Qwen3.8 kernel architecture and TDD ledger

This document is the Iron-side contract for the Qwen3.8 work consumed by Butter. It describes kernel ownership and test evidence on dev; historical branch names are not capability claims.

Canonical tracker: https://github.com/waffuruai/iron/issues/144

How to read this document

Start with the two model tracks and ownership boundary. Then use the dense-27B kernel map to find the relevant operation, the test tiers to choose the first valid gate, and the precision/transaction sections before changing target math or speculative state. The historical receipt at the end proves one component only; it does not override the current map.

Keep the two model tracks separate

The project has two distinct Qwen3.8 tracks. They share some primitives, but they do not share a model graph or a readiness claim.

Track	Graph	Current Iron scope	Whole-model state
Flash-Next 176B	48-layer experimental MoE, 512 experts/top-10, PLE n-gram tables	FP8/global-scale PLE gather and general quantized/MoE primitives	Metadata and component readiness only
Dense 27B NVFP4	64-layer qwen35 hybrid: 48 GDN + 16 GQA layers, dense SwiGLU	Production CUDA kernels used by Butter for prefill, scalar decode, exact verification, KV/state updates, and block drafting	Running on one DGX Spark

No successful dense-27B receipt proves that the 176B MoE graph is executable. No PLE gather test proves routed-expert, QSA, or full-model correctness.

Ownership boundary

Iron owns device semantics: kernel math, dtype/layout interpretation, bounds, strides, launch geometry, backend lowering, and component benchmarks. Butter owns model semantics: checkpoint/config parsing, layer schedule, tensor ownership, cache lifecycle, graph capture, speculative transactions, sampling, and end-to-end performance.

Every production change therefore needs two layers of evidence:

1. an Iron component oracle for the exact dtype, layout, shape, and boundary;
2. a Butter model or transaction oracle proving that composition, state, and token behavior remain correct.

Butter currently pins Iron commit 216d0c2034629cdd8df9b3776d377fda2a9c9436. Documentation against a newer Iron dev commit must state that Butter has not consumed it until the pin moves and Butter CI passes.

Current dense-27B kernel map

Model responsibility	Iron implementation	Primary evidence	State
Scalar GDN recurrence and fused output gate	kernels/ssm/gated_delta_qwen35_decode_fused.rs	Qwen production-shape CPU oracle; tail and offset-view cases	Production
Batched/prefill GDN recurrence	kernels/ssm/gated_delta_wy*.rs	one- and multi-chunk sequential-GDN oracles	Production; numerical policy remains Butter-gated
GDN alpha/beta transforms	kernels/ssm/gated_delta_gate_beta.rs, kernels/gemm/gemv.rs	separate/joined projection tests across f32/f16/bf16	Production
Full-attention d256 GQA	kernels/sdpa/sdpa_multi_d256.rs, kernels/sdpa/sdpa_multi_d256_gqa.rs	causal/non-causal, 24q/4kv, nq 1/7/8/16, long-context cases	Production
FP8 KV storage and reads	kernels/kv_cache/cache.rs, quantized SDPA kernels	physical-stride, dtype, packed-code, and tail oracles	Production
Paired and batched KV writes	kernels/kv_cache/update_many.rs, kernels/kv_cache/cache.rs	exact pair/many tests for f32/f16/bf16 and FP8	Production
NVFP4/FP8 projections	CUDA runtime matrix paths plus gemm/* kernels	checkpoint component oracles and exact-row tests in Butter; Iron kernel corpora for declarative siblings	Production, shape-selected
Draft dynamic convolution	kernels/convolution/grouped_dynamic_conv.rs	prepare/finish CPU oracles	Production component
Resident greedy/top-k selection	kernels/ops/argmax_rows.rs, kernels/sampling/row_topk.rs, kernels/sampling/ranked_path.rs	stable tie/order, production vocabulary width, and path-selection oracles	Production component
Flash-Next PLE gather	kernels/quant/dequant_gather_block_scaled.rs::iron_fp8_e4m3_global_dequant_gather	released-width FP8 landmarks, out-of-order rows, global-scale oracle	Component-ready only

Test tiers

Tier I0: pure Rust and codegen

Run before GPU work:

cargo test -p wh-iron-core
cargo test -p wh-iron-codegen
cargo test -p wh-iron-std --lib

These checks cover IR construction, lowering, manifests, and CPU-side helpers. They do not execute a CUDA kernel.

Tier I1: registered CUDA kernel oracles

Build a fresh runner after any declarative-kernel change; the registry is linked into the runner binary and a stale binary can execute stale kernels.

CARGO_INCREMENTAL=0 CARGO_PROFILE_DEV_DEBUG=0 CARGO_BUILD_JOBS=2 \
  cargo build -p wh-iron-std --bin __iron_runner --features cuda

target/debug/__iron_runner test --backend cuda --filter gated_delta
target/debug/__iron_runner test --backend cuda --filter sdpa_multi_d256
target/debug/__iron_runner test --backend cuda --filter grouped_dynamic_conv
target/debug/__iron_runner test --backend cuda --filter row_topk
target/debug/__iron_runner test --backend cuda --filter fp8_e4m3_global

The exact filter inventory may change. First list names when a filter returns zero tests; zero selected tests are not a green result.

Tier I2: Butter integration

Iron cannot prove checkpoint layout or model state in isolation. Before Butter moves its Iron pin, the paired Butter gates must cover:

• real-checkpoint NVFP4 and FP8 projection layouts;
• causal batched prefill versus scalar prompt ingestion;
• scalar decode token fingerprint;
• every speculative commit depth plus abort/drop restoration;
• eager versus captured verification;
• prompt-conditioned outputs across unrelated prompts;
• short and long context when an attention or KV kernel changes.

Tier I3: performance promotion

A kernel is not promoted from an isolated microbenchmark alone. Record exact commits, hardware, context/shape, warmup, every measured arm, median and dispersion, component time, end-to-end time, output fingerprint, and rollback. Use a same-binary selector where possible. A candidate that changes target numerics requires an explicit quality policy; draft-only numerical changes must still improve acceptance-adjusted end-to-end throughput.

Precision contracts

• Target verification is the correctness authority. Exact variants preserve the established reduction and tie order unless Butter explicitly adopts a new quality policy.
• Draft math may use a different precision only when target verification remains authoritative and acceptance-adjusted performance improves.
• Packed FP8/NVFP4 bytes are never reinterpreted as f16/bf16. Layout and scale metadata are tested independently.
• Cosine similarity alone is insufficient. Quantized changes report maximum error, norm/scale drift, non-finite behavior, and a multi-token model gate.
• FP8 KV changes must test physical stride and offset, not only logical shape.
• CUDA graph capture requires stable storage and independent arenas for simultaneous sessions; pointer lifetime is part of correctness.

Transaction and width vocabulary

Use distinct terms throughout code and documentation:

• trained_block_rows = 8: width encoded by the draft checkpoint;
• verify_rows: runtime target width, currently admitted in 2..=10;
• proposal_count = verify_rows - 1 for a particular round;
• committed_rows: accepted proposal prefix plus one target correction/bonus.

Do not use one unqualified “gamma” value for all four. Width-eight receipts are historical measurements, not the complete current runtime contract.

The target transaction must resolve exactly once. Commit publishes only the selected KV and GDN state. Abort, error, or unresolved drop restores the prior logical state. Rejected rows are poisoned in tests so accidental tail reads cannot pass silently.

Known gaps

• The 176B Flash-Next MoE/PLE graph is not a supported whole-model execution path. Router/expert placement, QSA, gated residual, PLE integration, and memory admission remain open.
• Dense block drafting has exact transaction coverage, but draft-cache storage is still a contiguous reference design rather than the final paged 2,048-row ring described by the long-term architecture.
• CUDA production paths include runtime/AOT matrix kernels that are not all represented by declarative #[test_kernel] inventory. Butter checkpoint tests are mandatory for those paths.
• Performance support is calibrated on the GB10 Spark. Other CUDA devices need their own selector and receipt; architectural correctness does not imply the same launch choice is fast.

Historical I1 receipt: 2026-08-26

The released Flash-Next PLE table shape is F8_E4M3 [2500012,160] with one BF16 scalar scale. The global-scale gather oracle covers E4M3 landmarks 00, 01, 38, 40, 7e, 80, b8, fe, repeated/out-of-order rows, and FP32/FP16/BF16 outputs. A mutation that cast encoded bytes directly to f32 failed all three contracts; decoding E4M3 before applying the global scale passed with zero observed error. This remains component evidence, not whole-model readiness.