Four-wide SwiGLU on M5 Max

This extracts iron_swiglu_x4 from PR #299 onto dev at dc856bf4cc808a69594af2f8dd2d13940f82e478. Four adjacent gate/up values are loaded together, activation arithmetic remains F32, and each thread writes four outputs. The existing scalar iron_swiglu implementation and default dispatch behavior are unchanged. No compiler or runtime changes from #299 are required.

The recommendation is for large BF16 tensors on M5 Max Metal. F32 loses in the screening comparison and should retain scalar. Smaller BF16 shapes stay scalar because their gains are small or negative. F16 is a separate preliminary measurement, not a broadly validated recommendation. These are GPU kernel/dispatch measurements, not model tokens per second or H100 results.

Dispatch contract

All three tensors have the same dtype and length. The raw x4 kernel requires exactly n / 4 launched threads and n % 4 == 0. The runtime’s ordinary elementwise bound is the output element count; it cannot protect extra x4 threads introduced by rounding up a threadgroup.

recommended_swiglu_x4(n) is a shape gate for callers that have already selected the measured backend/dtype. It requires n >= 409600 && n % 1024 == 0. Use TPG [256,1,1] and grid [n/1024,1,1]. This tightens #299’s previous multiple-of-four recommendation so it cannot suggest partial groups at TPG 256. Other shapes should use scalar or an explicitly exact x4 launch.

Reproduction

Run on an available M5 Max after builds finish. Retain the source commit, lockfile hash, device/OS/toolchain identity, exact command, and both stdout and stderr. Do not build or launch another GPU benchmark concurrently.

# In a dedicated checkout, restore this campaign's dependency resolution.
gzip -dc docs/benchmarks/swiglu-x4-2026-09-05/dependencies.lock.gz > Cargo.lock
CARGO_BUILD_JOBS=2 CARGO_PROFILE_DEV_DEBUG=0 \
  cargo build --locked -p wh-iron-std --example swiglu_ab
SWIGLU_DTYPE=bf16 SWIGLU_PACK=8 SWIGLU_ROUNDS=9 \
  target/debug/examples/swiglu_ab > forward.csv 2> forward.log
SWIGLU_DTYPE=bf16 SWIGLU_PACK=8 SWIGLU_ROUNDS=9 SWIGLU_REVERSE=1 \
  target/debug/examples/swiglu_ab > reverse.csv 2> reverse.log

The paired executable uses the unchanged dev scalar kernel and the extracted x4 kernel in the same process. It uploads identical deterministic, nonuniform inputs once. Each timed Metal command buffer contains eight serial independent dispatches with barriers, and its GPU time is divided by eight. Both variants use identical residency, output allocation, packing and timing. This includes dispatch/barrier GPU cost; host allocation/readback time is outside the GPU interval.

Each variant receives 20 warmups and 40 timed repetitions in each of nine paired rounds. Variant order alternates by round; the second session reverses both the initial variant order and shape order. Every measured sample is printed as CSV. SWIGLU_PACK=1 measures separate command buffers; SWIGLU_DTYPE=f16 or f32 selects diagnostic dtype comparisons.

Before timing, every output is checked against the unrounded F64 mathematical SiLU-product reference using the actual stored inputs. The unchanged absolute bounds are F32 1e-4, F16 5e-3, BF16 5e-2. Both variants must also produce bit-identical outputs. The unrounded reference avoids adding a second quantization discontinuity to the numerical error measurement.

Validation

The declarative GPU tests include 1,024, 409,600, 410,624 and 1,048,576 elements across F32/F16/BF16, with the existing independent CPU oracle and unchanged tolerances. Host tests check the selection threshold and reject shapes that would overhang a TPG-256 group. The registered kernel inventory is updated with the added dtype cases.

make test completed successfully: 1,216 Cargo tests passed and 59 were ignored, including the passing wrapper for the 5,310-case registered GPU corpus. Its existing supported-device skip policy is unchanged. The separately logged SwiGLU run executed 50 cases with 50 passes, zero failures and zero skips. make clippy (all targets/features), formatting, typos, buffer bindings and rendered inventory checks pass. Full Cargo output and filtered GPU results are retained.

Paired results

Frozen source: ce2c1ecd7f3d9e5dd15a489c53f5d82d55ef8459; M5 Max, macOS 27.0 build 26A5353q, Rust 1.98.0. The dependency lock, source/binary hashes, session timestamps and host load are in provenance.json. Host load ranged roughly 4.9–6.1; this is a shared interactive laptop, not a claim of an idle system.

Each latency below is the median of nine per-round medians. Reduction is the median of the nine paired reductions, rather than a ratio of those two independently aggregated columns.

BF16 elements	Session	Scalar µs	x4 µs	Paired time reduction	x4 wins
409,600	forward	2.8438	2.0052	29.5%	9/9
409,600	reverse	2.8281	1.9792	29.9%	9/9
1,048,576	forward	6.0443	4.2083	31.2%	9/9
1,048,576	reverse	6.0625	4.4089	29.8%	7/9

The threshold wins all 18 paired rounds, with a minimum paired reduction of 10.7%. The million-element reverse session contains two substantial losses; all samples and ranges are retained rather than discarded. Small BF16 controls are mixed or under 3% in median paired gain, so the existing small-shape scalar choice remains. F32 loses all three paired rounds at 409,600 elements (8.8% median slowdown). F16 is preliminary and has a noisy loss at that shape, so it is not included in the recommendation.

The separate-command-buffer BF16 control (pack=1, three rounds) improves 2.8% at 409,600 elements and 4.3% at 1,048,576. The packed result includes reuse of resident inputs and warm caches. It should not be presented as cold-streaming bandwidth or a serving-throughput result.

summary.csv includes small shapes, F32/F16 diagnostics and the separate-command-buffer control. Compressed raw CSVs and oracle logs accompany every session. Regenerate the summary with:

python3 docs/benchmarks/swiglu-x4-2026-09-05/summarize.py \
  docs/benchmarks/swiglu-x4-2026-09-05/*.csv.gz

During harness development, a storage-rounded BF16 CPU reference produced a 0.0625 discrepancy on the unchanged scalar baseline near an output rounding boundary. The retained reference evaluates the mathematical result in F64 without pre-rounding it to output storage. Inputs and absolute tolerances were not changed; all final sessions also require bit-identical scalar/x4 outputs. Earlier screening measurements are not used in the table above.