A megakernel 1.35× faster than vLLM, and what Kerneln found when it checked

On 23 October 2026 we ran Kerneln for about two hours on an NVIDIA H100 80GB HBM3 SXM rented from RunPod, through kerneln.com. This post covers what came back, including the result that didn’t pass.

A megakernel, measured and checked

A megakernel runs a whole model’s forward pass as one persistent GPU kernel instead of hundreds of small ones. We took Hazy Research’s open-source megakernel for Llama-3.2-1B-Instruct (commit 7309cec). The megakernel is theirs, not ours. Kerneln measured it and checked it, the same way it checks any speed-up.

Speed (batch 1 decode, bf16, 30 runs × 128 tokens):

Engine	ms per token	Megakernel is
Megakernel	1.12 (895 tokens/s)
vLLM 0.31	1.51	1.35× faster
Hazy’s own PyTorch path	7.76	6.9× faster
Hugging Face eager	8.85	7.9× faster

Kerneln also checked that the number is physically plausible: the time per token is consistent with streaming the model’s weights through the H100’s memory bandwidth. It is not too fast to be real.

Output. Kerneln compared the megakernel’s next-token choice with the Hugging Face reference over 16 prompts × 128 tokens. They matched at 98.75% of positions (mean KL 0.0024). Kerneln’s bar is 99%, so the run is labelled “Differs”. For scale: the reference compared with itself under small settings changes (sdpa bf16, fp16) agrees 99.2–99.6% of the time. So the megakernel is fast, and slightly less exact than the usual run-to-run noise. We publish both halves.

Schedule. The megakernel’s 8,644-instruction schedule across 132 SMs was checked for deadlocks and races and came back clean.

Everything else that ran on the GPU

• AI optimizer: Claude rewrote a deliberately wasteful test kernel three times. Kerneln kept one, 2.13× faster (it replaced about 15 elementwise kernel launches with a single Triton kernel), and reverted two that were slower (0.41× and 0.52×).
• Benchmark: 24 of 24 configurations measured and output-checked (PyTorch eager, torch.compile, TensorRT and ONNX Runtime; three batch sizes; fp32 and fp16).
• TensorRT precision matrix: FP32, FP16 and BF16 engines, each checked against FP32. BF16 was 1.57× faster than eager FP32 with matching output.
• PyTorch → Triton: converted gelu and silu_mul kernels ran on the H100 and matched PyTorch.
• LLM serving: Qwen3-0.6B on vLLM served 17.2 requests a second within the latency SLA, gained 4.2× from batching and reached its first token in 17.4 ms (p50). Every run also checks that the model answers sensibly and the same way twice.
• Cost: Kerneln tracked $2.20 of H100 job time across 37 jobs and priced the idle time separately.
• Live telemetry, op profiles, memory analysis, Nsight Systems report import and hardware comparison all ran against the H100.

What we fixed along the way

Real hardware finds real bugs. Jobs on the 224-core host hit a per-user thread limit and hung (now scaled to the host). Generated Triton kernels lost their source after a temporary file was cleaned up (now kept in memory). An infinite value broke a result upload, and 32 MB megakernel results were too big for the upload limit (now 100 MB). All of these fixes are live.

What did not run

• Kernel counters and warp-divergence analysis need hardware counter access, which RunPod pods deny. They need a full GPU VM.
• The in-app Nsight Systems timeline failed inside the job sandbox on the pod. Importing an Nsight report recorded outside the sandbox worked.
• Mirage, a second megakernel compiler, built but would not load, so it has no result.

Read this with care

• This is one model (1B parameters), one GPU and one session. Larger models may behave differently.
• The optimizer baseline was a deliberately wasteful test kernel, chosen to show the loop working.
• Clocks were not locked on the pod. The numbers are as-deployed.