Stopping silent kernel regressions in CI
How a platform team could wire Kernalin's regression gate into pull requests so slowdowns never reach production unnoticed.
This is a demonstration case study using illustrative data. It shows the shape of a Kernalin engagement, not a real customer or real numbers.
Problem
A platform team owns dozens of custom CUDA kernels shared across product teams. Twice in a quarter, an innocent refactor shaved single-digit percentages off a hot kernel — and nobody noticed until aggregate latency drifted in production.
Infrastructure
- Monorepo of CUDA C++ kernels
- GitHub Actions CI
- Mixed RTX 4090 / A100 / H100 runners
Baseline
Kernalin seeds an environment-keyed baseline store — each kernel’s baseline is keyed by GPU, driver, and toolkit, so comparisons are never made across incompatible hardware.
Optimization
No optimization here — the goal is protection. The team adds the Kernalin GitHub Action to its CI workflow.
Verification
On every pull request, the action runs correctness + anti-cheat, benchmarks the candidate against the stored baseline, and returns a tiered verdict: BLOCK on a credible regression, WARN on a borderline change requiring an acknowledged re-baseline, INFO/PASS otherwise. It posts an auditable metric-diff comment on the PR.
Result
Regressions are now caught in the pull request, on the day they land, with an audit trail attributing each delta to a source change or an environment change. The team’s rule is simple: a red gate means the number moved, and the number is not allowed to move silently.