Cursor open-sourced Mixture-of-Kittens (MoK), a fused MoE execution kernel that pushed multi-node signaling latency from 103us to 18us and lifted 512-GPU training throughput 41%. Why an IDE company rewrote the GPU kernel — and what it means for the rest of the AI stack.

Cursor Rewrote the GPU Kernel. Here's Why Your AI Coding Tools Will Too.

On August 7, 2026, Cursor open-sourced Mixture-of-Kittens (MoK) — a fully fused MoE execution layer that pushes token dispatch, cross-GPU communication, and expert compute into a single kernel on GB300 NVL72 racks. Two numbers from their benchmark explain why this matters: multi-node signaling latency dropped from 103 μs to 18 μs, and end-to-end training throughput rose 41% on 512 GPUs.

That second number is the headline. The first number is the story.

The bottleneck in modern MoE training is no longer bandwidth. It's the orchestration around it.

MoE forward and backward pipeline: Dispatch → Grouped GEMM → Combine, with the Reverse pair below. Blue nodes are forward steps MoK keeps in one Megakernel.

"Fast hardware" already stopped being the answer

In late July, NVIDIA disclosed that GB300 NVL72 hit 1,648 TFLOPS per GPU when training DeepSeek-V3 across 256 GPUs. The rack itself is wild: 72 GPUs in one NVLink domain, 130 TB/s of aggregate bandwidth. On paper, data should fly.

So why is anyone rewriting the kernel?

The honest answer is that MoE training doesn't look like a bandwidth chart. Each forward pass requires a sequence of cross-GPU operations: dispatch tokens to whichever GPU holds their assigned expert, run the grouped GEMM, combine the results back at the original token positions, then do the same three steps in reverse for backprop. Six communication-heavy steps per layer, every step.

The wire has been fast for two generations. The problem is what happens around the wire.

What "communication tax" actually means

Most engineers who hit this problem first think about bandwidth. Cursor's engineers thought about coordination.

In a Push-based dispatch (the conventional design), each source GPU writes tokens to target GPUs. Before it writes, it has to know exactly where on the target the data should land — so all the writers don't collide. Same-expert tokens should sit contiguous in memory, otherwise the downstream GEMM has to re-pack them. Each writer also needs a completion signal before the target can safely use the data.

At 72-way expert parallel, each rank waits on up to 71 peers. The data movement is fast. The signaling is the wall.

Push vs Pull dispatch: Push needs all 71 peers to finish writing before compute starts (~103 μs). Pull lets the target GPU fetch what it needs independently (~18 μs).

Three things hide inside the word "communication":

Layout generation — sorting tokens by expert so the GEMM gets contiguous input.
Synchronization — waiting for every peer to finish before compute can start.
Load imbalance — some experts get many tokens this step, few the next; the slow expert gates the whole layer.

130 TB/s describes the rack's peak. It doesn't describe what each step actually gets.

The three moves that bought back the time

1. Pull instead of Push. Each expert GPU now reads the tokens it needs from the source GPUs. The target controls its own address space; no global write-coordination. NVLink's two directions have independent channels, and Pull uses both the request path and the data path simultaneously. Under uneven expert load, Cursor measured up to 29% higher NVLink utilization. More importantly, signaling latency drops because the target GPU doesn't have to wait for 71 strangers — it just reads.

2. One Megakernel instead of three CUDA streams. MoK puts communication SMs and compute SMs into the same kernel and lets them coordinate through a GPU-local counter. There's no CUDA-stream contention. The two sides decide together how to divide the GPU. Cursor's reported gain on per-layer benchmarks goes up to 2.37× on MXFP8 forward versus the strongest public baseline (NCCL+PyTorch / DeepEP / TransformerEngine / HybridEP+Megatron).

3. Ring Token Buffer + minibatch tuning. MoE has an awkward chicken-and-egg: you don't know how many tokens each GPU will receive until the router runs. Pre-allocate the worst case and you waste memory. Re-allocate dynamically and you stall for the CPU. MoK uses a fixed-size ring buffer and overlaps one macrobatch's Combine with the next macrobatch's Dispatch.

The critical knob is minibatch size — how many tokens to hand to the expert GEMM at once. Too small, and many SMs sit idle. Too large, and the first batch waits forever.

On Kimi 2.5 shapes (Hidden 7168, expert intermediate 2048), Cursor found the sweet spot at ~2368 tokens — enough to form two full waves of expert compute. Below that, forward latency climbs to 5.981 ms at 512 tokens. At the sweet spot, it's 3.425 ms. Past it, no further gain.

That tradeoff is the whole game: overlap communication and compute without starving either side of work.

The honest boundaries

Cursor didn't publish a full ablation, so we can't say how much of the 2.37× comes from Pull, how much from Megakernel, how much from Ring Buffer. Treat each contribution as co-equal in the headline number.

The design is also not portable. Pull, the Megakernel, and the Ring Buffer all rely on low-latency remote reads across GPUs in the same NVLink domain — that's a Blackwell-and-NVL72-only feature. H100, AMD MI300X, and older NVLink topologies don't expose it. The optimal minibatch shifts with Hidden Size, Top-k, and expert count. Anyone copying MoK verbatim to a different model shape will get worse numbers than Cursor did.

So this is not a recipe. It's a proof of ceiling — proof that on the right hardware, you can still find a 2× inside the kernel even after NVIDIA already pushed the rack.

What this means for AI tooling

Three years ago, the move was DeepSeek publishing MLA and training tricks to the open-source community. That was a kernel-level move inside the model architecture.

MoK is one layer up. It's an execution-level move inside the model runtime.

Both moves have the same logic: when the platform vendor stops moving the bottleneck, the application vendor starts moving it.

For AI startups with a real inference or training cost problem, the next million dollars won't go into more GPUs. It'll go into engineers who can read PTX and write CUDA. Cursor is now one of those companies, alongside DeepSeek, Mistral, and a handful of others. The list will grow.

A reasonable prediction to leave open, not close: which other AI coding or infrastructure company publishes its own kernel in the next twelve months? The hardware is no longer the constraint. The willingness to commit a kernel team to live in the registers is.

Sources: Cursor Mixture-of-Kittens blog (Aug 7, 2026); NVIDIA GB300 NVL72 DeepSeek-V3 disclosure (Jul 21, 2026). All benchmark numbers are from Cursor's own published micro-benchmarks and end-to-end runs on 512× GB300.