Mixture of Kittens Speeds Up AI Training

Mixture-of-Kittens (MoK) is an open-source megakernel designed by Cursor to optimize Mixture-of-Experts (MoE) training on NVIDIA NVL72 systems. By fusing computation and communication into a single, deterministic kernel, MoK achieves significant speedups—up to 2.37x for specific passes—over existing distributed frameworks. The system utilizes a pull-based communication model to minimize signaling latency and employs a ring token buffer to eliminate inefficient CPU-GPU synchronizations. Furthermore, MoK offers a tunable minibatch architecture that allows developers to balance hardware saturation with network efficiency across forward and backward training stages. Together, these innovations address the communication bottlenecks inherent in scaling large-scale agentic models.

Note:  This podcast was AI-generated, and sometimes AI can make mistakes.  Please double-check any critical information.

Sponsored by Embersilk LLC