Perplexity makes MoE communication 10x faster
Perplexity just open-sourced pplx-kernels, a high-performance communication library for Mixture-of-Experts models that runs up to 10x faster than standard All-to-All primitives. It bypasses the CPU via GPU-to-NIC communication and overlaps data transfers with computation across NVLink, CX-7, and EFA networks. Why it matters: running massive MoE architectures across clusters just got substantially faster without sacrificing portability.
pplx-kernels is an open-source GPU communication library from Perplexity, built specifically to eliminate communication bottlenecks when running large-scale Mixture-of-Experts (MoE) models across multiple GPUs. It delivers up to 10x faster performance compared to standard All-to-All communication primitives.
When serving models like DeepSeek R1 or Mixtral across clusters, raw compute rarely dictates latency. The real bottleneck is moving tokens between GPUs hosting different experts. In DeepSeek R1, out of 671B parameters, only 37B activate per token, but those active experts are scattered across nodes, triggering massive cross-device traffic.
What it does
Standard PyTorch implementations rely heavily on the native all_to_all primitive to dispatch tokens. Because token routing is sparse, native routines often force expensive memory padding, constant CPU-GPU synchronizations, and heavy metadata broadcasts.
pplx-kernels replaces this pipeline with custom kernels tailored for expert dispatch and combination. It routes tokens directly to the GPUs holding the assigned experts and collects the outputs via weighted sums without traditional framework overhead.
On a single node, it cuts latency by 2.5x compared to the previously fastest implementation. Across multi-node clusters, it achieves throughput up to 10x faster than standard PyTorch all-to-all transfers.
How it works
The library achieves this speed by bypassing the host CPU and overlapping network transfers with active GPU compute.
On ConnectX network cards, it takes advantage of GPU-initiated communication through IBGDA. The GPU directly posts RDMA transfers to the network interface card and polls completion queues on its own, cutting host CPU overhead out of the loop entirely.
It also uses a split-kernel architecture. By isolating send and receive stages, the GPU can run local matrix math simultaneously while data travels over the wire.
Instead of juggling the full complexity of NVSHMEM and paying heavy penalties for global memory fences, pplx-kernels builds its entire synchronization pipeline around just three non-blocking primitives. It triggers transfers and signals completion atomically without stalling execution pipelines.
Compatibility and getting started
Portability is the major design highlight here. Highly specialized kernels tuned to specific proprietary setups can run roughly 2x faster, but they break outside narrow hardware setups. pplx-kernels prioritizes running cleanly across NVLink, ConnectX-7, and AWS EFA networks without constant rewriting.
The entire library is open source and available directly on GitHub at ppl-ai/pplx-kernels.
Why it matters
For any engineering team deploying massive MoE models in production, communication overhead is where GPU utilization goes to die. Standard collectives simply were not built for dynamic expert routing across 64 or 128 devices.
pplx-kernels gives distributed systems teams a drop-in path to reclaim massive latency margins without locking themselves into fragile, single-vendor network hacks. It is a clean, practical piece of infrastructure that solves a very real deployment headache right now.
Sources (1)
- perplexity.ai(opens in a new tab)Originalperplexity.ai