Kimi open-sources its 2.8T MoE monster K3

Kimi K3 is an open-source Mixture-of-Experts powerhouse just dropped by Moonshot AI on Hugging Face. The model packs 2.8 trillion total parameters, native visual understanding, and a massive 1-million-token context window.

If you need an open model that can ingest entire codebases, technical libraries, or complex visual documents without losing track of details, K3 is currently the heaviest hitter available on open weights.

What it can do

Despite boasting 2.8 trillion parameters, K3 does not fire every weight at once. Thanks to its MoE routing, only 104 billion parameters are active during any given token generation.

It handles visual inputs natively rather than relying on tacked-on adapters, powered by an integrated Moon VIC vision encoder. The team claims the new architecture delivers 2.5 times the intelligence per unit of compute compared to previous setups, focusing on efficiency rather than raw bloat.

Moonshot AI also open-sourced the underlying plumbing. That includes custom high-performance attention kernels, a dedicated MoE communication library, and runtime infrastructure built for scaling complex agent environments.

How to run it

Under the hood, K3 introduces proprietary mechanisms like delta attention and attention residuals to keep calculations stable and fast across long sequence lengths.

The catch is the sheer footprint. The full unquantized weights weigh in at roughly 1.56 terabytes on Hugging Face. Running the raw model requires a cluster of enterprise-grade GPUs with substantial VRAM pools.

The open-source community is already moving to tame it. Quantization projects like Unsloth have released compressed GGUF builds, bringing a 1-bit version down to around 594 GB. It is still massive, but far more manageable for on-premise clusters.

You can pull the weights directly from Hugging Face under moonshotai/Kimi-K3 and grab the technical report from GitHub.

Why it matters

For enterprise teams and researchers with serious hardware, K3 offers a frontier-grade alternative to proprietary cloud APIs. Having million-token context paired with native vision in an open model means organizations can build bespoke agent pipelines without data ever leaving their servers.

For solo builders, however, local execution remains largely out of reach due to the massive storage and compute requirements. We think K3's real impact will come from how the community dissects its attention architecture, tests its agent tooling, and downstream distillations that follow.