DeepSeek Coder V2 just outcoded GPT-4 Turbo

DeepSeek Coder V2 is a fully open-source coding model that just crossed a major milestone: it beats top proprietary models like GPT-4 Turbo, Claude 3 Opus, and Gemini 1.5 Pro on flagship coding and math benchmarks.

Built on the team's MoE foundation, the new release fixes the main pain points of the original DeepSeek Coder. It expands programming language support from 86 to 338 and blows past the old 16K context window, now supporting up to 128K tokens for tackling messy, repo-level codebases.

How it works under the hood

The secret sauce is an MoE setup that delivers massive capacity without painful inference costs. The base model was pre-trained on an additional 6 trillion tokens, bringing the total training dataset to 10.2 trillion tokens.

That training diet is 60% source code gathered from GitHub and CommonCrawl, 10% math datasets, and 30% natural language.

DeepSeek is offering two sizes: 16B and 236B total parameters. But because only relevant expert layers fire up, the 16B model activates just 2.4B parameters per token, while the flagship 236B variant activates only 21B. You get heavy-hitting reasoning power with the operational footprint of a much smaller network.

The benchmark showdown

On raw coding capability, the 236B model posted 90.2 on HumanEval, 76.2 on MBPP+, and 73.7 on Aider. Across the board, it outperformed GPT-4 Turbo, Claude 3 Opus, and leading open-weight rivals like Codestral and Llama 3-70B.

In fact, the only closed model that edged it out across multiple coding tests was GPT-4o, and even then, by razor-thin margins.

It holds its own in math benchmarks like GSM8K and MATH, too. On general language understanding via MMLU, it scored 79.2. That is slightly below general frontier models like GPT-4o at 88.7, but roughly on par with Llama 3-70B and well ahead of typical code-focused checkpoints.

One honest caveat from the creators: real-world instruction following still needs work. The model can stumble when given complicated, multi-condition programming prompts in messy dev environments.

How to get it

Everything is out under a permissive MIT license, meaning full freedom for both research and commercial deployment.

You can pull both the 16B and 236B models in base or instruction-tuned flavors directly from Hugging Face. If you lack the VRAM to host it locally, DeepSeek provides a pay-as-you-go API and a ready-to-test web chatbot.

Why it matters

If you build software in regulated environments or care about intellectual property, this release changes the calculus. You no longer have to send internal code to OpenAI or Anthropic just to get state-of-the-art code completion and bug fixing.

DeepSeek has proven that smart MoE architectures can match commercial frontier intelligence on coding tasks at a fraction of active compute. We think this will accelerate price cuts on commercial coding APIs, though developers integrating it into autonomous agent loops should watch its instruction-following quirks closely.