ByteDance drops open-source Seed-OSS-36B with 512K context

ByteDance's Seed Team just open-sourced Seed-OSS-36B, a new family of 36-billion-parameter language models designed for advanced reasoning, coding, and heavy document processing. Everything is released under the permissive Apache-2.0 license, meaning teams can freely customize and deploy it in commercial products without paying licensing or API fees.

What it does and how it works

The release includes three distinct variants. There are two base models: one trained with synthetic instruction data for higher general-purpose benchmark scores, and another without synthetic data that provides a clean, unbiased baseline for researchers. The third model is Seed-OSS-36B-Instruct, which is post-trained specifically for instruction following and complex task execution.

Under the hood, Seed-OSS-36B uses 64 layers with grouped query attention (GQA), SwiGLU activation, RMSNorm, and a 155,000-token vocabulary. Its standout capability is a native 512K token context window—roughly 1,600 pages of text, doubling the limit of OpenAI's GPT-5 family. On the long-context RULER benchmark at 128K length, it scored 94.6, marking a new high for open-source releases.

Another clever feature is the adjustable thinking budget. Similar to recent releases like Nvidia's Nemotron-Nano-9B-v2, developers can specify how many reasoning tokens the model should consume before generating an answer. Recommended in multiples of 512 tokens, setting the budget to zero puts the model into a direct response mode for minimal latency.

Performance-wise, the Instruct variant posted top-tier open-source scores: 91.7 percent on AIME24, 65 on BeyondAIME, and 67.4 on LiveCodeBench v6. Meanwhile, the synthetic Base version reached 65.1 on MMLU-Pro and 81.7 on MATH.

How to get started

All model weights are live on Hugging Face right now. You can pull them directly for self-hosted deployment. Note that processing deep 512K context sequences on a 36B model will demand substantial enterprise GPU memory, though dialing down the thinking budget helps keep inference costs manageable.

Why it matters

If you build software that analyzes full codebases or long legal and financial packets, Seed-OSS-36B offers an open alternative that legitimately rivals top closed models on reasoning depth.

ByteDance played this smart by giving developers full commercial permission and granular control over inference cost. You are no longer locked into paying OpenAI or Anthropic by the token for long-context workloads.