Microsoft drops open-source Phi-3.5 models

Microsoft just dropped its latest Phi-3.5 family of small AI models, and every single one is available under a completely permissive MIT license. If you're building local AI workflows or need high-efficiency models that don't cost a fortune to run, these belong on your radar right now.

The release includes three distinct flavors: the 3.8B parameter Phi-3.5-mini-instruct, the 41.9B total parameter Phi-3.5-MoE-instruct (which only uses 6.6B active parameters), and the 4.15B multimodal Phi-3.5-vision-instruct. Crucially, all three support an expansive 128k token context window.

What each model actually does

The Mini is built specifically for compute-constrained and low-memory environments. Despite its tiny 3.8B footprint, it punches well above its weight in logic, math, and coding. On the RepoQA long-context code benchmark, it outscored noticeably larger rivals like Llama-3.1-8B and Mistral-7B. Microsoft trained it on 3.4 trillion tokens using 512 H100 GPUs across just 10 days.

The MoE model is Microsoft's first mixture-of-experts entry in the series. While holding 41.9B parameters total, it only activates 6.6B parameters during inference, giving you deep multi-subject reasoning at small-model inference speeds. In fact, it topped GPT-4o mini on the 5-shot MMLU benchmark across STEM, humanities, and social sciences.

Rounding out the lineup is Vision, designed for mixed image and text tasks. It handles OCR, chart reading, table extraction, and multi-frame video summarization. With the 128k context window, it can digest complex visual streams without breaking a sweat, having been trained on 500 billion high-quality tokens using 256 A100s.

How to get started

All three models are live on Hugging Face right now. You can download the weights, inspect model cards, and start fine-tuning immediately.

Because Microsoft opted for a standard MIT license instead of a restrictive community agreement, you can modify, build into proprietary software, and monetize whatever you create without user-count caps or legal friction.

Hardware requirements are remarkably accessible. Mini and Vision fit comfortably on consumer GPUs and laptops. The MoE model requires enough VRAM to hold the full 41.9B footprint, but runtime compute demands stay light thanks to the sparse activation.

Why it matters

These models are a dream for developers building edge assistants, private document retrieval tools, or embedded coding agents. You get near-frontier small-model performance without sending a dime or a single byte of private data to third-party API endpoints.

Compared to Meta's Llama 3.1 series, Microsoft's move to release these under pure MIT terms removes the lingering licensing headaches that often stall enterprise adoption. For local and on-device AI, Phi-3.5 sets an impressive benchmark for pure efficiency.