Microsoft's 2.7B Phi-2 punches way above its weight
Microsoft just released Phi-2, a 2.7B parameter language model trained on textbook-quality synthetic and web data. Despite its compact footprint, it matches or outperforms models up to 25x larger like Llama-2-70B on complex reasoning and coding benchmarks without RLHF. Why it matters: it demonstrates that strategic data curation can unlock emergent reasoning even in lightweight models.