Microsoft's Orca 2 punches way above its weight
Microsoft just released Orca 2, a pair of small language models trained to use tailored reasoning strategies rather than blindly mimicking giant models. By learning when to reason step-by-step or answer directly, it matches or beats models 5 to 10 times larger on complex zero-shot benchmarks.