Microsoft's AI outdiagnoses doctors at half the cost

Diagnosing a patient has never looked like a multiple-choice board exam. Doctors don't get all the lab results handed to them on day one; they ask questions, weigh the cost of a scan against its diagnostic value, and eliminate possibilities piece by piece.

Now, Microsoft is trying to turn that exact clinical reasoning into software. In a paper published in Science, Microsoft AI unveiled MAI-DxO, an AI diagnostic orchestrator designed to mirror a multi-round debate among specialist physicians. The company claims the system clobbers generalist physicians on diagnostic accuracy while slashing the cost of ordered tests.

Mustafa Suleyman, CEO of Microsoft AI, framed the milestone as a tangible step toward what he calls medical superintelligence.

Virtual panels debating the chart

Standard medical benchmarks like the USMLE test textbook memorization with all facts known in advance. Microsoft built SDBench, an interactive benchmark that starts models with minimal patient intake data and forces them to choose which tests to order next while clocking estimated financial costs.

On 304 notoriously difficult cases pulled from The New England Journal of Medicine, MAI-DxO hit 83% diagnostic accuracy. Human physicians given open-book access to textbooks and search engines scored just 36%, and slipped to 22% without reference materials.

On typical emergency-room cases from the MIMIC-IV-ED dataset, MAI-DxO notched 80% accuracy against 47% for human physicians. In cost-optimized modes, the AI maintained roughly 81% accuracy while cutting clinical workup costs by more than half.

Under the hood, MAI-DxO doesn't rely on a single prompt. It sets up multiple AI agents acting as a clinical panel: proposing competing hypotheses, requesting selective tests, and challenging each other's conclusions across iterative rounds of debate.

Model muscle versus orchestrator smarts

While the numbers look staggering, detailed data reported by TechCrunch reveals an important nuance: much of the heavy lifting still comes down to the raw horsepower of frontier models.

Microsoft tested the framework across models from OpenAI, Google, Anthropic, Meta, and DeepSeek, with top marks landing on OpenAI's GPT-5 and GPT-5.5. Running standalone GPT-5 with a simple prompt on the NEJM cases yielded 78.9% accuracy at a steep price tag of roughly $8,400 per case. Wrapping GPT-5 inside DxO shaved about $3,000 off test costs but nudged accuracy up by just four points.

With GPT-5.5, DxO pushed complex-case accuracy from 72% to nearly 87%. Yet on everyday emergency room cases, the orchestrator offered little to no edge over standalone frontier models, with both hovering around 85% accuracy at similar costs. Where DxO really shone was propping up weaker models: Gemini 2.5 Flash jumped from 46.4% solo to nearly 69% inside the orchestration loop.

It is also worth noting that the human benchmark consisted of primary care and general internal medicine doctors. In actual clinical practice, NEJM-grade medical mysteries are rarely tackled by a solo generalist, but rather by coordinated teams of subspecialists.

Microsoft is open-sourcing SDBench for select NEJM cases, and has partnered with Beth Israel Deaconess Medical Center in Boston for upcoming real-world prospective trials.

Why it matters

For clinicians drowning in documentation and administrative triage, having an automated second opinion that purposefully minimizes expensive, unnecessary tests is a genuine breakthrough. Still, everyday patients shouldn't expect an AI doctor in their pocket just yet; the tool is slated for hospital pilots and clinical backstops rather than direct-to-consumer triage.

For the tech industry, Microsoft is shrewdly shifting the AI health battleground away from static licensing exams and toward workflow economics. By evaluating not just the final diagnosis but the financial burden of every lab ordered along the way, Microsoft is speaking directly to hospital CFOs.

That said, the data shows an uneasy tension. On straightforward emergency visits, modern frontier models do just fine on raw instinct without complex multi-agent debates, and the orchestrator's biggest wins relied on comparing generalist doctors against cases typically reserved for specialist tumor boards. Whether this virtual panel survives the messy reality of a packed Boston emergency room on a Friday night remains the real test.