Microsoft AI Beats Doctors in Complex Diagnosis Study

Microsoft has built an artificial-intelligence system that works through a medical case the way a real physician does, asking questions, ordering tests, and revising its thinking as new evidence arrives, and in a newly peer-reviewed study it diagnosed difficult cases more than twice as accurately as human doctors. The findings mark the clearest claim yet that AI can rival clinicians on complex diagnostic reasoning.

The tool, called the Microsoft AI Diagnostic Orchestrator (MAI-DxO), correctly identified 83.2% of 304 of the toughest diagnostic cases ever published in the New England Journal of Medicine. Generalist physicians working with textbooks and internet access reached just 36.1%,  and when those doctors were denied reference materials, their accuracy fell to roughly 22%, leaving the AI nearly four times ahead.

What Microsoft Actually Built

MAI-DxO is not a new AI model. It is an orchestration layer, software that coordinates existing large language models and makes them behave like a panel of specialists debating a case together. Rather than ingesting a patient's full symptom list, lab work, and imaging all at once, the system starts from a single chief complaint and then decides what history to take, what the exam should focus on, and which tests to order, updating its working hypotheses as results come back.

To do this, the orchestrator splits the reasoning across five AI "personas," each with a distinct job:

1. Diagnostician that maintains a running list of the three most likely diagnoses at every step.
2. The test-chooser that selects the highest-value next question or investigation.
3. Challenger that questions assumptions and guards against anchoring on an early, wrong idea.
4. A cost steward that enforces cost-conscious, judicious ordering of tests.
5. Quality-control agent that checks the panel's reasoning before a final diagnosis is committed.

Microsoft says this design helps counter the cognitive biases that trip up individual clinicians, such as anchoring on a first impression or ignoring alternative explanations.

The Headline Numbers: 83% vs 36%

The study evaluated MAI-DxO across two very different sets of real cases. The first was 304 clinicopathological conference cases from the New England Journal of Medicine — notoriously hard teaching cases that normally go to specialist teams. The second was 325 emergency-department admissions drawn from Beth Israel Deaconess Medical Center in Boston.

Paired with OpenAI's GPT-5, MAI-DxO reached 83.2% accuracy on the NEJM cases versus 36.1% for generalist physicians. On the emergency-department cases it scored 80.3% against 46.8% for the human doctors; the single best-performing physician managed 62.5%. The results held up across models from OpenAI, Google, Anthropic, Meta, xAI, and DeepSeek, though GPT-5 and GPT-5.5 produced the strongest scores.

The full benchmark results are summarized below.

Source: "Sequential Diagnosis with Language Models," Science; AAAS/Microsoft AI release. Cost figures are simulated estimates based on hospital chargemaster pricing.

A New Way to Grade Medical AI

A central argument of the paper is that the way we have been testing medical AI is flawed. Most benchmarks rely on static, multiple-choice exam questions, which reward memorized knowledge but say little about whether a system can reason through an unfolding case.

To fix that, Microsoft built the Sequential Diagnosis Benchmark (SDBench), which turns each NEJM case into a step-by-step encounter. A physician or AI begins with a short abstract and must actively request each additional detail from a "gatekeeper" model that reveals findings only when explicitly asked. A separate physician-trained "judge" model scores the final diagnosis against clinical criteria, and every test ordered along the way is assigned a dollar cost, so accuracy and spending are measured together.

The Cost Question Needs Care

Microsoft's blog framing emphasizes that the AI reaches the right answer with fewer tests and at lower cost. That is true in some configurations but not all. Run without any budget limit, MAI-DxO's average workup actually cost more per case than the physicians',  about $5,431 versus a physician average, and roughly $4,979 versus $3,400 on the emergency cases.

The savings appear when the system is told to economize. Budget-constrained settings still beat doctors on accuracy while cutting costs sharply: 80.9% at $2,241 per case, and 79.6% at just $1,303 on the NEJM set. Microsoft also notes the cost model is simplified, it does not account for the invasiveness of a test, patient discomfort, or how long results take to return.

From Preprint to Peer Review

This is not the first time MAI-DxO has made headlines. Microsoft first unveiled the system in June 2025 in a post titled "The Path to Medical Superintelligence," alongside a non-peer-reviewed preprint. At that time the orchestrator ran on OpenAI's o3 model and reported 80% accuracy.

The October 2026 Science paper is the peer-reviewed version, now re-run on newer frontier models. Microsoft says it repeated the entire evaluation suite on GPT-5 and then GPT-5.5 without changing its prompts or orchestration, and the accuracy advantage grew. On complex cases, DxO lifted GPT-5.5's score from about 72% to nearly 87%.

One nuance critics have flagged: against the strongest raw models, the orchestrator's added value shrinks. GPT-5 alone, given a simple prompt, already reached 78.9%, so on the hardest cases DxO's gain over well-prompted GPT-5 is only a few points, though it also trimmed roughly $3,000 per case. Microsoft's researchers acknowledge that accuracy gains naturally diminish as models approach the theoretical ceiling, and that the orchestrator helps most on complex cases and with less-capable models.

"Medical Superintelligence": Microsoft's Bigger Bet

Microsoft AI CEO Mustafa Suleyman has framed the work as a step toward what he calls “medical superintelligence” which the company defines as the capacity to surpass the strongest human-led health systems in medical reasoning and coordination across the entire arc of care. He has said Microsoft intends to field the best clinical-decision-support products in the industry over the next few years, and predicted a further breakthrough in interpreting medical records by 2027.

Dominic King, Microsoft AI's vice president of Health, told reporters there is "something very substantial and exciting here," and pointed to the core problem the company is targeting: a widening gap between the demand for medical expertise and the supply of it. Microsoft positions MAI-DxO as infrastructure to help close that gap, complementing clinicians rather than replacing them.

The Caveats: Why This Isn't a Doctor Replacement, Yet

The study is a striking benchmark result, but it comes with important limits that Microsoft itself acknowledges:

1. It is a benchmark, not a clinical trial. The AI was tested on historical, already-solved cases, not live patients.
2. The human comparison was with generalists, not specialists. These complex cases would normally go to specialist teams, and researchers expect the performance gap to narrow against expert specialists.
3. The doctors worked alone, without colleagues, specialist input, or the collaborative tools they would use in practice.
4. Suleyman has conceded the system "needs clinical validation" before it can be trusted in real care settings.
5. Questions remain about data, privacy, and dependency — Microsoft's headline numbers currently rely on OpenAI's frontier models.

What Happens Next

Microsoft is moving toward real-world evaluation. The company has announced a partnership with Beth Israel Deaconess Medical Center to assess MAI-DxO's clinical reasoning in hospital settings, where consenting patients will interact with the system alongside standard care so that its recommendations can be compared directly with doctors'.

Separately, Microsoft is working with the Mayo Clinic to train a dedicated healthcare foundation model slated for release in 2027, an effort aimed at reducing its reliance on outside models. In a companion article in BMJ Digital Health & AI, the company laid out five further capabilities it believes AI must master, including coordinating clinical care and making treatment decisions, before "medical superintelligence" is anything more than a slogan.

The Bottom Line

For patients and clinicians, the promise is real but not imminent: an AI that can reason through a diagnosis step by step could eventually extend scarce expertise to places that lack it, and cut unnecessary testing. For now, MAI-DxO has cleared a high bar on paper, outperforming generalist doctors on some of medicine's hardest puzzles, while the harder test, proving itself with real patients in real clinics, is only just beginning..