83% accuracy: Microsoft claims AI outperforms generalists in complex diagnoses

Eighty-three percent. That is the accuracy rate achieved by Microsoft's new diagnostic AI system in a simulation of complex medical cases, a figure that more than doubles the performance of generalist physicians. The results, published in Science on Thursday, position the company's technology as a significant leap in automated medical reasoning. Mustafa Suleyman, CEO of Microsoft AI, described the milestone as a critical step toward achieving what the company calls medical superintelligence.

The system, known as MAI-DxO, operates by coordinating a panel of artificial intelligence agents that mimic a team of doctors. Rather than relying on a single model, these agents interact with the patient, order necessary tests, and debate potential diagnoses before reaching a conclusion. In the study, the AI was tested against complex cases drawn from the New England Journal of Medicine. While the AI achieved an 83% success rate, primary care doctors and general internists working individually with reference materials were correct only 36% of the time. It is important to note that such complex cases are typically handled by specialists or physician teams, who were not included in this specific comparison.

Microsoft joins Google and OpenAI in asserting that its artificial intelligence can match or exceed human medical expertise. However, the study's data indicates that much of the improvement stems from the power of the underlying AI models themselves. Microsoft tested the system using models from various providers, including OpenAI, Google, Anthropic, Meta, and DeepSeek. The highest performance was observed when using OpenAI's GPT-5 and GPT-5.5 models. The orchestration system provided additional benefits, but the foundation model remained the primary driver of accuracy.

Cost and efficiency were also key metrics in the evaluation. When using GPT-5 with a basic prompt, the system scored 78.9% on complex cases at a cost of approximately $8,400 per case. The DxO orchestration reduced this cost by about $3,000 while increasing accuracy by roughly four points. For emergency department cases, the gains in both accuracy and cost were minimal when comparing the standalone model to the orchestrated system. With the newer GPT-5.5 model, released in April 2026, the results were mixed. On complex cases, accuracy jumped from 72% to nearly 87%, but no significant improvement was seen in emergency scenarios, where both versions scored around 85%.

The orchestration system proved most valuable when paired with less capable models. For instance, Gemini 2.5 Flash scored 46.4% on its own but improved to nearly 69% with the DxO assistance. Despite these gains, the gap between weaker and stronger models remained larger than the boost provided by the system. Microsoft researchers noted that accuracy gains naturally diminish as models approach the theoretical maximum. A spokesperson stated that the orchestration helps optimize both accuracy and cost, addressing the broader challenge of healthcare delivery.

Suleyman, who leads Microsoft's frontier AI efforts, revealed that the company is collaborating with the Mayo Clinic to train a foundation model specifically for healthcare. This model is planned for release next year and is expected to enable significant advancements. Suleyman predicts that by 2027, there will be a breakthrough in coding for medical records, allowing for highly accurate predictions of how a patient's condition might unfold. He views this foresight as invaluable for clinicians. Suleyman, a co-founder of DeepMind before joining Microsoft in 2024, has long focused on applying AI to medical problems.

The company expects healthcare models to improve in capability by an order of magnitude within the next 24 to 36 months. Suleyman believes that progress will accelerate rapidly once a flywheel effect is established, where models in production are iterated upon using real-world usage to identify and fix gaps. This process has already driven progress in coding and chat applications. He anticipates that a similar dynamic will begin for medical records and clinical decision support next year. Through a new partnership with Beth Israel Deaconess Medical Center in Boston, Microsoft will evaluate the clinical reasoning of DxO across different hospital settings.

Dominic King, vice president of Health at Microsoft AI, stated that the technology is ready for real-world evaluations. The study at Beth Israel will involve patients receiving standard treatment while also engaging with the AI system if they consent. Microsoft and its partners will then compare the AI's recommendations against those of human doctors. King called it the biggest study of its kind, covering both emergency and urgent care environments. This work follows a similar study by Google, which evaluated its AMIE diagnostic AI in urgent care visits in collaboration with Beth Israel. Microsoft aims to integrate its technology with clinicians and healthcare systems to improve outcomes beyond simple benchmarking, positioning itself as a leader in medical AI.

The study coincides with a manifesto outlining the path to medical superintelligence, defined as the capacity to surpass the strongest human-led health systems in reasoning and coordination across the full arc of care. Microsoft hopes to move beyond individual tasks like answering exam questions to deeper integration with clinical workflows. The company already sells dictation and radiology tools to clinicians and views this new system as an expansion of those efforts. Suleyman emphasized the competitive landscape, stating that Microsoft is committed to accelerating the introduction of this technology to the world. He believes that having the number one state-of-the-art medical clinical decision support product will be a key differentiator for the company in the coming years.