83%: The accuracy rate of Microsoft's new diagnostic AI panel

The integration of artificial intelligence into clinical decision-making has accelerated rapidly, with major technology firms now competing to prove their models can rival human expertise. In this context, Microsoft released a study in the journal Science on Thursday, showing that its new diagnostic system, MAI-DxO, correctly identified complex medical cases 83% of the time in a simulation environment.

The core claim is that this multi-agent system, which functions like a virtual panel of doctors, significantly outperforms generalist physicians. In the same simulation using cases from the New England Journal of Medicine, primary care doctors and general internists working individually with reference materials were correct only 36% of the time. Mustafa Suleyman, CEO of Microsoft AI, described the result as a step toward medical superintelligence, defined as surpassing the strongest human-led health systems in reasoning and coordination. However, a key caveat remains: the study did not compare the AI against specialist physicians or teams, who typically handle cases of this complexity in real-world settings.

The study’s data suggests that much of the performance gain comes from underlying model power rather than just the orchestration layer. Microsoft tested the system using models from OpenAI, Google, Anthropic, Meta, and DeepSeek. The best results were achieved with OpenAI’s GPT-5 and GPT-5.5. When using GPT-5 on its own with a basic prompt, the accuracy was 78.9% at an estimated cost of $8,400 per case. The DxO system reduced that cost by about $3,000 but improved accuracy by only four points. With the newer GPT-5.5 model, the orchestration layer had a more pronounced effect, raising accuracy from 72% to nearly 87% on complex cases while slightly lowering costs.

Cost and complexity remain central variables in these results. On emergency department cases, there was little improvement in either accuracy or cost between the standalone models and the orchestrated system. The benefits of the DxO framework were most evident with less capable models. For instance, Gemini 2.5 Flash scored 46.4% alone but nearly 69% when assisted by the system. Microsoft researchers noted that accuracy gains naturally shrink as models approach their theoretical maximum, suggesting the orchestration helps optimize the balance between accuracy and expense.

Looking ahead, Suleyman stated that Microsoft is collaborating with the Mayo Clinic to train a healthcare foundation model, with a release planned for next year. He expressed confidence that the company will see a coding style breakthrough for medical records in 2027, allowing for highly accurate predictions of how a medical record might unfold. He expects healthcare models to become an order of magnitude better in capability within the next 24 to 36 months, driven by a flywheel effect where real-world usage identifies gaps for improvement.

To validate these capabilities in real-world settings, Microsoft has partnered with Beth Israel Deaconess Medical Center in Boston. The goal is to evaluate DxO across different hospital environments, comparing its recommendations against those of human doctors. Patients will receive standard treatment, while those who consent will also engage with the AI. Dominic King, vice president of Health at Microsoft AI, noted that this study will cover both emergency and urgent care settings, aiming to provide compelling evidence for broader deployment. This effort follows a similar real-world evaluation of Google’s AMIE diagnostic AI, published in The Lancet earlier this month.

Microsoft views this work as an expansion of its existing clinical tools, such as dictation and radiology software. Suleyman emphasized that the company aims to build state-of-the-art clinical decision support products, arguing that this focus could differentiate Microsoft from other frontier labs. He remains optimistic about the trajectory, stating that the progress happens quickly once the flywheel of model iteration and real-world usage begins to turn in the medical sector.