Microsoft AI diagnostic system outperforms generalist physicians in complex case simulation

AI Diagnostic System Performance

Microsoft's AI diagnostic system, MAI-DxO, demonstrated a higher diagnostic accuracy than generalist physicians in a recent simulation. The system correctly diagnosed 83% of complex medical cases, while individual primary care doctors and general internists achieved 36% accuracy on the same cases. These cases were sourced from the New England Journal of Medicine.

System Architecture and Functionality

MAI-DxO operates by coordinating several AI agents that mimic a panel of doctors. These agents interact by questioning a patient, ordering tests, and debating a diagnosis. This orchestration system is designed to improve upon the performance of individual AI models.

Underlying AI Models and Cost Analysis

The study indicated that much of the accuracy improvement stemmed from more powerful underlying AI models, with MAI-DxO's orchestration adding further gains. Microsoft tested MAI-DxO with models from various companies, including OpenAI, Google, Anthropic, Meta, and DeepSeek. The most favorable results were obtained using OpenAI's GPT-5 and GPT-5.5 models.

Microsoft also evaluated MAI-DxO on cost. Running GPT-5 alone on complex cases cost approximately $8,400 per case with 78.9% accuracy. MAI-DxO reduced this cost by about $3,000 while increasing accuracy by four points. With GPT-5.5, MAI-DxO increased accuracy from 72% to nearly 87% at a slightly lower cost on complex cases. However, for emergency department cases, the improvements in accuracy and cost were minimal when comparing GPT-5 or GPT-5.5 with and without MAI-DxO.

Implications for Medical Expertise

Mustafa Suleyman, CEO of Microsoft AI, described these results as a step towards "medical superintelligence." Dominic King, Vice President of Health at Microsoft AI, noted that the biggest challenge in healthcare delivery is the growing mismatch between the demand and supply of medical expertise. This AI system aims to help address this gap, particularly in complex diagnostic scenarios.