Every year, 20 million people globally are diagnosed with cancer.1 Every patient is unique, with hundreds of distinct tumor sub-types, each demanding treatment protocols involving new drugs, combinations, clinical trials, and device-based therapies. Top cancer centers rely heavily on multidisciplinary tumor boards—dedicated sessions where radiologists, pathologists, surgeons, oncologists, genetic counselors, and other specialists undertake sophisticated analysis of vast patient data and knowledge to align on personalized care plans.
Because of the immense preparation and specialization required, less than 1% of these patients have access to these personalized treatment plans, which have demonstrably improved patient outcomes.
A recent American Society of Clinical Oncology (ASCO) study highlighted that clinicians spend between 1.5 to 2.5 hours per patient, meticulously reviewing imaging, pathology slides, clinical notes, and genomic data.2 And cancer care is just one example of the complex data analysis healthcare requires. Agentic AI holds the potential to reduce administrative friction and further transform care delivery.
The healthcare agent orchestrator is available now in the Azure AI Foundry Agent Catalog. It features pre-configured agents with multi-agent orchestration and open-source customization options that allow developers and researchers to build agents that coordinate multi-disciplinary multimodal healthcare data workflows, such as tumor boards, and streamline deployment into healthcare enterprise productivity tools (such as Microsoft Teams and Word). Modular, general reasoners as well as specialized, multimodal AI agents work together to address tasks that would take hours, with the goal to effectively augment clinician specialists with customized cutting-edge agentic AI.
By integrating the latest capabilities from across Microsoft, the healthcare agent orchestrator can manage analysis and reasoning over diverse healthcare data types—ranging from imaging (DICOM files) and pathology (whole-slide images) to genomics data and clinical notes from electronic health records (EHRs). Each agent is equipped with advanced AI models from Azure AI Foundry, combining general-purpose reasoning capabilities with healthcare-specific modality models to drive actionable insights grounded in multimodal clinical data.
Key capabilities of healthcare agent orchestrator
Orchestrating agentic capabilities that can reason over complex EHR data and augment time-consuming tasks like building a chronological patient timeline, determining cancer stage, using specific reference guidelines, reviewing radiology and pathology images, synthesizing current medical literature, referencing treatment guidelines, surfacing relevant clinical trials, and generating customized reports.
Providing tools that connect enterprise healthcare data through Microsoft Fabric and the fast healthcare interoperability resources (FHIR) data service.
Ensuring interoperability and integration into existing workflows, including distribution to familiar tools the majority of healthcare organizations already use—Teams, Word, PowerPoint, and Microsoft 365 Copilot—where users can interact with AI agents.
Providing robust explainability capabilities in agentic AI-generated outputs, such as grounding responses to the source EHR data—critical for validation, trust, and adoption in high-stakes healthcare environments.
Researchers and developers at leading cancer care institutions—including Stanford University, Johns Hopkins, Providence Genomics, Mass General Brigham, and the University of Wisconsin School of Medicine and Public Health—are currently exploring the healthcare agent orchestrator to study how agentic AI could deliver value to complex clinical tasks such as cancer care.
“Stanford Medicine sees 4,000 tumor board patients a year, and our clinicians are already using foundation model generated summaries in tumor board meetings today (via a PHI safe instance of GPT on Azure). The new healthcare agent orchestrator has the power to streamline this existing workflow by reducing fragmentation (saving time by avoiding copy-pasting) and enables surfacing new insights from data elements that were challenging to search, such as trial eligibility criteria, treatment guidelines, and real-world evidence. Stanford Health Care is excited further research the potential of using the healthcare agent orchestrator to build the first generative AI agent solution used in a production setting for real-world care for our cancer patients.”
—Dr. Mike Pfeffer, Chief Information Officer, Stanford Health Care and Stanford School of Medicine
“The vision of the healthcare agent orchestrator is to rapidly surface, summarize, and take action on relevant multimodal medical information for each complex cancer case, so hours of review can become minutes. Collaborating with Microsoft allows us to explore the value of these models for tumor boards and beyond.”
—Dr. Joshua Warner, Radiologist at UW Health and Assistant Professor of Radiology, UW School of Medicine and Public Health
Early development collaborations featured the integration of this multi-agent workflow into Teams chats, where, for example, group chats enabled conversations between multiple human experts and specialized healthcare AI agents connected to specific healthcare data. It demonstrated the promise to significantly enhance efficiency and collaboration among clinical providers. This capability is already bringing clinicians and developers together to build the agentic healthcare applications of the future: the catalyst is the powerful combination of healthcare-specific agents using general reasoning models and multimodal healthcare foundation models alongside the ability to interact directly with custom agents using Teams.
For example, Johns Hopkins oncologists Dr. Vasan Yegnasubramanian, Dr. Elsa Anagnostou, and Dr. Taxiarchis Botsis and their developer teams in the Johns Hopkins inHealth Precision Medicine program and Molecular Tumor Board are providing their expertise to refine and test the system to ensure it would have high utility if used in their clinical and precision medicine applications.
Coordinating collaboration of specialized agents
The healthcare agent orchestrator builds upon recent research and releases from Microsoft Research and our collaborators. It coordinates collaboration of specialized agents designed explicitly for complex multidisciplinary clinical workflows like cancer care.
The orchestrator leverages Semantic Kernel and Magentic-One to coordinate agents, maintain shared memory, and interact with the human in the loop.
The patient history agent leverages Universal Medical Abstraction to organize patient data chronologically.3 Manual work that can take experts over three hours happens in minutes.
The radiology agent leverages customer fine-tuned models like CXRRepotGen/MAIRA-2 to analyze radiology images for a second read.4
The pathology agent demonstrates how to connect to external agents like Paige.ai’s “Alba” pathology agent to address complex queries related to pathology images (available in preview).5
The cancer staging agent refers to the American Joint Committee on Cancer (AJCC) clinical guidelines to support accurate cancer staging.
The clinical guidelines agent refers to the National Comprehensive Cancer Network (NCCN) clinical guidelines to suggest recommended treatment plans.
The clinical trials agent identifies eligible clinical trials by matching patient profiles against databases such as ClinicalTrials.gov. This can result in more than double the recall improvement compared to the publicly available Critera2Query baseline.6
The medical research agent delivers actionable, evidence-based guidance grounded on graph-based knowledge from trusted medical journals.
The report creation agent automates comprehensive, integrated, richly formatted reporting that serves as a trusted reference during multidisciplinary meetings.
“As we progress towards the routine use of multi-agent systems, the healthcare agent orchestrator demonstrates the power to simplify the integration of various models and agents with productivity tools that clinicians are already using. The flexible orchestration framework will make it easy for us at Paige to continue to focus on our pathology agents while enabling their integration into the larger cancer care workflow and leverage access to multi-modal data.”
—Razik Yousfi, Chief Executive Officer of Paige.ai
The orchestrator is intentionally open-ended: any approved agent—including third-party—that exposes an API, tool wrapper, or MCP endpoint can be pulled into a Teams conversational thread. Paige.ai is shipping their Alba agent in preview, the first example of an external agent that can be connected to healthcare agent orchestrator. Built on Paige’s foundation-scale vision models and coupled with a conversational large language model (LLM) front-end, Alba delivers real-time conversational digital pathology insights such as tumor grade, morphology, and biomarker status directly from whole-slide images.
“Providence clinical researchers have begun leveraging advanced AI capabilities provided by the healthcare agent orchestrator to quickly and efficiently parse through large sets of publications, clinical trials and electronic health records. We are excited about its potential to enhance our ability to interpret genomics and match clinical trials in the molecular tumor boards, ultimately benefiting patient care by providing more precise and timely treatment options. Its integration into our workflows also will help streamline communication and collaboration among clinical providers, ensuring that critical clinical information is shared promptly and accurately. As we continue to explore new ways to understand the biology of cancer, its capabilities will be instrumental in driving medical discoveries and advancing cancer treatment.”
—Carlo Bifulco, MD, Chief Medical Officer of Providence Genomics and research faculty at the Earle A. Chiles Research Institute
Empowering developers to accelerate innovations for care teams
As clinical care complexity escalates, the healthcare agent orchestrator empowers developers to confidently navigate the accelerating era of agentic AI, collaborate with clinicians, and democratize precision medicine tools by surfacing these capabilities into existing workflows. The initial framework is designed to study the opportunity of assisting tumor boards. The ultimate vision is to empower healthcare and life science developers to research how agentic AI capabilities could impact clinicians and patients more widely by providing real-time support to multidisciplinary care teams across the healthcare ecosystem.
Healthcare developers and clinical organizations are invited to explore healthcare agent orchestrator, available through the Azure AI Foundry Agent Catalog. Engage with the next generation of AI-powered healthcare agents today.
4 MAIRA-2: Grounded Radiology Report Generation, June 6, 2024
Disclaimer
Healthcare agent orchestrator is intended for research and development use. It is not designed or intended to be deployed in clinical settings as-is nor is it intended for use in the diagnosis or treatment of any health or medical condition, and its performance for such purposes has not been established. You bear sole responsibility and liability for any use of healthcare agent orchestrator, including verification of outputs and incorporation into any product or service intended for a medical purpose or to inform clinical decision-making, compliance with applicable healthcare laws and regulations, and obtaining any necessary clearances or approvals.
Matthew Lungren, MD, MPH
Chief Scientific Officer, Health and Life Sciences
Dr. Matthew Lungren is the Chief Scientific Officer for Microsoft Health and Life Sciences, where he leads efforts to translate cutting-edge technologies—including generative AI and cloud services—into real-world healthcare applications. He is also a Clinical Associate Professor at UCSF and an Adjunct Professor in Biomedical Data Science at Stanford University.
See more articles from this author
GIPHY App Key not set. Please check settings