Beyond Benchmark Accuracy: Failure Modes of Multi-Agent Coordination in Safety-Critical Clinical Workflows

Yuanchen Bai1, Zijian Ding2, Shaoyue Wen3,4, Xiang Chang1, Angelique Taylor1
1 Cornell University, 2 University of Maryland, College Park, 3 Imperial College London 4 New York University
Paper Appendix Code arXiv
Study Design Pipeline

We designed a controlled test case in a healthcare setting that simulates real-world complexity, serving as a testbed to examine how hierarchical MARS systems operate under high-stakes conditions. Our exploration goes beyond surfacing coordination patterns by analyzing how three factors shape system-level performance: contextual knowledge, communication structures, and model reasoning. κ = 1 indicates the inclusion of contextual and procedural knowledge, while κ = 0 corresponds to its absence. σ = 1 denotes an enhanced communication structure, and σ = 0 reflects its absence. ω specifies the underlying model, either GPT-4o-2024-08-06 or o3-2025-04-16.

Abstract

Multi-agent systems are increasingly proposed to automate multi-step, safety-critical clinical workflows, yet benchmark accuracy is an insufficient proxy for deployment readiness: the coordination failures that matter for patient safety surface across the steps of the process, not in any single accuracy metric. In a simulated emergency-room onboarding workflow run entirely by LLM agents, we diagnose these failures across two studies that vary contextual knowledge, communication structure, and model reasoning. Our preliminary results suggest that team structure, rather than knowledge availability, may be the primary bottleneck: several failure modes persist even when an exhaustive knowledge base is supplied. We also surface a deployment risk that benchmark scores alone would miss: a stronger reasoning model (o3) plans more capably than a non-reasoning model (GPT-4o) yet appears to introduce a wider variety of failure modes. These early findings suggest that process-level evaluation, auditable coordination, and clinician-in-the-loop oversight are prerequisites for safe deployment.

Study 1: Evaluation
Study 1: Contextual Knowledge

We developed a knowledge base (KB) containing contextual and procedural knowledge as a shared resource, analogous to organizational documentation, to ground MARS team behavior and decision-making. We evaluated the effectiveness of this contextual knowledge on MARS performance at both the manager and subordinate levels across seven dimensions. Our analysis shows that five critical failure modes persist even with a detailed KB. Annotated example traces are attached below in section "Coordination Failure Modes" below. These findings indicate that while sufficient contextual knowledge is necessary, system structure remains the primary bottleneck for achieving robust coordination.

Study 2: Structure and Reasoning
Study 2: Reasoning Behavior Cards

We identify four major themes in MARS coordination patterns, each comprising several sub-themes. To contextualize these sub-themes, we annotate each with ‘✓’ or ‘✗’ to indicate whether its implications are positive or negative within our test scenario. We also report the frequency of each sub-theme across 20 traces for both GPT-4o and o3. We find distinct behavioral profiles which underscore trade-offs between reasoning and non-reasoning models. For each sub-theme, we provide representative examples along with accompanying comments (green box: [What went well:], red box: [What went wrong:]) in section "Reasoning Behavior Cards" below.

Coordination Failure Modes

Reasoning Behavior Cards

BibTeX

@misc{bai2025masmarscoordinationfailures,
        title={From MAS to MARS: Coordination Failures and Reasoning Trade-offs in Hierarchical Multi-Agent Robotic Systems within a Healthcare Scenario}, 
        author={Yuanchen Bai and Zijian Ding and Shaoyue Wen and Xiang Chang and Angelique Taylor},
        year={2025},
        eprint={2508.04691},
        archivePrefix={arXiv},
        primaryClass={cs.RO},
        url={https://arxiv.org/abs/2508.04691}, 
  }