Multi-agent system

What it is

In practice, this involves a coordinator agent that breaks down a high-level request into sub-tasks and assigns them to specialized agents with distinct system prompts or tools. These agents execute these tasks, often passing outputs through a message bus or state management layer to refine the final response. While a simple implementation might involve 2 or 3 agents, complex systems can scale to dozens of persistent workers interacting over several rounds of turn-based communication. Each interaction adds round-trip latency and cumulative token costs, significantly increasing the overhead per user request.

Why it matters

You should use this architecture when a single model call fails to provide reliable results for complex, multi-step workflows. If you ignore the overhead, you will likely hit API rate limits or experience runaway costs due to excessive token consumption across multiple internal agent rounds. Building this requires robust state management and error handling, as a failure in one agent often cascades through the entire chain. Choosing between a single, highly capable model versus a team of specialized agents directly impacts your infrastructure, latency budget, and testing strategy.

In practice

You typically implement this by managing a shared chat history or state machine that persists between agent turns. In production, you will monitor inter-agent message logs for cycles or stuck loops, often configuring a max-depth parameter to prevent infinite recursion. You might use libraries like LangGraph or AutoGen to handle the state transitions and message routing between your agents.

The tradeoff

The primary tradeoff is between system reliability and operational cost, as increasing agent complexity improves accuracy but exponentially inflates latency and token usage.

Where it appears

Research summaries that use Multi-agent system, each linked to its source paper.

• Debugging LLM Multi-Agent System Failures
  Repair or Resample? Rethinking Failure Debugging in LLM Multi-Agent Systems

  The authors introduce SymTrace, a framework that improves the reliability of debugging complex multi-agent systems by using controlled intervention anchors to replicate and repair execution failures.

• Polaris Multi Agent System for Analytics
  Polaris : Multi Agentic System for Conversational Enterprise Analytics

  Polaris is a multi agentic system designed to improve enterprise data querying and analysis through adaptive task coordination and specialized agent orchestration.

• Multi-Agent System for Deepfake Detection
  Multi-Agent Forensic Reasoning for Generalizable Deepfake Video Detection

  The paper introduces a multi-agent framework that uses specialized observation experts and reinforcement learning to identify forgeries in video content.

• Improving Multi-Agent Systems through Evidence-Guided Revision
  EMAS: Stabilizing Multi-Agent System Evolution through Evidence-Guided Revision

  EMAS evolves the topology and prompts of multi-agent systems using evidence-based revisions to increase accuracy and reduce token costs while keeping the base language model fixed.