Multi-agent system
A multi-agent system is a distributed architecture where autonomous AI instances collaborate by delegating tasks, sharing information, and verifying outputs to solve complex problems.
What it is
In practice, this involves a coordinator agent that breaks down a high-level request into sub-tasks and assigns them to specialized agents with distinct system prompts or tools. These agents execute these tasks, often passing outputs through a message bus or state management layer to refine the final response. While a simple implementation might involve 2 or 3 agents, complex systems can scale to dozens of persistent workers interacting over several rounds of turn-based communication. Each interaction adds round-trip latency and cumulative token costs, significantly increasing the overhead per user request.
Why it matters
You should use this architecture when a single model call fails to provide reliable results for complex, multi-step workflows. If you ignore the overhead, you will likely hit API rate limits or experience runaway costs due to excessive token consumption across multiple internal agent rounds. Building this requires robust state management and error handling, as a failure in one agent often cascades through the entire chain. Choosing between a single, highly capable model versus a team of specialized agents directly impacts your infrastructure, latency budget, and testing strategy.
In practice
You typically implement this by managing a shared chat history or state machine that persists between agent turns. In production, you will monitor inter-agent message logs for cycles or stuck loops, often configuring a max-depth parameter to prevent infinite recursion. You might use libraries like LangGraph or AutoGen to handle the state transitions and message routing between your agents.
The tradeoff
The primary tradeoff is between system reliability and operational cost, as increasing agent complexity improves accuracy but exponentially inflates latency and token usage.