When designing a multi-agent system, one of the most consequential architectural decisions is the orchestration pattern. The two dominant approaches — supervisor and pipeline — have distinct strengths that suit different problem types, and distinct operational costs that rarely appear in the diagram. Choosing wrong produces brittle systems that are expensive to maintain and hard to reason about during an incident.
The pipeline pattern
In the pipeline pattern, agents are arranged in a linear sequence where the output of one agent feeds directly into the next. Think of an assembly line: Agent A processes raw input, passes it to Agent B for enrichment, then Agent C for analysis, and finally Agent D for output generation. Each agent has a single, well-defined responsibility, and the path through the system is fixed before the first request arrives.
Pipelines excel when a process has clear, sequential steps with predictable data transformations. Document processing is the classic case: ingestion, extraction, validation, classification, output. The advantages are simplicity, predictability, and straightforward debugging — you can inspect the state between any two agents, replay a single step, and reason about cost per transaction with a calculator rather than a dashboard.
Where pipelines break
Pipelines break when the process is not really linear. Real workflows contain exceptions, rework loops, and conditional branches that people resolve by walking to a colleague's desk. Every one of those becomes a new edge in a static graph, and the graph stops being readable at roughly the point where it stops being a pipeline. The reliable tell is a routing configuration full of conditionals that nobody on the team can explain end to end.
The supervisor pattern
The supervisor pattern uses a central orchestration agent that decides at runtime which specialized agents to invoke and in what order. The supervisor receives the input, assesses the task, delegates to the appropriate agents, collects their results, and may make further delegation decisions based on what came back.
Supervisors shine when the process requires genuine decision-making. Customer operations is a good example: a supervisor might route a simple question to a knowledge agent, an account issue to a billing agent, and a complex complaint to an escalation path with a human at the end of it. The advantage is adaptability — the system can compose agents in orders nobody enumerated in advance.
The costs nobody prices in
A supervisor buys that adaptability with tokens, latency, and variance. Every delegation decision is an additional model call before any work happens, so cost per transaction rises and tail latency gets worse. More importantly, the same input can take two different paths on two different days. That makes regression testing harder, incident review harder still, and it means your evaluation suite has to assert on outcomes rather than on the trace. Pipelines are cheap to reason about precisely because the path is the code.
Failure modes differ, so instrumentation must too
A pipeline usually fails loudly, at a step boundary, with an inspectable input. A supervisor fails quietly: it loops, it over-delegates, or it confidently hands the task to the wrong specialist and returns a fluent answer built on the wrong evidence. Supervisors therefore need budget caps on steps, tokens, and wall-clock time; a hard stop that escalates to a human rather than retrying; and a delegation trace stored as a first-class artifact, not as log lines. Build that instrumentation with the first version — retrofitting it after the first production incident is how teams lose a quarter.
Hybrids are the normal answer
In practice, most production systems are hybrids: a supervisor owns the top-level workflow, while each specialist task inside it runs as a deterministic pipeline. Our MedAgent reference architecture is designed this way — a supervisor orchestrates the workflow, while a step such as document extraction runs as a fixed pipeline of ingestion, extraction, validation, and confidence scoring. Reference architecture means exactly that: a starting design we customize, build, and deploy around your data, systems, and compliance constraints, not a product you switch on.
How to choose
Choose a pipeline when the steps are known in advance, the branch count is small, volume is high, and you need predictable cost per transaction. Choose a supervisor when the input space is open, the correct next step depends on what the previous step found, and the value of the system is precisely in handling the cases a script hands back. When both are arguable, start with the pipeline — it is cheaper to run and easier to test, and adding a supervisor above a working pipeline later is a far smaller change than untangling a supervisor you did not need.
The architecture question is downstream of a process question. Map the real workflow first, exceptions included, and the pattern usually picks itself.
▸ SHARE THIS ARTICLE
▸ WRITTEN BY