Cheat sheet · LLMs & Agents

Multi-Agent Systems & Orchestration cheat sheet

The key ideas on one page — plus common pitfalls and interview tips. Skim it before an interview, or print it for revision.

A practitioner's reference for designing, orchestrating, and operating LLM-based multi-agent systems: when to use one agent vs. many, orchestration topologies, communication and memory, tool use and control flow, and production concerns like cost, reliability, evaluation, and security.

Foundations: What an Agent Is & When to Go Multi-Agent

An LLM agent = model + tools + memory + a control loop. The loop is usually the ReAct (reason-act-observe) cycle: the model reasons, calls a tool, reads the observation, and repeats until a stop condition (task done, max steps, or human handoff).
Single agent first: most tasks are cheaper and more reliable with one well-prompted agent and good tools. Reach for multi-agent only when work genuinely decomposes into parallelizable or specialized subproblems.
Strong reasons to split: (1) parallel breadth (research many sources at once), (2) separation of concerns (distinct system prompts and toolsets), (3) context isolation so one subtask's tokens don't pollute another, (4) role specialization (e.g. coder vs. reviewer).
Multi-agent costs: token usage is typically ~4x a single chat for a tool-using agent and up to ~15x for a multi-agent system, because each agent re-establishes context. Anthropic reported its multi-agent research system used ~15x the tokens of a normal chat. Add latency, coordination failures, and harder debugging.
Rule of thumb: parallelizable, read-heavy, breadth-first tasks (research, exploration) benefit; tightly coupled, sequential, write-heavy tasks (most coding, where edits depend on each other) usually do not.
Agentic is not the same as multi-agent. Adding planning, reflection, or tool loops to ONE agent (agentic patterns) frequently beats orchestrating several agents.

Orchestration Topologies

Orchestrator-worker (manager/lead): a lead agent decomposes the task, spawns subagents with focused objectives, then synthesizes results. Best for dynamic, open-ended breadth. The lead must give each worker an explicit objective, output format, tool guidance, and scope boundaries.
Sequential pipeline (chain): deterministic hand-off A->B->C, each agent transforms the output. Predictable and cheap, but brittle if an upstream step fails or drifts.
Parallel / map-reduce (fan-out, fan-in): run N independent workers concurrently, then a reducer merges. Great for breadth; watch for duplicated work and merge conflicts.
Group chat / debate / blackboard: agents share a message space and iterate; useful for critique-and-refine and consensus, but prone to loops and token blow-up without a turn limit or moderator.
Hierarchical: orchestrators of orchestrators (teams of teams). Powerful for very large tasks, but coordination overhead grows fast - keep depth shallow.
Graph / state machine: nodes = agents/steps, edges = transitions with conditional routing and cycles. The dominant production pattern (e.g. LangGraph) because it makes control flow, retries, and checkpoints explicit.
Routing / handoff (swarm): a triage agent classifies intent and hands off to a specialist; specialists can hand back. Good for customer-support-style partitioning.
Agent-as-tool: expose a subagent as a callable tool of a parent agent (supported in OpenAI Agents SDK and the Claude Agent SDK). The parent keeps control of the loop and gets a typed result, instead of a free-form conversation.

Communication, State & Memory

Message protocols: agents exchange structured messages (role, content, tool_calls, tool_results). Prefer typed/structured outputs (JSON schema, function-call args) over free text so hand-offs are parseable and validatable.
Shared vs. isolated context: a shared scratchpad/blackboard maximizes coordination but burns tokens and leaks noise; isolated contexts (each subagent gets only what it needs) scale better. Pass distilled summaries between agents, not raw transcripts.
Memory tiers: short-term (in-context working memory / conversation), long-term (vector store, knowledge graph, or files) retrieved on demand, and procedural (tools/skills). Persist intermediate artifacts to an external store so agents can resume.
State management: a central typed state object (e.g. a typed dict / graph state) threaded through the system enables checkpointing, time-travel/replay, and human-in-the-loop pauses - essential for long-running, durable workflows.
Interop standards: MCP (Model Context Protocol, Anthropic-originated, now broadly adopted) standardizes how agents connect to tools and data sources; A2A (Agent2Agent, donated to the Linux Foundation in 2025) targets cross-agent/cross-vendor communication. Knowing both signals current awareness.
Context engineering > prompt engineering at scale: actively curate what enters each agent's window (retrieval, compaction, summarization) to avoid 'context rot' and stay under token limits.

Tool Use, Routing & Control Flow

Tools are the agent's hands: define them with crisp names, descriptions, and JSON schemas. Vague tool descriptions are a top cause of failure - the model picks the wrong tool or malforms args. Be prescriptive about WHEN to call a tool, not just what it does.
Routing decides which agent/tool handles a request: LLM-driven (model classifies), rule/embedding-based (cheaper, deterministic), or hybrid. Use cheap deterministic routing where possible; reserve LLM routing for genuine ambiguity.
Loop control is mandatory: cap max steps/turns, add stop conditions, and detect no-progress loops. Unbounded ReAct loops are the classic runaway-cost bug.
Reflection / self-critique: a separate reviewer/critic step (or the same agent in a critique role) catches errors before final output; pairs well with evaluator-optimizer loops.
Human-in-the-loop checkpoints: gate irreversible/high-risk actions (payments, deletes, prod writes) behind approval; design the graph to pause and resume on human input.
Tool-side guardrails: validate and sandbox tool inputs/outputs, enforce least-privilege credentials per agent, and rate-limit - agents call tools in unexpected ways.

Reliability, Cost & Evaluation in Production

Failure modes compound: for N independent steps each required for success with reliability p, end-to-end success is approximately p^N. Five 90%-reliable steps give ~59% overall - so isolate, validate, and add per-step retries/fallbacks. Real failures are often correlated, so p^N is a rough model, not an exact figure.
Resumability: long runs hit transient errors (rate limits, tool timeouts). Use durable state + checkpoints so you resume from the last good step instead of restarting (and re-paying for) the whole run.
Cost & latency control: cap fan-out width, use prompt/context caching for shared system prompts, route easy steps to smaller/cheaper models (model cascading), and compact context aggressively. Budget tokens per agent.
Observability is non-negotiable: trace every step (inputs, tool calls, outputs, tokens, cost, latency) with a tracing tool (LangSmith, OpenTelemetry GenAI, Langfuse, etc.). You cannot debug emergent multi-agent behavior without full traces.
Evaluation: combine end-to-end task-success metrics with step-level/trajectory evals (right tool? right hand-off?). LLM-as-judge with a rubric scales eval but must be calibrated against human labels; eval small samples early since agents are nondeterministic.
Security: treat all agent inputs as untrusted - prompt injection via tool results or retrieved docs can hijack the loop. Enforce least privilege, isolate untrusted content, and constrain what each agent may do.

Frameworks & When to Use Them (2026)

LangGraph: low-level graph/state-machine orchestration with checkpointing, human-in-the-loop, and cycles - the go-to when you need explicit, durable, controllable control flow.
OpenAI Agents SDK: lightweight, production-ready handoff + routing model with built-in tracing and guardrails (the successor to the experimental Swarm, which is now deprecated/educational). Good for triage-and-specialist patterns with minimal abstraction.
CrewAI: role/crew abstraction (agents with roles, goals, tasks) - fast to prototype collaborative teams; less granular control than a graph.
AutoGen / AG2: conversational multi-agent + group chat; strong for research, code-execution agents, and human-in-the-loop chat patterns (AG2 is the community fork of AutoGen).
Anthropic Claude Agent SDK + MCP: build agents around Claude with standardized tool/data connectivity via MCP; pair with subagent orchestration for research-style breadth.
Choose by control needs: prototyping & roles -> CrewAI/AutoGen; production reliability, branching, retries, durable state -> LangGraph or a custom graph; simple routing/handoff -> OpenAI Agents SDK. Don't add a framework if one agent with tools suffices.

Common pitfalls

Reaching for multi-agent when one agent + good tools would be cheaper, faster, and more reliable - multi-agent can cost up to ~15x the tokens plus coordination risk.
Using multi-agent for tightly-coupled sequential work (e.g. editing code where later edits depend on earlier ones) - agents lack shared context and step on each other.
No loop guard or stop condition, leading to runaway ReAct loops and exploding cost.
Sharing full raw transcripts between agents instead of distilled summaries, causing context bloat, 'context rot', and window overflow.
Ignoring compounding failure: chaining many sub-90% steps yields low end-to-end success; teams forget the p^N intuition and skip per-step validation/retries.
Vague tool descriptions and unstructured (free-text) hand-offs, causing wrong tool selection and unparseable outputs.
No tracing/observability - emergent, nondeterministic behavior becomes impossible to debug or evaluate.
Treating retrieved/tool content as trusted, opening prompt-injection paths; giving every agent broad credentials instead of least privilege.
Confusing 'agentic' (one agent with planning/reflection/tools) with 'multi-agent' (several coordinating agents).
No checkpointing on long runs, so a transient error forces a full, costly restart.

Interview tips

Expect 'when would you NOT use multi-agent?' - answer with the single-agent-first principle and the cost/coupling tradeoffs, not just enthusiasm.
Be ready to sketch a topology (orchestrator-worker vs. pipeline vs. graph vs. handoff) and justify it for a given task (research vs. coding vs. support).
They probe reliability: explain p^N compounding (and that it assumes independent, all-required steps) plus mitigations - retries, validation, checkpoints, human-in-the-loop.
Expect questions on context/memory: isolated vs. shared context, summarization between agents, and the short/long/procedural memory tiers.
Tooling depth: describe how you'd define tools (schemas, when-to-call descriptions), route requests, and guard loops; mention MCP and A2A for current awareness.
Evaluation & observability: how do you test a nondeterministic agent system? (trajectory + end-to-end evals, LLM-as-judge calibrated to humans, full tracing).
Production concerns: cost control (caching, model cascading, fan-out limits), security/prompt-injection, and least-privilege credentials.
Framework awareness without dogma: compare LangGraph vs. CrewAI/AutoGen vs. OpenAI Agents SDK by control/durability needs; pick the simplest thing that works.
A strong closing line: 'start with one agent, add structure only when traces show a real bottleneck' - shows pragmatic engineering judgment.

Read the full lesson