The One Article That Unblocked My Multi-Agent Architecture

My team has been designing an agentic system recently. Building a single agent felt manageable, but the moment we tried to wire multiple agents together, the questions started piling up.

What orchestration structure do we use? How do agents communicate? How do we handle memory across the system?

Then I found an article by Rohit Ghumare that covered nearly everything I’d been struggling with. Here’s a breakdown of the key ideas alongside my own experience.

Why Multi-Agent at All

As I’ve mentioned in previous posts, multi-agent isn’t always the answer. But I spent all of last year failing at context management with a single agent.

The problem: a single agent’s context window fills up fast, and once it does, the agent starts forgetting. Worse, when one agent tries to handle multiple domains simultaneously, its judgment degrades.

Multi-agent solves this by splitting concerns - but it introduces coordination overhead. Managing that overhead is the real challenge, and the article tackles it head on.

Three Orchestration Patterns

This was the most practical section. Instead of organizing around “what’s impressive,” the article focuses on when to use what.

Supervisor Pattern

A manager agent decomposes tasks, distributes them to workers, and synthesizes the results.

• Best for: Tasks that split cleanly into subtasks, or when you need an audit trail
• Sweet spot: 3 - 8 workers
• Watch out: Every decision routes through the supervisor, creating a potential bottleneck

Swarm Pattern

No central manager. Agents communicate peer-to-peer and self-organize.

• Best for: Problems that require diverse perspectives, or where real-time responsiveness matters
• Watch out: Duplicate work, infinite loops, suboptimal convergence. Debugging is painful

Hierarchical Pattern

A recursive extension of the supervisor pattern. Top-level coordinator delegates to mid-level managers, who delegate to workers.

• Best for: Systems with 10+ agents, or when strategy and tactics need clear separation
• Watch out: Token costs spike with every coordination layer

From personal experience, the supervisor pattern has been the most stable for me. The key is getting worker distribution and error handling right - if you don’t, the supervisor itself becomes the point of failure.

How Agents Communicate

If orchestration defines the structure, communication defines how information actually flows between agents.

• Shared State: All agents read from and write to a single state object. Simple to implement, easy to debug. Most teams should start here
• Message Passing: Asynchronous communication through an event bus. Use this when you need loose coupling between agents
• Handoff: Explicit baton-passing with full context transfer. Works well for fixed-order pipelines

Memory Architecture for Multi-Agent Systems

The core memory problem is straightforward: how do you share state without causing collisions? The article breaks it into three patterns.

Session-Based Memory

Each agent works in an isolated local state. Changes merge into shared memory only when the session ends.

• Best for: Parallel workers that need to operate independently without interference
• How it works: Agent takes a snapshot of shared state at session start, works locally, merges deltas at session end
• Advantage: Collision-free parallel processing

Window Memory

A sliding window retains only the most recent N exchanges. Older entries get compressed into summaries.

• Best for: Long conversations where you need to preserve context without burning tokens
• How it works: When the window overflows, the oldest third gets summarized and compressed
• Advantage: Prevents unbounded state growth

Episodic Memory

Stores the collaboration history of specific agent combinations and uses it to inform future decisions.

• Best for: Agent teams that collaborate frequently and should improve based on past outcomes
• How it works: Records which agent combinations succeeded or failed on which tasks
• Advantage: Enables decisions like “this combination worked well last time - use it again”

Production Considerations

Token Costs

• A supervisor + 4 workers: ~1K for decomposition + ~12K across workers + ~2K for synthesis = roughly 15K tokens per task
• The same task with a single agent: about 4K tokens. Coordination costs nearly 4x more
• Optimization: Cache supervisor instructions, structure worker outputs as structured data, invoke workers only when necessary

Latency

• At 2 - 5 seconds per LLM call, 4 agents in series means 12+ seconds. In parallel, 3 - 4 seconds
• Independent tasks should always run in parallel

Error Propagation

• Timeouts: Mandatory at every layer
• Circuit breakers: Stop calling an agent after N consecutive failures
• Graceful degradation: Core functionality should work even if some agents are down
• State isolation: A worker failure must never corrupt shared state

If you can’t observe it, you can’t fix it. Monitoring is a day-one requirement.

Anti-Patterns to Avoid

• Over-orchestration: Linking agents that could run independently
• God agent: One agent doing everything defeats the purpose
• Ignoring costs: Deploying without token monitoring leads to surprise invoices
• No fallback: Assuming every agent will always be available

The Takeaway

The article’s conclusion stuck with me:

Build one agent. Identify where it breaks. Add a second agent at that breaking point. Layer in a supervisor if needed. Repeat.

I started with an ambitious hierarchical design and ended up simplifying to a supervisor with three workers. The lesson: build a stable two-agent collaboration first, then scale from there.

If you’re in the middle of designing a multi-agent system, I’d recommend giving the original article a read.

Source: Building Effective Multi-Agent Systems