Multi-Agent AI Collaboration: Orchestrating Specialized AI Roles in Production

Why one agent isn't enough — and how to build a team of AI specialists that research, analyze, synthesize, and critique each other's work. With full observability via Langfuse.

Why Single-Agent Systems Hit a Ceiling

I built a 35-tool agentic system that could query databases, build knowledge graphs, and generate research reports autonomously. It worked well — until users started asking complex research questions that required balancing multiple perspectives:

• "Compare drug sensitivity across 3 cancer subtypes and recommend therapeutic targets"
• "What contradicts the current hypothesis about X pathway?"
• "Synthesize findings from 5 datasets and identify gaps in the evidence"

A single agent with 35 tools tries to do everything at once. It becomes a generalist — decent at many things, excellent at nothing. Worse, it can't critique its own work.

The Multi-Agent Architecture

I evolved the system into a 4-role collaboration pattern, each role with a distinct responsibility and personality:

Why Different Models for Different Roles

This tiered approach means ~70% of total tokens are processed by the cheaper Sonnet model (scout + analyst), while the expensive Opus reasoning is reserved for the two steps that need it most. Langfuse makes this cost breakdown visible per-role.

The Orchestration Pattern

# Simplified multi-agent orchestration
import asyncio
from langfuse import observe

class ResearchOrchestrator:
    def __init__(self):
        self.analyst = Agent(model="sonnet", tools=ANALYSIS_TOOLS, system_prompt=ANALYST_PROMPT)
        self.synthesizer = Agent(model="opus", tools=[], system_prompt=SYNTH_PROMPT)
        self.critic = Agent(model="opus", tools=[], system_prompt=CRITIC_PROMPT)

    def _new_scout(self):
        """Fresh scout per task — agents carry state, can't share concurrently."""
        return Agent(model="sonnet", tools=SCOUT_TOOLS, system_prompt=SCOUT_PROMPT)

    @observe(name="multi-agent-research")
    async def research(self, query: str) -> ResearchReport:
        # Phase 1: Parallel data gathering (fan-out)
        # Strands Agent.__call__ is synchronous; use invoke_async() for await
        # Each scout gets its own instance to avoid shared-state corruption
        scout_results = await asyncio.gather(
            self._new_scout().invoke_async(f"Find primary data: {query}"),
            self._new_scout().invoke_async(f"Find ontology context: {query}"),
            self._new_scout().invoke_async(f"Find contradicting evidence: {query}")
        )

        # Phase 2: Analysis (sequential — needs scout output)
        analysis = await self.analyst.invoke_async(
            f"Analyze these findings:\n{scout_results}"
        )

        # Phase 3: Synthesis
        report = await self.synthesizer.invoke_async(
            f"Synthesize into a coherent report:\n{analysis}"
        )

        # Phase 4: Adversarial critique
        critique = await self.critic.invoke_async(
            f"Find flaws in this report:\n{report}"
        )

        # Phase 5: Revision (if critique found issues)
        if critique.has_issues:
            report = await self.synthesizer.invoke_async(
                f"Revise based on critique:\n{report}\n\nCritique:\n{critique}"
            )

        return report

Observability with Langfuse: You Can't Optimize What You Can't Measure

# Langfuse integration pattern (SDK v3)
from langfuse import observe, get_client

@observe(name="multi-agent-research")
async def research(query: str):
    # Trace metadata attaches to the current span automatically
    langfuse = get_client()
    langfuse.update_current_trace(
        metadata={"query_type": classify_query(query)},
        tags=["multi-agent", "research"]
    )

    @observe(name="scout-phase")
    async def scout_phase():
        return await asyncio.gather(
            run_scout("primary", query),
            run_scout("ontology", query),
            run_scout("contradictions", query)
        )

    @observe(name="analyst-phase")
    async def analyst_phase(data):
        return await analyst(data)

    # Each phase gets its own Langfuse span
    scout_data = await scout_phase()
    analysis = await analyst_phase(scout_data)
    # ... synthesis and critique traced similarly

    # Score the output for quality monitoring
    langfuse.score_current_trace(
        name="user-satisfaction",
        value=1.0,  # Updated later via user feedback webhook
    )

What Langfuse Revealed That Surprised Me

1. The Critic saved money. Counter-intuitively, adding an expensive Opus critic reduced total cost because it caught errors that would have required users to re-run queries. Fewer retries = less total spend.
2. Scout parallelism was bottlenecked. The "fast" scout phase was actually the slowest because one scout was making 4 tool calls sequentially. Langfuse waterfall made this obvious.
3. Per-role token tracking revealed waste. The Analyst was receiving the full scout output (~8K tokens) but only using 30% of it. Adding a summarization step between scout and analyst cut costs 40%.

The Critic Pattern: Why Adversarial AI Matters

The most valuable addition was the Critic agent. Its prompt is explicitly adversarial:

CRITIC_PROMPT = """You are a rigorous scientific reviewer. Your job is to FIND FLAWS.

For each claim in the report:
1. Is it supported by the evidence provided?
2. Are there alternative interpretations the report ignores?
3. Is the confidence level appropriate, or is it overstated?
4. What critical data is MISSING that would change the conclusion?

Be ruthless. A report that passes your review should be publishable."""

In production, the Critic triggers a revision loop ~23% of the time. When it does, the revised report is measurably better (scored via Langfuse user feedback). The other 77% of the time, it adds confidence that the report is solid — which users notice and appreciate.

Deployment: Each Role is a Separate Service

The multi-agent system isn't just logical separation — it's physical separation on Kubernetes. Each role runs as its own Deployment with its own resource profile:

# deployment-data-scout.yaml (excerpt)
spec:
  replicas: 2              # High throughput, parallel queries
  containers:
    - resources:
        requests:
          memory: "512Mi"  # Lightweight — just tool calls
          cpu: "250m"

# deployment-synthesizer.yaml (excerpt)
spec:
  replicas: 1              # One at a time, deep reasoning
  containers:
    - resources:
        requests:
          memory: "1Gi"    # Larger context window handling
          cpu: "500m"

This lets you scale each role independently. Data Scouts scale out for throughput. Synthesizer stays at 1 replica to bound concurrent synthesis — a single pod serializes cross-user requests; size the concurrency semaphore to match your throughput target and expect 429s beyond it. The Critic can be temporarily scaled to 0 in low-priority environments to save cost, but requires a feature flag in the orchestrator to skip the stage — scaling to 0 without one causes connection failures.

Lessons Learned

1. Start with one agent, split when you hit diminishing returns

Don't design multi-agent from day one. I ran a single agent for 6 months before splitting. The split was justified only when I could clearly identify where "thinking harder" wasn't helping and "thinking differently" would.

2. The Synthesizer prompt is the hardest to write

Getting an LLM to resolve contradictions rather than just listing them requires very specific prompting. The breakthrough was telling it to assign confidence levels (0-1) to each finding and use those to weight the synthesis.

3. Langfuse per-role tracing is mandatory, not optional

Without it, you can't answer: "Is the critic actually improving output?" You need data — token costs, latency impact, and revision rates — to justify each role's existence.

If you're building multi-agent systems or have questions about orchestrating AI roles on Kubernetes, feel free to connect.