Lift the hood and it’s one machine.

Strip the domain logic away and every production vertical agent is the same architecture: a workflow engine that delegates bounded decisions to LLMs, assembles fresh context every turn, treats every model output as an unverified proposal, and logs everything into a trace stream that feeds evals and training. The agent loop itself is mostly a while loop—I’ve written about that. This post is about everything production wraps around the loop, and the interesting part: almost every wrapper is now a purchasable component. Walk the machine once, end to end, and you get two things at the same time—the generic pattern, and a map of the startup landscape that sells it.

Here’s the pattern. Everything below is a tour of it, one box at a time.

For each component: what it does, why it exists, and who sells it.

1. Channels and voice I/O ¶

Everything enters through a channel adapter—web chat, SMS, email, API, phone. Text channels are easy. Voice is where this stage becomes a product category, because a natural conversation gives you roughly 800 milliseconds per turn, and that budget has to cover speech-to-text, deciding the caller has actually finished talking (endpointing—genuinely hard), the model, and text-to-speech, all streaming.

The landscape splits into layers people routinely conflate. STT: Deepgram, AssemblyAI. TTS: ElevenLabs, Cartesia. And a layer above both: the voice pipeline—Vapi, Retell, LiveKit, Pipecat—which wires STT, LLM, and TTS together with interruption handling and telephony. Vapi is not a speech model; it’s the plumbing that makes speech models conversational. Holding that budget while the agent reasons mid-turn is precisely why Giga rebuilt this stage instead of buying it.

2. Intent extraction and routing ¶

Before the expensive model wakes up, a small fast one decides what this request is: classify the intent, verify the caller, pick the workflow. This is almost always an in-house fine-tune—the latency budget demands a small model, and the training labels are your own production traffic, so there’s little to buy.

The adjacent buyable layer is model routing—OpenRouter, LiteLLM, Martian—deciding which model serves which call. When a company says its platform is “LLM-agnostic,” this component is what they mean: the architecture owns the routing, so no single model is load-bearing.

3. Session, state, and memory ¶

The agent needs three kinds of state, and collapsing them causes most memory confusion. Conversation state: this session’s transcript, which must survive disconnects and channel hops—the same borrower on a call, then SMS. Entity state: the order, the borrower, the deal—read live from your system of record, never cached in the model’s head. Long-term memory: what previous sessions taught, retrieved per turn.

The first two are yours to build on ordinary infrastructure. The third is a product category: Letta, Zep, Mem0. Salient markets “borrower-level memory”—prior calls, promises, disputes informing every interaction. That’s this component, wearing a compliance department.

4. The knowledge pipeline ¶

RAG is three components wearing one acronym, and they run on different clocks.

Offline, throughput-bound: parsing—Unstructured, LlamaParse, Reducto—turning PDFs, tables, and scans into structure; then embedding—Voyage, Cohere—into an index: Pinecone, Qdrant, Weaviate, Chroma, pgvector, Turbopuffer. Online, latency-bound: hybrid retrieval plus a reranker, inside about 100ms because it sits on the hot path.

The offline half is where verticals live or die. F2’s entire wedge is parsing messy data rooms at financial-grade fidelity—the fanciest retrieval cannot fix a bad parse. One taxonomy correction worth internalizing: Exa is not a vector database. It, along with Tavily, is neural search over the web—which makes it a tool the agent calls, and it belongs two sections down.

5. The orchestrator and the bounded loop ¶

The center of the diagram, and the part demos most misrepresent. Production vertical agents are not open-ended ReAct loops pointed at a pile of tools. The outer structure is a deterministic workflow state machine—compiled from flows that domain experts draw in builders like Giga’s Agent Canvas or Salient’s Agent Studio. The LLM gets bounded autonomy inside each node: reason, act, observe, against a whitelisted tool set, under a step cap, with a defined surrender path. Autonomy is a per-node budget, not a global property. This single design decision is why these systems pass enterprise procurement while impressive demos don’t.

The buyable layer: LangGraph, CrewAI, and the labs’ own agent SDKs for the loop; Temporal underneath for durable execution, so a crash mid-loop resumes instead of restarting the phone call.

6. Tools, MCP, and the policy gate ¶

The defining trust decision of the whole pattern: the model never calls anything. It emits a proposal—tool name, arguments. A declarative policy gate evaluates the proposal against your rules and approves, denies, or escalates. A separate executor with scoped credentials runs what survives.

MCP is becoming the wiring standard for the tools themselves. Composio and Arcade sell integration catalogs; Exa, Tavily, and Browserbase are tools in their own right—web search and browser automation. Guardrails vendors—Guardrails AI, NeMo Guardrails, Lakera—cover input/output screening and injection defense. But the policy engine itself (OPA/Cedar-shaped) stays in-house at every serious company, because it encodes the rules that are the domain: refund limits, FDCPA disclosure requirements, IAM scopes. The prompt can say “don’t refund over $50”; the gate is what makes it true.

7. Sandboxed execution ¶

When actions are code, blast radius becomes a component. E2B, Modal, and Daytona sell disposable isolation—Firecracker microVMs in E2B’s case, gVisor’s user-space kernel in Modal’s—where an agent can execute, test, and fail without consequence. Kanu’s variant inverts the idea: the sandbox is your actual AWS account, with isolation enforced by tag-scoped IAM and a battery of security checks rather than a separate VM. Either way the principle holds—the agent acts inside a boundary that was drawn before it started acting.

The underestimated flavor is the browser. Most verticals sit on systems of record with no API—insurer portals, government sites, legacy EHRs—so the tool call degrades to an agent driving a web UI, and that live session needs the same containment plus session persistence, auth handling, and a human-takeover path. Browserbase (already on the tools shelf), Steel, and Hyperbrowser sell the managed browser; Browser Use, Stagehand, and Skyvern sell the agent that drives it.

And this component is one primitive serving three demand curves, which is why the category is bigger than “run the agent’s Python”:

Inference-time execution is the visible curve. The steeper one is training: RL on agentic tasks needs thousands of parallel, resettable environments per experiment, because each trajectory is an agent trying whatever maximizes reward—side effects included—and every rollout needs a clean world to wreck. Snapshot-and-fork is the primitive vendors compete on because RL wants to branch state cheaply, and an environments market is forming on top of it (Prime Intellect’s Environments Hub, Mechanize). The third curve belongs two sections down: the eval harness replays traces and runs simulations inside the same isolation.

One macro note. Sovereign AI doesn’t add executions so much as it moves them—data-residency rules and national-AI programs push the execution layer inside jurisdictional or corporate boundaries, which favors isolation you can deploy (BYOC, on-prem, or the open primitives underneath: Firecracker, gVisor, Kata) over pure SaaS. Kanu’s inversion above is the same force at company scale. Expect every sandbox vendor to grow a deployable story, or cede that demand to the open source they’re built on.

8. Verification and human handoff ¶

Guardrails constrain actions before they happen; verification validates outputs after, pre-commit. Schema checks, business invariants (the refund must not exceed the order total), grounding checks (every policy claim traces to a retrieved source), and increasingly a judge model scoring against a rubric. This stays mostly in-house for the same reason the policy gate does: the checks are the domain expertise.

When verification fails—or stakes exceed the autonomy budget—the handoff is engineered, not apologized for: a queue plus a context packet (transcript, retrieved policy, attempted and blocked actions) into the desk your humans already use (Zendesk, Genesys); HumanLayer sells the approval-flow primitive for agents. These are partial-autonomy systems by construction. The escalation rate is simultaneously the pricing model, the safety valve, and—because the human’s resolution comes back as a labeled example—the training-data faucet.

9. Observability, evals, and the learning loop ¶

Every model call, retrieval, proposal, gate decision, and execution appends to one trace stream with four consumers: debugging, audit, evals, training data. One artifact, four jobs—which is why the eval/observability shelf is the most crowded corner of the agent-infrastructure map. Tracing and evals: Langfuse, Arize, Braintrust, AgentOps, Weights & Biases Weave. Voice simulation and QA—synthetic callers hammering your agent before real ones do: Hamming, Coval, Cekura (formerly Vocera).

The loop’s output lands in three destinations with three latencies:

Facts go in context. Rules go in config—versioned and rolled back like deployments, which is what makes an audit answerable. Reflexes go in weights: fine-tunes of the edge models—the intent classifier, the reranker, the judge—not the frontier model in the middle. OpenPipe, Fireworks, and Predibase sell that capability. The sequencing is invariant: tracing, then evals, then weights. A fine-tune without regression evals is a bet you can’t measure.

The five patterns, composed ¶

If you know the five classic agentic design patterns—reflection, tool use, ReAct, planning, multi-agent—they’re all in the opening diagram. Not as a menu; as a fixed composition. Planning is the orchestrator’s state machine, with most of the plan pre-compiled by domain experts as flows. ReAct is the bounded loop inside each node. Tool use is the proposal–gate–executor triple. Reflection runs at three timescales: in-loop critique, pre-commit verification, and the eval pipeline reflecting over the whole deployment. Multi-agent is the org chart—router, workers, judge—differing in model size as much as in role. The textbook teaches them as five choices. Production ships them as one machine.

The box you rebuild is the business ¶

Every company in this space—swap in Decagon, Sierra, Parloa, Harvey, pick your vertical—buys most of the catalog above and rebuilds exactly one box: the one the catalog can’t satisfy. For Giga, no rented pipeline stayed sub-second once retrieval, policy, and tool calls sat inside the turn. For F2, no generic parser could read a data room. For Salient, no off-the-shelf guardrail could encode FDCPA. For Kanu, no hosted sandbox could safely ship changes to a live cloud account. The exception proves the rule: Chatbase, at the self-serve end of the market, rebuilds none of it—pure assembly of the catalog, which caps differentiation exactly as the thesis predicts: when every box is bought, the only levers left are distribution and price. The rebuilt box is the wedge; the trace stream compounds it into the moat, because the terminal asset of the learning loop is a model and a policy set that only your traffic could have produced.

The pattern is generic. The box you rebuild is the business.

Summary ¶

Vendor names will drift—this corner of the industry reshuffles quarterly. The boxes won’t. Learn the machine, and the landscape becomes legible: every funding announcement is somebody selling one box harder, and every vertical AI success story is somebody who rebuilt the right one.