Applied GenAI Curriculum for AI PMs  ·  Core Sequence

Unit 10 — The Production Architecture, Assembled

The five-step build-up of a real GenAI system — start with one model call, then add context, guardrails, a router/gateway, caches, and agent patterns, one problem at a time.

Tags
productionarchitecture
Layer
CORE — read in sequence
Objective
Read any architecture diagram an engineer shows you — know what each box does and what question to ask about it.
Connects
The convergence point of Units 04, 05, and 08. Monitoring and the feedback loop extend it in Unit 21.

Why this unit is the convergence point

Everything so far — workflows vs. agents (04), the four patterns (05), prompting (08) — becomes concrete boxes on a diagram here. The key insight is how the architecture is built: not designed all at once, but grown one component at a time, each added to solve a specific problem. Learn the progression and you can read any real system's diagram, name every box, and ask the one question that matters about each. Start from the simplest possible thing:

User query Model API Response

No context, no guardrails, no optimization. Every step below is a response to a problem this bare version has. (Follow the order that fits your app — it's a common progression, not a law.)

The five steps

1

Enhance context

Give the model the information it needs per query — via retrieval (text, image, tabular / RAG) and tools that gather info through APIs (web search, weather, internal data). This is feature engineering for foundation models, and it's the single biggest lever on output quality.

Ask: Where does the model's knowledge come from for a given query — and do providers' upload/tool limits constrain us?

2

Add guardrails

Protect the system and users wherever there's risk, on both sides:

Input guardrails — stop private data leaking to external APIs (detect & mask PII, e.g. [PHONE NUMBER], then unmask on the way back) and block malicious prompts (Unit 22). Output guardrails — catch failures (malformed JSON, hallucination, toxicity, brand risk, private-data leaks) and set a policy: retry (works because output is probabilistic; parallel retries keep latency down), or fall back to a human (on anger, keywords, or too many turns).

Ask: What failure modes do we catch, and what's the policy for each? And watch the false-refusal rate — a system too secure blocks legitimate requests. Note the reliability-vs-latency trade: guardrails add latency and don't play well with streaming.

3

Add a model router and gateway

Router — instead of one model for everything, an intent classifier sends each query to the right destination: specialized models (better), cheap models for simple queries (cheaper), stock replies for out-of-scope asks (no API call wasted), or a clarifying question for ambiguous ones. Routers must be small, fast, and cheap. Gateway — a unified, secure interface to all your models: one place to change when an API changes, plus centralized access control, cost limits, logging, and fallback when an API fails or rate-limits.

Ask: How are queries routed, and does the gateway give us one control point for keys, cost, and failover? (Common flow: route → retrieve → generate → score.)

4

Add caches

Cut latency and cost by reusing past work. Exact caching — reuse only on identical requests (great for multi-step or slow actions like retrieval/SQL); needs an eviction policy (LRU/LFU/FIFO) and a rule for what not to cache (user-specific or time-sensitive queries). Semantic caching — reuse for similar queries via embeddings + a similarity threshold; higher hit rate but fragile (depends on good embeddings, vector search, threshold tuning) and can return wrong answers.

Ask: What's cached, and how do we prevent a cache leak? (A personalized answer wrongly cached as generic can serve one user's private data to another.)

5

Add agent patterns

Move beyond a straight-line flow to loops, parallelism, and branching (Unit 04). The system can feed a response back in to retrieve more and try again, and — most consequentially — take write actions (send an email, place an order, move money) that change the real world. Write actions make a system far more capable and far more dangerous; grant them with the utmost care.

Ask: Does this system have write actions — and what's the blast radius if one fires wrongly? Every loop and write is a new failure mode to debug.

Query Guardrails Router / Gateway Context Model Cache Guardrails Response ↺ agent loop: response can feed back in · model can take write actions
The PM's read on any architecture diagram You don't need to design this — you need to interrogate it. For each box: context → where does knowledge come from? guardrails → what's caught and what's the false-refusal cost? router/gateway → how are queries routed and controlled? cache → what's reused and could it leak? agent loop/writes → what can it change in the world? Complexity solves more tasks but multiplies failure modes — every added box is a new place to debug.

The through-line: complexity is earned, not assumed

Notice the discipline baked into the progression — it's the same message as Unit 04. Each component exists because the previous architecture had a concrete problem. A team that starts at step 5 (agents, write actions) without earning steps 1–4 has built something powerful, unobservable, and hard to trust. The right question at the start of any build is not "how sophisticated can we make this?" but "what's the simplest architecture that meets the bar, and what problem forces the next box?"

What "good" looks like after this unit

You can now:

Two pieces are deliberately held for Unit 21: monitoring/observability (which should be designed in, not bolted on) and the user-feedback loop that turns production traffic into improvement.