Applied GenAI Curriculum for AI PMs · Core Sequence
The five-step build-up of a real GenAI system — start with one model call, then add context, guardrails, a router/gateway, caches, and agent patterns, one problem at a time.
Everything so far — workflows vs. agents (04), the four patterns (05), prompting (08) — becomes concrete boxes on a diagram here. The key insight is how the architecture is built: not designed all at once, but grown one component at a time, each added to solve a specific problem. Learn the progression and you can read any real system's diagram, name every box, and ask the one question that matters about each. Start from the simplest possible thing:
No context, no guardrails, no optimization. Every step below is a response to a problem this bare version has. (Follow the order that fits your app — it's a common progression, not a law.)
Give the model the information it needs per query — via retrieval (text, image, tabular / RAG) and tools that gather info through APIs (web search, weather, internal data). This is feature engineering for foundation models, and it's the single biggest lever on output quality.
Ask: Where does the model's knowledge come from for a given query — and do providers' upload/tool limits constrain us?
Protect the system and users wherever there's risk, on both sides:
Input guardrails — stop private data leaking to external APIs (detect & mask PII, e.g. [PHONE NUMBER], then unmask on the way back) and block malicious prompts (Unit 22). Output guardrails — catch failures (malformed JSON, hallucination, toxicity, brand risk, private-data leaks) and set a policy: retry (works because output is probabilistic; parallel retries keep latency down), or fall back to a human (on anger, keywords, or too many turns).
Ask: What failure modes do we catch, and what's the policy for each? And watch the false-refusal rate — a system too secure blocks legitimate requests. Note the reliability-vs-latency trade: guardrails add latency and don't play well with streaming.
Router — instead of one model for everything, an intent classifier sends each query to the right destination: specialized models (better), cheap models for simple queries (cheaper), stock replies for out-of-scope asks (no API call wasted), or a clarifying question for ambiguous ones. Routers must be small, fast, and cheap. Gateway — a unified, secure interface to all your models: one place to change when an API changes, plus centralized access control, cost limits, logging, and fallback when an API fails or rate-limits.
Ask: How are queries routed, and does the gateway give us one control point for keys, cost, and failover? (Common flow: route → retrieve → generate → score.)
Cut latency and cost by reusing past work. Exact caching — reuse only on identical requests (great for multi-step or slow actions like retrieval/SQL); needs an eviction policy (LRU/LFU/FIFO) and a rule for what not to cache (user-specific or time-sensitive queries). Semantic caching — reuse for similar queries via embeddings + a similarity threshold; higher hit rate but fragile (depends on good embeddings, vector search, threshold tuning) and can return wrong answers.
Ask: What's cached, and how do we prevent a cache leak? (A personalized answer wrongly cached as generic can serve one user's private data to another.)
Move beyond a straight-line flow to loops, parallelism, and branching (Unit 04). The system can feed a response back in to retrieve more and try again, and — most consequentially — take write actions (send an email, place an order, move money) that change the real world. Write actions make a system far more capable and far more dangerous; grant them with the utmost care.
Ask: Does this system have write actions — and what's the blast radius if one fires wrongly? Every loop and write is a new failure mode to debug.
Notice the discipline baked into the progression — it's the same message as Unit 04. Each component exists because the previous architecture had a concrete problem. A team that starts at step 5 (agents, write actions) without earning steps 1–4 has built something powerful, unobservable, and hard to trust. The right question at the start of any build is not "how sophisticated can we make this?" but "what's the simplest architecture that meets the bar, and what problem forces the next box?"
Two pieces are deliberately held for Unit 21: monitoring/observability (which should be designed in, not bolted on) and the user-feedback loop that turns production traffic into improvement.