Applied GenAI Curriculum for AI PMs · Series Close
Every source behind the 24 units, collected here textbook-style — the three works the series draws on, and a unit-by-unit map to the exact chapters, sections, and pages for anyone who wants to read deeper.
Chip Huyen. O'Reilly Media, 2025. ISBN 978-1-098-16629-8.
The backbone of the series — the source for all CORE units except 04 and 06, and most DEPTH units. Official table of contents (used to verify the page ranges below): github.com/chiphuyen/aie-book · O'Reilly: oreilly.com
The applied-evaluation spine of the series — the source for the eval thread's practitioner units (06, 14, 15, 16).
Posts cited: Your AI Product Needs Evals · A Field Guide to Rapidly Improving AI Products · Creating a LLM-as-a-Judge That Drives Business Results · LLM Evals FAQ · "It's Hard to Eval" Is a Product Smell
The source for the foundational architecture distinction in Unit 04.
Article cited: Building Effective Agents (free, ~25 min)
Citation format for the book: [AIE] Ch.N → "Section" (pp. X–Y). CORE units are in accent; DEPTH units in brown.
| CORE — the operating model (Units 01–13) | ||
| 01 | What AI Engineering Is (and Isn't) | [AIE] Ch.1 → "The Rise of AI Engineering" (pp. 2–15) + "The AI Engineering Stack" (pp. 35–47) |
| 02 | Why the Output Is Never the Same Twice | [AIE] Ch.2 → "Sampling Fundamentals" + "Sampling Strategies" (pp. 88–96) and "The Probabilistic Nature of AI" (pp. 105–111) |
| 03 | Should This Even Be an AI Feature? | [AIE] Ch.1 → "Planning AI Applications" (pp. 28–35) |
| 04 | Workflows vs. Agents | [Anthropic] "Building Effective Agents" |
| 05 | The Four-Way Decision | [AIE] Ch.6 → "RAG" intro + "RAG Architecture" (pp. 253–256) and "Agent Overview" (pp. 276–278); Ch.7 → "When to Finetune" incl. "Finetuning and RAG" (pp. 311–318) |
| 06 | Why Evals Are the Whole Game | [Hamel] "Your AI Product Needs Evals" |
| 07 | Defining "Good": Criteria & Model Selection | [AIE] Ch.4 → "Evaluation Criteria" (pp. 160–179) + "Model Selection" (pp. 179–199) |
| 08 | Prompting as an Engineering Discipline | [AIE] Ch.5 → "Introduction to Prompting" (pp. 212–218) + "Prompt Engineering Best Practices" (pp. 220–233) |
| 09 | Designing the Eval Pipeline | [AIE] Ch.4 → "Design Your Evaluation Pipeline" (pp. 200–208) |
| 10 | The Production Architecture, Assembled | [AIE] Ch.10 → "AI Engineering Architecture," Steps 1–5 (pp. 449–465) |
| 11 | Data Curation: The Unsexy Blocker | [AIE] Ch.8 → "Data Curation" (pp. 365–380) |
| 12 | Cost & Latency | [AIE] Ch.9 → "Inference Overview" + "Inference Performance Metrics" (pp. 406–419) |
| 13 | Capstone (CORE checkpoint) | None — produced, not read. |
| DEPTH — pulled in on demand (Units 14–24) | ||
| 14 | Error Analysis & the Improvement Flywheel | [Hamel] "A Field Guide to Rapidly Improving AI Products" |
| 15 | LLM-as-Judge, Done Properly | [Hamel] "Creating a LLM-as-a-Judge That Drives Business Results" · then [AIE] Ch.3 → "AI as a Judge" (pp. 136–148) |
| 16 | Evals Reference Shelf | [Hamel] "LLM Evals FAQ" and "'It's Hard to Eval' Is a Product Smell" · optional paid course (above) |
| 17 | RAG Internals: Retrieval Quality | [AIE] Ch.6 → "Retrieval Algorithms" (pp. 257–268) + "Retrieval Optimization" (pp. 268–273) + "RAG Beyond Texts" (pp. 273–275) |
| 18 | Agent Internals | [AIE] Ch.6 → "Tools" (pp. 278–281) + "Planning" (pp. 281–298) + "Agent Failure Modes and Evaluation" (pp. 298–300) + "Memory" (pp. 300–305) |
| 19 | Finetuning: What It Actually Involves | [AIE] Ch.7 → "Finetuning Overview" (pp. 308–311) + "Finetuning Tactics" (pp. 357–360) |
| 20 | Synthetic Data & Distillation | [AIE] Ch.8 → "Data Augmentation and Synthesis" (pp. 380–396) |
| 21 | Monitoring, Observability & Feedback | [AIE] Ch.10 → "Monitoring and Observability" + "AI Pipeline Orchestration" (pp. 465–474) + "User Feedback" (pp. 474–492) |
| 22 | Defensive Prompting | [AIE] Ch.5 → "Defensive Prompt Engineering" (pp. 235–251) |
| 23 | Where Foundation Models Come From | [AIE] Ch.2 → "Training Data" (pp. 50–58) + "Post-Training" (pp. 78–88) |
| 24 | Foundation-Model Evaluation Concepts | [AIE] Ch.3 → "Challenges of Evaluating Foundation Models" (pp. 114–118) + "Exact Evaluation" (pp. 125–136) + "Ranking Models with Comparative Evaluation" (pp. 148–156) |
The units can be read in several deliberate orders beyond the default sequence:
Each unit distilled and reframed its source(s) for a product-management audience — translating engineering depth into the decisions and questions a PM actually owns, rather than reproducing the material. The units themselves carry no inline citations by design; this page is the single, textbook-style home for all attribution and further reading. Page ranges follow the book's official table of contents and are inclusive of the named section, stopping where the next section begins.
Deliberate exclusions across the series — model internals and information-theory math ([AIE] pp. 58–77, 96–105, 118–124), finetuning memory/quantization math (pp. 319–356), and accelerator/kernel optimization (pp. 419–447) — are the engineers' pages, left out by design to keep the curriculum on the role boundary a PM operates within.