Applied GenAI Curriculum for AI PMs · Core Sequence · Eval Thread 1/5
The empirical claim that failed AI products share one root cause — no evaluation system — plus the three-level eval structure and the iterate-fast flywheel.
Unit 02 showed that AI output is non-deterministic; Units 03–05 decided whether and how to build. Now the question that determines whether any of it works in production: how do you know it's good — and getting better? This is Unit 1 of a five-part thread deliberately woven through the rest of the curriculum, because evaluation isn't a phase you complete; it's the instrument you read at every phase. If you take one reflex from this unit, make it the question that opens every AI feature review: "What's our eval plan?"
Across many AI products, successful and failed, the failures share one common cause: the absence of a robust evaluation system. Teams pour effort into the model and prompts, ship a demo, and then plateau — unable to tell whether a change helped or hurt. The claim is strong and worth holding onto because it reframes your job: your leverage as a PM isn't in the model, it's in insisting the team can measure the model.
Success with AI, like software, hinges on how fast you can iterate. That requires three capabilities working together in a loop:
Measure quality — tests and metrics that say whether output is good.
Log and inspect data to find why something failed.
Adjust behavior — prompt, RAG, finetune, or code.
Most teams pour all their energy into ③ — endlessly tweaking prompts — which is exactly what keeps them stuck at demo quality. Without ① and ②, you can't tell whether a change improved anything; you're playing whack-a-mole in the dark. Streamline evaluation and the other two get easy. Evaluation is the bottleneck that gates everything else.
Lucy, the AI assistant inside the real-estate SaaS Rechat, is the canonical arc. Early on, prompt engineering drove rapid progress. Then, as Lucy's surface area grew, quality plateaued — with three telltale symptoms every PM should recognize:
| Whack-a-mole | Fixing one failure mode spawned new ones elsewhere. |
| Vibe-check visibility | No real insight into performance across tasks beyond gut feel. |
| Bloated prompts | Prompts ballooned into long, unwieldy forms trying to patch every edge case. |
The fix wasn't a better model or a cleverer prompt — it was building a systematic evaluation system at the center of the process. Eval and curation sits in the middle of everything, and it's where the team should spend most of its time.
Evaluation isn't one thing. It's three levels, ordered by cost — which dictates how often you run each. Get a good chunk of Level 1 working before leaning on Level 2.
Fast, cheap code assertions (like pytest) scoped by feature × scenario. For a "listing finder," scenarios might assert the query returns exactly one, many, or zero results; generic tests catch things like leaked internal IDs. Generate test inputs synthetically (an LLM can write them), and update them constantly as real failures surface.
Run on every code change. Note: you don't need a 100% pass rate — the target is a product decision about which failures you'll tolerate.
For what assertions can't catch. Prerequisite: log your traces (the full conversation/request flow). Then look at your data — remove all friction from viewing traces, and label examples simply (binary good/bad beats granular scores to start). You can also automate grading with an LLM judge, but you must validate it against human labels and keep checking agreement. Human review never fully goes away.
Run on a set cadence. This is where the highest-leverage learning happens — Unit 14 is built on it.
The real-world check that your product actually drives the user behavior and outcomes you want. Mechanically similar to A/B testing any product.
Run only after significant product changes, once you're confident it's ready for real users. Appropriate for more mature products.
Here's why this is a strategic investment, not overhead. The infrastructure you build for evaluation is the same infrastructure that powers everything else — so once it exists, capabilities come nearly for free:
| Fine-tuning data | Fine-tuning is 99% assembling high-quality data covering your product's surface — which a good eval system already generates and curates. Your evals become a data engine. |
| Debugging | A complaint or error is fast to diagnose when you already have searchable traces, assertions that flag bad behavior, and tools to test a fix. Eval infra is debug infra. |
| Faster iteration | Every improvement is measurable, so the evaluate → debug → change loop spins faster over time — the flywheel that separates great AI products from stuck ones. |
This is the reframe to carry into budget conversations: evals look like an operational cost, but they're the asset that compounds — turning each fix into reusable test cases, training data, and debugging power.
This unit made the case for evals. The rest of the thread makes them concrete: Unit 07 defines the criteria you're measuring against, Unit 09 designs the pipeline, Unit 14 runs error analysis on real traces, and Units 15–16 handle LLM judges and the reference details.