Applied GenAI Curriculum for AI PMs · Core Sequence · Eval Thread (supporting)
Define domain-specific criteria (with cost and latency as first-class), run a defensible model-selection workflow, decide build-vs-buy, and understand why public benchmarks mislead.
Unit 06 argued that evals are the whole game. This unit answers the prerequisite question: good at what, and measured how? It sits deliberately before prompting — you should know the target you're optimizing toward before you learn the levers to hit it. The payoff is being able to answer, with a straight face, "which model and how do we know it's good enough?" — not by pointing at a leaderboard, but with a process you can defend.
The worst outcome isn't an app that never ships — it's one that's deployed but nobody knows if it works. It costs to run, and might cost even more to take down. Apps with murky ROI are common precisely because teams didn't define how they'd be evaluated up front.
Every AI application should start with a list of criteria specific to it. They fall into four buckets. Picture summarizing a legal contract: each bucket answers a different "is it good?" question.
Can the model do the task at all? Bounded by its training data and size — a model that never saw Latin can't translate it.
Does it understand legal contracts?Is the output well-formed and trustworthy? Includes factual consistency and safety (the metrics that matter most now).
Is the summary coherent and faithful?Does it do what you asked, in the form you asked? A model can understand the task yet ignore your format.
Did it honor the length limit?Can you afford it, and is it fast enough? A first-class criterion, not an afterthought.
How much, and how long to wait?Evaluated with domain benchmarks and exact evaluation where possible. Coding uses functional correctness (does the code run and pass tests) — though you may also care about efficiency, cost, and readability. Non-coding capabilities are often tested with multiple-choice questions, because close-ended outputs are easy to verify and reproduce (the reason most public benchmarks lean on them).
Old NLG metrics like fluency and coherence matter less now that models write near-human text (still useful for weak models or creative/low-resource work). The pressing modern metrics:
| Factual consistency | Checked two ways: local (does the output match a provided context — critical for summarization, support bots, analysis) and global (does it match open-world knowledge — for general chat, fact-checking). It's far easier to verify against explicit context than open knowledge; the hardest part is often deciding what the facts even are. |
| Safety | Umbrella for toxicity and bias — inappropriate content, harmful instructions, hate speech, violence, stereotypes, ideological slant. Detectable via general AI judges or smaller specialized classifiers. |
These two are examples — you'll define your own (conciseness, friendliness, controversiality, whatever your product needs) using the same techniques.
If the model is bad at following instructions, it won't matter how good your instructions are. A model can have the domain capability (correctly reads tweet sentiment) yet fail instruction-following (outputs HAPPY/ANGRY when you asked for POSITIVE/NEGATIVE). More powerful models are generally better at this.
A model that's excellent but too slow or expensive is useless; many teams knowingly choose lower quality for better cost and latency. This is Pareto optimization — multiple objectives at once. The discipline: be explicit about what you can't compromise on. If latency is non-negotiable, filter out every model that misses your latency bar first, then pick the best of what remains.
The output of this whole section is a concrete criteria table — each criterion with a metric, a benchmark or dataset, a hard requirement, and an ideal. That table is the target the rest of your work optimizes toward.
You don't care which model is "best" — only which is best for your application. Selection recurs constantly (each adaptation technique triggers it again). First, separate two kinds of attributes:
| Hard attributes | What you can't or won't change — license, training data, model size (provider decisions) or privacy/control (your policies). These shrink the candidate pool. |
| Soft attributes | What you can improve — accuracy, toxicity, factual consistency. Estimating how much you can improve them is judgment (one task jumped 20% → 70% just by splitting it in two; another stayed unusable after weeks). |
Note the same attribute can be hard or soft depending on context: latency is soft if you host the model and can optimize it, hard if someone else hosts it. Then run the four-step funnel:
Drop models whose non-negotiables don't fit (license, privacy, hosting).
Use benchmarks and leaderboards to shortlist promising candidates, balancing quality/latency/cost.
Run your evals on the shortlist to find the real winner — the step that actually decides.
Detect failures and collect feedback continuously (Unit 21).
The steps are iterative — a later step can overturn an earlier decision (you planned to self-host, but private evals show open models fall short, so you switch to an API).
Since you won't train a foundation model from scratch, "build vs. buy" means commercial API vs. self-hosting an open model — and the answer prunes your candidate pool hard. It helps to know the "open" gradations:
| Open weight | Weights are downloadable, but training data is not released. |
| Open model | Weights and training data are public — enables retraining, auditing, and deeper understanding. |
The trade-offs run along control, privacy, cost-at-scale, and the engineering burden of serving — which is why this is a question teams revisit more than once as they grow.
There are thousands of benchmarks, and leaderboards aggregate a handful into a ranking. Useful for a first-pass shortlist — but not to be trusted as the answer, for concrete reasons:
| Thin coverage | Compute limits mean leaderboards include only a few benchmarks (some drop important-but-expensive ones). A handful can't represent a model's vast capabilities and failure modes. |
| Opaque selection | Different leaderboards pick different benchmarks with unclear rationale, so rankings don't compare cleanly. If they overlap only slightly, which do you believe? |
| Correlation | Two highly-correlated benchmarks add no information but can exaggerate biases in the aggregate score. |
| Saturation | As models improve, benchmarks saturate and get replaced — a moving target, and a sign yesterday's leaderboard win means little today. |
| Contamination | Models may have seen benchmark data in training, inflating scores in ways that won't hold on your real inputs. |