Applied GenAI Curriculum for AI PMs · Core Sequence
Sampling and the probabilistic nature of AI — why identical inputs produce different outputs, and why this one fact reshapes testing, QA, user expectations, and sprint planning.
Unit 01 established that a foundation model is a completion machine making predictions based on probabilities. This unit is where that sentence grows teeth. Ask an AI the same question twice and the answer can change — not because of a bug, but because of how the model chooses each word. That single property, non-determinism, is the hidden reason behind nearly every "why is AI different to ship?" question a PM faces. Almost everything downstream in this curriculum — evals, staged rollouts, feedback loops, guardrails — is an answer to a problem introduced here.
To generate a token, the model doesn't "know" the answer — it computes a probability distribution over every token in its vocabulary, then picks one. Given "My favorite color is …", it might assign "green" 50%, "red" 30%, and a long tail to thousands of other words. The mechanics, in one line:
The network emits a raw score for each token in the vocabulary.
Converts those scores into probabilities that sum to 1.
Picks one token according to those probabilities.
The crucial word is sampling. Always taking the single highest-probability token is greedy sampling — fine for a spam classifier, but it makes a language model boring, forever answering in the most common words. So instead the model samples from the distribution: if "red" has a 30% chance, it gets picked 30% of the time. That deliberate roll of the dice is the source of everything in this unit. You don't need the softmax math; you need the consequence: the model is designed to not always say the same thing.
Sampling isn't uncontrolled — there are a handful of sampling variables that shape how adventurous the output is. You won't set these, but you should know what they trade off, because "make it more consistent" and "make it more creative" are requests to move these dials in opposite directions.
| Variable | What it does |
|---|---|
| Temperature | The master creativity dial. Higher (~0.7+) flattens the odds so rarer, more interesting tokens surface — creative but less coherent. Lower makes the model favor the obvious token — consistent but duller. 0 in practice means "always take the top token" (as consistent as it gets). |
| Top-k | Only consider the k most likely tokens (e.g. 50–500), then sample among those. Smaller k = more predictable, less interesting. |
| Top-p (nucleus) | Consider the smallest set of tokens whose probabilities add up to p (e.g. 0.9). Adapts the pool to context — narrow for "yes/no", wide for "meaning of life". |
| Stopping condition | Stop tokens / max length that end generation. Keeps latency and cost down — but stopping too early can cut off a sentence or break required formats like JSON. |
Consistent, predictable, "boring." Good for extraction, classification, structured tasks where you want the same answer every time.
Creative, varied, occasionally incoherent. Good for brainstorming, drafts, ideation — anywhere variety is the point.
Because the model samples from "a world of possibilities," anything with non-zero probability — however wrong — can come out. This shows up as two distinct problems. Keeping them separate matters, because they have different fixes.
Inconsistency is the model giving very different responses to the same or barely-different prompts. A documented example: asking ChatGPT to score an essay gave 3/5 one run and 5/5 the next, same prompt. It comes in two flavors:
| Same input → different output | Identical prompt, two different answers. The more tractable case. |
| Slightly different input → drastically different output | An accidental capital letter flips the response. Harder to tame. |
For the first case there are real mitigations: cache the answer so a repeated question returns the stored response; fix the sampling variables (temperature, top-p, top-k); and fix the seed (the starting point for the random-number generator). But note the honest caveat — even with all of that pinned, consistency isn't guaranteed 100% of the time, because the hardware executing the model can nudge outputs, and if you're on a provider API like OpenAI or Google, you don't control that hardware. The second case (tiny input change, huge output change) can't be fixed by these settings at all — it's addressed later with careful prompting (Unit 08) and memory systems.
Hallucination is a response not grounded in facts — the model stating something untrue as if it were real. It's fatal wherever factuality matters: in June 2023 a law firm was fined for submitting fictitious, ChatGPT-generated legal research to a court. Crucially, hallucination is not the same root cause as inconsistency. Inconsistency comes from sampling randomness; hallucination is more nuanced — the model can produce something seemingly never in its training data. Two leading hypotheses:
| Self-delusion | The model can't distinguish the text you gave it from text it generated. Once it says "this person is an architect," it treats that as established fact and builds on it — "snowballing" into confidently wrong output. |
| Knowledge mismatch | During training the model was taught to imitate human labelers who used knowledge the model itself lacked — effectively training it to make things up to fill the gap. |
The practical upshot for you: hallucination can be reduced but not switched off. Partial levers include prompting the model to say "I don't know" when unsure, asking for concise answers (fewer tokens = fewer chances to invent), and — most importantly downstream — grounding it in real data via RAG (Units 05, 17) and catching failures via evals (Unit 06+). Even detecting a hallucination is hard, much like detecting when a person is lying.
Hold this unit next to normal software, where the same input reliably yields the same output. Non-determinism breaks that assumption, and the fallout runs through the rest of the curriculum:
| PM activity | What non-determinism changes |
|---|---|
| Testing / QA | You can't assert output == expected. "Passed once" doesn't mean "passes." This is the reason evals (Unit 06) exist as a discipline. |
| User expectations | Users assume consistency; the system won't always deliver it. Expectation-setting (Unit 03) is hard because of this. |
| Acceptance criteria | "Correct answer" becomes "acceptable distribution of answers" — you spec ranges and failure rates, not exact strings. |
| Rollout & monitoring | A demo that worked can fail in production on the same input. Staged rollouts and live feedback loops (Units 10, 21) are the response. |