Applied GenAI Curriculum for AI PMs · Core Sequence
In-context learning, system vs. user prompts, context-length economics, and the best-practices set — plus how to tell a team doing disciplined prompt iteration from one running on vibes.
Unit 07 set the target — the criteria you're optimizing toward. This unit is the first and cheapest lever to hit it. The trap is treating prompting as casual wordsmithing. Done well, it's a disciplined loop: change one thing, measure against your evals, version it, keep or revert. Your job as a PM isn't to write the prompt — it's to recognize whether the team is iterating like engineers or guessing like it's a chat window.
A prompt is an instruction to a model to perform a task. It generally has up to three parts:
Prompting only works if the model can follow instructions in the first place (Unit 07's instruction-following criterion). How much prompt-fiddling you need depends on the model's robustness to perturbation — whether writing "5" instead of "five," adding a newline, or changing capitalization swings the output. Both instruction-following and robustness track overall model strength, which yields a practical shortcut: a stronger model often saves you days of prompt-tweaking.
The foundational idea: a model can learn a desired behavior from examples in the prompt alone, no weight updates — this is in-context learning. Each example is a "shot":
| Zero-shot | No examples — just the instruction. |
| Few-shot | Several examples in the prompt (five examples = 5-shot). Generally, more examples help — bounded by context length and cost. |
This is also a form of continual learning: to teach a model about a new library version, you don't retrain — you put the new info in its context, letting it answer beyond its training cutoff. One nuance worth carrying: as models get stronger, few-shot adds less over zero-shot for general tasks — but examples still matter a lot for domain-specific content the model rarely saw in training.
Most APIs split the prompt in two: the system prompt (think: the task description / role, set by you the developer) and the user prompt (the task, from the end user). A real-estate assistant might carry "You're an experienced real estate agent…" in the system prompt, and the uploaded disclosure + "How old is the roof?" in the user prompt.
How much you can put in a prompt is capped by the model's context length. Windows have exploded (from ~1K tokens to millions), but two economic facts matter more than the ceiling:
| Every token costs | APIs charge per input and output token, and longer outputs add latency (tokens are generated one at a time). "Be concise" is a cost-and-speed lever, not just style. |
| The middle gets lost | Models attend best to the beginning and end of a prompt, worst to the middle. The "needle in a haystack" test measures this by hiding a fact at various positions and checking recall. Put critical instructions at the edges. |
The takeaway: a bigger context window is not free capacity. If quality degrades as prompts grow, the fix is often to shorten, not stuff.
These techniques work across a wide range of models and outlast the throwaway tricks ("offer the model a $300 tip"). This is the checklist to hold a team's prompt work against.
Remove ambiguity: state the exact scoring scale, whether "I don't know" is allowed, integer vs. fractional. Assign a persona to set perspective ("act as a first-grade teacher"). Give examples to pin down the response you want. Specify the output format precisely (JSON keys, no preambles) and use end markers for structured outputs so the model doesn't just continue your input.
Like giving a student a reference text — supplying context improves answers and curbs hallucination (without it, the model falls back on unreliable internal knowledge). Either hand it the context or give it tools to fetch it (context construction — RAG, web search, Units 05/17). Restricting a model to only its context (for roleplay/grounding) is genuinely hard; prompting helps but doesn't guarantee it.
Decompose into chained prompts (e.g. classify intent → respond to that intent). Models handle simple instructions better, and decomposition adds real benefits: you can monitor and debug each step, parallelize independent ones, and write simpler prompts. Costs can even drop (smaller prompts, cheaper models for easy steps). The trade: more steps can raise perceived latency. One team's support prompt bloated past 1,500 tokens; decomposing it improved quality and cut cost.
Chain-of-thought (CoT) — "think step by step" or supplying the steps — nudges systematic reasoning and reliably improves results (and reduces hallucination). Self-critique asks the model to check its own output. Both cost latency, since the model works through intermediate steps before the user sees a token.
Prompting is back-and-forth; each model has quirks (numbers, roleplay, instruction placement). But iteration must be systematic: version your prompts, use an experiment-tracking tool, standardize eval metrics and data so you can compare, and evaluate each prompt against the whole system — a change can help a subtask yet hurt the end-to-end result.