Applied GenAI Curriculum for AI PMs · Core Sequence

Unit 05 — The Four-Way Decision: Prompting vs. RAG vs. Finetuning vs. Agents

Just enough of each pattern to tell them apart — and combined with Unit 04, a working decision tree for "why this approach and not the alternatives?"

Why this unit pairs with Unit 04

Unit 04 split the world into workflows vs. agents by control flow. This unit cuts a different way — by how you give the model what it's missing. Together they form the decision tree you'll apply to almost every AI feature. The goal here isn't to implement any of these four; it's to differentiate them well enough that when an engineer proposes one, you can ask the single question that exposes whether it's the right call.

The four adaptation techniques at a glance

All four are ways to close the gap between a general model and your specific task. They differ in what they change and how much they cost.

Cheapest

Prompting

Adapt via instructions and context — no weight changes. Where you always start.

Add data

RAG

Retrieve relevant external info per query and feed it in. Gives the model facts it lacks.

Change weights

Finetuning

Further-train the model on your data. Changes the model's behavior and form.

Most complex

Agents

Let the model plan and use tools in a loop to accomplish multi-step tasks.

1. RAG — giving the model facts it doesn't have

Retrieval-augmented generation enhances a model's output by retrieving relevant information from an external memory source — an internal database, a user's past chats, or the internet — and feeding it into the prompt. The shape is two components:

Retriever	Pulls the information most relevant to the query from external memory. Its two jobs: indexing (processing data so it's fast to fetch later) and querying (fetching what's relevant now). The whole system's quality rides on the retriever.
Generator	The model that writes the response using the retrieved information.

The mental model: RAG constructs context specific to each query instead of stuffing the same context into every call — the foundation-model equivalent of feature engineering. Because it supplies relevant facts, it produces more detailed answers and reduces hallucinations. It also naturally handles per-user data (include a user's data only in that user's queries).

"Won't long context kill RAG?" A question you'll hear. The answer is no, for two reasons worth knowing: (1) data always outgrows any context window — we generate far more than we delete; (2) a model given long context doesn't necessarily use it well — the longer the context, the more likely it fixes on the wrong part, and every extra token adds cost and latency. RAG feeds only the most relevant slice, which can both cut tokens and improve performance. Rule of thumb: if your entire knowledge base is small enough to fit comfortably in the prompt, you may not need RAG at all.

2. Agents — plan, act, observe, repeat

An agent is anything that can perceive its environment and act upon it — defined by its environment and its set of actions. The actions are extended by the tools it can call. On this definition, familiar systems are already agents: a chatbot that can search the web and run code is one; a RAG system is one, with its retrievers and SQL executors as tools. The model is the brain: it processes the task and environmental feedback, plans a sequence of actions, and judges when the task is done — looping reason → act → observe until complete.

Why agents need stronger (and pricier) models — the compounding-error math Agents take many steps, and accuracy multiplies. At 95% accuracy per step: 10 steps → ~60% overall; 100 steps → ~0.6%. Add higher stakes (tools let an agent do real damage), and you see why agents demand more capable models, sandboxing, and guardrails. This is the quantitative backbone of Unit 04's "default to a workflow."

3. Finetuning — changing the model's behavior

Finetuning continues training the model on your data, updating its weights. It's the heaviest lever, and the discipline around it is mostly about restraint.

Reasons to finetune	Reasons not to
Improve quality on a specific task the base model wasn't trained well for (e.g. an uncommon SQL dialect). Reliably produce a structured format. Mitigate bias with curated data. A small finetuned model can beat a much larger general one on its task, and be cheaper/faster to serve.	High up-front and ongoing cost: data, ML talent, serving, and a monitoring/update policy. It can degrade performance on other tasks. Base models improve so fast they may outrun your finetuned model. Most "prompting doesn't work" cases are really unsystematic prompting.

The order that matters Start with prompting, systematically — clear instructions, representative examples, defined metrics. Only when thorough prompting proves inadequate should you reach for RAG or finetuning. Beware the reflex that "general models can't do domain-specific tasks, so we must finetune" — as base models improve, they increasingly beat specialized ones. Prompting experiments also build the eval pipeline and data guidelines that finetuning will later depend on, so the effort is never wasted.

The heuristic that decides RAG vs. finetuning

Once prompting is maxed out, the choice between RAG and finetuning comes down to what kind of failure you're seeing:

Information-based failure → RAG

"RAG is for facts."

Outputs are factually wrong or outdated — the model lacks the info (private data) or has stale info. Give it the sources. For up-to-date questions, RAG reliably beats finetuning.

Behavior-based failure → Finetuning

"Finetuning is for form."

Outputs are correct but wrong in kind — irrelevant to the task, or in the wrong format (won't compile, wrong structure). Finetune on well-formed examples to fix the behavior.

These aren't mutually exclusive — real systems often combine prompting, RAG, and finetuning. But the facts-vs-form split is the fastest way to sort which lever a problem calls for.

The decision tree (Units 04 + 05 combined)

Walk top to bottom; stop at the first "yes." Simpler is better.

1. Can a single well-crafted prompt (with examples) do it?

Yes → Prompting. Don't add complexity you can't justify.

2. Does it fail because it lacks facts (private / current info)?

Yes → RAG. Information-based failure.

3. Does it fail on behavior/format that prompting + examples can't fix?

Yes → Finetuning — but only after systematic prompting. Behavior-based failure.

4. Does the task need multi-step planning and tool use you can't predefine?

Yes → Agent (clear Unit 04's five-point checklist first). No / steps predictable → a workflow.

The one incisive question When a team proposes an approach, ask what it's for: "Is our problem missing facts or wrong behavior?" (RAG vs. finetuning) and "Are the steps predictable?" (workflow vs. agent). Most over-engineering dies on those two questions.

What "good" looks like after this unit

You can now:

Describe RAG as retriever + generator, and explain why it reduces hallucinations and survives long context.
Define an agent as perceive + act via tools, and cite the compounding-error math for why agents need stronger models.
Apply "RAG is for facts, finetuning is for form" to route a failure to the right lever.
Walk the combined decision tree and ask the two questions (facts-vs-form, predictable-vs-not) that justify a pattern.

Each pattern's real depth — retrieval quality, agent failure-mode budgeting, finetuning mechanics — waits in Units 17, 18, and 19, to be pulled in when a live decision demands it.