Applied GenAI Curriculum for AI PMs · Depth Layer
Why and when to synthesize data, how AI does it, model distillation — and the quality risks that make "we'll just generate the training data" a claim to probe, not accept.
Unit 11 established that data is the usual blocker. The seductive escape hatch is "let's just generate it." Synthetic data is genuinely useful and increasingly common — but the phrase hides real risks that can quietly poison a model. This unit gives you both halves: when synthesis legitimately helps, and the four failure modes that should make you probe any "we'll synthesize it" proposal before it's funded.
Synthesis targets the same "golden trio" from Unit 11 — quantity, coverage, quality — plus two more:
Produce data at scale where real data is scarce or dangerous to collect (rare weather, deep sea, car-accident scenarios).
Generate targeted cases: very long/short texts, adversarial examples, rare classes, toxic examples to train detectors.
Sometimes better than human — AI-generated preference labels are more consistent than humans whose ratings vary by mood.
Often the only option where real data is legally off-limits — synthetic patient records or insurance claims with no real PII.
Generate data from a big model to train a cheaper, faster small one (its own section below).
You don't need the recipes, just the vocabulary so you know what a team means. Two traditional families predate AI, and one is new:
| Rule-based / templates | Fill a template with a random generator (fake transactions, invoices, resumes); apply transforms (rotate/crop images; swap words for synonyms — also handy for de-biasing by flipping gendered terms); add perturbation (noise) for robustness. |
| Simulation | Run experiments virtually instead of dangerously — self-driving cars meeting a "horse" in a simulator, robot-arm movements, rare financial or climate scenarios. |
| AI-powered | Use a model to generate data: simulate API outcomes without calling them, simulate humans, self-play (a bot playing itself for the equivalent of ~180 years of games a day), or generate instruction data. |
Recall from Unit 14 the key discipline for AI-generated inputs: use structured dimensions (features × scenarios × personas), not "give me test queries," or you get generic mush.
This is what to raise when someone proposes synthesizing training data. AI-generated data will keep growing in importance, but it likely can't fully replace human data, for four reasons:
Generated data can be low quality, and if you can't verify it, you can't trust it. The gate on synthetic data is having reliable ways to evaluate it — which loops straight back to your eval system (Units 09, 14, 15). No verification, no confidence.
Training a small "student" on a big "teacher's" outputs often copies the teacher's style but not its factual accuracy or ability to generalize. Worse: if the teacher can solve hard math and you train the student on those solutions, you teach the student to produce answers that look like solutions even when it can't actually solve them — i.e. to hallucinate confidently. Real reasoning gains come from better base models, not imitation.
Recursively training on AI-generated data can cause irreversible degradation: models over-represent probable events and forget rare ones over generations (a cancer-detection model that increasingly "forgets" the rare positive case). Collapse is likely if the training set is entirely synthetic; mixing in real data helps — though there's no settled recipe for the right ratio. All-synthetic is the danger zone.
Generating with model X hides where the data came from. If X was trained on copyrighted material, your model may inherit the violation. If X saw benchmark B, your strong score on B is contaminated and meaningless. Without clear lineage you can't fully trust performance or commercial safety.
The balanced view: some teams do succeed with heavy synthetic data (one instruction-tuned model used ~98% synthetic data) — but typically for a single iteration and with verified synthetic data only. The pattern that works is "generate, then verify, then use only what passed," never "generate and trust."
Distillation is a specific, valuable use of synthesis: train a small student to mimic a big teacher, so you get a cheaper, faster model with comparable performance. DistilBERT kept ~97% of BERT's capability at 40% smaller and 60% faster; Alpaca was a small model finetuned on a large model's outputs. It pairs naturally with lightweight adapters (LoRA) from Unit 19 to slash inference cost.
That closes the data-cost deep dives. The remaining DEPTH units turn to launch: monitoring and the feedback loop (Unit 21) and defensive prompting (Unit 22).