Signal / Noise

Essays & working notes

Thoughts

Ideas tested against real products, written down once they've survived contact with one.

The eval is the spec

For probabilistic systems, requirements written in prose are opinions. Requirements written as evals are testable. Write the eval first.

The model is the easy part

Trace failed enterprise AI projects back to their first crack and it's almost never the model. The difficulty moved — spend your early weeks where it went.

How I decide an agent is good enough to ship

The demo tells you what a system can do. The eval tells you what it will do. Only one of those is a shipping decision.

Workflow or agent? Do the arithmetic first

Autonomy is not an ambition decision. It is a cost decision, and the costs compound per step.

If they don't use it, it doesn't exist

A model that everyone overrides has failed, even though every launch metric says otherwise. Adoption is the metric the others report to.

← Back home Disagree with something? LinkedIn →