How I decide an agent is good enough to ship

The demo tells you what a system can do. The eval tells you what it will do. Only one of those is a shipping decision.

Every agent demo I have ever seen was impressive. That is not a compliment to the agents; it is a property of demos. A demo is a system meeting the three inputs it was rehearsed on, in front of people who want it to work. The question a demo answers — can this work? — stopped being interesting the moment foundation models got good. The question that decides whether you ship is different: what does this system do on the thousand inputs nobody rehearsed, and can I live with the ways it fails?

That second question has a tool, and the tool is an eval. Not a vibe check, not "we tried twenty prompts and it seemed fine" — a harness: a fixed set of real cases, a definition of what good means for each, and a score you can re-run every time anything changes. Building one is unglamorous. It is also the closest thing this field has to a shipping criterion.

What the harness actually contains

Mine has four parts, and none of them is optional.

A golden set that looks like production, not like the pitch. Real inputs, pulled from the actual workflow, including the ugly ones — the ambiguous request, the question that's really two questions, the input that's missing half its context. If your test set is cleaner than your traffic, your eval is a second demo.

A definition of "good" written before you look at outputs. For each case: what must be true for this answer to be acceptable? Writing the criteria after seeing outputs is how teams convince themselves that whatever the system does is what they wanted. The criteria are the spec. Write them first.

A failure taxonomy, because failures are not equal. An answer that is unhelpfully vague, an answer that is confidently wrong, and an answer that fabricates a policy are three different products. I bucket every miss: harmless miss · wrong but visible · wrong and plausible. The last bucket is the one that matters, because it is the one users won't catch — and in an enterprise workflow, plausible-and-wrong compounds silently downstream.

Thresholds tied to blast radius, not to pride. A drafting assistant with a human reviewing every output can ship at a failure rate that would be disqualifying for a system that acts on its own. "Good enough" is not a number; it is a number for a given level of autonomy and a given cost of being wrong. Same model, same score — different shipping decisions.

The decision, stated plainly

I ship when three things are true. The eval score clears the threshold for the failure modes that matter — not on average, but specifically in the plausible-and-wrong bucket. The failure modes that remain are ones the workflow can absorb — a human catches them, a guardrail contains them, or the cost of one is small. And there is a way to notice change: the harness runs on every prompt edit, model swap, and retrieval change, because a system that passed in March and silently degraded in June is worse than one that never shipped — people have stopped checking it by June.

Notice what is not on the list: whether the demo went well, whether the model is state-of-the-art, whether a competitor announced something. None of those move the decision, because none of them change what the system does to the thousandth unrehearsed input.

The uncomfortable part is that the harness is usually more work than the agent. A capable prototype takes an afternoon now; a golden set with honest criteria takes days of arguing about what good means. That inversion bothers people. It shouldn't. Arguing about what good means is the product work — the model is just the part that came pre-built.

If you take one thing: a demo is evidence about the system's ceiling. An eval is evidence about its floor. You ship on the floor.