Adoption is a product metric, not a change-management afterthought
If the producers don't use it, the model doesn't exist. Why adoption belongs on the dashboard next to precision and recall, and how to design for it from day one.
Here is a model with a precision of 0.91 and a recall of 0.88, validated, documented, and deployed. And here is the underwriter it was built for, who glances at its recommendation, ignores it, and prices the risk the way she always has. What is that model's real precision? It's undefined — because a recommendation nobody acts on has no outcomes to be right or wrong about. The dashboard says 0.91. Reality says zero. The gap between those two numbers is adoption, and most teams treat it as somebody else's problem, to be handled after launch by a "change management" workstream with no seat at the design table.
That's the error. Adoption isn't the thing that happens to a finished product. It's a property of the product, as measurable and as designable as precision, and it should sit on the same dashboard. Precision and recall tell you whether the model is right. Adoption tells you whether being right matters. You need both numbers, and only one of them is on most teams' walls.
Why offline metrics quietly lie to you
Offline metrics answer a question posed in a vacuum: given this input, is the output correct? Adoption answers the question that actually pays the bills: given this output, in this workflow, under this person's incentives and time pressure and professional pride, does the decision change? A model can win the first question and lose the second decisively, and nothing in your validation suite will warn you, because your validation suite never met the user.
The reasons a good model goes unused are specific and, crucially, mostly designable:
| Why they don't use it | What it's really about |
|---|---|
| "I don't know why it said that." | No reason attached to the output. Experts don't act on assertions they can't interrogate. |
| "It's not how I work." | The tool sits beside the workflow instead of inside it — another tab, another login, another step. |
| "It was wrong once." | A single salient miss with no graceful failure mode resets trust to zero. |
| "If it's wrong, it's on me." | The person carries the accountability but the tool took the judgment. Nobody trades authority for exposure. |
Notice what's not on that list: "the model wasn't accurate enough." Accuracy is table stakes and rarely the binding constraint. The binding constraints are legibility, fit, failure behavior, and the alignment of authority with accountability — and every one of them is a design decision you make early or forfeit.
Stop asking "is the model good enough to ship?" and start asking "what would make a busy expert change a decision they're personally accountable for?" The second question designs a product. The first one only validates a model.
Designing for adoption from day one
Instrument it like any other metric. Pick an adoption measure before you build, not after you launch. Not logins — decision influence: how often does the output actually change what the human does? Override rate is the most honest number in the building. A high override rate isn't a user-training problem; it's the product telling you the truth. Put it on the dashboard next to precision and watch how differently the team behaves when both are visible.
Attach a reason to every output. In regulated financial services you likely need this for the examiner anyway — but the same explanation that satisfies governance is what earns the expert's trust. An output an underwriter can interrogate is one she can decide to trust or overrule on the merits. An unexplained score is one she can only ignore. Explainability isn't only a compliance surface; it's your primary adoption surface, and it's cheaper to build once for both.
Get inside the workflow, not beside it. The single largest adoption tax is asking someone to leave the system they already live in. A suggestion that appears where the decision is already made — in the underwriting screen, in the claims queue — starts from trust. A suggestion behind a separate login starts from friction and rarely recovers.
Design the failure, not just the success. Experts don't abandon tools for being imperfect; they abandon them for being confidently, opaquely wrong with no recourse. Show uncertainty honestly. Make the low-confidence case look different from the high-confidence one. A model that says "I'm not sure here, take a look" survives its mistakes. A model that is equally certain when it's wrong as when it's right gets switched off the first time it's caught.
Keep authority with the accountable human. Position the model as an input to a decision the expert still owns, not a verdict they must overturn. People adopt tools that make them faster and better at a job that remains theirs. They quietly sabotage tools that seem to be auditioning for their judgment while leaving them holding the risk.
Do this and the payoff compounds in a way the offline metrics can't show you: an adopted model generates real outcomes, real outcomes generate the feedback that improves the next version, and the improvement earns more adoption. A model nobody uses gets none of that. It just decays quietly at 0.91, a correct answer to a question no one asked it, on a dashboard that was measuring the wrong thing all along.
If you take one thing: put override rate on the dashboard next to precision. The model's real accuracy is whatever survives contact with a user who didn't have to listen.