Insurance AI initiatives stall in production not because the technology fails but because of gaps in governance, tacit knowledge capture, and organizational readiness. John Deahl of Telos argues that closing the pilot-to-production gap requires capturing experienced judgment, defining use cases before buying platforms, and building governance from day one.
Key takeaways for insurance leaders
- The gap between AI pilots and production is not a technology gap, it is a governance, knowledge capture, and organizational readiness gap.
- Decades of underwriting and claims judgment exist only in experienced staff nearing retirement; if that knowledge isn’t captured and embedded in systems, no technology purchase can replace it.
- Buying AI platforms without a defined, financially material business use case is how programs get large and underdeliver.
- Governance must start on day one, not at the production gate, “judgment infrastructure” that captures how decisions are made is the audit-ready standard insurers need to reach.
The pattern is familiar enough in insurance AI that it has become a source of real organizational frustration: a pilot produces compelling results, leadership enthusiasm builds, the team moves toward production release, and then legal or security halts deployment. The technology worked. The organization was not ready for it. Several months later, with goodwill eroded and budgets strained, the initiative is either quietly shelved or relaunched with a narrower scope than anyone originally intended.
John Deahl, at Telos, names the underlying condition with precision: the gap between AI pilots that produce demos and production systems that can be defended to regulators, E&O carriers, and shareholders is not primarily a technology gap. It is a governance gap, a knowledge capture gap, and an organizational readiness gap. Closing it requires a different kind of investment than most AI programs are currently making.
The knowledge walking out the door
The most underappreciated risk in insurance AI adoption is not a technology failure. It is the departure of the people whose judgment the organization has never formally captured. Deahl’s framing is direct: “This isn’t data that lives in a dashboard or shows up in an audit. It’s 20 to 30 years of pattern recognition, judgment calls, risk calculations, and relationship context that exists only in people’s heads.”
Insurance organizations are in the midst of a significant wave of experienced workforce exits. Underwriting judgment, claims handling expertise, and complex account knowledge that took careers to develop are concentrated in people who are within a decade of retirement. When those individuals leave without their knowledge having been extracted and embedded into systems, the organization loses capabilities that no technology purchase can replace. AI that runs on well-documented, well-governed institutional knowledge is qualitatively different from AI running on transaction logs. The former captures how decisions were made. The latter captures only that they were made.
Platform first is usually investment second
Deahl identifies a second structural failure that is worth naming plainly: vendors are selling platforms, and organizations are buying them without defined business use cases. A compelling demonstration is not a use case. The absence of a specific, financially material business problem to solve, and an honest assessment of whether the organization has the data and process infrastructure to address it, means the investment is driven by vendor momentum rather than organizational need. That sequencing is how insurance AI programs get large and underdeliver.
The corrective is not complex, but it requires discipline that procurement and innovation processes often undermine: identify the specific decision process the AI is meant to improve, document the current state with enough rigor to define what improvement looks like numerically, and then evaluate technology against that standard rather than against demo performance.
Governance as a day-one activity
The third failure mode Deahl identifies is governance timing. “Governance has to start from day one, not when you’re ready to commit to production.” The legal, compliance, and security review that stops an AI initiative at the production gate is not a late-stage obstacle. It is an early-stage gap that was never addressed. Organizations that treat governance as a deployment checklist rather than a design input will repeatedly encounter this problem. The fix is structural: business and technology need to work together from project initiation, with legal and compliance involved in framing the use case, not reviewing the output.
What Telos describes as “judgment infrastructure”, systems that capture how decisions are made, measure whether those decisions improve over time, and create traceable records for every output including human overrides, is the audit-ready, regulatorily defensible production standard that AI systems in insurance ultimately need to reach. Building toward that standard from day one is less friction than retrofitting it after a legal hold.
A ReSource Pro perspective
The organizations building production-grade AI in insurance are doing foundational work before they deploy: documenting decision workflows, cleaning and governing data, and establishing oversight structures that satisfy audit requirements. The AI trust gap closes through operational discipline and process rigor, not through model selection. Tacit knowledge capture, governance architecture, and documented workflow design are not peripheral AI-readiness tasks. They are the core of what determines whether AI deployed in insurance is defensible, scalable, and durable.
Telos’ John Deahl: Why insurance’s AI problem isn’t the models
Author: John Deahl
Publication: The Insurance Lead
Original Publication Date: June 9, 2026
