Operationalizing Agentic AI in Claims

The panel opened on what Chris Huff described as the most overlooked problem in agentic claims. Insurers are racing to deploy agents on top of intake processes where the underlying documents are messy, multi-format, and often handwritten. The agent inherits every weakness in the input.

Jun Yamada framed the same issue from a carrier’s perspective. The fastest way to make agentic adoption tractable, he argued, is to stop thinking about end-to-end claims automation and instead decompose the lifecycle into discrete steps: FNOL, ingestion, extraction, validation, triage. Some of those steps are information processing problems, where AI can carry real load. Others involve actual judgment, where the regulatory and litigation exposure means a human has to remain accountable.

That decomposition is also how trust gets built internally. Tokio Marine has been productionizing the lower-risk capabilities first — document ingestion, summarization, validation — while staying cautious on anything that touches adjudication or litigation. Chris reinforced that this pattern is industry-wide, not unique to Tokio Marine, and the reason is structural: regulated industries have spent years building deterministic processes, and agentic AI feels like a move back toward probabilistic ones. The work is putting in place the guardrails that let probabilistic technology produce deterministic-enough outputs to survive an audit.

Asked what a carrier could do to test whether their data foundation could support AI, Chris was direct: most carriers test on their best documents because they want a great outcome, and that’s exactly why the resulting pilot won’t scale.

He outlined three concrete tests any insurer can run before committing to an agentic deployment. The first is to pull a representative sample of real intake — handwritten FNOLs, labelled PDFs, 40-page packets stitched from multiple formats — and measure straight-through yield on the worst portion of it. The results will usually be sobering. The second is to calibrate confidence. When the system reports 90% confidence, does that mean it’s right 90% of the time? Those are not the same number, and most teams haven’t checked. The third is the reproducibility test: pick a random claim and try to reconstruct every input, every transformation, every decision, and every output. If that can’t be done in minutes, the system is not production-ready.

Frederic Stallaert pushed back on the idea that full automation is the destination most insurers should be targeting. In practice, none of Paperbox’s 75+ insurance customers have asked for fully unattended intake. The thing they’re actually optimizing for is operational efficiency and the quality of customer service — not automation for its own sake.

Three things hold back true automation in his experience. The first is multi-document ambiguity: a claim that arrives across five attachments, multiple emails, and a follow-up requires contextual reasoning the model has to do well across all of them, not just one. The second is the persistent reality of handwriting and low-quality scans. The third is ambiguity in business processes themselves — when two claims handlers in the same team use different typologies and label things differently, there’s no standard for the automation to follow. That last point reframes the work: before deploying AI, much of the value is in business process standardization and the governance around it.

Jun laid out a five-part framework for thinking about explainability and audit trail in regulated claims environments. The starting point is human in the loop — AI output should never be the sole decision-making instrument, and the adjuster needs to remain accountable. The second is purpose clarity: there’s a meaningful difference between a tool that organizes documents and one that influences valuation or settlement, and that boundary needs to be explicit.

The third is auditability: a clear record of inputs, outputs, human review, and final actions. The fourth is explainability at the business level — not the technical workings of the model, but how the decision can be explained in terms of facts, evidence, and rationale that a regulator or claimant can follow. The fifth is discoverability: AI outputs create new exposure in litigation, and carriers without traceable accountability struggle to respond when a claim is uncovered in court.

Frederic added a critical refinement: the audit trail is more than a log. Dumping data as a proof point isn’t enough. It needs to be viewable, workable, and managed well enough that the carrier can respond confidently to difficult questions when they come — not theoretically retrievable somewhere in the system.

The most pointed argument of the panel came from Jun, who described an asymmetry that’s quietly reshaping the economics of defense. Plaintiffs’ attorneys are deploying AI aggressively. Carriers, constrained by regulation and internal caution, are not keeping pace.

Jun’s point to his own group companies has been the same: litigation management is a stated pain point, but the cautious approach to AI on the defense side means every demand packet a carrier receives is now AI-optimized to find the highest-payout angle, while the response is still mostly human. The expense and indemnity curves on litigation are moving in the same direction the plaintiff-side AI investment is, and that correlation is unlikely to be coincidence.

Asked where carriers most commonly cut corners in governance, Chris named one pattern that fails the first real audit: capturing the outcome of an AI decision without capturing what produced it.

Most teams log that an agent approved a claim at, say, 92% confidence, and stop there. What’s missing is the underlying chain of custody — which version of the model ran, on what prompt, with what raw inputs, through what transformations, and why the confidence landed where it did. In the moment, none of that feels necessary. The dashboard passes. The demo looks beautiful. Then the first audit or subpoena arrives and the record is too thin to defend.

The implication is that the chain of evidence has to be designed in from day one, at the field level. Retrofitting it after an incident is not a strategy.

Frederic described two failure modes he sees repeatedly when carriers try to move from pilot to production. The first is mathematical. In a controlled pilot environment, an edge case is something a clever solution can absorb. At scale, edge cases compound — each one triggers a chain of downstream actions, and the work the automation was supposed to eliminate gets replaced by new work managing exceptions.

The second failure mode is human, and harder to engineer around. A pilot is usually run by a team that believes in the technology and trusts the purpose. When the same system is deployed to claims handlers and account managers who weren’t part of that team, they often resist it — finding clever workarounds rather than using the system as intended. The implication is unambiguous: if the end users who will actually interact with the tool aren’t involved during the pilot, the project fails after deployment, regardless of how the model performs.

Both panellists pointed to a similar pattern in where agentic AI is creating real value today. Summarization is effectively table stakes — ingesting 500-page demand packets and surfacing what the adjuster needs in time to respond inside a 24-hour window. FNOL intake is another high-impact area, particularly for CAT response where temporary intake staff often capture incomplete information; agentic systems can come back to claimants with a list of what’s missing instead of accepting whatever was submitted.

Jun shared a particularly vivid example from one of Tokio Marine’s high-net-worth group companies. For property fire losses, an AI tool now scans the claim file alongside fire and police reports, then cross-references the equipment and appliances in the home against active product recall databases. An adjuster handling a $3M+ home fire isn’t going to manually check whether the homeowner’s barbecue or appliances had open recall notices — but the AI surfaces them, opening subrogation paths against OEMs that would otherwise be impossible to pursue.

Frederic noted that the right application of agentic AI varies by line of business. Personal lines claims benefit from heavier automation because customers usually want fast, standardized settlement of low-value losses. Commercial lines claims are more data-heavy, so the value of agentic AI is in structuring and streamlining intake to give the human claims handler the context to decide faster — not in making the decision itself.

An audience question about lock-in risk — what happens if a third-party API gets restricted or a vendor changes terms — drew an architectural answer from both Frederic and Chris. Five years ago, Frederic noted, the advice to CIOs was to plan an exit strategy from any major technology investment on a five-year horizon. That horizon has compressed to two or three. The implication is that AI infrastructure needs to be designed for swap-ability from day one, with the ability to replace underlying models and providers as accuracy, cost, and policy conditions change.

Chris added a forward-looking framing on how that affects architecture overall: on a go-forward basis, the population of digital workers (agents) will grow faster than the population of human employees. Humans care about UIs. Agents don’t — they care about clean APIs. The investment that gets de-prioritized in most CIO shops today — API management, increasingly via emerging standards like MCP (Model Context Protocol) — is the investment that becomes structurally important next.

Asked what carriers should do starting today to avoid a litigation reckoning later, Chris distilled the panel’s central argument into one line.

Frederic added the second half of the answer. The carriers who survive will also have kept humans meaningfully in the loop on decisions that carry risk — not as a rubber stamp, but as a genuine review.

Jun, asked to add to it, declined to add anything: “Chris and Frederic took the words out of my mouth. We should probably take their words and build our governance model around what they said.”