The problem
Underwriters spent much of their day reading policy documents to assemble rationales by hand. An earlier vendor pilot had produced fluent summaries that occasionally invented clauses — which made it unusable in a regulated process.
What we did
- Built the evaluation harness first, scored against a set of underwriter-graded cases
- Grounded every generated claim in a retrieved passage with a visible citation
- Added a human review step that captures corrections as future evaluation data
- Versioned prompts and models so any output could be reproduced on demand
