Last week, an agent at one of our pilot agencies told the PolicyIQ chat: "Looks like maybe we should use your competitor."
The competitor was ChatGPT.
She had asked PolicyIQ a renters question about vandalism. PolicyIQ replied with a confident answer — citing a Commercial Property form. Wrong policy entirely. The right policy (the Squire HO form, vandalism is Peril 9) was sitting in the library all along; we just hadn't grounded the answer in it. She pasted the same question into ChatGPT-in-a-project, and ChatGPT gave a clean answer that pointed at the right form name.
That message hit us hard. Not because ChatGPT was better than PolicyIQ — it wasn't, on most dimensions — but because in that one moment, with that one wrong citation, we'd done the worst thing an insurance tool can do: we'd sounded confident and been wrong.
So we rebuilt the chat pipeline. This post is what changed, and why "just use ChatGPT" is still the wrong answer for a multi-producer agency — even after that incident.
The Problem: Confident Wrong Answers
Large-language-model retrieval has a known failure mode. When the model can't find a grounded answer, it doesn't stop — it generates one that sounds right. The output looks like a citation. It names a form. It quotes some text. But the quoted text isn't in the form, or the form isn't relevant to the question, or both.
In a coding context, that's annoying. In an insurance context, it costs trust. An agent who reads PolicyIQ's answer to a client and gets corrected by the carrier two days later doesn't come back to PolicyIQ. And neither does the agent next to her.
ChatGPT has this same failure mode. It's just better at hiding it, because most ChatGPT users aren't fact-checking every citation against the source PDF. We were.
The Fix: A Deterministic Citation Validator
The core change in PolicyIQ v2 is plain-code logic that runs after the AI writes its answer and before your agent sees it.
The validator does one thing, very specifically:
- The AI produces an answer plus a list of citations (document, page, snippet).
- For each citation, the validator looks up the actual text on the cited page of the cited document.
- If the snippet appears word-for-word on that page, the citation passes.
- If it doesn't, the entire answer is killed.
That last bullet is the wedge. The AI cannot ship a fabricated citation past the validator. It can ship an answer that says "I don't have a confident citation for this." It cannot ship "according to Form CG 00 01, Section II, Paragraph B" when those words don't actually exist on that page. That's gone.
For the agent, the visible change is small: most answers come back the same, just with citations they can trust. Behind the scenes, every answer is gated.
ChatGPT-in-a-project has no equivalent gate. It can — and does — generate citations that don't survive a fact-check.
The Other Rebuilds That Mattered
The validator was the big one, but the original incident had several causes. We fixed each:
1. Carrier-Specific Vocabulary
The renters question failed partly because the AI didn't know that "renters" in a Squire HO form lives under Coverage C, "vandalism" is Peril 9 on the Perils Insured Against list, and the question would never resolve from a Commercial Property form. We added a vocabulary map: consumer terms (renters, vandalism, teen driver, water damage) translate to carrier-form terms (Coverage C, Peril 9, youthful driver, water damage incl. flood/burst pipe/sewer) before retrieval runs.
ChatGPT pattern-matches generic insurance language. It doesn't know that your carrier's form numbers exist, let alone what's in each section.
2. Page-Anchored Citations
Every PolicyIQ citation opens the source PDF directly to the cited page, with the cited snippet highlighted. The agent doesn't trust us — they verify in one click.
ChatGPT-in-a-project links to "files in this project" but doesn't anchor to a specific page or show the snippet. The verification step is on you.
3. Honest Fallback Behavior
When PolicyIQ can't find the answer in your uploaded documents, it doesn't dead-end the agent. It returns a senior-P&C-style answer behind a clear banner: "This answer is outside your uploaded policy documents." The fallback model never sees the retrieved excerpts, so it can't invent a citation. The agent knows exactly what they're getting: industry knowledge, not your specific policy.
ChatGPT's confidence stays high whether or not the source disappears. There's no banner. The tone for grounded and ungrounded answers is the same.
4. Admin Audit Trail
Every PolicyIQ chat is logged. Admins can review what each agent asked, what was retrieved, and what was cited. A wrong answer doesn't just disappear — we (and your principal) can see it and fix the gap.
ChatGPT-in-a-project gives the agency owner zero visibility into what their team is asking or trusting.
Why "Just Use ChatGPT" Is Still the Wrong Answer
The agent who said she'd switch to ChatGPT didn't actually switch. After the rebuild, she didn't have to. But it's worth answering the question directly, because we get it a lot:
Why not just upload our policies to ChatGPT and call it done?
Five specific reasons, in order of how often each matters in a real agency:
- No citation gate. ChatGPT will generate a confident-sounding citation for language that isn't in your PDFs. PolicyIQ's validator makes that impossible.
- No carrier vocabulary. ChatGPT doesn't know that Coverage C is renters in your HO form, that I282 is a real endorsement code on a Squire farm policy, or that an NCCI class code lookup belongs in a workers' comp manual. PolicyIQ does.
- No shared library. Files you upload to a ChatGPT project stay on one user's account. PolicyIQ is multi-user from day one — every producer and CSR works from the same indexed library, and new hires start fully loaded instead of starting from zero.
- No page-anchored citations. ChatGPT can't open the source PDF to page 27 with the cited snippet highlighted. PolicyIQ does that on every citation, so verification takes one click.
- No admin audit trail. The owner of a ChatGPT-in-a-project setup can't see what their team is asking. PolicyIQ logs every chat for admin review and gap reporting.
For a solo producer doing low-stakes research, ChatGPT-in-a-project can work. For a 5-producer agency where one wrong answer to one client erodes trust across the book, it's the wrong tool — even after our incident, even after the rebuild was clearly necessary.
Where This Fits in Tool Selection
If you're choosing an AI tool for insurance policy work in 2026, three things to look for, in order:
- Does the tool verify its own citations? Ask the vendor. If the answer is anything other than "yes, every citation is checked against the source document before the answer ships," assume citations can be fabricated.
- Does it know your carrier's vocabulary? Test it with a question that requires carrier-specific knowledge ("Is vandalism covered on a Squire renters policy?" / "What does endorsement I282 do on a farm policy?"). A tool that gives a generic answer is pattern-matching, not retrieving.
- Is it built for an agency, not a single user? Multi-user, shared library, admin audit, role-based document access. These aren't nice-to-haves for an agency at any size — they're table stakes.
For a deeper category-level breakdown, read our roundup of the best policy search tools for insurance agents, which compares PolicyIQ against the alternatives across these dimensions and others. For how the search actually works under the hood, see how AI answers insurance policy questions in seconds.
The Honest Version
We almost lost a pilot agency to ChatGPT because we shipped a confident wrong answer. We took that personally and rebuilt the pipeline around a citation validator that makes fabricated citations impossible to ship. The agency stayed. The agent who flagged it is still using PolicyIQ daily.
That doesn't mean we'll never get an answer wrong. It means when we do, the failure mode is "I can't answer that from your library" — not "according to Form CG 00 01, which actually doesn't say what I just quoted."
If you want to see PolicyIQ v2 in action, read the full overview and pricing or try the live demo. If you'd rather talk through whether it fits your agency, book a 30-minute discovery call — we'll be honest about when PolicyIQ is the right tool and when something else is.
And if your agency is still on the "let's just use ChatGPT" path, that's the conversation we'd love to have. The free Stop Digging Through PDFs guide is a fine starting point either way.
Common Questions About AI Citations
Can ChatGPT cite the wrong insurance policy?
Yes — and it can do it confidently. ChatGPT-in-a-project will happily generate a paragraph that looks like a citation from your uploaded form, even when the cited language doesn't actually appear on the cited page. It has no gate that verifies the snippet against the document before showing it to your agent.
What is a citation validator?
Plain-code logic that runs after the AI writes its answer and before your agent sees it. The validator takes every citation the AI produced — document, page, and snippet — and checks the snippet word-for-word against the actual text on that page. If the snippet doesn't match, the answer is killed and replaced with a fallback. Fabricated citations cannot ship.
Why does PolicyIQ verify citations when ChatGPT doesn't?
Because we built PolicyIQ for insurance agents, and a confident wrong citation in front of a client costs more than the time the AI saved. ChatGPT is a general-purpose tool. It's optimized for fluent text, not for grounded citations in an agency setting. Adding a deterministic validator means we accept some answers that say 'I can't answer that from your library' in exchange for never quoting language that isn't really in the policy.
What happens if PolicyIQ can't find the answer in our documents?
PolicyIQ gives the agent a senior-P&C-style answer behind a clear "outside your uploaded policy documents" banner — so they're not dead-ended on the call. The question is also logged in the gap report so admins can see what's missing from the document library and which policies to upload next.
Is ChatGPT-in-a-project good enough for an insurance agency?
It depends on what you'll accept. ChatGPT-in-a-project handles a single user, doesn't verify citations against your forms, doesn't anchor citations to a specific page in the PDF, and stays trapped on one laptop. For a solo producer doing low-stakes research, it can work. For a multi-producer agency that needs cited answers in front of clients, shared knowledge across the team, and admin visibility into what's being asked, it isn't built for the job.