Avoiding AI Hallucinations in Case Law Discovery and Search

Most firms now treat generative AI as table stakes for case law research. The failure mode has been written into the public record often enough that it no longer qualifies as a surprise: a plausible-sounding citation, a fluent parenthetical, a holding that happens not to exist. The remedy is not another warning memo to associates. It is architecture.

Case law search is the single workflow where hallucinations have cost lawyers money, reputation, and admission. A Thomson Reuters Westlaw study of cases between June 30 and August 1 identified 22 different cases in which courts or opposing parties found non-existent cases within filings, leading to discipline motions or sanctions in many instances. The scale is wider than any single survey captures: Damien Charlotin's AI Hallucination Cases database lists 2,202 court and tribunal decisions involving AI-fabricated citations as of its 19 October 2026 update, with United States courts accounting for 1,501. So how should a firm build a research stack that stops this at the infrastructure layer rather than the human one?

Why General-Purpose Retrieval Is Not Enough

The common assumption is that connecting a model to a legal database solves the problem. The empirical record disagrees. The Stanford RegLab "Hallucination-Free?" study concluded that while legal RAG systems can reduce hallucinations compared to general-purpose AI like GPT-4, hallucinations remain substantial, wide-ranging, and potentially insidious. The retrieval layer in a research tool can locate relevant authorities and still allow the model to miscite them, misquote them, or synthesize a holding the opinion never reached.

There is a structural reason. A 2025 OpenAI and Georgia Tech paper argues that language models hallucinate because standard training and evaluation procedures reward guessing over acknowledging uncertainty. A model trained to produce a confident answer will produce one even when the retrieved passage does not support the proposition it is asked to defend. Retrieval raises the ceiling of what the system can know; it does not change what the model is incentivized to say.

The practical implication for firm IT buyers is that the question is no longer "does this tool use RAG." Every serious vendor does. The question is which specific controls sit between retrieval and the attorney's draft, and whether those controls are inspectable. That is a procurement conversation, and it belongs upstream of any pilot. For deeper treatment of the verification question itself, see the discussion of verifying AI legal reasoning before client use.

A Closed-Corpus Retrieval Layer, Not an Open Web

Grounded legal retrieval begins by narrowing what the model is allowed to see. In the production architecture appropriate for case law research, the corpus is scoped before the first token of generation: a curated set of jurisdictional reporters, the firm's own matter library, treatises licensed for use, and nothing else. The model is not permitted to fall back on parametric memory or on web search when the retriever returns thin results. It is instructed, and gated, to abstain.

Closing the corpus has three effects. It eliminates the entire category of fabricated citations that arise when a model invents a plausible-looking reporter cite to fill a gap. It makes every passage the model uses traceable to a specific document, page, and paragraph in the index. And it makes the retrieval layer itself auditable, which matters when a court or an insurer asks what the system saw. Firms with jurisdictionally diverse practices should pair this with jurisdiction-aware routing so that a Delaware query cannot be answered from a California corpus.

For a mid-sized litigation group, the practical form of this is a private LLM running against the firm's indexed corpus, with the index itself versioned and dated. The question of where that index and model sit — on-premise, in a VPC, or in a hybrid pattern — is treated in detail in a separate deployment comparison, and the economics are covered in the cost analysis.

Citation Resolver Services as a Hard Gate

The second architectural element is a citation resolver that runs after generation and before anything reaches a human. Every citation emitted by the model — case name, reporter, pin cite, parenthetical holding — is parsed, normalized, and submitted to a resolver service that verifies the citation exists in the corpus, that the reporter volume and page match, that the quoted language appears on the cited page, and that the parenthetical characterization is consistent with the surrounding opinion text.

A citation that fails any of these checks is not shown to the attorney as a soft warning. It is removed from the draft, logged, and the model is re-prompted with the failure as context. This is a hard gate, not a nudge. It is the single control most directly responsive to the sanction pattern: in February 2025, three Morgan & Morgan attorneys were sanctioned by Judge Kelly H. Rankin of the U.S. District Court for the District of Wyoming after submitting a motion in limine containing eight fabricated case citations generated by the firm's in-house AI platform in a product liability suit against Walmart. Lead attorney Rudwin Ayala lost his pro hac vice admission and was fined $3,000, while two colleagues each paid $1,000.

A resolver would have caught every one of those citations before the brief left the firm, because none of them existed in any reporter the resolver could reach. The failure was not of judgment. It was of infrastructure.

Holding-Level Grounding and Shepardizing Hooks

Citation existence is necessary but not sufficient. The next layer verifies that the proposition the model attributes to a case matches the holding the court actually reached. This is what the Stanford researchers called the misgrounding problem, and it is the harder of the two to catch because the cite is real.

Holding-level grounding operates on two signals. The first is passage-level attribution: for every substantive assertion in the generated memo, the system records the exact retrieved passage used to support it and surfaces that passage inline so the attorney can read source and claim side by side. The second is a treatment check that calls a Shepardizing or KeyCite-equivalent service and flags any cited authority that has been reversed, overruled, questioned, or distinguished since publication. Cases flagged as negatively treated are not silently cited; they are either annotated with the current treatment or removed from the draft.

In a well-formed pipeline, retrieval, resolution, grounding, and treatment checks run concurrently rather than in strict sequence. The architectural pattern resembles the orchestration logic described in work on graph-based legal workflow orchestration, where each stage emits structured outputs the next stage can consume without waiting for a monolithic completion. The practical benefit is latency; the governance benefit is that every stage logs its own result, so a failure can be attributed to a specific step rather than to the pipeline as a whole.

Attorney Sign-Off Gates and Audit Trails

No credible case law research architecture ships without a human-in-the-loop gate at the end. The U.S. District Court for the District of Colorado sanctioned two attorneys for submitting a brief containing nearly thirty defective citations generated by AI, finding Federal Rule of Civil Procedure 11 requires attorneys to verify filings even when AI tools are used. The duty does not transfer to the vendor, and no amount of grounding engineering changes that.

What the architecture can do is make the sign-off substantive rather than ceremonial. The reviewing attorney sees the generated passage, the retrieved source passage, the resolver result, the treatment status, and a diff when the model's language departs materially from the source. Sign-off is captured as a signed event tied to the user, the matter, the model version, the index version, and the retrieved document set. That record is what a firm hands to a court, a bar counsel, or a malpractice carrier when the question of reasonable care is asked after the fact. The broader governance frame — policies, roles, retention — is covered under legal AI governance and human-in-the-loop controls.

The regulatory environment is converging on the same expectation. A legal AI governance tracker found that as of October 3, 2026, there were 251 entries addressing AI in court filings — including judges' standing orders, district local rules, state court rules and agency policies — with 228 in force and 182 of those binding attorneys. On July 10, 2026, the U.S. Court of Appeals for the Eleventh Circuit referred attorney Anthony F. Sabatini for possible discipline after finding his briefs relied on fake, AI-generated citations, formally routing hallucinated citations into attorney-discipline machinery in a published opinion.

What a Defensible Case Law Stack Looks Like in Procurement

The components above are not aspirational. They are the minimum a firm should be able to point to when a managing partner, a general counsel, or an insurer asks how the research stack prevents the failure mode that produced the Morgan & Morgan, Sabatini, and Colorado sanctions. In procurement, that translates to specific requirements: a closed and versioned corpus; a citation resolver with an audit log; holding-level attribution surfaced to the reviewing attorney; a treatment-check hook; a hard sign-off gate; and a signed event trail per matter.

None of this is unique to any one vendor. It is the shape of the problem. Firms that build or buy against this shape have a defensible answer when the question comes. Firms that treat hallucination controls as a feature to be added later tend to meet the question for the first time in a courtroom. For buyers evaluating how these controls fit a specific deployment, LAW.co's enterprise AI deployment practice is organized around exactly this architecture.

AI in case law research is a tool, not a shortcut around Rule 11. The firms that have learned this the expensive way did so because they treated the model as the system. The system is the model, the corpus, the resolver, the grounding layer, the treatment hook, and the attorney's signature — in that order, with each one accountable to the next.