A lawyer cited fake cases invented by ChatGPT, and the judge is furious

If you thought AI hallucinations were just good for a few online laughs, a federal court in New York is here to show how catastrophic they can be in real life.

A practicing attorney turned to ChatGPT to conduct legal research for a personal injury suit, and the chatbot did what it so often does: it hallucinated six completely fictional court cases out of thin air. Instead of double-checking reliable legal databases when challenged, the legal team doubled down, asked the AI to fabricate the full opinions, and filed mobile screenshots of the chat right into the court docket.

The federal judge is furious, ordering a formal sanctions hearing that could derail multiple legal careers.

How a fake airline citation spiraled out of control

The dispute began with Mata v. Avianca, a lawsuit over injuries allegedly suffered aboard a flight in 2019. The airline filed for Chapter 11 bankruptcy the following year, leading to a legal fight over whether that bankruptcy discharged the plaintiff's claims or tolled the statute of limitations.

To counter the airline's motion to dismiss, the plaintiff's attorneys filed an opposition brief packed with what looked like rock-solid precedent. They cited cases like Varghese v. China Southern Airlines and Miller v. United Airlines, complete with volume numbers, page citations, and elaborate judicial quotes.

There was just one problem: none of those decisions existed. When the defense tried to look them up on Westlaw and PACER, they found nothing. Docket numbers pointed to entirely unrelated matters.

On April 11, the district judge ordered plaintiff's counsel Peter LoDuca to produce copies of the cited opinions. Rather than realizing the blunder, the team returned to ChatGPT, had the bot write out full-length bogus rulings, and submitted PDFs of scanned mobile screenshots. The interface's telltale down arrow was visible right on the court documents.

Asking the bot if it lied, and trusting its answer

On May 4, the judge issued an order to show cause, calling the filing an unprecedented circumstance packed with bogus judicial decisions, quotes, and citations.

Affidavits filed before the deadline revealed that another lawyer at the firm, Steven Schwartz, had run the queries. Schwartz admitted he had never used ChatGPT for legal research before and had no idea it could fabricate case law from scratch.

Even wilder was the affidavit's attached chat history. When Schwartz realized the cases were being challenged, he actually asked ChatGPT to confirm whether the Varghese case was real.

ChatGPT politely doubled down, claiming that upon double-checking, the case indeed existed and could be found on databases like Westlaw and LexisNexis. Schwartz took the bot at its word and attached the screenshot to show he had not intentionally misled the court.

Judge P. Kevin Castel was entirely unimpressed and set an in-person sanctions hearing for June 8, targeting LoDuca, Schwartz, and their law firm.

Why it matters

This mess is an urgent case study in what happens when professionals treat an autoregressive language model like a reliable fact repository. Large language models predict the next plausible token; they do not verify reality. When pushed for specific, authoritative citations, they will gladly synthesize plausible-sounding fakes.

For lawyers, researchers, and developers, the lesson is clear: AI can draft prose, summarize notes, or brainstorm angles, but it cannot be your primary source of truth. Every statute, case number, and statistical claim must be verified against primary sources.

In our view, while falling for the initial hallucination shows a glaring lack of technical literacy, the team's decision to double down was a collapse of professional duty. Trusting an AI model when it claims it double-checked itself borders on comical negligence.

Expect courts and professional bodies worldwide to formalize mandatory AI disclosure rules and verification protocols in response to this spectacle. The human who signs the brief carries full legal liability, no matter what the chatbot promised.