OpenAI hit with first defamation suit over ChatGPT hallucination

AI hallucinations are usually just annoying, but things get messy when a chatbot fabricates a multi-million-dollar felony out of thin air and pins it on you.

On June 5, Georgia radio host Mark Walters filed a defamation lawsuit against OpenAI in the Superior Court of Gwinnett County. The lawsuit followed an incident where ChatGPT generated a fabricated legal summary claiming Walters was accused of defrauding and embezzling funds from a non-profit organization. Reported on June 7, this appears to be the first-ever defamation lawsuit filed over false information invented by ChatGPT.

A fake summary of a PDF it couldn't even read

The details of the incident border on the absurd. A journalist named Fred Riehl was looking into an actual federal court case and provided ChatGPT with an online link to a PDF, asking the bot to summarize it.

At the time, ChatGPT lacked the required plug-ins to access external web links. Instead of telling the journalist it couldn't read the file, the system simply hallucinated. It produced an intricate, convincing summary that mixed factual details with outright fabrications. Crucially, the bot claimed Walters was believed to have embezzled "in excess of $5,000,000" from a gun rights group called the Second Amendment Foundation.

Walters had never faced any such allegations. Riehl never published the chatbot's output and instead verified the claims with another source, which caught the error. While court filings don't reveal how Walters caught wind of the output, he filed for unspecified monetary damages against OpenAI.

When Section 230 stops protecting the machine

Chatbot hallucinations have caused plenty of real-world headaches already. We have seen a college professor threaten to flunk an entire class after ChatGPT falsely accused students of cheating, and a lawyer facing sanctions after submitting hallucinated case law to a judge. But Walters' lawsuit takes the problem to uncharted legal territory.

For decades, Section 230 has protected internet companies from liability over defamatory content posted by third parties on their platforms. But ChatGPT does not host user content; it synthesizes and outputs entirely new text. Whether Section 230 shields a company when its AI generates defamatory claims from scratch is an untested question.

Winning the case will still be an uphill battle. UCLA law professor Eugene Volokh noted that while libel claims against AI developers are legally viable in theory, this specific case will be hard to sustain. Walters did not notify OpenAI beforehand to request removal, and because Riehl never published the false claims, showing tangible harm or damages will be difficult.

Why it matters

We think this lawsuit exposes a glaring contradiction at the heart of OpenAI's marketing strategy.

On one hand, OpenAI puts a small disclaimer at the bottom of ChatGPT warning that it may produce incorrect info. On the other hand, the company markets the tool as a reliable way to "get answers" and "learn something new," with CEO Sam Altman repeatedly talking about how he prefers learning from ChatGPT over reading books. You cannot boast about delivering answers while treating blatant lies as a minor disclaimer.

For anyone using generative AI, this is a clear warning sign. When a model cannot verify facts, it will happily invent details with absolute confidence. Treating ChatGPT like a search engine or using it for legal research without external verification remains a recipe for disaster.

For the AI industry, this lawsuit marks the beginning of a reckoning. If courts ultimately rule that AI-generated fabrications fall outside Section 230 protections, tech companies will suddenly face real legal liability for their models' outputs. A tiny disclaimer on a homepage will no longer be enough to dodge the bill.