ChatGPT falsely accused an innocent man of murdering his kids

Curiosity can lead to dark places, especially when you ask an AI what it knows about you.

A Norwegian man named Arve Hjalmar Holmen recently queried ChatGPT about himself. Instead of a standard bio, the bot generated a horrific story, claiming he was a convicted murderer who had killed two of his own children and tried to murder a third, currently serving a 21-year prison sentence in Norway.

Even worse, the bot didn't just hallucinate a generic horror story. It blended fabricated murder charges with Holmen’s real-world details, accurately identifying his hometown, as well as the exact number and gender of his children.

A tiny disclaimer won't cut it

Austrian privacy advocacy group Noyb has now stepped in, filing a formal complaint on Holmen’s behalf with the Norwegian data protection authority, Datatilsynet. The complaint accuses OpenAI of violating the EU’s General Data Protection Regulation (GDPR).

Noyb is demanding regulators fine OpenAI, force it to erase the defamatory claims, and mandate technical overhauls so the model stops spitting out damaging falsehoods about individuals.

Joakim Söderberg, a data protection lawyer at Noyb, made the case plain. Under GDPR, personal data must be accurate, and when it is wrong, individuals have a legal right to get it corrected.

Söderberg blasted OpenAI’s standard defense, saying that displaying a tiny disclaimer at the bottom of the chat warning that the bot can make mistakes clearly isn't enough. In his words, a company cannot spread devastating false information and think it can walk away clean just by whispering that it might not be true.

The complaint redacts the exact date of Holmen’s initial prompt, but Noyb noted the query occurred before ChatGPT rolled out integrated web search. In an ironic twist, running the query today simply surfaces news stories about Noyb’s filing against OpenAI.

This marks Noyb’s second major regulatory strike against ChatGPT. In April 2024, the group filed a complaint after the tool repeatedly hallucinated a public figure’s date of birth. OpenAI argued back then that it couldn't actually correct internal model data, offering only to filter and block specific queries. Noyb countered that blocking prompts fails GDPR’s demand that inaccurate data be erased or rectified without delay.

Why it matters

This case lays bare the collision course between generative AI’s architecture and European privacy law.

For regular people, hallucinations like this are a living nightmare. When an AI hallucinates a celebrity’s birthday, it is a minor nuisance. But when an engine accurately lists your hometown and your children's genders while labeling you a child murderer, the potential for real-world reputational damage is terrifying. Proving your innocence against an authoritative-sounding machine output is an exhausting, unfair burden for any private citizen.

For the AI industry, OpenAI’s playbook of slapping a small disclaimer on the screen and setting up keyword filters is hitting a legal brick wall. LLMs predict tokens based on probabilities; they don't store facts like relational databases. When an error is baked into neural weights, companies genuinely struggle to surgically remove it without retraining, which is why OpenAI relies on post-hoc prompt blocking.

We think European regulators won't accept that excuse for long. GDPR does not grant exceptions for probabilistic math. If Datatilsynet rules that prompt blocking does not satisfy the legal right to rectification, OpenAI and its rivals will face an ugly choice: fundamentally overhaul how their models handle personal data, or risk crushing fines and potential service suspensions across Europe.