OpenAI's invisible watermark to catch ChatGPT cheats

Students are turning in term papers written by bots, and propaganda farms can now flood the web with eerily convincing comments. Once text is on the screen, proving a machine wrote it feels almost impossible. OpenAI wants to change that by hiding a mathematical fingerprint inside the text itself.

Speaking at the University of Texas, OpenAI researcher Scott Aaronson revealed that the company has a working prototype for statistically watermarking text generated by models like GPT-3 and ChatGPT.

Turning token choices into a secret code

Under the hood, GPT generates text by picking the next token from a pool of around 100,000 choices based on probability. A little randomness is usually mixed in, which is why running the same prompt twice gives you two different answers.

Instead of true randomness, OpenAI engineer Hendrik Kirchner built a prototype that uses a pseudo-random function tied to a secret cryptographic key known only to OpenAI. To a human reader, the words flow naturally. But across a few hundred tokens, basically a single paragraph, the system leaves behind a detectable statistical trail.

Crucially, you cannot just beat the system by moving a few sentences around or deleting some words. Because the score is calculated across the broader sequence of tokens, Aaronson noted that the signal is robust against casual edits. In theory, it could even scan a long essay and pinpoint which paragraphs came from GPT and which ones came from a human.

OpenAI did consider the blunt-force approach of logging every single response into a giant database for later lookup. But Aaronson said that idea creates massive privacy issues, making mathematical watermarking a far cleaner path.

Fighting essay mills and digital troll armies

OpenAI is building this because the abuse cases are already knocking on the door.

In academia, people are already calling this AI-assisted plagiarism "AIgiarism." Aaronson pointed out that current models can easily pull a solid B in most college courses. Without a reliable way to spot it, students will be sorely tempted to let ChatGPT do their homework.

Outside the classroom, the stakes are much higher. Aaronson warned about automated propaganda campaigns that could spam comment sections with fake pro-war arguments without needing a building full of human trolls in Moscow. Bad actors could also clone a real person's writing style to frame them for controversial posts. In his view, almost every malicious use of AI relies on hiding the machine's involvement, meaning a solid watermark attacks the root of the problem.

Why it matters

We think this cryptographic approach is a brilliant piece of engineering. Storing everyone's prompt history in a database would be a privacy disaster, and third-party AI detectors that guess based on text perplexity are notoriously unreliable. Tucking a hidden mathematical pattern into token probability distributions solves the verification problem cleanly without logging personal chats.

For teachers, editors, and platform moderators, an official watermarking tool would be a game changer. It offers a tangible defense against essay fraud and can protect honest writers from being falsely accused by janky heuristic checkers.

However, the real-world limitation is glaring. This watermark only works if OpenAI holds the keys and runs the model. If a college student uses an open-source model running on their own laptop, or if a state-sponsored troll farm runs a modified local LLM, OpenAI's secret keys are useless. The watermarking system sets an important ethical benchmark, but bad actors looking to cause real chaos will simply use tools that do not play by OpenAI's rules.