OpenAI rethinks its ChatGPT cheat detector

OpenAI built a tool that can catch students using ChatGPT to write their papers, but it is terrified of actually releasing it.

According to The Wall Street Journal, the company has developed a highly effective text watermarking system for ChatGPT. Yet, executives are locked in an internal debate over whether the public should ever see it.

An OpenAI spokesperson confirmed the research, stating that the company is taking a deliberate approach because of the complexities involved and the broader fallout it could create.

An invisible stamp on every word

Detecting AI text has historically been a fool's errand. OpenAI learned this the hard way last year when it quietly shut down its own detection tool due to embarrassingly low accuracy.

This new watermarking approach takes a completely different path. It does not try to detect AI text across the web. Instead, it focuses solely on ChatGPT.

By slightly tweaking how the model selects words, OpenAI embeds an invisible mathematical pattern into the generated text. To a human reader, the prose looks completely normal. Run it through OpenAI's detector, however, and the watermark lights up immediately.

In an updated research post, OpenAI noted that the system is remarkably accurate and can easily survive local paraphrasing.

Trivial to beat, easy to harm

So why leave it on the shelf? Because bad actors can defeat it in seconds.

OpenAI admitted that the watermark crumbles under basic circumvention. Running the output through a translation tool, asking another model to rewrite it, or simply inserting and deleting special characters between words wipes the watermark out completely.

Then there is the collateral damage. OpenAI pointed out that watermarking could easily stigmatize non-native English speakers who rely on ChatGPT purely as a writing assistant to clean up grammar.

Why it matters

This dilemma puts OpenAI between a rock and a hard place.

If OpenAI releases the detector, it handicaps itself. Cheaters and casual users will simply flock to rival models that do not watermark their output, costing OpenAI massive user engagement while solving none of the underlying cheating problems.

For educators and businesses, this confirms what many researchers suspected. We believe the dream of a silver-bullet AI detector is effectively dead.

Watermarking looks like a lock that only keeps out honest people. Until detection works across all models and resists basic rewriting, relying on software to police originality will likely punish innocent learners while letting genuine cheaters walk right through.