AI text detectors are biased against non-native English speakers

If English is your second language and you just spent hours drafting an essay from scratch, there is a very good chance an algorithm thinks you cheated.

According to a study published in the journal Patterns, popular AI text detectors show a massive, systematic bias against non-native English speakers. The research was led by James Zou, an assistant professor of biomedical data science at Stanford University.

A failing grade for native-bias detectors

Zou and his team ran 91 English essays from the Test of English as a Foreign Language, or TOEFL, through seven widely used GPT detectors.

The results were brutal. More than half of the non-native essays were falsely flagged as AI-generated. One tool went so far as to label a staggering 98% of the student essays as machine-written.

Yet when the researchers fed the same detectors essays written by native eighth graders in the US, the tools flipped. Over 90% of those essays were correctly identified as human-written.

While companies selling these detection tools regularly claim 99% accuracy, the researchers warned that those figures are misleading at best.

The perplexity trap

To understand why this happens, you have to look at how these detectors work. Most of them rely on a metric called text perplexity, which basically measures how surprised a language model is when trying to guess the next word in a sentence.

LLMs like ChatGPT naturally output low-perplexity text because they are built to generate the most probable words. But human non-native speakers also rely on common words, simpler phrasing, and familiar sentence structures. To a detector, that lack of lexical surprise looks identical to a robot.

Then the researchers pulled off a truly wild experiment. They took those falsely accused TOEFL essays and asked ChatGPT to rewrite them using richer, more sophisticated vocabulary.

When they fed the AI-edited versions back into the detectors, every single one passed as human. As the authors pointed out, detectors might paradoxically force non-native writers to use AI just to prove they are human.

Why it matters

We think this study exposes a massive, dangerous blind spot in how schools and hiring managers are rushing to police AI.

For international students, academics, and job seekers, the consequences of a false positive can be life-altering. You risk getting expelled, losing a visa, or seeing your resume quietly trashed by an automated filter, simply because your vocabulary is straightforward.

Trying to catch generative AI by measuring text predictability was always a flawed approach. As Jahna Otterbacher from the Cyprus Center for Algorithmic Transparency noted, ChatGPT learns fast and will easily outsmart static detection models. Fighting AI with more AI is a losing game.

If institutions keep relying on these flawed tools, they will not stop cheating. They will just end up punishing the very people who are working the hardest to bridge a language gap.