Perplexity (AI detection)

In language modeling, perplexity is a measure of surprise: given the words so far, how hard is it for a model to predict the next one? A text where each next word is exactly what a model would guess has low perplexity; a text full of unexpected turns has high perplexity.

AI detectors borrow this metric with a simple assumption: language models generate the words they themselves find most probable, so low perplexity is treated as evidence of AI authorship.

The problem is that low perplexity is not unique to machines. Non-native English writers are taught textbook phrases and safe collocations — "plays an important role in," "it is widely known that" — and reach for them deliberately, because tested phrasing means fewer grammar mistakes. That is careful learner English, and to a detector it looks statistically similar to model output.

Compare what the metric rewards:

Predictable: Social media plays an important role in modern society. Less predictable: Social media has quietly rewired how a generation argues, flirts, and grieves.

Both are legitimate sentences; only the second scores as "human-like."

This is the core mechanical reason detectors misfire on ESL writing. A Stanford study (Liang et al., published in Patterns, 2023) found AI detectors falsely flagged 61% of TOEFL essays by non-native English speakers.

Perplexity is usually paired with burstiness (variation in sentence rhythm); together they form the backbone of most first-generation detectors, and together they explain much of the flagxiety non-native writers report. Knowing the metric helps you understand a score — it does not obligate you to write worse English to satisfy it.