I was just reading this in the Wall Street Journal: “Is It Human or AI? New Tools Help You Spot the Bots”. The detectors are arriving almost as fast as the thing they detect. ChatGPT has been public for six weeks — it launched on November 30 and gained a million users in five days — and there is already a cottage industry in catching it.
One of them is GPTZero. NPR reported yesterday that its author, Edward Tian, is “a 22-year-old senior at Princeton University”, a computer science major minoring in journalism, and that he released the app on January 2. It uses two measurements. “Perplexity measures the complexity of text; if GPTZero is perplexed by the text, then it has a high complexity and it’s more likely to be human-written.” Burstiness “compares the variations of sentences. Humans tend to write with greater burstiness, for example, with some longer or complex sentences alongside shorter ones.” The logic is simple enough. A language model picks the likeliest next word, so its prose is predictable to another model — and predictable is the tell.
“More than 30,000 people had tried out GPTZero within a week of its launch”, NPR says, and the app crashed under the load. Tian himself concedes it “isn’t foolproof”. That candor is the most reassuring thing about it. For anyone grading essays, hiring writers or just reading their own inbox, these tools are going to matter as much as the models do.
Also, and not surprisingly: I think Google leads the other major three in image creation.
