An AI detector that reads the ideas, not the prose

A team from the University of Maryland and Google DeepMind posted a detector this week that asks a different question about AI writing. Not "was this prose generated," but "did the ideas come from a person."

It strips a document down to a paraphrased outline of what each section actually does, then judges that outline, so the surface style stops carrying the signal.

The numbers show why that matters. As documents were written from more and more detailed human plans, a standard detector kept flagging them as AI 92% of the time. This one's false alarms fell from 95% to 7%, tracking the actual source of the thinking. On 50 stories humans wrote from AI-generated plans, it flagged 68% as machine-led, against 8% for the comparison tool.

It's a preprint. Its 0.6% false-positive rate on human writing still trails the best existing detectors, it stumbles on some newer models, and the team says plainly it isn't ready to settle authorship disputes.

Students and journal editors are the first users everyone pictures. But the real customer is anyone who needs to know whose judgment they're buying. A grant reviewer. A hiring manager reading a take-home. An investor reading a memo. Detecting AI prose was always a losing game, because prose is easy to launder. Detecting whose ideas these are is the question underneath, and it reaches a lot further than the classroom.

Read the primary source →