Substack’s AI Detector and the Return of the Witch Hunt
Substack now scans your posts for AI with a tool called Pangram. It scored this one 100% AI, then 100% human.
In Salem, the accusation was the evidence. To be named was to be halfway to guilty, and the naming needed nothing more than a neighbour’s certainty and a room willing to believe them.
This week Substack gave every reader a machine that names.
In this post I will:
Read Substack’s new AI detector against the logic of a witch hunt.
Lay out the evidence that these tools fail, both in the lab and in social media.
Show you, with this very post and a free ‘humanising’ tool, how little the scan is worth.
What Substack actually launched
On Monday, Chris Best (Substack’s CEO) announced a partnership with Pangram, described as the ‘leading’ AI-detection tool. You can now scan notes, replies, comments, and posts over 100 words to see an estimate of how much was written by a human hand and how much with AI. The word for the thing it hunts is ‘Claudefishing’: the con of investing your attention in writing with no human thought behind it.
I share the worry underneath it. Feeds are filling with text that no one meant. Trust between readers and writers is why we write, and it is worth protecting.
I use these tools myself, and I said so this week in my own statement. Honest work gets made with AI every day. The trouble is that a scanner cannot tell that work from the con, so it treats everyone as the con.
The scanner protects none of it. It hands us a number and asks us to feel something about a person.
The accusation becomes the evidence
Arthur Miller wrote The Crucible about a town that mistook suspicion for proof. Once the machinery of accusation was running, doubt itself became damning. The question stopped being ‘what did this person do’ and became ‘why are they so sure they are innocent’.
A detector installs that machinery in your reading app. Every post now arrives with a button that asks a question about the person who wrote it. The reader becomes an examiner. The writer becomes a suspect who must, at any moment, prove a human wrote each sentence.
No one hangs for a high ‘human’ score, and I am not pretending they do. The parallel is narrower and more stubborn than that. It is the structure of the thing: a community that installs a device to find hidden guilt starts to see hidden guilt, and the person under suspicion is left proving a negative to a room already leaning towards the machine.
Chris is careful. He says the tool shows results only to those who ask, that Pangram is not perfect, that people should make their own judgements. Salem was careful too, in its way. It had procedures. Underneath them sat a prior belief that hidden guilt was everywhere, and a will to build the device that would find it. The care rode on top of the suspicion and gave it a clean face.
The tool does not work, and we already know it
I have written about this before…
Set the philosophy aside and look at the record.
Analysis of Turnitin (another ‘leading’ AI detection software tool), published earlier this year found it scores blended writing backwards. It over-flags light human editing and under-detects heavy AI use. Text pushed through a cheap humaniser came back at zero. The tool is least reliable exactly where the stakes run highest.
This is not one bad detector. A 2023 Stanford study tested seven of them against essays written by real people who speak English as a second language. On average, 61% of those genuine human essays were flagged as AI. On one detector, 97% of the essays written by non-native English speakers were called machine-written. Every one of those students wrote every word.
Chris tells you independent research suggests high accuracy. The same independent research Pangram points to actually states:
“Findings show that while detection tools can provide useful initial flags, they should not be used as sole evidence in high-stakes decision-making but should be implemented in a broader evaluation strategy.”
It punishes the people already punished for how they write
The false positives have a pattern. They land on people who write in a second language, and on neurodiverse writers whose rhythm, repetition, and structure a model reads as synthetic.
These are the same people who have spent their whole lives being told they speak wrong and write wrong. Now a scanner offers a fresh, numerical way to say it, with the authority of a machine and the deniability of an estimate. A tool that fails hardest on the least protected is an old prejudice with a new interface.
We are using an AI to accuse you of using an AI
Chris Best opens his announcement with an engraving of The Turk, the eighteenth-century machine that toured Europe beating people at chess. The Turk was a trick. A human chess master sat hidden inside, working the arm. It faked machine intelligence by concealing a person.
We have now built the mirror image. Pangram is a real machine, trained on vast amounts of text to make a probabilistic guess, and we have pointed it at your writing to work out whether a person is hidden inside. To decide if there is a human on the other end, Substack asks a machine.
We are outsourcing a judgement that was supposed to stay human. The reader used to decide whether a piece felt alive by reading it. That is the skill. Handing it to a detector retires the muscle we most need to keep.
If this is landing, my book goes deeper on exactly this: when to trust these tools, and when to leave them alone.
So I ran this post through a humaniser
Before publishing, I put this entire post through through a free humanising tool (superhumanizer.ai) and nothing else. I copied my published text in, pasted the result straight into a new post with no edits, and scanned it with Substack’s Pangram tool. It came back 100% human. You can check the humanised version yourself here.
According to Substack and Pangram the version I am actually publishing (the one you are reading now) scored 100% AI. The same argument, same ideas, one free tool in between. That is the whole point of the piece, proved on the piece itself.
The figure moved because I passed my writing through one more machine. The authorship never changed. A score that shifts when you add or remove a tool is measuring tools, while the person who wrote the sentences stands right here, unmoved.
It runs both ways. The same scanner a free humaniser can talk out of an accusation will, on another day, flag a real writer who never touched AI.
The honest version of this already exists
Substack shipped the good idea in the same announcement. The ‘How I make this’ statement lets a writer tell you, in their own words, how their work is made. I wrote mine this week:
I use AI to research, to edit, and now and then for a turn of phrase I keep. The thinking is mine, the argument is mine, and every source here is one I have checked myself.
AI detection does not work. Trust what a writer tells you about their process before you trust a percentage from a tool that guesses. These scores fall hardest on people who write in a second language and on neurodiverse writers, who have already been judged their whole lives for how they speak and write.
Be kind. Be fair.
It sets an expectation and asks you to hold me to it.
So here is the ask, since Chris invited one. Keep the statements. Keep the reply rules that let a community set its own norms, and give readers the choice over what reaches them. Drop the scan. A percentage stamped next to a person’s name reads as a verdict, and you have built it on a tool your own post admits is not perfect.
That is trust doing what trust does. A person tells you something and stakes their name on it. You believe them until they give you reason not to. Chris ends his post with a good line: when he wants Claude’s opinion, he will ask Claude. I would only add that when you want to know if there is a person behind the writing, you can ask the person.
Scan me if you like. I have told you how I work. The number will tell you less than this article already has.
Go slow.
Pangram: 0% accuracy, 100% confidence.
Thanks for this, Sam. I know you've written extensively about AI detection tools in the past, so it's good to get your measured take on this. It's also interesting to read the responses across the Substack community to these latest developments, especially since many people are ESL or neurodivergent.
Like you, I use AI for research, structure and editing, but the ideas and thinking behind my pieces are all mine. Surely we should rely on the reader to judge the quality of the work. It's like the library telling you which books to read!
I can't help thinking that Substack has shot itself in the foot here. I can understand their motivation, but I think they are missing the point somewhat. Because in my experience, Substack is about three things: content, connection and community and with this bungled attempt at protecting the quality of the content, they risk damaging the connections and alienating the communities the platform is built upon.