319 Comments
User's avatar
Nick Chivall's avatar

Pangram: 0% accuracy, 100% confidence.

Dr Sam Illingworth's avatar

Just like all AI tools…

The Silver Sofa's avatar

Pangram could run for office someday

Dr Sam Illingworth's avatar

It certainly has the colour scheme...

Des Kennedy's avatar

Thanks for this, Sam. I know you've written extensively about AI detection tools in the past, so it's good to get your measured take on this. It's also interesting to read the responses across the Substack community to these latest developments, especially since many people are ESL or neurodivergent.

Like you, I use AI for research, structure and editing, but the ideas and thinking behind my pieces are all mine. Surely we should rely on the reader to judge the quality of the work. It's like the library telling you which books to read!

I can't help thinking that Substack has shot itself in the foot here. I can understand their motivation, but I think they are missing the point somewhat. Because in my experience, Substack is about three things: content, connection and community and with this bungled attempt at protecting the quality of the content, they risk damaging the connections and alienating the communities the platform is built upon.

Dr Sam Illingworth's avatar

Thanks Des! And honestly all they needed to do was ask a few actual writers and readers on this platform for their input.

Des Kennedy's avatar

Absolutely. It's business 101: listen to the customers' voice. Instead, they plumped for adding a feature that instantly makes everyone on the platform distrust each other. Go figure.

Heidi Theis Travel Matchmaker's avatar

💯💯💯

Taka Toyoshima's avatar

I had the same concern.

My first language is Japanese, but I write my newsletter in English for an international audience.

I don’t want to rely on automatic translation, so I write in Japanese first and then use ChatGPT and DeepL to help translate it into natural English.

As a result, AI detectors often classify my writing as AI-generated.

But the ideas, analysis, and judgment are still mine. AI helps me express them in another language—it doesn’t replace my thinking.

Dr Sam Illingworth's avatar

Exactly this Taka! I also wish I could use AI effectively to improve my Japanese!

TheAiBuildGuide's avatar

That's not a translation problem, it's that Pangram has no way to see origin, only the finished text. It can't tell if a sentence came from you writing it straight, translating it, or a model generating it start to finish, because none of that history reaches it. All it sees is the shape of the words after the fact, and plenty of genuinely human writing reads the same way to it.

CrossPlay Press's avatar

Look up things like rule of three, was x but was y, and metaphor stacking. Ai heavily use these formulas to write so when you ask for it to translate into natural English it could be applying these formulas. I’m noticing this when I ask it to rewrite my pieces to read more like an article. There are some drastic structural changes that I go back and fix.

Davina - Belonging to Myself's avatar

Taka, that really is an excellent point. This too is part of an important conversation - about all ESL writers.

Todd McKeever's avatar

The larger danger may be what this does to the reader. Instead of asking, “Did this help me think, feel, or see something differently?” we start inspecting every sentence for evidence of guilt.

A probability score cannot tell us who supplied the judgment, experience, or conviction behind a piece. Once suspicion replaces reading, strong human writers will get caught alongside the slop.

Dr Sam Illingworth's avatar

Exactly my worries as well Todd. 😢

Carey Lening's avatar

Well said. I support people explaining their process, and I agree that's a positive outcome. If they'd stopped with that, it would have been a great first step.

But what Substack actually did is exactly as you described: they brought witch-hunts to Substack.

Dr Sam Illingworth's avatar

And as you so brilliantly pointed out in your article I suspect it is the law which will cause them to back peddle…

Valentina's avatar

Thank you for devoting so much space to us non-native writers! English is my third language, and now I have two options: be judged for the imperfections in my English, or be judged for using AI to improve it. Checkmate!

Another thought on discrimination: it seems obvious that groups already subject to more scrutiny and discrimination — women, people of color, and others — will also be judged more harshly for using AI. We can already see this pattern emerging in research. Sadly, that is how these things usually work: the weakest and most marginalized people end up paying the highest price.

I disagree with you slightly about the statement, though. On its own, a “How I make this” statement might not be a bad idea. But placed where it is, right next to the detector’s verdict, it starts to look like an explanatory note from a schoolkid who failed to do their homework. That is why mine is aggressively sarcastic. Sorry not sorry, but I refuse to put myself in the position of someone pleading their innocence.

And of course, none of this is really about protecting us from slop. It is about looking good in investor decks. Which is very very sad.

Dr Sam Illingworth's avatar

You are so welcome. And thank you for your disagreement and perspective! Where do you think we could place this statement instead? As I genuinely think it is a good idea, but I am also conscious that I always speak from the epicentre of White, male privilege...

Valentina's avatar

I would place it somewhere that makes it part of the author’s normal context, e.g. on their profile, on the publication’s About page, or as an optional “How I made this” note at the end of the post. Somewhere the writer chooses to put it. Right now it feels more like a box for the accused to explain themselves.

Dr Sam Illingworth's avatar

This is a great suggestion. I would find this really valuable as a writer and a reader.

Woofs & Wisdom's avatar

What you have suggested, I have done with all of my Substack articles at the end of every post. Even before the introduction of this AI detection tool, I thought it was important to be transparent with my readers.

Dr Sam Illingworth's avatar

This is great to hear. Well done! 💪💪💪

Davina - Belonging to Myself's avatar

You have articulated something that I realise I was feeling uncomfortable about Valentina. Thank you.

J. Steven Robertson's avatar

We're not even tracking the right operator.

I read for content. I don't care how it's produced.

Bring me good logic and argumentation — teach me — and it could have been written by an elephant for all I care.

Dr Sam Illingworth's avatar

Exactly the same! 💪💪💪

Bryan Caballero @ The Shield's avatar

Amen!

Patricia Russo's avatar

Exactly!

Edwin Canizalez's avatar

I’ve never met a techbro that used the line “if you have nothing to hide, you should not fear this product” that I liked. Mostly because they make obscene amounts of money at the expense of people’s privacy.

I'm repurposing an answer I shared with another author (mostly out of laziness):

I did a random “test” with some of my pieces. The AI scan said almost all my poems with AI written with 0% human input. 😂 Then I did the same thing with some of my essays and it said only 10% was human in most cases 😂😂

The false positives aren’t evidence of AI authorship; it's just evidence that I have high standards. AI detectors measure surface artifacts: grammatical symmetry, thematic continuity, and statistically probable word pairings.

Like you said, when an AI “matches” my work, it’s performing a probability trick, concluding that after a word like frequency, the mathematically sensible companions are loss or signal. Not every piece I write is a grand slam, but my baseline craft is tight, evocative, and conceptually disciplined enough that machines mistake rigor for automation. And NO. I’m not going to contort my future work to dodge these alarms. I’m not going to lace my sentences with artificial “human irregularities” or run pre‑publication diagnostics to appease a detector. That’s not my job. My job is to write with the methodologies that built my voice in the first place. This is an arms race I don't care to engage in.

I started two podcasts to show folks that the author of my pieces and the guy talking on video are one in the same. I also started sharing essays with accompanying drafts belonging to my poetry pieces. The essays trace the process from first impulse to final version, accounting for the logic of what stayed. I did think of being transparent; I just wanted to share some knowledge.

Here's the thing at the end of the day, If people think I’m using AI to write my pieces, they can listen to my podcast, hear how I speak and realize I write the same way I speak. It's really not that hard. And if they still want to troll me, I politely suggest they can blow me...( a kiss before they say goodbye).

Dr Sam Illingworth's avatar

I love that you are creating podcasts but hope that you (and others) weren't 'forced' into it to prove your humanity. 😢

Edwin Canizalez's avatar

It ended up serving a dual utility. It gave me a chance to share my thoughts and for people to see I'm real. But the main driver wasn't proving it :)

Lee Shand's avatar

Sam, you honestly make this place a better place mate. I really don't have the energy to give you a full on Lee Shand reply on this subject anymore, but please know this place is better for having you.

Dr Sam Illingworth's avatar

Thanks so much Lee. I really appreciate that pal. 🙏

JHong's avatar

Sam… as you may have witnessed, I went on a 14-notes tirade about this integration. I was still in light testing mode, and figured many notes would be scanned, so I did scan my own notes. All 100% human.

Then… I found a better statistic. And I also decided to add sharper language on the founder (I added disclaimer-> exception, since he says it shouldn’t be the sole arbiter but has no qualms outing journalists and authors with the tool on Twitter).

I made two light edits. A reader later told me its scanning as 100% AI written. It flipped 100% in the other direction from 2 edits.

Make it make sense!

Dr Sam Illingworth's avatar

This is absolute insanity! And thank you for all the digging and work you have been doing here JHong!

Rachel @ This Woman Votes's avatar

There is a big difference between the confidently wrong, uninterrogated, boldly published AI slop, being done by the laziest thinkers on this planet, and some of the best uses of AI that I have seen on Substack, people that edit and review and, most importantly, fact check that idiot AI. Because at the end of the day, this is a tool that is tuned for emotional resonance, not epistemic authority, every single output must be fully reviewed and questioned and researched.

But the problem with Pangram in particular is even when it’s confidence is LOW it flags as AI - so it is itself, boldly wrong some of the time. If they are going to hand out a % AI Written, % AI Assisted, and % Human Written, it should all be paired with % Confidence.

I have been working in commercial ML/AI implementations since 2013, and I have seen some really incredible use cases, if Substack wanted to do something really amazing for humanity, they would enable a Trust Score and a Fact Checker, THAT is the use case that humanity needs, because like every other platform that has ZERO responsibility for the quality of content published, this one is prone to absolute bullshit, even when it is 100% human written.

%100 guaranteed, all that spewed directly from my human brain 🫠

Dr Sam Illingworth's avatar

Love this! And also you absolutely highlight how they seem to have picked one of the worst AI tools in terms of false confidence and fabrications. 🫠🫠🫠

Diane's avatar

Just another tool to bully or falsely accuse I can so this happening lord lol

Dr Sam Illingworth's avatar

It's not good is it Diane. 😢

Diane's avatar

It would be ok if used for the right reasons and responsibly. It’s not perfect I mean like everyone states you are asking an ai tool to check for ai generated material lol I mean…

Billy Spencer's avatar

Thank you for writing this. I put a few of my articles through the tool and some came back human and others AI. I do use AI with editing and counter arguments (you helped me hone this skill) so I’m not surprised the flag of AI is raised. Like you, the arguments and ideas are mine, with some help of a tool that is available. I’m sure some of the Grammarly edits or even edits from any spellcheck will flag as AI.

2 be more hooman is two misspell and uz bad grammer

Dr Sam Illingworth's avatar

Thank you Billy! And exactly. Also if we all wanted 💯'human scores' we could just used a humaniser without any engagement or thought. Go figure. 😬

Billy Spencer's avatar

Right. So use an AI tool to trick the other AI tool that I am a human. Not what I’m imagining Substack had in mind here! 😂

Dr Sam Illingworth's avatar

Exactly! And yet… here we are. 🫠🫠🫠

Cornelia Krom's avatar

Very funny!

Kyle Hetrick's avatar

Sam, this gives sharper language to a concern I recently tried to name in my own essay. The deepest failure here is a category error. Pangram can identify linguistic patterns. It cannot measure authorship, integrity, lived experience, or whether a writer takes responsibility for what bears his name. Yet the interface invites readers to turn a probability into a moral verdict.

I use AI openly in my own process as a research aid, editor, and thinking partner. But my work begins in recovery, ministry, prayer, study, conviction, and questions I am actually living. I verify, revise, reject, and accept responsibility for every published word. No percentage can describe that process.

And the irony is nearly perfect: AI is now being used as a purity test for human expression, while human readers outsource their discernment to the machine. If the goal is trust, machine-generated suspicion cannot create it. Honest disclosure, accountable authorship, and careful reading might.

Thank you for your perspective and insight.

Dr Sam Illingworth's avatar

Thank you Kyle. It is great to hear about your own process as well. Ans it sounds completely driven by human judgement. 💪

Brian Elliott's avatar

The first post I wrote and put through their tool scored 70% human. I rewrote it substantially. The score dropped to 62%.

My human editing scores lower than the Claude project I use for editing (and then re-edit).

Dr Sam Illingworth's avatar

Thanks, Brian. This says it all, really, doesn't it...

Leif Linden's avatar

Detecting plagiarism is fair game: copying someone's argument whole cloth violates the rules and institutions that have been built, for good reason, around authorship.

Detecting whether writing "sounds AI" is a different act. There's no offense to verify, only a judgment about what a human is supposed to sound like, and that falls hardest on people whose natural voice reads as unusual: non-native speakers, the neurodivergent, anyone formally trained into a stiffer register. That's no longer plagiarism detection. That's punishing someone for how they choose to write.

Dr Sam Illingworth's avatar

Great distinction Leif. And even the plagiarism tools we have had for decades now is not really fully fit for purpose.