Another wonderful piece around critical literacy around AI (and punishing seemingly AI-generated content), Sam! I'd never thought about the global implications of the systems misguided by AI unless I read about how critical industries like healthcare rely on research papers passing through the same AI funnel. You are right in enforcing better human judgement and that we cannot do that by playing the volumes game since humans are not like machines.
I'm also curious to know more about your experiences in peer-reviews (and how they differ from now) before AI came to our lives.
Thanks Raghav, great question. In my opinion, things have changed in terms of the fact that the amount of publications we're seeing submitted has increased exponentially. Part of this was already happening after Covid, as researchers had more time to write, but the proliferation of, at first, quite poor and more recently relatively high-quality articles that are clearly written by AI has become out of control. However, it's the system that's broken, as ultimately academics are in a publish or perish situation and publishers stand to make millions of dollars off of our efforts. In my opinion, AI is just highlighting once again a system that is very broken.
TOTALLY Agree - SO MANY papers are written now - just like the proliferation of Substack posts - and I’d suggest many Substack posts are higher quality than some submitted papers I review?!
And when I review - the process has asked me to “tick an online box” to verify “NO AI support has been used in your review”!
BUT - I don’t know how this is checked by the Editors of that journal???
I don’t publish papers but I do like to use them to back up facts. With the information that you shared here, I can see people losing trust in all information in general. I also worry about medical journals where research impacts healthcare.
Peer review is so important! Not sure if this is general, but in my field there are usually two to three reviewers, and I find it necessary. Some focus more on the technical details, others on the big picture, like how well the paper fits into the literature.
However, reviews take months, as we tend to do them for free, in our own time, anonymously, just for the sake of the scientific community. Therefore, a human reviewer will never be able to compete with an AI reviewer’s speed. In the case you are talking about, was it the reviewer who delegated the task to AI? Could it have been just an AI revision of the review’s language, or was there certainly no human involved? I don’t really see the reason why anyone would agree to review and then ask AI to do it instead since there is no payment involved.
Excellent points and TBH the whole system is broken. Most of the big publishing houses triple dip an make a fortune whilst outsourcing all of the work to academics for unpaid 'service.' As usual AI has probably just highlighted something that needed fixing way before 2022.
I think the ‘reviewers’ (many of whom are undergrads with no training) copy and paste into GenAI tools and then share the output with no validation or checking. I have seen this myself as Editor on multiple journals across several disciplines. 😢
That’s so interesting, thanks for sharing! I just can’t imagine handing in a bad quality, obviously AI-written review, it seems like an obvious way to damage one’s reputation. Maybe people are afraid to refuse the task and that’s why they accept even when they don’t have the time.
Love this article. Especially how it calls out that this is a structural issue, I find it with my students too, I don’t suspect that they use AI all the time, but that sometimes when tired and met with unreasonable demands, AI takes over. Peer review research is essential to so many walks of life, this discussion needs to be had more, and the solutions have to structural
What strikes me here is that the deepest problem isn’t just AI use in peer review, but the substitution of output for judgement inside a validation system.
If reviewers are overloaded, undertrained, and working inside a publish-or-perish economy, then LLM use becomes less an individual shortcut than a structural symptom. But peer review only means anything if a human reader is doing the hard part: deciding what matters, what is noise, what is genuinely novel, and what doesn’t quite ring true.
That’s why this matters beyond academia. Once a system starts confusing fluent output with informed judgement, the problem is no longer just misconduct. It is trust infrastructure beginning to fail.
This is a great observation, Steven, and definitely something that's gonna be plaguing academia for the foreseeable future. Unfortunately, AI helps to further exacerbate a system in which quantity is often prioritised over quality.
Thanks, Sam — that’s exactly what struck me. AI feels less like the origin of the problem here than an accelerant for a system already drifting toward quantity over judgement. Really thought-provoking piece.
This is a powerful piece because it shows that the issue is not just that reviewers are using AI, but that the review system itself has become structurally impossible to sustain at this scale. When nearly 20,000 submissions depend on overextended volunteer reviewers working under time pressure, AI use starts to look less like an isolated ethical lapse and more like a predictable response to institutional overload.
What I find most important is your framing of this as a trust infrastructure problem. The combination of identity leaks, reviewer harassment, and AI-generated review at scale suggests that the crisis is deeper than misconduct. It raises the question of whether peer review, in its current form, can still support the level of legitimacy academia expects from it.
At the same time, I think the next question is even harder: if human judgement is what peer review is supposed to protect, then what institutional conditions are needed for that judgement to remain real rather than symbolic? In that sense, AI may not just be undermining peer review. It may be exposing how fragile the model had already become.
This is a great point. And I agree that in cases like this, AI is the catalyst ha helps us better understand how and why a particular system is broken.
Same question: if AI is reviewing AI and AI starts getting trained with AI generated content, should we start removing the AI prefix from everything for simplicity?
Thanks Sam - Another WONDERFUL post highlighting a significant issue across many levels of education - from elementary school to doctoral level!!
Peer-review is one issue - and I think your post nails many issues around weaknesses that are increasing exponentially in academia!
I don’t always trust “peer-reviewed” as a recommendation for an educational intervention that schools should use… I have to READ at least one of their “peer-reviewed” research papers - to actually confirm that the research is “evidence-based” and includes data on students’ performance that shows significant improvements are due to that intervention!
NEXT - I look at the research design & the variables reported to “control” for the impact of these variables related to and yet separate from the intervention target skills & knowledge? So, I mean variables that might be correlated with the dependent variable…
For example oral reading fluency is correlated with reading comprehension and so is vocabulary knowledge… So - I ask myself “Are these variables measured or controlled for in this paper about comprehension?”
I believe this logic about research design & control of potential variables is critical in peer review? MY Personal view!
As well, some “peer-review” academics think theoretical papers & survey research can be used to recommend educational interventions? Again, my personal view is that they cannot guide classroom teachers as to what might work for their students’ learning…
Back to AI and peer-review - I think this logical thinking process (my sort of thinking at academic level) needs to be applied, with examples, for students & teachers in schools!
I’m still thinking this through & your posts provide good directions for me!!
MANY THANKS yet AGAIN!! Best, Gail 👍👍😊
Ps that “longer responses” for AI is clearly a pattern, as previously discussed!
Dear Sam, that’s a brilliant analysis of the ICLR situation!
You’re absolutely right: when we allow AI to 'accelerate' critical processes, we lose what matters most,human depth and accountability.
Your blog is a rare island of clarity in a world trying to automate even our thinking. I’m glad to see you’re still championing the 'slow-down' where the industry demands a blind sprint!!
I sat for quite a few years on the (12 person) scientific committee of a UK national radiological society and part of our job was to select oral presentations for the annual conference. However, the submissions were abstracts with a maximum word count of about 300, which is typical for a medical conference. We had about 500 abstract submissions so filtering them down to about 200 (for a 3 day conference) was manageable by us, based on a scoring system. Each person would only be asked to score about 50 abstracts (within their specialised area), which was about 8-10 hours work.
I can't quite imagine how the 'analysis' system worked in this (ICLR 2026) case (how would AI score a submission?) but having 19,000 submissions and peer reviews that ran to 3000 words sounds truly bizarre. A conference that attracts 19,000 submissions is basically 'out of control' as you would need a small army of experts to sift that many. And using AI (glorified pattern recognition) for this sort of complex 'triage task' is a bit of a non-starter anyway.
Exactly this Philip. I have also been involved with conferences that accept around 21,000 abstracts, and there are definitely ways of doing it, but it becomes almost a full-time job in itself for the organising committee.
Absolutely Paul! I honestly think AI is just shining a light on the rapidly decaying infrastructure of academia in all its many guises. The optimist in me thinks that this is an amazing opportunity, but we have been here before…
Another wonderful piece around critical literacy around AI (and punishing seemingly AI-generated content), Sam! I'd never thought about the global implications of the systems misguided by AI unless I read about how critical industries like healthcare rely on research papers passing through the same AI funnel. You are right in enforcing better human judgement and that we cannot do that by playing the volumes game since humans are not like machines.
I'm also curious to know more about your experiences in peer-reviews (and how they differ from now) before AI came to our lives.
Thanks Raghav, great question. In my opinion, things have changed in terms of the fact that the amount of publications we're seeing submitted has increased exponentially. Part of this was already happening after Covid, as researchers had more time to write, but the proliferation of, at first, quite poor and more recently relatively high-quality articles that are clearly written by AI has become out of control. However, it's the system that's broken, as ultimately academics are in a publish or perish situation and publishers stand to make millions of dollars off of our efforts. In my opinion, AI is just highlighting once again a system that is very broken.
TOTALLY Agree - SO MANY papers are written now - just like the proliferation of Substack posts - and I’d suggest many Substack posts are higher quality than some submitted papers I review?!
And when I review - the process has asked me to “tick an online box” to verify “NO AI support has been used in your review”!
BUT - I don’t know how this is checked by the Editors of that journal???
I don’t publish papers but I do like to use them to back up facts. With the information that you shared here, I can see people losing trust in all information in general. I also worry about medical journals where research impacts healthcare.
Exactly this! I wish more people were talking about it. As it is not just an issue for academics. 🙏
Peer review is so important! Not sure if this is general, but in my field there are usually two to three reviewers, and I find it necessary. Some focus more on the technical details, others on the big picture, like how well the paper fits into the literature.
However, reviews take months, as we tend to do them for free, in our own time, anonymously, just for the sake of the scientific community. Therefore, a human reviewer will never be able to compete with an AI reviewer’s speed. In the case you are talking about, was it the reviewer who delegated the task to AI? Could it have been just an AI revision of the review’s language, or was there certainly no human involved? I don’t really see the reason why anyone would agree to review and then ask AI to do it instead since there is no payment involved.
Excellent points and TBH the whole system is broken. Most of the big publishing houses triple dip an make a fortune whilst outsourcing all of the work to academics for unpaid 'service.' As usual AI has probably just highlighted something that needed fixing way before 2022.
So who is it who uses the AI, the researcher who agreed to review, or is it never even sent to a real reviewer?
I think the ‘reviewers’ (many of whom are undergrads with no training) copy and paste into GenAI tools and then share the output with no validation or checking. I have seen this myself as Editor on multiple journals across several disciplines. 😢
That’s so interesting, thanks for sharing! I just can’t imagine handing in a bad quality, obviously AI-written review, it seems like an obvious way to damage one’s reputation. Maybe people are afraid to refuse the task and that’s why they accept even when they don’t have the time.
I just honestly think many people have never been trained how to do peer review properly. 😢
Love this article. Especially how it calls out that this is a structural issue, I find it with my students too, I don’t suspect that they use AI all the time, but that sometimes when tired and met with unreasonable demands, AI takes over. Peer review research is essential to so many walks of life, this discussion needs to be had more, and the solutions have to structural
Thanks Ellen. And I agree entirely, in many ways AI is just catalysing the structural erosion of issues that have been decades in the making. 😢
YES, it is devastating but just a fact.
What strikes me here is that the deepest problem isn’t just AI use in peer review, but the substitution of output for judgement inside a validation system.
If reviewers are overloaded, undertrained, and working inside a publish-or-perish economy, then LLM use becomes less an individual shortcut than a structural symptom. But peer review only means anything if a human reader is doing the hard part: deciding what matters, what is noise, what is genuinely novel, and what doesn’t quite ring true.
That’s why this matters beyond academia. Once a system starts confusing fluent output with informed judgement, the problem is no longer just misconduct. It is trust infrastructure beginning to fail.
This is a great observation, Steven, and definitely something that's gonna be plaguing academia for the foreseeable future. Unfortunately, AI helps to further exacerbate a system in which quantity is often prioritised over quality.
Thanks, Sam — that’s exactly what struck me. AI feels less like the origin of the problem here than an accelerant for a system already drifting toward quantity over judgement. Really thought-provoking piece.
Thank you Stephen. 🙏
This is a powerful piece because it shows that the issue is not just that reviewers are using AI, but that the review system itself has become structurally impossible to sustain at this scale. When nearly 20,000 submissions depend on overextended volunteer reviewers working under time pressure, AI use starts to look less like an isolated ethical lapse and more like a predictable response to institutional overload.
What I find most important is your framing of this as a trust infrastructure problem. The combination of identity leaks, reviewer harassment, and AI-generated review at scale suggests that the crisis is deeper than misconduct. It raises the question of whether peer review, in its current form, can still support the level of legitimacy academia expects from it.
At the same time, I think the next question is even harder: if human judgement is what peer review is supposed to protect, then what institutional conditions are needed for that judgement to remain real rather than symbolic? In that sense, AI may not just be undermining peer review. It may be exposing how fragile the model had already become.
This is a great point. And I agree that in cases like this, AI is the catalyst ha helps us better understand how and why a particular system is broken.
Same question: if AI is reviewing AI and AI starts getting trained with AI generated content, should we start removing the AI prefix from everything for simplicity?
Now THAT is the question that nobody is asking... 😔
Thanks Sam - Another WONDERFUL post highlighting a significant issue across many levels of education - from elementary school to doctoral level!!
Peer-review is one issue - and I think your post nails many issues around weaknesses that are increasing exponentially in academia!
I don’t always trust “peer-reviewed” as a recommendation for an educational intervention that schools should use… I have to READ at least one of their “peer-reviewed” research papers - to actually confirm that the research is “evidence-based” and includes data on students’ performance that shows significant improvements are due to that intervention!
NEXT - I look at the research design & the variables reported to “control” for the impact of these variables related to and yet separate from the intervention target skills & knowledge? So, I mean variables that might be correlated with the dependent variable…
For example oral reading fluency is correlated with reading comprehension and so is vocabulary knowledge… So - I ask myself “Are these variables measured or controlled for in this paper about comprehension?”
I believe this logic about research design & control of potential variables is critical in peer review? MY Personal view!
As well, some “peer-review” academics think theoretical papers & survey research can be used to recommend educational interventions? Again, my personal view is that they cannot guide classroom teachers as to what might work for their students’ learning…
Back to AI and peer-review - I think this logical thinking process (my sort of thinking at academic level) needs to be applied, with examples, for students & teachers in schools!
I’m still thinking this through & your posts provide good directions for me!!
MANY THANKS yet AGAIN!! Best, Gail 👍👍😊
Ps that “longer responses” for AI is clearly a pattern, as previously discussed!
Thank you Gail, both for your kind words and also for a perfect example of critical thinking. 🙏
Thanks Sam!! 👍👍
Dear Sam, that’s a brilliant analysis of the ICLR situation!
You’re absolutely right: when we allow AI to 'accelerate' critical processes, we lose what matters most,human depth and accountability.
Your blog is a rare island of clarity in a world trying to automate even our thinking. I’m glad to see you’re still championing the 'slow-down' where the industry demands a blind sprint!!
Thank you so much Mila. For me this is definitely where the human in the loop is vital. 🙏
I sat for quite a few years on the (12 person) scientific committee of a UK national radiological society and part of our job was to select oral presentations for the annual conference. However, the submissions were abstracts with a maximum word count of about 300, which is typical for a medical conference. We had about 500 abstract submissions so filtering them down to about 200 (for a 3 day conference) was manageable by us, based on a scoring system. Each person would only be asked to score about 50 abstracts (within their specialised area), which was about 8-10 hours work.
I can't quite imagine how the 'analysis' system worked in this (ICLR 2026) case (how would AI score a submission?) but having 19,000 submissions and peer reviews that ran to 3000 words sounds truly bizarre. A conference that attracts 19,000 submissions is basically 'out of control' as you would need a small army of experts to sift that many. And using AI (glorified pattern recognition) for this sort of complex 'triage task' is a bit of a non-starter anyway.
Exactly this Philip. I have also been involved with conferences that accept around 21,000 abstracts, and there are definitely ways of doing it, but it becomes almost a full-time job in itself for the organising committee.
Absolutely Paul! I honestly think AI is just shining a light on the rapidly decaying infrastructure of academia in all its many guises. The optimist in me thinks that this is an amazing opportunity, but we have been here before…