What Peer Review Actually Is (and Why It Matters)
When a researcher submits a paper to a journal, it typically gets sent to two or three experts in the field who assess whether the work is valid, well-reasoned, and worth publishing. This process is usually anonymous and unpaid. It’s called peer review, and for roughly 50 years, it’s been the standard gatekeeping mechanism for scientific knowledge.
It wasn’t always this way. Peer review only became universal in the 1970s — partly as a political move to legitimize public funding for science. The credibility it offered was the point, not necessarily the quality control.
That origin matters now, because the system was never designed to handle the volume it’s facing today.
The Numbers Behind the Pressure
The scale of the problem is significant:
- Papers indexed in major databases like Scopus and Web of Science are growing at roughly 5.6% per year — exponential growth.
- Researchers collectively spend an estimated 15,000 years of work on peer review annually.
- That unpaid labor would cost approximately $1.5 billion in the US alone if compensated.
- Some editors now contact 30 or more researchers just to find a single willing reviewer. Five years ago, five to ten contacts would yield three reviewers.
When editors can’t find qualified reviewers, they settle for less qualified ones. When reviewers are stretched thin, they read less carefully. The result is what happened to health economist Jason Semprini — his paper on HPV vaccine policy was rejected because a single reviewer misread the central argument. One reviewer, one misunderstanding, one rejection.
That’s not an edge case anymore. It’s increasingly the norm.
Why the Volume Keeps Growing
Several forces are feeding the publication explosion simultaneously.
Journals proliferated between 1960 and 2020, and many now run special issues that generate additional content on demand. Online-only publishing removed the physical constraint of fitting research onto printed pages. AI tools make it easier to write up research and easier for non-native English speakers to submit to English-language journals.
Then there are paper mills — services that charge researchers to put their names on fabricated but convincing-looking publications. These add noise to an already overwhelmed system.
Research is also more interdisciplinary than it used to be, which means individual papers require more expertise and time to review properly. The workload per paper has gone up at the same time the pool of willing reviewers has shrunk.
How AI Is Entering the Review Process
More than half of peer reviewers already use AI in some part of the review process, according to a survey by Frontiers journals. That’s happening faster than any formal policy can keep up with.
Some uses are benign — using AI to help understand an unfamiliar reference or check grammar. But the more problematic use is reviewers generating entire reviews with AI tools and submitting them as their own work.
Linguist Marijn van Putten spent two days trying to track down a medieval Arabic manuscript a reviewer had recommended he cite — only to conclude the reference had been hallucinated by an AI. The review contained other telltale signs: excessive em dashes, generic phrasing, a certain hollow confidence. An AI detection tool confirmed his suspicion.
Beyond wasted time, there’s a confidentiality issue. Uploading an unpublished manuscript to a commercial AI chatbot that retains data could expose research before it’s published — a serious breach of academic ethics.
What Responsible AI Use Could Look Like
The AI conference NeurIPS is trialing a more structured approach: a custom AI tool that helps reviewers understand submissions and do background research, but explicitly does not write the review itself. The distinction matters. AI as a research aid is different from AI as a replacement for expert judgment.
Locally run language models or services with strict data retention policies could address the confidentiality problem — but they require the same reviewers who are already cutting corners to exercise additional care. That’s a tension without an easy resolution.
Preprints
Researchers increasingly post papers to platforms like arXiv before formal peer review is complete. This makes findings visible immediately and allows the field to react in real time. In some disciplines, preprints are now accepted as legitimate credentials for jobs and grants — an acknowledgment that waiting for formal publication can stall careers for no good reason.
Decoupled Peer Review
Some organizations are experimenting with separating peer review from journal submission entirely. A paper gets reviewed once by a community of researchers, and authors can then submit to multiple journals using that single review. If the first journal passes, the authors move to the next without starting over. This reduces duplicated effort significantly.
Paying Reviewers
There’s evidence that compensation in the $100–$300 range per review speeds up turnaround without hurting quality. Some prominent economics journals already do this. The counterargument is that it adds to the publication fees researchers already pay — but the current unpaid model is clearly not sustainable.
The Blog Experiment in AI Research
AI researchers are arguably furthest along in abandoning traditional publishing altogether. Submissions to top AI conferences have increased between two- and ten-fold since 2019. Many researchers in the field don’t need formal publications for career advancement because much of the work happens outside academia.
Blogs have filled the gap. Platforms like the AI Alignment Forum use upvoting, downvoting, and community commentary instead of anonymous peer review. Posts get far more reactions than the handful of reviews a journal would generate, and the system surfaces counter-arguments alongside the original work.
The tradeoffs are real. Accessibility can let charismatic voices rise regardless of rigor. Work published outside peer-reviewed venues is harder to cite in traditional journals. And there’s no guarantee that an upvote reflects careful reading.
But the speed advantage is hard to argue with. In a field moving as fast as AI, a paper that takes 18 months to clear peer review may be obsolete before it’s published.
What’s Actually Likely to Change
A wholesale collapse of peer review is unlikely. The system is too embedded in how careers are built and how public trust in science is maintained. But the version of peer review that survives will probably look different from what exists today.
Some things are already shifting:
- The average number of reviewers per paper in immunology has dropped from three to closer to two.
- Preprints are now standard in many fields.
- AI tools are entering the process whether journals sanction them or not.
- Some fields are quietly accepting that not everything needs the same level of scrutiny.
The more likely outcome is fragmentation — peer review meaning different things in different fields, with varying levels of rigor, different tools, and different timelines.
The Practical Takeaway
If you’re a researcher, the system you’re navigating is under real pressure, and the shortcuts being taken around you are creating real risks for your own work. Understanding which alternatives — preprints, decoupled review, community platforms — are accepted in your field is worth knowing now, not after a frustrating rejection.
If you’re building tools for researchers or academic publishers, the gap between what the current system can handle and what it’s being asked to do is wide and growing. The demand for better solutions is genuine.
And if you’re a reader of scientific research, it’s worth knowing that “peer reviewed” covers a wider range of scrutiny than it once did — and that the label alone is no longer a guarantee of careful evaluation.
The bedrock is still there. But it needs reinforcement.
Comments (0) No comments yet
Want to join this discussion? Login or Register.
No comments yet. Be the first to share your thoughts!