Harvard study finds AI peer review outperforms human experts

A recent study from Harvard University researchers indicates that artificial intelligence can outperform human experts in academic peer review. The study compared 134 sets of AI-generated and human-written review comments, finding that AI led across seven evaluation dimensions, including identifying research weaknesses, suggesting references, and evaluating methodology.

In blind tests conducted by three former academic journal editors, the AI was preferred in 26 out of 30 comparisons. Furthermore, the AI demonstrated a lower factual error rate of 5.5%, compared to 15.9% for human reviewers. The research utilized Claude Opus 4.8 to generate reviews and GPT-5.5 to act as a judge.

While the findings suggest a potential shift in academic publishing, experts note challenges. Some researchers argue that AI may become overly pedantic, while others emphasize that the current peer review system faces a 'prisoner's dilemma' due to the surge in AI-assisted paper submissions, which places an unsustainable burden on human reviewers.