Jury Theorems for Peer Review | The British Journal for the Philosophy of Science: Vol 0, No ja

Abstract:  Peer review is often taken to be the main form of quality control on academic writings. Usually this is carried out by journals. Parts of math and physics appear to have now set up a parallel, crowd-sourced model of peer review, where papers are posted on the arXiv to be publicly discussed. In this paper we argue that crowd-sourced peer review is likely to do better than journal-solicited peer review at sorting papers by quality. Our argument rests on two key claims. First, crowd-sourced peer review will lead to there being on average more reviewers per paper than journal-solicited peer review. Second, due to the wisdom of the crowds, more reviewers will tend to make better judgments than fewer. We make the second claim precise by looking at the Condorcet Jury Theorem as well as two related, novel jury theorems developed specifically to apply to the case of peer review.