SocArXiv Papers | The limitations to our understanding of peer review
flavoursofopenscience's bookmarks 2019-09-02
Summary:
Authors
Submitted on
August 29, 2019
Last edited
August 30, 2019
Supplemental Materials
Abstract:
Peer review is embedded in the core of our scholarly knowledge generation systems, conferring legitimacy on research while distributing academic capital and prestige on individuals. Despite its critical importance, it curiously remains poorly understood in a number of dimensions. In order to address this, we have programmatically analysed peer review to assess where the major gaps in our theoretical and empirical understanding of it lie. We distill this into core themes around editorial accountability, the subjectivity and bias of reviewers, the function and quality of peer review, the role of the reviewer, the social and epistemic implications of peer review, and regarding innovations in open peer review platforms and services. We use this to present a guide for the future of peer review, and the development of a new research discipline based on the study of peer review. Such a field requires sustained funding and commitment from publishers and research funders, who both have a commitment to uphold the integrity of the published scholarly record. This will require the design of a consensus for a minimal set of standards for what constitutes peer review, and the development of a shared data infrastructure to support this. We recognise that many of the criticisms attributed to peer review might reflect wider issues within academia and wider society, and future care will be required in order to carefully demarcate and address these.