The Future of Peer Review - The Scholarly Kitchen

"Based on last November’s SpotOn London conference (which, sadly, I couldn’t attend myself), the report makes seven recommendations for the research community in the coming years:

1. Find new ways of matching expertise and reviews by better identifying, verifying and inviting peer reviewers (including using AI)
2. Increase diversity in the reviewer pool (including early career researchers, researchers from different regions, and women)
3. Experiment with different and new models of peer review, particularly those that increase transparency
4. Invest in reviewer training programs
5. Find cross-publisher solutions to improve efficiency and benefit all stakeholders, such as portable peer review
6. Improve recognition for review by funders, institutions, and publishers
7. Use technology to support and enhance the peer review process, including automation

[...]

Certainly, if we put machines in charge of even some of the peer review process itself, transparency would be even more essential than it is today. I can’t imagine that many of us would like to be in a Google-like situation of not knowing anything about the algorithms that are being used to determine the quality of a paper, nor being informed when, why, how, and by whom those algorithms were created and get updated.

Last but not least, greater transparency can, of course, be of great value in increasing diversity. Studies like this one by Jory Lerback and Brooks Hanson provide evidence of bias in the review process — in this case, showing that 'women were used less as reviewers than expected…The bias is a result of authors and editors, especially male ones, suggesting women as reviewers less often, and a slightly higher decline rate among women in each age group when asked.' Other studies have shown that, for example, Chinese researchers are far less likely and US researchers far more likely to be reviewers. These kinds of studies don’t just shed light on a long-suspected problem in peer review, they also provide a solution to a well-known problem: the need for a larger pool of qualified peer reviewers."