Premiering pre-registration at PLOS Biology

peter.suber's bookmarks 2022-04-06


"Two years ago, PLOS Biology launched Pre-Registered Research Articles (PRAs; more widely known as Registered Reports) as a new article type [1, 2]. Fast-forward through two years of research, a pandemic and a baby (welcomed by two of the main authors), and this month we have published our first PRA [3]. We thought it would be a good moment to take stock and reflect on what the fuss is about and what lessons we have learned along the way.

As editors, we know all too well that seemingly very interesting work can fall flat on its face because the experiments do not conclusively justify the claims and this problem arises when it is too late to intervene. Unlike regular research articles, PRAs undergo peer review immediately after the study design stage and before experiments are conducted. If these so-called “Stage 1” PRAs are issued an accept-in-principle decision, then the research is performed and results and discussion are added to create a single, fully integrated “Stage 2” article. The Stage 2 is assessed on its adherence to the approved approach, the appropriateness of any deviations, and the accuracy of the conclusions. This approach aims to minimise bias towards beautiful, unequivocal (and often cherry-picked and irreproducible) results, while also maximising study quality. It focuses peer review on the conclusiveness and rigour of the proposed methodology, rather than the perceived “coolness” of the results, when there is still time to do something about it. Our launch of PRAs coincided with a more general change in how we editorially assess all submissions to PLOS Biology, with a focus on the importance of the research question being asked, the approach used and the rigor of the execution, reducing the emphasis on the results obtained [1]. Launching PRAs at this time was therefore a natural fit.

To date, we have received 48 PRA submissions, of which 23% have been peer-reviewed, a figure that is in line with our journal average. However, of the articles that we have peer-reviewed, 88% received an accept-in-principle decision. This proportion is substantially higher than our average of approximately 50% and suggests that assessing the work at an early stage decreases the publication bias that arises from judging the perceived advance of the results. All of the Stage 1 PRAs that we have accepted underwent a major revision that led to modifications in the proposed study design and/or analysis. Thus, external expert input at this stage of the process clearly has value and should lead to more robust articles, as was suggested by a recent analysis of registered report research quality [4]. Over the past two years, we have engaged with many reviewers and Academic Editors who were unfamiliar with the PRA format, which has been an interesting experience. Some people are confused at first, finding reviewers is generally more difficult and they tend to take more time to review a manuscript as they familiarise themselves with the concept. However, most have become supportive and are even excited about the possibilities that this format affords. The majority of our PRAs are assigned two Academic Editors; one who oversees compliance with the PRA criteria and process and one who has relevant expertise on the topic of the work—the PRA that we have published this month is unusual in that, owing to the topic, one Academic Editor was able to fulfil both roles...."


From feeds:

Open Access Tracking Project (OATP) » peter.suber's bookmarks

Tags: oa.preregistration oa.plos oa.journals

Date tagged:

04/06/2022, 09:43

Date published:

04/06/2022, 05:43