Estimating the reproducibility of psychological science | Science

peter.suber's bookmarks 2018-02-21

Summary:

The co-authors of this work call themselves the Open Science Collaboration.

"No single indicator sufficiently describes replication success, and the five indicators examined here are not the only ways to evaluate reproducibility. Nonetheless, collectively these results offer a clear conclusion: A large portion of replications produced weaker evidence for the original findings despite using materials provided by the original authors, review in advance for methodological fidelity, and high statistical power to detect the original effect sizes. Moreover, correlational evidence is consistent with the conclusion that variation in the strength of initial evidence (such as original P value) was more predictive of replication success than variation in the characteristics of the teams conducting the research (such as experience and expertise). The latter factors certainly can influence replication success, but they did not appear to do so here.

 

Reproducibility is not well understood because the incentives for individual scientists prioritize novelty over replication. Innovation is the engine of discovery and is vital for a productive, effective scientific enterprise. However, innovative ideas become old news fast. Journal reviewers and editors may dismiss a new test of a published idea as unoriginal. The claim that “we already know this” belies the uncertainty of scientific evidence. Innovation points out paths that are possible; replication points out paths that are likely; progress relies on both. Replication can increase certainty when findings are reproduced and promote innovation when they are not. This project provides accumulating evidence for many findings in psychological research and suggests that there is still more work to do to verify whether we know what we think we know...."

Link:

http://science.sciencemag.org/content/349/6251/aac4716

From feeds:

Open Access Tracking Project (OATP) » peter.suber's bookmarks

Tags:

oa.reproducibility oa.psychology oa.open_science oa.ssh

Date tagged:

02/21/2018, 11:10

Date published:

02/21/2018, 06:10