Open science and trustworthy data | The Psychologist

abernard102@gmail.com 2015-12-12

Summary:

"In our letter (November 2015), we urged the Society’s boards and senior committees to respond to the very serious problems of replicating psychological research that were revealed by the meagre 36 per cent success rate of the Reproducibility Project’s report of 100 attempted replications. In reply, Professor Andy Tolmie commented that ‘low n research may be a more endemic part of the problem than any deliberate attempts at massaging data’. However, low ns were not the problem for the Reproducibility Project because a priori power analyses for the replications indicated that a 92 per cent replication rate was predicted based on the originally reported effect sizes. The Project’s report (Open Science Collaboration, 2015) noted that the best predictor of replication success was the effect size observed in the replication, which is independent of sample size. Sadly, the average effect size for the replications was less than half of that for the original studies. The report described the original studies as having ‘upwardly biased effect sizes’. It seems likely that the psychology literature reflects questionable research practices that can inflate effect sizes, such as: p-hacking, unreported removal of troublesome data, and capitalising on chance through selective publishing after adjusting a paradigm to produce significant results or reporting a ‘successful’ dependent variable but not those showing smaller effects. One issue that we raised in our letter was the temptation faced by junior researchers to further their careers by removing, adjusting or inventing data. The rewards for such data manipulation can be considerable, while the dangers of discovery under present systems are very small. A recent case provides an illustration ..."

Link:

http://thepsychologist.bps.org.uk/volume-29/january-2016/open-science-and-trustworthy-data

From feeds:

Open Access Tracking Project (OATP) » abernard102@gmail.com

Tags:

oa.new oa.comment oa.data oa.open_science oa.quality oa.credibility oa.reproducibility oa.psychology oa.ssh

Date tagged:

12/12/2015, 20:12

Date published:

12/12/2015, 15:12