What’s Next for Open Science - Making the Case for Open Methods - The Scholarly Kitchen

"But data alone is not enough, and an enormous hole in the open science movement has been the lagging attention paid to the reporting of research methodologies. Being able to review the data behind a study does indeed allow one to see if a researcher’s analysis and the conclusions drawn are accurate for that dataset. But it does little to validate the quality and accuracy of the dataset itself. If I don’t know how you got that data, I have no idea if it’s any good, and I certainly don’t stand any chance of replicating it.

A big problem here is that the scant information offered by most journals’ Materials and Methods sections is insufficient to have any chance of repeating what the original authors did. Often when describing a technique, an author will merely cite a previous paper where they used that technique…which also cites a previous paper, which also cites a previous paper and the wild goose chase is on. This lack of detailed methodology reporting is something of an anachronism, driven by decades of a print-dominant publication model aimed at reducing the number of pages in journal issues, along with a lack of incentives to improve methods reporting.

As open data requires the public availability of the data behind any published research conclusions, so open methods would require the public availability of detailed documentation of the procedures used to gather and analyze those data. Like open data, this can happen through a variety of routes — publication of the method as a standalone paper cited by the research paper, detailed documentation of the methods used in the paper itself (or its supplementary materials), or citation of a deposited documentation of the method in a repository such as protocols.io...."