The Reproducibility Initiative: A Good Idea in Theory that Won’t Work in Practice | Xconomy

"The failure of scientists to independently confirm much of the data contained in 'hot' academic publications is casting a long shadow over the biopharmaceutical industry. Research groups at Amgen and Bayer reported that the data in a significant percentage of published “breakthrough” papers from academic scientists could not be confirmed in their labs. Given that Big Pharma has increasingly turned to academic investigators as a source of molecular targets for new drugs, this represents a Big Problem. I have argued that this lack of experimental reproducibility represents an especially acute problem for virtual biotech companies, who lean on contractors to do most of their R&D, and have no internal lab facilities in which they can try to replicate the data. A new proposal, the Reproducibility Initiative, has recently been established to create a pathway for verifying experimental data outside of the lab that generated it. Science Exchange, a for-profit online marketplace of laboratory services, is coordinating this new initiative in partnership with the open access journal PLoS ONE. The basic concept behind the Science Exchange marketplace is that it enables scientists to hire service providers to perform experiments that are beyond the capabilities of their own labs. The Reproducibility Initiative is layered on top of this marketplace, with the goal of addressing this irreproducibility problem. Researchers sign up with the Reproducibility Initiative to have their work replicated. An advisory panel finds an outside lab group to perform the studies (which are done anonymously), and the results are then shared with the investigators. Researchers who wish to avail themselves of the Initiative must pay for the confirmation work to be done. In addition, they must also pay a 5 percent transaction fee to Science Exchange for tapping into their network. While the goal of this new initiative is admirable (and likely profitable for the Science Exchange), I don’t think the approach will work in the real world. Here’s why this initiative will not pan out, despite its good intentions..."