More trial, less error: An effort to improve scientific studies | Reuters

“A year-old Palo Alto, California, company, Science Exchange, announced on Tuesday its ‘Reproducibility Initiative,’ aimed at improving the trustworthiness of published papers. Scientists who want to validate their findings will be able to apply to the initiative, which will choose a lab to redo the study and determine whether the results match. The project sprang from the growing realization that the scientific literature - from social psychology to basic cancer biology - is riddled with false findings and erroneous conclusions, raising questions about whether such studies can be trusted... In March, Lee Ellis of M.D. Anderson Cancer Center and C. Glenn Begley, the former head of global cancer research at Amgen, reported that when the company's scientists tried to replicate 53 prominent studies in basic cancer biology, hoping to build on them for drug discovery, they were able to confirm the results of only six... The new initiative, said Begley, senior vice president of privately held biotechnology company TetraLogic, ‘recognizes that the problem of non-reproducibility exists and is taking the right steps to address it.’ The initiative's 10-member board of prominent scientists will match investigators with a lab qualified to test their results, said Elizabeth Iorns, Science Exchange's co-founder and chief executive officer. The original lab would pay the second for its work. How much depends on the experiment's complexity and the cost of study materials, but should not exceed 20 percent of the original research study's costs. Iorns hopes government and private funding agencies will eventually fund replication to improve the integrity of scientific literature. The two labs would jointly write a paper, to be published in the journal PLoS One, describing the outcome. Science Exchange will issue a certificate if the original result is confirmed. It may not be obvious why scientists would subject their work to a test that might overturn its results, and pay for the privilege, but Iorns is optimistic. ‘It would show you are a high-quality lab generating reproducible data,’ he said. ‘Funders will look at that and be more likely to support you in the future.’ If results are reproduced, ‘it will increase the value of any technology the researcher might try to license,’ she said, adding that it would also provide assurance to, say, a pharmaceutical company that the result is sound and might lead to a new drug...” Typically, scientists must show that results have only a 5 percent chance of having occurred randomly. By that measure, one in 20 studies will make a claim about reality that actually occurred by chance alone, said John Ioannidis of Stanford University, who has long criticized the profusion of false results. With some 1.5 million scientific studies published each year, by chance alone some 75,000 are probably wrong. In addition, Ioannidis said, ‘people start playing with how they handle missing data, outliers, and other statistics,’ which can make a result look real when it's not. ‘People are willing to cut corners’ to get published in a top journal, he said...”