Will Facebook's New Research Initiative Make The Replication Crisis Worse?
ab1630's bookmarks 2018-08-13
"The era of “big data” has transformed our understanding of the human world, making it possible for researchers to study billions of users at once, while at the same time making it impossible for other researchers to replicate their work. The ever-larger datasets that increasingly define modern quantitative social science are controlled by an ever-shrinking number of researchers that have exclusive access to the digital riches of our modern world. Facebook’s new academic research initiative with Social Science One was supposed to fix all of this, granting researchers across the world access to the private data of Facebook’s two billion users to mine, but the unanswered question is how initiatives like Social Science One will address the replication crisis. The growing “replication crisis” in academia refers to the inability to reproduce the findings of published papers either due to access restrictions that prevent others from being able to examine the underlying data or due to the data and methods outlined in the paper not producing the results described by the researchers. Solving the replication crisis requires that researchers commit to making the data that underlies their studies available to other academics where legally and ethically possible. This can be as simple as posting a giant ZIP file to their faculty website or as sophisticated as providing an advanced interactive data mart capable of customized extracts and interactive online exploration and analysis. Yet, in spite of all of the calls for researchers to make their data more open, few scholars actually release the datasets undergirding their work, preferring to keep the data confidential so they can exclusively publish with it. Others might not want to spend the effort required to document and box up the data to make it available or may not want to risk that other researchers could spot errors in their data and call their findings into question...."