BITSS Preprints | Why We Need Open Policy Analysis

Abstract:  The evidence-based policy movement promotes the use of empirical evidence to inform policy decision-making. While this movement has gained traction over the last two decades, several concerns about the credibility of empirical research have been identified in scientific disciplines that use research methods and practices that are commonplace in policy analysis. As a solution, we argue that policy analysis should adopt the transparent, open, and reproducible research practices espoused in related disciplines. We first discuss the importance of evidence-based policy in an era of increasing disagreement about facts, analysis, and expertise. We then review recent credibility crises of empirical research (difficulties reproducing results), their causes (questionable research practices such as publication biases and p-hacking), and their relevance to the credibility of evidence-based policy (trust in policy analysis). The remainder of the paper makes the case for "open" policy analysis and how to achieve it. We include examples of recent policy analyses that have incorporated open research practices such as transparent reporting, open data, and code sharing. We conclude with recommendations on how key stakeholders in evidence-based policy can make open policy analysis the norm and thus safeguard trust in using empirical evidence to inform important policy decisions.