bitter carrots* | sometimes i'm wrong - simine vazire
ab1630's bookmarks 2018-05-14
"if you had told me five years ago that even one in twenty social/personality psych papers would provide links to their data and code, or to a pre-registration, i would have thought that would be huge progress.** i've long been a fan of the nudges that encourage these kinds of practices (e.g., badges), and until recently i thought going as far as to require this kind of transparency (even with room for legitimate exceptions) was probably unrealistic - our field didn't seem ready for that. i was sympathetic to the carrots-not-sticks approach. but there's a problem with carrots-not-sticks. we're asking researchers to eat the carrots, but some of the carrots are pretty bitter. sometimes, when researchers are transparent, that brings information to light that undermines their claims, and readers don't buy the claims. that's a necessary side effect of transparency. and it means we can't in good faith tell researchers that transparency is always in their best interest and will be its own reward. we can't lure people with carrots, and pretend all of the carrots are delicious and fun to eat. sometimes carrots are hard to swallow. i think it's time to admit that the main argument for transparency isn't self-interest - it's that transparency is just better for science.*** imagine the following scenarios: scenario 1: you get a paper to review, and the authors have shared their data and code. you look at the data and realize there is a coding error, or something else that makes the results uninterpretable (i.e., suggests the study needs to be re-run to fix the error). you point this out in your review, the editor agrees and rejects the manuscript....