“I mean, what exact buttons do I have to hit?”

This American Life reporter Gabriel Rhodes says:

This is one of the big differences between Jon and Anthony, between scientist and non-scientist. For Jon, having a year’s worth of work suddenly thrown into question is a normal day at the office. But for Anthony, that’s not normal. And it’s not OK. The time in Jon’s lab was a year of his life, where he felt like Jon kept moving the goal posts. . . . But now, Anthony wants to know, before he starts turning his life upside down again, what will count as proof enough for Jon? How many experiments?

Anthony Holland: So let’s say I do three weeks of experiment, and I only concentrate on these leukemia cells. And if I can kill at least 20% every single time, every week, will that do it? Would that be enough? Or do you want to see pancreatic die, or do you want to see—I mean, what exact buttons do I have to hit?

This captures a big problem with the research enterprise, as I see it. There’s this attitude that if you can reach some threshold, whether it be statistical significance or 80% power or whatever, that your work should be published/funded and should be taken as correct. People really don’t want to live with uncertainty.

The post “I mean, what exact buttons do I have to hit?” appeared first on Statistical Modeling, Causal Inference, and Social Science.