Unintentional parody of Psychological Science-style research redeemed by Dan Kahan insight

Cat-owner Kahan’s no Freud expert but he spied the above piece of work and sent it to me along with the comment:

I can pretty much tell (meaning I put odds at 9.7:1; obviously I’ll revise when I read) that this is a disaster b/c it tries to tell me what the inferences are and why they are sound by adverting to statistical tests. Inferences come from the validity of the design—what observations were determined to be relevant to the question of interest & what those observations were. Then come the statistics to discipline & extend the infrerences. There’s a style of analysis that thinks that inferences are like neutrinos or dark matter to be extracted w/ elaborate detection devices that will catch a glimpse of these otherwise elusive & fleeting streaks of insight into how the world “really” works… wrong wron gwrong wrong wrong.

Gwrong indeed. Kahan’s remark about neutrinos and dark matter seems to me to capture something real in how these researchers see science, and it fits their fear of their statistics being audited or their claims being checked by replication.

Anyway, nobody cares what gets published in the Journal of Retailing and Consumer Services. But it was all worth it to get this insight from Kahan.

P.S. Just to be clear: the purpose of this post is to highlight Kahan’s interesting statement criticizing the indirect nature of the sort of research shown in the above abstract, the idea that instead of just directly studying something, they’ll go around rejecting null hypotheses that they don’t believe. That’s what’s “gwrong” with the cited work. It might well be that the substantive claims made in that paper are correct. I’m not criticizing the Seo and Fiore’s paper or saying they are wrong regarding the effects of fitting rooms. I’m saying that I agree with Kahan that this sort of reasoning-by-refuting-null-hypothesis is not a good way to do science, even if it can work in some settings.

The post Unintentional parody of Psychological Science-style research redeemed by Dan Kahan insight appeared first on Statistical Modeling, Causal Inference, and Social Science.