Credit where due to NPR regarding science data fraud, and here’s how they can do even better

We associate NPR with uncritical promotion of junk science (etc etc etc) and the scientist-as-hero narrative (which I absolutely hate).

So we should give them credit when they do the right thing and introduce some skepticism into their reporting. A couple people pointed me to NPR story, Fabricated data in research about honesty. You can’t make this stuff up. Or, can you?, which featured these bits:

GUO: The company told us in a statement that they’d pulled the original data set they sent to Dan Ariely, and it looked dramatically different from the published data from Ariely’s experiment. The company said they’d only given Ariely data for about 3,700 insurance policies. But in Ariely’s paper, he claimed he had data for more than 13,000 policies. That’s a difference of almost 10,000.

FOUNTAIN: In their statement, the company told us, quote, “though some of the data in the published study data is originally sourced from our data, it is clear that the data was manipulated inappropriately and supplemented by synthesized or fabricated data.”

GUO: The company basically confirmed everything that Uri and the Data Colada people had said about Ariely’s numbers. And the company made it clear that whatever had gone wrong with the data set had gone wrong after they had already given it to Ariely.

FOUNTAIN: We shared parts of the insurance company statement with Ariely, and he responded in an emailed statement, quote, “as I said two years ago, I was responsible for the relationship with the insurance company that provided the data for the paper. I got the data file from the insurance company in about 2007, and I can’t tell now who had access to it. Getting the data file was the extent of my involvement with the data.”

FOUNTAIN: But Michael says the problem with all these fake studies is bigger than the feelings of a bunch of academics like him. In Guatemala, there was this big investment. Hundreds of thousands of people were put through this experiment that was never going to work.

SANDERS: All of that time and money could have been spent on something different, a different intervention with a better chance of working if we’d known, right? This is not a petty, academic squabble about he said, she said. This is – like, this has real impact in the real world. So I’m pissed off about that. I’m also pissed off by how stupid it is. So when you see, like, eminent Harvard professor has committed research fraud, what you want is really sophisticated, clever ways of cheating, not, oh, I thought I’d copy and paste these observations ’cause they were higher, and that’ll make my result come together. This is dumb. And that annoys me. But it’s also a source of terror because that means there’ll be people out there who have cheated in a really sophisticated way who we haven’t caught and who we may never catch.

I agree with my correspondent that it’s good that they discuss the fraud and also the consequences of it.

Just one thing . . .

Two things, actually.

First, it’s great to see NPR doing some critical reporting on science hype. I’d also like to see them go back and address their past credulity in this area.

I think that one reason for the spectacular success of junk science in recent decades has been its promotion and endorsement by trusted news outlets such as NPR. So, once they realize they’ve been snookered, it would be good for them to address that: to forthrightly say in their story that, yes, they, NPR, had been promoting this junk science for years, and to consider what that implies about their science reporting more generally.

We can learn a lot from our mistakes, but only if we first face up to them.

Second, yes, science fraud is horrible, but there’s tons and tons of junk science that’s not fraud, it’s just bad work, in the sense of making strong claims that are not supported by the data. Himmicanes, air rage, ages ending in 9, etc etc etc. Some of this work (for example, the claim that beautiful parents are 36% more likely to have girls, or the claim that women were 20% were more likely to support Barack Obama during certain times of the month) were ridiculous on their face; other claims were internally inconsistent (for example, the paper with the voodoo dolls that described a three-day study as “long-term”); others had some surface plausibility but disintegrated under careful investigation.

My point is: Yes, let’s shine the light on science fraud. But don’t think that, just cos some highly promoted bit research is not fraudulent, that it should be taken at face value.

You’ve taken the first step, NPR. Don’t stop there!