Advice for science writers!

I spoke today at a meeting of science journalists, in a session organized by Betsy Mason, also featuring Kristin Sainani, Christie Aschwanden, and Tom Siegfried.

My talk was on statistical paradoxes of science and science journalism, and I mentioned the Ted Talk paradox, Who watches the watchmen, the Eureka bias, the “What does not kill my statistical significance makes it stronger” fallacy, the unbiasedness fallacy, selection bias in what gets reported, the Australia hypothesis, and how we can do better.

Sainani gave some examples illustrating that journalists with no particular statistical or subject-matter expertise should be able to see through some of the claims made in published papers, where scientists misinterpret their own data or go far beyond what was implied by their data. Aschwanden and Siegfried talked about the confusions surrounding p-values and recommended that reporters pretty much forget about those magic numbers and instead focus on the substantive claims being made in any study.

After the session there was time for a few questions, and one person stood up and said he worked for a university, he wanted to avoid writing up too many stories that were wrong, but he was too busy to do statistical investigations on his own. What should he do?

Mason replied that he should contact the authors of the studies and push them to explain their results without jargon, answering questions as necessary to make the studies clear. She said that if an author refuses to answer such questions, or seems to be deflecting rather than addressing criticism, that this itself is a bad sign.

I expressed agreement with Mason and said that, in my experience, university researchers are willing and eager to talk with reporters and public relations specialists, and we’ll explain our research at interminable length to anyone who will listen.

So I recommended to the reporter that, when he sees a report of an interesting study, that he contact the authors and push them with hard questions: not just “Can you elaborate on the importance of this result?” but also “How might this result be criticized?”, “What’s the shakiest thing you’re claiming?”, “Who are the people who won’t be convinced by this paper?”, etc. Ask these questions in a polite way, not in any attempt to shoot the study down—your job, after all, is to promote this sort of work—but rather in the spirit of fuller understanding of the study.

The post Advice for science writers! appeared first on Statistical Modeling, Causal Inference, and Social Science.