Evidence of things not seen

A couple of studies out recently look at coffee and health.  One from Harvard(and reported by CNBC)  says coffee (but not decaf or tea) increases the chance of healthy aging in women. Another, from Tufts, (and reported by Newsweek and The Independent) says that unsweetened black coffee, or coffee with very small amounts of sugar or normal coffee amounts of milk, reduces death rates slightly, but not coffee with more sugar or milk (as in everything from a Kiwi small flat white to American-style lattes)

There are two problems with these studies.  The first is that I can’t see them.  One is an abstract from a conference presentation; the other is a paper in an academic journal, but not one the University of Auckland gives me access to.

Compounding this, the two abstracts only give information for their preferred beverages. It’s not possible to tell whether “Decaffeinated coffee and tea intake were not significantly associated with odds of HA nor any domains” means that there’s evidence the correlations were different for tea and decaf or whether there was just a bit more uncertainty around plausibly the same correlation.  Similarly, “However, the mortality benefits were restricted to black coffee [HR (95% CI): 0.86 (0.77, 0.97)] and coffee with low added sugar and saturated fat content [HR (95% CI): 0.86 (0.75, 0.99)]” doesn’t tell us what they found for other coffee types. Nor is the information in the press releases I could find. Since the difference between ways of drinking coffee was the main news tag for these studies, that’s a bit unsatisfactory.

I’ll also note that the Tufts team published an abstract in 2020 with a slightly smaller version of the data from the same survey series, and concluded “Adding milk/cream, alone or with sugar/sweetener, did not significantly change the results.”

A basic principle for studies like these is that conclusions about difference require evidence of difference.  This applies to conclusions in the paper, and even more so to conclusions you want the press to report.

The post Evidence of things not seen first appeared on Stats Chat.