How did the press do on that “black spatula” story? Not so great.
Statistical Modeling, Causal Inference, and Social Science 2024-12-19
Act 1: The journal Chemosphere publishes a research article finding that black plastic kitchen items contain dangerous toxins arising from reprocessing of plastics that contained carcinogenic flame retardants. This result is publicized by the advocacy group that helped conduct the study, and it is picked up in the major media, with headlines such as, “Black-colored plastic used for kitchen utensils and toys linked to banned toxic flame retardants,” as CNN reported from 1 Oct. Various news organizations pick up the story and give recommendations about not reusing plastic containers, etc.
Act 2: It turns out that the journal article had a factor-of-10 error, where a given exposure was stated to be 80% of the legal limit for a certain toxin, but it was really only 8%.
We wrote about this the other day, citing, linking, and quoting from an article of 11 Dec from Canada’s National Post newspaper, that reported:
Plastics rarely make news like this . . . the media uptake was enthusiastic on a paper published in October in the peer-reviewed journal Chemosphere.
“Your cool black kitchenware could be slowly poisoning you, study says. Here’s what to do,” said the LA Times. “Yes, throw out your black spatula,” said the San Francisco Chronicle. Salon was most blunt: “Your favorite spatula could kill you,” it said.
The study, by researchers at the advocacy group Toxic-Free Future . . . estimated that using contaminated kitchenware could cause a median intake of 34,700 nanograms per day of Decabromodiphenyl ether, known as BDE-209. . . . The trouble is that, in the study’s section on “Health and Exposure Concerns,” the researchers said this number, 34,700, “would approach” the reference dose given by the United States Environmental Protection Agency. . . .
The paper correctly gives the reference dose for BDE-209 as 7,000 nanograms per kilogram of body weight per day, but calculates this into a limit for a 60-kilogram adult of 42,000 nanograms per day. . . . But 60 times 7,000 is not 42,000. It is 420,000. This is what [McGill University’s] Joe Schwarcz noticed. . . .
The article goes on to interview Schwarcz about the spatula claim and general issues of summarizing uncertainty.
The starting point of Act 2 of our story seems to have been this press release by Schwarcz from 6 Dec on the McGill University website, entitled, “Are Black Plastic Spatulas and Serving Spoons Safe to Use? For me, the risk would not be enough to discard them, but science should find a way to keep flame retardants out of such items.”
It’s a complicated problem. Indeed, only just now I’m realizing that “median intake” is a strange thing to look at when summarizing a public health risk.
Act 3: The error was corrected in the journal and in a press release by the advocacy organization, but in a minimizing way: “it is important to note that this does not impact our results.” This was mentioned in the above-linked National Post article.
Act 4: Several news organizations followed up on the correction.
And that’s the subject of today’s post. How did the news organizations do?
I’d say their record is mixed. I’ll go through them in the order they came up in my Google searches:
Here’s SF Gate, a website that was formerly part of the San Francisco Chronicle:
Did you read that black spatulas were poisoning you and everyone you love and then throw away multiple kitchen utensils in a blind panic? . . . Publications ranging from the Atlantic (“Throw Out Your Black Plastic Spatula”) to the Los Angeles Times (“Your cool black kitchenware could be slowly poisoning you, study says”) to Salon.com (“Your favorite spatula could kill you”) all spread the panic. . . . But then Joe Schwarcz, director of McGill University’s Office for Science and Society, noticed a mistake in the study. . . .
The SF Gate article, which is dated 17 Dec, has some strong similarities to the National Post article–I’m not saying it’s plagiarized, exactly, more that they seem to have read that earlier news article and rewritten it. The good news is they kept the credit to Schwarcz; the bad news is . . . well, let’s see if you notice. Read carefully:
From the National Post: “‘Your cool black kitchenware could be slowly poisoning you, study says. Here’s what to do,’ said the LA Times. ‘Yes, throw out your black spatula,’ said the San Francisco Chronicle. Salon was most blunt: ‘Your favorite spatula could kill you,’ it said.”
From SF Gate: “Publications ranging from the Atlantic (‘Throw Out Your Black Plastic Spatula’) to the Los Angeles Times (‘Your cool black kitchenware could be slowly poisoning you, study says’) to Salon.com (‘Your favorite spatula could kill you’) all spread the panic.”
SF Gate kept the LA Times and Salon.com, they added the Atlantic, but they removed the San Francisco Chronicle. Kind of funny to get rid of the only local angle, no? Especially given that the SF Gate article has the title, “Turns out everyone in SF didn’t need to throw away their black spatulas.”
I can’t figure out why SF Gate didn’t mention the San Francisco Chronicle. Here are two, somewhat contradictory, guesses:
1. SF Gate and the Chronicle are competitors and so, wherever possible, they will avoid linking to or mentioning each other.
2. SF Gate and the Chronicle are sister organizations and so SF Gate removed any mention of the Chronicle in their article so as not to cause any embarrassment to that newspaper.
I don’t know which it is; either way, sure, it’s not the world’s biggest scandal but it’s kind of uncool.
In any case, SF Gate did not credit the National Post at all, which seems really bad given that it looks like they were just rewriting that National Post story from a week earlier. Again, I’m not saying it was plagiarism; it’s just bad practice to not credit an earlier source.
Google also turns up an article, “Viral study about black plastic spatulas had a big math problem,” from the San Francisco Chronicle itself. This one also cites Schwarcz, but not the National Post, even though, again, it seems to be a recycling (ha ha, geddit?) of that earlier news article.
On the plus side, the Chronicle does explicitly say that they had covered the earlier story, and they link to that story, although without reproducing their incendiary (ha ha, geddit?) headline, “Yes, throw out your black spatula.”
Here’s the New York Times, my hometown newspaper, with an article entitled, “Do I Really Need to Throw Out My Black Plastic Spatula? A new study detected dangerous chemicals in a variety of household items. But experts say the health risks aren’t clear-cut.” In true Gray Lady fashion, they’ve worked to make their headline and sub-headline as boring as possible—and they’ve pretty much succeeded!
This Times article does the worst job of reporting we’ve seen so far! In some ways it’s good: it takes a measured tone and includes interviews with multiple experts. But . . . the article was published 10 Dec and updated 17 Dec, and it doesn’t mention the factor-of-10 problem at all! Even though Schwarcz’s press release appeared on 6 Dec. OK, fair enough–the reporter didn’t go to the trouble of a careful internet search that could’ve found the press release, and as of 10 Dec, when the NYT article was first published, the National Post article hadn’t yet appeared. It was really the National Post that broke the story. But it says right there on the Times article that it was updated on 17 Dec, indeed that it appeared in the physical newspaper on that day. By then, even a cursory internet search on some variant of *black plastic study* would’ve revealed the problem.
Back on 14 Nov, the Times had run an article, “Black Plastic Kitchen Tools Might Expose You to Toxic Chemicals. Here’s What to Use Instead.” They didn’t know about the factor-of-10 error, and they pretty much buy the “Toxic-Free Future” alarmist pitch straight-up, no skepticism. On the plus side, unlike all the other sources, this Times article has some useful suggestions, as they discuss metal, silicone, and wood alternatives.
On the positive side, here’s Slate, which did very well on this story. Back on 4 Nov, long before any of the scandal had occurred, they ran a story, “I’m Not Throwing Away My Black Plastic Spatula. Yes, they can have scary chemicals in them. But let’s take a closer look at the research,” with lots of details going into the science. And, as a bonus, on 16 Dec they added an update at the very beginning of their story, “Black plastic is even less worrisome than it initially seemed—one of the papers sparking concern contained a math error,” with that last link going back to . . . the National Post story from 11 Dec!
Good job, Slate! Excellent science reporting. And I say this not just as an occasional science reporter for Slate.
The Los Angeles Times, to its credit, also got on the bandwagon, with a story dated 19 Dec entitled “Your black plastic kitchen utensils aren’t so toxic after all. But you should still toss them, group says,” and continuing:
Published in the peer-reviewed journal Chemosphere, experts from the nonprofit Toxic-Free Future said they detected flame retardants and other toxic chemicals in 85% of 203 items made of black plastic including kitchen utensils, take-out containers, children’s toys and hair accessories. . . .
But in an update to the study, the authors say they made an error in their calculations and the real levels were “an order of magnitude lower” than the EPA’s thresholds. The error was discovered by Joe Schwarcz, director of McGill University’s Office for Science and Society in Canada.
In a blog post, Schwarcz explained that the Toxin-Free Future scientists miscalculated the lower end of what the EPA considered a health risk through a multiplication error. Instead of humans being potentially exposed to a dose of toxic chemicals in black plastic utensils near the minimum level that the EPA deems a health risk, it’s actually about one-tenth of that.
The L.A. Times didn’t link to or credit the National Post, but they did credit Schwarcz, which is important. Oddly enough, though, they did not link to Schwarcz’s press release (which they label as a blog, which is I don’t think is quite accurate, but maybe it’s close enough). The L.A. Times does link to their earlier article, but they do a bad thing by (a) not pointing out that it was their newspaper that fell for that earlier hype (to know it was the L.A. Times, you’d need to hover over or click on that link) and (b) not pointing out their earlier, alarmist, headline, “Your cool black kitchenware could be slowly poisoning you, study says. Here’s what to do.”
Some further internet searching took me to the world of news aggregators–those sites that manage to show up in your google search but don’t do any reporting themselves. The site msn.com offered two articles dated today, one from the South African site DW and one from the U.S.? site Newser. The latter article is just a rewrite of the National Post article. To its credit, it cites and links to the National Post, so in that sense, it’s the best of the bunch. On the other hand, the whole thing is kind of sketchy: Newser describes itself as “a human + ai experiment,” and what it’s doing is paraphrasing an existing article and then putting it on some new platform with a bunch of ads, which is then scraped by msn.com, which adds no value but is then itself included in a google search. Lots of middlemen in this system, and at no stage is any value added.
Here’s Scientific American, with a story dated 18 Dec, “Any Level of Flame Retardants in Black Plastic Spatulas Concerns Scientists. The scientists behind a popular study on the health effects of flame retardants in black plastic cooking utensils and toys made a calculation error but still say their revised findings are alarming.”
I’m not so happy about this story, for two reasons. First, yes, they mention the factor-of-10 error but without citing or linking to Schwarcz at all, indeed not mentioning him at all. Second, they give lots of quotes from the advocacy group, pretty much just following their spin. I guess this fits into Scientific American’s recent reputation as being a politicized anti-industry propaganda sheet.
From the other side, here’s Plastics Today, representing the plastics industry, or some segment of it. Their article is called, “The Case of the Black Plastic Spatula. Did you throw yours out? That might have been premature.” They don’t mention Joe Schwarcz, but they correctly do cite and link to the National Post.
USA Today reports the correction, which is fine, but they only cite the advocacy group that published the paper, without crediting Schwarcz. They link to the advocacy group’s updated press release, which minimizes their factor-of-10 mistake (“The error does not impact the study’s findings, recommendations, or conclusions,” which makes you wonder why they bothered putting the number in their paper in the first place, also makes you wonder if they’d said it didn’t impact anything had the error been a factor of 10 in a direction that did not favor their advocacy) and, yes, they say “We are grateful to the scientists who brought this forward and we apologize for the error,” but, no, they do not link to Schwarcz’s report or credit him in any way.
As you might expect, the tech news site Gizmodo does a better job, citing Schwarcz and linking to the National Post article.
Bill Bottenberg pointed me to an article dated 16 Dec in the news site Ars Technica, “Huge math error corrected in black plastic study; authors say it doesn’t matter. Correction issued for black plastic study that had people tossing spatulas.” It’s pretty bad: zero credit to Schwarcz or the National Post, it just goes over the original published paper and the correction notice from the journal.
Regarding the substance of the matter, Bottenberg writes:
Being a natural born physical chemist I noticed the earlier claim about the dangers of black plastic in the kitchen. Of course I examined immediately my kitchen tool drawers and found: narrow spatula with a lotta holes, wide spatula with a flat front edge – good for flipping omeletes, nice slotted spoon good for removing stuff from a boiling broth for various reasons, and a deep wide spoon good for serving from a soup or stew. These were all as black and plasticky as could be. I looked at them and let them be.
Why would I do this? Well, maybe a feeling that the mass transfer processes in water (with stuff) didn’t seem likely at the temperature ranges that might be encountered. But then, the mass transfer rates in dang hot oil (with stuff) might be meaningful. These considerations might be for aromatic molecules, maybe for nasty bromine containing organics aromatic or not.
What a puzzle. I closed the drawers and continued using them as I mulled over the possibilities.
This is how we human beings do stuff. Either we throw them all out, keep them because we don’t care, or just think for a while about the ramifications…
To the current points you have been making, which I really enjoy and appreciate – the authors have admitted to the mistake, but not so much as to change their recommendations. It’s a strange world.
Ha ha, the jokes on me. Now I have to go back and really read the original paper, rethink the Ars Technica article and figure out what’s next.
I’m not emptying the drawers of black plastic…
My recommendations to Bottenberg are:
1. Don’t “rethink the Ars Technica article.” Just go back to the original sources. And I’d recommend spending more time on Gizmodo and less time on Ars Technica (and zero time on Scientific American!).
2. Use a metal ladle to serve your soup, wooden spoons for stirring, a metal spatula for frying on your usual pans, and a silicone spatula for your nonstick pans. If you don’t have any of these items, then, sure, no rush to get them all at once, but you must already have some of these in your kitchen already, no?
Summary
The news organizations did a reasonable job covering the correction, except for four things:
1. Many fewer outlets reported on the error than on the original story.
2. Even when reporting the error, they mostly relied too strongly on the spin provided by the advocacy organization.
3. They mostly did not cite the National Post, even though that was the newspaper that broke the story. Unfortunately, this seems to be standard practice, most notoriously with the New York Times but with other newspapers as well, to not credit the reports from other newspapers that got to a story first. Here’s a particularly bad example from last year of the NYT retelling a story without acknowledging the original source, which was a Canadian newspaper.
4. Many of the outlets did not credit Joe Schwarcz, the person who discovered the error. This was a consequence of items 2 and 3 above, as Schwarcz was not credited in the correction notice and press release from the authors of the study.
This is not plagiarism, but a similar issue arises here, something that Thomas Basbøll and I have written about: When a news article obscures is sources, information is lost, and the reader pays the price.
In this case, it’s clear that several of the followup articles were directly based on the National Post article; by not mentioning or linking to that earlier reporting, they were covering their tracks and making it harder for readers to see beyond the spin being provided by the advocacy organization.
I’m not saying here that plastic kitchen items are safe (or that they’re not); what I’m talking about is the reporting.
Also, don’t trust Scientific American.