“Gold standard science”
Statistical Modeling, Causal Inference, and Social Science 2025-06-03
I got this email the other day from a journalist at a major news organization:
I’m a science reporter from **, wondering if you’d have some time to talk/reflect on the Trump administration’s embrace and use of concepts like “Gold Standard Science” and replicability – and the extent to which this is being used to improve science, or not.
I responded that it’s impossible to take this seriously, given that the same people who claiming to advocate so-called gold standard science have been energetically pushing junk social science such as unsupported claims of widespread election fraud and junk biological science such as, most notoriously, a discredited paper on vaccines and autism. This is the absolute opposite of gold-standard science.
Don’t get me wrong. The government has supported bad science in the past–both Brian Wansink and Cass Sunstein have held government posts (under Bush Jr. and Obama, respectively), and I guess that lots of papers in the glory days of Psychological Science and PNAS from 2010-2015 were conducted in part using public funding. And the notorious Excel error paper is said to have influenced government funding. That was all a little bit different, though, because these were projects that seemed reasonable at first and then only in retrospect were recognized to be fatally flawed. The vaccines and autism stuff, though: that’s junk science that the government is endorsing, years after the fraud has been revealed.
Then this news recently came out, “White House Health Report Included Fake Citations”:
A report on children’s health released by the Make America Healthy Again Commission referred to scientific papers that did not exist. . . .
“It makes me concerned about the rigor of the report, if these really basic citation practices aren’t being followed,” said Katherine Keyes, a professor of epidemiology at Columbia University who was listed as the author of a paper on mental health and substance use among adolescents. Dr. Keyes has not written any paper by the title the report cited, nor does one seem to exist by any author.
Props to Keyes to stating her objections so mildly. I’d be screaming right now had that happened to me. “Concerned about the rigor of the report,” indeed!
From the official “Restoring Gold Standard Science” statement:
Employees shall not engage in scientific misconduct nor knowingly rely on information resulting from scientific misconduct. . . . Except as prohibited by law, and consistent with relevant policies that protect national security or sensitive personal or confidential business information, agency heads shall . . . make publicly available the following information within the agency’s possession: the data, analyses, and conclusions associated with scientific and technological information produced or used by the agency that the agency reasonably assesses will have a clear and substantial effect on important public policies or important private sector decisions . . . the models and analyses (including, as applicable, the source code for such models) . . .
The good news, I guess, is that the Wakefield autism data and the recent White House health report have no actual data, nor do they have any source code, so . . . nothing needs to be made publicly available! There’s no way to publicly share data and references that don’t exist. On the other hand, it does seem to be the case that government employees are “engaging in scientific misconduct” and “knowingly relying on information resulting from scientific misconduct” by promoting these fraudulent statements, so that seems to be a problem.
That report with the fake citations was released on 22 May 2025 and the Gold Standard Science document is dated 23 May 2025 so maybe the authors of that report are off the hook, as their violation occurred before this new policy was announced.
According to the news article, a government spokesperson “did not answer a question about the source of the fabricated references and downplayed them as ‘minor citation and formatting errors.” She said that “the substance of the MAHA report remains the same.”
We’ve seen that before! Fake data, fake references, garbled analysis, whatever it is, when critics point out the problems, the reaction is not to reassess but to double down. This is absolutely horrible behavior. The right thing would be to step back and say, “Hey, if we’re relying on discredited research and backing up our claims with fake citations, maybe we shouldn’t be so sure of ourselves.” But nooooo, they don’t do that. This is classic junk-science behavior.
Again, this is the opposite of anything that should be called “gold standard science.”
Beyond all that, the term “gold standard science” makes me uncomfortable, as I associate it with research in various fields where there is some causal identification and statistical significance which is then used to bully readers into accepting iffy claims. Setting aside anything about the current government, I’d be happy if the terms “gold standard” and “science” were never used in the same sentence (so I’d slightly change what I wrote on page 1 of this article from a few years back)