Whassup with the FDA approval of that Alzheimer’s drug? A “disgraceful decision” or a good idea?

Andrew Klaassen writes:

Any chance you’ll be weighing in on your blog on the apparently wobbly studies supporting the FDA’s approval of Aduhelm? I’m hearing angry things being said about it by the random people I know in medical research, but don’t know much beyond that.

Here’s the one link on the story [by Beth Mole] that I’ve read so far.

And Deborah Mayo points to this discussion by Geoff Stuart that is critical of the FDA’s reasoning.

Beyond all that, I talked with someone who works more on the policy side who said she thought the drug approval was a terrible idea and asked me how the FDA could ever have made this decision.

Here are my two thoughts.

First, I defer to the experts on this. If everybody thinks the FDA approval was a bad idea, it probably was. I base this conclusion not on the statistical details I’ve seen but rather on a more general impression of rules and fairness, that if other drugs with similar trial results wouldn’t get approved, that some better justification would be needed. Again, I say this not based on any analyses of mine, just based on my respect for all the people who expressed this view. Also the usual concerns about burdening the taxpayer (if this drug is approved for Medicare payments), diverting resources from other treatments, etc.

Second, I can see a rationale for approving a drug in this case even if there’s no evidence that it work. It goes like this. If you approve the drug, some people will try it. If some people will try it, we’ll get some data. Not randomized data—but it’s not clear that randomized data are really what we need. If the treatment is approved now, then we’ll get real-life data, and in 2022 we’ll have one year of real-life data, in 2026 we’ll have five years, etc. It seems likely that this treatment probably won’t do so much to help people right now, but some real-world longitudinal data could help understand what it does do, and that could be valuable in helping to develop future treatments.

If this argument is correct, then the rationale for approval is not about whether this particular drug does the job, but rather whether this line of research might ultimately be successful. Approve the first crude attempt now, and then this will put us on the escalator to developing the thing that really works. Don’t approve, and you delay this future development.

Again, I’m not saying I think the FDA made the right decision in approving the drug—I’ll defer to the experts who say otherwise—I’m just saying I can see a justification in general terms.

It is perhaps helpful to consider this future-looking justification when evaluating the arguments for and against approval. For example, the above-linked news article shared this post from the director of the FDA’s Center for Drug Evaluation and Research:

We ultimately decided to use the Accelerated Approval pathway—a pathway intended to provide earlier access to potentially valuable therapies for patients with serious diseases where there is an unmet need, and where there is an expectation of clinical benefit despite some residual uncertainty regarding that benefit… [T]reatment with Aduhelm was clearly shown in all trials to substantially reduce amyloid beta plaques. This reduction in plaques is reasonably likely to result in clinical benefit.

From the other side is this statement from Mark Dallas, a neuroscientist at University of Reading:

This sets a dangerous precedent for future drugs in the fight to combat Alzheimer’s and other complex diseases. In many ways the clinical trials undertaken do not present a clear picture that this medicine will be of tangible benefit to individuals living with dementia.

And this from Robert Howard, a psychiatry professor at University College London:

As a dementia clinician and researcher with personal family experience of Alzheimer’s disease, I want to see effective dementia treatments as much as anyone. I consider the approval of aducanumab represents a grave error that will have only negative impact on patients and their families and that could derail the ongoing search for meaningful dementia treatments for a decade.

This last quote is interesting because he offers a forward-looking argument in the No direction. If I’m interpreting his statement correctly, Howard is saying not just that the new drug has not been proven effective but also that he suspects this line of research is a dead end.

This to me connects to a general issue of statistics and policy analysis:

Decisions are made based on immediate questions that are (partially) resolvable by available data: Did this particular drug work on this particular group of people? But the best reasons for the decision are long-term: Will approving this drug take us in a useful direction going forward? It’s tricky because government decisions should be based on some principles, and once we move away from the hard numbers there is the risk of bad decisions and political interference.

But still. When considering, say, a construction project, the government will do some cost-benefit analysis and this will be forward-looking: some estimate, for example, of the number of people who in five years will be driving over this hypothetical bridge or riding this particular train or whatever. These projections will be based on assumptions, and these assumptions should be stated clearly, but at least the accepted framing is about the future. I am bothered that much of the discussion of drug approval is backward-looking, all about details of a particular study that’s already done. This seems related to the general problem of statistical studies being analyzed in isolation and a focus on noisy summaries such as statistical significance.

Just to be clear: I believe that the FDA and its critics are ultimately thinking about long-term benefits. I just feel that much of the available language for this discussion is focused on static analyses of a single study, so that the long-term questions are implicit and not foregrounded.