Bayesian decision analysis for the drug-approval process (NSFW)

Bill Jefferys points me to a paper, “Is the FDA Too Conservative or Too Aggressive?: A Bayesian Decision Analysis of Clinical Trial Design,” by Vahid Montazerhodjat and Andrew Lo. Here’s the abstract:

Implicit in the drug-approval process is a trade-off between Type I and Type II error. We propose using Bayesian decision analysis (BDA) to minimize the expected cost of drug approval, where relative costs are calibrated using U.S. Burden of Disease Study 2010 data. The results for conventional fixed-sample randomized clinical-trial designs suggest that for terminal illnesses with no existing therapies such as pancreatic cancer, the standard threshold of 2.5% is too conservative; the BDA-optimal threshold is 27.9%. However, for relatively less deadly conditions such as prostate cancer, 2.5% may be too risk-tolerant or aggressive; the BDA-optimal threshold is 1.2%. We compute BDA-optimal sizes for 25 of the most lethal diseases and show how a BDA-informed approval process can incorporate all stakeholders’ views in a systematic, transparent, internally consistent, and repeatable manner.

Hey—the acronym “BDA” is already taken! But let’s set that aside . . . In all seriousness, here are my reactions to the above abstract:

(a) I like the idea of applying formal decision analysis to the drug approval problem, an idea I discussed in this article a few years ago but have never actually done. So I’m glad to see this new paper.

(b) As always, I find the Type 1 and Type 2 error framework to be inappropriate. I continue to be frustrated by researchers, starting perhaps with Jimmie Savage, who want to apply Bayesian inference but remain struck in this essentially deterministic or discrete way of thinking. In general, I doubt it makes sense to say that a drug works or does not work; rather, a drug will have different effects on different people, and these effects are themselves unknown. That is, there is variation and there is uncertainty. But no Type 1 or Type 2 errors. To put it another way, Yes Yes Yes on the drug-approval process as tradeoff, No No No on the discrete framing of therapies as “effective” or “not effective.” To try to perform a sophisticated, data-based analysis but tie yourself to the Type 1 and Type 2 error framework, that’s like, ummmm, I already used the “paint a picture using salad tongs” analogy, but I still like it so, yeah, that’s it.

(c) It seems a bit un-Bayesian, to present numbers like “27.9%,” given the uncertainty that must be present in these estimates.

Anyway, my overall reaction is positive and I hope these comments can inspire these and other researchers in this area to do even better.

The post Bayesian decision analysis for the drug-approval process (NSFW) appeared first on Statistical Modeling, Causal Inference, and Social Science.