Science in the Age of Covid-19, Part 3

Here is Part 3 of my talk on research methods used in Covid-19 science. In Part 2, I point out the key differences between randomized clinical trials (RCTs) and real-world studies. In particular, real-world studies are subject to a variety of real-world biases. The quality of such studies hinges on whether the researchers make judicious decisions about how to correct for such biases.

In Part 3, I explain why even RCT results merit careful interpretation. RCTs are not perfect either, and these vaccine studies offer great materials to learn about the unintended shortcomings. The key topics covered are interim analyses, case-counting windows, staged enrollment, week-to-week variability, placebo effects, immortal time bias (again), conditionality, and story time.

Towards the end of these slides, I discuss the trend of "re-analysis" or "post-hoc analysis". This is one of the more worrisome trends in Covid-19 research. Post-hoc data mining is an about-face to prespecifying an analysis plan, and is the mother of spurious results.

Having run well beyond an hour of materials, I then very quickly describe another research method - using models and simulations. I may come back with a Part 4 to give this section justice. In the meantime, you can review the set of posts I wrote last year about the Oxford model (here).

Here are the links to Part 1, Part 2, and Part 3 of the talk.