Survey Statistics: Margin of Error
Statistical Modeling, Causal Inference, and Social Science 2026-01-13
What is a poll’s margin of error ? Let’s narrow the discussion (and the margins of error themselves) to MRP with Bayesian inference and simplifying assumptions:
- Estimate E(Y) via E(E(Y | X, sample)) with MRP using two steps:
- MR: Fit a model E(Y | X, theta, sample) to our sample with Bayesian inference to get posterior draws of parameters theta^(s)
- P: For each theta^(s), average over known population distribution p(X) to get E^(s)(Y)
- Use this first flavor of calibration only (no second flavor of calibration, i.e. logit-shifting)
- Assume E(Y | X, sample) = E(Y | X), i.e. missing at random, i.e. no coverage or nonresponse bias after adjusting for X
- Assume no measurement error in X or Y
- Assume our model for E(Y | X) is correct
Next week we will see additional uncertainty because these assumptions don’t hold, see Disentangling Bias and Variance in Election Polls by Shirani-Mehr, Rothschild, Goel, and Gelman.
(The polar bear is back in the Appalachians, where summit views are uncertain.)
Below are some snippets of how our simplified case is discussed in the MRP papers we’ve looked at so far.
…
MRPW draft paper by Andrew Gelman, Yajuan Si, and Brady T. West:
