Graphical display of election forecast uncertainty
Statistical Modeling, Causal Inference, and Social Science 2025-02-02
Josh Goldstein, author of The Formal Demography of Peak Population and other things, writes:
What do you think about this kind of graphical display in today’s NYT?
The margins of error are hugely overlapping, but, if I did my calculation correctly, the SE on the difference of 3% is only about 1.4% or so, and so the difference itself is > 2SEs from 0. If we want to know if “Harris is really ahead”, should you believe this picture and say that it’s too uncertain to say? Or, should you believe the SE of difference approach and say that it appears very likely that she’s ahead, at least by a little?
More generally, it makes me think that when we do coefficient plots with the intention of visually trying to figure out if there are important differences between coefficients that it might be better to plot +/- 1 SE and not +/- 2.
I replied that Harris is currently ahead in the popular vote, as estimated from the polls. But the polls have nonsampling error, so maybe she’s not really ahead, or maybe she’s ahead by less or more than the polls indicate. Our rough calculation is to double the standard error to account for nonsampling error. Also, it’s expected that Harris needs something like 51% or 52% of the national two-party vote in order to win the electoral vote this year (an effect that has varied over time), so being ahead in the popular vote isn’t enough.
Goldstein adds:
On separate topic, also kind of interesting that we seem as a society to be post-election-forecasting. Maybe I’m just projecting my own feelings, but it seems that this time, everyone is just willing to wait and not screaming about the polls/forecasts etc. Maybe we’re just worn out!
I dunno, I think it’s just that the forecasts are so close. If the election were decided by national popular vote, or if it wasn’t expected that there will be an electoral/popular vote mismatch, then I think there’d be a lot of discussion of whether Harris’s win probability is 60% or 70% or 80% or 90% or whatever. But since there’s so much uncertainty in the electoral vote, we just can’t say much. Some poll aggregators can try to construct news out of nothing by reporting win probabilities to the nearest fraction of a percentage point, but, as we’ve discussed before, that’s pretty meaningless.