Survey Statistics: equivalent models, equivalent weights (locally)

Last month we saw that the Times/Siena Poll is now using energy balancing weights (Huling & Mak, 2024). In a toy example, we saw under which outcome models these weighting methods might do well. I was inspired by Little 2004, who saw under which outcome model the inverse-probability-weighted estimator (a.k.a. Horvitz-Thompson) does well. This explains the HT estimator’s poor performance in Basu’s (1971) elephants example (which I’ve used in my post Basu’s Bears).

From Little 2004:

Andrew’s 2007 “Struggles” paper and the 2026 MrPlew paper go the other way: start with an outcome model and back out the weights. These folks all worked pretty hard. I wondered if I could just use Thomas Lumley’s survey package to get the weights. See my posts “struggles with equivalent weights” and my continued struggles. But in the simulation I used a linear outcome model. It would have been more interesting with a logistic outcome model !

The linear case was handled by Andrew’s 2007 paper. The 2026 MrPlew paper extends this by noticing that the linear equivalent weights are derivatives, and using this to define locally equivalent weights, green highlighting by me:

Then the 2026 MrPlew folks use these equivalent weights to compare the target population to the weighted sample, as an MrP diagnostic. (Unlike the Huling & Mak paper on energy balancing weights, the MrPlew folks look at one covariate function at a time, not the entire covariate distribution at once.) One of their examples is from Lax & Phillips 2009 Gay Rights in the States, which I blogged about for pride. In this example, the implied covariate (im)balance for MrP looks not great ! Here’s their Figure 5b:

As Andrew blogged: “It makes sense that implied covariate balance can sometimes be worse for MRP than for raking. MRP is a smoothed version of raking, and unsmoothed raking can overfit.” Yes, but here it’s worse than the raw uncorrected balance ! The MrPlew authors caution here that the locally linear approximation may be extrapolating poorly, and that we shouldn’t over-rely on this model check. They also say that “logistic regression generally balances the variance-weighted covariates, but not the covariates themselves”. I am glad they included this illustrative example and I want to think more about it.

Thoughts ?