How to understand coefficients that reverse sign when you start controlling for things?

Denis Cote writes:

Just read this today and my unsophisticated statistical mind is confused.

“Initial bivariate analyses suggest that union membership is actually associated with worse health. This association disappears when controlling for demographics, then reverses and becomes significant when controlling for labor market characteristics.”

From my education about statistics, I remember to be suspicious about multiple regression coefficients that are in the opposite direction of the bivariate coefficients. What I am missing? I vaguely remember something about the suppression effect.

My reply:

There’s a long literature on this from many decades ago. My general feeling about such situations is that, when the coefficient changes a lot after controlling for other variables, it is important to visualize this change, to understand what is the interaction among variables that is associated with the change in the coefficients. This is what we did in our Red State Blue State paper, for example, and we also developed some such tools in our paper on police stop-and-frisk. I have not read the particular article you cite, so I can’t comment on that particular application, but in general I think these multiple regression analyses can be fine but I like to understand where in he data the switching of sign is coming from.

I think there’s still a lot of useful work to be done on graphical methods for understanding the effects of conditioning on regression models.

The post How to understand coefficients that reverse sign when you start controlling for things? appeared first on Statistical Modeling, Causal Inference, and Social Science.