Selection effects can go both ways (for taxi drivers as well as the rest of us)

You know how we talk about the two modes of microeconomic reasoning? For example, here:

The logic of social science can work in two directions: generative modeling predicts behavior given assumed preferences, and inferential reasoning deduces preferences given observed behavior. Both these modes of reasoning can be valuable, but in choosing which mode to use, social scientists have the freedom to come to essentially opposite conclusions for any problem that comes in.

And here on the two modes of pop-microeconomics:

1. People are rational and respond to incentives. Behavior that looks irrational is actually completely rational once you think like an economist.

2. People are irrational and they need economists, with their open minds, to show them how to be rational and efficient.

Argument 1 is associated with “why do they do that?” sorts of puzzles. Why do they charge so much for candy at the movie theater, why are airline ticket prices such a mess, why are people drug addicts, etc. The usual answer is that there’s some rational reason for what seems like silly or self-destructive behavior.

Argument 2 is associated with “we can do better” claims such as why we should fire 80% of public-school teachers or Moneyball-style stories about how some clever entrepreneur has made a zillion dollars by exploiting some inefficiency in the market.

The trick is knowing whether you’re gonna get 1 or 2 above. They’re complete opposites!

More generally, almost any social science argument can be turned around 180 degrees, as I discussed here in the context of risk aversion.

Alex Tabarrok provides a great explanation of the challenges of two-way thinking, in this case regarding selection effects:

Should I be worried or reassured that my taxi driver isn’t wearing a seat belt? An econ puzzle.

I should be worried . . . and it reveals something of importance. First note that there is an incentive and a selection effect. All else equal, a driver without a seat belt should drive more carefully–that’s the rational response to increased personal risk. But drivers who forgo seat belts are probably more risk-loving or less safety-conscious across many dimensions. I think . . . the second effect, the selection effect, dominates: be worried. . . .

What makes this an economics puzzle is that it reveals a failure of the standard adverse selection story. . . . The taxi driver puzzle is a clean real-world case where the selection effect runs opposite to what adverse selection theory predicts. Adverse selection theory is correct that information asymmetries can challenge markets but it’s often not obvious which way the asymmetry runs . . . Moreover, preferences and norms can make the selection run the opposite way . . .

It’s a good example of how to avoid the one-way street fallacy.

I like Alex’s framing that there are real effects going in both directions, so that you can get a net positive or a net negative depending on which effect is bigger. I think this makes much more sense than supposing a single effect that goes in one direction or another.

This two-way thinking is also helpful when thinking about generalization. If someone does a study finding a positive effect of some treatment, and then you want to think about what will happen in a new setting, just think of it like this: There’s a positive effect A and a negative effect B, and the study finds that A – B > 0 in a certain setting. What will happen in a new setting? It depends on both A and B. There’s no reason to assume that the sign of A – B is some kind of universal invariant.

This comes up a lot in statistics too, that you have to be careful to avoid setting up your model in a way that would restrict the direction of the effects.