How effective are football coaches?

Dave Berri writes:

A recent study published in the Social Science Quarterly suggests that these moves may not lead to the happiness the fans envision (HT: the Sports Economist). E. Scott Adler, Michael J. Berry, and David Doherty looked at coaching changes from 1997 to 2010. What they found should give pause to people who demanded a coaching change (or still hope for one). Here is how these authors summarize their findings:

. . . we use matching techniques to compare the performance of football programs that replaced their head coach to those where the coach was retained. The analysis has two major innovations over existing literature. First, we consider how entry conditions moderate the effects of coaching replacements. Second, we examine team performance for several years following the replacement to assess its effects.

We find that for particularly poorly performing teams, coach replacements have little effect on team performance as measured against comparable teams that did not replace their coach. However, for teams with middling records—that is, teams where entry conditions for a new coach appear to be more favorable—replacing the head coach appears to result in worse performance over subsequent years than comparable teams who retained their coach.

Berri points to other studies from hockey, soccer, and basketball that come to similar conclusions: replacing a head coach does not, on average, lead to improvement. He concludes:

What these studies did is look at teams or players with different coaches and failed to find much of a difference. That suggests that coaches in sports are not very different from each other. It may be true (and more than likely very true) that you are better off with a professional coach than with a random person grabbed from the stands (or no one at all). But it doesn’t appear that the choice of professional coach matters much.

Berri continues:

And that means, if it costs a small fortune to fire your coach – and often it does – then a team is probably better off just keeping who they have on the sideline. Yes, this may not make the fans of the losers very happy today. But it doesn’t make sense for universities to make decisions that cost the school money and don’t systematically change the outcomes we see on the field.

I have a few thoughts on this. Let me start by saying that it’s good that Berri is pointing to research rather than just speculating (as I’m about to do). That said, I wonder if he is going too far in his interpretation of those findings.

I have not read the linked studies but, from Berri’s description, they found that a replacement coach is no better, on average, than the coach that came before. But how do you get from there to “it doesn’t appear that the choice of professional coach matters much”? Nearly all coaches change jobs at some point, good coaches as well as bad coaches. Several blog commenters make relevant points along these lines, but unfortunately I didn’t see a response from Berri. (Our blog here is unusual in that I apparently have nothing better to do than to read (and often respond to) your comments!)

The other thing that struck me, given that Berri’s post appeared in the Freakonomics blog, was how his story fits paradigm 2 of microeconomics. Recall that economists are often making one of two arguments:

1. People are rational and respond to incentives. Behavior that looks irrational is actually completely rational once you think like an economist.

2. People are irrational and they need economists, with their open minds, to show them how to be rational and efficient.

Argument 1 is associated with “why do they do that?” sorts of puzzles. Why do they charge so much for candy at the movie theater, why are airline ticket prices such a mess, why are people drug addicts, etc. The usual answer is that there’s some rational reason for what seems like silly or self-destructive behavior.

Argument 2 is associated with “we can do better” claims such as why we should fire 80% of public-schools teachers or Moneyball-style stories about how some clever entrepreneur has made a zillion dollars by exploiting some inefficiency in the market.

Recently I remarked on a Freakonomics article that had a delightful mix of the two stories: Stephen Dubner and Steven Levitt wrote that “there is one betting strategy that will routinely beat a bookie, and you don’t even have to be smart to use it” (argument 2: with careful data analysis, you can cleverly beat the odds) but then went on to mock some (unnamed) “academics” for using this fact to underestimate the intelligence of street-smart bookies (that’s argument 1).

Berri’s example seems more like a pure case of argument 2: sports teams are spending millions of dollars on top coaches, and that’s a waste of money because professional coaches are essentially all the same. With some effort you can make an “argument 1″ story here—the idea being that teams fire coaches because it makes the fans happy—but I don’t buy it. My impression is that fans want wins, they want championships, and to a lesser extent they want star players and exciting action. There’s a lot better ways of spending millions of dollars to get all these things, than to blow it on coaches with no ability.

I should also point out that, in the best form of a social-science story, arguments 1 and 2 go together. Argument 2 identifies the $20 bill on the sidewalk, and argument 1 explains why nobody was picking it up. In practice, though, I feel like we typically see one side of the story without the other. When argument 1 is uncorked, there is a tendency to treat the identified behavior as rational and normative, while with argument 2 there is typically an assumption that the newly discovered strategy is simply better.

P.S. To return to sports, I also wanted to bring up one of my pet ideas on why coaches get fired when they do:

Dan [Goldstein] has another question that I think I have the answer to. He writes that he “has always wondered why teams are so eager to fire their coaches after they lose a few big games. Don’t they realize that their desired state of having won those same few big games would have been mostly due to luck?”

My guess is that these are situations where the management has already decided they want to fire the coach, and they’re just waiting for a convenient time to do it so as not to antagonize the fans.

P.P.S. Also this on coaching.