I don’t like discrete models (hot hand in baseball edition)

Bill Jefferys points us to this article, “Baseball’s ‘Hot Hand’ Is Real,” in which Rob Arthur and Greg Matthews analyze a year of pitch-by-pitch data from Major League Baseball.

There are some good things in their analysis, and I think a lot can be learned from these data using what Arthur and Matthews did, so my overall impression is positive. But here I want to point to two aspects of their analysis that I don’t like, that I think they could do better.

First and most obviously, their presentation is incomplete. I don’t know exactly what they did or what model they fit. They say they fit a hidden Markov model but they don’t say how many states the model had. From context I think they were fitting 3 states—hot, cold, or normal—for each player, but I’m not sure. The problem is . . . they shared no code. It would be the simplest thing in the world for them to have shared their Stan code, or R code, or Python code, or whatever—but they didn’t. This doesn’t make Arthur and Matthews uniquely bad—I’ve published a few hundred papers not sharing my code too—it’s commonplace. But it’s still a flaw. It’s hard to understand or evaluate work when you can’t see the math or the code or the data.

Second, I don’t think the discrete model makes sense. I do believe that pitchers have days when they throw harder or less hard, and I’m sure a lot more can be learned from these data too, but I would not model this as discrete states. Rather I’d say that the max pitch speed varies continuously over time.

I can see how a discrete model could be easier to fit—and I can certainly see the virtue of a discrete model in a simulation study, indeed I’ve used simulations of discrete models to understand the “hot hand fallacy fallacy” in basketball—but I think that any discrete model here should be held at arms length, as it were, and not taken too seriously.

In particular, I don’t see how we can get much out of statements such as “the typical pitcher goes through 57 streaks in a season, jumping between hot and cold every 24 pitches,” which seems extremely sensitive to how the model is parameterized.

Again, I say this not to slam Arthur and Matthews (hey, they linked to this blog! We’re all on the same side here!) but rather to point to a couple places where I think their analysis could be improved.

Alos, let me emphasize that my comments above do not reflect any particular baseball knowledge on my part; I’d say the same thing if the analysis were done for football, or golf, or tennis, or any other sport.

Speaking generally, it should be much easier to study hotness using continuous measurements (such as pitch speed in baseball or ball speed and angle in basketball) than using discrete measurements (such as strikeouts in baseball or successful shots in basketball). With continuous data you just have so much more to work with. Remember the adage that the most important aspect of a statistical method is not what it does with the data but what data it uses.

The post I don’t like discrete models (hot hand in baseball edition) appeared first on Statistical Modeling, Causal Inference, and Social Science.