Beware scientists who don’t dogfood it.

I dogfood it. The statistical methods in our books are the methods we use in teaching, consulting, and research; I use Stan to solve applied problems; etc.

But not everybody does. Sometimes maybe it doesn’t matter. If that notorious Stanford medical school professor wants to take cold showers or if the now-retired Cornell food guru wants to serve himself out of a perhaps-apocryphal bottomless soup bowl or if the disgraced primatologist wants to have pseudo-conversations with monkeys, or if the nudgelords want to rearrange the food in their kitchen, whatever. They should go for it.

The bigger problem comes when an entire field doesn’t eat its own dogfood.

This issue arose in a discussion with Megan Higgs and Pamela Reinagel a couple years ago regarding why the replication crisis didn’t seem to be as big a problem in biology (at least of the wet lab variety) than in psychology.

We came up with the following explanation:

In biology, researchers have a clear incentive to try to replicate published work, because they’re using it in their own research. That’s what’s meant by biology being a “cumulative science.” Lab biologists eat their own (and each others’ dogfood).

Certain glamour areas of cognitive and social psychology are different. For example, consider social priming of the elderly-words-and-slow-walking variety. Psychologists publish this work, but it’s not like they’re giving themselves subliminal tapes featuring the speeches of Speedy Gonzalez. In contrast, biologists are using published biology research in order to do better biology research. Biology is cumulative, not just in the sense of new research building old research, but in the sense of methods cumulating as well.

Lots of times we’re not dogfooding it because the our research is intended for others. For example, political scientists are (usually) not practicing politicians and we’re rarely applying any of our political insights to our own work.

But when researchers in a field don’t eat their own dogfood, I can see how unreplicated and unreplicable results can flourish.

The limits of dogfooding

Just to be clear, I’m not saying that scientists should only dogfood it. I use lots of methods and tools that I was not involved in developing. Nor am I proposing that scientists should dogfood everything they do. As noted above, I use the methods in my books—but I don’t use the methods in all of my research articles. Research articles are speculative: they include some ideas that ultimately become useful and some that do not. I dogfood it with many of my published research ideas but not all of them. Sometimes my colleagues and I are producing delicious dogfood that we can eat and recommend to others (as with MRP, Stan, loo, PSIS, PPC, etc.); other times we’re constructing some intermediate product that, if not directly edible, might contribute someday to the sort of dogfood that can sustain us. That’s research!