The replication crisis may also be a theory crisis

Enlarge / A jumbled jigsaw puzzle, AKA the state of theory in the behavioral sciences. (credit: flickr user: giveawayboy)

A replication crisis has called into question results from behavioral (and other) sciences. Complaints have focused on poor statistical methods, the burying of negative results, and other “questionable research practices” that undermine the quality of individual studies.

But methods are only part of the problem, as Michael Muthukrishna and Joseph Henrich argue in a paper in Nature Human Behaviour this week. It’s not just that individual puzzle pieces are low in quality; it’s also that there’s not enough effort to fit those pieces into a coherent picture. "Without an overarching theoretical framework,” write Muthukrishna and Henrich, “empirical programs spawn and grow from personal intuitions and culturally biased folk theories.”

Doing research in a way that emphasizes joining the dots constrains the questions you can ask in your research, says Muthukrishna. Without a theoretical framework, “the number of questions that you can ask is infinite.” This makes for a scattered, disconnected body of research. It also feeds into the statistical problems that are widely considered the source of the replication crisis. Having too many questions leads to a large number of small experiments—and the researchers doing them don't always lay out a strong hypothesis and its predictions before they start gathering data.