Where do theories come from?

Lee Sechrest sends along this article by Brian Haig and writes that it “presents what seems to me a useful perspective on much of what scientists/statisticians do and how science works, at least in the fields in which I work.” Here’s Haig’s abstract:

A broad theory of scientific method is sketched that has particular relevance for the behavioral sciences. This theory of method assembles a complex of specific strategies and methods that are used in the detection of empirical phenomena and the subsequent construction of explanatory theories. A characterization of the nature of phenomena is given, and the process of their detection is briefly described in terms of a multistage model of data analysis. The construction of explanatory theories is shown to involve their generation through abductive, or explanatory, reasoning, their development through analogical modeling, and their fuller appraisal in terms of judgments of the best of competing explanations. The nature and limits of this theory of method are discussed in the light of relevant developments in scientific methodology.

I found this very difficult to read and forwarded it to Cosma Shalizi, who writes:

Like a lot of what I read about abduction, it seems much more a theory (or sketch of a theory) of how scientists think, than of scientific method. Put another way, the H-D account of scientific method has always tended to “black-box” the issue of where hypotheses come from, in favor of what to do with them once you have them. I think this is usually helpful, but there’s no reason not to try to open up the black box, and study the origin of hypotheses; if there’s a role for abduction, it’s there, in explicating the “generate” part of generate-and-test. (In fact, if memory serves, Peirce later repented of his term “abduction”, and just called it “hypothesis” or “hypothesizing”.) If one could show, or even plausibly suggest, that certain modes of hypothesizing are systematically more reliable or fruitful than others, that would be extremely valuable. This paper in particular seems to have some odd confusions of levels between what are presumably fairly permanent parts of how scientists think (analogy), and current technological artifacts — I love bootstrapping, but it hardly belongs in the same category as a component of scientific method. (And as for stem-and-leaf plots…)

Sechrest wrote:

Too much research seems to be addressed to determining whether “it is,” or “it isn’t.” The more important question very often is “Why is (or isn’t) it?” To me, abduction seems more likely to occur in the aftermath of having seen something. Isaac Asimov once said The most exciting phrase to hear in science, the one that heralds new discoveries, is not ‘Eureka!’ but ‘That’s funny…’

And that is when abduction begins, the attempt to identify explanations and reason toward the best one. For example, at least some drug trials begin with seemingly sensible expectations that a drug will work; those expectations are often wrong. Usually that is the end of the matter. But, to me, an important question may well be “Why didn’t the drug work as expected?” (A “why isn’t it? question) The abductive process cannot lead directly to a clear-cut answer, but it can get us closer. And that is what, I think, good scientists do. Ineffective scientists (and I have seen them many times) say, “Well, that didn’t work. Anybody got another idea?”

I have nothing to add to the above discussion, except to point to our recent discussion of the challenges of systematizing model building. As I see it, new ideas arise from anomalies in data with respect to existing theories.

The post Where do theories come from? appeared first on Statistical Modeling, Causal Inference, and Social Science.