Books to Read While the Algae Grow in Your Fur, December 2015

Attention conservation notice: I have no taste. Also, this month when I wasn't reading textbooks on regression, I was doped to the gills on a mixture of binged TV shows, serial audio fiction and flu medicine.

Michael H. Kutner, Chris J. Nachtsheim and John Neter, Applied Linear Regression Models

J. J. Faraway, Linear Models with R

Sanford Weisberg, Applied Linear Regression

Having taught undergraduate linear regression for the first time this year, I had to pick a textbook, which meant reading a lot of them. These were the three I made it through cover to cover. Kutner et al. (henceforth KNW) is the one previously assigned for the class, and which we ended up keeping for reasons of continuity. Farawy's and Weisberg's were optional.

I have to say that over the course of the semester I came to really dislike KNW. The mathematical level is very low — I don't think anyone could read it and come away with any notion of why the numerator and the denominator in an $F$ test statistic are independent, or even that an $F$ test is a specialization of a likelihood ratio test. Which, OK, there's room for regression textbooks which aren't deeply into probability. But it has a most unhelpful devotion to things which were never more than kludges adapted to the computing hardware of 1950 or even 1920, like ANOVA tables, and endless attention to transformations which try to make things look more Gaussian and/or additive, never mind what they do to the interpretations. Against this, there are literally four pages on the bootstrap, and while leave-one-out cross-validation is mentioned, multi-fold CV isn't. It's almost as though the last forty years of statistics never happened. This in turn makes the explanation of regression trees fitting (another four pages, including examples) totally obscure. Now, I am sure that KNW know all this stuff perfectly well, but they don't teach it, at least not here, and I can't begin to fathom why. Even the data examples are small and antiquated and often just weak. (Who tries to predict house prices without information on location? Seriously, who?)

Both Faraway and Weisberg are superior in several ways: neither goes far into probability, but they move much faster, they are more up to date about things like Gaussianity, ANOVA tables, non-linear models, etc. (*), their computing is better, and their examples are more serious. Faraway has more material on shrinkage estimators (ridge regression, lasso) than Weisberg, and several chapters on experimental design, which Weisberg hardly touches on. On the other hand, Weisberg does have a more gentle opening with material on scatterplots and on "simple" regression (i.e., with one predictor variable). At least with undergrads, starting soft like that is probably a good idea.

None of the three books has an adequate discussion of causal inference, though again Faraway has the most; at least none of them say anything actively harmful on the matter. All three put model diagnostics after parametric inference within the model, which I realize is the traditional order but makes little sense — why bother testing whether such-and-such a slope is exactly zero if the model is rubbish in the first place? (**)

All three are outrageously priced, with KNW being by far the worst. (When the Revolution comes, Big Textbook won't be the first up against the wall, but they'll get a low number***.)

Clearly, I do not recommend KNW for self-study, though either Faraway or Weisberg should be fine. I would need a truly compelling reason to assign KNW again in the future. I would be happy to use either Faraway or Weisberg, leaning towards the former.

*: E.g., Weisberg (sec. 9.3, p. 204): "The assumption of normal errors plays only a minor role in regression analysis. It is needed primarily for inference with small samples, and even then the bootstrap ... can be used for inference. Furthermore, nonnormality of the unosbervable errors is very difficult to diagnose in small samples by examination of residuals." Or Faraway (sec. 3.2, p. 35): "It is not really necessary to specifically compute all the elements of the [ANOVA] table. As the originator of the table, Fisher said in 1931, it is 'nothing but a convenient way of arranging the arithmetic.' Since he had to do his calculations by hand, the table served a necessary purpose, but is not essential now."

**: Yes, there are circumstances where one might be interested in testing hypotheses about the best linear approximation, using a fixed set of variables, to the true