Books to Read While the Algae Grow in Your Fur, August 2015

Attention conservation notice: I have no taste.

Roland and Sabrina Michaud, Mirror of the Orient

The Michaud's gorgeous photos from the 1960s and 1970s — mostly of Afghanistan, but also Turkey, Iran, and India — aptly paired with Persianate miniature paintings. This is a wonderful book I have coveted for many years, and I am very pleased to have finally scored a copy I could afford.

Alain Barrat, Marc Barthelemy and Alessandro Vespignani, Dynamical Processes on Complex Networks

Survey of the state of the field as of 2008. It is decent and generally clear, if not especially fast-paced, and covers ideas about network structure, percolation, synchronization of oscillators, epidemic models, diffusion of innovations (mapped on to epidemic models), and Kauffman's Nk model in some detail. (They're pretty good on linkages between these.) On other biological processes they are vaguer.

I found the emphasis on results presuming exact power-law degree distributions less than compelling, and the apologia for this emphasis in the conclusion surprisingly wrong-headed. (It does no good to defend them as approximations unless you also show that conclusions continue to hold when the assumptions are in fact only approximately true --- that there is, as Herbert Simon once put it, continuity of approximation. And in many cases, you'd need very, very robust continuity of approximation indeed.) But I recognize that I am abnormally picky about this subject.

ObDisclaimer: I've met Prof. Vespignani once or twice, but I don't think I've ever met or corresponded with the other authors.

Kelley Armstrong, Sea of Shadows and Empire of Night

Mind candy: First two-thirds of a fantasy trilogy about the adventures of a pair of teenage shamans. It's surprisingly enjoyable, with surprisingly effective monsters. The human setting is inspired not by a vaguely feudal Europe, but by more-or-less Heian-era Japan, though there seems to be no equivalent of Buddhism (maybe the bit with the monks in the second book?), and making the !Ainu blonds and redheads hints at pandering to the audience.

Arthur E. Albert and Leland A. Gardner, Jr., Stochastic Approximation and Nonlinear Regression

This is all about on-line learning and stochastic gradient descent before it was cool:

This monograph addresses the problem of "real-time" curve fitting in the presence of noise, from the computational and statistical viewpoints. Specifically, we examine the problem of nonlinear regression where observations $ \{Y_n: n= 1, 2, \ldots \} $ are made on a time series whose mean-value function $ \{ F_n(\theta) \} $ is known except for a finite number of parameters $ (\theta_1, \theta_2, \ldots \theta_p) = \theta^\prime $. We want to estimate this parameter. In contrast to the traditional formulation, we imagine the data arriving in temporal succession. We require that the estimation be carried out in real time so that, at each instant, the parameter estimate fully reflects all of the currently available data. The conventional methods of least-squares and maximum-likelihood estimation ... are inapplicable [because] ... the systems of normal equations that must be solved ... are generally so complex that it is impractical to try to solve them again and again as each new datum arrives.... Consequently, we are led to consider estimators of the "differential correction" type... defined recursively. The $ (n+1) $st estimate (based on the first $ n $ observations) is defined in terms of the $ n $th by an equation of the form \[ t_{n+1} = t_n + a_n[Y_n - F_n(t_n)] \] where $ a_n $ is a suitably chosen sequence of "smoothing" vectors.

(It's not all time series though: section 7.8 sketches applying the idea to experiments and estimating response surfaces.) Accordingly, most of the book is about coming up with ways of designing the $ a_n $ to ensure consistency, i.e., $ t_n \rightarrow \theta $ (in some sense), especially $ a_n $ sequences which are themselves very fast to compute.

Mathematically, of course, we've got much more powerful machinery for proving theorems about