Whassup with Why We Sleep?

Last month we reported on the book Why We Sleep, which had been dismantled in a long and detailed blog post by Alexey Guzey. A week later I looked again, and Walker had not responded to Guzey in any way. In the meantime, Why We Sleep has also been “>endorsed by O.G. software entrepreneur Bill Gates. Programmers typically have lots of personal experience of sleep deprivation, so this is a topic close to their hearts.

As of this writing, it seems that Walker still has not responded to most of the points Guzey made about errors in his book. The closest thing I can find is this post dated 19 Dec 2019, titled “Why We Sleep: Responses to questions from readers.” The post is on a site called On Sleep that appears to have been recently set up—I say this because I see no internet record of it, and it currently has just this one post. I’m not trying to be some sort of sleuth here, I’m just trying to figure out what’s going on. For now, I’ll assume that this post is written by Walker.

The post begins:

The aim of the book, Why We Sleep, is to provide the general public access to a broad collection of sleep research. Below, I address thoughtful questions that have been raised regarding the book and its content in reviews, online forums and direct emails that I have received. Related, I very much appreciate being made aware of any errors in the book requiring revision. I see this as a key part of good scholarship. Necessary corrections will be made in future editions.

The first link above goes to a page of newspaper and magazine reviews, and the second link goes to Guzey’s post. I didn’t really see any questions raised regarding the book in those newspaper and magazine reviews, so I’m guessing that the “thoughtful questions” that Walker is referring to are coming entirely, or nearly entirely, from Guzey. It seems odd for Walker to cite “online forums” and only link to one of them. Also, although Walker links to Guzey, he does not address the specific criticisms Guzey made of his book.

And this makes me think Dan Davies’s maxim, Good ideas do not need lots of lies told about them in order to gain public acceptance. (Oddly enough, that classic quote appeared many years ago on a blog called “Economics and similar, for the sleep-deprived.”)

If Walker really knows his stuff, why would he write a book with so many errors?

On the other hand, it’s possible that Walker does know his stuff, or at least some of his stuff, and is just sloppy. He’s a sleep expert, not a statistics expert.

Looking at one specific claim

To try to get a sense of what’s going on, I decided to take a look at some particular disputed claim. I picked an item that’s purely statistical, with no confusing issues of biology: “Does the number of vehicular accidents caused by drowsy driving exceed the combined number of vehicular accidents caused by alcohol and drugs?”

From Guzey’s blog:

“In Chapter 1, Walker writes ‘vehicular accidents caused by drowsy driving exceed those caused by alcohol and drugs combined’. This shows how dangerous it is to not sleep and you have not refuted this part.”

I [Guzey] did look into this. I was not able to find any data on vehicular accidents caused by drowsy driving. However, the data by the National Highway Traffic Safety Administration on accidents that involve drowsy driving (a) and that involve drugs and alcohol (a) does not support this assertion.

According to this data, 1.2-1.4% (Table 2 in linked file) of car crashes involved drowsy driving, while 2.8% (Table 7 in linked file) of car crashes involved driving while having the alcohol blood concentration above the legal limit in the US (and this is not including accidents involving drugs).

From Walker’s blog:

Fatal vehicle accidents associated with drowsy driving are estimated to be around 15-20% (see also data from the AAA Foundation for Traffic Safety).

This is a big difference! Guzey says “1.1%-1.4% of car crashes involved drowsy driving”; Walker says “Fatal vehicle accidents associated with drowsy driving are estimated to be around 15-20%.” Guzey’s talking all accidents, Walker’s just talking fatal accidents. But that can’t explain a factor of 14!

Fortunately, both bloggers give references here, so we can follow the links and check.

Indeed, if you follow Guzey’s source, which is a report from the U.S. National Highway Traffic Safety Administration, you’ll see an estimate that drowsy driving was involved in 2.3% of fatal crashes.

Walker’s first link is to a news article that states “Drowsy driving is estimated to be a factor in 20 percent of fatal crashes,” but provides no sources. His second link is to a report from the AAA Foundation for Traffic Safety that begins:

According to the National Highway Traffic Safety Administration (NHTSA), approximately . . . 2.5% of all fatal crashes in years 2005 – 2009 involved a drowsy driver . . . However, the official government statistics are widely regarded as substantial underestimates of the true magnitude of the problem. . . . unlike impairment by alcohol, impairment by sleepiness, drowsiness, or fatigue does not leave behind physical evidence, and it may be difficult or impossible for the police to ascertain in the event that a driver is reluctant to admit to the police that he or she had fallen asleep . . . This inherent limitation is further compounded by the design of the forms that police officers complete when investigating crashes, which in many cases obfuscate the distinction between whether a driver was known not to have been asleep or fatigued versus whether a driver’s level of alertness or fatigue was unknown.

They take a sample of crashes and use a statistical modeling approach with multiple imputation to estimate drowsiness levels, concluding that 21% of all fatal crashes involved a drowsy driver.

So it seems like Walker’s claim is reasonable here. I can see how Guzey could’ve stopped at the NHTSA data, but this AAA report seems to be making some good points, and the 20% of fatal crashes involving drowsy driving seems as reasonable as any other number out there.

Summary

Based on his book and his Ted talk, it seems that Walker has a message to send, and he doesn’t care much about the details. He’s sloppy with sourcing, gets a lot wrong, and has not responded well to criticism.

But this does not mean we should necessarily dismiss his message. Ultimately his claims need to be addressed on their merits.