Left-NIMBYs Take Another Loss — And This One Is Really Embarrassing

As I wrote last year, the Urban Institute produced a study purporting to show that land use reforms did not substantially increase housing production. I noted that the study has become very influential, particularly among Left-NIMBYs, who want to maintain strict land use controls but not acknowledge that such controls make housing more expensive (which obviously impacts low- and middle-income people the most). If you don’t like the term “Left-NIMBYs,” then use “supply skeptics”: they reject the notion that allowing more housing construction will stabilize prices.

I expressed a lot of my own skepticism about this study, mainly because of the data quality issues. The study purports to show that jurisdictions that have enacted “land use reforms” have built virtually no new housing over the baseline – only 0.8% more. The problem is that “land use reforms” can mean about a million different things, so the number really means very little.

Well, others have looked more closely at the study, and as it turns out, that was the least of its problems. The Urban Institute study used an innovative method to figure out whether jurisdictions had enacted land use reforms: an AI tool trawling through newspapers for references to them. But then the center-right American Enterprise Institute looked more carefully at those data points. And it found something very very ugly:

Our independent review reveals that 60 of these articles should be disqualified outright due to duplication, incorrect geographic attribution, or policies that only affected commercial or industrial areas or properties. Among the remaining 120 entries, 118 are either incorrectly classified in direction (more vs. less restrictive), are either not major municipality-wide reforms, or have insufficient information to accurately determine policy direction. After a thorough and painstaking review of all 180 cases, we found only two that plausibly qualify as “major” reforms—and even these warrant caution given uncertainty about whether their supply impact was measurable using Stacy et al.’s methodology.

My gob is smacked. How in the world could anyone produce something this sloppy? Ned Resnikoff (whose emphasized the lines in the previous passage) comments:

No less than a fifth of the 180 land use changes were “attributed to the wrong city.” For example, the Urban Institute researchers said that a height limit increase in Aventura, Florida had actually taken effect in Miami. Why? Because the article about this increase appeared in the Miami Herald. (The article’s lead sentence is: “Developers who had plans to build in Aventura on hold can now proceed, but face stricter zoning laws.”).

Put another way, this very-cited study is a hot flaming piece of garbage.

When I was in practice, a senior attorney told me: “Do the easy things right.” Proofreading (even with spellcheck), being on time, etc. etc. Well, here’s another one: check the computer’s work. It was obvious even last year that there were significant problems with relying on AI, and these guys didn’t bother to do it. In a reply to the AEL critique, the Urban Institute researchers say that they had human beings check the results. That makes it even worse!

Resnikoff thinks that they were so convinced that their program was right, they didn’t check. The cynic in me suggests that maybe it’s that they got great results and didn’t really want to find out if they were right. And now they have egg on their face.

Or maybe not. AEI of course covered their own study. Resnikoff wrote about it. But at least a Google search does not reveal any press coverage about such a spectacular failure – and importantly, one that has dominated Left-NIMBYist discourse. Perhaps this flameout will just mean that people won’t cite it anymore.

But maybe not. Scholars and popular writers routinely cite books that have been comprehensively debunked because they tell the story that the scholars and writers want to tell. Attempts to set the record straight (as I have tried to do in areas like water policy and fair housing) get the worst academic treatment: they are not attacked, but simply ignored.

I wonder if the same thing will happen here. Lots of people want to tell the supply skepticism story. So it will be told.