“Deciphering the Neighborhood Atlas Area Deprivation Index: The Consequences of Not Standardizing”

Steve Petterson writes:

Given your interest/concerns about the Area Deprivation Index (ADI), I thought you would be interested in my paper that has just been accepted by Health Affairs Scholar. The supplemental materials include much more details about the problems with the ADI.

The main results have been replicated by a team at Stanford (affiliated with Bob Phillips) and Northwestern (affiliated with Bernard Black). Through Bob, CMS and ASPE are aware of my findings (that build on your findings).

So you know, I was the analyst behind the Social Deprivation Index (SDI) that you used in your maps for your blog.

From Petterson’s new article:

The Area Deprivation Index is a widely used measure recently selected for several Federal payment models that adjust payments based on where beneficiaries live. A recent debate in Health Affairs focuses on seemingly implausible ADI rankings in major cities and across New York. At the root of the issue is the importance of standardization of measures prior to calculating index scores. . . . The main finding is that without standardization, the ADI is reducible to a weighted average of just two measures—income and home values, certainly not the advertised multidimensional measure. . . .

This last bit resonates with me, when an index is created that nominally uses all sorts of information but actually is based on very little. Similar issues arose with the notorious Electoral Integrity Project (where the data on North Korea were based on respondents from something like 3 people) and that Philadelphia crime analysis (where the trends in that city were compared to a weighted average from only 3 other cities) and various other indexes and measures floating around.

From my perspective, the big issue here with the Area Deprivation Index is not standardization so much as the problem of people uncritically using a “Deprivation Index” without thinking hard about what goes into it.