The research on SAT bias is great data science

I just put up a follow-up video on the SAT controversy, including a speculation about what might happen if the SAT were to be toppled from college admission. Link to video.

***

One of the key arguments against the SAT is its alleged bias against certain subgroups of test-takers. There has been a lot of research into ways to measure and combat such biases. Chapter 5 of Numbers Rule Your World (link) gets into the science behind this.

In the video, I discuss the following example featured also in the book

Is this test item biased against some subgroup? If so, which subgroup, by how much, and for what reason? It turns out that it is basically impossible to answer these questions purely by intuition. Test designers run experiments to measure the bias, and are often surprised by what they find.

This whole area of research is a great example of a practical application of data science. The issues range from how to collect the right data, and how to analyze such data to make causal conclusions. You can read about it in my book (link).

Or try the video first.