Bias detectives: the researchers striving to make algorithms fair

One big obstacle to accountability is that agencies often do not collect data on how the tools are used or their performance, says Goel. “A lot of times there’s no transparency because there’s nothing to share.” The California legislature, for instance, has a draft bill that calls for risk-assessment tools to help reduce how often defendants must pay bail — a practice that has been criticized for penalizing lower-income defendants. Goel wants the bill to mandate that data are collected on instances when judges disagree with the tool and on specific details, including outcomes, of every case. “The goal is fundamentally to decrease incarceration while maintaining public safety,” he says, “so we have to know — is that working?”