The Challenges of Creating Equitable Algorithms

Irineo Cabreros examines the trade-offs to consider when designing algorithms to be equitable.
How can algorithms be made both accurate and equitable?
Read about using AI for social good.

Late last year, the Justice Department joined the growing list of agencies to discover that algorithms don't heed good intentions. An algorithm known as PATTERN placed tens of thousands of federal prisoners into risk categories that could make them eligible for early release. The rest is sadly predictable: Like so many other computerized gatekeepers making life-altering decisions—presentencing decisions, resume screening, even health care needs—PATTERN seems to be unfair, in this case to Black, Asian, and Latino inmates.

A common explanation for these misfires is that humans, not equations, are the root of the problem. Algorithms mimic the data they are given. If that data reflect humanity's sexism, racism, and oppressive tendencies, those biases will be baked into the algorithm's predictions.

But there is more to it. Even if all the shortcomings of humanity were stripped away, equity would still be an elusive goal for algorithms for reasons that have more to do with mathematical impossibilities than backward ideologies. In recent years, a growing field of research in algorithmic equity has revealed fundamental—and insurmountable—limits to equity. The research has deep implications for any decisionmaker, human or machine.

Imagine two physicians. Dr. A graduated from a prestigious medical school, is up on all the latest research, and carefully tailors her approach to each patient's needs. Dr. B takes one cursory glance at every patient, says “you're fine,” and mails them a bill.

If you had to pick a doctor, the decision might seem obvious. But Dr. B has one redeeming attribute. In a sense, she is more fair: Everyone is treated the same.

Read the full article about algorithms and equity by Irineo Cabreros at RAND Corporation.