Janelle Shane @JanelleCShane I blog at aiweirdness.com/. My book "You Look Like a Thing and I Love You" is out Nov 5th! Research Scientist in optics. she/her. wandering.shop/@janellecshane Oct. 17, 2019 1 min read

Very effective interactive demo of how human bias in training data affects any algorithm that tries to use it.

Especially in the US, predicting who will commit a crime is not the same as predicting who will be arrested. And here all the AI has is arrest data.

I talk about this example in my book too. Neatly illustrates how if a machine learning algorithm tackles a problem like parole decisions without knowing about the bias in its data (and how can it? it doesn't know what's going on), it'll treat the bias as valid.

One solution to building fairer algorithms is to change the training data to reflect the world as it ought to be - in other words, to debias the data.

Here, one could adjust assuming a fair justice system would have produced equal arrest rates for Black vs white defendants.

The demo in the article shows how doing that might produce an algorithm that looks unfair when you look at its rearrest rates for Black vs white defendants - but that's only because the real-world arrest rate itself has bias built in.

As is often the case, it's so important to look carefully at what the AI is really being asked to predict.

Here, it's important to understand the difference, especially in the US, between the rates at which people commit crimes vs the rates at which they're arrested.

The algorithm doesn't know the difference. All it's doing is predicting human behavior based on the numbers it's given.

Making it predict the fair behavior we want, vs the biased behavior we have, requires careful work.

I talk more about the extreme literalmindedness & general obliviousness of machine learning algorithms in my book, You Look Like a Thing and I Love You: How AI Works and Why It's Making the World a Weirder Place  https://www.janelleshane.com/book-you-look-like-a-thing 


You can follow @JanelleCShane.



Bookmark

____
Tip: mention @threader_app on a Twitter thread with the keyword “compile” to get a link to it.

Enjoy Threader? Sign up.

Threader is an independent project created by only two developers. The site gets 500,000+ visits a month and our iOS Twitter client was featured as an App of the Day by Apple. Running this space is expensive and time consuming. If you find Threader useful, please consider supporting us to make it a sustainable project.