Machine Learning Shows us that Systemic Racism is Very Real

With the recent events around George Floyd; IBM, Amazon and Microsoft have all moved out of the facial recognition business in the last week citing the likelihood of them being used for bias Three Big Tech Players Back Out of Facial Recognition Market. They have done this for the same reason I give when asked about bias in ML algorithms, the only way to stop it is to starve the algorithm of the data that would lead to the bias being formed. These algorithms by design can lead to the classification of people by race, sex or any other characteristic that we can tell by looking at people.

Just a quick note on bias vs. racism. The terms systemic racism or institutional racism can probably be replaced with systemic bias. That is to say there may not be a conscious effort to disadvantage a particular group but from the perspective of a member of that group, such a fine distinction is probably immaterial. Either way, they don't get the job or get treated a certain way by the legal system, etc. A claim of, "It's not racism, it's just bias" is likely a cold comfort.

Without going into the technical weeds on how machine learning works, suffice it to say it is modeled after how our brains learn, by forming patterns. Based on a history of data, it makes predictions on future events. For the most part those future predictions are made with the assumption that the status quo will be maintained. That is so say, if A led to B in the past fifty times, A will still lead to B the next time it happens.

Let's take a very simple example of what such an algorithm would look like if we wanted to say, make predictions on what job candidates would be successful in the tech industry. We feed the training algorithm with information from past candidates like, were they hired, what was in their resumes, their names, their pictures, their age, starting salaries if hired, etc. Based on the makeup of today's tech industry, what conclusions will such a model likely come to on what makes up a successful candidate?

- Likely some education and technical experience, that's all good
- White males are more likely to be successful candidates. Uh oh. We can argue how this came to be, but the current industry is dominated by white males and they are currently the most successful candidates, so that will be picked up
- In some cases minorities and women can be paid less doh!
- People with "white and male sounding" names are more likely to be successful
- People who live in more affluent neighborhoods and suburbs make candidates more likely to be successful
- Most viable candidates are likely between, eh 30-50 years old

Feed the trained model data on new candidates and all other things being equal, the model is likely to conclude that 30-50 year old white males are the ones to hire. Now is that model racist? Is it biased? Are the people using the model even aware of how it is coming to its conclusions under the hood? I'm not sure the answer to any of these questions is per se important. What is important are the results.

Edit to add: based on some feedback, I want to make sure to point out my hiring in the tech industry example is not completely theoretical. Amazon scraps secret AI recruiting tool that showed bias against women

How does this relate to systemic racism? It relates because these models learn in very similar ways to how we learn. Our learning and experience, consciously or unconsciously, trains us to reinforce the status quo. Past experience is what our minds use to give us conclusions for future events. If our past experience shows us that it is mostly white males are who we interacted with successfully in the tech industry, we will form the impression that is what will most likely work in the future. Not necessarily purposely, or even consciously, but that's the most likely conclusion that our brains will reach, regardless of your race or sex.

Our minds can't do what I suggested to students with bias, that is to say we can't simply starve our brain of the data that would lead to biased outcomes. This is why people that say they "don't see color" should rethink their position. Like my hypothetical job applicant model above, we definitely see color, race, sex, you name it. Unlike ML that can be starved of certain data, we can't choose to not see it or not hear it. All kinds of factors are being fed into our brains every single day and in every single interaction.

The important thing to realize is that without conscious effort to change things, our minds will default to coming to conclusions that are supported by past experience. What is successful today is what will be successful tomorrow. Problems I see today will be the problems I see tomorrow. Like machine learning, it's just the way our minds work. If we are not conscious of the fact that our minds are coming to conclusions based on that data then we will forever be locked into our current cycle. We have to be very cognizant of what is happening and why we are likely getting certain impressions on people. It needs to be a conscious effort to really analyze what data we have available to us on a situation by situation basis. If we just allow our minds to stick with their first impressions, there may be all kinds of immaterial biases at play.

BTW, all of our minds work like this. This means that minorities and women are just as likely to be unconsciously biased against themselves. The more I work with machine learning, the more inescapable this conclusion has become.