- As you can imagine, a company like Amazon
receives hundreds of applications for each opening position.
So in 2014, they created a human resources system
to help them find the best candidates.
The system would process these applications
and then rate each one with a five star rating system.
It was designed to be sent simple and elegant.
Each current employee was ranked by their manager
using a five star system.
Then they looked at all the five star employees
and had a machine learning algorithm
look for patterns in their original applications.
Then the algorithm would look for similar patterns
in the new job applications.
In 2015, they started using the system
on all their new job applicants.
The problem was that about 60% of Amazon employees were men.
So there were more five star male employees.
The machine learning algorithm learned
that males are better applicants.
And so the system began the strongly favor men.
That meant that the system was penalizing applicants
who listed things like women's chess club
or attended traditional women's colleges.
The machine learning algorithm reinforced this bias.
And so the vast majority
of the new five star applicants were men.
Eventually, the system became so biased
that they had to shut it down.
Now Amazon decided to throw away the system
instead of trying to fix the machine learning algorithm.
But instead, let's imagine that they hired you
to help understand the data ethics issues.
You could look at it using the deontological approach
of Immanuel Kant.
Remember that Kant puts a categorical imperative
on always telling the truth.
You have a duty to tell the truth
no matter the consequences.
The machine learning algorithm wasn't lying.
It was simply learning the bias in Amazon's hiring process.
Then it used that bias to hire new candidates.
In a sense it was getting into trouble
because it was doing too good of a job
at telling the truth.
To replace the truth with a lie,
just because you don't like the consequences,
takes power away from Amazon to better understand the truth.
Plus if you penalize men to make it fair,
then you would be limiting their autonomy,
which is another categorical imperative.
So Kant might argue that you're penalizing men
by removing the objectivity and the data
and tilting the scales in favor of women.
Now, obviously there are a lot of people
who would be concerned about the consequences
of allowing this bias to continue.
If you're a woman, you're not as concerned
about Amazon seeing the truth.
You just want to make sure that you have a fair opportunity
to get hired for the job.
So as a consultant, you might argue that Amazon
should try a more utilitarian approach.
There are many advantages
to having a more diverse workplace.
Studies show that they're less prone to groupthink
and have fewer blind spots.
So a utilitarian approach would be to fix the algorithm
so it has less bias.
Finally, virtue ethics looks at the motivations
behind a virtuous person.
A virtuous person tries to eliminate inequality
because it's the right thing to do.
That means you have to stop using the algorithm
until it's fair for everyone.