AI Auditing with Deborah Raji

[Music]

hi uh my name is Deborah rajie and uh

I'm a Milla fellow I work with the

algorithmic Justice League so the

algorithmic Justice League is a research

organization um that works very hard to

make sure that AI is developed in a way

that is inclusive and effective for

everyone right now a lot of our work has

also involved doing audits ourselves of

these deployed systems so we analyze

situations um like I mentioned anything

from like healthcare to hiring to facial

recognition what we do is we come into

those situations and we try to

understand how the deployment of that

system impacts different marginalized

groups is a project called gender Shades

where we looked at uh facial recognition

systems that were deployed in the real

world and asked the question of is this

A system that works for everyone these

systems although they were operating at

almost 100% for for example lighter

skinned um male faces um they were uh

performing at less than 70% accuracy for

darker skinned women um this was a huge

story and it kind of escalated uh in the

press and and that's a lot of what we're

known for is that

project so you might have um a company

that builds a tool for doctors or for

teachers um whereas the affected

population in that situation would

actually be the students or the patients

and those guys very rarely have any kind

of influence on the types of features

that are emphasized in the development

of the AI system uh the type of data

that's collected uh and as a result um

those that are sort of experiencing the

weight of the decision-making that these

tools make uh end up almost uh erased

from the entire process of development

unless actively sought

out yeah so there's a lot of situations

in which humans are making very

important decisions uh an example being

hiring or a judge making a decision in a

criminal case and there's certainly a

lot of bias involved in that there's a

lot of the perspective of that person

making that decision that influences the

nature of that outcome in the same way

if you replace that human decision maker

with an algorithm there's bound to be

some level of bias involved in that the

other sort of aspect of this is that we

tend to trust algorithms and see them as

neutral in a way that we don't with

humans yeah so I got into this field

almost accidentally um I studied

robotics Engineering in University and I

was sort of playing a lot with um

uh AI as like just a form of of of part

of my experience in terms of coding and

and my experience in hackathons and

building projects and realize very

quickly that a lot of the data sets for

example um do not include a lot of

people that look like me so a lot of the

data sets that we use to uh you know to

to pretty much teach these algorithmic

systems uh what a face looks like what a

hand looks like what a human looks like

um don't actually include uh a lot of

people of color um and other different

demographics so that was is probably the

biggest uh sort of red flag that I saw

in the industry

immediately um I think a lot of the

times we think of AI systems as these

sci-fi sentient robot

overlords um but they're really just a

bunch of decisions being made by actual

humans and um our understanding of AI

systems as the separate thing makes it

really hard to hold anyone accountable

when a bad decision is made

[Music]