WEBVTT

00:00:02.240 --> 00:00:05.490
[Music]

00:00:07.560 --> 00:00:10.160
hi uh my name is Deborah rajie and uh

00:00:10.160 --> 00:00:12.040
I'm a Milla fellow I work with the

00:00:12.040 --> 00:00:14.120
algorithmic Justice League so the

00:00:14.120 --> 00:00:17.160
algorithmic Justice League is a research

00:00:17.160 --> 00:00:19.600
organization um that works very hard to

00:00:19.600 --> 00:00:22.160
make sure that AI is developed in a way

00:00:22.160 --> 00:00:26.199
that is inclusive and effective for

00:00:28.599 --> 00:00:31.000
everyone right now a lot of our work has

00:00:31.000 --> 00:00:33.360
also involved doing audits ourselves of

00:00:33.360 --> 00:00:35.879
these deployed systems so we analyze

00:00:35.879 --> 00:00:38.120
situations um like I mentioned anything

00:00:38.120 --> 00:00:40.200
from like healthcare to hiring to facial

00:00:40.200 --> 00:00:42.399
recognition what we do is we come into

00:00:42.399 --> 00:00:43.960
those situations and we try to

00:00:43.960 --> 00:00:45.760
understand how the deployment of that

00:00:45.760 --> 00:00:49.879
system impacts different marginalized

00:00:52.680 --> 00:00:55.199
groups is a project called gender Shades

00:00:55.199 --> 00:00:57.680
where we looked at uh facial recognition

00:00:57.680 --> 00:00:59.079
systems that were deployed in the real

00:00:59.079 --> 00:01:01.600
world and asked the question of is this

00:01:01.600 --> 00:01:03.800
A system that works for everyone these

00:01:03.800 --> 00:01:06.080
systems although they were operating at

00:01:06.080 --> 00:01:08.799
almost 100% for for example lighter

00:01:08.799 --> 00:01:12.799
skinned um male faces um they were uh

00:01:12.799 --> 00:01:15.280
performing at less than 70% accuracy for

00:01:15.280 --> 00:01:17.960
darker skinned women um this was a huge

00:01:17.960 --> 00:01:20.720
story and it kind of escalated uh in the

00:01:20.720 --> 00:01:22.400
press and and that's a lot of what we're

00:01:22.400 --> 00:01:25.320
known for is that

00:01:27.759 --> 00:01:30.880
project so you might have um a company

00:01:30.880 --> 00:01:34.159
that builds a tool for doctors or for

00:01:34.159 --> 00:01:36.280
teachers um whereas the affected

00:01:36.280 --> 00:01:38.119
population in that situation would

00:01:38.119 --> 00:01:40.759
actually be the students or the patients

00:01:40.759 --> 00:01:43.399
and those guys very rarely have any kind

00:01:43.399 --> 00:01:46.200
of influence on the types of features

00:01:46.200 --> 00:01:48.119
that are emphasized in the development

00:01:48.119 --> 00:01:50.560
of the AI system uh the type of data

00:01:50.560 --> 00:01:54.240
that's collected uh and as a result um

00:01:54.240 --> 00:01:56.640
those that are sort of experiencing the

00:01:56.640 --> 00:01:58.640
weight of the decision-making that these

00:01:58.640 --> 00:02:03.159
tools make uh end up almost uh erased

00:02:03.159 --> 00:02:04.920
from the entire process of development

00:02:04.920 --> 00:02:08.200
unless actively sought

00:02:10.760 --> 00:02:13.440
out yeah so there's a lot of situations

00:02:13.440 --> 00:02:15.319
in which humans are making very

00:02:15.319 --> 00:02:17.760
important decisions uh an example being

00:02:17.760 --> 00:02:20.239
hiring or a judge making a decision in a

00:02:20.239 --> 00:02:22.560
criminal case and there's certainly a

00:02:22.560 --> 00:02:24.440
lot of bias involved in that there's a

00:02:24.440 --> 00:02:26.599
lot of the perspective of that person

00:02:26.599 --> 00:02:28.480
making that decision that influences the

00:02:28.480 --> 00:02:30.840
nature of that outcome in the same way

00:02:30.840 --> 00:02:33.000
if you replace that human decision maker

00:02:33.000 --> 00:02:34.879
with an algorithm there's bound to be

00:02:34.879 --> 00:02:37.040
some level of bias involved in that the

00:02:37.040 --> 00:02:38.959
other sort of aspect of this is that we

00:02:38.959 --> 00:02:41.239
tend to trust algorithms and see them as

00:02:41.239 --> 00:02:45.360
neutral in a way that we don't with

00:02:48.560 --> 00:02:51.400
humans yeah so I got into this field

00:02:51.400 --> 00:02:54.280
almost accidentally um I studied

00:02:54.280 --> 00:02:56.959
robotics Engineering in University and I

00:02:56.959 --> 00:03:00.640
was sort of playing a lot with um

00:03:00.640 --> 00:03:04.120
uh AI as like just a form of of of part

00:03:04.120 --> 00:03:05.920
of my experience in terms of coding and

00:03:05.920 --> 00:03:07.080
and my experience in hackathons and

00:03:07.080 --> 00:03:09.280
building projects and realize very

00:03:09.280 --> 00:03:11.280
quickly that a lot of the data sets for

00:03:11.280 --> 00:03:13.560
example um do not include a lot of

00:03:13.560 --> 00:03:14.879
people that look like me so a lot of the

00:03:14.879 --> 00:03:17.360
data sets that we use to uh you know to

00:03:17.360 --> 00:03:18.840
to pretty much teach these algorithmic

00:03:18.840 --> 00:03:21.239
systems uh what a face looks like what a

00:03:21.239 --> 00:03:23.840
hand looks like what a human looks like

00:03:23.840 --> 00:03:25.799
um don't actually include uh a lot of

00:03:25.799 --> 00:03:28.200
people of color um and other different

00:03:28.200 --> 00:03:31.080
demographics so that was is probably the

00:03:31.080 --> 00:03:33.720
biggest uh sort of red flag that I saw

00:03:33.720 --> 00:03:36.360
in the industry

00:03:38.599 --> 00:03:41.040
immediately um I think a lot of the

00:03:41.040 --> 00:03:43.799
times we think of AI systems as these

00:03:43.799 --> 00:03:46.840
sci-fi sentient robot

00:03:46.840 --> 00:03:49.879
overlords um but they're really just a

00:03:49.879 --> 00:03:52.120
bunch of decisions being made by actual

00:03:52.120 --> 00:03:55.439
humans and um our understanding of AI

00:03:55.439 --> 00:03:57.120
systems as the separate thing makes it

00:03:57.120 --> 00:04:00.000
really hard to hold anyone accountable

00:04:00.000 --> 00:04:04.280
when a bad decision is made

00:04:04.680 --> 00:04:08.879
[Music]