0:00:00.900,0:00:01.900
♪ [music] ♪

0:00:03.800,0:00:05.700
<i>- [Narrator] Welcome to [br]Nobel conversations.</i>

0:00:07.300,0:00:10.240
<i>In this episode, Josh Angrist [br]and Guido Imbens,</i>

0:00:10.240,0:00:12.000
<i>sit down with Isaiah Andrews</i>

0:00:12.000,0:00:14.303
<i>to discuss how the research[br]was initially received</i>

0:00:14.900,0:00:17.736
<i>and how they responded [br]to criticism.</i>

0:00:18.700,0:00:19.300
At the time, did you feel like[br]you are on to something,

0:00:20.400,0:00:24.000
you felt like this was [br]the beginning of a whole line of work

0:00:24.000,0:00:27.400
that you felt like was going to be important or...?

0:00:27.600,0:00:30.100
Not so much that it was[br]a whole line of work,

0:00:30.100,0:00:32.600
but certainly I felt like, "Wow, this--"

0:00:32.600,0:00:34.700
We proved something be proved up[br]that people didn't know before,

0:00:34.700,0:00:39.000
that it was worth knowing.

0:00:39.000,0:00:40.000
Yeah, going back compared to my

0:00:40.000,0:00:41.400
job market papers having--

0:00:41.600,0:00:45.900
I felt this was actually a very clear crisp result.

0:00:46.400,0:00:48.400
But there were definitely

0:00:48.900,0:00:53.200
was mixed reception and I don't[br]think anybody said that,

0:00:53.300,0:00:56.900
"Oh, wow, this is already,[br]something."

0:00:57.100,0:00:59.600
No, which is the nightmare scenario for a researcher

0:01:00.300,0:01:03.200
where you think you've discovered[br]something and then somebody else,

0:01:03.300,0:01:04.800
says, "Oh, I knew that."

0:01:05.000,0:01:08.600
But there were definitely was a need to[br]convince people that this was worth knowing,

0:01:09.000,0:01:12.800
that instrumental variables estimates[br]a causal effect for compliers.

0:01:13.200,0:01:18.000
Yeah, but even though it[br]took a long time to convince

0:01:18.600,0:01:20.400
a bigger audience,

0:01:20.700,0:01:24.600
sometimes even fairly quickly, the[br]reception was pretty good

0:01:24.800,0:01:27.000
among a small group of people.

0:01:27.200,0:01:31.500
Gary, clearly liked it a lot from the beginning

0:01:31.800,0:01:34.600
and I remember, because at that point Josh had left for Israel,

0:01:34.700,0:01:37.400
but I remember explaining it to Don Ruben

0:01:37.600,0:01:43.700
and he was like, "Yeah, this really is something here."

0:01:43.800,0:01:47.200
Not right away though.[br]Don took some convincing.

0:01:47.500,0:01:48.400
By the time you got to Don,

0:01:48.500,0:01:51.500
there have been some back[br]and forth with him

0:01:51.800,0:01:53.500
and in correspondence actually.

0:01:53.700,0:01:56.700
But I remember at some[br]point getting a call or email from him

0:01:56.700,0:02:02.300
saying that he was sitting at the[br]airport in Rome

0:02:02.500,0:02:03.700
and looking at the paper and thinking,

0:02:03.700,0:02:07.000
"Yeah, no actually, you guys are onto something."

0:02:07.000,0:02:10.600
We were happy about but that[br]took longer than I think you remember.

0:02:10.800,0:02:12.500
Yeah, it wasn't right away

0:02:12.600,0:02:13.600
[laughter]

0:02:13.700,0:02:16.500
because I know that I was back in[br]Israel by the time that happened.

0:02:16.500,0:02:18.300
I'd left for Israel in the summer--

0:02:18.400,0:02:22.300
I was only at Harvard for two years. [br]We had that one year.

0:02:22.600,0:02:25.700
It is remarkable, I mean, that[br]one year was so fateful for us.

0:02:25.900,0:02:27.200
- [Guido] Yes.

0:02:28.500,0:02:30.200
I think we understood there was[br]something good happening,

0:02:30.200,0:02:34.000
but maybe we didn't think it was[br]life-changing, only in retrospect.

0:02:34.400,0:02:35.400
♪ [music] ♪

0:02:35.800,0:02:39.300
- [Isaiah] As you said, it sounds like a small group[br]of people were initially quite receptive,

0:02:39.300,0:02:41.000
perhaps took some time for

0:02:41.100,0:02:44.200
a broader group of people to come[br]around to

0:02:44.400,0:02:47.500
seeing the LATE framework[br]as a valuable way to look at the world.

0:02:47.700,0:02:50.000
I guess, in over the[br]course of that, did you

0:02:50.100,0:02:52.200
were their periods[br]where you thought,

0:02:52.300,0:02:53.200
maybe the people

0:02:53.300,0:02:55.800
saying this wasn't a useful way to[br]look at the world were right?

0:02:55.800,0:02:58.400
Did you get discouraged? [br]How did you think about?

0:02:58.400,0:03:00.900
I don't think I was discouraged[br]but the people who were saying

0:03:00.900,0:03:03.900
that we're smart people, well informed metricians,

0:03:05.000,0:03:08.000
sophisticated readers

0:03:08.900,0:03:11.800
and I think the substance[br]of the comment was,

0:03:11.800,0:03:15.600
this is not what econometrics is about.

0:03:16.300,0:03:20.700
Econometrics was being transmitted at that time was about structure.

0:03:21.300,0:03:24.700
There was this idea that[br]there's structure in the economy

0:03:25.100,0:03:27.100
and it's our job to discover it

0:03:27.200,0:03:31.200
and what makes it structure[br]is it's essentially invariant

0:03:31.900,0:03:34.800
and so we're saying, in the late theorem,

0:03:34.900,0:03:39.100
that every instrument produces[br]its own causal effect,

0:03:39.300,0:03:42.100
which is in contradiction to that[br]to some extent

0:03:42.400,0:03:45.300
and so that was where the tension was.

0:03:45.300,0:03:46.300
People didn't want to give up that idea.

0:03:46.300,0:03:47.700
Yeah, I remember

0:03:48.100,0:03:50.500
once people were started

0:03:51.200,0:03:56.100
arguing more more vocally against that,

0:03:56.900,0:04:00.700
that never really bothered me that much.

0:04:01.000,0:04:03.700
It seems clear that we had a result there

0:04:04.900,0:04:08.100
and it was somewhat controversial,

0:04:08.100,0:04:09.100
but controversial in a good way.

0:04:09.100,0:04:10.400
It was clear that people felt

0:04:10.700,0:04:12.800
they had to come out against it because--

0:04:14.100,0:04:17.500
Well, I think what we think it's good now

0:04:17.500,0:04:20.800
we might not have loved it at the time.

0:04:20.800,0:04:21.800
I remember being somewhat,[br]the more upset--

0:04:21.800,0:04:26.400
there was some dinner where someone said,

0:04:26.700,0:04:28.300
"No, no, that paper with Josh,

0:04:28.800,0:04:32.600
that was doing a disservice[br]to the profession."

0:04:32.600,0:04:34.400
We definitely had reactions like that.

0:04:34.800,0:04:38.200
At some level, that may be indicative of the culture

0:04:38.400,0:04:40.000
in general in economics at the time.

0:04:41.400,0:04:44.300
I thought back later, what if that'd happened now,

0:04:44.600,0:04:48.200
if I was a senior person sitting[br]in that conversation,

0:04:48.300,0:04:52.200
I would call that out because it[br]really was not appropriate--

0:04:53.000,0:04:54.200
- [Josh] But it wasn't so bad.

0:04:54.600,0:04:57.300
I think the criticism is--

0:04:57.700,0:05:01.500
It wasn't completely[br]misguided, it was maybe wrong.

0:05:01.800,0:05:04.700
No, no, but you can[br]say the paper is wrong

0:05:05.400,0:05:09.300
but it's saying that it's a disservice to[br]the profession, that's not really--

0:05:09.300,0:05:10.300
Personal.

0:05:10.300,0:05:14.700
Yes, and doing that, not to me, [br]but in front of my senior colleagues.

0:05:14.900,0:05:17.700
But nobody was saying the result was wrong

0:05:17.700,0:05:18.700
and I remember also,

0:05:18.700,0:05:20.900
some of the comments were thought-provoking

0:05:20.900,0:05:24.000
so we had some negative reviews,

0:05:24.400,0:05:26.300
I think on the average causal response paper.

0:05:26.500,0:05:30.600
Somebody said, "These compliers[br]you can't figure out who they are."

0:05:31.500,0:05:31.900
Right.

0:05:32.000,0:05:33.000
It's one thing to say

0:05:33.000,0:05:36.200
you're estimating the effect of[br]treatment on the treated or something like that.

0:05:36.200,0:05:38.400
You can tell me who's treated,

0:05:38.600,0:05:42.600
people in the CPS,[br]you can't tell me who's a complier.

0:05:42.800,0:05:46.800
So that was a legitimate challenge.

0:05:46.800,0:05:47.800
That's certainly fair and I can see why

0:05:47.800,0:05:53.700
that part made people[br]a little uneasy and uncomfortable.

0:05:53.800,0:05:54.200
Yeah.

0:05:54.300,0:05:56.400
But it's a at the same time

0:05:56.900,0:06:00.800
because it showed that you couldn't[br]really go beyond that,

0:06:01.500,0:06:05.500
it was very useful thing to realize.

0:06:05.500,0:06:06.500
I remember on the day,

0:06:06.500,0:06:11.900
we got to the key result that I was thinking,[br]"Wow, this is as good as it gets.

0:06:12.100,0:06:16.500
Here we actually have an insight[br]but clearly--"

0:06:17.500,0:06:21.000
And we had to sell it.

0:06:21.000,0:06:22.000
For quite a few years, we had to sell

0:06:22.000,0:06:24.800
and it's proven to be quite useful.

0:06:25.500,0:06:29.300
I don't think we understood that it[br]would be so useful at the time.

0:06:30.100,0:06:34.600
No, I did feel early on this was[br]a substantial insight.

0:06:34.600,0:06:35.600
- [Josh] Yeah we [learned] something.

0:06:35.600,0:06:40.400
But I did not think goals were there.

0:06:40.500,0:06:42.400
I don't think we were aiming for the Nobel.

0:06:42.650,0:06:43.650
[laughter]

0:06:43.900,0:06:46.300
We were very happy to get[br]that note in <i>Econometrica.</i>

0:06:47.600,0:06:48.600
♪ [music] ♪

0:06:49.900,0:06:52.800
- [Isaiah] Are there factors or are ways of approaching[br]problems that lead people to be better at

0:06:53.200,0:06:56.600
recognizing the good stuff and taking the[br]time to do it as opposed to dismissing it?

0:06:56.600,0:06:57.700
- [Josh] Sometimes I think it's helpful.

0:06:57.700,0:07:01.300
If you're trying to convince somebody[br]that you have something useful to say

0:07:01.900,0:07:04.500
and maybe they don't speak your language,

0:07:04.700,0:07:09.900
you might need to learn their language.

0:07:10.300,0:07:10.850
Yes. Yes, exactly.

0:07:10.850,0:07:11.400
That's what we did with Don,[br]we figured out how to--

0:07:12.200,0:07:17.600
I remember we had a very hard time explaining[br]the exclusion restriction to Don,

0:07:17.800,0:07:19.700
maybe rightfully so,

0:07:20.000,0:07:24.600
I think Guido and I eventually figured out[br]that it wasn't formulated very clearly,

0:07:25.400,0:07:29.700
and we came up with a way to do that in[br]the potential outcomes framework

0:07:29.700,0:07:32.700
that I think worked[br]for the three of us.

0:07:33.400,0:07:35.800
Yeah, well, it worked for[br]the bigger literature

0:07:35.800,0:07:38.100
but I think what you're saying there [br]is exactly right,

0:07:38.100,0:07:39.100
you need to figure out how not just say, okay well, I've got this language

0:07:41.400,0:07:43.900
and this this works great

0:07:43.900,0:07:45.900
and I've got to convince someone[br]else to use the language.

0:07:45.900,0:07:47.600
You could first figure out what language they're using

0:07:49.600,0:07:51.200
and then only then, can you try to say,

0:07:51.200,0:07:55.700
"Wow, but here you thinking of it this way,"

0:07:55.700,0:07:56.700
but that's actually a pretty hard thing to do,

0:07:56.700,0:08:00.000
get someone from a different discipline, [br]convincing them,

0:08:00.200,0:08:03.300
two junior faculty in a different department [br]actually have something to say to you,

0:08:03.300,0:08:06.600
that's that takes a fair amount of effort.

0:08:07.500,0:08:10.200
Yeah, I wrote on a number of times, in fairly long letters.

0:08:10.700,0:08:14.500
I remember thinking this is worth doing,

0:08:14.600,0:08:17.600
that if I could convince Don [br]that would validate the framework to some extent.

0:08:20.300,0:08:25.000
I think both you and Don were a little[br]bit more confident that you were right.

0:08:25.000,0:08:28.200
Well, we used to argue a lot and[br]you would sometimes referee those.

0:08:28.500,0:08:29.500
[laughter]

0:08:29.800,0:08:30.700
That was fun.

0:08:30.800,0:08:34.100
It wasn't hurtful.

0:08:35.200,0:08:39.800
I remember getting a little testy once,[br]we had lunch in The Faculty Club

0:08:40.600,0:08:44.400
and we're talking about the draft lottery paper.

0:08:45.200,0:08:47.600
we were talking about never takes

0:08:47.700,0:08:51.000
as people wounded serve in[br]the military irrespective of

0:08:51.200,0:08:53.700
whether they were getting drafted

0:08:54.500,0:08:58.700
and you and Don said something[br]about shooting yourself in the foot,

0:08:59.800,0:09:03.400
as a way of getting out of the military[br]and that may be the exclusion restriction

0:09:03.400,0:09:05.300
for never takes wasn't working

0:09:06.300,0:09:08.900
and then the other one would say,

0:09:08.900,0:09:12.100
"Well, yes you could do that [br]but why would you want to shoot yourself in the foot?"

0:09:12.250,0:09:12.825
[laughter]

0:09:12.825,0:09:13.400
It got a little [inaudible] there.

0:09:13.400,0:09:17.600
I usually go for moving to Canada for my example[br]when I'm teaching that.

0:09:19.700,0:09:24.000
But he thinks it's tricky,

0:09:24.100,0:09:27.900
I get students coming[br]from computer science and they want to do

0:09:28.100,0:09:29.900
things on causal inference

0:09:30.400,0:09:33.700
and it takes a huge amount of[br]effort to figure out how

0:09:33.700,0:09:36.800
they actually thinking about problem[br]and whether there's something there

0:09:37.000,0:09:38.500
and so, now over the years,

0:09:38.500,0:09:40.600
I've got a little more[br]appreciation for the fact that Don

0:09:40.800,0:09:42.100
was actually willing to--

0:09:42.200,0:09:46.000
It took him a while, but he did[br]engage first with Josh

0:09:46.400,0:09:47.500
and then with both of us

0:09:47.700,0:09:51.000
and rather than dismissing[br]and say,

0:09:51.500,0:09:55.700
"Well, okay I can't figure out what these guys are[br]doing and it's probably just not really that

0:09:55.800,0:09:56.700
that interesting."

0:09:57.200,0:10:00.300
Everybody always wants to[br]figure out quickly,

0:10:00.300,0:10:04.300
you want to save time and you want to save[br]your brain cells for other things.

0:10:05.000,0:10:08.000
The fastest route to that is to figure[br]out why you should dismiss something.

0:10:08.000,0:10:09.000
Yes.

0:10:09.000,0:10:11.300
I don't need to spend time on this.

0:10:11.500,0:10:12.500
♪ [music] ♪

0:10:12.700,0:10:15.900
<i>- [Narrator] If you'd like to watch more[br]Nobel conversations click here,</i>

0:10:16.300,0:10:18.600
<i>or if you'd like to learn[br]more about econometrics,</i>

0:10:18.700,0:10:21.300
<i>check out Josh's "Mastering[br]Econometrics" series.</i>

0:10:21.800,0:10:24.700
<i>If you'd like to learn more[br]about Guido, Josh, and Isaiah</i>

0:10:24.900,0:10:26.400
<i>check out the links in the description.</i>

0:10:26.702,0:10:28.017
♪ [music] ♪