1
00:00:00,900 --> 00:00:02,760
♪ [music] ♪

2
00:00:03,800 --> 00:00:05,700
<i>- [Narrator] Welcome to 
Nobel conversations.</i>

3
00:00:07,300 --> 00:00:10,240
<i>In this episode, Josh Angrist 
and Guido Imbens,</i>

4
00:00:10,240 --> 00:00:12,000
<i>sit down with Isaiah Andrews</i>

5
00:00:12,000 --> 00:00:14,303
<i>to discuss how the research
was initially received</i>

6
00:00:14,900 --> 00:00:17,090
<i>and how they responded 
to criticism.</i>

7
00:00:18,700 --> 00:00:21,540
At the time, did you feel like
you are on to something,

8
00:00:21,540 --> 00:00:25,152
you felt this was the beginning
of a whole line of work

9
00:00:25,152 --> 00:00:27,202
that you felt like was going 
to be important or...?

10
00:00:27,600 --> 00:00:30,100
Not so much that it was
a whole line of work,

11
00:00:30,100 --> 00:00:32,417
but certainly I felt like, 
"Wow, this--"

12
00:00:32,417 --> 00:00:35,045
- [Josh] We proved something 
that people didn't know before,

13
00:00:35,045 --> 00:00:36,430
that it was worth knowing.

14
00:00:36,430 --> 00:00:41,143
Yeah, going back compared to 
my job market papers having--

15
00:00:41,600 --> 00:00:45,560
I felt this was actually 
a very clear crisp result.

16
00:00:46,400 --> 00:00:49,530
But there were definitely 
was mixed reception

17
00:00:49,530 --> 00:00:52,420
and I don't think anybody 
said that,

18
00:00:52,420 --> 00:00:55,461
"Oh, wow, this is already,
something."

19
00:00:55,461 --> 00:00:59,386
No, which is the nightmare scenario
for a researcher

20
00:01:00,300 --> 00:01:02,259
where you think you've 
discovered something

21
00:01:02,283 --> 00:01:04,461
and then somebody else says, 
"Oh, I knew that."

22
00:01:05,000 --> 00:01:07,412
But there were definitely was 
a need to convince people

23
00:01:07,430 --> 00:01:10,346
that this was worth knowing,
that instrumental variables

24
00:01:10,370 --> 00:01:12,712
estimates a causal effect 
for compliers.

25
00:01:13,200 --> 00:01:16,178
Yeah, but even though it
took a long time

26
00:01:16,178 --> 00:01:19,348
to convince a bigger audience,

27
00:01:19,820 --> 00:01:24,346
sometimes even fairly quickly, 
the reception was pretty good

28
00:01:24,800 --> 00:01:26,645
among a small group of people.

29
00:01:27,200 --> 00:01:31,297
Gary, clearly liked it a lot 
from the beginning

30
00:01:31,800 --> 00:01:35,645
and I remember, because at that point
Josh had left for Israel,

31
00:01:35,645 --> 00:01:38,886
but I remember explaining it 
to Don Ruben

32
00:01:39,696 --> 00:01:43,700
and he was like, 
"Yeah, this really is something here."

33
00:01:43,700 --> 00:01:47,100
Not right away though,
Don took some convincing.

34
00:01:47,500 --> 00:01:49,210
By the time you got to Don,

35
00:01:49,210 --> 00:01:51,226
there have been some back
and forth with him

36
00:01:51,226 --> 00:01:53,304
and in correspondence actually.

37
00:01:53,700 --> 00:01:57,103
But I remember at some point 
getting a call or email from him

38
00:01:57,103 --> 00:02:00,020
saying that he was sitting 
at the airport in Rome

39
00:02:00,020 --> 00:02:03,700
and looking at the paper 
and thinking,

40
00:02:03,700 --> 00:02:07,000
"Yeah, no actually, 
you guys are onto something."

41
00:02:07,490 --> 00:02:08,744
We were happy about

42
00:02:08,744 --> 00:02:10,670
but that took longer 
than I think you remember.

43
00:02:10,800 --> 00:02:12,500
Yeah, it wasn't right away

44
00:02:12,500 --> 00:02:13,700
[laughter]

45
00:02:13,700 --> 00:02:16,500
because I know that I was back 
in Israel by the time that happened.

46
00:02:16,500 --> 00:02:18,400
I'd left for Israel in the summer--

47
00:02:19,390 --> 00:02:22,600
I was only at Harvard for two years.
We had that one year.

48
00:02:22,600 --> 00:02:25,700
It is remarkable, I mean, that
one year was so fateful for us.

49
00:02:25,900 --> 00:02:27,200
- [Guido] Yes.

50
00:02:27,690 --> 00:02:30,200
I think we understood there was
something good happening,

51
00:02:30,200 --> 00:02:34,000
but maybe we didn't think it was
life-changing, only in retrospect.

52
00:02:34,000 --> 00:02:35,720
♪ [music] ♪

53
00:02:35,720 --> 00:02:37,495
- [Isaiah] As you said, it sounds like 
a small group of people

54
00:02:37,495 --> 00:02:39,190
were initially quite receptive,

55
00:02:39,190 --> 00:02:42,190
perhaps took some time for
a broader group of people

56
00:02:43,090 --> 00:02:46,022
to come around to seeing 
the LATE framework

57
00:02:46,022 --> 00:02:47,700
as a valuable way to look 
at the world.

58
00:02:47,700 --> 00:02:50,100
I guess, in over the course of that,

59
00:02:50,100 --> 00:02:51,888
were their periods 
where you thought,

60
00:02:51,888 --> 00:02:54,290
maybe the people saying 
this wasn't a useful way

61
00:02:54,290 --> 00:02:55,751
to look at the world were right?

62
00:02:55,751 --> 00:02:58,400
Did you get discouraged? 
How did you think about?

63
00:02:58,400 --> 00:03:00,900
I don't think I was discouraged
but the people who were saying

64
00:03:00,900 --> 00:03:05,900
that we're smart people, 
well informed econometricians,

65
00:03:05,900 --> 00:03:07,800
sophisticated readers

66
00:03:08,900 --> 00:03:11,800
and I think the substance
of the comment was,

67
00:03:11,800 --> 00:03:14,297
this is not what econometrics 
is about.

68
00:03:14,297 --> 00:03:20,572
Econometrics was being transmitted 
at that time was about structure.

69
00:03:21,300 --> 00:03:24,290
There was this idea that
there's structure in the economy

70
00:03:25,100 --> 00:03:27,200
and it's our job to discover it

71
00:03:27,200 --> 00:03:30,952
and what makes it structure
is it's essentially invariant

72
00:03:31,900 --> 00:03:34,900
and so we're saying, 
in the LATE theorem,

73
00:03:34,900 --> 00:03:37,699
that every instrument produces
its own causal effect,

74
00:03:37,699 --> 00:03:41,386
which is in contradiction to that
to some extent

75
00:03:41,386 --> 00:03:43,802
and so that was 
where the tension was.

76
00:03:43,802 --> 00:03:45,551
People didn't want 
to give up that idea.

77
00:03:46,300 --> 00:03:50,203
Yeah, I remember once 
people were started

78
00:03:51,200 --> 00:03:55,784
arguing more more vocally 
against that,

79
00:03:56,900 --> 00:03:59,483
that never really 
bothered me that much.

80
00:03:59,483 --> 00:04:03,051
It seemed clear that 
we had a result there

81
00:04:03,051 --> 00:04:06,048
and it became somewhat 
controversial,

82
00:04:06,048 --> 00:04:08,395
but controversial in a good way.

83
00:04:08,620 --> 00:04:10,190
It was clear that people felt

84
00:04:10,700 --> 00:04:13,835
they had to come out against it 
because--

85
00:04:13,970 --> 00:04:15,959
Well, I think what 
we think it's good now

86
00:04:17,426 --> 00:04:19,238
we might not have loved it 
at the time.

87
00:04:20,168 --> 00:04:22,984
I remember being somewhat,
the more upset--

88
00:04:22,984 --> 00:04:24,910
there was some dinner 
where someone said,

89
00:04:24,910 --> 00:04:27,655
"No, no, that paper with Josh,

90
00:04:27,655 --> 00:04:30,808
that was doing a disservice
to the profession."

91
00:04:32,050 --> 00:04:33,850
We definitely had 
reactions like that.

92
00:04:35,410 --> 00:04:38,200
At some level, that may be 
indicative of the culture

93
00:04:38,400 --> 00:04:40,000
in general in economics 
at the time.

94
00:04:41,400 --> 00:04:44,097
I thought back later, 
what if that'd happened now,

95
00:04:44,600 --> 00:04:47,682
if I was a senior person sitting
in that conversation,

96
00:04:48,300 --> 00:04:51,898
I would call that out because it
really was not appropriate--

97
00:04:53,000 --> 00:04:54,200
- [Josh] It wasn't so bad.

98
00:04:54,600 --> 00:04:56,600
I think the criticism is--

99
00:04:57,700 --> 00:04:59,298
It wasn't completely misguided,

100
00:05:00,210 --> 00:05:01,351
it was maybe wrong.

101
00:05:01,800 --> 00:05:04,485
No, no, but you can say 
the paper is wrong

102
00:05:05,400 --> 00:05:06,600
but it's saying that

103
00:05:06,600 --> 00:05:08,112
it's a disservice 
to the profession,

104
00:05:08,128 --> 00:05:09,300
that's not really--

105
00:05:09,300 --> 00:05:10,300
Personal.

106
00:05:10,300 --> 00:05:12,646
Yes, and doing that, not to me,

107
00:05:12,646 --> 00:05:14,612
but in front of 
my senior colleagues.

108
00:05:15,391 --> 00:05:17,489
But nobody was saying 
the result was wrong

109
00:05:17,489 --> 00:05:18,700
and I remember also,

110
00:05:18,700 --> 00:05:21,699
some of the comments 
were thought-provoking

111
00:05:21,699 --> 00:05:23,159
so we had some negative reviews,

112
00:05:23,159 --> 00:05:25,861
I think on the average 
causal response paper.

113
00:05:26,500 --> 00:05:30,361
Somebody said, "These compliers
you can't figure out who they are."

114
00:05:31,450 --> 00:05:33,000
Right, it's one thing to say

115
00:05:33,000 --> 00:05:35,040
you're estimating 
the effect of treatment

116
00:05:35,040 --> 00:05:36,765
on the treated 
or something like that.

117
00:05:36,788 --> 00:05:38,400
You can tell me who's treated,

118
00:05:38,700 --> 00:05:42,289
people in the CPS,
you can't tell me who's a complier.

119
00:05:42,929 --> 00:05:44,679
So that was a legitimate challenge.

120
00:05:44,679 --> 00:05:47,800
That's certainly fair 
and I can see why

121
00:05:49,880 --> 00:05:53,700
that part made people
a little uneasy and uncomfortable.

122
00:05:54,300 --> 00:05:56,400
But at the same time

123
00:05:56,900 --> 00:06:00,244
because it showed that you couldn't
really go beyond that,

124
00:06:00,800 --> 00:06:03,775
it was very useful thing 
to realize.

125
00:06:04,630 --> 00:06:06,500
I remember on the day,

126
00:06:06,500 --> 00:06:09,911
we got to the key result 
that I was thinking,

127
00:06:09,911 --> 00:06:14,221
"Wow, this is as good as it gets.

128
00:06:14,221 --> 00:06:16,978
Here we actually have an insight
but clearly--"

129
00:06:17,500 --> 00:06:19,760
And we had to sell it 
at some point.

130
00:06:19,760 --> 00:06:22,000
For quite a few years, 
we had to sell it

131
00:06:23,480 --> 00:06:24,892
and it's proven to be quite useful.

132
00:06:25,500 --> 00:06:28,761
I don't think we understood that
it would be so useful at the time.

133
00:06:28,761 --> 00:06:29,871
No.

134
00:06:30,170 --> 00:06:34,600
I did feel early on this was
a substantial insight.

135
00:06:34,600 --> 00:06:36,440
- [Josh] Yeah we [did] something.

136
00:06:36,440 --> 00:06:40,041
But I did not think 
goals were there.

137
00:06:40,500 --> 00:06:42,400
I don't think we were aiming 
for the Nobel.

138
00:06:42,650 --> 00:06:43,650
[laughter]

139
00:06:43,730 --> 00:06:46,243
We were very happy to get
that note in <i>Econometrica.</i>

140
00:06:46,859 --> 00:06:48,829
♪ [music] ♪

141
00:06:49,900 --> 00:06:51,500
- [Isaiah] Are there factors 
or are ways of approaching problems

142
00:06:51,500 --> 00:06:54,346
that lead people to be better 
at recognizing the good stuff

143
00:06:54,346 --> 00:06:56,600
and taking the time to do it 
as opposed to dismissing it?

144
00:06:56,600 --> 00:06:57,700
- [Josh] Sometimes 
I think it's helpful.

145
00:06:57,700 --> 00:06:59,672
If you're trying to 
convince somebody

146
00:06:59,698 --> 00:07:01,355
that you have something 
useful to say

147
00:07:01,900 --> 00:07:04,026
and maybe they don't 
speak your language,

148
00:07:04,894 --> 00:07:06,761
you might need 
to learn their language.

149
00:07:06,761 --> 00:07:08,020
Yes, yes, exactly.

150
00:07:08,020 --> 00:07:11,736
That's what we did with Don,
we figured out how to--

151
00:07:11,736 --> 00:07:14,052
I remember we had a very hard time

152
00:07:14,052 --> 00:07:15,686
explaining the exclusion 
restriction to Don,

153
00:07:17,430 --> 00:07:18,993
maybe rightfully so,

154
00:07:19,804 --> 00:07:22,244
I think Guido and I 
eventually figured out

155
00:07:22,510 --> 00:07:24,420
that it wasn't formulated 
very clearly,

156
00:07:25,400 --> 00:07:27,450
and we came up 
with a way to do that

157
00:07:27,450 --> 00:07:29,456
in the potential outcomes framework

158
00:07:29,456 --> 00:07:32,218
that I think worked
for the three of us.

159
00:07:32,218 --> 00:07:33,419
Yeah.

160
00:07:33,419 --> 00:07:35,544
Well, it worked for
the bigger literature

161
00:07:35,544 --> 00:07:37,730
but I think what you're saying 
there is exactly right,

162
00:07:37,749 --> 00:07:41,080
you need to figure out 
how not just say,

163
00:07:41,080 --> 00:07:43,894
"Okay well, I've got this language
and this this works great

164
00:07:43,894 --> 00:07:45,900
and I've got to convince someone
else to use the language.

165
00:07:45,900 --> 00:07:48,357
You could first figure out 
what language they're using

166
00:07:48,680 --> 00:07:50,738
and then only then, 
can you try to say,

167
00:07:50,738 --> 00:07:53,270
"Wow, but here you thinking of it 
this way,"

168
00:07:53,270 --> 00:07:56,880
but that's actually 
a pretty hard thing to do,

169
00:07:56,880 --> 00:07:59,268
you get someone from 
a different discipline,

170
00:07:59,268 --> 00:08:02,300
convincing them, two junior faculty
in a different department

171
00:08:02,300 --> 00:08:04,366
actually have something 
to say to you,

172
00:08:04,366 --> 00:08:06,600
that's that takes 
a fair amount of effort.

173
00:08:07,500 --> 00:08:10,240
Yeah, I wrote Don 
a number of times,

174
00:08:10,240 --> 00:08:12,048
in fairly long letters.

175
00:08:12,048 --> 00:08:13,805
I remember thinking 
this is worth doing,

176
00:08:14,600 --> 00:08:16,006
that if I could convince Don

177
00:08:16,780 --> 00:08:19,444
that would validate the framework 
to some extent.

178
00:08:20,300 --> 00:08:23,174
I think both you and Don were

179
00:08:23,174 --> 00:08:25,000
a little bit more confident 
that you were right.

180
00:08:25,000 --> 00:08:26,438
Well, we used to argue a lot

181
00:08:26,438 --> 00:08:28,500
and you would sometimes 
referee those.

182
00:08:28,500 --> 00:08:29,500
[laughter]

183
00:08:29,800 --> 00:08:30,800
That was fun.

184
00:08:32,580 --> 00:08:33,967
It wasn't hurtful.

185
00:08:35,200 --> 00:08:37,935
I remember it getting 
a little testy once,

186
00:08:37,935 --> 00:08:40,076
we had lunch in The Faculty Club

187
00:08:40,600 --> 00:08:44,077
and we're talking about 
the draft lottery paper.

188
00:08:44,930 --> 00:08:47,700
We were talking about never takes

189
00:08:47,700 --> 00:08:51,000
as people wounded serve 
in the military irrespective of

190
00:08:51,000 --> 00:08:53,500
whether they were getting drafted

191
00:08:54,500 --> 00:08:58,831
and you or Don said something
about shooting yourself in the foot,

192
00:08:59,800 --> 00:09:01,530
as a way of getting 
out of the military

193
00:09:01,530 --> 00:09:03,400
and that may be 
the exclusion restriction

194
00:09:03,400 --> 00:09:05,223
for never takes wasn't working

195
00:09:06,300 --> 00:09:08,560
and then the other one would say,

196
00:09:08,560 --> 00:09:09,951
"Well, yes you could do that

197
00:09:09,951 --> 00:09:12,128
but why would you want 
to shoot yourself in the foot?"

198
00:09:12,128 --> 00:09:13,225
[laughter]

199
00:09:13,225 --> 00:09:15,400
It got a little [out of hand] there.

200
00:09:15,400 --> 00:09:17,600
I usually go for moving to Canada,
for my example,

201
00:09:18,690 --> 00:09:20,096
when I'm teaching that.

202
00:09:22,030 --> 00:09:24,100
But things are tricky,

203
00:09:24,860 --> 00:09:26,595
I get students coming
from computer science

204
00:09:26,595 --> 00:09:29,943
and they want to do things 
on causal inference

205
00:09:30,400 --> 00:09:33,293
and it takes a huge amount 
of effort to figure out

206
00:09:33,460 --> 00:09:35,230
how they actually thinking 
about problem

207
00:09:35,230 --> 00:09:37,000
and whether there's something there

208
00:09:37,000 --> 00:09:38,310
and so, now over the years,

209
00:09:38,310 --> 00:09:40,242
I've got a little more appreciation
for the fact

210
00:09:40,250 --> 00:09:41,958
that Don was actually willing to--

211
00:09:42,200 --> 00:09:46,000
It took him a while, 
but he did engage first with Josh

212
00:09:46,400 --> 00:09:47,500
and then with both of us

213
00:09:48,380 --> 00:09:50,413
and rather than dismissing
and say,

214
00:09:50,413 --> 00:09:53,348
"Okay, well I can't figure out
what these guys are doing

215
00:09:53,368 --> 00:09:56,195
and it's probably just 
not really that interesting."

216
00:09:57,200 --> 00:09:59,736
Everybody always wants 
to figure out quickly,

217
00:09:59,736 --> 00:10:01,376
you want to save time

218
00:10:01,376 --> 00:10:04,133
and you want to save your brain cells
for other things.

219
00:10:05,000 --> 00:10:07,000
The fastest route to 
that is to figure out

220
00:10:07,000 --> 00:10:08,460
why you should dismiss something.

221
00:10:08,460 --> 00:10:09,560
Yes.

222
00:10:09,560 --> 00:10:11,300
I don't need to spend time on this.

223
00:10:11,300 --> 00:10:12,670
♪ [music] ♪

224
00:10:12,670 --> 00:10:14,070
<i>- [Narrator] If you'd like 
to watch more </i>

225
00:10:14,070 --> 00:10:15,737
<i>Nobel conversations, click here,</i>

226
00:10:16,300 --> 00:10:18,354
<i>or if you'd like to learn
more about econometrics,</i>

227
00:10:18,700 --> 00:10:21,300
<i>check out Josh's "Mastering
Econometrics" series.</i>

228
00:10:21,800 --> 00:10:24,700
<i>If you'd like to learn more
about Guido, Josh, and Isaiah</i>

229
00:10:24,860 --> 00:10:26,702
<i>check out the links 
in the description.</i>

230
00:10:26,702 --> 00:10:28,017
♪ [music] ♪