0:00:01.000,0:00:03.000 I want to talk to you today a little bit 0:00:03.000,0:00:06.000 about predictable irrationality. 0:00:06.000,0:00:10.000 And my interest in irrational behavior 0:00:10.000,0:00:13.000 started many years ago in the hospital. 0:00:13.000,0:00:17.000 I was burned very badly. 0:00:17.000,0:00:20.000 And if you spend a lot of time in hospital, 0:00:20.000,0:00:23.000 you'll see a lot of types of irrationalities. 0:00:23.000,0:00:28.000 And the one that particularly bothered me in the burn department 0:00:28.000,0:00:32.000 was the process by which the nurses took the bandage off me. 0:00:33.000,0:00:35.000 Now, you must have all taken a Band-Aid off at some point, 0:00:35.000,0:00:38.000 and you must have wondered what's the right approach. 0:00:38.000,0:00:42.000 Do you rip it off quickly -- short duration but high intensity -- 0:00:42.000,0:00:44.000 or do you take your Band-Aid off slowly -- 0:00:44.000,0:00:48.000 you take a long time, but each second is not as painful -- 0:00:48.000,0:00:51.000 which one of those is the right approach? 0:00:51.000,0:00:55.000 The nurses in my department thought that the right approach 0:00:55.000,0:00:58.000 was the ripping one, so they would grab hold and they would rip, 0:00:58.000,0:01:00.000 and they would grab hold and they would rip. 0:01:00.000,0:01:04.000 And because I had 70 percent of my body burned, it would take about an hour. 0:01:04.000,0:01:07.000 And as you can imagine, 0:01:07.000,0:01:11.000 I hated that moment of ripping with incredible intensity. 0:01:11.000,0:01:13.000 And I would try to reason with them and say, 0:01:13.000,0:01:14.000 "Why don't we try something else? 0:01:14.000,0:01:16.000 Why don't we take it a little longer -- 0:01:16.000,0:01:21.000 maybe two hours instead of an hour -- and have less of this intensity?" 0:01:21.000,0:01:23.000 And the nurses told me two things. 0:01:23.000,0:01:27.000 They told me that they had the right model of the patient -- 0:01:27.000,0:01:30.000 that they knew what was the right thing to do to minimize my pain -- 0:01:30.000,0:01:33.000 and they also told me that the word patient doesn't mean 0:01:33.000,0:01:35.000 to make suggestions or to interfere or ... 0:01:35.000,0:01:38.000 This is not just in Hebrew, by the way. 0:01:38.000,0:01:41.000 It's in every language I've had experience with so far. 0:01:41.000,0:01:45.000 And, you know, there's not much -- there wasn't much I could do, 0:01:45.000,0:01:48.000 and they kept on doing what they were doing. 0:01:48.000,0:01:50.000 And about three years later, when I left the hospital, 0:01:50.000,0:01:53.000 I started studying at the university. 0:01:53.000,0:01:56.000 And one of the most interesting lessons I learned 0:01:56.000,0:01:58.000 was that there is an experimental method 0:01:58.000,0:02:02.000 that if you have a question you can create a replica of this question 0:02:02.000,0:02:06.000 in some abstract way, and you can try to examine this question, 0:02:06.000,0:02:08.000 maybe learn something about the world. 0:02:08.000,0:02:10.000 So that's what I did. 0:02:10.000,0:02:11.000 I was still interested 0:02:11.000,0:02:13.000 in this question of how do you take bandages off burn patients. 0:02:13.000,0:02:16.000 So originally I didn't have much money, 0:02:16.000,0:02:20.000 so I went to a hardware store and I bought a carpenter's vice. 0:02:20.000,0:02:24.000 And I would bring people to the lab and I would put their finger in it, 0:02:24.000,0:02:26.000 and I would crunch it a little bit. 0:02:26.000,0:02:28.000 (Laughter) 0:02:28.000,0:02:31.000 And I would crunch it for long periods and short periods, 0:02:31.000,0:02:33.000 and pain that went up and pain that went down, 0:02:33.000,0:02:37.000 and with breaks and without breaks -- all kinds of versions of pain. 0:02:37.000,0:02:39.000 And when I finished hurting people a little bit, I would ask them, 0:02:39.000,0:02:41.000 so, how painful was this? Or, how painful was this? 0:02:41.000,0:02:43.000 Or, if you had to choose between the last two, 0:02:43.000,0:02:45.000 which one would you choose? 0:02:45.000,0:02:48.000 (Laughter) 0:02:48.000,0:02:51.000 I kept on doing this for a while. 0:02:51.000,0:02:53.000 (Laughter) 0:02:53.000,0:02:57.000 And then, like all good academic projects, I got more funding. 0:02:57.000,0:02:59.000 I moved to sounds, electrical shocks -- 0:02:59.000,0:03:04.000 I even had a pain suit that I could get people to feel much more pain. 0:03:04.000,0:03:08.000 But at the end of this process, 0:03:08.000,0:03:11.000 what I learned was that the nurses were wrong. 0:03:11.000,0:03:14.000 Here were wonderful people with good intentions 0:03:14.000,0:03:16.000 and plenty of experience, and nevertheless 0:03:16.000,0:03:20.000 they were getting things wrong predictably all the time. 0:03:20.000,0:03:23.000 It turns out that because we don't encode duration 0:03:23.000,0:03:25.000 in the way that we encode intensity, 0:03:25.000,0:03:29.000 I would have had less pain if the duration would have been longer 0:03:29.000,0:03:31.000 and the intensity was lower. 0:03:31.000,0:03:34.000 It turns out it would have been better to start with my face, 0:03:34.000,0:03:36.000 which was much more painful, and move toward my legs, 0:03:36.000,0:03:39.000 giving me a trend of improvement over time -- 0:03:39.000,0:03:40.000 that would have been also less painful. 0:03:40.000,0:03:42.000 And it also turns out that it would have been good 0:03:42.000,0:03:44.000 to give me breaks in the middle to kind of recuperate from the pain. 0:03:44.000,0:03:46.000 All of these would have been great things to do, 0:03:46.000,0:03:49.000 and my nurses had no idea. 0:03:49.000,0:03:50.000 And from that point on I started thinking, 0:03:50.000,0:03:53.000 are the nurses the only people in the world who get things wrong 0:03:53.000,0:03:56.000 in this particular decision, or is it a more general case? 0:03:56.000,0:03:58.000 And it turns out it's a more general case -- 0:03:58.000,0:04:01.000 there's a lot of mistakes we do. 0:04:01.000,0:04:06.000 And I want to give you one example of one of these irrationalities, 0:04:06.000,0:04:09.000 and I want to talk to you about cheating. 0:04:09.000,0:04:11.000 And the reason I picked cheating is because it's interesting, 0:04:11.000,0:04:13.000 but also it tells us something, I think, 0:04:13.000,0:04:16.000 about the stock market situation we're in. 0:04:16.000,0:04:19.000 So, my interest in cheating started 0:04:19.000,0:04:21.000 when Enron came on the scene, exploded all of a sudden, 0:04:21.000,0:04:24.000 and I started thinking about what is happening here. 0:04:24.000,0:04:25.000 Is it the case that there was kind of 0:04:25.000,0:04:28.000 a few apples who are capable of doing these things, 0:04:28.000,0:04:30.000 or are we talking a more endemic situation, 0:04:30.000,0:04:34.000 that many people are actually capable of behaving this way? 0:04:34.000,0:04:38.000 So, like we usually do, I decided to do a simple experiment. 0:04:38.000,0:04:39.000 And here's how it went. 0:04:39.000,0:04:42.000 If you were in the experiment, I would pass you a sheet of paper 0:04:42.000,0:04:46.000 with 20 simple math problems that everybody could solve, 0:04:46.000,0:04:48.000 but I wouldn't give you enough time. 0:04:48.000,0:04:50.000 When the five minutes were over, I would say, 0:04:50.000,0:04:53.000 "Pass me the sheets of paper, and I'll pay you a dollar per question." 0:04:53.000,0:04:57.000 People did this. I would pay people four dollars for their task -- 0:04:57.000,0:04:59.000 on average people would solve four problems. 0:04:59.000,0:05:02.000 Other people I would tempt to cheat. 0:05:02.000,0:05:03.000 I would pass their sheet of paper. 0:05:03.000,0:05:05.000 When the five minutes were over, I would say, 0:05:05.000,0:05:06.000 "Please shred the piece of paper. 0:05:06.000,0:05:09.000 Put the little pieces in your pocket or in your backpack, 0:05:09.000,0:05:12.000 and tell me how many questions you got correctly." 0:05:12.000,0:05:15.000 People now solved seven questions on average. 0:05:15.000,0:05:20.000 Now, it wasn't as if there was a few bad apples -- 0:05:20.000,0:05:23.000 a few people cheated a lot. 0:05:23.000,0:05:26.000 Instead, what we saw is a lot of people who cheat a little bit. 0:05:26.000,0:05:29.000 Now, in economic theory, 0:05:29.000,0:05:32.000 cheating is a very simple cost-benefit analysis. 0:05:32.000,0:05:34.000 You say, what's the probability of being caught? 0:05:34.000,0:05:37.000 How much do I stand to gain from cheating? 0:05:37.000,0:05:39.000 And how much punishment would I get if I get caught? 0:05:39.000,0:05:41.000 And you weigh these options out -- 0:05:41.000,0:05:43.000 you do the simple cost-benefit analysis, 0:05:43.000,0:05:46.000 and you decide whether it's worthwhile to commit the crime or not. 0:05:46.000,0:05:48.000 So, we try to test this. 0:05:48.000,0:05:52.000 For some people, we varied how much money they could get away with -- 0:05:52.000,0:05:53.000 how much money they could steal. 0:05:53.000,0:05:56.000 We paid them 10 cents per correct question, 50 cents, 0:05:56.000,0:05:59.000 a dollar, five dollars, 10 dollars per correct question. 0:05:59.000,0:06:03.000 You would expect that as the amount of money on the table increases, 0:06:03.000,0:06:06.000 people would cheat more, but in fact it wasn't the case. 0:06:06.000,0:06:09.000 We got a lot of people cheating by stealing by a little bit. 0:06:09.000,0:06:12.000 What about the probability of being caught? 0:06:12.000,0:06:14.000 Some people shredded half the sheet of paper, 0:06:14.000,0:06:15.000 so there was some evidence left. 0:06:15.000,0:06:17.000 Some people shredded the whole sheet of paper. 0:06:17.000,0:06:20.000 Some people shredded everything, went out of the room, 0:06:20.000,0:06:23.000 and paid themselves from the bowl of money that had over 100 dollars. 0:06:23.000,0:06:26.000 You would expect that as the probability of being caught goes down, 0:06:26.000,0:06:29.000 people would cheat more, but again, this was not the case. 0:06:29.000,0:06:32.000 Again, a lot of people cheated by just by a little bit, 0:06:32.000,0:06:35.000 and they were insensitive to these economic incentives. 0:06:35.000,0:06:36.000 So we said, "If people are not sensitive 0:06:36.000,0:06:41.000 to the economic rational theory explanations, to these forces, 0:06:41.000,0:06:44.000 what could be going on?" 0:06:44.000,0:06:47.000 And we thought maybe what is happening is that there are two forces. 0:06:47.000,0:06:49.000 At one hand, we all want to look at ourselves in the mirror 0:06:49.000,0:06:52.000 and feel good about ourselves, so we don't want to cheat. 0:06:52.000,0:06:54.000 On the other hand, we can cheat a little bit, 0:06:54.000,0:06:56.000 and still feel good about ourselves. 0:06:56.000,0:06:57.000 So, maybe what is happening is that 0:06:57.000,0:06:59.000 there's a level of cheating we can't go over, 0:06:59.000,0:07:03.000 but we can still benefit from cheating at a low degree, 0:07:03.000,0:07:06.000 as long as it doesn't change our impressions about ourselves. 0:07:06.000,0:07:09.000 We call this like a personal fudge factor. 0:07:10.000,0:07:14.000 Now, how would you test a personal fudge factor? 0:07:14.000,0:07:18.000 Initially we said, what can we do to shrink the fudge factor? 0:07:18.000,0:07:20.000 So, we got people to the lab, and we said, 0:07:20.000,0:07:22.000 "We have two tasks for you today." 0:07:22.000,0:07:23.000 First, we asked half the people 0:07:23.000,0:07:25.000 to recall either 10 books they read in high school, 0:07:25.000,0:07:28.000 or to recall The Ten Commandments, 0:07:28.000,0:07:30.000 and then we tempted them with cheating. 0:07:30.000,0:07:33.000 Turns out the people who tried to recall The Ten Commandments -- 0:07:33.000,0:07:35.000 and in our sample nobody could recall all of The Ten Commandments -- 0:07:36.000,0:07:40.000 but those people who tried to recall The Ten Commandments, 0:07:40.000,0:07:43.000 given the opportunity to cheat, did not cheat at all. 0:07:43.000,0:07:45.000 It wasn't that the more religious people -- 0:07:45.000,0:07:46.000 the people who remembered more of the Commandments -- cheated less, 0:07:46.000,0:07:48.000 and the less religious people -- 0:07:48.000,0:07:49.000 the people who couldn't remember almost any Commandments -- 0:07:49.000,0:07:51.000 cheated more. 0:07:51.000,0:07:55.000 The moment people thought about trying to recall The Ten Commandments, 0:07:55.000,0:07:56.000 they stopped cheating. 0:07:56.000,0:07:58.000 In fact, even when we gave self-declared atheists 0:07:58.000,0:08:02.000 the task of swearing on the Bible and we give them a chance to cheat, 0:08:02.000,0:08:04.000 they don't cheat at all. 0:08:06.000,0:08:08.000 Now, Ten Commandments is something that is hard 0:08:08.000,0:08:10.000 to bring into the education system, so we said, 0:08:10.000,0:08:12.000 "Why don't we get people to sign the honor code?" 0:08:12.000,0:08:14.000 So, we got people to sign, 0:08:14.000,0:08:18.000 "I understand that this short survey falls under the MIT Honor Code." 0:08:18.000,0:08:21.000 Then they shredded it. No cheating whatsoever. 0:08:21.000,0:08:22.000 And this is particularly interesting, 0:08:22.000,0:08:24.000 because MIT doesn't have an honor code. 0:08:24.000,0:08:29.000 (Laughter) 0:08:29.000,0:08:33.000 So, all this was about decreasing the fudge factor. 0:08:33.000,0:08:36.000 What about increasing the fudge factor? 0:08:36.000,0:08:38.000 The first experiment -- I walked around MIT 0:08:38.000,0:08:41.000 and I distributed six-packs of Cokes in the refrigerators -- 0:08:41.000,0:08:43.000 these were common refrigerators for the undergrads. 0:08:43.000,0:08:46.000 And I came back to measure what we technically call 0:08:46.000,0:08:50.000 the half-lifetime of Coke -- how long does it last in the refrigerators? 0:08:50.000,0:08:53.000 As you can expect it doesn't last very long; people take it. 0:08:53.000,0:08:57.000 In contrast, I took a plate with six one-dollar bills, 0:08:57.000,0:09:00.000 and I left those plates in the same refrigerators. 0:09:00.000,0:09:01.000 No bill ever disappeared. 0:09:01.000,0:09:04.000 Now, this is not a good social science experiment, 0:09:04.000,0:09:07.000 so to do it better I did the same experiment 0:09:07.000,0:09:09.000 as I described to you before. 0:09:09.000,0:09:12.000 A third of the people we passed the sheet, they gave it back to us. 0:09:12.000,0:09:15.000 A third of the people we passed it to, they shredded it, 0:09:15.000,0:09:16.000 they came to us and said, 0:09:16.000,0:09:19.000 "Mr. Experimenter, I solved X problems. Give me X dollars." 0:09:19.000,0:09:22.000 A third of the people, when they finished shredding the piece of paper, 0:09:22.000,0:09:24.000 they came to us and said, 0:09:24.000,0:09:30.000 "Mr Experimenter, I solved X problems. Give me X tokens." 0:09:30.000,0:09:33.000 We did not pay them with dollars; we paid them with something else. 0:09:33.000,0:09:36.000 And then they took the something else, they walked 12 feet to the side, 0:09:36.000,0:09:38.000 and exchanged it for dollars. 0:09:38.000,0:09:40.000 Think about the following intuition. 0:09:40.000,0:09:43.000 How bad would you feel about taking a pencil from work home, 0:09:43.000,0:09:45.000 compared to how bad would you feel 0:09:45.000,0:09:47.000 about taking 10 cents from a petty cash box? 0:09:47.000,0:09:50.000 These things feel very differently. 0:09:50.000,0:09:53.000 Would being a step removed from cash for a few seconds 0:09:53.000,0:09:56.000 by being paid by token make a difference? 0:09:56.000,0:09:58.000 Our subjects doubled their cheating. 0:09:58.000,0:10:00.000 I'll tell you what I think 0:10:00.000,0:10:02.000 about this and the stock market in a minute. 0:10:03.000,0:10:07.000 But this did not solve the big problem I had with Enron yet, 0:10:07.000,0:10:10.000 because in Enron, there's also a social element. 0:10:10.000,0:10:11.000 People see each other behaving. 0:10:11.000,0:10:13.000 In fact, every day when we open the news 0:10:13.000,0:10:15.000 we see examples of people cheating. 0:10:15.000,0:10:18.000 What does this cause us? 0:10:18.000,0:10:19.000 So, we did another experiment. 0:10:19.000,0:10:22.000 We got a big group of students to be in the experiment, 0:10:22.000,0:10:23.000 and we prepaid them. 0:10:23.000,0:10:26.000 So everybody got an envelope with all the money for the experiment, 0:10:26.000,0:10:28.000 and we told them that at the end, we asked them 0:10:28.000,0:10:32.000 to pay us back the money they didn't make. OK? 0:10:32.000,0:10:33.000 The same thing happens. 0:10:33.000,0:10:35.000 When we give people the opportunity to cheat, they cheat. 0:10:35.000,0:10:38.000 They cheat just by a little bit, all the same. 0:10:38.000,0:10:41.000 But in this experiment we also hired an acting student. 0:10:41.000,0:10:45.000 This acting student stood up after 30 seconds, and said, 0:10:45.000,0:10:48.000 "I solved everything. What do I do now?" 0:10:48.000,0:10:52.000 And the experimenter said, "If you've finished everything, go home. 0:10:52.000,0:10:53.000 That's it. The task is finished." 0:10:53.000,0:10:57.000 So, now we had a student -- an acting student -- 0:10:57.000,0:10:59.000 that was a part of the group. 0:10:59.000,0:11:01.000 Nobody knew it was an actor. 0:11:01.000,0:11:05.000 And they clearly cheated in a very, very serious way. 0:11:05.000,0:11:08.000 What would happen to the other people in the group? 0:11:08.000,0:11:11.000 Will they cheat more, or will they cheat less? 0:11:11.000,0:11:13.000 Here is what happens. 0:11:13.000,0:11:17.000 It turns out it depends on what kind of sweatshirt they're wearing. 0:11:17.000,0:11:19.000 Here is the thing. 0:11:19.000,0:11:22.000 We ran this at Carnegie Mellon and Pittsburgh. 0:11:22.000,0:11:24.000 And at Pittsburgh there are two big universities, 0:11:24.000,0:11:27.000 Carnegie Mellon and University of Pittsburgh. 0:11:27.000,0:11:29.000 All of the subjects sitting in the experiment 0:11:29.000,0:11:31.000 were Carnegie Mellon students. 0:11:31.000,0:11:35.000 When the actor who was getting up was a Carnegie Mellon student -- 0:11:35.000,0:11:37.000 he was actually a Carnegie Mellon student -- 0:11:37.000,0:11:41.000 but he was a part of their group, cheating went up. 0:11:41.000,0:11:45.000 But when he actually had a University of Pittsburgh sweatshirt, 0:11:45.000,0:11:47.000 cheating went down. 0:11:47.000,0:11:50.000 (Laughter) 0:11:50.000,0:11:53.000 Now, this is important, because remember, 0:11:53.000,0:11:55.000 when the moment the student stood up, 0:11:55.000,0:11:58.000 it made it clear to everybody that they could get away with cheating, 0:11:58.000,0:12:00.000 because the experimenter said, 0:12:00.000,0:12:02.000 "You've finished everything. Go home," and they went with the money. 0:12:02.000,0:12:05.000 So it wasn't so much about the probability of being caught again. 0:12:05.000,0:12:08.000 It was about the norms for cheating. 0:12:08.000,0:12:11.000 If somebody from our in-group cheats and we see them cheating, 0:12:11.000,0:12:15.000 we feel it's more appropriate, as a group, to behave this way. 0:12:15.000,0:12:17.000 But if it's somebody from another group, these terrible people -- 0:12:17.000,0:12:19.000 I mean, not terrible in this -- 0:12:19.000,0:12:21.000 but somebody we don't want to associate ourselves with, 0:12:21.000,0:12:23.000 from another university, another group, 0:12:23.000,0:12:26.000 all of a sudden people's awareness of honesty goes up -- 0:12:26.000,0:12:28.000 a little bit like The Ten Commandments experiment -- 0:12:28.000,0:12:32.000 and people cheat even less. 0:12:32.000,0:12:36.000 So, what have we learned from this about cheating? 0:12:36.000,0:12:39.000 We've learned that a lot of people can cheat. 0:12:39.000,0:12:42.000 They cheat just by a little bit. 0:12:42.000,0:12:46.000 When we remind people about their morality, they cheat less. 0:12:46.000,0:12:49.000 When we get bigger distance from cheating, 0:12:49.000,0:12:53.000 from the object of money, for example, people cheat more. 0:12:53.000,0:12:55.000 And when we see cheating around us, 0:12:55.000,0:12:59.000 particularly if it's a part of our in-group, cheating goes up. 0:12:59.000,0:13:02.000 Now, if we think about this in terms of the stock market, 0:13:02.000,0:13:03.000 think about what happens. 0:13:03.000,0:13:06.000 What happens in a situation when you create something 0:13:06.000,0:13:08.000 where you pay people a lot of money 0:13:08.000,0:13:11.000 to see reality in a slightly distorted way? 0:13:11.000,0:13:14.000 Would they not be able to see it this way? 0:13:14.000,0:13:15.000 Of course they would. 0:13:15.000,0:13:16.000 What happens when you do other things, 0:13:16.000,0:13:18.000 like you remove things from money? 0:13:18.000,0:13:21.000 You call them stock, or stock options, derivatives, 0:13:21.000,0:13:22.000 mortgage-backed securities. 0:13:22.000,0:13:25.000 Could it be that with those more distant things, 0:13:25.000,0:13:27.000 it's not a token for one second, 0:13:27.000,0:13:29.000 it's something that is many steps removed from money 0:13:29.000,0:13:33.000 for a much longer time -- could it be that people will cheat even more? 0:13:33.000,0:13:35.000 And what happens to the social environment 0:13:35.000,0:13:38.000 when people see other people behave around them? 0:13:38.000,0:13:42.000 I think all of those forces worked in a very bad way 0:13:42.000,0:13:44.000 in the stock market. 0:13:44.000,0:13:47.000 More generally, I want to tell you something 0:13:47.000,0:13:50.000 about behavioral economics. 0:13:50.000,0:13:54.000 We have many intuitions in our life, 0:13:54.000,0:13:57.000 and the point is that many of these intuitions are wrong. 0:13:57.000,0:14:00.000 The question is, are we going to test those intuitions? 0:14:00.000,0:14:02.000 We can think about how we're going to test this intuition 0:14:02.000,0:14:04.000 in our private life, in our business life, 0:14:04.000,0:14:07.000 and most particularly when it goes to policy, 0:14:07.000,0:14:10.000 when we think about things like No Child Left Behind, 0:14:10.000,0:14:13.000 when you create new stock markets, when you create other policies -- 0:14:13.000,0:14:16.000 taxation, health care and so on. 0:14:16.000,0:14:18.000 And the difficulty of testing our intuition 0:14:18.000,0:14:20.000 was the big lesson I learned 0:14:20.000,0:14:22.000 when I went back to the nurses to talk to them. 0:14:22.000,0:14:24.000 So I went back to talk to them 0:14:24.000,0:14:27.000 and tell them what I found out about removing bandages. 0:14:27.000,0:14:29.000 And I learned two interesting things. 0:14:29.000,0:14:31.000 One was that my favorite nurse, Ettie, 0:14:31.000,0:14:35.000 told me that I did not take her pain into consideration. 0:14:35.000,0:14:37.000 She said, "Of course, you know, it was very painful for you. 0:14:37.000,0:14:39.000 But think about me as a nurse, 0:14:39.000,0:14:41.000 taking, removing the bandages of somebody I liked, 0:14:41.000,0:14:44.000 and had to do it repeatedly over a long period of time. 0:14:44.000,0:14:47.000 Creating so much torture was not something that was good for me, too." 0:14:47.000,0:14:52.000 And she said maybe part of the reason was it was difficult for her. 0:14:52.000,0:14:55.000 But it was actually more interesting than that, because she said, 0:14:55.000,0:15:00.000 "I did not think that your intuition was right. 0:15:00.000,0:15:01.000 I felt my intuition was correct." 0:15:01.000,0:15:03.000 So, if you think about all of your intuitions, 0:15:03.000,0:15:07.000 it's very hard to believe that your intuition is wrong. 0:15:07.000,0:15:10.000 And she said, "Given the fact that I thought my intuition was right ..." -- 0:15:10.000,0:15:12.000 she thought her intuition was right -- 0:15:12.000,0:15:17.000 it was very difficult for her to accept doing a difficult experiment 0:15:17.000,0:15:19.000 to try and check whether she was wrong. 0:15:19.000,0:15:23.000 But in fact, this is the situation we're all in all the time. 0:15:23.000,0:15:26.000 We have very strong intuitions about all kinds of things -- 0:15:26.000,0:15:29.000 our own ability, how the economy works, 0:15:29.000,0:15:31.000 how we should pay school teachers. 0:15:31.000,0:15:34.000 But unless we start testing those intuitions, 0:15:34.000,0:15:36.000 we're not going to do better. 0:15:36.000,0:15:38.000 And just think about how better my life would have been 0:15:38.000,0:15:40.000 if these nurses would have been willing to check their intuition, 0:15:40.000,0:15:41.000 and how everything would have been better 0:15:41.000,0:15:46.000 if we just start doing more systematic experimentation of our intuitions. 0:15:46.000,0:15:48.000 Thank you very much.