Not Through Ignorance

Home » Science » That problem was framed! – The failure of intuition

That problem was framed! – The failure of intuition


I have a new favorite math problem, courtesy of Daniel Kahneman’s recent book Thinking, Fast and Slow (2011).  The problem isn’t original to Kahneman, though it related rather well to his thesis that when we approach problems of any kind of complexity, we favor a system of intuition (which he calls System 1) over a system of deliberate thinking-it-out (System 2).  Sadly, at least for the idea of humans as rational agents, we are burdened by two curses: we rely way too much on the flawed System 1, and we’re just not all that good at System 2, even when we do employ it.

Anyway, here’s the question, slightly modified in my telling.

A family owns 3 vehicles.  The dad generally drives a pickup truck which gets 12 miles to the gallon.  He needs some sort of hauling vehicle because he has a side business taking trash to to dump for people.  The mom has a commuter car that gets 30 miles to the gallon, but has seating for 5 which the family needs for the occasional family trip.  The daughter has a two-seater that gets 50 miles to the gallon.  As it turns out, each car gets driven about the same total distance each year.

The family recently inherited some money, and they decide they can upgrade one vehicle, but they will still need a pickup, a commuter car, and a two-seater.  As an environmentally-conscious family, their primary goal is to minimize their fuel consumption.  A secondary goal is to spend less money at the gas station.  After pricing their options, they come up with three possibilities:

  • Dad can upgrade from his 12 mpg pickup to a 14 mpg pickup
  • Mom can upgrade her 30 mpg commuter car to a 40 mpg commuter
  • Daughter can upgrade her 50 mpg two-seater to a 65 mpg two-seater.

Which should they choose?

Now, option 3 looks rather appealing, because this is the option that gets the best mpg vehicle.  Furthermore, it represents the biggest mpg upgrade – an improvement of 15 mpg.

Option 2 might also look appealing.  Although this option “only” gains 10 mpg, it represents a 33% improvement in mpg, whereas option 3 only represents a 30% improvement.  In comparison, option 1 looks pretty poor indeed – only a 2 mpg gain, which is a little under 17%, to achieve a pretty pathetic 14 mpg.

By this point you’ll likely be surprised and not surprised that the family should choose option 1.  Surprised because it goes so much against our intuition – our System 1 method of looking for a quick solution by comparing the numbers we’ve been given.  But not surprised in that everyone suspects a trick question in cases like this.  But what’s the trick?

There’s no trick.  It’s just that the problem has been framed around miles per gallon, and therefore the numbers lead you to the wrong conclusion.  But we can switch on System 2 and work it out.

The problem stipulates that the three vehicles are driven equally.  It doesn’t matter what that distance is, but we can choose 10,000 miles per year as a reasonable American example.  Since the family goal is to save gasoline, the next step is to figure out who would save more gallons in a typical year.  Working that out is as trivial as dividing 10,000 by the rated mpg:

  • Dad requires 833 gallons to go 10,000 miles; with the upgrade he would require 714 (a savings of 119 gallons)
  • Mom requires 333 gallons to go 10,000 miles; with the upgrade, she would require 250 (a savings of 83 gallons)
  • Daughter requires 200 gallons to go 10,000 miles; with the upgrade, she would require 154 (a savings of 46 gallons)

Not only is option 1 the best choice, it is the best choice in a landslide.  So shockingly counterintuitive is this result – a result than can be checked by anyone who survived 4th grade math class – that it has been dubbed the MPG illusion.  Indeed, had some member of the family driven a car that got 100 mpg and had the option to upgrade to a revolutionary vehicle that required no gasoline – dad would still be better off getting the upgrade!  (A savings of 119 gallons per year versus a savings of 100.)

New car window stickers in the United States have only lately been changed to present not only miles per gallon but gallons per 100 miles (note the arrow marked #2 in the graphic above).  This is indeed the same information as is presented in the larger graphic above (marked #1), but now framed in a way that people can compare vehicles using their preferred intuitive reasoning strategy.

Intuition in Life or Death Decisions

Kahneman has had a long history of research on framing.  In another counterintuitive result, Kahneman asked volunteers in a psychology experiment to imagine that a new disease is expected to kill 600 people.  Two alternative programs have been proposed in order to combat the disease:

  • If Program A is adopted, 200 people will be saved
  • If Program B is adopted, there is a 1/3 probability that 600 people will be saved and a 2/3 probability that no people will be saved

The subjects of the experiment are invited to choose one of the two programs to support.

Any economist can tell you that the two programs are, for decision-making purposes, identical.  The expected utility of each choice is the saving of 200 lives and the death of 400.  But to a human decision maker, the choices do not feel identical.  In this situation, about 75% of the subjects of the experiment will opt for Program A.  It is apparently hard for people to gamble away the sure thing of saving 200 lives, even for the small chance of saving everyone.

But that’s not the interesting part of the experiment.  The interesting part of the experiment is that another group of subjects are given the same scenario, but the following choices:

  • If Program A is adopted, 400 people will die
  • If Program B is adopted, there is a 1/3 probability that nobody will die and a 2/3 probability that 600 people will die

About 75% of subjects given this question choose Program B.

Notice that this is the opposite of what people do in the first version of the question, because saying that 200 people will be saved (the first description of Program A) is equivalent to saying that 400 people will die (the second description of Program A).  In the first case, people were unwilling to gamble the lives of the 200 people who were guaranteed to be saved by the program, but in the second case, they were willing to take the gamble – based not on the statistics but based on the framing of the problem.  (Of course, advertisers, salesmen, police officers, and lawyers are well-aware – perhaps unconsciously, perhaps consciously – that the way in which a question is asked can be more important than the question itself.)

Intuitions in the Courtroom

Speaking of lawyers, another rich literature in the psychology of thinking and decision making concerns the reliability of eyewitness testimony.  One of the more well-known studies comes from the research of psychologist Elizabeth Loftus.  She showed subjects a slide show depicting the aftermath of a car accident (not much more than a fender bender).  She asked subjects to estimate how fast the cars were going when they hit each other.

You’ll probably not be surprised to learn that the estimates of subjects were significantly lower than the estimates of another group of subjects who were asked to estimate how fast the cars were going when they smashed into each other.  Subjects had seen the same still photographs, but the leading question “smashed into” vs. “hit” was enough to affect the guess at speed.

But that’s not the cool part (or scary part, depending on whether your fate has ever been in the hands of eye witnesses).  One week later, subjects were given a memory test about the photographs they had been shown.  One question on the memory test was “Did you see any broken glass?”

Remember, all subjects had seen the same photographs (in which, by the way, there was no broken glass).  The subjects who had been asked, a week before, how fast the cars were going when they smashed into each other were much more likely to remember (misremember) having seen broken glass (often confidently) than those who’d been asked about cars “hitting” each other.  To be dramatic about it, a couple of words in a question had produced the hallucination of broken glass.

In other experiments false memories could be created by seemingly innocuous differences.  “Did you see a broken headlight” and “Did you see the broken headlight” are not the same question, and subjects – and real life eyewitnesses – do not treat them the same.

Intuitions in Medical Decisions

I’ll end with another of my favorite math problems.

A new disease is known to infect 1 out of every 1,000 people in the United States (or insert your own country here).  Initially it causes no symptoms, but early detection is absolutely crucial.  Fortunately a blood test exists which always detects an infection when present.  The test does have a false-positive rate of 5%.

You decide to get the test.  Unfortunately for you, the test comes back positive.  Given this, what are the chances you do, indeed, have the disease?

This is a great one to try and figure out before reading further.  The math is easy, but figuring out what numbers to use is hard for most people.

Most people believe that their odds are about 95%, and the scary thing is, a lot of doctors believe this too.  Not only are your odds much lower than this, they are so low that you almost shouldn’t be nervous.

The first thing to do is to take into account the base rate of the disease – 1 in 1,000.  (Failure to do so is a reasoning error known as the base-rate fallacy.)  Before you get the blood test, you have no reason to think your odds of having the disease is any different than 1 in 1,000.  After getting a positive test, obviously your odds go up – but how much?

The key is to think about this problem for a statistical point of view.  It isn’t just you getting the blood test, lots of people get the blood test.  So let’s imagine 1,000 people go for the test, you being one of them.  Odds are, one of the 1,000 people going in for the test have the disease (that’s the base rate), and that one person is guaranteed to get a positive result.  But what about the other 999?

The test – like all medical tests – occasionally gives someone a positive result when they are in fact disease-free.  We are told that the test has a 5% false positive rate.  So people who are disease free get a positive test result 5% of the time.

So, out of our 999 disease-free people, about 50 of them (5%) will get a positive test result.

In the initial group of 1,000 people, we have 51 people with a positive test result – the one person who actually has the disease, and 50 other people who have a false positive.  Therefore, if you get a positive result, your odds of having the disease are 1 in 51, or just under 2%.

It’s hard to wrap one’s brain around this.  The disease isn’t very common – 1 in 1,000 – so getting a positive test is definitely bad news – your odds go up from 0.1% to 2%.  On the other hand, who worries about a 2% chance?  It can’t be good news to have a positive test, but can you really call it bad news?

It gets even crazier.  Imagine your doctor, aware of the false positive rate, recommends you get a second test.  That second test comes back positive.  Should you panic now?

Again, imagine the 51 people who got a positive test take the test again.  We know our 1 real disease-carrier will get a second positive result.  Of the other 50, who are disease-free, each has a 5% chance of getting a false positive – between 2 and 3 of those 50 people.  So if you get another positive result, you’re one of 3 or 4 people getting two consecutive positives, but you are still more likely to be disease-free than to have the disease!

Naturally, this isn’t true will all diseases and all tests.  A base rate of 1 in 1,000 is rather high (it would mean that 300,000 people in the United States have this disease) and a false positive rate of 5% is also rather high (but not out of the range of existing medical tests).  Some diseases are very rare and some tests are very good.  But some diagnostic tests do have high false positives – anything involving radiography, for example, or blood tests that attempt to detect elevated levels of compounds normally found in the body (human growth hormone drug tests for athletes come to mind) – so it’s really important to put particular medical tests into perspective.  Anyone who has ever gotten bad news – even from a doctor well aware of these issues – knows that having a sense of perspective about positive tests are really hard to achieve.

Take a deep breath, though, and do the math.

One last thing

The rates of organ donation in France are much higher than they are in the U.K.  Why?  Is France a country full of humanitarians and the U.K. full of selfish organ-hoggers?  (They could be forgiven, given this scene in Monty Python’s Meaning of Life.)

The real answer, you won’t be surprised to know, is the way the request is framed, as Dan Ariely explains in this informative TED talk:


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: