PDF Summary:The Undoing Project, by Michael Lewis

The human mind is more prone to error than most of us would like to believe. Many of the ways the mind fools itself were discovered by the team of Daniel Kahneman and Amos Tversky, whose research changed how we understand human behavior. At a glance, the two psychologists were opposites in personality and style, yet both had an interest in the workings of the mind and a need to undo our most common misperceptions. During Tversky and Kahneman’s intense collaboration, they identified the mental errors that skew our everyday decisions. Their work gave us tools to sharpen our thinking and avoid the unconscious pitfalls of the mind.

This guide will look at Kahneman and Tversky’s partnership, the historical forces that shaped their lives, and the prevailing psychological theories they overturned. It will also examine how their insights reached beyond the field of psychology into economics, politics, and other domains of human knowledge.

(continued)...

The Duo in Action (1973-1977)

By 1973, Kahneman and Tversky were ready to summarize their findings, but once again, war intervened in their lives. The following military conflict gave fresh urgency to the problem of uncovering the foibles of the mind. Lewis writes that in the years following the 1973 Yom Kippur War, Kahneman and Tversky were particularly troubled by their government’s failure to predict the attack. They shifted their focus to examine the decision-making process and gave birth to a theory that overturned our understanding of human behavior in general.

(Shortform note: While much has been written in hindsight about failures in military intelligence, the problem of accurate forecasting remains an issue to this day. In Superforecasting, Philip Tetlock and Dan Gardener emphasize that accurate prediction hinges on avoiding cognitive bias, as well as focusing on statistical probabilities and taking a wide perspective of events. However, in Fooled by Randomness, Nassim Nicholas Taleb argues that not only do chaotic situations defy prediction, but that our very predictions themselves change the future.)

The Yom Kippur War began when Syria and Egypt launched a joint attack on October 6, 1973. When the fighting began, Tversky and Kahneman were assigned to the psychology unit embedded in the Israeli army. Lewis states that their primary job was to find ways to improve troop morale. What they in fact were able to accomplish was to let the troops open up about their experiences. Doing so was so heartbreaking that it reaffirmed the psychologists’ desire to find practical applications for their work, specifically to prevent another unexpected war.

(Shortform note: The employment of psychologists for military purposes began during the two World Wars, and not only to help soldiers deal with trauma. Military psychologists are often called upon to evaluate recruits and officer candidates, as well as to provide mental health interventions for soldiers operating under great stress.)

Decision-Making and Prospect Theory

In order to stave off a similar surprise attack in the future, Kahneman and Tversky tried to convince the country’s policy makers to calculate probabilities and play the odds. However, when they tried to teach the government about statistical analysis, they learned that generals and leaders always favor their instincts, even after their instincts have been proven wrong. Lewis says this led Tversky and Kahneman to conclude that people need stories, not numbers. Realizing that providing leaders with information wasn’t sufficient, Kahneman and Tversky decided to examine the decision-making process itself.

(Shortform note: The idea that people value stories over data still holds true today. A 2015 report by PriceWaterhouseCoopers shows that business executives place data third on the list of factors that go into decision-making, after their own intuition and the experience of others. This degree of self-confidence flies in the face of the long list of dangers that come from overvaluing intuition.)

Lewis writes that prior studies of decision-making revolved around hypothetical gambles, such as whether people would prefer an 80% chance of winning $100 or a certain gain of $80. The reigning explanation for how people made decisions was expected utility theory—the idea that people are rational actors who calculate the utility of a situation to make the optimal decision. To address why this theory failed to predict actual human behavior, Swiss mathematician Daniel Bernoulli introduced the idea that people also calculate the psychological value of outcomes. Tversky and Kahneman were of the opinion that decision-making was far more chaotic.

(Shortform note: In Decisive, authors Chip and Dan Heath highlight several mental fallacies that make decision-making more chaotic, namely binary thinking, confirmation bias, clinging to a status quo, and overconfidence. In Nudge, Richard Thaler and Cass Sunstein suggest that those very fallacies often guide decision-making, a result that Kahneman and Tversky may have hoped for.)

In order to build a more accurate model of human decisions, Tversky and Kahneman decided that emotions must be taken into account, not just probabilities and economic value. Lewis says that in working the problem, the emotion Kahneman latched onto first was regret, or more specifically, that anticipation of regret guides many decisions. Bernoulli’s risk aversion, then, can be seen as the price of avoiding regret. However, they still couldn’t understand what prompted people to make risky decisions.

(Shortform note: While Kahneman would shortly abandon his theory that regret is the primary motivator in making decisions, regret’s psychological power continues to be a subject of much study. In The Paradox of Choice, Barry Schwartz goes into detail on the psychology of regret and the negative impact it can have on our lives. However, he also describes regret as a useful emotion that forces us to take decisions seriously and reduce the harm of our past mistakes.)

A flaw Tversky and Kahneman noticed in economics research was that the decision-making gambles in expected utility theory were always framed as a choice between two gains. Lewis writes that the breakthrough came when Kahneman and Tversky turned the problem on its head and framed experiments with gambles involving loss. What they found was that people are much more open to taking risks if it means avoiding loss instead of making gains.

(Shortform note: Kahneman and Tversky approached the subject of risk from a behavioral and economics perspective, but risk-taking also involves neurological factors. The surges of adrenaline and dopamine produced while engaging in risky activities can become addictive, and some people resort to high-risk activities as a response to emotional pain and depression.)

In 1975 they worked together on creating “risk-value theory,” also known as “prospect theory,” which balanced human decisions based on gains and losses and laid bare the patterns of seemingly irrational human behavior. At the time, economics theory asserted that the irrational errors people made were random and would be corrected by the market. Prospect theory, however, showed that humans are irrational in consistent, systematic ways, and that irrationality is built into the economic market itself. In other words, according to Lewis, prospect theory was the iceberg that could sink all of the prevailing economic thought. We’ll discuss prospect theory at length later in the guide.

(Shortform note: Decades after prospect theory’s initial publication, it is still widely held as the best predictive tool for explaining how people evaluate economic risk. It has also been applied to problems in political science, organizational management, marketing, and even travel.)

Separate Ways (1978-1996)

While Kahneman and Tversky were knocking down the pillars of psychology and economics, they were also undergoing upheavals themselves. Lewis claims that their newfound stardom in the academic world, coupled with changes in their personal lives, put their partnership on tenuous ground that affected them both in different ways. Their relationship shifted through the 1970s and ’80s such that Tversky received more and more credit, while Kahneman began to feel resentful of his partner’s antagonistic style, until he felt they could no longer work together.

The first major change to affect their partnership was when Kahneman divorced his wife and left Israel to strike up a relationship with a colleague, Anne Treisman. Tversky left Israel to follow Kahneman, and American universities scrambled to hire him. Lewis states that many in academia gave Tversky more credit for their work, mainly because he was more outgoing, and his contributions were more grounded in math. In 1978, Kahneman settled at the University of British Columbia, while Tversky took a posting at Stanford. Their partnership would now be conducted long-distance.

(Shortform note: The University of British Columbia is a well-regarded research institution, but Stanford University is considered more prestigious, counting Nobel laureates and billionaires among its faculty and alumni. While the two psychologists researched and published their work jointly, Tversky was often given singular credit, such as in this 1998 article which implies that Kahneman was merely Tversky’s coauthor. However, it’s understandable that people were drawn to the more extroverted Tversky—in Quiet, Susan Cain acknowledges that society itself values the qualities of extroverts more than those of introverts.)

Undoing Theory and Public Perception

Following prospect theory, Lewis says that Kahneman turned his attention back to the power of regret, but this time as it connects to grieving events in the past. He’d observed that people experiencing grief try to “undo” the pain of a tragedy by dwelling on what might have happened to prevent it (hence the term Undoing; we explore this concept further in the next part of the guide).

(Shortform note: The most common understanding of grief is based on the five stages identified by Elisabeth Kübler-Ross in On Death and Dying—denial, anger, bargaining, depression, and acceptance. However, the “five stages'' concept has been criticized as unscientific. To be fair, in her book On Grief and Grieving, Kübler-Ross herself points out that she never intended for the five stages to be interpreted as a literal, linear progression.)

Because of their geographic separation, Kahneman developed most of his work on Undoing without being able to bounce ideas off his partner. Lewis notes that Tversky contributed some material to the project, but he spent much of his time giving lectures abroad. Tversky, for his part, thought they were still a team, though over the next few years he began to receive awards and recognition, while Kahneman was largely left out.

Lewis makes it clear that Tversky rebuked anyone who painted him as the more important of the pair. Nevertheless, he was the one whom other academics came to for insights on how prospect theory related to their work. Kahneman was aware of Tversky’s spotlight and recognized that he’d grown envious of the recognition his partner was receiving. (Shortform note: In academia, where reputation is valuable currency, it matters very much who’s given credit for an idea. However, reacting with anger and resentment is often counterproductive. Claiming credit back when it’s unfairly given elsewhere does require an active effort to assert ownership for your contributions, which unfortunately went against Kahneman’s style.)

Throughout the ’80s, they continued to publish papers jointly, though each was written mostly by one and not the other. Their work also began to be attacked by academics whose research they’d thrown into question. Some challenged prospect theory in particular, while others rejected Kahneman and Tversky’s basic premise that human beings are fundamentally irrational.

(Shortform note: Present-day opponents of prospect theory criticize it for being too mathematical, relying on restrictive assumptions, and not taking neurological processes into account. Those who dispute Tversky and Kahneman’s findings on heuristics claim that they overstate their evidence, misrepresent the complexity of intuition, and unfairly malign the benefits of heuristics.)

Conflict and Tversky’s Death

Lewis writes that Kahneman’s impulse was to avoid conflict, whereas Tversky wanted to go on the warpath. He pushed Kahneman to coauthor a paper that would show conclusively that the mind ignored logic. Their paper was published in 1983, but the process of writing it made Kahneman miserable; it reflected Tversky’s antagonism toward their critics more than Kahneman’s level-headed approach.

(Shortform note: In addition to personality differences, another major source of contention between research psychologists arises from the problem that many studies cannot be reproduced, to the point that researchers have hotly debated whether a study on the reproducibility problem itself is reproducible. While some claim that psychological research is inherently flawed to begin with, recent studies have continued to confirm Tversky and Kahneman’s findings.)

In 1986, Kahneman took a position at UC Berkeley, only 40 miles from where Tversky worked at Stanford, but being in such proximity made Kahneman so unhappy that in 1992 he left California for Princeton in New Jersey. Lewis says that Kahneman considered their partnership over, but Tversky reached out in 1993 to cajole him into writing one more paper responding to a critic, Gerd Gigerenzer. The process of crafting their reply was so contentious that for Kahneman it not only ended their collaboration, but it also marked the close of their friendship.

He told Tversky as much, and almost immediately afterward, Tversky was diagnosed with terminal cancer. Lewis recounts that Kahneman was one of the first people Tversky reached out to, and though Tversky faced his death with peace and stoic calm, his mortality reminded Kahneman of the value of their friendship. Tversky and Kahneman spoke nearly every day until Tversky passed away in June 1996.

The Marriage of True Minds

While many people found Kahneman and Tversky’s intellectual marriage surprising, given their different personalities, their intense relationship and subsequent breakup wouldn’t seem strange to experts on marriage. In Getting the Love You Want, Harville Hendrix and Helen Hunt explain that people are attracted to partners who fill in the missing parts of themselves, but that over time, the traits that connect them become the same ones that drive them apart.

Hendrix and Hunt contend that to manage these differences and maintain a strong bond, partners should engage in a structured dialogue that forces them to communicate rationally. This involves mirroring the other person’s statements, validating their point of view, and responding with empathy.

Lewis writes that after Tversky’s death, Kahneman finally began to receive the attention and acclaim he’d been denied, culminating in a Nobel Prize in Economics in 2002 and the publication of his seminal work, Thinking, Fast and Slow, in 2011. (Shortform note: In honor of Tversky, Stanford University held a multidisciplinary symposium to highlight his legacy. Kahneman moved on to study hedonic psychology, the science of pleasure and suffering. In addition to his Nobel Prize, in 2013 he was awarded the Presidential Medal of Freedom.)

Three Major Breakthroughs

While describing the history of Kahneman and Tversky’s collaboration, Lewis shows that between 1969 and 1979 they overturned our understanding of how we make decisions not once, but three times. The lessons from that decade of research call into question much of what we think about what we think. Tversky and Kahneman revealed that the mind relies on impressions over logic, that we can be tricked into making illogical decisions based on how choices are presented, and that given the chance, our minds rewrite reality in order to avoid the pain of regret. We’ll explore each of these ideas in detail.

The Theory of Heuristics

Kahneman and Tversky’s first breakthrough was determining that when making judgments, the mind doesn’t unconsciously calculate statistics, as was the common belief of the time. Instead, they showed that the mind applies stories and stereotypes through processes that Tversky and Kahneman called heuristics. In short, our minds use a variety of shortcuts to make guesses when we don’t have enough information.

In their research, they identified three separate heuristics that systematically cloud human judgment—representativeness, availability, and anchoring. (Shortform note: Heuristics have become a standard psychological tool for describing the mental shortcuts the brain takes when evaluating judgments and decisions. In Algorithms to Live By, Brian Christian and Tom Griffiths argue that you can train yourself to use heuristics derived from computer programming to make better decisions and optimize your time. Algorithms, they claim, let us make more efficient use of our limited memory and attention so we can avoid analysis paralysis and decision fatigue.)

Representativeness

The representativeness heuristic describes the way our minds persistently compare people and events to stereotypes and other assumptions based on past interactions. Lewis points out that from an evolutionary perspective, this heuristic is a handy mental measure to speed up decision-making in crucial situations—such as determining whether that shadow up ahead is a panther about to attack you from a tree. However, Tversky and Kahneman showed that this heuristic breaks down when random elements are involved.

According to Kahneman and Tversky, the human mind has a fundamental misunderstanding of randomness, to the point that we concoct inaccurate beliefs to explain why random things happen. In particular, as Lewis explains, people find it hard to accept that randomness naturally generates clusters that look like patterns even when they’re not. Instead, we wrongly expect randomness to create an even spread. For example, we think flipping a coin will produce heads and tails in equal numbers, when nothing guarantees that that will be the case. To our minds, an even distribution is more “representative” of what we believe a random sample will produce, so we concoct erroneous stories to explain any coincidences that naturally occur.

(Shortform note: There’s nothing special about coincidence; randomness guarantees that it will occur. In The Improbability Principle, mathematician David Hand invokes the Law of Truly Large Numbers to explain that the sheer number of possibilities and opportunities in the world makes oddball occurrences statistically inevitable. They only seem significant because our minds demand a narrative to explain why they happen.)

Lewis says that this cognitive error is problematic in disciplines like psychology, social science, and even medicine, where research is performed on small sample groups that may not represent the larger population because of random factors in test group selection. (Shortform note: The smaller the sample size used in a study, the larger its margin of error. In a paper published in 2005, medical professor John Ioannidis asserts that most current research findings are flawed due to the statistical limitations of the studies they’re based on.)

Availability

Availability, the second heuristic, states that you will consider any given scenario more likely if you can easily recall a similar situation. Lewis states that this heuristic makes us draw conclusions based on common occurrences, recent events, or anything that’s heavy on our minds. For example, after watching a movie about a serial killer, you may suddenly be afraid of becoming a victim yourself, even though the actual likelihood did not change after seeing the film. (Shortform note: While this effect may seem innocuous, it can have a negative impact on decision-making, such as when a manager overlooks an employee’s good record in favor of one recent mistake. It can also increase your anxiety by making you dwell on unlikely events.)

Lewis says that as with representativeness, the availability heuristic makes sense from an evolutionary standpoint—scenarios that occur more frequently may indeed be more likely than others. However, on the societal level, this heuristic leads to self-reinforcing systemic bias. For an individual, it can trick you into drawing poor conclusions when the proper evidence isn’t readily available, but misleading information is.

(Shortform note: In Biased, Jennifer Eberhardt goes into detail about the self-reinforcing societal impact of both the availability and representativeness heuristics, though she does not cite the heuristics by name. Availability comes into play when negative depictions of Black people in media create more “available” memories from which people make associations. Representativeness comes into play in what Everhardt dubs the other-race effect, in which people judge individuals based on preconceptions about the group they belong to.)

Anchoring

Anchoring, the third heuristic, is a phenomenon related to how the mind deals with numbers. Tversky and Kahneman found that when asking test subjects to estimate numbers, their guesses could be manipulated by “priming” their subjects with irrelevant information.

For example, if students were told, “There are 14 lines in a sonnet,” then asked to guess how many countries there are in Africa, their answers would tend to be low. If another group were told, “There are 5,000 students enrolled in this college,” their guesses to the number of countries would be high. (The correct answer as of 2022 is 54.) Lewis claims that Kahneman and Tversky weren’t able to identify why the brain behaves like this, but the fact that it does reveals another way that the mind is vulnerable to error.

How Numbers Fool the Mind

Kahneman elaborates on the anchoring effect in Thinking, Fast and Slow, where he identifies two mechanisms that cause it. First, the mind uses the anchor number as an initial guess from which to adjust. Second, the mind makes an association with the anchor—high or low, big or small, long or short—that colors the narrative within which the guess is made.

Others have noted that the anchoring effect is of particular use to marketers, who influence customers’ perceptions about price by anchoring their impressions about how much a product should cost. Some research has suggested that anchoring affects moral judgments as well, such as when one person’s ethical opinion is used as an anchor for those of others.

The Theory of Risk

After establishing the systematic ways in which the mind can be fooled into error, Kahneman and Tversky turned their attention to how we assess risk when making decisions. Whereas economists approached the problem strictly in terms of financial gain, Lewis writes that Tversky and Kahneman took a broader tack and discovered that people are far more influenced by the desire to avoid potential loss. Their “prospect theory” explains why we take risks, how we define loss, and how our choices can be influenced by the way risks are presented.

When Kahneman and Tversky began to explore how people reacted to possible losses rather than theoretical gains, they learned the drive to avoid loss is very strong. Lewis says that in order for subjects to accept a gamble with a potential loss, the promised payout has to far exceed the possible cost they might incur, as it does when playing the slots at a casino. Generalized more broadly, people dislike losing what they have far more than they enjoy getting what they want.

(Shortform note: The promise of large payouts for relatively small risks is common to both Las Vegas and Wall Street, leading to what’s known as the casino mentality of investing. This mentality contributed to the 2008 global financial crisis when lenders were encouraged to make high-risk investments disguised as low-risk opportunities. Nevertheless, there are those who still recommend gambling strategies in financial investment.)

Tversky and Kahneman also discovered that people react to changes in value, not the absolute value of their situation. For example, if Bob is demoted to a middle management position, while Lucy is promoted into middle management, Lucy is happy while Bob is upset, even though they both now hold the same job. Lewis points out that the determining factor is the change in each person’s status quo, and it matters how that status quo is framed.

(Shortform note: In framing wealth, most people perceive “the rich” to be those earning more than themselves, regardless of their personal income. Investor Jacob Schroeder says that by using the framing effect on yourself and choosing to look for potential gains even in negative situations, you can make better choices about your wealth and overall happiness. This echoes the advice of Norman Vincent Peale, who contended In The Power of Positive Thinking that happiness is rooted in a mindset you can choose to adopt by reframing your outlook on life.)

In summary, Kahneman and Tversky found that people are risk-averse when considering gains, and more risk-taking when facing a loss. Therefore, Lewis says, our judgments can be manipulated depending on whether our circumstances are framed as positive or negative in relation to our perceived status quo. For example, if Lucy expects a $5,000 bonus, but then receives only $4,000, her gain is perceived as a loss. (Shortform note: What’s key to our perception of loss vs. gain is our individual reference point from which each is measured. Business professors have used this concept in the investment market to determine whether stocks are over- or undervalued.)

“Framing,” it turns out, is a powerful tool that can be employed by businesses marketing goods, politicians swaying votes, or doctors persuading patients to have surgery. Lewis relates that Tversky and Kahneman’s results led to an axiom—we don’t choose between options, we choose between how those options are described.

(Shortform note: Being aware of the power of framing is a useful tool for reminding us that anyone presenting information may have an unspoken agenda. In Nudge, Richard Thaler and Cass Sunstein explicitly lay out a program on how to use framing to advance a particular political agenda. Tools they suggest for reframing political decisions include offering “default” choices, providing clear links between decisions and outcomes, and narrowing the number of options to choose from.)

The Theory of Undoing

According to Lewis, the last major project to come out of Kahneman and Tversky’s collaboration was when they pivoted from looking at how people make decisions about the future to how they deal with the past. At the root of this shift were Kahneman’s thoughts about how people avoid feelings of regret, and the mental permutations they go through in order to cope with regret in the present. In their studies, Tversky and Kahneman uncovered a fourth heuristic of the mind, one in which we create alternate realities to avoid the pain of tragedy and frustration.

Lewis says they named their new mental model the simulation heuristic, referring to the power of “what might have been” to cloud present-day judgments and decision-making. Imagining an alternate, happier life offers a temporary salve to feelings of sorrow, but it also contaminates our perception of reality by evoking feelings of envy and regret for paths not taken.

Dwelling on the Past vs. Living in the Present

Many wellness experts warn against the dangers of engaging the simulation heuristic; instead, they espouse living in the moment as a balm for anxiety and depression. In The Power of Now, Eckhart Tolle explains that ruminating on the past leads to resentment and bitterness, while accepting the highs and lows of the present allows you to face them without wishing them away.

This thought is echoed in Radical Acceptance by psychologist Tara Brach, who says the stories we create to undo our frustrations are unhealthy coping mechanisms that only lead to us feeling unworthy and unhappy. She argues that accepting the present as it is allows us to recognize our reality and treat ourselves with compassion.

Kahneman and Tversky suggested that these “might have been” fantasies use counterfactual emotions to cover up uncomfortable realities. Lewis explains that the strength of those emotions depends on how close the alternate reality is to the present. For example, imagining a different career choice made 20 years ago carries less emotional weight than imagining you made a different choice yesterday. What also determines the strength of counterfactual feelings is how realistic and desirable the alternate reality seems. For example, imagining that you could have dealt with a problem at work more gracefully engenders stronger feelings than imagining you could’ve avoided the issue by spending the last 10 years as a beachcomber.

(Shortform note: The feelings aroused by missed opportunities may become even more harmful as we age. Research has shown that letting go of what might have been leads to better mental health later in life. In The Power of Positive Thinking, Norman Peale recommends creating a daily ritual to move on from whatever mistakes you might have made.)

Kahneman dubbed the process of rewriting painful events “undoing,” which he interpreted as a coping mechanism to deal with life’s infinite possibilities. Lewis states that in their research, Kahneman and Tversky established four rules the mind follows when undoing the past:

1. The more separate events we have to undo at once, the less likely we are to do so.
2. The farther events retreat into the past, the harder they become to undo.
3. In undoing, we imagine a person making a different choice, and not merely a different random circumstance.
4. In undoing an event, we tend to disregard any unexpected aspect of what occurred.

Because of the process of undoing, Tversky observed that for the mind, reality isn’t fixed. Instead, it’s a haze of possibilities. (Shortform note: Further work on the process of undoing was continued by Kahneman’s colleague Barbara Fredrickson, who revealed that positive emotions can undo the physiological effects of negative emotions. However, a later study by Melissa Falkenstern showed that the cognitive effects of negative emotions aren't as easily undone.)