PDF Summary:Continuous Discovery Habits, by

Book Summary: Learn the key points in minutes.

Below is a preview of the Shortform book summary of Continuous Discovery Habits by Teresa Torres. Read the full comprehensive summary at Shortform.

1-Page PDF Summary of Continuous Discovery Habits

Has your company been struggling to meet the needs of consumers despite extensive work on product development? Your problem may lie in how well you understand your customers. In Continuous Discovery Habits, product discovery coach and consultant Teresa Torres presents a transformative approach to product management, arguing that successful product teams must engage in continuous discovery alongside their delivery efforts. Rather than treating customer research as a sporadic activity, Torres advocates making it an integral weekly practice.

In our guide, we’ll first explain what discovery is. Then, we’ll break Torres’s ideas down into five steps you can incorporate into your current product management process. In our commentary, we’ll supplement Torres’s ideas with other experts’ perspectives on product development, advice for putting Torres’s ideas into practice, and research that supports Torres’s ideas.

(continued)...

Once you’ve gathered all this information, have your three-person product development team reflect on it and draw conclusions about customer needs and wants. From there, determine what problems you want to address and what end goal you want to achieve. This will lead you to your next step: brainstorming solutions.

The Drawbacks of Story-Based Interviewing

While story-based interviewing can help you get a clearer picture of a customer’s experience, it may still be subject to biases or errors. People may demonstrate memory biases or distortions—the recency bias can cause customers to primarily remember experiences that happened more recently, for example. The choice-supportive bias might lead them to overemphasize aspects of their experience that were the result of choices they made rather than aspects that happened to them without their choosing. Episodic memory (the memory of experiences we’ve had) is also especially prone to distortion, meaning that customers’ recalling of events may not always be accurate.

You may find it beneficial to devise other means of gathering information on customer needs and wants to inform your product development. For example, some experts recommend gathering feedback from customers immediately after providing them with a service, which could prevent the memory distortions that happen over time from affecting their input.

Step 2: Brainstorm Solutions

Torres points out that product development teams often already have a few ideas compiled for solving issues they’ve noted. However, these tend to be the first or second solutions they’ve thought of, and research shows that our first ideas are rarely our best ones. Instead, it’s important to generate many ideas in order to stimulate the creative process and come up with the best solution.

(Shortform note: In Where Good Ideas Come From, Steven Johnson expounds on how good ideas develop. He explains that ideas don’t magically appear out of nothing; rather, they build on existing knowledge and ideas, which can help explain why the more ideas you come up with, the better each new idea will be. These earlier ideas act as platforms on top of which you can develop higher-level ideas, resulting in higher-quality solutions the more time you spend ideating.)

Torres argues in favor of switching between brainstorming as individuals and brainstorming as a group. Research shows that individuals tend to come up with more ideas—and more creative ones—than people working in groups. However, individuals can sometimes get stuck, which slows progress. By brainstorming individually first, then coming together and sharing ideas as a group, product development teams can achieve optimal productivity and creativity in coming up with solutions.

To put this process into action, alternate between individual and group brainstorming until you have at least 15 ideas. Then, take multiple group votes, gradually eliminating the least popular solutions, until you’re down to three. You’ll examine these solutions more closely in the following steps, beginning with the assumptions that underpin them.

(Shortform note: Another benefit of switching between individual and group brainstorming is that it can help equally distribute the cognitive load the process requires. Research shows that, in group settings, the majority of the work is often done by just a few individuals, with other group members participating little or not at all. Certain groups may be more prone to this unequal distribution than others; women, for example, often do more work in group settings and get less credit than men. By expecting team members to come to the group session having already generated ideas individually, teams can reduce this discrepancy and create a collaborative setting that’s both more equitable and more productive.)

Step 3: Identify Your Assumptions

The next step is to identify the underlying assumptions behind the decisions you’re making—assumptions you’re likely unaware of. Every assumption you make represents a risk that your solution won’t succeed. You may have come up with three solutions that seem promising, but if they’re based on faulty assumptions, they’ll be doomed to failure.

Torres outlines five key categories of assumptions that product teams need to consider: Desirability assumptions relate to whether customers want and will value the solution. Viability assumptions concern whether the solution makes business sense and will provide adequate returns. Feasibility assumptions address whether the team can build the solution from both technical and organizational perspectives. Usability assumptions examine whether customers can use the solution. Ethical assumptions consider potential harms and negative impacts of the solution.

(Shortform note: We make assumptions because our brains evolved to take shortcuts in order to reduce the amount of information we’d need to process to make decisions. These shortcuts helped our ancestors survive in situations where their lives depended on acting quickly, but today these same shortcuts can sometimes prevent us from making the best decisions—for example, by assuming customers want a product they don’t, assuming a solution will be profitable when it won’t, or assuming a business practice is ethical when it’s not.)

Story Mapping to Identify Assumptions

To uncover your underlying assumptions, Torres recommends using story mapping. Story mapping involves laying out each step users need to take to get value from a solution, which helps reveal underlying assumptions at each step. To do this, note which people or entities need to interact for the customer to access your solution and map these out chronologically. Then note any assumptions you’re making at each stage of these interactions.

For example, if you’re developing a video game, your important people and entities could include the customer, the games store, and the console the customer uses. The interactions could be as follows: 1) The customer goes to the online game store to look for something to buy. 2) The game store shows your game as purchasable. 3) The customer buys the game. 4) The customer plays the game on their console. At stage 1—the customer visiting your online store—you might be assuming that customers want the type of game you’re offering—a desirability assumption. At stage 2, you’re assuming the customer knows how to navigate the game store—a usability assumption. There’ll also be other assumptions for stages 3 and 4.

(Shortform note: Story mapping was developed by Jeff Patton in the early 2000s and was intended to help product teams better understand their customers. Story mapping works best when it’s done collaboratively, with everyone on the team participating equally and challenging or editing wherever they see fit. It also works best when the maps are continually updated in response to new information and customer feedback.)

You may not need to go through the whole story-mapping process every time. Torres explains that, as teams develop their skills at spotting assumptions, they may naturally move away from using formal methods. The key is to use whatever methods help address the team’s particular blind spots, as most teams tend to have biases toward certain categories of assumptions while overlooking others. For example, the product development team in our video game example might be adept at spotting desirability assumptions—knowing what will appeal to gamers—but struggle with viability assumptions, leading them to try to add features that can’t be supported with current technology.

(Shortform note: Expertise development explains why teams may move from relying on the formal story-mapping process to eventually being able to do it informally. As people spend time on certain tasks or skills, they gradually develop greater proficiency, which can eventually turn into mastery with enough practice. As teams get more accustomed to noticing their blind spots, they’ll also get better at fixing those blind spots—eventually eliminating them as “blind” spots altogether. However, depending on the skill, the people involved, and the amount of practice, expertise development can take a very long time, so don’t be concerned if your team is still relying on formal methods months or even years into the process.)

Step 4: Test Your Assumptions

Once you’ve identified your assumptions, test them one at a time. Torres says your goal is to assess whether you have sufficient evidence to believe that an assumption is true. If you don’t, you’ll need to remove that assumption from your decision-making process and revise your product development pipeline accordingly. This helps you mitigate your risks and ensure you’re making the best, most well-thought-out product development decisions possible.

(Shortform note: To determine which assumptions to test first, consider using the assumption slam model. This is a chart that allows you to graph your assumptions based on their importance and how much evidence (or knowledge) you already have that they’re correct. You should prioritize testing the assumptions that are high in importance but low in knowledge, as these represent the greatest risk if they turn out to be wrong.)

To test your assumptions, Torres recommends that you simulate a customer experience. Rather than asking customers what they would do hypothetically, recruit customers to participate in simulations where they can demonstrate actual behavior. These simulations should be kept minimal, focusing only on the specific moment needed to test the assumption. You also need to decide in advance what type of results will serve as the evidence you need so you know what to do with the data as you’re gathering it.

(Shortform note: One potential drawback of simulating customer experiences is that customers may alter their behavior when they know they’re being observed—a tendency called the Hawthorne effect. Similarly, subjects may engage in certain behaviors because of subtle cues from experimenters—called demand characteristics—that make them think that behavior is expected. To avoid these risks, conduct your experiment in as natural a setting as possible—that is, a setting that’s very close to the experience a user would have outside of the experiment. You can also avoid telling participants what hypothesis you’re testing for so they’ll be less tempted to change their behavior to confirm the hypothesis.)

Torres advocates starting with small-scale tests before moving to larger experiments. While small tests may result in false positives or false negatives, these risks are generally acceptable because the cost of being wrong is relatively low. She explains that product teams aren’t conducting scientific research—they’re trying to mitigate risk and create value for customers. Therefore, while teams should adopt scientific thinking, they don’t need the same level of rigor as academic research.

Returning to our video game example, you might decide to test your first assumption (that customers want the type of game you’re offering). You decide that, to verify that this assumption is true, at least four out of 10 simulation participants need to choose your game type over other game types. You carry out your simulation by presenting participants with a list of games available to purchase. If your game is a first-person shooter, but the participants in your simulation only purchase platform games and puzzle games, you have evidence that your assumption may be wrong. This represents a risk in your solution idea, which you should take into account as you continue to make product development decisions.

(Shortform note: Other experts support Torres’s advice to start with small-scale tests before doing larger projects. In Great by Choice, Jim Collins and Morten T. Hansen researched companies that performed 10 times better than other companies during times of upheaval. They found that companies that started with small experiments to find what works (what they refer to as “bullets”) before taking on larger, riskier experiments or product rollouts (which they refer to as “cannonballs”) were more likely to be among these high-performance companies. The authors advise that in designing your small tests or “bullets,” you make sure they’re low-cost, have minimal consequences for failure, and don’t disrupt your company on a large scale.)

Step 5: Evaluate Effectiveness of Discovery

Now that you’ve gathered all this discovery information, you can apply it to product development. At this stage, you’ll start doing your delivery work in tandem with your ongoing discovery work. While some view delivery and discovery as separate processes, Torres argues that discovery and delivery are deeply intertwined—discovery work often requires some delivery to test assumptions in a real environment, and delivery work generates new insights that can be fed back into discovery.

(Shortform note: Some companies separate teams for discovery and delivery, but this stage of integrating the two means that it’s important to have the same people working on both of these processes. While separate teams can offer different benefits and innovations, it’s impractical to have one team coming up with innovative solutions and a completely different team implementing them—separating your discovery and delivery teams means that your delivery team often won’t know how to implement solutions from discovery, and the discovery team might not be able to incorporate insights gained from delivery. Thus, product management expert Marty Cagan asserts that the same team responsible for discovery should be responsible for delivery.)

Torres says that in Step 5, you’ll measure how effective your solutions are in meeting your customers’ needs and wants and use that information to refine your product. The key here is to evaluate how effective your assumption tests are using real-world—rather than experimental—data collection. You’re putting your solutions into action and assessing whether they’re bringing you closer to your end goal. Torres emphasizes that teams shouldn’t try to measure everything right away. Instead, they should begin by identifying what metrics are needed to evaluate their current assumption tests.

One important distinction she makes is between measuring the number of people who take an action versus the number of actions taken. She explains that the choice depends on whether value comes from many people taking an action once or fewer people taking an action multiple times. In our video game example, you might measure the views you get for your game in the online game store. Since most people won’t buy multiple copies of the same game, the number of people who view your game will likely be the most useful metric. On the other hand, if you’re trying to evaluate the usefulness of a new in-game mechanic to see how often players will use it, the total number of actions will probably give you the best insight.

Choose what metrics will help you measure the effectiveness of your tests, integrate your delivery and discovery work so you can apply what you learn in each process to the other, and repeat the cycle. This, in combination with the other steps listed above, is how you can embody the process of ongoing discovery.

Choosing Metrics: Avoid Vanity Metrics and Map Value Streams

In The Lean Startup, Eric Ries provides some advice on what metrics to choose to measure progress. He warns against relying on “vanity metrics,” or those that look good but provide a false sense of progress without offering meaningful insights. He notes that the total number of anything (such as sales, downloads, or followers) will usually be a vanity metric because they increase over time regardless of what solutions you implement. Make sure the metrics you choose reflect the actual quality of the solutions you’re testing and aren’t just attractive but meaningless numbers.

To identify important metrics, consider using value-stream mapping to chart out where value is being created by your product and what pain points are detracting from that value. By mapping out where current value is coming from and comparing it to what you want to see in the future, you can more easily identify areas of improvement and the corresponding metrics (such as the number of people taking an action versus the number of actions taken) you need to use to measure progress in those areas.

Want to learn the rest of Continuous Discovery Habits in 21 minutes?

Unlock the full book summary of Continuous Discovery Habits by signing up for Shortform.

Shortform summaries help you learn 10x faster by:

  • Being 100% comprehensive: you learn the most important points in the book
  • Cutting out the fluff: you don't spend your time wondering what the author's point is.
  • Interactive exercises: apply the book's ideas to your own life with our educators' guidance.

Here's a preview of the rest of Shortform's Continuous Discovery Habits PDF summary:

What Our Readers Say

This is the best summary of Continuous Discovery Habits I've ever read. I learned all the main points in just 20 minutes.

Learn more about our summaries →

Why are Shortform Summaries the Best?

We're the most efficient way to learn the most useful ideas from a book.

Cuts Out the Fluff

Ever feel a book rambles on, giving anecdotes that aren't useful? Often get frustrated by an author who doesn't get to the point?

We cut out the fluff, keeping only the most useful examples and ideas. We also re-organize books for clarity, putting the most important principles first, so you can learn faster.

Always Comprehensive

Other summaries give you just a highlight of some of the ideas in a book. We find these too vague to be satisfying.

At Shortform, we want to cover every point worth knowing in the book. Learn nuances, key examples, and critical details on how to apply the ideas.

3 Different Levels of Detail

You want different levels of detail at different times. That's why every book is summarized in three lengths:

1) Paragraph to get the gist
2) 1-page summary, to get the main takeaways
3) Full comprehensive summary and analysis, containing every useful point and example