What hidden beliefs might be sabotaging your product’s success? How can you uncover and test these critical assumptions before investing significant resources?
Product development teams often make crucial decisions based on untested assumptions, which can lead to costly failures. In Continuous Discovery Habits, Teresa Torres provides a systematic approach to identifying and testing product assumptions across five key categories.
Keep reading to discover practical methods for uncovering your product assumptions and learning how to test them effectively.
Identify Product Assumptions
Torres asserts that it’s important to identify the underlying assumptions behind the decisions you’re making—assumptions you’re likely unaware of. Every assumption you make represents a risk that your solution won’t succeed. You may have come up with three solutions that seem promising. But, if they’re based on faulty assumptions, they’ll be doomed to failure.
Torres outlines five key categories of product assumptions that teams need to consider:
- Desirability assumptions relate to whether customers want and will value the solution.
- Viability assumptions concern whether the solution makes business sense and will provide adequate returns.
- Feasibility assumptions address whether the team can build the solution from both technical and organizational perspectives.
- Usability assumptions examine whether customers can use the solution.
- Ethical assumptions consider potential harms and negative impacts of the solution.
(Shortform note: We make assumptions because our brains take shortcuts in order to reduce the amount of information we’d need to process to make decisions. These shortcuts helped our ancestors survive in situations where their lives depended on acting quickly, but today these same shortcuts can sometimes prevent us from making the best decisions—for example, by assuming customers want a product they don’t, assuming a solution will be profitable when it won’t, or assuming a business practice is ethical when it’s not.)
Story Mapping to Identify Assumptions
To uncover your underlying assumptions, Torres recommends using story mapping. Story mapping involves laying out each step users need to take to get value from a solution, which helps reveal underlying assumptions at each step. To do this, note which people or entities need to interact for the customer to access your solution and map these out chronologically. Then note any assumptions you’re making at each stage of these interactions.
For example, if you’re developing a video game, your important people and entities could include the customer, the games store, and the console the customer uses. The interactions could be as follows: 1) The customer goes to the online game store to look for something to buy. 2) The game store shows your game as purchasable. 3) The customer buys the game. 4) The customer plays the game on their console. At Stage 1—the customer visiting your online store—you might be assuming that customers want the type of game you’re offering—a desirability assumption. At Stage 2, you’re assuming the customer knows how to navigate the game store—a usability assumption. There’ll also be other assumptions for Stages 3 and 4.
You may not need to go through the whole story-mapping process every time. Torres explains that, as teams develop their skills at spotting assumptions, they may naturally move away from using formal methods. The key is to use whatever methods help address the team’s particular blind spots, as most teams tend to have biases toward certain categories of assumptions while overlooking others. For example, the product development team in our video game example might be adept at spotting desirability assumptions—knowing what will appeal to gamers—but struggle with viability assumptions, leading them to try to add features that can’t be supported with current technology.
Test Your Product Assumptions
Once you’ve identified your assumptions, test them one at a time. Torres says your goal is to assess whether you have sufficient evidence to believe that an assumption is true. If you don’t, you’ll need to remove that assumption from your decision-making process and revise your product development pipeline accordingly. This helps you mitigate your risks and ensure you’re making the best, most well-thought-out product development decisions possible.
(Shortform note: To determine which assumptions to test first, consider using the assumption slam model. This is a chart that allows you to graph your assumptions based on their importance and how much evidence (or knowledge) you already have that they’re correct. You should prioritize testing the assumptions that are high in importance but low in knowledge, as these represent the greatest risk if they turn out to be wrong.)
To test your assumptions, Torres recommends that you simulate a customer experience. Rather than asking customers what they would do hypothetically, recruit customers to participate in simulations where they can demonstrate actual behavior. These simulations should be kept minimal, focusing only on the specific moment needed to test the assumption. You also need to decide in advance what type of results will serve as the evidence you need so you know what to do with the data as you’re gathering it.
(Shortform note: One potential drawback of simulating customer experiences is that customers may alter their behavior when they know they’re being observed—a tendency called the Hawthorne effect. Similarly, subjects may engage in certain behaviors because of subtle cues from experimenters—called demand characteristics—that make them think that behavior is expected. To avoid these risks, conduct your experiment in as natural a setting as possible—that is, a setting that’s very close to the experience a user would have outside of the experiment. You can also avoid telling participants what hypothesis you’re testing for so they’ll be less tempted to change their behavior to confirm the hypothesis.)
Torres advocates starting with small-scale tests before moving to larger experiments. While small tests may result in false positives or false negatives, these risks are generally acceptable because the cost of being wrong is relatively low. She explains that product teams aren’t conducting scientific research—they’re trying to mitigate risk and create value for customers. Therefore, while teams should adopt scientific thinking, they don’t need the same level of rigor as academic research.
Returning to our video game example, you might decide to test your first assumption (that customers want the type of game you’re offering). You decide that, to verify that this assumption is true, at least four out of 10 simulation participants need to choose your game type over other game types. You carry out your simulation by presenting participants with a list of games available to purchase. If your game is a first-person shooter, but the participants in your simulation only purchase platform games and puzzle games, you have evidence that your assumption may be wrong. This represents a risk in your solution idea, which you should take into account as you continue to make product development decisions.
(Shortform note: Other experts support Torres’s advice to start with small-scale tests before doing larger projects. In Great by Choice, Jim Collins and Morten T. Hansen researched companies that performed 10 times better than other companies during times of upheaval. They found that companies that started with small experiments to find what works (what they refer to as “bullets”) before taking on larger, riskier experiments or product rollouts (which they refer to as “cannonballs”) were more likely to be among these high-performance companies. The authors advise that in designing your small tests or “bullets,” you make sure they’re low-cost, have minimal consequences for failure, and don’t disrupt your company on a large scale.)
Exercise
Consider Torres’s five categories of assumptions (desirability, viability, feasibility, usability, and ethical). List at least three key assumptions you’re making about your product or service, and identify which category each assumption falls into.
