What makes product discovery and delivery work together effectively? How can you measure whether your product solutions are actually working?
In Continuous Discovery Habits, Teresa Torres explores the interconnected nature of product discovery and delivery. She explains how to gather meaningful data, choose the right metrics, and use real-world testing to refine your products.
Keep reading to learn how successful teams integrate product discovery and delivery to create solutions that truly serve their customers’ needs.
Product Discovery and Delivery
Once you’ve gathered discovery information, you can apply it to product development. At this stage, you’ll start doing your delivery work in tandem with your ongoing discovery work. While some view product discovery and delivery as separate processes, Torres argues that they are deeply intertwined—product discovery work often requires some delivery to test assumptions in a real environment, and product delivery work generates new insights that can be fed back into discovery.
(Shortform note: Some companies separate teams for product discovery and delivery, but this stage of integrating the two means that it’s important to have the same people working on both of these processes. While separate teams can offer different benefits and innovations, it’s impractical to have one team coming up with innovative solutions and a completely different team implementing them—separating your discovery and delivery teams means that your delivery team often won’t know how to implement solutions from discovery, and the discovery team might not be able to incorporate insights gained from delivery. Thus, product management expert Marty Cagan asserts that the same team responsible for discovery should be responsible for delivery.)
Torres says that, in this step of the product discovery process, you’ll measure how effective your solutions are in meeting your customers’ needs and wants and use that information to refine your product. The key here is to evaluate how effective your assumption tests are using real-world—rather than experimental—data collection. You’re putting your solutions into action and assessing whether they’re bringing you closer to your end goal. Torres emphasizes that teams shouldn’t try to measure everything right away. Instead, they should begin by identifying what metrics are needed to evaluate their current assumption tests.
One important distinction she makes is between measuring the number of people who take an action versus the number of actions taken. She explains that the choice depends on whether value comes from many people taking an action once or fewer people taking an action multiple times. In our video game example, you might measure the views you get for your game in the online game store. Since most people won’t buy multiple copies of the same game, the number of people who view your game will likely be the most useful metric. On the other hand, if you’re trying to evaluate the usefulness of a new in-game mechanic to see how often players will use it, the total number of actions will probably give you the best insight.
Choose what metrics will help you measure the effectiveness of your tests, integrate your delivery and discovery work so you can apply what you learn in each process to the other, and repeat the cycle. This, in combination with the other steps listed above, is how you can embody the process of ongoing discovery.
| Choosing Metrics: Avoid Vanity Metrics and Map Value Streams In The Lean Startup, Eric Ries provides some advice on what metrics to choose to measure progress. He warns against relying on “vanity metrics,” or those that look good but provide a false sense of progress without offering meaningful insights. He notes that the total number of anything (such as sales, downloads, or followers) will usually be a vanity metric because they increase over time regardless of what solutions you implement. Make sure the metrics you choose reflect the actual quality of the solutions you’re testing and aren’t just attractive but meaningless numbers. To identify important metrics, consider using value-stream mapping to chart out where value is being created by your product and what pain points are detracting from that value. By mapping out where current value is coming from and comparing it to what you want to see in the future, you can more easily identify areas of improvement and the corresponding metrics (such as the number of people taking an action versus the number of actions taken) you need to use to measure progress in those areas. |
Exercise
Based on Torres’s guidance about measuring effectiveness, identify which metrics would be most meaningful for evaluating your product’s success. Explain why you chose these metrics and how they connect to your desired outcome.
