This article is an excerpt from the Shortform summary of "The Lean Startup" by Eric Ries. Shortform has the world's best summaries of books you should be reading.
Like this article? Sign up for a free trial here .
You know that in Lean Startup, you should be avoiding Vanity Metrics. These tell you less useful facts about your business and can be a misleading sign of progress.
Instead, look at real metrics with cohort analysis. But what is cohort analysis, and how do you do it?
Cohort Analysis
Instead of looking at cumulative vanity metrics over time, the more accurate analysis is to separate users into groups based on the time they joined, then measure your metric for each group independently. Each group of users is called a cohort.
For example, say we wanted to measure engagement in an app by number of photos sent. Each week, we take all the users who joined that week, and then look at the average number of photos each users send in their first day. We work really hard for 4 weeks, and we hope to see this number rise. Instead we see this:
| Number of photos sent per user | Vanity metric – total photos sent | |
| Week 1 | 5 | 100 |
| Week 2 | 5 | 200 |
| Week 3 | 5 | 350 |
| Week 4 | 5 | 500 |
Notice how despite our hard work, the number of photos each new user sends has not risen – we haven’t made any real progress per cohort. However, the vanity metric on the right – total photos – has risen, due to the cumulative actions of all users.
In contrast, here’s what progress would look like:
| Number of photos sent per user | Vanity metric – total photos sent | |
| Week 1 | 5 | 100 |
| Week 2 | 6 | 250 |
| Week 3 | 8 | 400 |
| Week 4 | 10 | 600 |
The key metric of photos sent per user is rising across cohorts – hard work is paying off and actually improving the metric! (Improvement in this metric also improves the vanity metric of total photos sent, since users are sending more photos.)
Choosing Good Metrics
You need to define metrics that really matter to your business, and measure improvements to that metric.
The metrics that really matter vary from business to business. Often, they reflect your Value Hypothesis and your Growth Hypothesis.
For example, an app might aim for these metrics:
———End of Preview———
Like what you just read? Read the rest of the world's best summary of "The Lean Startup" at Shortform . Learn the book's critical concepts in 20 minutes or less .
Here's what you'll find in our full The Lean Startup summary :
- How to create a winning Minimum Viable Product
- How to understand how your startup will grow
- The critical metrics you need to track to make sure your startup is thriving
