The Measure of Success (which tends to be customer-centric) is a common way to measure the success of a product.
“Our Customer Retention score is 90%.”
“Our Net Promoter Score is 80%.”
“Our Churn Rate has never crossed 10%.”
“Our conversation rate from trial to paying is 50%.”
“Our Monthly Active Users are growing at a rate of 210% month-on-month.”
“Our Monthly Recurring Revenue is growing at a rate of 230% month-on-month.”
“Our Customer Lifetime Value is $300 and growing.”
Such metrics are very important and should be well crafted. They allow a business to focus on the right behaviour expected from a product.
The Flip Side of Measure of “Success”
Consider this scenario. Your KPI is the conversion of trial users to paying customers. And you have chosen a conversion rate of 50% as a Measure of Success.
Say you see a 49% conversion rate. What does that mean? Has the product failed? Not really. Just a bit behind but still a success. You continue on the same path along with continuous improvement.
Say instead, you had seen a 45% conversion rate. What would that mean? Had the product failed? Or, just a bit behind but still a success? Most probably yes and you would have continued on the same path along with continuous improvement.
Maybe the same at a 40% conversion rate.
At what point will the alarm bells ring? At what point will you decide you got the basics wrong and need a pivot?
Say instead, you had seen a 30% conversion rate. What would that mean? Had the product failed now? Still a success? Most probably, at this threshold, you would decide to go back to basics and maybe pivot.
Say then it falls down to a 20% conversion rate. More user studies, more analysis, more advice seeking, more market research. Not calling it a success but not giving up. Changes and updates would occupy your mind.
Maybe the same at a 10% conversion rate.
At what point will the alarm bells ring? At what point will you decide you have it all wrong and need to consider closing the product as sunk cost or do a hard pivot?
Maybe when the conversion rate is less than 5%? For straight 3 months? Even when you have made drastic changes based on real data and feedback? Most probably, at this threshold, you would decide to wind up.
Startups get this. Enterprises don’t. And that is a whole another topic.
A Case for Measure of “Failure”
In my Product Management Canvas, there is a clear place for calling out Failure Metrics. In the example below, there are three clear thresholds:
| Threshold | Duration | A Product Manager’s Analysis |
| Conversion Rate > 50% | – | “Success! Keep learning and keep improving” |
| Conversion Rate <= 50% | – | “O-Oh! Go back to basics. Something is wrong. We are not successful but we are not giving up either. We will get there.” |
| Conversion Rate <= 5% | Over 3 month | “Am I throwing good money on a bad idea?” “Am I stuck in sunk-cost fallacy?” “Time to call it quits? Looks like that.” “What should I pivot to?” |
Remember: While you struggle to find the right product-market fit, you have not failed.
Not achieving Success Metrics does not imply failure.
It is obvious that not achieving Success Metrics does not imply failure. It just means you are in the Zone of Struggle.
Then how does one know when to stop and move on? For that, we need a very clear Measure of Failure.
Measure of Failure
In the above example, it is obvious that Measure of Success and Measure of Failure are distinct measurements and face values. Defining Metrics of Failure is an important activity that we may find hard to dwell on. It is human nature to avoid dwelling too much on the worst case. This is where a disciplined Product Manager can make a difference.
Some characteristics of Metrics of Failure:
- Like Metrics of Success, Metrics of Failure is a band of values rather than a single value
- Unlike Metrics of Success, Metrics of Failure has a definite time component. In the above example, it is “Conversion Rate <= 5%, over 3 month“.
- It can be the same as Metrics of Failure but a band on the other side of the spectrum
- There is a sufficient gap between Metrics of Success and Metrics of Failure for the Zone of Struggle
- Metrics of Failure can possibly be a totally different metric than the products Metrics of Success
The Measure of Success, the Zone of Struggle and The Measure of Failure
Defining Measure of Success and Measure of Failure is an important part of your Product Ideation process.
| Metrics | Measure Band | Duration | Action |
| Measure of Success | Conversion Rate > 50% | – | Keep going. Keep learning. Keep improving. |
| Zone of Struggle | Conversion Rate <= 50% | – | Renewed Product Analysis, User Research, Market Research, Ecosystem Research and Competition Research to find the root cause of low conversation rates |
| Measure of Failure | Conversion Rate <= 5% | Over 3 month | Record Learning and wind up or pivot. |
Share Your Experience
What’s your experience with this? Please share, especially if you are in an enterprise.
