You need to select the right metrics to measure the success of the product or feature you're building.Once selected, metrics help the product development in many ways:<ul><li>They provide direction for the product development</li><li>They offer data to make better decisions</li><li>They help measure the impact of new features</li><li>They provide alignment across teams</li><li>They help identify problems</li></ul>

Benefits of product metrics

Answer these questions to find the right metrics for your product:<ul><li>Primary output metric — When will you call your product a success or failure?</li><li>Input metrics — What milestones will your product hit before it becomes a success?</li><li>Check metrics — Are you screwing up other company metrics while building your product?</li><li>External dependent metrics — What external metrics outside of your control are required for your product to be a success?</li></ul>Tap the image to see example metrics for improving revenue/ride at Uber.There are two more examples in the article if you're curious (Netflix and Amazon Prime).

The right set of metrics

<ul><li>Define metrics in such a way that they are measurable and can be tracked on a regular basis</li><li>Define the increment in the metric that your product will bring: "Increase average session length from 5min to 8min"</li><li>Define the time period in which that metric will be achieved: "Increase survey completion rate from 2% to 5% within 3 months"</li></ul>

Best practices for defining metrics

<figure><img alt="" src="https://miro.medium.com/max/500/0*hqxX-ETh2H7" width="450" height="450"><figcaption dir="auto">The maze of metrics</figcaption></figure><h1 data-selectable-paragraph="">Problems that a Product Manager Faces in defining metrics</h1><UL><LI>Unable to get your products prioritized because the company’s priority is not the metric you are moving to.</LI><LI>You have chosen the wrong metric for your product and figure this out later after launching the product.</LI><LI>I have not been able to conclusively achieve the success of the products you have launched.</LI><LI>I have been dependent on other teams to achieve your success metric and have been unable to do so.</LI><LI>Have achieved success metric for your own product but later you figured out that you have screwed some other metric for some other team in the process of doing so.</LI></UL><h1 data-selectable-paragraph="">The success of a Product depends on choosing the right metric</h1>The difference between a successful product and an unsuccessful product is the selection of the right metric to measure the success of the product. Without a measurable success metric your product is directionless and hence there is no way to figure out when to take what actions. It’s like a race when one doesn’t know where the finish line is.<h1 data-selectable-paragraph="">What is the product metric?</h1>These are well defined quantifiable and directly measurable entities that determine the progress made by a product and provide input to further improve the product to achieve better results.<h1 data-selectable-paragraph="">Why are they important?</h1>Because :<UL><LI>Metrics define a product decision</LI><LI>Helps in effective communication and collaboration with stakeholders about what is important</LI><LI>Helps to identify problems and take a corrective course</LI><LI>They help to provide a feedback loop for the product launched.</LI><LI>Ensures that product success is identified and recognized throughout the organization</LI></UL><h1 data-selectable-paragraph="">What are some product decisions dependent on metrics?</h1><UL><LI>Estimation of the potential impact of the product</LI><LI>Prioritizing amongst conflicting initiatives by normalizing products to a common metric</LI><LI>Scoping user stories within an initiative based on the highest impact first</LI><LI>Defining the features of the product based on the metrics to move</LI></UL><h1 data-selectable-paragraph="">What are the different types of products?</h1>There are two different types of products on which metrics can be defined and they are as follows :<UL><LI>New Product Inception Metrics</LI><LI>Optimizations/Additions on top of existing product metrics</LI></UL>Piece of Advice: If you don’t have the right measurable metric, you should not build the product, and conversely if you chose to build a product ensure you have the metrics in place to measure success.<h1 data-selectable-paragraph="">Ask the right questions to get your metric?</h1><h1 data-selectable-paragraph="">Primary Output Metric</h1>Question — When will you call your product a success or a failure?Why is this metric important?This is the most important measurable outcome of your product which establishes the true impact of your product. The reason why this is important :<UL><LI>This is the most important key result for the product and significantly helps to align stakeholders to start product development.</LI><LI>Over a period of time product managers should be able to judge the success and failure of the product based on this metric and also figure out if additional product investments are required</LI><LI>Before building the product PM’s should estimate the change in the success metric so that they can set their targets accordingly</LI><LI>This metric should be defined in such a way that it is measurable and can be tracked on a regular basis. (if possible should be traceable without an experiment)</LI></UL>Eg: Monthly ad revenue (See Case Study Below for Details)<h1 data-selectable-paragraph="">Input metrics</h1>Question — What will be the important milestones your product will reach before it becomes an overall success?Why is this metric important?These metrics help to segregate the monolithic success metric into various independent secondary success metrics which are functions of different components of the product. The reason for the importance of this metric is as follows :<UL><LI>Assign the success metric for each of the stakeholders to ensure proper accountability</LI><LI>Use these metrics as milestones to ensure that the primary output metric is right on track.</LI><LI>Break the monolithic output metric equation as a function of dependent input variables to achieve the output metric</LI></UL>Eg: Click Through Rate * Cost Per Click, Fill Rate of Ads (See Case Study Below for Details)<h1 data-selectable-paragraph="">Check metric</h1>Question — Are you screwing up some other metric of the company while building your product?Why is this metric important?These metrics prevent the moving round and round in a circle paradox where the net value the product delivers to the company is zero. The reason why this is important :<UL><LI>It helps product managers to think more holistically about the general good of the company rather than only their own stream.</LI><LI>Also since most metrics are interconnected and improvement in one metric may lead to the degradation of the other and hence this fact should be known while making product decisions.</LI></UL>Eg: Orders/discovery session on Swiggy (See Case Study Below for Details)<h1 data-selectable-paragraph="">External Dependent Metric</h1>Question — What is the most important external metric which the product does not control and which is required for your product to attain the desired primary output metric?Why is this metric important?The success of the product is also dependent on the right inputs and if these right inputs are not achieved the desired outcomes are not reached. This metric can be of two different types :<UL><LI>Type 1: The input metric is not dependent on your product and is a function of the market forces which is beyond your control.</LI><LI>Type 2: The input metric is a dependent metric and is either the outcome metric of another product.</LI></UL>Eg: Total monthly impressions on ads (See Case Study Below for Details)<h1 data-selectable-paragraph="">Best Practices of defining metrics</h1>Few Practices to define metrics :<UL><LI>Make your primary output much independent from other teams as much as possible. (For Eg: at Zomato if a product manager is responsible for bringing discounts from merchants than his primary output metric should be y% of GMV funded as discounts by merchants and not x million $ of discounts)</LI><LI>Always define the increment in the metric that your product will bring. (For example: Increase metric y from x1 to x2 and not Increase metric y to x2)</LI><LI>Always define a time period as to when that metric will be achieved. (For example: Increase metric y from z1 to z2 within x months)</LI><LI>The input metrics can also be attributed to different teams/stakeholders so that there is clear accountability on the product.</LI></UL><h1 data-selectable-paragraph="">Iterative Product Development</h1><figure><img alt="" src="https://miro.medium.com/max/500/0*9kl5b-Ct7o8" width="692" height="306"><figcaption dir="auto">Flow Diagram of Iterative Product Development</figcaption></figure>This is a simple approach to product development based on metrics. The key points here are as follows :<UL><LI>Check for external dependent metrics and ensure that they are right or normalize input/output metrics so that the output metrics become independent.</LI><LI>Iterate the product to improve the input metrics.</LI><LI>Once input metrics are right after a lag period check the output metrics and they should 99% be right. (If it does not happen you have defined the wrong input metrics)</LI><LI>Once primary metrics are achieved, then see the check metric to confirm that your product is not screwing up the other metrics.</LI><LI>If you achieve your output metric within the time period then your product is a success.</LI></UL><h1 data-selectable-paragraph="">Case Study: Let’s apply the framework in three different problem statements</h1>Case 1: Amazon Prime Subscription<figure><img alt="" src="https://miro.medium.com/max/500/1*6UtLCz9FdNf3yt7d0mvJy" width="692" height="701"><figcaption dir="auto">Amazon Prime Subscription Metrics</figcaption></figure>Case 2: Netflix Recommendations<figure><img alt="" src="https://miro.medium.com/max/500/1*0ZUhDP7OyVuU4oe1UMD_A" width="692" height="635"><figcaption dir="auto">Netflix Recommendation Metrics</figcaption></figure>Case 3: Uber User-Based Pricing<figure><img alt="" src="https://miro.medium.com/max/500/1*Yiy9UiJrJZwjMj3pnHraD" width="692" height="421"><figcaption dir="auto">Uber User-Based Pricing Metrics</figcaption></figure><h1 data-selectable-paragraph="">Change in metrics defines the roadmap</h1>The way product features are added to a product is changed by the metrics in which that particular product is chasing and these metrics for a product change with time leading to different features on the product roadmap to optimize for the newer metrics.<figure><img alt="" src="https://miro.medium.com/max/500/1*uaY88K8kfernLxBrFx4pd" width="692" height="254"></figure>Let me know your thoughts and improvements in this framework in the comments and do use this framework in your products.

The difference between a successful product and an unsuccessful product is the selection of the right metric to measure the success of the product. Without a measurable success metric your product is…

Product Metrics

Problems that a Product Manager Faces in defining metrics

<ul><li>Unable to get your products prioritized because the company’s priority is not the metric you are moving to. </li><li>You have chosen the wrong metric for your product and figure this out later after launching the product. </li><li>I have not been able to conclusively achieve the success of the products you have launched. </li><li>I have been dependent on other teams to achieve your success metric and have been unable to do so. </li><li>Have achieved success metric for your own product but later you figured out that you have screwed some other metric for some other team in the process of doing so. </li> </ul>

The success of a Product depends on choosing the right metric

The difference between a successful product and an unsuccessful product is the selection of the right metric to measure the success of the product . Without a measurable success metric your product is directionless and hence there is no way to figure out when to take what actions. It’s like a race when one doesn’t know where the finish line is.

These are well defined quantifiable and directly measurable entities that determine the progress made by a product and provide input to further improve the product to achieve better results.

<ul><li>Metrics define a product decision </li><li>Helps in effective communication and collaboration with stakeholders about what is important </li><li>Helps to identify problems and take a corrective course </li><li>They help to provide a feedback loop for the product launched. </li><li>Ensures that product success is identified and recognized throughout the organization </li> </ul>

What are some product decisions dependent on metrics?

<ul><li>Estimation of the potential impact of the product </li><li>Prioritizing amongst conflicting initiatives by normalizing products to a common metric </li><li>Scoping user stories within an initiative based on the highest impact first </li><li>Defining the features of the product based on the metrics to move </li> </ul>

What are the different types of products?

There are two different types of products on which metrics can be defined and they are as follows :

<ul><li>New Product Inception Metrics </li><li>Optimizations/Additions on top of existing product metrics </li> </ul>

Piece of Advice : If you don’t have the right measurable metric, you should not build the product, and conversely if you chose to build a product ensure you have the metrics in place to measure success.

Ask the right questions to get your metric?

Question — When will you call your product a success or a failure?

Why is this metric important?

This is the most important measurable outcome of your product which establishes the true impact of your product. The reason why this is important :

<ul><li>This is the most important key result for the product and significantly helps to align stakeholders to start product development. </li><li>Over a period of time product managers should be able to judge the success and failure of the product based on this metric and also figure out if additional product investments are required </li><li>Before building the product PM’s should estimate the change in the success metric so that they can set their targets accordingly </li><li>This metric should be defined in such a way that it is measurable and can be tracked on a regular basis. (if possible should be traceable without an experiment) </li> </ul>

Eg: Monthly ad revenue (See Case Study Below for Details)

Question — What will be the important milestones your product will reach before it becomes an overall success?

These metrics help to segregate the monolithic success metric into various independent secondary success metrics which are functions of different components of the product. The reason for the importance of this metric is as follows :

<ul><li>Assign the success metric for each of the stakeholders to ensure proper accountability </li><li>Use these metrics as milestones to ensure that the primary output metric is right on track. </li><li>Break the monolithic output metric equation as a function of dependent input variables to achieve the output metric </li> </ul>

Eg: Click Through Rate * Cost Per Click, Fill Rate of Ads (See Case Study Below for Details)

Question — Are you screwing up some other metric of the company while building your product?

These metrics prevent the moving round and round in a circle paradox where the net value the product delivers to the company is zero. The reason why this is important :

<ul><li>It helps product managers to think more holistically about the general good of the company rather than only their own stream. </li><li>Also since most metrics are interconnected and improvement in one metric may lead to the degradation of the other and hence this fact should be known while making product decisions. </li> </ul>

Eg: Orders/discovery session on Swiggy (See Case Study Below for Details)

Question — What is the most important external metric which the product does not control and which is required for your product to attain the desired primary output metric?

The success of the product is also dependent on the right inputs and if these right inputs are not achieved the desired outcomes are not reached. This metric can be of two different types :

<ul><li>Type 1: The input metric is not dependent on your product and is a function of the market forces which is beyond your control. </li><li>Type 2: The input metric is a dependent metric and is either the outcome metric of another product. </li> </ul>

Eg: Total monthly impressions on ads (See Case Study Below for Details)

Few Practices to define metrics :

<ul><li>Make your primary output much independent from other teams as much as possible. (For Eg: at Zomato if a product manager is responsible for bringing discounts from merchants than his primary output metric should be y% of GMV funded as discounts by merchants and not x million $ of discounts) </li><li>Always define the increment in the metric that your product will bring. (For example: Increase metric y from x1 to x2 and not Increase metric y to x2) </li><li>Always define a time period as to when that metric will be achieved. (For example: Increase metric y from z1 to z2 within x months) </li><li>The input metrics can also be attributed to different teams/stakeholders so that there is clear accountability on the product. </li> </ul>

This is a simple approach to product development based on metrics. The key points here are as follows :

<ul><li>Check for external dependent metrics and ensure that they are right or normalize input/output metrics so that the output metrics become independent. </li><li>Iterate the product to improve the input metrics. </li><li>Once input metrics are right after a lag period check the output metrics and they should 99% be right. (If it does not happen you have defined the wrong input metrics) </li><li>Once primary metrics are achieved, then see the check metric to confirm that your product is not screwing up the other metrics. </li><li>If you achieve your output metric within the time period then your product is a success. </li> </ul>

Case Study: Let’s apply the framework in three different problem statements

Case 1: Amazon Prime Subscription

Case 2: Netflix Recommendations

Case 3: Uber User-Based Pricing

The way product features are added to a product is changed by the metrics in which that particular product is chasing and these metrics for a product change with time leading to different features on the product roadmap to optimize for the newer metrics.

Let me know your thoughts and improvements in this framework in the comments and do use this framework in your products.

Bite-sized knowledge
to upgrade
your career

Input-Output Metrics for Product Managers

Benefits of product metrics

The right set of metrics

Best practices for defining metrics

IDEAS ABOUT

MORE LIKE THIS

Retention Metrics That Matter

How product teams can measure progress through outcome OKRs

Metrics Versus Experience

It's time to

READ

LIKE

A PRO!

Bite-sized knowledgeto upgradeyour career

Input-Output Metrics for Product Managers

Benefits of product metrics

The right set of metrics

Best practices for defining metrics

IDEAS ABOUT

MORE LIKE THIS

Retention Metrics That Matter

How product teams can measure progress through outcome OKRs

Metrics Versus Experience

It's time to

READ

LIKE

A PRO!

Bite-sized knowledge
to upgrade
your career