Product Metrics

This post takes a deep dive into some of the critical metrics that are applicable to specific product types. This blog is compiled from different sources and the list of metrics is quite exhaustive. However, in reality, PMs create an optimized list of metrics that most support the growth strategy of the product. The idea is to start small and then gradually build on top of that.

Types of consumer business apps

Consumer apps are designed to meet specific target customers’ unique needs and wants. The goal is to strengthen the relationship with customers which leads to upsell and cross-sell.

The 5 major types of such businesses are:

Subscription — trial-based (e.g. Netflix)

Subscription — freemium (e.g. Duolingo)

Ad-based (e.g. Twitter)

Marketplaces (e.g. Amazon)

DTC (e.g. Country Delight)

Subscription — trial-based

Key success factors: low acquisition costs, a high conversion rate (post the trial period), and high retention.

Trials growth: New trials per period,

Conversion: % of trials that convert to customers after the trial period ends,

Customer retention: Month 1/3/6 cohort retention ( retention curve in Metrics for a product manager)

Revenue: Monthly recurring revenue (MRR)

CAC: Cost of acquiring a trial. MRR and conversion become very critical metrics to track when the acquisition is through paid channels.

Payback period: This is also known as the growth spend efficiency which is the time taken to recover the CAC.

The general benchmark for startups to recover CAC is 12 months or less. High-performing SaaS companies have an average payback period of 5–7 months. Great B2C businesses have a payback period of less than 1 month. 6 months is considered good, and 12 months is average. And in exceptional cases, payback of acquisition costs happens on the very first transaction. B2B businesses that have small and medium businesses as their consumers 6 months of payback can be considered as great, 12 months as good, and 18 months as average. For enterprise customers less than 12 months of payback period is great, 18 months is good, and 24 months is average. Larger enterprises with greater access to capital can afford a longer payback period.

Subscription — freemium

Key success factors: Ability to build large pools of free users through word of mouth or virality (growth generated from existing users), sustained conversion rate from free to paid, and high retention.

User growth: New signups

User retention: Week 1/2/12 cohort retention

Customer growth: New paying customers

Customer retention: Month 1/3/6 cohort retention

Conversion: % of free users that end up converting to paying customers after a finite time period (1–3 months is typically good)

Engagement: Key action each day/week/month

Revenue: Monthly recurring revenue

Payback period: Explained above

Ad-based

Key success factors: Sustained and inexpensive virality-driven user growth, high engagement, and high retention.

User growth: New signups

Engagement: DAU, WAU, or MAU

Intensity: DAU/MAU, WAU/MAU, or L7/L30. While DAU divided by MAU is a common metric for measuring engagement, it has its shortcomings because it gives a single number whereas L30 (coined by the Facebook growth team) or the activity histogram gives a detailed view of a product’s engagement. It provides a richer understanding of user engagement. This is also called the “Power User Curve”. It is a histogram of users’ engagement (whatever action is important to measure the top-level activity) over the total number of days they were active in a month.

The Power User Curve smiles when things are good

The Power User Curve has the following advantages over the DAU by MAU value:

1. It shows if the most engaged segment is coming back every day

2. It shows the variability among different users. Some are less engaged, whereas others are highly engaged

3. Power User Curves can be shown for different user actions and not just for a specific action. This matters if the core activity that matters for the product is deeper in the funnel.

4. When mapped to cohorts, Power User Curves shows if user engagement is getting better over time — which in turn helps assess product launches and performance of other feature changes

Retention: Week 2/4/8 cohort retention

Activation: % of new users who “activate,” whatever that means for your product

CAC: Cost of acquiring a new user

Marketplaces

Key success factors: Efficient acquisition costs, high-enough buyer conversion, high average order value which is the average dollars spent per transaction, and high purchase frequency.

Match rate or Fill Rate: The job of any marketplace is to facilitate the matching of supply with demand. It is the rate at which buyers or consumers can find sellers or producers and vice versa. It varies from industry to industry. e.g. for a ridesharing company, the driver utilization time or what % of the time are drivers driving around with a passenger, vs. empty? A related metric is to measure unsuccessful matches. For ridesharing, what percentage of users open the app but drop off without requesting a ride? It could be due to long wait times, surge pricing, etc. when the marketplace was unable to meet the demand. Marketplace PMs should investigate and identify reasons why matches don’t happen and eliminate or reduce these blockers. If the core issues are not addressed quickly, multi-tenanting (which may be there already) will increase and users will naturally be incentivized and flock to competitor products.

Bookings growth: Number of completed transactions per week/month. Bookings tell directly whether marketplace usage is growing or shrinking. e.g. for Uber and Ola, it is the number of rides. For Airbnb, it is the number of nights booked.

Supply growth or Market depth: New active supply per week/month. It is important to measure and optimize market depth because it directly impacts the user experience. One of the primary goals of any marketplace is to reduce search costs by making it easy for participants to find and match with the other side. However, too much supply can cause challenges in discovery which may result in negative network effects. Consumers experience decision fatigue or a paradox of choice. Conversion rates could fall in such cases. Generally, for heterogeneous (each supplier is different) supply marketplaces like Airbnb, where a user’s tastes may be quite specific, every additional listing on the platform is useful to see. Whereas, for homogeneous (all suppliers are similar) supply marketplaces like Yulu, market depth impacts ease of use, e.g., if there were 6 Yulu scooters in my vicinity, this is no more valuable than if there were only 4 scooters available for me to use — user value plateaus despite the addition of more supply. So it is recommended to track “activated supply” (supply that has reached some value milestone) and not just raw new supply.

GMV growth: Dollars going through the system per week/month. GMV or Gross merchandise volume is the total sales dollar volume of merchandise in a specific period. It’s the real top line, what the consumer side of the marketplace is spending. It is a measure of the size of the marketplace.

Revenue: This is different from GMV, even though, GMV and revenue are frequently used interchangeably. Revenue is a fraction of GMV that the marketplace “takes.” It consists of various fees that the marketplace earns from the participants for providing its services; such as transaction fees based on GMV successfully transacted, advertisement revenues, sponsorships, subscription fees, etc.

Buyer retention: Month 1/3/6 purchaser retention

Supply retention: Month 1/3/6 supply cohort retention

Conversion: % of visitors who end up purchasing

Time to fill: How long it takes to fill a customer request (e.g. Uber request)

Average order value (AOV): Average dollars spent per transaction

DTC

Key success factors: Efficient acquisition costs, sustainable margins, and high AOV.

Customer growth: Number of new purchases/subscribers

Customer retention: Month 3/6/12 cohort retention

Revenue: Monthly (recurring) revenue

Growth spend efficiency: Payback period, and/or contribution margin, and/or ROAS

Gross margins: Net sales revenue minus the cost of goods sold (per purchase or from an annual subscription)

CAC: Cost to acquire a customer

AOV: Average order value

Key considerations while selecting the right set of metrics

1. How do you know you are acquiring new users sustainably?

2. How do you know that your users are experiencing the value that they expected?

3. How do you know your users are ready to pay?

4. How do you become certain that your users still want to pay for using the product?

5. How do you know that the product is making a profit for the business?