Why Product Analytics Matter and How to Get Started

Part of defining your product is deciding how you are going to deliver value to your users. But as nice as that intention sounds, it has no value for you as the product owner in itself. You’ll need to find a way to make sure that you’re doing what you set out to do at the start of your product journey.

This is where the complex world of product analytics comes into play.

Your first thought might be: ‘But there are so many features in my product, how will I know which engagements I should measure?‘

Here’s the scary answer first: you’ll have to measure ALL your events and engagements. The secret is placing them within a framework that helps you make sense of them. That’s because not all user actions have the same value, not even for the user, and definitely not for the business you’re running.

Pirate metrics for product analytics

It’s easier to keep track of all your product metrics if you have a framework to put them into context. One framework is McClure’s pirate metrics for startup growth:

Acquisition
Activation
Retention
Referral
Revenue

If you look at the initials of this list, you’ll get a very nice AARRR, and if you added the pirate accent as you read it, then you already know where the name comes from.

Before going into the specific list of metrics for each step of this framework, you should know that it helps if you overlay it with the long-term customer experience of an ideal user who decides to stick with you. Just as you think of a user persona when it comes to UX decisions and changes, think of a user persona when the time comes to figure out how their behaviour might reflect in the numbers you’re tracking.

Common metrics to track

User acquisition

Funnel conversion points:

Impressions
Clicks
Free trials
Conversions to subscriptions
Customer acquisition cost

Note that a funnel might look different depending on the channel you’re working with. For example, if you’re running a Facebook ads campaign for a subscription-based product, it might look something like this:

If you’re running a search-based campaign in the App Store, the funnel is much shorter:

User activation

There are two ways to measure user activation, and both are important. At their core, a user is considered activated if you can answer ‘yes’ consistently to these two questions:

Is the user opening and using the product consistently over a set time (e.g. a month)?
Has the user gone through the features of the product to understand its value? (Did they have an ‘aha moment’?)

Depending on the type of product you’re building, activation metrics and aha moments will likely look very different. When it comes to metrics, you should be measuring:

Daily active users (DAU)
Weekly active users (WAU)
Monthly active users (MAU)

But remember these numbers are meaningless if they don’t reflect the real-life behaviour of your users as they’re incorporating their product in their day-to-day life. Here are just some examples to make you think:

For a workout app, an ideal user would be active daily or at least once a week, because that reflects a healthy habit of working out that’s being tracked in the app
For a grocery shopping app, an ideal user would be active weekly, twice a month or once a month, depending on how good a meal planner they are. So even a user who makes one order a month could be considered an active user.
A finance app for a freelancer or a small business could be used weekly or monthly, depending on the workload of the business.

It makes a lot of sense to have some user interviews with your audience, segmented depending on your personas, to understand what you should expect as average interaction with your product. Not only will this help you set a baseline, but it will also help you set the language for notifications and other avenues of engagement to keep your users active without annoying them.

The second dimension of activation is the ‘aha moment’. In other words, is the user getting a moment of understanding early on the value that the product can offer them? This is easily tracked by making sure they walk through what you consider the core feature flow of your product. If you see drop-off rates here, inquiries, or users who just stop using your product, you’ll need to do further research to understand what is wrong.

Retention

Activation is only the first step in a user’s journey with your product. For them to get the most of your product and for your business to recoup the investment you made, you’ll need to do your best to have a high rate of retention: that is, keep users active for as long as possible.

Here are some of the metrics you can track for this:

Customer retention
Churn rate
Cancelled orders
Adoption of new feature sets

From a business perspective, retention has the highest impact on revenue, especially if your product has a subscription-based monetisation model. If a user stops paying for their subscription, you stop receiving income from them. On the other hand, a 5% increase in customer retention produces more than a 25% increase in profit.

Referral

One way great products stand out amongst all the other products that exist is in the way that their users talk about them. Of course, there are two layers to this. The first one – and rather obvious – is that your product should deliver the kind of value that kindles the enthusiasm of users to want to recommend your product to their friends, acquaintances and coworkers.

The second layer is to make these actions easy. In UX speak, we call these opportunities for referrals or virality loops. These can take the form of:

Social shares
Social invites
Personal promo codes
Credit metrics

Revenue

One last category of metrics you should pay attention to is that which are relevant to the revenue streams and cost structures of your business. These are directly related to your bottom line, as they will predict if you will make a profit or not at the end of a year.

Cost to acquire a new customer
The yearly average revenue per customer
The lifetime value (LTV) of your average customer
Price satisfaction

If you’ve got to this point, you’re probably still thinking these are a lot of metrics to keep track of, while none seem to stand out as a compass to show you the direction towards making your product be successful. But don’t worry, that’s what the next section is for.

The North Star metric

The North Star metric is the guiding output metric that can help drive business and product success. Its goal is to provide clear evidence that your product is solving the problem it was built for and providing value for your target market.

Several characteristics make a North Star metric stand out and we’ll be listing them all here.

It measures the moment a user finds value from your product

That means that like every metric, it is measurable, timebound, not influenced by external factors, and may be thought of as the outcome of the core UX flow of your product. Because of this, it is a non-revenue indicator (revenue is an indicator of value for your business, but not for the user – they’re paying to get something of value). Nevertheless, the North Star metric is predictive of your business model. If your product cannot deliver value consistently to the user and you cannot reliably measure this, it follows that you won’t be able to generate revenue through a product like this.

It stands in for the core of your product strategy

If your product has a vision statement, the north star metric is how you count for many users you’ve achieved said vision. For example:

If you’re building Quora, with a vision of becoming a source of knowledge on the internet, your North Star metric is the number of questions answered
If you’re Spotify, with a vision to entertain people with the best music, your North Star metric is time spent listening to music
If you’re Uber, Lime, or any ride-sharing app with a vision to help people get from one place to another, your North Star metric is the number of rides taken in a week in within a reasonable amount of minutes (e.g. between 5 to 10 minutes). In most two-sided market places, the North Star metric is a result of both demand and supply, not just the demand of users. Tracking the supply side ensures the success of the whole marketplace.

It focuses on one way in which a product can compete for your user’s time

If you test and use enough products, you realise soon that they fall into three categories when it comes to the mindshare and effort used by a person while engaging with a specific product. Amplitude defines these as three games:

The attention game: how much of your users’ time can your product take up
The transaction game: how many commercial transactions does your user make through your product
The productivity game: how many high-value digital tasks can a user perform in the ecosystem your product creates

One simple rule is that a product can only play one of these games as it exists. If you try to play more than one, you dilute both your strategy and your resources and set yourself up for failure.

It reflects the output of the whole AARRR framework

A good North Star metric is a leading metric when it comes to the future business outcomes your team cares about. Moreso, it is supported by all the metrics (and therefore the actions) that a user triggers as they get value from the app, and use and turn using the app into a habit.

Does this mean the North Star metric is a singular metric that you need to focus on? Well, it’s easier to think of it as an output metric. You can’t focus on it specifically, because it’s too broad, too big and not actionable enough for your team to act on it.

But it is a scoreboard that tells you how you are doing when it comes to the product vision you’ve set yourself. To change the score, you need to look for the specific input metrics, that will move the needle of your North Star metric as well.

Here’s a case study for Spotify as an example (remember, their North Star metric is the time spent listening to music):

Based on Brian Balfour

The tech stack for product analytics

Up till now, we’ve talked about the why and what of product metrics. Next, we’ll dig in the how. Before explaining the tech stack we prefer to recommend here at Tapptitude, we’ll first explain the mental model you need to have in mind when choosing these tools.

The data scheme

When it comes to setting up product analytics, every team needs a data warehouse that can perform data analytics and gather up the data from the various channels you work with, a marketing attribution tool, and then the specific tools to work with the owned and social channels where you’re running distribution. Finally, to make sense of all this, you’ll want to plug into the data warehouse a data visualisation tool that will help you make sense of the analytics and give you a big picture of where you are and where you’re heading.

The tools we use

1. Segment – as a data warehouse

Segment is a customer data platform that helps you collect, clean, and control your customer data. An alternative to it is Singular.

Because Segment centres all data around the user, it plays the role of the data warehouse, telling you what the user does inside the product, as well as on the journey to towards acquiring your product.

Segment is used to collect:

events and properties that come from the product interaction
User traits – used to associate specific properties to a user (i.e. product preferences, device properties, personal data)
Automatically tracked events – App Installed, or partner-specific events (i.e. Deep Links opened, through other Partners integration)

Segment is used to unify multiple integrations and simplifies the code base needed to make them work, having the ability to work with many partners: Facebook Analytics, Deep Linking tools, Firebase, Mixpanel. The full list of integrations is available here: https://segment.com/catalog/.

A product owner can add a Segment destination with only a few lines of code, but otherwise it should work out-of-the box and this saves significant development time.

2. Branch.io – as the marketing attribution tool

Branch provides the leading mobile linking platform, with solutions that unify user experience and measurement across different devices, platforms, and channels. Its main roles are to provide reliable linking and attribution for those links.

The tool is used to:

track installs, campaigns, attribution.
generate trackable links (deep links)
Detect whether the product already exists or isn’t installed on the user’s device
Attribute and track app installs across multiple channels and aggregate them in a unified way.
Integrate with Major Ad Partners (Google Adwords, Facebook Ads, Apple Search Ads) and Customer Data Platforms
Provide DeepViews of the product – offer a preview of the product content. They are an interface between the product and the user, allowing the user to install the app and land him on the accessed detail.

3. Mixpanel – as the data visualisation analytics

Mixpanel is a self-serve product analytics reporting tool. In our data scheme, it plays the role of the data visualisation analytics tool that helps us make sense of all the data that’s gathered in the product and the acquisition campaigns.

Specifically, it:

Interprets data received from Segment (in our case) and other platforms (Branch)
Aggregates ALL data received from different sources with the help of Segment.
Allows the Product Strategist or Product Owner to have a full picture of the results of the product tactics, current user engagement and behaviour of the user cohorts, as well as track any other experiments.
Displays events tracked in Segment through a visual interface
Merges users with the same identity coming from different sources (web/mobile app/new logins/ new devices)

4. SendGrid as the email automation tool

SendGrid is an email marketing and automation company. Recently acquired by Twilio, they also have plans for text, chat, voice, and video messaging, offering the perfect platform to coordinate and monitor all communications with a user happening outside the app. This may include:

Marketing, promotional and community based messaging
Support messaging
Subscription and legal based messaging

Of course, every product we’ve listed here has an alternative. With the myriad of channels and ways people discover and download apps today, you’re bound to find new products and approaches to include in your data infrastructure. As long as you remember:

That every tool should have a clear objective and role in the whole process and
That variables should be consistent and reliable across platforms

then you’re on the right track to build a data infrastructure that will help you in your product analytics endeavours.

Product analytics is the backend of great product management

As a wrap-up, always remember that you’re not measuring for the sake of measuring. While every founder starts out building a product focused on solving a problem for their users, product metrics is how a product manager worth their salt tracks if they’re actually succeeding with their goal.

Product metrics will help you:

Predict what will resonate with your users.

Specifically, they will tell you which features are contributing to the aha moment, which are keeping them engaged, where they lose interest, as well as help you keep track of your various acquisition funnels. Last, but not least, product metrics play an essential role in predicting which categories of users you’re most likely to retain or lose, helping you further refine your user personas.

Spend your limited resources more effectively.

When you’re focusing on scaling your startup with first rounds of funding, you’ll be very aware that every penny you spend matters. That’s why product metrics are a key part in strategising how to best spend the limited resources you have, whether it comes to your budget, or your team’s time.

Gain support from executives, board or investors.

It’s easier to gain stakeholder support when you can support your position with evidence. Product metrics, especially well-documented, well-argued metrics show both your team’s expertise in product management, as well as your product’s potential to reach the goals you set.

Align your team and help you towards working on a common goal.

Depending on how large your startup is, they may work in different teams and focus on smaller goals that are relevant and closer to their own work, whether it’s a specific feature set for a product team, or acquisition if it’s the product marketing team. Having a North Star metric set and the different teams working on action metrics they can directly influence will change the dynamic of the team, aligning them towards a common goal.

At the end of the day, you have all the reasons to get product analytics right, and none to avoid it. Your product’s long-term success depends on it.