Most of us are familiar with the idea of a customer health score, especially in the subscription economy. But what’s the payback from that score? And how do you determine it?
The predictive lift of a health score—i.e., its predictive performance measured against random choice—is the key to understanding payback. If the predictive lift is high and actionable, the score has good economic value—because you’ll use it to apply resources to priorities that impact revenue retention and growth.
But if the predictive lift of a health score is low, its economic value is low—because acting on the score is only slightly better than random or “gut-based” customer interaction.
So how do you measure economic value of your customer health scoring? In predictive analytics, we have the idea of “information value,” which quantifies how strongly a particular factor such as a health score contributes to prediction.
We derive information value with computation involving statistical correlation to an outcome. An information value is normalized to be anywhere from zero (i.e., same predictive value as random choice) to infinity (i.e., perfect prediction). In practice, a health score that has an information value somewhere above 0.5 will provide good predictive lift and economic value.
Let’s walk through some examples. When looking across its whole customer base, a given company might have 15% churn. So from a probability standpoint, that means if you were to randomly select a customer to approach as a retention risk, there’s a 15% chance that you’d actually be engaging an at-risk customer.
Predictive analytics can transform that 15% probability into 50% probability or better—and that’s the goal of a health score, to increase the probability of engaging the right customer w
ith the right interaction.
But what if you don’t know the information value of your health score? Suppose it has an information value of just 0.05? Using that health score, the probability that you’re engaging an at-risk c
ustomer rises to 21%—just a 6% lift above not using the health score at all.
The chart below shows the results of segmenting customers by the health score, assuming you have 1,000 customers total. Each block in the chart represents ten customers with green blocks representing customers that will renew and red representing customers that will cancel.
A perfect prediction would identify 150 customers—i.e., the 15% out of 1,000 that are actually at risk. But with an information value of 0.05, 430 are identified as at risk, and only 90 of those are actually at risk.
Consequently, 80% of customers flagged as unhealthy would actually be healthy—so you’d be spending time and resources on many customers that aren’t at risk. In addition, sixty of the actual at-risk customers are predicted to be healthy, so you’ll have unanticipated churn.
This is how we evaluate the economic value of a health score: Does it accurately focus resources where needed?
So what happens as the information value increases? At an information value of 0.4, the probability that a predicted churn would actually churn goes up to 36%. With an information value of 0.8, the probability gets to be 50%. At 0.8, the total number of identified customers is reduced from 430 down to 250, with only 23 at-risk customers going unidentified. A big improvement!
All this means that if your health score has a high information value, you can reduce the amount of unnecessary intervention while increasing the coverage of at-risk customers. That is, you can reduce retention costs while increasing retention rates—and that’s the economic value you should expect from any customer health score.
The Implication
Don’t build your customer health scoring based on intuition. Start relying on data. When designing and developing a customer health score, be sure to evaluate the information value of the factors you include, and be sure to back-test your final scoring mechanism. Just as importantly, start to think about different scoring mechanisms for different cohorts of customers. There is no one-size-fits-all health scoring system. Ideally, start to think about individualized health scores to match the expectations of individual customers.
About Scout by ServiceSource
Contact
