Essential Guide to Understanding Customers & Prospects – Session 6: Scoring and how to improve business targeting performance

Nigel Magson, Managing Director of Talking Numbers considers:

How to acquire new business customers – through a methodology of building propensity scores and applying these back to the Business Universe

In this issue we consider how to profitably prospect, and create scorecards to decide how deep to select into a prospect file or base, such as Marketscan’s Business Universe. This issue builds on some of the concepts introduced in previous e-shot of how to examine and profile your customers, assess market penetration and also lever the value from adding additional data to your file.

So why build a scorecard?

Good questions, so let’s take a typical scenario. You have been marketing to (e.g. mailing, e-mailing or telemarketing) to prospective customers and some of those have converted into customers. You have bought or acquired various data to support this process. You may have done this from simple profiling principles (see our earlier issue on profiling) but you are now seeking to improve the efficiency of this targeting or rollout across a larger universe base.

But which records will be best selected? Can you create a smaller more responsive target set of data, and if so how far down should you contact and therefore purchase?

This is the realm of a scorecard, and building a statistical model which will provide and demonstrate this uplift. Scorecards are often employed effectively where there is a specific or smaller universe that cannot be identified simply through profiling techniques. For example, for one client recently, we were looking to target the UK Fork lift truck market, but in fact a subset of this market. The base universe of this market does not exist commercially and so a modelled solution from consolidating different data sources was required. The answer was a combination of profiling and modelling.

This article will give some examples and insights into the process, some of the techniques that are employed, and some of the things to bear in mind. Please remember the actual process of developing a model is the preserve of an experienced data analyst with good statistical and marketing knowledge. You may have this person in-house or you may work with an independent specialist provider of analytics services.

So how do I build a B2B scorecard? – Case Study example from one of Marketscan’s clients

Context – The client is a vendor of business gifts (pens & diaries) into the UK business market via a telemarketing/mailing operation. The client was seeking to uplift response to past targeting & mailing and decide which data to purchase.

You remember in the first edition on applying data analysis in the B2B environment we explored the area of profiling and market penetration, and the use of indexes and z-scores to describe and compare data populations. In this instance together with the client’s data the following variables were added to enhance the dataset and support further targeting.

  • Phone numbers
  • Fax numbers
  • Named senior execs
  • Actual turnover figures
  • Modelled turnover figures
  • Employee sizes
  • SIC codes
  • Premise types
  • Branches
  • Company type
  • Incorporation Date
  • Time at premise
  • Subsidiary
  • Company Reg no
  • Pre Tax Profit
  • Net Worth
  • Export Sales
  • SOHO Flags

The following variables in addition to the client’s own were found to have high significance and could be introduced into a modelling process. These variables were then used to form score cards based on a multivariate statistical model.

Rank Variable Significance
1 Premise Type High
2 Number of Branches High
3 Company Type High
4 Number of Employees High
5 Number of Employees High
6 Turnover High

The model was developed using a regression technique, after a number of methods had been explored and compared back against the Business Universe to look at both new and historic customers. It is often worth considering whether your customer base is changing by looking at recent vs older customer profiles.

The scorecard below resulted and was applied into the Marketscan business file.

It is easy to recognize the optimisation possibilities using the scorecards. For example if you selected the first two deciles within the scorecard you already find 42.33% of the new customers but only 20% of the companies in the Marketscan business file. The index within the score class reflects the relationship between the customers and the business file. The index of 2.51 for the new customers in decile 1 shows that the amount of new customers in this segment is 2.51 times larger in comparison to amount of companies in the Marketscan business file. The reverse of the penetration statistic is the potential remaining, which can be scored using the common (non client factors) into the business universe.

Scorecard validation

In the case study example, the model was validated in a number of ways:

It was built a number of times using different samples and population sizes to ensure that the model was robust. A group of cases was put aside during modeling as a test dataset. The modeled and test dataset were then compared to examine any differences. The new customers a subset of actives) were examined to understand how good the model was at predicting these companies. The power of the variables in the model were then compared with the information found during the profiling exercise.

These are standard methods of checking the power and robustness of a model to ensure that it is not over trained (i.e. only good at recognizing the dataset used to build it). Scorecards are often validated using a gains chart, which can show the difference or uplift a new model might make over an existing model or indeed no model at all. The example below is illustrative.

If you are feeling brave or inspired we’ve added a little more detail on some of the modelling techniques available to the analyst when tackling model building, with some of the typical applications and potential drawbacks to the technique. There are various statistical methods which can be used for modelling purposes, some of which are listed below: 2

  • Discriminant Analysis
  • Linear and multiple regression
  • Log-linear models
  • CART & CHAID (covered in earlier bulletins)
  • Neural Networks and other AI methods (not covered here)

The most important point which must be kept in mind when modelling data of any kind is to keep it simple! Simple models are easier to build, understand and apply to your data and they tend to be almost as powerful as more complex ones. They are also easier modify when they need updating or changing, and often the underlying ruleset can be programmed to score new records onto the database. There are two main types of model, group assignment and scoring.

Discriminant analysis is a good example of group assignment. Discriminant analysis tries to predict which group a customer should be placed in and then compares this with a known value. The model can then overlaid onto data where the end ruleset might simply be represented as follows.

Y=Ax1+Bx2+Cx3+D

e.g. Contact_score=1.3*Income-0.75*number_of_children+4*age+1.54

Traditional regression models prefer to use continuous data, such as age and income, unfortunately the data that marketers have access to is generally categorical in nature; e.g. sex, Agree/Disagree. There are a number of methods that have been designed to work with this sort of information, two of the main ones are Log Linear modelling and CHAID. These methods can be used to create scores or assign customers to groups, e.g. high value/Low value. Unfortunately both of these methods work best when trying to differentiate between two groups such as Yes/No rather than Yes/No/Maybe type questions.

Linear Regression

This assumes that there is a roughly linear relationship between two variables, e.g. Height and weight. A ‘best fit’ straight line plotted using both of the variables.

It is mainly used to estimate one variable where the other is known. The fit of the linear relationship can be investigated using the correlation coefficient; the closer this is to 1 the better the fit, a correlation of zero means that there is no linear relationship between the two variables. The correlation squared gives a measure of how much variation in one variable can be explained by variation in the other. Linear regression is of little use in marketing as relationships are generally more complicated than catered for here.

Limitations and cautions

This method is best used on continuous (Interval) variables and should be used with caution, if at all, when dealing with categorical data. The residual plot should always be examined to see if there is any pattern in the data; if there is it generally means that the actual relationship is non-linear. The results obtained by this method must be examined to investigate whether two highly correlated variables are not related to each other but to a third variable common to both; for instance time can affect two variables without the two variables interacting between themselves.

Multiple Regression (Linear)

This method is similar to linear regression in that it tries to calculate an unknown value using linear combinations of other key variables. The form the equation takes is :

y=Ax1+Bx2+Cx3+D

e.g. credit_score=0.45*Income-.25*sex+4.25*age+0.97

Uses

Credit scoring is one use of this method. Unfortunately a great deal of marketing data is categorical in nature and this technique is not quite as robust using this form of data.

Limitations and cautions

This technique assumes that the multivariate normal distribution is followed by the data. This method is best suited for continuous data, categorical binary fields (Male/Female or Yes/No) can be used but the results must be examined carefully. The greater the amount of categorical data there is the greater the likelihood that the multivariate normal distribution is not true.

Log-Linear Modelling

Method

Log linear modelling is a technique for analysing categorical data. It investigates the inter-relationships between the variables which form cross tabs of the data. It predicts the cell counts of a multivariate cross tabulation.

Uses

Log-linear modelling is very useful because it was designed to be used with categorical data. It can be used in the same way as discriminant analysis to try and predict groups such as likely to respond/no respond. It is more robust than discriminant analysis because it does not assume any underlying distribution.

Limitations and cautions

It can be adversely affected by very large data sets; firstly because of the size of the tables that it writes to memory and secondly because it can find relationships solely because the data set is large. A certain amount of pre-processing may be required prior to using this model.

Discriminant Analysis

In discriminant analysis the dependent variable in nominal or categorical, e.g. uses/does not use. discriminant analysis is used to try to predict which of the groups, or categories that a subject belongs to, e.g. respond/non respond, using a test data set. The ability to discriminate between the groups is shown in a ‘confusion matrix’ this shows the number of times that a person has been mis-classified and the number of times correctly classified.

Uses

Discriminant analysis is useful in finding those variables that are effective in predicting group membership, or what variables discriminate well between the groups, or categories. A credit scoring model is one example of a discriminant function with the dependent variable being good/bad risk. The results from a discriminant analysis are fairly easy to interpret and it can be simple to understand how well the function has performed. The variables that are most important to the model can be extracted from the underlying statistics.

Limitations and cautions

Discriminant analysis assumes that the data-set has the multivariate normal distribution. This means that a data set which contains a large number of categorical variables may produce a poor discrimination function.

Output

Below is a territory map produced using discriminant analysis. The main descriptors of the groupings, found using the discriminant analysis and profiling are affluence and commuting. Group 3 is a high affluence group who do not travel far to work, group 4 is high affluence and high travel to work.

Territory Map

Summary

Business scorecards can be extremely powerful and can be applied in a number of business scenarios e.g.

  • Identifying records within a larger universe such as the Business Universe.
  • Improve targeting – timing & propensity
  • Indicate the product propensity to be offered next
  • Identify lapsing, niche or problem groups
  • To create allowable marketing costs & ROI
  • Identify potential purchase opportunities

Scorecards can be built using discriminant, regression or logistic regression models. Although, the usefulness of using artificial intelligence models such as neural networks or fuzzy logic is worth investigating too. The performance of the model should be evaluated each time it is used by building-in “test cells” to allow you to examine response independent of the model.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>