Behavioral Space Modeling Blog Series: The Marketing Funnel

This is the first in a multi-part series on performing basic behavioral modeling with digital datasets and its applications in marketing.

The Behavioral Space

We cannot read our customers’ minds. The closest we can probably come is directly surveying consumers, but continually surveying every individual consumer is infeasible and would lead to a customer experience falling somewhere between bizarre and excruciating. We can, however, observe certain behavioral characteristics, especially in the digital arena. Clicks and impressions from advertising, mobile app tracking and on-site analytics are all well-established vectors for measuring behavior and engagement. By combining everything from time-on-site to display ad impressions, a huge number of different behavioral traits can be synthesized, leading to a high-dimensional behavioral space. An individual’s recent behavior can be mapped in the behavioral space and, if analyzed properly, can serve as a strong indicator of intent.

This behavioral space is, in a sense, actually a mathematical representation of the marketing funnel. If we can identify a group of recent converters that all fall in the same region of the behavioral space, we can define that region as the bottom of the funnel. Any nearby non-converters are likely to be in the final stages of the consideration process – a crucial time to reach out and nudge them towards conversion with advertising and CRM. After identifying these high-potential prospects, they can be targeted by all of the usual means and monitored for lift. Lists of these prospects can be continually updated to power evergreen lower-funnel campaigns.

Defining the Space

For our example, we have chosen 21 distinct behavioral traits, or dimensions, to define consumer behavior. High-dimensional spaces are not difficult to work with in an abstract, mathematical sense, but they are impossible to interpret of visually. Dimensionality reduction techniques are very effective at combining multiple dimensions into a smaller subset. There are quite a few dimensionality reduction techniques available, each one having advantages and disadvantages.¹ In this case, we have chosen a technique that is specifically designed to maintain the relationships between nearby points. In other words, any points that were close together in our high-dimensional space will remain close together in our low-dimensional space.

Stepping back for a moment, what are we hoping to accomplish here? We would like to be able to plot our marketing funnel in such a way that we can actually see what consumers may be nearing conversion. After dimensionality reduction, the axes for our plot will not correspond directly to single behavioral traits and will be difficult to interpret, but the plots themselves will be immensely informative nonetheless. Visualizations assist in validating datasets (see the very famous example of Anscombe’s quartet²) and dimensionality reduction can help “clean” the data by eliminating noisy behavioral signals – a good idea before employing machine learning.

So, does it work? In the plot below, we have reduced our original 21 dimensions to 3 and plotted converter behavior as blue points, with non-converter behavior as red points.

Fig 1. The behavioral space representation of converters (blue) and non-converters (red). Regions occupied by both converters and non-converters represent high potential for prospecting.

There are already a few interesting observations to be had here that validate our approach thus far:

• There are regions that are occupied purely by converters. These converters exhibit behavior that differs substantially from non-converters – perhaps these are extremely loyal customers that behave in ways that non-customers cannot.
• Similarly, there are groups of non-converters that lie far away from any converters. These are likely to be either prospects that inhabit a higher section of the marketing funnel or individuals who have engaged erroneously.
• The final category comprises regions where there are strong overlaps between converters and non-converters. The non-converters residing in these groups are behaving very much like converters and are therefore likely to be close to conversion themselves. The best example of this is the group in the top right of figure 1, which we will call “cluster 1.”

This is an excellent first step and clearly shows promise. However, we still need to rigorously define the groupings in such a way that prospects can be consistently and automatically identified.

Behavioral Clustering

We can clearly see separation between groups in figure 1 and want to define them algorithmically. Clustering is the most obvious method to employ for this. Much like dimensionality reduction, there are many clustering algorithms and choosing one is performed on a case-by-case basis. In this case, we want a flexible algorithm with two characteristics:

• It handles heterogeneous clusters well. Not all of our clusters are spheroidal. We see quite a few elongated and oddly shaped groups.
• It can identify outliers or noise. There are many points that do not fall close to the denser, natural groups. As we do not know how to classify these individuals, we should be able to automatically discard them.

In general, we should aim for our clustering method to be able to identify the same patterns that we do. Although machine learning is making rapid progress, it still has nowhere near the pattern recognition capabilities of humans yet. Without going into much technical detail here, the DBSCAN algorithm is a strong choice for this application and the results are encouraging.

Fig. 2: Clustering results. Cluster 1 (light blue) is separated from the rest of the clusters. Outliers (mauve) appear to be identified appropriately. The patterns are generally clean, with clear delineations between clusters.

The clustering performed as expected, and we see cluster 1 (light blue) separated nicely from the rest of the points. Cluster 1 contains many high-potential prospects and is now rigorously defined. As we monitor the behavioral data, new individuals will move into cluster 1 and we can begin highly targeted evergreen lower-funnel advertising activity.

Next Steps

Everything looks good so far, but how do we know this has really identified high-potential prospects? This is a very complicated question when using unsupervised techniques (such as clustering) in general and will be tackled in a future post. For now, we have made considerable progress towards segmenting the behavioral space in an actionable way. Assuming we believe these clusters and this definition of the behavioral space to be legitimate, marketers can look at aggregate statistics for KPIs or quantities of interest for each cluster and target based on those observations. Alternatively, multiple clusters can be targeted simultaneously to determine what region of the behavioral space is best suited for specific messaging. We will discuss approaches for both in silico and in vivo testing approaches to validate the methodology in the next post.

1. Van Der Maaten, Laurens, Eric Postma, and Jaap Van den Herik. “Dimensionality reduction: a comparative.” J Mach Learn Res 10.66-71 (2009): 13.
2. Anscombe, Francis J. “Graphs in statistical analysis.” The American Statistician 27.1 (1973): 17-21.