In Part 1 and Part 2 of this blog posting series, we discussed: Our current viewpoints on marketing attribution and conversion journey analysis in 2017. The selection criteria of the best measurement approach. Introduced our vision on handling marketing attribution and conversion journey analysis. We would like to conclude this [...]
Algorithmic marketing attribution and conversion journey analysis [Part 3] was published on Customer Intelligence Blog.
In Part 1 of this blog posting series, we discussed our current viewpoints on marketing attribution and conversion journey analysis in 2017. We concluded on a cliffhanger, and would like to return to our question of which attribution measurement method should we ultimately focus on. As with all difficult questions [...]
Algorithmic marketing attribution and conversion journey analysis [Part 2] was published on Customer Intelligence Blog.
Editor's note: This blog post was authored by Malcolm Lightbody (SAS Customer Intelligence Product Management) and Suneel Grover (SAS Principal Solutions Architect).
Everyone has a marketing attribution problem, and all attribution measurement methods are wrong. We hear that 
all the time. Like many urban myths, it is founded in truth. Most organizations believe they can do better on attribution. They all understand that there are gaps, for example, missing touchpoint data, multiple identities across devices, arbitrary decisions on weightings for rules, and uncertainty about what actions arise from the results.
Broadly speaking, the holy grail of media measurement is to analyze the impact and business value of all company-generated marketing interactions across the complex customer journey. In this post, our goal is to take a transparent approach in discussing how SAS is building data-driven marketing technology to help customers progress beyond typical attribution methods to make the business case for customer journey optimization.
Being SAS, we advocate an analytic approach to addressing the operational and process-related obstacles that we commonly hear from customers. We want to treat them as two sides of the same coin. The output of attribution analytics informs marketers about what touch points and sequence of activities drive conversions. This leads marketers to make strategic decisions about future investment levels, as well as more tactical decisions about what activities to run. In an ideal world, the results of subsequent actions are fed back into the attribution model to increase not only its explanatory power, but also its predictive abilities, as shown below:
The diagram above shows the main parts of an attribution project. The actual analysis is just part of the process, with upstream and downstream dependencies. But this doesn’t always happen as it should. Consider a standard attribution report. Let us for the moment ignore what technique was used to generate the result and place ourselves in the shoes of the marketer trying to figure out what to do next.
In the graph above, we see the results of an attribution analysis based on a variety of measurement methods. Before answering the question of which method should we focus on, let's do a quick review of rules-based and algorithmic measurement techniques.
This type of attribution allocates 100 percent of the credit to either the last or first touch of the customer journey. This approach has genuine weaknesses, and ignores all other interactions with your brand across a multi-touch journey.
Linear attribution arbitrarily allocates an equal credit weight to every interaction along the customer journey. Although slightly better than the last- and first-touch approaches, linear attribution will undercredit and overcredit specific interactions.
Time-decay attribution arbitrarily biases the channel weighting based on the recency of the channel touches across the customer journey. If you support the concept of recency within RFM analysis, there is some merit to approach. Position-based attribution places more weight on the first and last touches, while providing less value to the interactions in between.
In contrast, algorithmic attribution (sometimes referred to as custom models) assigns data-driven conversion credit across all touch points preceding the conversion, and uses math typically associated with predictive analytics or machine learning to identify where credit is due. It analyzes both converting and non-converting consumer paths across all channels. Most importantly, it uses data to uncover the correlations and success factors within marketing efforts. Here is a video summarizing a customer case study example to help demystify what we mean.
Although many marketers recognize the value and importance of algorithmic attribution, adopting it hasn’t been easy. There are several reasons:
Returning to our question of which method should we ultimately focus on, the answer is it depends. An attribution report on its own cannot decide this. And it doesn’t even matter if the attribution report is generated using the most sophisticated algorithmic techniques. There are four things that the report won't tell you:
In Part 2 of this blog posting series, we will dive into specific detail within these areas, as well as introduce our vision within SAS Customer Intelligence 360 on handling algorithmic marketing attribution and conversion journey analysis.
Algorithmic marketing attribution and conversion journey analysis [Part 1] was published on Customer Intelligence Blog.
Optimization is a core competency for digital marketers. As customer interactions spread across fragmented touch points and consumers demand seamless and relevant experiences, content-oriented marketers have been forced to re-evaluate their strategies for engagement. But the complexity, pace and volume of modern digital marketing easily overwhelms traditional planning and design approaches that rely on historical conventions, myopic single-channel perspectives and sequential act-and-learn iteration.
SAS Customer Intelligence 360 Engage was released last year to address our client needs for a variety of modern marketing challenges. Part of the software's capabilities revolve around:
Regardless of the method, testing is attractive because it is efficient, measurable and serves as a machete cutting through the noise and assumptions associated with delivering effective experiences. The question is: How does a marketer know what to test?
There are so many possibilities. Let's be honest - if it's one thing marketers are good at, it's being creative. Ideas flow out of brainstorming meetings, bright minds flourish with motivation and campaign concepts are born. As a data and analytics geek, I've worked with ad agencies and client-side marketing teams on the importance of connecting the dots between the world of predictive analytics (and more recently machine learning) with the creative process. Take a moment to reflect on the concept of ideation.
Is it feasible to have too many ideas to practically try them all? How do you prioritize? Wouldn't it be awesome if a statistical model could help?
Let's break this down:
Here is the really sweet part. The space of visual analytics has matured dramatically. Creative minds dreaming of the next digital experience cannot be held back by hard-to-understand statistical greek. Nor can I condone the idea that if a magical analytic easy-button is accessible in your marketing cloud, one doesn't need to understand what's going on behind the scene.That last sentence is my personal opinion, and feel free to dive into my mind here.
Want a simple example? Of course you do. I'm sitting in a meeting with a bunch of creatives. They are debating on which pages should they run optimization tests on their website. Should it be on one of the top 10 most visited pages? That's an easy web analytic report to run. However, are those the 10 most important pages with respect to a conversion goal? That's where the analyst can step up and help. Here's a snapshot of a gradient boosting machine learning model I built in a few clicks with SAS Visual Data Mining and Machine Learning leveraging sas.com website data collected by SAS Customer Intelligence 360 Discover on what drives conversions.
I know what you're thinking. Cool data viz picture. So what? Take a closer look at this...
The model prioritizes what is important. This is critical, as I have transparently highlighted (with statistical vigor I might add) that site visitor interest in our SAS Customer Intelligence product page is popping as an important predictor in what drives conversions. Now what?
The creative masterminds and I agree we should test various ideas on how to optimize the performance of this important web page. A/B test? Multivariate test? As my SAS colleague Malcolm Lightbody stated:
"Multivariate testing is the way to go when you want to understand how multiple web page elements interact with each other to influence goal conversion rate. A web page is a complex assortment of content and it is intuitive to expect that the whole is greater than the sum of the parts. So, why is MVT less prominent in the web marketer’s toolkit?
One major reason – cost. In terms of traffic and opportunity cost, there is a combinatoric explosion in unique versions of a page as the number of elements and their associated levels increase. For example, a page with four content spots, each of which have four possible creatives, leads to a total of 256 distinct versions of that page to test.
If you want to be confident in the test results, then you need each combination, or variant, to be shown to a reasonable sample size of visitors. In this case, assume this to be 10,000 visitors per variant, leading to 2.56 million visitors for the entire test. That might take 100 or more days on a reasonably busy site. But by that time, not only will the marketer have lost interest – the test results will likely be irrelevant."
SAS Customer Intelligence 360 provides a business-user interface which allows the user to:
Continuing with my story, we decide to set up a test on the sas.com customer intelligence product page with four content spots, and three creatives per spot. This results in 81 total variants and an estimated sample size of 1,073,000 visits to get a significant read at a 90 percent confidence level.
Notice that Optimize button in the image? Let's talk about the amazing special sauce beneath it. Methodical experimentation has many applications for efficient and effective information gathering. To reveal or model relationships between an input, or factor, and an output, or response, the best approach is to deliberately change the former and see whether the latter changes, too. Actively manipulating factors according to a pre-specified design is the best way to gain useful, new understanding.
However, whenever there is more than one factor – that is, in almost all real-world situations – a design that changes just one factor at a time is inefficient. To properly uncover how factors jointly affect the response, marketers have numerous flavors of multivariate test designs to consider. Factorial experimental designs are more common, such as full factorial, fractional factorial, and mixed-level factorial. The challenge here is each method has strict requirements.
This leads to designs that, for example, are not orthogonal or that have irregular design spaces. Over a number of years SAS has developed a solution to this problem. This is contained within the OPTEX procedure, and allows testing of designs for which:
The OPTEX procedure can generate an efficient experimental design for any of these situations and website (or mobile app) multivariate testing is an ideal candidate because it applies:
The OPTEX procedure is highly flexible and has many input parameters and options. This means that it can cover different digital marketing scenarios, and it’s use can be tuned as circumstances demand. Customer Intelligence 360 provides the analytic heavy lifting behind the scenes, and the marketer only needs to make choices for business relevant parameters. Watch what happens when I press that Optimize button:
Suddenly that scary sample size of 1,070,000 has reduced to 142,502 visits to perform my test. The immediate benefit is the impractical multivariate test has become feasible. However, if only a subset of the combinations are being shown, how can the marketer understand what would happen for an untested variant? Simple! SAS Customer Intelligence 360 fits a model using the results of the tested variants and uses them to predict the outcomes for untested combinations. In this way, the marketer can simulate the entire multivariate test and draw reliable conclusions in the process.
So you're telling me we can dream big in the creative process and unleash our superpowers? That's right my friends, you can even preview as many variants of the test's recipe as you desire.
The majority of today’s technologies for digital personalization have generally failed to effectively use predictive analytics to offer customers a contextualized digital experience. Many of today’s offerings are based on simple rules-based recommendations, segmentation and targeting that are usually limited to a single customer touch point. Despite some use of predictive techniques, digital experience delivery platforms are behind in incorporating machine learning to contextualize digital customer experiences.
At the end of the day, connecting the dots between data science and testing, no matter which flavor you select, is a method I advocate. The challenge I pose to every marketing analyst reading this:
Can you tell a good enough data story to inspire the creative minded?
How does a marketer know what to test? was published on Customer Intelligence Blog.
How can you tell if your marketing is working? How can you determine the cost and return of your campaigns? How can you decide what to do next? An effective way to answer these questions is to monitor a set of key performance indicators, or KPIs.
KPIs are the basic statistics that give you a clear idea of how your website (or app) is performing. KPIs vary by predetermined business objectives, and measure progress towards those specific objectives. In the famous words of Avinash Kaushik, KPIs should be:
An example that fits this description, with applicability to profit, nonprofit, and e-commerce business models, would be the almighty conversion rate. In digital analytics this metric is interpreted as the proportion of visitors to a website or app who take action to go beyond a casual content view or site visit, as a result of subtle or direct requests from marketers, advertisers, and content creators.
Although successful conversions can be defined differently based on your use case, it is easy to see why this KPI is uncomplex, relevant, timely, and useful. We can even splinter this metric into two types:
Macro conversion – Someone completes an action that is important to your business (like making you some money).
Micro conversion – An indicator that a visitor is moving towards a macro conversion (like progressing through a multi-step sales funnel to eventually make you some money)
Regardless of the conversion type, I have always found that reporting on this KPI is a popular request for analysts from middle management and executives. However, it isn't difficult to anticipate what is coming next from the most important person in your business world:
"How can we improve our conversion rate going forward?"
You can report, slice, dice, and segment away in your web analytics platform, but needles in haystacks are not easily discovered unless we adapt. I know change can be difficult, but allow me to make the case for machine learning and hyperparameters within the discipline of digital analytics. A trendy subject for some, a scary subject for others, but my intent is to lend a practitioner's viewpoint. Analytical decision trees are an excellent way to begin because of their frequent usage within marketing applications, primarily due to their approachability, and ease of interpretation.
Whether your use case is for supervised segmentation, or propensity scoring, this form of predictive analytics can be labeled as machine learning due to algorithm's approach to analyzing data. Have you ever researched how trees actually learn before arriving to a final result? It's beautiful math. However, it doesn't end there. We are living in a moment where more sophisticated machine learning algorithms have emerged that can comparatively increase predictive accuracy, precision, and most importantly – marketing-centric KPIs, while being just as easy to construct.
Using the same data inputs across different analysis types like Forests, Gradient Boosting, and Neural Networks, analysts can compare model fit statistics to determine which approach will have the most meaningful impact on your organization's objectives. Terms like cumulative lift or misclassification may not mean much to you, but they are the keys to selecting the math that best answers how conversion rate can be improved by transparently disclosing accurate views of variable importance.
So is that it? I can just drag and drop my way through the world of visual analytics to optimize against KPIs. Well, there is a tradeoff to discuss here. For some organizations, simply using a machine learning algorithm enabled by an easy-to-use software interface will help improve conversion rate tactics on a mobile app screen experience as compared to not using an analytic method. But an algorithm cannot be expected to perform well as a one size fits all approach for every type of business problem. It is a reasonable question to ask oneself if opportunity is being left on the table to motivate analysts to refine the math to the use case. Learning to improve how an algorithm arrives at a final result should not be scary because it can get a little technical. It's actually quite the opposite, and I love learning how machine learning can be elegant. This is why I want to talk about hyperparameters!
Anyone who has ever built a predictive model understands the iterative nature of adjusting various property settings of an algorithm in an effort to optimize the analysis results. As we endlessly try to improve the predictive accuracy, the process becomes painfully repetitive and manual. Due to the typical length of time an analyst can spend on this task alone - from hours, days, or longer - the approach defies our ability as humans to practically arrive at an optimized final solution. Sometimes referred to as auto tuning, hyperparameters address this issue by exploring different combinations of algorithm options, training a model for each option in an effort to find the best model. Imagine running 1000s of iterations of a website conversion propensity model across different property threshold ranges in a single execution. As a result, these models can improve significantly across important fit statistics that relate directly to your KPIs.
At the end of running an analysis with hyperparameters, the best recipe will be identified. Just like any other modeling project, the ability to action off of the insight is no different, from traditional model score code to next-best-action recommendations infused into your mobile app's personalization technology. That's genuinely exciting, courtesy of recent innovations in distributed analytical engines with feature-rich building blocks for machine-learning activities.
If the subject of hyperparameters is new to you, I encourage you to watch this short video.
This will be one of the main themes of my presentations at Analytics Experience 2017 in Washington DC. Using digital data collected by SAS Customer Intelligence 360 and analyzing it with SAS Visual Data Mining & Machine Learning on VIYA, I want to share the excitement I am feeling about digital intelligence and predictive personalization. I hope you'll consider joining the SAS family for an awesome agenda between September 18th-20th in our nation's capital.
Hyperparameters, digital analytics, and key performance indicators was published on Customer Intelligence Blog.
Today in higher education, savvy users expect to have the information they need to make data-informed decisions at their fingertips. As such, leaders in institutional research (IR) are under pressure to provide these users with accurate data, reports and analyses. IR has been tasked with transforming data and reports in [...]
6 examples of data management, reporting and analytics in higher education was published on SAS Voices by Georgia Mariani
Digital intelligence is a trending term in the space of digital marketing analytics that needs to be demystified. Let's begin by defining what a digital marketing analytics platform is:
Digital marketing analytics platforms are technology applications used by customer intelligence ninjas to understand and improve consumer experiences. Prospecting, acquiring, and holding on to digital-savvy customers depends on understanding their multidevice behavior, and derived insight fuels marketing optimization strategies. These platforms come in different flavors, from stand-alone niche offerings, to comprehensive end-to-end vehicles performing functions from data collection through analysis and visualization.
However, not every platform is built equally from an analytical perspective. According to Brian Hopkins, a Forrester analyst, firms that excel at using data and analytics to optimize their digital businesses will together generate $1.2 trillion per annum in revenue by 2020. And digital intelligence — the practice of continuously optimizing customer experiences with online and offline data, advanced analytics and prescriptive insights — supports every insights-driven business. Digital intelligence is the antidote to the weaknesses of analytically immature platforms, leaving the world of siloed reporting behind and maturing towards actionable, predictive marketing. Here are a couple of items to consider:
As consumer behavioral needs and preferences shifts over time, the proportion of digital to non-digital interactions is growing. With the recent release of Customer Intelligence 360, SAS has carefully considered feedback from our customers (and industry analysts) to create technology that supports a modern digital intelligence strategy in guiding an organization to:
The question now is - are you ready? Learn more here of why we are so excited about enabling digital intelligence for our customers, and how this benefits testing, targeting, and optimization of customer experiences.
Digital intelligence for optimizing customer engagement was published on Customer Intelligence.
We've been saying that the customer is queen or king for quite some time now. And in the coming year, that will be truer than ever. The customer determines where he or she finds information and which channel and which supplier gets the sale. And there is an abundance of these suppliers (certainly online). Customer loyalty, it seems, is as good as dead. Yes, of course, we are faithful to our local baker and tailor, but for items we don't buy everyday and where there is no personal relationship with the supplier (nor does there always need to be one), we don't really care where we order from. Right?
Today, many consumers make their choice based on only two criteria: price and reviews – the latter providing some confidence about product quality and supplier reliability. And it's an obvious choice. Why would you pay top price for an OEM device charger that you can get from a Chinese web shop for a fraction of the price – unless you need it tomorrow, of course? Virtually no supplier has a monopoly today, and you can switch to a new supplier with just one mouse click.
So does that leave all companies having to compete solely on price? No, that would create an unhealthy market situation. Aiming for good reviews is a great idea, of course, but is merely a partial solution. To encourage customer loyalty in the long term, you need to focus heavily on the last touch point in the customer journey. The three elements are essential in these efforts: data, analytics and real-time decisioning.
Customers leave a data trail behind in various channels. This data enables you to build up a wealth of information about the customer. This is nothing new, but I have noticed that a lot of companies have difficulty in determining what data from this data stream they should add to the customer profile. By analysing the data, you can determine whether data can be assigned as a fixed value to the customer, or is of only temporary relevance, such as a location, for example. In addition, you can really get to know your customer by analysing this data, using this knowledge to predict behaviour and responding to this behaviour in real time.
Using analytics to predict customer behaviour is the key to success in the last step of the customer journey. In this way, you can create the ultimate balance between customer service-driven interactions and marketing and sales-driven interactions. Just think how valuable it would be to know at this last touch point whether you should persuade the customer with your service, or use a combination offer with a product from the same line?
By using data strategically, you can predict where the customer has a need. You know what motivates him to actually make a purchase at that critical decision point. Responding smartly to this will increase customer satisfaction and make those customers more loyal. As a result, you will see that price and reviews are indeed important, but that customers still need a supplier who knows and recognises them, and responds to their needs.
To learn more about creating fiercely loyal customers, download our free ebook, Keep them coming back: You guide to building customer loyalty with analytics.
How do you revive customer loyalty in the digital age? was published on Customer Intelligence.
Machine learning has a high profile currently and is riding a wave of exposure in the media that includes articles about subjects from self-driving cars and self-landing rockets, to computers beating the world’s best players at Go, the most computationally complex board game in the world. Is there an opportunity for your organisation, and the marketers within it, to make use of this “new” technology?
Machine learning techniques were developed as long ago as the 1950s, but with the advent of big data and large analytical engines, the prevalence and the ease of applying the techniques has increased. 
Additionally, organisations now understand the value that analytics can bring, so are willing to place it front and center in their plans and invest more time and resources in exploring new and better techniques. Segmentation and predictive models, for instance, have proven themselves time and again in the marketing world, but to a certain extent, they require a higher degree of knowledge to understand. In some cases, a machine learning technique unburdens the user of the statistical work, but provides just as good an answer as a traditional technique. More people, with more data, trying to make more decisions lends itself to a technique that requires less manual intervention.
Organizations, large and small, can have huge, complex data that can from the latest advances in machine learning – banks have transaction records, telcos have call details, retailers have purchase records.
Take marketing in our omnichannel world as an example. There are huge amounts of customer interactions and there are business problems, such as attribution and optimizing the customer experience, that are perfect for the latest machine learning techniques. For real-time personalization of experience and real-time calculation of recommendations, great benefit can be gained from self-learning algorithms in reinforcement learning.
But it is important to remember that organizations also have many analytically driven challenges that are smaller, simpler and just as important and valuable to the bottom line of the organization. Again, for marketing, more traditional disciplines like segmentation and propensity modeling are still extremely useful, and organizations need to keep using capabilities like these to ensure the continued benefits from their use.
SAS has embraced machine learning techniques for many years, and recently took a further step forward with the latest release of our SAS Customer Intelligence 360 suite of products. SAS has built a recommendation engine with the best of both worlds – a predictive model built using traditional techniques (logistic regression) and a machine learning algorithm (using naïve Bayes classifiers). Fortunately, your customers don’t need to understand these techniques – they just want your website to make better recommendations!
Machine learning and what it means for marketing was published on Customer Intelligence.
For the uninitiated, SAS 360 Engage enables organizations to interact with consumers by allowing them to create, manage and deliver digital content over web and mobile channels. Wait a minute. SAS does more than the analytics? That is correct. SAS 360 Engage is a marketing super force serving as a one-stop shop for data capture all the way through delivering highly-targeted, personalized digital experiences.
Being able to dynamically place content and offers into digital channels – across devices and points in time – is nothing new for savvy marketing brands focused on optimization. As customer journeys spread across fragmented touch points while customers are demanding seamless and relevant experiences, content-oriented marketers have been forced to reevaluate their strategies for engagement. But the complexity, pace and volume of modern marketing easily overwhelms traditional planning and design approaches that rely on historical conventions, myopic single-channel perspectives and sequential act-and-learn iteration.
Presently, marketers primarily use a variety of content optimization approaches that include A/B testing and multivariate testing. A/B testing, at its simplest, is a method of website or mobile optimization in which the conversion rates of two versions of a page are compared using visitor response rates. By tracking the way visitors interact with the content– the videos they watch, the buttons they click, or whether they sign up for a newsletter – you can infer which version of the content is most effective.
Due to the popularity of this technique, SAS 360 Engage supports A/B/n testing. A/B/n testing is an extension of A/B testing, where “N” refers to the number of versions being tested, anywhere from two versions to the “nth” version. For example, when a brand has more than one idea for what the ideal digital experience should be, A/B/n can be used to compare each hypothesis and produce an optimized decision based on data, not subjectivity.
Testing is attractive because it is efficient, measurable and serves as a machete cutting through the noise and assumptions associated with delivering effective experiences. In parallel, the evolving marketing landscape is driving a greater mandate for testing: to operate in more channels, handle more data and support more users. Testing must mature beyond traditional on-site experimentation to fully optimize a multifaceted customer journey.
The majority of today’s technologies for personalization have generally failed to effectively use data science to offer consumers a contextualized digital experience. Many of today’s offerings are based on simple rules-based segmentation to drive recommendations. Building off the benefits of multi-channel A/B/n testing, this is where SAS 360 Engage injects its analytical muscle to differentiate from other personalization technologies. Let's break this down:
Performing algorithmic segmentation sounds awesome, but who really has the time to do it? We have so many tests to run.
The time has arrived for predictive marketing to have its moment in the sun, and with Forrester recently naming SAS the leader in customer analytics, it's official - the 800-pound gorilla in advanced analytics is locked in on solving complex issues facing the space of data-driven marketing. Making digital personalization more relevant for target audiences is just like preparing a delicious meal; it all comes down to the ingredients and preparation process to rise to the occasion!
A beautiful and interpretable visualization is generated highlighting what is unique about this segment, as compared to everyone else who was exposed to the test. If the brand wants to target this audience in future campaigns, a single click populates this segment in the platform for future journey orchestration.
If you look closely at the image, you will note in the upper half of the report that the winner of the A/B/n test is variant A. However, the lower half of the report showcases a newly discovered segment. It turns out that when a specific customer segment with recent purchase, stay and amenity activity interacts with this hospitality brand, variant B produces better results. How did SAS 360 Engage do this? By applying automated firepower (i.e. algorithmic clustering) to produce this prescriptive and actionable insight. To learn more about this segment, marketers can profile the audience:
SAS 360 Engage was built with the recognition that some marketing teams don't have data scientists available, and have real needs for analytical automation. To improve upon the concept of A/B/n testing, augmenting this capability with automated, algorithmic segmentation with prescriptive results addresses an important need. Let's assume you've run an A/B/n test with four versions of a page, and variant A was crowned the champion. Wouldn't it be nice to know that if a specific segment arrived at your website, an alternative experience would facilitate a better result?
SAS 360 Engage: A/B testing and algorithmic segmentation was published on Customer Intelligence.
