In a previous posting, SAS Customer Intelligence 360 was highlighted in the context of delivering relevant product, service, and content recommendations using automated machine learning within digital experiences. Shifting gears, SAS recognizes there are different user segments for our platform. This post will focus on building custom analytical recommendation models [...]
In part one of this blog posting series, we took an introductory tour of recommendation systems, digital marketing, and SAS Customer Intelligence 360. Helping users of your website or mobile properties find items of interest is useful in almost any situation. This is why the concept of personalized marketing is [...]
Regardless of industry, it has become a frequent occurrence that behind every data-driven marketer is an analytical ninja. Together, they formulate recipes in addressing the customer-centric paradigm that considers the different actions that a brand can take for a specific individual, and decides on the “best” one. The goal of [...]
Why does your organization’s website or mobile app exist? What are you hoping to accomplish with your business by being digital? What are the most important priorities for your digital presence? Here are three goals that most organizations share: Sell more stuff. Make marketing more effective. Delight customers. Goals are [...]
Path analysis is an exploration of a chain of consecutive events that a given user or cohort performs during a set period while using a website, online game or mobile app (although other use cases can apply outside of digital analytics). As a subset of behavioral analytics, path analysis is [...]
When data meets geography, use cases revolve around mapping and spatial analytics. But what happens when you combine digital analytics and powerful visualization for customer location analysis? Leveraging data collection mechanisms, SAS 360 Discover captures first-party behavioral information across the entire digital customer experience with a brand’s websites and mobile [...]
Optimization is a core competency for digital marketers. As customer interactions spread across fragmented touch points and consumers demand seamless and relevant experiences, content-oriented marketers have been forced to re-evaluate their strategies for engagement. But the complexity, pace and volume of modern digital marketing easily overwhelms traditional planning and design approaches that rely on historical conventions, myopic single-channel perspectives and sequential act-and-learn iteration.
SAS Customer Intelligence 360 Engage was released last year to address our client needs for a variety of modern marketing challenges. Part of the software's capabilities revolve around:
Regardless of the method, testing is attractive because it is efficient, measurable and serves as a machete cutting through the noise and assumptions associated with delivering effective experiences. The question is: How does a marketer know what to test?
There are so many possibilities. Let's be honest - if it's one thing marketers are good at, it's being creative. Ideas flow out of brainstorming meetings, bright minds flourish with motivation and campaign concepts are born. As a data and analytics geek, I've worked with ad agencies and client-side marketing teams on the importance of connecting the dots between the world of predictive analytics (and more recently machine learning) with the creative process. Take a moment to reflect on the concept of ideation.
Is it feasible to have too many ideas to practically try them all? How do you prioritize? Wouldn't it be awesome if a statistical model could help?
Let's break this down:
- Predictive analytic or machine learning projects always begin with data. Specifically training data which is fed to algorithms to address an important business question.
- Ultimately, at the end of this exercise, a recommendation can be made prescriptively to a marketer to take action. This is what we refer to as a hypothesis. It is ready to be tested in-market.
- This is the connection point between analytics and testing. Just because a statistical model informs us to do something slightly different, it still needs to be tested before we can celebrate.
Here is the really sweet part. The space of visual analytics has matured dramatically. Creative minds dreaming of the next digital experience cannot be held back by hard-to-understand statistical greek. Nor can I condone the idea that if a magical analytic easy-button is accessible in your marketing cloud, one doesn't need to understand what's going on behind the scene.That last sentence is my personal opinion, and feel free to dive into my mind here.
Want a simple example? Of course you do. I'm sitting in a meeting with a bunch of creatives. They are debating on which pages should they run optimization tests on their website. Should it be on one of the top 10 most visited pages? That's an easy web analytic report to run. However, are those the 10 most important pages with respect to a conversion goal? That's where the analyst can step up and help. Here's a snapshot of a gradient boosting machine learning model I built in a few clicks with SAS Visual Data Mining and Machine Learning leveraging sas.com website data collected by SAS Customer Intelligence 360 Discover on what drives conversions.
I know what you're thinking. Cool data viz picture. So what? Take a closer look at this...
The model prioritizes what is important. This is critical, as I have transparently highlighted (with statistical vigor I might add) that site visitor interest in our SAS Customer Intelligence product page is popping as an important predictor in what drives conversions. Now what?
The creative masterminds and I agree we should test various ideas on how to optimize the performance of this important web page. A/B test? Multivariate test? As my SAS colleague Malcolm Lightbody stated:
"Multivariate testing is the way to go when you want to understand how multiple web page elements interact with each other to influence goal conversion rate. A web page is a complex assortment of content and it is intuitive to expect that the whole is greater than the sum of the parts. So, why is MVT less prominent in the web marketer’s toolkit?
One major reason – cost. In terms of traffic and opportunity cost, there is a combinatoric explosion in unique versions of a page as the number of elements and their associated levels increase. For example, a page with four content spots, each of which have four possible creatives, leads to a total of 256 distinct versions of that page to test.
If you want to be confident in the test results, then you need each combination, or variant, to be shown to a reasonable sample size of visitors. In this case, assume this to be 10,000 visitors per variant, leading to 2.56 million visitors for the entire test. That might take 100 or more days on a reasonably busy site. But by that time, not only will the marketer have lost interest – the test results will likely be irrelevant."
SAS Customer Intelligence 360 provides a business-user interface which allows the user to:
- Set up a multivariate test.
- Define exclusion and inclusion rules for specific variants.
- Optimize the design.
- Place it into production.
- Examine the results and take action.
Continuing with my story, we decide to set up a test on the sas.com customer intelligence product page with four content spots, and three creatives per spot. This results in 81 total variants and an estimated sample size of 1,073,000 visits to get a significant read at a 90 percent confidence level.
Notice that Optimize button in the image? Let's talk about the amazing special sauce beneath it. Methodical experimentation has many applications for efficient and effective information gathering. To reveal or model relationships between an input, or factor, and an output, or response, the best approach is to deliberately change the former and see whether the latter changes, too. Actively manipulating factors according to a pre-specified design is the best way to gain useful, new understanding.
However, whenever there is more than one factor – that is, in almost all real-world situations – a design that changes just one factor at a time is inefficient. To properly uncover how factors jointly affect the response, marketers have numerous flavors of multivariate test designs to consider. Factorial experimental designs are more common, such as full factorial, fractional factorial, and mixed-level factorial. The challenge here is each method has strict requirements.
This leads to designs that, for example, are not orthogonal or that have irregular design spaces. Over a number of years SAS has developed a solution to this problem. This is contained within the OPTEX procedure, and allows testing of designs for which:
- Not all combinations of the factor levels are feasible.
- The region of experimentation is irregularly shaped.
- Resource limitations restrict the number of experiments that can be performed.
- There is a nonstandard linear or a nonlinear model.
The OPTEX procedure can generate an efﬁcient experimental design for any of these situations and website (or mobile app) multivariate testing is an ideal candidate because it applies:
- Constraints on the number of variants that are practical to test.
- Constraints on required or forbidden combinations of content.
The OPTEX procedure is highly flexible and has many input parameters and options. This means that it can cover different digital marketing scenarios, and it’s use can be tuned as circumstances demand. Customer Intelligence 360 provides the analytic heavy lifting behind the scenes, and the marketer only needs to make choices for business relevant parameters. Watch what happens when I press that Optimize button:
Suddenly that scary sample size of 1,070,000 has reduced to 142,502 visits to perform my test. The immediate benefit is the impractical multivariate test has become feasible. However, if only a subset of the combinations are being shown, how can the marketer understand what would happen for an untested variant? Simple! SAS Customer Intelligence 360 fits a model using the results of the tested variants and uses them to predict the outcomes for untested combinations. In this way, the marketer can simulate the entire multivariate test and draw reliable conclusions in the process.
So you're telling me we can dream big in the creative process and unleash our superpowers? That's right my friends, you can even preview as many variants of the test's recipe as you desire.
The majority of today’s technologies for digital personalization have generally failed to effectively use predictive analytics to offer customers a contextualized digital experience. Many of today’s offerings are based on simple rules-based recommendations, segmentation and targeting that are usually limited to a single customer touch point. Despite some use of predictive techniques, digital experience delivery platforms are behind in incorporating machine learning to contextualize digital customer experiences.
At the end of the day, connecting the dots between data science and testing, no matter which flavor you select, is a method I advocate. The challenge I pose to every marketing analyst reading this:
Can you tell a good enough data story to inspire the creative minded?
How can you tell if your marketing is working? How can you determine the cost and return of your campaigns? How can you decide what to do next? An effective way to answer these questions is to monitor a set of key performance indicators, or KPIs.
KPIs are the basic statistics that give you a clear idea of how your website (or app) is performing. KPIs vary by predetermined business objectives, and measure progress towards those specific objectives. In the famous words of Avinash Kaushik, KPIs should be:
- Instantly useful.
An example that fits this description, with applicability to profit, nonprofit, and e-commerce business models, would be the almighty conversion rate. In digital analytics this metric is interpreted as the proportion of visitors to a website or app who take action to go beyond a casual content view or site visit, as a result of subtle or direct requests from marketers, advertisers, and content creators.
Although successful conversions can be defined differently based on your use case, it is easy to see why this KPI is uncomplex, relevant, timely, and useful. We can even splinter this metric into two types:
Micro conversion – An indicator that a visitor is moving towards a macro conversion (like progressing through a multi-step sales funnel to eventually make you some money)
Regardless of the conversion type, I have always found that reporting on this KPI is a popular request for analysts from middle management and executives. However, it isn't difficult to anticipate what is coming next from the most important person in your business world:
"How can we improve our conversion rate going forward?"
You can report, slice, dice, and segment away in your web analytics platform, but needles in haystacks are not easily discovered unless we adapt. I know change can be difficult, but allow me to make the case for machine learning and hyperparameters within the discipline of digital analytics. A trendy subject for some, a scary subject for others, but my intent is to lend a practitioner's viewpoint. Analytical decision trees are an excellent way to begin because of their frequent usage within marketing applications, primarily due to their approachability, and ease of interpretation.
Whether your use case is for supervised segmentation, or propensity scoring, this form of predictive analytics can be labeled as machine learning due to algorithm's approach to analyzing data. Have you ever researched how trees actually learn before arriving to a final result? It's beautiful math. However, it doesn't end there. We are living in a moment where more sophisticated machine learning algorithms have emerged that can comparatively increase predictive accuracy, precision, and most importantly – marketing-centric KPIs, while being just as easy to construct.
Using the same data inputs across different analysis types like Forests, Gradient Boosting, and Neural Networks, analysts can compare model fit statistics to determine which approach will have the most meaningful impact on your organization's objectives. Terms like cumulative lift or misclassification may not mean much to you, but they are the keys to selecting the math that best answers how conversion rate can be improved by transparently disclosing accurate views of variable importance.
So is that it? I can just drag and drop my way through the world of visual analytics to optimize against KPIs. Well, there is a tradeoff to discuss here. For some organizations, simply using a machine learning algorithm enabled by an easy-to-use software interface will help improve conversion rate tactics on a mobile app screen experience as compared to not using an analytic method. But an algorithm cannot be expected to perform well as a one size fits all approach for every type of business problem. It is a reasonable question to ask oneself if opportunity is being left on the table to motivate analysts to refine the math to the use case. Learning to improve how an algorithm arrives at a final result should not be scary because it can get a little technical. It's actually quite the opposite, and I love learning how machine learning can be elegant. This is why I want to talk about hyperparameters!
Anyone who has ever built a predictive model understands the iterative nature of adjusting various property settings of an algorithm in an effort to optimize the analysis results. As we endlessly try to improve the predictive accuracy, the process becomes painfully repetitive and manual. Due to the typical length of time an analyst can spend on this task alone - from hours, days, or longer - the approach defies our ability as humans to practically arrive at an optimized final solution. Sometimes referred to as auto tuning, hyperparameters address this issue by exploring different combinations of algorithm options, training a model for each option in an effort to find the best model. Imagine running 1000s of iterations of a website conversion propensity model across different property threshold ranges in a single execution. As a result, these models can improve significantly across important fit statistics that relate directly to your KPIs.
At the end of running an analysis with hyperparameters, the best recipe will be identified. Just like any other modeling project, the ability to action off of the insight is no different, from traditional model score code to next-best-action recommendations infused into your mobile app's personalization technology. That's genuinely exciting, courtesy of recent innovations in distributed analytical engines with feature-rich building blocks for machine-learning activities.
If the subject of hyperparameters is new to you, I encourage you to watch this short video.
This will be one of the main themes of my presentations at Analytics Experience 2017 in Washington DC. Using digital data collected by SAS Customer Intelligence 360 and analyzing it with SAS Visual Data Mining & Machine Learning on VIYA, I want to share the excitement I am feeling about digital intelligence and predictive personalization. I hope you'll consider joining the SAS family for an awesome agenda between September 18th-20th in our nation's capital.
Digital marketing analytics platforms are technology applications used by customer intelligence ninjas to understand and improve consumer experiences. Prospecting, acquiring, and holding on to digital-savvy customers depends on understanding their multidevice behavior, and derived insight fuels marketing optimization strategies. These platforms come in different flavors, from stand-alone niche offerings, to comprehensive end-to-end vehicles performing functions from data collection through analysis and visualization.
However, not every platform is built equally from an analytical perspective. According to Brian Hopkins, a Forrester analyst, firms that excel at using data and analytics to optimize their digital businesses will together generate $1.2 trillion per annum in revenue by 2020. And digital intelligence — the practice of continuously optimizing customer experiences with online and offline data, advanced analytics and prescriptive insights — supports every insights-driven business. Digital intelligence is the antidote to the weaknesses of analytically immature platforms, leaving the world of siloed reporting behind and maturing towards actionable, predictive marketing. Here are a couple of items to consider:
- Today's device-crazed consumers flirt with brands across a variety of interactions during a customer life cycle. However, most organizations seem to focus on website activity in one bucket, mobile in another, and social in . . . you see where I'm going. Strategic plans often fall short in applying digital intelligence across all channels — including offline interactions like customer support or product development.
- Powerful digital intelligence uses timely delivery of prescriptive insights to positively influence customer experiences. This requires integration of data, analytics and the systems that interact with the consumer. Yet many teams manually apply analytics and deliver analysis via endless reports and dashboards that look retroactively at past behavior — begging business leaders to question the true value and potential impact of digital analysis.
As consumer behavioral needs and preferences shifts over time, the proportion of digital to non-digital interactions is growing. With the recent release of Customer Intelligence 360, SAS has carefully considered feedback from our customers (and industry analysts) to create technology that supports a modern digital intelligence strategy in guiding an organization to:
- Enrich your first-party customer data with user level data from web and mobile channels. It's time to graduate from aggregating data for reporting purposes to the collection and retention of granular, customer-level data. It is individual-level data that drives advanced segmentation and continuous optimization of customer interactions through personalization, targeting and recommendations.
- Keep up with customers through machine learning, data science and advanced analytics. The increasing pace of digital customer interactions requires analytical maturity to optimize marketing and experiences. By enriching first-party customer data with infusions of web and mobile behavior, and more importantly, in the analysis-ready format for sophisticated analytics, 360 Discover invites analysts to use their favorite analytic tool and tear down the limitations of traditional web analytics.
- Automate targeting, channel orchestration and personalization. Brands struggle with too few resources to support the manual design and data-driven design of customer experiences. Connecting first-party data that encompasses both offline and online attributes with actionable propensity scores and algorithmically-defined segments through digital channel interactions is the agenda. If that sounds mythical, check out a video example of how SAS brings this to life.
The question now is - are you ready? Learn more here of why we are so excited about enabling digital intelligence for our customers, and how this benefits testing, targeting, and optimization of customer experiences.
Digital intelligence for optimizing customer engagement was published on Customer Intelligence.
In April, SAS 360 Discover was introduced at SAS Global Forum 2016. Since my career started at SAS over five years ago, I have been anticipating this important announcement. In my opinion, this is a major breakthrough for the space of digital intelligence.
In my first year working at SAS, I learned of research and development to address industry needs for digital marketers. Although technologies from Google, Adobe and others address web analytics with measurement reporting, there was a shortcoming.
Historically, web analytics has always had a huge data challenge to cope with since its inception. And when the use case for analysts is to run summary reports, clickstream data is normalized:
It nicely organizes raw clickstream into small, relevant data for reporting. However, this approach presents challenges when performing customer-centric analysis. Why? Holistic customer analysis requires the collection and normalization of digital data at an individual level. This is one of the most important value props of SAS 360 Discover.
Multi-source data stitching and predictive analytics require a data collection methodology that summarizes clickstream:
The data is prepared to contextualize all click activity across a customer's digital journey in one table row, including a primary key to map to all visits across browsers and devices. The data table view shifts from being tall and thin to short and wide. The beauty of this is it enables sophisticated analysis to prioritize what is important, and what isn't. This concept of data collection and management is considered a best practice for advanced customer analytics.
How many marketers do you know who wake up in the morning and claim they can't wait to hear about how analysts are spending 80 percent of their time preparing raw web behavior data, rather than focusing on analysis and actionable insights? None, you say? Exactly! Wouldn't you rather hear your marketing analysts spend their time doing this?
I have always appreciated SAS for what it can do with structured, semi-structured, and unstructured information, but there has always been one dependency – where do I point SAS to obtain the originating data? SAS 360 Discover eliminates this requirement, and provides data collection mechanisms for your brand's website(s) and mobile apps.
In addition, the raw semi-structured data streams SAS natively collects are run through a pre-built relational data model using SAS Data Management for various forms of contextualization that stretch far beyond traditional web analytic use cases.
The output of this data model schema summarizes all digital visitor behavior at this level of detail:
- Anonymous visitors.
- Sessions (or visits).
- Interactions (or clicks/hits).
The data model schema will allow for additional configurations and introduction of other digital data sources to accommodate your organization's evolving needs. More importantly, the benefits of the output are profound, and listed below is a summary of SAS 360 Discover benefits:
- Digital data normalization to support online and offline data stitching of customers.
- When offline data is residing in your organization's data warehouse, information is available at the customer level (not a click or hit level). That's a problem when you want to link it with web or app data. The amount of time analysts spend reshaping raw HIT extracts from their web analytics solution is astonishing, and quite difficult. Customer analysis requires online/offline data stitching, and overcoming this obstacle was a problem SAS set out to solve.
- Measurement reporting and visualization of customers and segments.
- The reporting remains critical as an entry stage for analytics. SAS believes there should be no limit to how many reports and dashboards can be produced to meet business objectives. In other words, unlimited ad hoc reports using SAS Visual Analytics, which is the analysis tool that is packaged with SAS 360 Discover.
- Predictive analytics, machine learning, and data science of customers and anonymous traffic.
- Fueling the SAS customer decision hub
- Brands gain a competitive edge if they stop perceiving customer engagement as a series of discrete interactions and instead see it as customers do: a set of interrelated interactions that, when combined, make up the customer experience. By folding in all known customer level information into a common hub, SAS can analyze, score and take intelligent, contextual actions across channels.
The path to digital intelligence from traditional web analytics covers the diversity of data, advanced analytic techniques, and injection of prescriptive insights to support decision-making and marketing orchestration. Digital intelligence is a transformation — making it a competitive differentiator. It aims to convert brands to become:
- Customer-centric rather than channel-centric
- Focused on enterprise goals as opposed to departmental
- Enabled for audience activation and optimization
- Analytical workhorses
I suspect you would love to see demonstrations of the data that SAS 360 Discover collects from websites and mobile apps in action:
In addition, here is the on-demand video of the SAS Global Forum 2016 keynote presentation of SAS Customer Intelligence 360.
As a marketing analyst at heart, it is extremely gratifying to share my excitement for SAS 360 Discover. The time for predictive customer marketing in the digital ecosystem is here, and the 800-pound gorilla in advanced analytics has just unleashed your new secret weapon.