Data Mining

3月 052020
 

Have you heard that SAS offers a collection of new, high-performance CAS procedures that are compatible with a multi-threaded approach? The free e-book Exploring SAS® Viya®: Data Mining and Machine Learning is a great resource to learn more about these procedures and the features of SAS® Visual Data Mining and Machine Learning. Download it today and keep reading for an excerpt from this free e-book!

In SAS Studio, you can access tasks that help automate your programming so that you do not have to manually write your code. However, there are three options for manually writing your programs in SAS® Viya®:

  1. SAS Studio provides a SAS programming environment for developing and submitting programs to the server.
  2. Batch submission is also still an option.
  3. Open-source languages such as Python, Lua, and Java can submit code to the CAS server.

In this blog post, you will learn the syntax for two of the new, advanced data mining and machine learning procedures: PROC TEXTMINE and PROCTMSCORE.

Overview

The TEXTMINE and TMSCORE procedures integrate the functionalities from both natural language processing and statistical analysis to provide essential functionalities for text mining. The procedures support essential natural language processing (NLP) features such as tokenizing, stemming, part-of-speech tagging, entity recognition, customized stop list, and so on. They also support dimensionality reduction and topic discovery through Singular Value Decomposition.

In this example, you will learn about some of the essential functionalities of PROC TEXTMINE and PROC TMSCORE by using a text data set containing 1,830 Amazon reviews of electronic gaming systems. The data set is named Amazon. You can find similar data sets of Amazon reviews at http://jmcauley.ucsd.edu/data/amazon/.

PROC TEXTMINE

The Amazon data set has already been loaded into CAS. The review content is stored in the variable ReviewBody, and we generate a unique review ID for each review. In the proc call shown in Program 1 we ask PROC TEXTMINE to do three tasks:

  1. parse the documents in table reviews and generate the term by document matrix
  2. perform dimensionality reduction via Singular Value Decomposition
  3. perform topic discovery based on Singular Value Decomposition results

Program 1: PROC TEXTMINE

data mycaslib.amazon;
    set mylib.amazon;
run;

data mycaslib.engstop;
    set mylib.engstop;
run;

proc textmine data=mycaslib.amazon;
    doc_id id;
    var reviewbody;

 /*(1)*/  parse reducef=2 entities=std stoplist=mycaslib.engstop 
          outterms=mycaslib.terms outparent=mycaslib.parent
          outconfig=mycaslib.config;

 /*(2)*/  svd k=10 svdu=mycaslib.svdu outdocpro=mycaslib.docpro
          outtopics=mycaslib.topics;

run;

(1) The first task (parsing) is specified in the PARSE statement. Parameter “reducef” specifies the minimum number of times a term needs to appear in the text to be included in the analysis. Parameter “stop” specifies a list of terms to be excluded from the analysis, such as “the”, “this”, and “that”. Outparent is the output table that stores the term by document matrix, and Outterms is the output table that stores the information of terms that are included in the term by document matrix. Outconfig is the output table that stores configuration information for future scoring.

(2) Tasks 2 and 3 (dimensionality reduction and topic discovery) are specified in the SVD statement. Parameter K specifies the desired number of dimensions and number of topics. Parameter SVDU is the output table that stores the U matrix from SVD calculations, which is needed in future scoring. Parameter OutDocPro is the output table that stores the new matrix with reduced dimensions. Parameter OutTopics specifies the output table that stores the topics discovered.

Click the Run shortcut button or press F3 to run Program 1. The terms table shown in Output 1 stores the tagging, stemming, and entity recognition results. It also stores the number of times each term appears in the text data.

Output 1: Results from Program 1

PROC TMSCORE

PROC TEXTMINE is used with large training data sets. When you have new documents coming in, you do not need to re-run all the parsing and SVD computations with PROC TEXTMINE. Instead, you can use PROC TMSCORE to score new text data. The scoring procedure parses the new document(s) and projects the text data into the same dimensions using the SVD weights derived from the original training data.

In order to use PROC TMSCORE to generate results consistent with PROC TEXTMINE, you need to provide the following tables generated by PROC TEXTMINE:

  • SVDU table – provides the required information for projection into the same dimensions.
  • Config table – provides parameter values for parsing.
  • Terms table – provides the terms that should be included in the analysis.

Program 2 shows an example of TMSCORE. It uses the same input data layout used for PROC TEXTMINE code, so it will generate the same docpro and parent output tables, as shown in Output 2.

Program 2: PROC TMSCORE

Proc tmscore data=mycaslib.amazon svdu=mycaslib.svdu
        config=mycaslib.config terms=mycaslib.terms
        svddocpro=mycaslib.score_docpro outparent=mycaslib.score_parent;
    var reviewbody;
    doc_id id;
run;

 

Output 2: Results from Program 2

To learn more about advanced data mining and machine learning procedures available in SAS Viya, including PROC FACTMAC, PROC TEXTMINE, and PROC NETWORK, you can download the free e-book, Exploring SAS® Viya®: Data Mining and Machine Learning. Exploring SAS® Viya® is a series of e-books that are based on content from SAS® Viya® Enablement, a free course available from SAS Education. You can follow along with examples in real time by watching the videos.

 

Learn about new data mining and machine learning procedures in SAS Viya was published on SAS Users.

1月 132020
 

Are you ready to get a jump start on the new year? If you’ve been wanting to brush up your SAS skills or learn something new, there’s no time like a new decade to start! SAS Press is releasing several new books in the upcoming months to help you stay on top of the latest trends and updates. Whether you are a beginner who is just starting to learn SAS or a seasoned professional, we have plenty of content to keep you at the top of your game.

Here is a sneak peek at what’s coming next from SAS Press.
 
 

For students and beginners

For beginners, we have Exercises and Projects for The Little SAS® Book: A Primer, Sixth Edition, the best-selling workbook companion to The Little SAS Book by Rebecca Ottesen, Lora Delwiche, and Susan Slaughter. Exercises and Projects for The Little SAS® Book, Sixth Edition will be updated to match the updates to the new The Little SAS® Book: A Primer, Sixth Edition. This hands-on workbook is designed to hone your SAS skills whether you are a student or a professional.

 

 

For data explorers of all levels

This free e-book explores the features of SAS® Visual Data Mining and Machine Learning, powered by SAS® Viya®. Users of all skill levels can visually explore data on their own while drawing on powerful in-memory technologies for faster analytic computations and discoveries. You can manually program with custom code or use the features in SAS® Studio, Model Studio, and SAS® Visual Analytics to automate your data manipulation and modeling. These programs offer a flexible, easy-to-use, self-service environment that can scale on an enterprise-wide level. This book introduces some of the many features of SAS Visual Data Mining and Machine Learning including: programming in the Python interface; new, advanced data mining and machine learning procedures; pipeline building in Model Studio, and model building and comparison in SAS® Visual Analytics

 

 

For health care data analytics professionals

If you work with real world health care data, you know that it is common and growing in use from sources like observational studies, pragmatic trials, patient registries, and databases. Real World Health Care Data Analysis: Causal Methods and Implementation in SAS® by Doug Faries et al. brings together best practices for causal-based comparative effectiveness analyses based on real world data in a single location. Example SAS code is provided to make the analyses relatively easy and efficient. The book also presents several emerging topics of interest, including algorithms for personalized medicine, methods that address the complexities of time varying confounding, extensions of propensity scoring to comparisons between more than two interventions, sensitivity analyses for unmeasured confounding, and implementation of model averaging.

 

For those at the cutting edge

Are you ready to take your understanding of IoT to the next level? Intelligence at the Edge: Using SAS® with the Internet of Things edited by Michael Harvey begins with a brief description of the Internet of Things, how it has evolved over time, and the importance of SAS’s role in the IoT space. The book will continue with a collection of chapters showcasing SAS’s expertise in IoT analytics. Topics include Using SAS Event Stream Processing to process real world events, connectivity, using the ESP Geofence window, applying analytics to streaming data, using SAS Event Stream Processing in a typical IoT reference architecture, the role of SAS Event Stream Manager in managing ESP deployments in an IoT ecosystem, how to use deep learning with Your IoT Digital, accounting for data quality variability in streaming GPS data for location-based analytics, and more!

 

 

 

Keep an eye out for these titles releasing in the next two months! We hope this list will help in your search for a SAS book that will get you to the next step in updating your SAS skills. To learn more about SAS Press, check out our up-and-coming titles, and to receive exclusive discounts make sure to subscribe to our newsletter.

Foresight is 2020! New books to take your skills to the next level was published on SAS Users.

10月 162019
 

Introduction

Generating a word cloud (also known as a tag cloud) is a good way to mine internet text. Word (or tag) clouds visually represent the occurrence of keywords found in internet data such as Twitter feeds. In the visual representation, the importance of each keyword is denoted by the font size or font color.

You can easily generate Word clouds by using the Python language. Now that Python has been integrated into the SAS® System (via the SASPy package), you can take advantage of the capabilities of both languages. That is, you create the word cloud with Python. Then you can use SAS to analyze the data and create reports. You must have SAS® 9.4 and Python 3 or later in order to connect to SAS from Python with SASPy. Developed by SAS, SASPy a Python package that contains methods that enable you to connect to SAS from Python and to generate analysis in SAS.

Configuring SASPy

The first step is to configure SASPy. To do so, see the instructions in the SASPy Installation and configuration document. For additional details, see also the SASPy Getting started document and the API Reference document.

Generating a word cloud with Python

The example discussed in this blog uses Python to generate a word cloud by reading an open table from the data.world website that is stored as a CSV file. This file is from a simple Twitter analysis job where contributors commented via tweets as to how they feel about self-driving cars. (For this example, we're using data that are already scored for sentiment. SAS does offer text analytics tools that can score text for sentiment too -- see this example about rating conference presentations.) The sentiments were classified as very positive, slightly positive, neutral, slightly negative, very negative, and not relevant. (In the frequency results that are shown later, these sentiments are specified, respectively, as 1, 2, 3, 4, 5, and not_relevant.) This information is important to automakers as they begin to -design more self-driving vehicles and as transportation companies such as Uber and Lyft are already adding self- driving cars to the road. Along with understanding the sentiments that people expressed, we are also interested in exactly what is contributors said. The word cloud gives you a quick visual representation of both. If you do not have the wordcloud package installed, you need to do that by submitting the following command:

pip install wordcloud

After you install the wordcloud package, you can obtain a list of required and optional parameters by submitting this command:

?wordcloud

Then, follow these steps:

  1. First, you import the packages that you need in Python that enable you to import the CSV file and to create and save the word-cloud image, as shown below.

  2. Create a Python Pandas dataframe from the twitter sentiment data that is stored as CSV data in the data file. (The data in this example is a cleaned-up subset of the original CSV file on the data.world website.)

  3. Use the following code, containing the HEAD() method, to display the first five records of the Sentiment and Text columns. This step enables you to verify that the data was imported correctly.

  4. Create a variable that holds all of the text in a single row of data that can be used in the generation of the word cloud.

  5. Generate the word cloud from the TEXTVAR variable that you create in step 4. Include any parameters that you want. For example, you might want to change the background color from black to white (as shown below) to enable you to see the values better. This step includes the STOPWORDS= parameter, which enables you to supply a list of words that you want to eliminate. If you do not specify a list of words, the parameter uses the built-in default list.

  6. Create the word-cloud image and modify it, as necessary.

Analyzing the data with SAS®

After you create the word cloud, you can further analyze the data in Python. However, you can actually connect to SAS from Python (using the SASPy API package), which enables you to take advantage of SAS software's powerful analytics and reporting capabilities. To see a list of all available APIs, see the API Reference.

The following steps explain how to use SASPy to connect to SAS.

  1. Import the SASPy package (API) . Then create and generate a SAS session, as shown below. The code below creates a SAS session object.

  2. Create a SAS data set from the Python dataframe by using the DATAFRAME2SASDATA method. In the code below, that method is shown as the alias DF2DS.

  3. Use the SUBMIT() method to include SAS code that analyzes the data with the FREQ procedure. The code also uses the GSLIDE procedure to add the word cloud to an Adobe PDF file.

    When you submit the code, SAS generates the PDF file that contains the word-cloud image and a frequency analysis, as shown in the following output:

Summary

As you can see from the responses in the word cloud, it seems that the contributors are quite familiar with Google driverless cars. Some contributors are also familiar with the work that Audi has done in this area. However, you can see that after further analysis (based on a subset of the data), most users are still unsure about this technology. That is, 74 percent of the users responded with a sentiment frequency of 3, which indicates a neutral view about driverless cars. This information should alert automakers that more education and marketing is required before they can bring self-driving cars to market. This analysis should also signal companies such as Uber Technologies Inc. and Lyft, Inc. that perhaps consumers need more information in order to feel secure with such technology.

Creating a word cloud using Python and SAS® software was published on SAS Users.

3月 232018
 

We all have different learning styles. Some learn best by seeing and doing; others by listening to lectures in a traditional classroom; still others simply by diving in and asking questions along the way. Traditional face-to-face classroom instruction, real-time classes over the Internet, or self-paced instruction with exercises, SAS Education [...]

The post SAS introduces the blended classroom appeared first on SAS Learning Post.

2月 012018
 

Groups and organizations lauding artificially intelligent solutions are popping up everywhere with promises to create the next battlefield advantage using next generation weapons, gear, or satellites.  The term artificial intelligence (AI) splashes the headlines with promises that we're moments away from revolutionizing the battlefield. As a former intelligence analyst, I [...]

13 ideas for AI in military intelligence was published on SAS Voices by Mary Beth Ainsworth

5月 202016
 

If you're looking for higher pay and better opportunities, what career skills should you seek to acquire? You might think leadership or communication skills would top the list, but a recent study says otherwise. According to a massive study from MONEY and Payscale.com, SAS Analytics skills are the most valuable skills to […]

New study confirms: SAS most valuable career skill was published on SAS Voices.

4月 152016
 

As promised a couple of weeks ago, I am very happy to share Part 2 of a webcast series highlighting how SAS participates in the space of digital analytics for data-driven marketing with applications for personalization and attribution. Before launching the video, let me set some context for what you are about to see.

Why do we care about the intersection of digital analytics and personalization? Honestly, it is increasingly important to predict how customers will behave so you can personalize experiences with relevance. The deeper your understanding of customer behavior and lifestyle preferences, the more impactful personalization can be. However, digital personalization at the individual level remains elusive for most enterprises who face challenges in data management, analytics, measurement, and execution. As customer interactions spread across fragmented touch points and consumers demand seamless and relevant experiences, content-oriented marketers have been forced to re-evaluate their strategies for engagement. But the complexity, pace and volume of modern marketing easily overwhelms traditional planning and design approaches that rely on historical conventions, myopic single-channel perspectives and sequential act-and-learn iteration.

The majority of technologies in use today for digital personalization have generally failed to effectively use predictive analytics to offer customers a contextualized digital experience. Most are based on simple rules-based recommendations, segmentation and targeting that are usually limited to a single customer touch point. Predictive MarketingDespite some use of predictive techniques, digital experience delivery platforms are behind in incorporating predictive analytics to contextualize experiences using 1st-, 2nd- and 3rd-party customer data. In my opinion, I believe the usage of digital data mining and predictive analytics to prioritize and inform the marketing teams on what to test, and to analytically define segment audiences prior to assigning test cells, is a massive opportunity. Marketers are very creative, and can imagine hundreds of different testing ideas – which tests do we prioritize if we cannot run them all? This is where advanced analytics can help inform our strategies in support of content optimization, as it allows the data to prioritize our strategy, and help us focus on what is important.

Moving on to our second subject of interest, we transition to the wonderful world of marketing attribution. At the very core of this topic, modern marketers recognize that customers expect brands to deliver relevant conversations across all channels at any given moment. The challenge is to uncover the interactions that drive conversions through integrated measurement and insights. However, organizations struggle to employ a holistic measurement approach because:

  1. It's confusing to distinguish among the measurement approaches available.
  2. Marketers bombard customers with extraneous content.
  3. Today's misaligned data makes customer level measurement a very difficult task.

It seems like attribution has been a problem for marketers for a very long time. According to a popular quote by Avinash Kaushik of Google:

“There are few things more complicated in analytics (all analytics, big data and huge data!) than multichannel attribution modeling."

The question is: Why is it challenging? SAS strongly believes three years later that we are living in a game-changing moment within digital analytics. Marketers are being enabled with approachable and self-service analytic capabilities, and this trend directly impacts our ability to improve our approaches to problems like attribution analysis. However, rules-based methods of attribution channel weighting continue to be far more popular in the industry to date, which contradicts the recent analytic approachability trend. The time has arrived for algorithmic attribution . . . Attribution

 

Did I whet your appetite? I hope so...please enjoy episode two of our two-part webcast series, now available for on demand viewing:

 

SAS for Digital Analytics: Personalization and Attribution [Part 2]

 

SAS Customer Intelligence offers a one-stop modern marketing platform to comprehensively support the objectives of predictive personalization and algorithmic attribution - from digital data collection, management, predictive analytics, omnichannel journey orchestration, delivery across online and offline channels, and measurement. On April 19 at SAS Global Forum 2016, SAS Customer Intelligence 360 will make its debut, and subjects like digital intelligence and predictive personalization will be primary topics. This new offering will drive unprecedented innovation in customer analytics and data-driven marketing, putting predictive analytical intelligence directly in the hands of digital and integrated marketers responsible for the customer experience.

If you enjoyed this article, be sure to check out my other work here. Lastly, if you would like to connect on social media, link with me on Twitter or LinkedIn.

tags: Advanced Analytics, customer intelligence, Data Mining, data science, Digital Analytics, Digital Attribution, Digital Intelligence, digital marketing, Digital Personalization, marketing analytics, Predictive Marketing, Predictive Personalization, segmentation

Introduction to SAS for digital personalization and attribution was published on Customer Intelligence.

3月 312016
 

Digital analytics primarily supports functions of customer and prospect marketing. When it comes to the goals of digital analysis, it literally mirrors the mission of modern marketing. But what exactly is today's version of marketing all about?Modern Marketing

Honestly, we've been talking about this for years. And years. We ALL know it's what we should be doing and conceptually it's very simple, but practically, it has been very hard to achieve. Why?

Even with great web analytics, there have always been critical missing insights, which meant we didn't know for certain what the next-best-interaction for each customer was at any point in time. In addition, the development of insights and the use of analytics to define high-propensity audience segments has been distinctly slow and batch-driven in nature, delaying relevant delivery of targeted interactions. So we may get the message right, but we probably don't deliver it in a timely, consistent way, which has a dramatic impact on customer responsiveness and marketing effectiveness.

So in today's connected, always-on, highly opinionated world, we need to be a little sharper in meeting our customer's basic expectations, never mind surprising, delighting, and impressing them. While the concept of customer-centricity continues to increase in importance, improving our analytical approach to support this premise is vital.

SAS recognizes today's modern marketing challenges with digital and customer analytics. It is our mission to enable marketers to benefit from approachable and actionable advanced analytics to make more powerful decisions within today’s complex and interconnected business environments. That sounds great, right? I sense some of you reading this are raising an eyebrow of suspicion at this very moment.

Practically speaking, we want to show you exactly what that means. On March 29th, 2016, we aired episode one of a two-part webcast series, and it is now available for on demand viewing:

SAS for Digital Analytics: Introduction & Advancing Segmentation [Part 1]

We genuinely hope the webcast provided a proper introduction to how SAS participates in the space of digital analytics for data-driven marketing, and please come back in a couple of weeks when we will post Part 2 in this series entitled: SAS for Digital Analytics: Personalization & Attribution [Part 2]

If you enjoyed this article, be sure to check out my other work here. Lastly, if you would like to connect on social media, link with me on Twitter or LinkedIn.

tags: Advanced Analytics, customer analytics, customer intelligence, data integration, data management, Data Mining, data science, Digital Analytics, Digital Intelligence, digital marketing, Integrated Marketing, marketing analytics, predictive analytics, Predictive Marketing, segmentation, web analytics, webcast

Introduction to SAS for digital analytics and segmentation was published on Customer Intelligence.

3月 232016
 

The business opportunity to intelligently manage customer journeys across their lifecycle with your brand has never been greater, but so is the danger of not meeting their expectations and losing out to savvier competitors. In my opinion, the current state of most digital analytic practices continue to be siloed, tactical, and narrowly fixated on channel-obsessed dashboard reporting. That might come across as presumptuous, but keep this in mind - customer-centricity is a hot topic at the C-Suite level, and your CMO has  stated (or will very soon) that your organization is transforming into a personalization super force that will be marketing to the segment of one. If that is the case, the category of digital analytics has got to step up its game!

The antidote is digital intelligence which represents a strategic shift in approach to marketing analysis that uses insights from traditional and modern channels (we're talking online AND offline) to enable actionable, customer-obsessed analytical brilliance.

The era of the empowered customer is unraveling itself — trends in which consumers, not brands, own influence, backed by the rapid rise of digital. I strongly believe that no matter how important a company's products or services are with my life, the majority of brands I do business with continue to perform channel-centric analysis, and remain unaware of the different interactions I have with them across ALL channels. I don't care about your email or search marketing KPIs. What I care about is how you treat Suneel, no matter what device, channel, or platform I select to interact with you on.

Meanwhile, digital marketing spend continues to grow at a tenacious pace, cementing the importance of digital channels in managing the customer journey. Digital marketing is effective in all phases of the customer life cycle, ranging from acquisition, upsell/cross-sell, retention, and winback, proven by the ongoing shift of wallet share to online channels. While these are exciting times for omnichannel marketers, these more holistic approaches bring challenges. In today's fragmented digital landscape, long-established methods focused on web analytics and aggregated customer views are ill-equipped to keep pace with:

Digital interaction bread crumb trails

Customers (and prospects) interact with brands across an array of online channels and devices, creating new paths to generate incremental value associated with marketing-centric KPIs. However, customers expect personalized relevance in moments of truth, raising the bar for analytics and marketing execution. A brand's digital presence is much more than a website, such as social media, mobile applications, and wearable technologies. Conventional web analytics only track onsite behavior and lack the ability to comprehend tech-savvy customers in 2016.

The collapse of the digital silo

Brands typically construct offline and online interaction channels confined from one another, so let's reflect on that for a moment. Isn't it time we recognize that customer data is customer data, regardless of where the ingredients are collected? To deliver comprehensive customer insights, brands seek to merge digital and offline data sources together. Digital & customer analytics teams are attempting to work together, but their projects struggle due to a clash of approaches & culture. Some of the main drivers are:

  1. Data — Customers leave trails of information for marketers to chew on, and are available in structured, semistructured, and unstructured formats. There's no excuse anymore for brands to not be able to work with all three. Approachable technology exists to integrate multiple sources of online and offline customer data in meaningful ways to analyze and take action on.
  2. Skills — Have you ever sat in a meeting with data scientists and web analytic ninjas? It's like they speak two different languages, and communication between these two segments is critical for an organization to innovate in its commitment to customer analytics.
  3. Analysis — There is a reason why there is so much discussion around the application of advanced analytics. In many ways, digital marketing is ripe for analytical maturity, ranging across segmentation, attribution, and personalization. The discipline has proven its value to help differentiate a brand from its competition. When are the days of Data-Scientist“good enough” analytics going to end? Let's keep the science in data science, and stop succumbing to the false hype that sophisticated predictive marketing can be accomplished through black box, easy-button solutions.

Dynamic interaction management

Brands seek to react intelligently to shifts in consumer behavior in milliseconds, which makes the intersection of predictive analytics and data-driven marketing vital for orchestrating the customer journey. To reach your target audience in opportunistic micro-moments, the requirement of real-time actionable analytics with direct connections to personalization and marketing automation systems is the queen bee. The sole dependence on isolated, retrospective reports and dashboards of aging web analytic solutions has serious limitations in modern marketing.

Given the investment and revenue at stake for most brands, it is increasingly important to champion support of the development and continuous optimization of digital channels. Simply put, analytical sophistication lives at the center of that process. Yet most organizations continue to approach digital analytics focused on discerning traffic sources and aggregated website user behaviors. Given the intricate complications and aspirational promise of digital marketing, brands should consider modernizing and maturing their approaches to customer analytics because:

  • CX matters: Customers don't care about the challenges related to identity management across multiple visits (or sessions), browsers, channels, and devices. Does your web analytic platform support your team's abilities to recognize and track customers, not clicks or hits, across the fragmentation of touch points? With careful consideration towards the areas of data management, data integration, and data quality, analyzing customer-centric (or visitor-centric) digital activity on their journeys to making (or not making) a purchase with your brand is absolutely feasible.
  • "Good enough" analytics must end: Digital analytic teams must graduate from machine gunning their organizations with traffic-based reports that summarize the past to producing predictive insights that marketers can interpret, and take action with. I'm always impressed by web analytic teams that produce an array of historical reports with beautiful visualizations, segmenting and slicing away at their tsunami of clickstream data. However, how much impact and relevance to the business can this approach have? Customer-centricity demands that we re-engineer our thinking, and make the shift from reactive to predictive marketing analytics.
  • There's nothing exciting about siloed channel analysis: To deliver the elusive and mythical 360 degree view of customer insights, it turns out you don't need magical wizards like Gandalf or Albus Dumbledore by your side. Have you ever wondered why web analytic software doesn't allow you to perform data stitching with offline data sources? How about data mining and predictive analytic capabilities? Well, it boils down to how digital data is collected, aggregated, and prepared for downstream use cases.

Web analytics has always had a BIG data challenge to cope with since it's inception in the mid 1990's, and when the use case for analysts is to run historical summary reports and visual dashboards, clickstream data is collected and normalized in a structured format as shown in this schematic:

Data Aggregation for Web Analytics

This format does a very nice job of organizing clickstream data in such a way that we go from big data to small, more relevant data for reporting. However, this approach presents challenges when performing customer-centric analysis which requires data stitching across online and offline data sources. Why you ask? Because you cannot de-aggregate data that was designed for channel and campaign performance summarizations. Holistic customer analysis, from a digital viewpoint, requires the collection and normalization of granular, detailed data at an individual level. Can it be done? Of course it can.

Multi-source data stitching, data mining and predictive analytics require a specific digital data collection methodology that summarizes clickstream data to look like this:

Data Aggregation for Advanced Analytics

Ultimately, the data is collected and prepared to contextually summarize all click activity across a customer's digital journey in one table row, including a primary customer key to map to all visits across channels and devices. The data table view shifts from being tall and thin, to short and wide. The more attributes or predictors an analyst adds, the wider the table gets. The beauty of this approach is it allows marketers and analysts to be curious, add more data sources, and allow algorithmic analysis to prioritize what is important, and what isn't. This concept is considered a best practice for advanced customer analytics.

  • Beware of blind spots: As time passes, customers in every industry are progressively sharing more data about themselves through existing and emerging digital outlets, such as mobile applications, wearables, and other connected technology. The opportunity to ingest and analyze these new sources should excite any marketer who claims to be data-driven. However, does your web analytics platform allow you to analyze these new digital touchpoints? A brand's ability to absorb, integrate, analyze, and derive marketable insights from emerging data sources is key in this new paradigm to avoid being blindsided by customers and the competition.

The path to digital intelligence from traditional web analytics needs to cover the diversity of data, advanced analytic techniques, and injection of prescriptive insights to support decision-making and marketing orchestration. Digital intelligence is a transformation for web analytic teams — making it a competitive differentiator if executed well. It aims to transform brands to become:

  1. Customer-centric rather than channel-centric: As customers and prospects weave across an ocean of marketing channels and connected devices, digital intelligence supports the integrated analysis of interactions in concert, rather than with disconnected channel views. In addition to visibility across all channels, analysis is highly granular to identify, track, and prioritize next-best-actions for individuals. In other words, hyper-personalization to the segment of one!
  2. Focused on enterprise goals as opposed to departmental: To enable omnichannel analytics, digital intelligence is highly dependent on customer data management capabilities across all data types – structured, semistructured, and unstructured. This includes fusing interaction and behavioral data across all digital channels with first-party offline customer data, as well as second- and third-party data (if available). This enriched potpourri of data must be prepared to feed the analytical ninjas that sit within the marketing organization, line of business or centralized customer intelligence team, because it is their job to exploit this stream of information and generate insights for the organization as a whole.
  3. Enabled for audience activation and optimization. The mission of digital intelligence is the direct application of analytics to generate data-driven evidence that helps business stakeholders make clearer decisions. The potential of data mining exponentially increases with richer customer data to support segmentation, personalization, optimization, and targeting - in other words, connecting data and analytics to the delivery of relevant content, offers, and awesome experiences.
  4. Analytical workhorses: The incredibly fast-moving world of digital interactions and campaigns mean that marketers desperately need quicker analysis. Waiting days or weeks for reports and research equates to failure. Digital intelligence delivers efficiency at a pace that more nearly matches users' decision-making schedules.

SAS Customer Intelligence offers a one-stop modern marketing platform to comprehensively support the mission of digital intelligence - from digital data collection, management, predictive analytics, and marketing delivery across online and offline channels. On April 19 at SAS Global Forum 2016, SAS Customer Intelligence 360 will make its debut, and digital intelligence will be a primary topic. This new offering will drive unprecedented innovation in customer analytics, putting predictive analytical intelligence directly in the hands of digital marketers, business analysts, and data scientists. In the last few months, industry analysts have previewed and validated our abilities in advanced and customer analytics.

We are very excited for the future and potential of digital intelligence. The question is...

Are you excited?

 

If you enjoyed this article, be sure to check out my other work here. Lastly, if you would like to connect on social media, link with me on Twitter or LinkedIn.

tags: customer intelligence, Data Mining, data science, Digital Analytics, Digital Intelligence, marketing analytics, personalization, predictive analytics, Predictive Marketing, segment of one, web analytics

Web analytics vs. digital intelligence - what's the difference? was published on Customer Intelligence.

1月 262016
 

I begin this blog post with one goal in mind. I want to raise awareness on the subject of customer and marketing analytics, and why this field is exploding in interest and popularity. Let's begin with a primer for the uninitiated, and lay down some definitions:

Customer Analytics: The processes, technologies, and enablement that give brands the customer insight necessary to provide offers that are anticipated, relevant and timely.

Marketing Analytics: The processes and technologies that enable brands to assess the success of their marketing initiatives by evaluating performance using important business metrics, such as ROI, channel attribution, and overall marketing effectiveness.

If you aren't a fan of textbook definitions, here is a creative alternative:

Still not on board? Here's my perspective on the subject:

Customers are more empowered and connected than ever before, with access to information anywhere, any time – where to shop, what to buy and how much to pay. Brands realize it is increasingly important to predict how customers will behave to respond accordingly. Simply put, the deeper your understanding of customer buying habits and lifestyle preferences, the more accurate your predictions of future buying behaviors will be.

Marketers need to be enabled to benefit from approachable and actionable advanced analytics to make more powerful decisions within today’s complex and interconnected business environments.  In my mind, the big picture boils down to one, two or three core enablers, based on your organization's goals and preferences:

Image 2

 

 

 

 

Marketing analysts tasked with making sense of customer data, big or small, have to migrate through a complex maze of myths and realities about technology platforms, advanced analytics solutions and, most importantly, the magnitude of customer analytics efforts. On the surface, it appears that customer analytics is a well-entrenched discipline in many organizations, but under the hood, old problems persist around data integration and data quality while new ones emerge around the real-time application of insights and the ability to rein in digital data for customer-based analysis.When I speak with clients, there are two key themes that I continually hear:

  1. Data is a big challenge. As customer interactions with brands increase and diversify, brands need to integrate data effectively in order to provide the contextual and real-time insights their customers are growing to expect. Haven't you grown tired of saying we spend 80 percent of our time on data management related tasks, and 20 percent on analysis?
  2. Analytic talent is hard to find. Brands struggle to find individuals with the right analytic skills to meet the challenges they are facing today. Without the talent to unlock actionable insights, modern customer analytics cannot meet its potential. (Given my public affiliation with The George Washington University's M.S. in Business Analytics program, I'd recommend checking it out if you are hunting for quality talent.)

To me, these themes point to a workflow entitled the marketing analytics lifecycle:

Image 3

 

 

 

 

 

 

With the growing importance of customer analytics in organizations, the ability to extract insight and embed it back into organizational processes is at the forefront of business transformation. However, this requires considerations for where relevant data resides, the ability to reshape it for downstream analytic tasks (predictive modeling vs. reporting), and how to take action on the derived insights. Furthermore, there are the roles of different people within the organization that need to be considered:

  • Marketing Analyst/Technologist
  • Data Scientist/Statistician
  • Marketing Manager
  • Supporting IT Team

Customer analysis touches all of these roles, and to enable this audience comprehensively, all aspects of the marketing analytics lifecycle must be supported. To directly address this, I want to to highlight what SAS is doing to help our clients meet these challenges.

Marketing Analytics Lifecycle Stage #1: Integrate and Prepare Data

Customer analytics is highly dependent on the quality of the ingredients we feed into analysis. Now, the digital marketing industry has been taken by storm by the emergence of Digital DMPs, like Oracle BlueKai, Neustar, and Krux, who aim to provide marketers support in programmatic ad buying and selling. Marketers and publishers are learning that harnessing their first-party data; developing single and consistent identities for their consumers across devices and systems, like email and site optimization; and gaining access to second-party data are mission critical. However, the subject of data mining and predictive analytics has largely been ignored by the Digital DMP space. Brands who want to exploit the benefits of advanced analytics have additional considerations to support their data management challenges. The following video highlights how SAS helps manage and prepare data of all sizes, from 1st party customer data to clickstream and IoT, specifically for analytics:

 

Some of you might be questioning the value of this, so let me offer a different perspective. Over the past few years, I have developed a personal frustration of attending various marketing conferences and repeatedly observing high-level presentations about the potential of analytics. Even more challenging has been the recent trend of companies presenting magical (i.e., "easy-button") black-box marketing cloud solutions that address every imaginable analytical problem; in my opinion, high-quality advanced analytics has not reached a point of commoditization, and remains a point of competitive differentiation. Do not be mislead by sleight-of-hand magic!

Marketing Analytics Lifecycle Stage #2 & #4: Explore Data, Develop Models, and Deploy

What types of marketing challenges are you attempting to solve with customer analytics? Srividya Sridharan and Brandon Purcell are two leading researchers in the space of customer insights, and recently released a report entitled How Analytics Drives Customer Life-Cycle Management recommending the deployment of various analytical techniques across the customer life cycle to grow existing customer relationships and provide insight into future behavior. Highly recommended reading! Let's review some of the most common problems (or opportunities) we view at SAS with our clients.

Image 4

Within each of the categories, a myriad of analytic techniques can be executed to assist and improve your brand's abilities to address them. The following video is a demonstration of how I used SAS Visual Statistics and Logistic Regression analysis to understand drivers by marketing channel of business conversions on a website or mobile app. The benefit of understanding these data-driven drivers is to influence downstream marketing personalization and acquisition campaigns. In addition, capabilities related to group-by modeling, deployment scoring and model comparison with other algorithmic approaches are highlighted.

 

 

Big digital data, scalable predictive analytics, visualization, approachability, and actionability. Stay thirsty my friends, because it is our clients who are expressing their needs, and SAS is stepping up to meet their challenges!

If you would like to learn more on how we address other marketing and customer analytic problems, please click on any of the following topics:

  1. Personalization
  2. Attribution
  3. Segmentation
  4. Acquisition
  5. Optimization

With that said, we have one final stage of the lifecycle to review.

Marketing Analytics Lifecycle Stage #3: Explain Results and Share Insights

An individual's ability to communicate clearly, succinctly and in the appropriate vernacular when presenting analytical recommendations to a marketing organization is extremely important when focused on driving change with data-driven methods. I recently wrote a blog post on this topic entitled Translating Predictive Marketing Analytics, and if you're tired of reading, here's another video - this time focused on explaining the results of analytical exercises in easy-to-consume business language.

 

As I close this blog post, I want to leave you with a few thoughts. For your brand's customers, technology is transparent, user-enabling, and disintermediating. The journey they embark with you on is fractured and takes place across channels, devices, and points in time. The question becomes – are you prepared for moments of truth as they occur across these channels over time? Customer analytics represents the opportunity to optimize every consumer experience, and revisiting a point I made earlier, the deeper your understanding of customer buying habits and lifestyle preferences, the more accurate your predictions of future buying behaviors will be.

If you enjoyed this article, be sure to check out my other work here. Lastly, if you would like to connect on social media, link with me on Twitter or LinkedIn.

 

tags: Business Analytics, business intelligence, customer analytics, customer intelligence, Customer Lifecycle Management, Data Mining, Digital Intelligence, marketing analytics, Marketing Attribution, personalization, Predictive Marketing, segmentation

The analytics of customer intelligence and why it matters was published on Customer Analytics.