sas intelligent decisioning

6月 292020
 

Companies have recognized that the key to maintaining customer loyalty and increasing engagement is to anticipate customer’s needs and desires. To that end, companies have invested heavily in AI technologies to create recommendation engines that present offers, communications, and products to fulfill those needs. Nowadays, recommendation engines can be found just about everywhere from news websites to e-commerce sites to online streaming services. However, customers frequently encounter recommendations that are inappropriate or repetitive. Just because you accidentally clicked on something 6 months ago, doesn’t mean that you should receive 100 variations of that same article/product today.

The challenge is that companies have often focused too much on building advanced AI algorithms to power their recommendation systems, but frequently miss out on rapid changes in the marketplace. Since recommendation algorithms are based heavily on historical behavior, they may fail under rapidly changing environments where new products are introduced, consumer tastes change rapidly, or market conditions deteriorate.

Recommendation engines also frequently fail to account for real-time events and context. For example, during holidays, people’s tastes can be highly seasonal. Using recommendations based on purchases made during other times of the year may have no relevance to what people want today.

Companies are also under pressure to recommend products and content that are most profitable and high value. Unfortunately, the most profitable products may not be the products most preferred by customers. This can result in conflict between marketers who want to push products and data scientists who want to create good recommendation engines. If the marketers override the recommendations, this can result in consumers losing trust in the recommendations.

To address these limitations, companies need to think about developing an approach to recommendation engines that account for the following factors:

  • Business objectives: Which products are most profitable? How do we optimize the recommendations to generate the highest revenue?
  • Context: What do we know about the customer before we deliver a recommendation? Are they at home/school/work/vacation? Did a significant life event occur such as getting married, having a baby, or buying a house?
  • Historical behavior: Analyzing past transactions to generate recommendations. This involves using AI and machine learning techniques such as collaborative filtering, market basket analysis, factorization machines to look at previous purchase history and compare them to others who purchased similar products.
  • Real-time trends: Using real-time information to address sudden changes in consumer demand. This real-time information can come from social media feeds or by analyzing real-time streaming data.

To address these different aspects of a recommendation engine, I want to walk you through an example of building a recommendation engine that incorporates these different aspects.

Scenario: A cable company would like to develop an app for subscribers that provides real-time recommendations for live TV. To improve the relevancy of the recommendations, they would like to consider several factors. Firstly, they would like to better predict if a family or child is watching at that moment and make age-appropriate recommendations. Secondly, if a show or content is extremely popular right now with other viewers (such as a breaking news event or a sports event in their area), they would like to override the default recommendation with the show that is extremely popular. This can be accomplished in 5 steps:

1. Use factorization machines to analyze historic viewing behavior and come up with personalized recommendations

Factorization machines are one of the most powerful recommendation algorithms currently available. It uses matrix factorization to project ratings on a very sparse matrix of users and products. SAS Viya provides a powerful distributed in-memory engine to train factorization machines on extremely large, sparse datasets consisting of thousands of products (or TV shows) and millions of users.

In the example below, we trained a factorization machine on set-top box viewing data. Our target variable was viewing seconds of the show. The factorization machine attempts to predict how long people will watch a program that they haven’t seen before based on the viewing habits of similar viewers. After training the factorization machine, we can generate a prediction for every program that an individual hasn’t watched before. Using this prediction, we can then rank-order all shows by the predicted viewing time from the factorization machine algorithm.

2. Build predictive models to predict who’s watching

To determine who is watching at any given time, you can use predictive models to best predict whether a child or family is watching. By collecting data on when and where users were historically watching family-friendly content, we can train a model that will predict the likelihood that a family or a child may be watching TV at that time. Using SAS Visual Data Mining and Machine Learning, users can build scalable modeling pipelines that take in historical viewing data, transform data for modeling, and build out a series of candidate models (such as a gradient boosting, neural network, random forest, etc.). After evaluating the modeling performance on a hold-out sample, the champion model can be published in production and leveraged within a decisioning flow.

3. Use SAS Event Stream Processing to capture what is popular right now

To better calculate what’s happening right now, we need a tool that can analyze real-time streaming data and act on it. SAS Event Stream Processing was designed precisely to analyze real-time streaming results before it lands in a data lake or database. Real-time tuning records from set-top boxes, mobile apps, websites, and smart TVs can be aggregated and analyzed in real-time to determine the most popular shows that are playing in real-time for a demographic, region, or genre.

4. Use SAS Intelligent Decisioning to deploy business rules

SAS Intelligent Decisioning is a solution for orchestrating real-time decisions that incorporate business rules and predictive models. It allows non-technical users to design decision flows using an easy GUI interface. After a decision flow is created, it can then be published as a REST API that can be called in real-time from edge devices (such as set-top boxes, mobile apps, Smart TVs, etc.) to receive a real-time recommendation. We can also export these decisions and embed them directly within SAS Event Stream Processing Engine. For more sophisticated users with a strong programming background, these business rules can also be coded directly in SAS Event Stream Processing without using Intelligent Decisioning.

In the example below, we can orchestrate a decision flow that determines what to recommend given certain circumstances. If the predictive model predicts that a child or family is watching, then a family-friendly recommendation will be presented. If event stream processing determines that a certain show is extremely popular right now, then it will override the baseline recommendation with the popular show. Otherwise, it will send the recommendation that was generated by the factorization machine.

5. Orchestrate the entire decisioning process using SAS Event Stream Processing

To bring this all together into a single flow that can work in real-time, we need a tool that can ingest real-time streaming data, enrich the data with all the relevant information we need to make an intelligent recommendation, aggregate real-time data, and execute the decisioning flow. This will result in a final recommendation. Event stream processing can orchestrate all these elements into a single project.

In the example below, SAS Event Stream Processing takes in real-time streaming set-top box records and then enriches it with data from the customer data warehouse. The event stream processing engine then aggregates real-time TV viewing across all devices and determines which shows are most popular right now. Then it scores the data using the predictive model to determine whether a child or family is watching. Finally, it executes the decision flow created in SAS Intelligent Decisioning to determine what the final recommendation will be. Event Stream Processing has a REST API that allows third-party applications or devices to connect to this flow and receive the requested recommendation.

Conclusion

The example above demonstrates how an organization can design sophisticated recommendation engines that incorporate not only AI algorithms, but also business rules, real-time streaming, and predictive models. This allows businesses to provide far superior recommendations than using AI algorithms alone. It allows context, real-time information, and business objectives to be incorporated when making the final recommendation. By leveraging tools like Event Stream Processing and SAS Intelligent Decisioning, business users can design, orchestrate, and operationalize the entire recommendation process. To learn more, check out these additional resources:

How to improve recommendation engines with real-time context and business rules was published on SAS Users.

1月 312020
 

Everyone is talking about artificial intelligence (AI) and how it affects our  lives -- there are even AI toothbrushes! But how do businesses use AI to help them compete in the market? According to Gartner research, only half of all AI projects are deployed and 90% take more than three [...]

Driving faster value from analytics – how to deploy models and decisions quickly was published on SAS Voices by Janice Newell

8月 202019
 

You can now easily embed a Python script inside a SAS decision within SAS Intelligent Decisioning. If you want to execute in SAS Micro Analytic Service (MAS), you no longer need to wrap it in DS2 code. The new Python code node does it for you. Here is how you can achieve it in less than 5 minutes:

Ready? Steady? Go!

The Python Script

If you want to run the following in MAS:

X1=1
X2=2
if X1 == None:
   X1 = 0
if X2 == None:
   X2 = 0
Y = 0.55 + 1 * X1 + 2 * X2 
print(Y)

Convert it to a Python function to meet the PyMAS requirements:

def execute(X1, X2):
       "Output: Y"
       if X1 == None:
            X1 = 0
       if X2 == None:
            X2 = 0
        Y = 0.55 + 1 * X1 + 2 * X2
       return Y
 
X1=1
X2=2
print(execute(X1,X2))

In a Jupyter Notebook, it will look like this:

Create an input data set to test the results

In SAS Studio V:

cas mysession sessopts=(metrics=true);
caslib _all_ assign;
options dscas;
 
data CASUSER.X1X2 (promote=yes);
length X1 8 X2 8;
X1=1; X2=1; output;
X1=1; X2=2; output;
X1=1; X2=3; output;
X1=1; X2=4; output;
run;
cas mysession terminate;

Create a decision in SAS Intelligent Decisioning 5.3

Choose New Python code file and call it python_logic. Copy the code from the Jupyter Notebook: from def until return Y. Watch out for your indentation!

Save and Close. Go to Variables:

Click on variables X1, X2, Y and change their type to Decimal.

Save the Decision.

Publish the decision to MAS

Test the publishing destination

Click on the published validation. Choose the data set you created:

Run. The code is executed.

Check the execution results

Y is the output of the python function. For the second line in the X1X2 data set, where X1 = 1 X2 =2, we get the result 5.55. Just as in the Jupyter Notebook.

Concepts

About Decisions in SAS

Put simply, there are three main components to a decision in SAS: inputs, logic, and outputs.

Inputs: the decision needs input variables. These can come from a CAS data set, a REST API or manual inputs.

Logic: a decision is defined by business rules, conditions, analytic models, custom code (DS2), etc. The new version allows execution of Python code in PyMAS (see below).

Outputs: a decision computes an output based on inputs and logic.

About SAS Micro Analytic Service (MAS)

A picture says a thousand words; here is a simplified diagram of MAS architecture (thanks to Michael Goddard):

MAS Architecture: Execution engine

You can apply or publish a decision using MAS. The SAS Micro Analytic Service provides the capability to publish a decision into operational environments.

When deployed as part of SAS Decision Manager, MAS is called as a web application with a REST interface by both SAS Decision Manager and by other client applications. MAS provides hosting for DS2 and Python programs and supports a "compile-once, execute-many-times" usage pattern.

The REST interface provides easy integration with client applications and adds persistence and clustering for scalability and high availability.

Prerequisites for Python decisions

You need SAS Intelligent Decisioning 5.3 in SAS Viya 3.4. SAS Intelligent Decisioning 5.3 is the wiz-kid of SAS Decision Manager 5.2. You do not need a certain Python version in your environment, but if you use certain libraries (e.g.: numpy, scipy, etc.), they might depend on the Python version.

Debugging your Python-based decisions

If you cannot replicate the example, it might be useful to consult the MAS logs. Log with MobaXtrem (or the software of your choice) to your server. Browse to the log of the concerned microservice, e.g.: microanalyticservice = MAS.

cd /opt/sas/viya/config/var/log/microanalyticservice/default/

Connect to the node using SFTP and open the log files. Check for errors, such as:

2019-06-27T21:31:12,251 [00000007] ERROR App.tk.MAS – Module ‘python1_0’ failed to compile in user context ‘provider’.

Resolve the Python error, per the messages you find in the log.

Solution for some errors

When you've made changes in your environment and have trouble getting your Python decisions to work, try to restart the following services:

  • decisionmanager
  • compsrv
  • launcher
  • runlauncher
  • microanalyticservice

Acknowledgements

Thanks to Marilyn Tomasic for finding the solution on what to do if you do not get the expected results. Thanks to Yi Jian Ching for sharing his knowledge and material.

References

Execute Python inside a SAS Decision: Learn how in less than 5 minutes was published on SAS Users.

8月 132019
 

Raw data doesn’t change an organization, and neither do analytics on their own. It’s making decisions based on that data and the results of analytics that drives change through a company. Every decision is important and influences an organization. Thousands of decisions need to be made every day and many decisions are dependent on other decisions in an interconnected network.

SAS Intelligent Decisioning combines business rules management, decision processing, real-time event detection, decision governance and analytics to automate and manage decisions across the enterprise. It supports customer-facing activities such as personalized marketing and next-best action, plus decisions affecting customers, including credit services and fraud prevention.

Overview

Business rules

An integrated business rule management platform enables fast rule construction, testing, governance and integration within decision flows. You can manage rule versions for tracking and governance. The solution allows users to create complex business logic supported by sophisticated functions and integration with Lookup Tables.

Decision flows

A graphical drag-and-drop interface allows users to build decisions with minimal programming effort. Decisions are created in a decision flow that orchestrates business rules, analytical models, database access, custom code objects and more.

Graphical editor to create decisions

Further, it is possible to test and maintain different versions of decisions and business rules before deploying them for production real-time or batch execution.

The high-performance, real-time Micro Analytics Services (MAS) engine can handle more than 5,000 real-time transactions per second with response times of 10 milliseconds per transaction. The REST interface to call decisions or business rules in real-time provides simple integration with most third-party applications.

Monitor test results through Decision Path tracking

New Features

Recently, the latest release of SAS intelligent Decisioning was released and I’d like to highlight some of the new features.

SQL Query Node

Users can now submit SQL directly into a SQL Query node without supplying any additional coding logic. The SQL Query node supports SELECT, INSERT, DELETE and UPDATE.

To link a SQL statement to a decision, just point tables and columns to the decision variables as shown below in the curly brackets. Intelligent Decisioning will then automatically pass data into the SQL as appropriate.

If you query data via a select statement, the result is returned in a Datagrid. A Datagrid is a data type for an object in Intelligent Decisioning and represents data in a table format that belongs to a single record.

Datagrids are used in many places in Intelligent Decisioning and there is a rich set of Datagrid functions to access and work with data in a Datagrid.

Python Code Node

Intelligent Decisioning provides an environment that aims to minimize the need to write code to build decisions. But if necessary, it is possible to submit code. Intelligent Decisioning supports writing code in Python as part of a decision flow. Data from a decision flow can be passed into the Python code and return values will be passed back from Python into the decision flow.

To enable coding in Python, a Python execution environment needs to be installed alongside Intelligent Decisioning. If a decision flow contains a Python Code Node, the Python code will automatically be executed in the Python environment as part of the overall decision.

Decision Flow containing Python code node

A code editor in Intelligent Decisioning allows you to edit your Python code within the environment.

A Python code editor is part of Intelligent Decisioning

Decision Node

Decision flows can call other decision flows. This opens the way to designing and building modular decisions with “pluggable” components. You can also build reusable decisions which are called by different decision flows. Building decisions in such a modular way makes it easier to read and maintain decision flows.

Drill down from one decision to the next

Treatments

Treatments are lists of attributes with fixed or dynamic values.

Treatments are used to define offers to present to a customer as a result of an inbound marketing campaign. Or treatments can be used as parameter lists to control engine settings. There are numerous use cases for treatments.

Treatment attribute list

To determine if a treatment is valid for a decision, you can set Eligibility Rules to decide when a treatment will be used. For audit reasons and to track changes over time, you can also have different versions of a treatment.

To utilize treatments, you group them together in treatment groups, which can then be called from a decision flow.

Conclusion

Manging and analysing high volumes of data to make thousands of decisions every day in an automated fashion and applying analytics to real-time customer interactions require a sophisticated and complete solution like SAS Intelligent Decisioning. It enables users to create, test, control versioning and trace analytically driven decisions all in one solution.

By making decisions, smarter organizations become more efficient. As mentioned in the beginning: Data doesn’t change your organization, decisions do!

Learn more

Video: SAS Intelligent Decisioning | Product Overview
Documentation: SAS Intelligent Decisioning
Product: SAS Intelligent Decisioning

SAS Intelligent Decisioning: Intro and Update was published on SAS Users.