SAS, in partnership with Intel, is sponsoring the 2020 Esri User Conference taking place virtually on Jul. 13-16. The Esri User Conference is the world’s largest virtual Geographic Information System (GIS) event and will include three days of inspiring plenary and technical sessions of how GIS is making a difference [...]
How have healthcare providers and governmental agencies predicted the fast-changing, potentially exponential increase in the need for medical services and equipment through the various stages of the COVID-19 pandemic? Mathematical techniques that attempt to model and understand the likely spread of the disease have been instrumental. The SEIR model is [...]
Adverse outcomes, and the rapid spread of COVID-19, have accelerated research on all aspects of the disease. You may have found it overwhelming, and very time-consuming, to find relevant and specialized insights in all the scientific literature out there. To aid researchers in quickly identifying relevant literature about key topics [...]
First introduced in SAS Visual Analytics 8.3, common filters are filters that can be shared between objects in your reports.
Common filter benefits include:
Easy to assign the same filter conditions to other report objects.
When you edit a common filter, it is updated everywhere that the common filter is used.
A common filter is available for the entire report, across pages.
Common filter limitations include:
Objects must share the same data source as the common filter definition.
Common filters are not available across multiple reports.
While the benefits speak for themselves, common filters also expedite designing reports and exploring data by quickly reusing the same filter conditions with only a few mouse clicks.
Let’s look at some examples of using common filters. In the screenshot below, I defined a filter to return data for the last 30 days. I converted that filter to a common filter and now the Last 30 Days common filter can be applied to any object in this report that uses the same data source. Then I applied it to the bottom treemap object.
In my second example, I parameterized a text input control to capture a user-defined value. Then I defined a parameter-driven filter for the bar chart object. Next, I converted that filter to a common filter named Text Input Contains Filter and applied it to the list table object below.
In my third example, I parameterized two drop-down list controls to capture date boundaries. Then I defined a parameter-driven filter for the waterfall chart object. Next, I converted that filter to a common filter named FromToWeekFilter and applied it to the key value object.
Pro tip: Embrace meaningful filter names. Recall that these common filters are associated with the data source for which they are defined. Being able to quickly identify a common filter definition by its name will save you additional mouse clicks in the long run.
Example 1: Last 30 Days Filter
Let’s pick up where we need to define the filters. Therefore, both objects have been added to the report page with roles assigned.
Next, select the line chart object and open the Filters pane. Click + New filter then select Advanced filter.
Now we will use the built-in filter conditions offered in SAS Visual Analytics. First, select the date/datetime data item, Day. Second, scroll down in the available conditions until you see Last 30 days, then double click on the condition to add it to the expression editor. Third, give the filter condition a meaningful name and as a final step, be sure the number of returned observations is expected. Click OK.
Now we need to convert this filter to a common filter. With the line chart object still selected, on the Filters pane, use the Last 30 Day overflow menu and select Change to common filter.
You should now see the Last 30 Days listed as a Common Filter in the Data pane.
Lastly, we can apply this common filter to the treemap object. Select the treemap object, then open the Filters pane. Next click + New Filter and select the common filter Last 30 Days.
It’s good practice to title your objects to reflect any filters that may be defined for them. Especially if there are no prompts, i.e. control objects, driving the subset of data. This is so your report consumers quickly understand that they are only seeing, in this case, the last 30 days of data.
Example 2: Text Input Contains Filter
This next example will define a parameter-driven common filter. If you are not familiar with using parameters in SAS Visual Analytics, start with this article and refer to the additional materials at the end.
See the screenshot below for more information on the data item role assignments for each object. The most important role that will drive the parameter-driven common filter is the parameter. The parameter, TextInputParameter, is a character parameter assigned to the text input control object and it is the only role assignment. If there is nothing entered in this prompt, i.e. control object, then the filter will return all of the data. We will define a contains expression to only return data rows that contain the entered text.
With the bar chart object selected, open the Filters pane and click + New filter and select Advanced filter.
Now we need to build our parameter-driven filter expression. In this filter we will be checking if the Product Description data item Contains the entered text stored in the TextInputParameter. I wrapped each string expression in an UpCase function so that mixed case is ignored. I could have just as easily used the LowerCase function to get the same result.
And finally, remember to give your filter a meaningful name. The returned observations number is not reflective of an applied filter since I do not have any text entered in the control object. If you had text entered there, then your returned observation number may show a subset of matched rows.
Now we need to convert this object-level filter to a common filter. With the bar chart still the active object, open the Filters pane and use the Text Input Contains Filter overflow menu and select Change to common filter.
You should now see the Text Input Contains Filter in the Data pane.
To add this common filter to the lower list table object, select the list table object and open the Filters pane. Click on the + New filter and select the Text Input Contains Filter.
Now let’s test the filter. In the screenshot below, I’ve typed the word red. Recall that our filter expression is to return rows where Product Description contains the entered text. Notice that even though I do not have the data item Product Description assigned as a data role in the bar chart object, SAS Visual Analytics is still able to apply the filter appropriately. The list table object does have a role assigned for Product Description so that filter application is easier to identify.
I used the contains operator instead of the equals to return a partial match to the Product Description to help identify trends across multiple Products. In these next examples, I entered the text (F) and (M) to return the rows where the Product Description indicates gender-specific products.
Example 3: From – To Week Filter
This last example will also define a parameter-driven common filter. If you need a more step-by-step guide to similar examples, refer to this YouTube video tutorial:
See the screenshot below for more information on the data item role assignments for each object. This example differs from the last in that these control objects, the drop-down lists, use both Category and Parameter Role assignments. The Year-Week data item will provide the available values and the selections will be stored in the parameters FromWeekParameter and ToWeekParameter. I will then create a common filter for an inclusive between of the selected year week values to apply to both the key value object and the waterfall chart.
With the waterfall chart object selected, open the Filters pane and click + New filter and select Advanced filter.
Now we need to build our parameter-driven filter expression. In this filter, I will subset the Year-Week date values which are inclusively between the FromWeekParameter and ToWeekParameter boundaries. Follow these steps:
Select the date/datetime data item, Year-Week.
Scroll down in the available conditions till you see Year-Week BetweenInclusive(‘x’,’y’), then double click on the condition to add it to the expression editor.
Drag the FromWeekParameter and ToWeekParameter from the parameter data items list to the expression.
Give the filter condition a meaningful name. Click OK.
Test the filter by adjusting the values in the drop-down list controls.
Now we need to convert this object-level filter to a common filter. With the waterfall chart still the active object, open the Filters pane and use the FromToWeekFilter overflow menu and select Change to common filter.
You should now see the FromToWeekFilter in the Data pane.
To add this common filter to the key value object; select the key value object and open the Filters pane. Click on the + New filter and select the FromToWeekFilter.
Remember that once a common filter is defined, it can be used for any object in the report that uses the same data source. In these examples, I applied the common filter on the same report page but you can use a common filter on any page within the report. Recall that all of the common filters are listed and available when I click to apply a new filter to an object.
Hopefully these examples have shown you new ways to explore data faster!
Additional materials for using parameters in SAS Visual Analytics:
In the early days and weeks of any widespread global health concern, particularly in a fast-moving outbreak like the coronavirus, there are many unknowns. Data visualization can be a good starting point to understand trends and piece data points together into a meaningful story. The ability to visualize the spread [...]
Administrators like to be able to keep track of resource usage and who is using what in a system. When an administrator has this capability, they can look out for issues of high resource usage that may have an impact on overall system performance. In Viya, data is accessed in caslibs. In this post, I will show you how an administrator can track and control resource usage of personal caslibs.
A Caslib is an in-memory space to hold tables, access control lists, and data source information. In GEL enablement classes as we have discussed CAS and caslibs, one of the big asks we have had was related to personal caslibs and how can an administrator keep track of the resources that they use. Until recently we didn’t have a great answer, the good news is now we do.
Personal caslibs are, by default, the active caslib when a user starts a CAS session. This gives each user a default location to access and load data (sort of like a home directory). As the name suggests, they are personal and only the user who starts the session can access the data. In early releases, administrators saw this as a big problem because personal caslibs were basically invisible to them. There was no way to monitor what was going on with a personal caslib, leaving the system open to issues where one user could consume a lot of resources and have an adverse impact.
Introduced in Viya 3.4, the accessControl.accessPersonalCaslibs action brings all existing personal Caslibs into a session where an administrator has assumed the data or superuser role.
Running the accessControl.accessPersonalCaslibs action has the following effects. The administrator can:
See all personal caslibs that existed at the time the action was run
View promoted tables characteristics within the personal caslibs
Drop promoted tables within other users’ personal caslibs.
This elevation of access remains in effect for the duration of the session. The action does not give an administrator full access to personal caslibs, an administrator can never fetch data from other users’ personal caslibs, drop any personal caslib, set access controls on any personal caslib, or set access controls on any table in any personal caslib. What it does is give the administrator a view into personal caslibs to be able to monitor and troubleshooting their resource usage.
Let’s look at how it works. Logged into Viya as Viya administrator (by default also a CAS administrator), I can use the table.caslibinfo action to see all the caslibs that the administrator has permission to view. In the output below, I see my own personal caslib, and all the other caslibs that the administrator has permissions (set by the CAS authorization system) to view.
cas mysess terminate;
In the code below, the super user role is assumed for this session (SAS Administrators by default can also assume the super user role in CAS). With the role assumed, the administrator can execute the accessControl.accessPersonalCaslibs action and the subsequent table.caslibinfo action returns all caslibs including any personal caslibs that existed when the session started.
/* need to be a super user or data administrator */
accessControl.assumeRole / adminRole="superuser";
cas adminsession terminate;
That helps, but what about the details? How can the administrator see how many resources the tables in a personal CASLIB are using? To get the details, we can access an individual CASLIB and its tables, and for each table, execute the table.tabledetails action. The program below will loop all the personal caslibs and, for each of the caslibs, it will loop the in-memory tables and execute the table.tabledetails action. The output of tabledetails gives an idea of how many resources (memory, disk etc.) a table is using.
/* need to be a super user */
accessControl.assumeRole / adminRole=”superuser”;
/* loop caslibs */do cvalue over casliblist;
if cvalue.name==: ‘CASUSER’ thendo; /* only look at caslibs that contain CASUSER */table.tableinfo result=tabresult / caslib=cvalue.name;
if x>1thendo; /* there are tables available */do tvalue over tablelist; /* loop all tables in the caslib */table.tabledetails / caslib=cvalue.name name=tvalue.name;
table.tableinfo / caslib=cvalue.name name=tvalue.name;
end; /* loop all tables in the caslib */end; /* there are tables available */end; /* only look at caslibs that contain CASUSER */end; /* loop caslibs */
accessControl.dropRole / adminRole=”superuser”;
cas adminsession terminate;
Two fields that can give an administrator an idea of how big a table is are:
Data size: the size of the SAS Dataset in memory
Memory Mapped: the part of the data that has been “memory mapped” and is backed up in the CAS Disk Cache memory-mapped files.
The table below show the output for one users personal caslib.
If one table in particular is causing problems, it is possible for the administrator to drop the table from memory.
accessControl.assumeRole / adminRole=”superuser”;
sessionProp.setSessOpt / caslib=”CASUSER(gatedemo499)”;
table.droptable / name=”TRAIN”;
cas adminsession terminate;
Visibility into personal caslibs will be a big help to Viya administrators monitoring CAS resource usage. Check out the following for more details:
Using Customer Lifetime Value in your business decision making is often important and crucial for success. Businesses that are customer-centric often spend thousands of dollars acquiring new customers, “on-boarding” new customers, and retaining those customers. If your business margins are thin, then it can often be months or quarters before you start to turn a profit on a particular customer. Additionally, some business models will segment the worth of their customers into categories that will often give different levels of service to the more “higher worth” customers. The metric most often used for that is called Customer Lifetime Value (CLV). CLV is simply a balance sheet look at the total cost spent versus the total revenue earned over a customer’s projected tenure or “life.”
In this blog, we will focus on how a business analyst can build a functional analytical dashboard for a fictional company that is seeing its revenue, margins, and a customer’s lifetime value decrease and what steps they can take to correct that.
We will cover 3 main areas of interest:
First, screenshots of SAS Visual Analytic reports, using Customer Lifetime Value and how you can replicate them.
Next, we will look at the modeling that we did in the report, with explanations on how we got used the results in subsequent modeling.
Lastly, we talk about one example of how we scored and deployed the model, and how you can do the same.
Throughout this blog, I will also highlight areas where SAS augments our software with artificial intelligence to improve your experience.
1. State of the company
First, we will look at the state of the company using the dashboard and take note of any problems.
Our dashboard shows the revenue of our company over the last two years as well as a forecast for the next 6 months. We see that revenue has been on the decline in recent years and churns have been erratically climbing higher.
Our total annual revenue was 112M last year with just over 5,000 customers churning.
So far this year, our revenue is tracking low and sits at only 88M, but the bad news is that we have already tripled last year's churn total.
If these trends continue, we stand to lose a third of our revenue!
2. The problems
Now, let’s investigate as to where the problems are and what can be done about them.
If we look at our current metrics, we can see some interesting points worth investigating.
The butterfly chart on the right shows movement between customer loyalty tiers within each region of the country with the number of upgrades (on the right) and downgrades (on the left).
The vector plots show us information over multiple dimensions. These show us the difference between two values and the direction it is heading. For example, on the left, we see that Revenue is pointed downward while churns (x axis) are increasing.
The vector plot on the right shows us the change in margin from year to year as well as the customer lifetime value.
What’s interesting here is that there are two arrows that are pointing up, indicating a rise in customer lifetime value. Indeed, if we were to click on the map, we would see that these two regions are the same two that have a net increase in Loyalty Tier.
This leads me to believe that a customer’s tier is predictive of margin. Let’s investigate it further.
3. Automated Analysis
We will use the Automated Analysis feature within Visual Analytics to quickly give us the drivers of CLV.
This screenshot shows an analysis that SAS Visual Analytics(VA) performed for me automatically. I simply told VA which variable I was interested in analyzing and within a matter of seconds, it ran a series of decision trees to produce this summary. This is an example of how SAS is incorporating AI into our software to improve your experience.
Here we can see that loyalty tier is indeed the most important factor in determining projected annual margin (or CLV).
4. Influential driver
Once identified, the important driver will be explored across other dimensions to assess how influential this driver might be.
A cursory exploration of Loyalty Tier indicates that yes, loyalty tier, particularly Tier 5, has a major influence on revenue, order count, repeat orders, and margin.
5. CLV comparison models
We will create two competing models for CLV and compare them.
Here on our modeling page are two models that I’ve created to predict CLV. The first one is a Linear Regression and the second is a Gradient Boosting model. I've used Model Comparison to tell me that the Linear Regression model delivers a more accurate prediction and so I use the output of that model as input into a recommendation engine.
6. Recommendation engine
Based on our model learnings and the output of the model, we are going to build a recommendation engine to help us with determine what to do with each customer.
Represented here, I built a recommendation engine model using the Factorization Machine algorithm.
Once we implement our model, customers are categorized more appropriately and we can see that it has had an impact on revenue and the number of accounts is back on the rise!
Even though Customer Lifetime Value has been around for years, it is still a valuable metric to utilize in modeling and recommendation engines as we have seen. We used it our automated analysis, discovered that it had an impact on revenue, we modeled future values of CLV and then incorporated those results into a recommendation engine that recommended new loyalty tiers for our customers. As a result, we saw positive changes in overall company revenue and churn.
“The future is already here — it's just not very evenly distributed.” ~ William Gibson, author The same can be said for climate change – global warming is here, in a big way, but its effects are still an arm's length away for many of us. How is climate change [...]
Designing interactive reports can be a fun and unique challenge. As user interface experience designers can attest, there are several aspects that go into developing a successful and effective self-service tool. Granted I’m not designing the actual software, but reports require a similar approach to be sure that visualizations are clear and that users can get to the answers they are looking for. Enter prompts.
Reports prompt users to better understand trends, how their data points compare to the whole, and to narrow the scope of data. Being able to pick the placement of these prompts quickly and easily will open the possibilities of your report layouts! I’m specifically speaking about Report and Page level prompts. Traditionally, these global prompt controls were only able to be placed at the top; see the yellow highlighted areas below.
Let’s take a look at an example report with the traditional Report and Page prompt layout. The Report prompts are extremely easy to pick out, since they sit above the pages, but the Page prompts can sometimes blend in with other prompts contained in the report body.
Introduced in the SAS Visual Analytics 8.4 release is the ability to control the layout position of these prompts. Using my example report, let’s change the placement of these prompts. In Edit mode, open the Options pane and use the top level drop-down to select the report name. This will activate the report level, and the report level Options will display. Next, under the Report Controls subgroup, move the placement radio button to the west cardinal point.
Depending on the type of control objects you are using in your report, you may not like this layout yet. For instance, you can see here that my date slider is taking up too much space.
When you activate the slide control, use the Options pane to alter the Slider Direction and Layout. You can even use the Style option to change the font size. You can see that after these modifications, the Report prompt space can be configured to your liking.
Next, let’s change the placement for the Page prompts, for demonstration purposes. From the Options pane, use the top drop-down to select the page name. This will activate the page level, and the page level Options will display. Next, under the Page Controls subgroup, move the placement radio button to the west cardinal position.
You can see that the direction of the button bar control was automatically changed to vertical. Now we can clearly see which prompts belong to the page level.
If I switch to view mode, and adjust the browser size, you can get a better feel for the Report and Page prompt layout changes.
Notice that I kept the Report prompts along the top but moved the Page prompts to the left of the report. I also added two containers and configured a gray border for each container to better separate the objects. This helps the user quickly see that the drop-down will filter the word cloud is only. I also used the yellow highlighting through styling and a display rule to emphasize the selected continent. The bar chart is fed from an aggregated data source which is why the report prompt is not filtering out the other continents.
Feel free to send me your latest report design ideas!
Additional material related to Report and Page prompts:
What can data tell us about the easiest hole at your favorite golf course? Or which hole contributes the most to mastering the course? A golf instructor once told me golf is not my sport, and that my swing is hopeless, but that didn’t stop me from analyzing golf data. [...]