Tech

6月 042016
 

In DataFlux Data Management Studio, the data quality nodes (e.g., Parsing, Standardization, and Match Codes) in a data job use definitions from the SAS Quality Knowledge Base (QKB).  These definitions are based on a locale (Language and Country combination).  Sometimes you would like to work with multi-locale data within the same data job and these data quality nodes have LOCALE attributes as part of their Advanced Properties to help you do this.

For example, you may want to work with data from the United States, Canada, and the United Kingdom within the same data job.  Note:  You must have the QKB locale data installed and be licensed for any locales that you plan to use in your data job.

The Advanced properties you will need to use are LOCALE_FIELD and LOCALE_LIST.  LOCALE_FIELD specifies the column name that contains the 5-character locale value to use for each record.  LOCALE_LIST specifies the list of locales that should be loaded into memory for use within the node.

The first step in using these Advanced properties in a data quality node in a data job is you need a field that contains the 5-character QKB locale information.  The first 2-characters represent the language and the last 3-characters represent the country.  For example, ENUSA represents the English – United States locale and ESESP represents the Spanish – Spain locale.  You can use the Locale Guessing node to create a field that contains a locale value based on a Locale Guess definition from the Quality Knowledge Base (QKB).  Alternatively, you can use a custom Standardization scheme to assign the 5-character locale information as shown in the example below.

 

Using Multiple Quality Knowledge Base Locales01
QKB Locale Standardization Scheme

 

Using Multiple Quality Knowledge Base Locales02
Apply QKB Locale Standardization Scheme

The application of the Standardization scheme is then followed up with an Expression node to assign the locale FRCAN (French – Canada) if the province is Quebec.  Now each record has its 5-character locale information in a field called DF_Locale.

Using Multiple Quality Knowledge Base Locales03
QKB Locale Field Results

Once you have the Locale field as part of your input data, you enter the information as usual for the data quality node.

Using Multiple Quality Knowledge Base Locales04
Match Codes Node Properties

Then you map the field with the 5-character locale information to the LOCALE_FIELD Advanced property for the data quality node. You also need to list the locales that should be loaded into memory in the LOCALE_LIST advanced property. Note: You could pass in this list as values using a macro variable.

Using Multiple Quality Knowledge Base Locales05
Match Codes Node Advanced Properties

Note: The definition used in the selected node must exist in all locales referenced.  For example, the State/Province Match definition only exists in the English – United States, English – Canada, and French – Canada locales.  Therefore, if you are using that definition in a Match Codes node you can only pass in data that is from one of those three locales; otherwise, executing the data job will produce an error.

Here is an example data job that uses the Advanced properties of LOCALE_FIELD and LOCALE_LIST to generate match codes for multi-locale data. Notice that there is minimal branching in the data flow.  The only branching that was needed is the United States and Canada data records are branched to generate the match codes for its State/Province data.

Using Multiple Quality Knowledge Base Locales06
Multi-Locale Data Job Example

In conclusion, the Advanced properties of LOCALE_FIELD and LOCALE_LIST are useful when you want to work with data from multiple locales within the same data job. For more information on Advanced properties for nodes in DataFlux Data Management Studio, refer to the topic “Advanced Properties” in the DataFlux Data Management Studio 2.7: User’s Guide.

tags: data management, DataFlux Data Management Studio, SAS Professional Services

Using Multiple Quality Knowledge Base Locales in a DataFlux Data Management Studio Data Job was published on SAS Users.

5月 272016
 

favorite big data presentations from SAS Global Forum 2106Nowadays, nearly every organization analyzes data to some degree, and most are working with “Big Data.”  At SAS Global Forum 2016 in Las Vegas, a vast number of papers were presented to share new and interesting ways our customers are using data to IMAGINE. CREATE. INNOVATE., as this year’s conference tagline reminds us.

Below you’ll find a collection of a few of my favorites on big data topics, ranging from SAS Grid Manager to Hadoop to SAS Federation Server. The common point? It’s easier than ever to modernize your architecture now. I hope these papers help you continue to advance your organization.

Paper 2020-2016: SAS® Grid Architecture Solution Using IBM Hardware
Whayne Rouse and Andrew Scott, Humana Inc.

This paper is an examination of Humana journey from SAS® 9.2 to SAS® 9.4M3 and from a monolithic environment to a SAS Grid Manager environment on new hardware and new storage. You can find tips such as the importance of understanding the old environment before starting and applying that understanding to building the new environment.

Paper 11684-2016: Multi-Tenancy in SAS® – Is It Worth the Fight?
Christopher Blake, RBS; Sergey Iglov, RBS; Dileep Pournami, RBS; Ekaitz Goienola, SAS UK

This paper explains how Royal Bank of Scotland analyzed the benefits of creating a shared Enterprise Business Intelligence platform in SAS alongside the risks and organizational barriers to the approach. The objective is to have a single, multi-tenant environment offering a variety of SAS software, including SAS Grid Manager, SAS Visual Analytics, SAS Enterprise Miner, and others.

Paper 11562-2016: Beyond Best Practice: Grid Computing in the Modern World
Jan Bigalke, Allianz Managed Operation & Services SE & Gregory S. Nelson, ThotWave Technologies

After an introduction detailing how SAS Grid Manager works, this paper describes multiple best practices implemented at Allianz Managed Operation & Services SE to configure and administer a multi-tenant grid environment.

Paper SAS6281-2016: Introducing SAS® Grid Manager for Hadoop
Cheryl Doninger and Doug Haigh, SAS Institute Inc.

With this paper you can learn the capabilities and benefits of SAS Grid Manager for Hadoop as well as some configuration tips. You will also find sample SAS Grid jobs to illustrate different ways to access and analyze your Hadoop data with your SAS Grid jobs.

Paper SAS2140-2016: Best Practices for Resource Management in Hadoop
James Kochuba, SAS Institute Inc.

YARN is the Hadoop answer to resource orchestration. SAS technologies can take advantage of a shared Hadoop environment, creating the ultimate analytic environment. This paper helps to set the expectations and tuning choices when setting up and using a shared Hadoop environment encompassing SAS technologies such as SAS/ACCESS® Interfaces, SAS® In-Database Analytics, SAS® In-Memory Analytics and SAS® Grid Computing.

Paper SAS2180-2016: How to Leverage the Hadoop Distributed File System as a Storage Format for the SAS® Scalable Performance Data Server and the SAS® Scalable Performance Engine
Steven Sober, SAS Institute Inc.

In this paper Steven explores how the SAS Scalable Performance Data Server and SAS Scalable Performance Data Engine leverage the Hadoop Distributed File System (HDFS). He explores the different capabilities of both engines and shows benchmarks detailing the differences in reading/writing compressed tables versus regular ones.

Paper SAS6223-2016 What’s New in SAS® Federation Server 4.2
Tatyana Petrova, SAS Institute Inc.

This paper provides an overview of the latest features of the product and includes examples for leveraging SAS Federation Server capabilities, such as data quality functions, data masking, and query performance optimization.

tags: big data, SAS Global Forum, SAS Professional Services

Seven of my favorite big data presentations from SAS Global Forum 2106 was published on SAS Users.

5月 252016
 

Pick your category? If this title seems familiar, that’s because in my last blog, Use parameters to pick your metric in VA Reports, I covered how to use parameters to allow your users to pick which metric they want to view in their visualizations. This is a great technique that offers a solution to many report requirements.

But, what if your users require specific axes labels and titles for your visualizations? What if your users require reference lines? If you encounter these requirements then consider using a stack container to meet these needs.

Let’s take a look; but first, here is a breakdown of the report we will be looking at in this blog. This report does not have any report level prompts but it does have two section prompts. Section prompts filter the data for every object on this section. There is a drop-down list control object that prompts the user for Year, and there is also a button bar control object that prompts the user for Continent.

Then in the report body there is a list control object, a text box and a stack container. The list control object prompts the user for Country. The text box provides the first half of the report title. And the stack container provides a way to organize multiple visualizations on your report; it layers or “stacks” the objects as if they were in a slide deck. The stack container provides navigation options to cycle through the visualization objects that were added. In this example, I added two bar charts and one line chart object to the stack container.

Use a stack container in SAS Visual Analytics

In this first view, I have the Product Line bar chart selected from the stack container. Notice in this bar chart I have two reference lines defined for the Y-axis: one at 250,000 and one at 500,000 for Order Total. Having these reference lines remain static as I select different continents from the button bar section prompt helps to compare these values across the different countries and regions. You can use reference lines to help identify goals or targets, acceptable or unacceptable ranges, previous year average or even previous month average, etc., basically any metric that puts a frame of reference around the values on the visualization.

Use a stack container to pick your category in Visual Analytics02

In this second view, I have the Vendor Type bar chart selected from the stack container. Notice in this bar chart I do not have any reference lines.

Use a stack container to pick your category in Visual Analytics03

In this third view, I have the Month line chart selected from the stack container.

Use a stack container to pick your category in Visual Analytics04

I have used the text box in this report to help with the custom titles. In this case, the title of this report is Total Orders By then each visualization in the stack container uses the category role as its title: either Product Line, Vendor Type or Month to complete the report title.

Use a stack container to pick your category in Visual Analytics05

What can you do with a stack container?

You cannot create an interaction or link to or from the stack container itself, but you can from the individual objects that are inside the stack container.

The way the stack container is used in this report allows the user to focus on the metric Order Totals and then examine these values across several category data items. As you can see, I have created an interaction from the list control object to each visualization in the stack container which allows the report user to filter out outliers or compare specific countries.

Use a stack container to pick your category in Visual Analytics06

In this screenshot, I have North America selected for my continent. We can see that the values for United States greatly exceed the values for Mexico and Canada. We can also see how far the United States values are from our reference lines.

Use a stack container to pick your category in Visual Analytics07

If I remove United States from this visualization - by using the list control to select Canada and Mexico - I can see more clearly how these values compare to each other and to the reference lines.

Use a stack container to pick your category in Visual Analytics08

How is the stack container implemented?

Simply drag and drop the Stack Container object from the Objects tab on to your report. Then, drag and drop the different objects into the stack container. In my example, I added two bar charts and a line chart. From the stack container’s Properties tab you can order the objects you added to the stack container by using the up and down arrows.

Use a stack container to pick your category in Visual Analytics09

How do you name the stack container objects?

These are the individual Names of each visualization object from the Properties tab. In this screenshot, I have the first bar chart selected, and I named this bar chart Product Line.

Use a stack container to pick your category in Visual Analytics10

You can also select different navigation properties for the stack container:

  • Navigation control location: Top left, Top Center, Top Right, Bottom Left, Bottom Center, or Bottom Right
  • Navigation button type: Buttons, Links, or Dots.

Use a stack container to pick your category in Visual Analytics11

Just like in the pick your metric blog, you can use this report technique to allow your report users to pick either different metrics, categories or both!

The stack container is one of my favorite report objects and really provides an interactive reporting experience to examine and explore the data.

tags: SAS Professional Services, SAS Visual Analytics

Use a stack container to pick your category in SAS Visual Analytics Reports was published on SAS Users.

5月 242016
 

Solving Business Problems with SASBeing a SAS consultant is about solving problems. In our day-to-day work we solve myriads of all sorts of problems – technical problems, data problems, programming problems, optimization problems – you name it. And in the grand scheme of things we solve business problems.

But without a well-defined business problem, all our problem-solving efforts become irrelevant. What is the benefit of optimizing a SAS program so it runs 2 seconds instead of 20? Well, you can claim a ten-fold improvement, but “so what?” if that program is intended to run just once! Given the number of hours you spent on such an optimization, you were definitely solving the wrong problem.

The ice cream maker

There was one event early in my life that made an unforgettable impression on me and forever changed my problem-solving mindset. When I was a teenager, my father and I bought Mom a present for her birthday – an ice cream maker – a little bowl with a slow electrical mixer that you place into a fridge. Yes, it was a cleverly self-serving gift on our part, but, hey, that was what she really wanted!

Unfortunately for all of us, the gift turned out to be totally useless, as our old refrigerator had such a construction that its door’s rigid rubber-covered-metal gasket would just cut off the ice cream maker’s wire, which was supposed to be plugged in outside of the fridge. Obviously, we were all pretty upset about it. My father was contemplating making a hole somewhere in the refrigerator to accommodate the wire. Mom pretended she never really cared about the ice cream maker in the first place. The whole gift situation was just terrible.

A couple weeks later we discovered a lottery ticket amongst the ice cream maker documentation – you know, one of those worthless no-chance promotional lottery tickets no one ever wins with.

Never in my life have I won anything with a lottery… except that time. No, we didn’t win a cash jackpot. Nor had we won a book titled “Converting an ice cream maker into a wireless device for dummies.” The win was much more meaningful. We had won a new refrigerator. It had a nice, soft rubber gasket around its door that perfectly allowed for the ice cream maker’s wire. The beauty of this win was not just that it was the perfect solution to our problem, but that, in fact, the solution (the winning lottery ticket) came packaged with the source of our problem (the ice cream maker)!

I can’t tell you how much that lucky event shaped my professional and overall attitude. From then on, whenever I faced a problem, I would always search for that little “lottery ticket.”  I know it’s there; I just need to find it, and that will turn the problem on its head. From then on, I saw the opportunity in any problem, and the satisfaction of solving the right one.

Problem-driven or purpose-driven solution?

The customer was straight to the point. They had a huge SAS data table that needed to be updated on a daily basis, at about 10 am when all the primary data sources were finished refreshing. The problem was that updating that SAS table was taking about 30 minutes, during the entirety of which the table was not available to their business analysts, as it was locked by the updating process. This created a 30 minute downtime for the business analysts. The customer formulated the problem as “too long a data table update,” with the suggested solution being “to optimize the process of updating the table”.

Sounds logical, doesn’t it? But we should never take for granted the customer’s vision of the problem, especially when it’s accompanied by a suggested solution.  In most cases, this will lead to solving the wrong problem. Just keep your mind open and help your customers to keep their minds open too.

Problems do not exist by themselves without a purpose, a goal. The customer is the ultimate authority of setting those goals and purposes. Get that purpose to be articulated first before jumping to formulating a problem. Picture this dialogue:

- What is your goal - to minimize the data table refresh time or to minimize the business analysts’ downtime?
- Aren’t those the same?
- Let’s assume not.
- But how?
- Let’s not ask “how” just yet. If I reduce your business analysts’ downtime to zero, would you care if your data table is being updated 30 minutes or 15 minutes?
- No. Well, then our goal is to minimize the business analysts’ downtime.

We just made a huge leap – from an imaginary problem to a purpose. In fact, the duration of the table update is not important as long as the analysts’ downtime is minimized. So the real problem is the downtime, which we need to reduce or, ideally, eliminate.

When the real problem is well-defined, finding a solution is a matter of practice. As Albert Einstein once said, “If I were given one hour to save the planet, I would spend 59 minutes defining the problem and one minute resolving it.”  It makes a lot of sense, since spending all your time on solving the wrong problem does not move you an iota in the right direction.

Creative problem-solving

There are many proven methods and techniques of creative problem-solving. One of them is using analogies. To arrive at “how” (the solution), let’s consider the following analogy. Suppose we want to renovate a restaurant but we don’t want to stop its operations and lose its patrons during that renovation. What would we do? An ideal solution would be to build a new restaurant building next to the old one. After we finish the construction, we would simply take the sign from the old building and affix it to the new one. Problem solved. We can demolish or reuse the old building.

For SAS data tables that would translate into this. Let’s say our data table is called ANALYTICS. Instead of rebuilding it “in place,” we build a new table, let’s call it ANALYTICS_NEW. We can spend 30 minutes or 15 minutes doing this – it does not matter. After we finish table-building we can just make two quick renames:

  1. Rename  ANALYTICS table to ANALYTICS_OLD;
  2. Rename ANALYTICS_NEW table to ANALYTICS.

Using CHANGE statement of PROC DATASETS these 2 renames would take about 0 seconds – that is our new downtime. After that we can delete the ANALYTICS_OLD table. Looks like we just found an ideal purpose-driven solution to effectively eliminate (not just reduce) the downtime.

I can foresee some inquisitive minds asking why we can’t do the table update in place without locking it out by using SAS/SHARE software.  Yes, we can, and that would be a “refrigerator” solution to an “ice-cream maker” problem. Unfortunately, not all the refrigerators are won in a lottery.

Are you solving the wrong problem? was published on SAS Users.

5月 242016
 

Solving Business Problems with SASBeing a SAS consultant is about solving problems. In our day-to-day work we solve myriads of all sorts of problems – technical problems, data problems, programming problems, optimization problems – you name it. And in the grand scheme of things we solve business problems.

But without a well-defined business problem, all our problem-solving efforts become irrelevant. What is the benefit of optimizing a SAS program so it runs 2 seconds instead of 20? Well, you can claim a ten-fold improvement, but “so what?” if that program is intended to run just once! Given the number of hours you spent on such an optimization, you were definitely solving the wrong problem.

The ice cream maker

There was one event early in my life that made an unforgettable impression on me and forever changed my problem-solving mindset. When I was a teenager, my father and I bought Mom a present for her birthday – an ice cream maker – a little bowl with a slow electrical mixer that you place into a fridge. Yes, it was a cleverly self-serving gift on our part, but, hey, that was what she really wanted!

Unfortunately for all of us, the gift turned out to be totally useless, as our old refrigerator had such a construction that its door’s rigid rubber-covered-metal gasket would just cut off the ice cream maker’s wire, which was supposed to be plugged in outside of the fridge. Obviously, we were all pretty upset about it. My father was contemplating making a hole somewhere in the refrigerator to accommodate the wire. Mom pretended she never really cared about the ice cream maker in the first place. The whole gift situation was just terrible.

A couple weeks later we discovered a lottery ticket amongst the ice cream maker documentation – you know, one of those worthless no-chance promotional lottery tickets no one ever wins with.

Never in my life have I won anything with a lottery… except that time. No, we didn’t win a cash jackpot. Nor had we won a book titled “Converting an ice cream maker into a wireless device for dummies.” The win was much more meaningful. We had won a new refrigerator. It had a nice, soft rubber gasket around its door that perfectly allowed for the ice cream maker’s wire. The beauty of this win was not just that it was the perfect solution to our problem, but that, in fact, the solution (the winning lottery ticket) came packaged with the source of our problem (the ice cream maker)!

I can’t tell you how much that lucky event shaped my professional and overall attitude. From then on, whenever I faced a problem, I would always search for that little “lottery ticket.”  I know it’s there; I just need to find it, and that will turn the problem on its head. From then on, I saw the opportunity in any problem, and the satisfaction of solving the right one.

Problem-driven or purpose-driven solution?

The customer was straight to the point. They had a huge SAS data table that needed to be updated on a daily basis, at about 10 am when all the primary data sources were finished refreshing. The problem was that updating that SAS table was taking about 30 minutes, during the entirety of which the table was not available to their business analysts, as it was locked by the updating process. This created a 30 minute downtime for the business analysts. The customer formulated the problem as “too long a data table update,” with the suggested solution being “to optimize the process of updating the table”.

Sounds logical, doesn’t it? But we should never take for granted the customer’s vision of the problem, especially when it’s accompanied by a suggested solution.  In most cases, this will lead to solving the wrong problem. Just keep your mind open and help your customers to keep their minds open too.

Problems do not exist by themselves without a purpose, a goal. The customer is the ultimate authority of setting those goals and purposes. Get that purpose to be articulated first before jumping to formulating a problem. Picture this dialogue:

- What is your goal - to minimize the data table refresh time or to minimize the business analysts’ downtime?
- Aren’t those the same?
- Let’s assume not.
- But how?
- Let’s not ask “how” just yet. If I reduce your business analysts’ downtime to zero, would you care if your data table is being updated 30 minutes or 15 minutes?
- No. Well, then our goal is to minimize the business analysts’ downtime.

We just made a huge leap – from an imaginary problem to a purpose. In fact, the duration of the table update is not important as long as the analysts’ downtime is minimized. So the real problem is the downtime, which we need to reduce or, ideally, eliminate.

When the real problem is well-defined, finding a solution is a matter of practice. As Albert Einstein once said, “If I were given one hour to save the planet, I would spend 59 minutes defining the problem and one minute resolving it.”  It makes a lot of sense, since spending all your time on solving the wrong problem does not move you an iota in the right direction.

Creative problem-solving

There are many proven methods and techniques of creative problem-solving. One of them is using analogies. To arrive at “how” (the solution), let’s consider the following analogy. Suppose we want to renovate a restaurant but we don’t want to stop its operations and lose its patrons during that renovation. What would we do? An ideal solution would be to build a new restaurant building next to the old one. After we finish the construction, we would simply take the sign from the old building and affix it to the new one. Problem solved. We can demolish or reuse the old building.

For SAS data tables that would translate into this. Let’s say our data table is called ANALYTICS. Instead of rebuilding it “in place,” we build a new table, let’s call it ANALYTICS_NEW. We can spend 30 minutes or 15 minutes doing this – it does not matter. After we finish table-building we can just make two quick renames:

  1. Rename  ANALYTICS table to ANALYTICS_OLD;
  2. Rename ANALYTICS_NEW table to ANALYTICS.

Using CHANGE statement of PROC DATASETS these 2 renames would take about 0 seconds – that is our new downtime. After that we can delete the ANALYTICS_OLD table. Looks like we just found an ideal purpose-driven solution to effectively eliminate (not just reduce) the downtime.

I can foresee some inquisitive minds asking why we can’t do the table update in place without locking it out by using SAS/SHARE software.  Yes, we can, and that would be a “refrigerator” solution to an “ice-cream maker” problem. Unfortunately, not all the refrigerators are won in a lottery.

Are you solving the wrong problem? was published on SAS Users.

5月 202016
 

ProblemSolversPROC FREQ is often the first choice when you want to generate basic frequency counts, but it is the last choice when it is compared to other statistical reporting procedures. People sometimes consider PROC FREQ last because they think they have little or no control over the appearance of the output. For example, PROC FREQ does not allow style options within the syntax, which the REPORT and TABULATE procedures do allow. Also, you cannot control the formats or headings with statements in the procedure step.

Sometimes, a simple frequency (via a one-way table) is all you want, and you don’t want to have to create an output data set just to add a format. A one-way table in PROC FREQ is unique also in that it includes a cumulative count and percent. These calculations cannot be done in other procedures without additional code or steps. However, there is a simple way to make some basic modifications to your table. By adding a PROC TEMPLATE step to modify either the Base.Freq.OneWayFreqs or the Base.Freq.OneWayList table template, you can change the formats of the statistics, move the label of the variable, change the labels of the statistics, and suppress the Frequency Missing row that appears below the table. These changes apply to all output destinations, including the traditional listing output.  You can also use PROC TEMPLATE to make small modifications to the nodes that are generated by one-way tables in the table of contents for non-listing ODS destinations.

Customize the Formats and Labels for Statistics

If you want to modify the format for the statistics Frequency, Cumulative Frequency, Percent, and Cumulative Percent, you can use the FORMAT= option in PROC TEMPLATE. This option accepts SAS formats and user-defined formats. If you want to change the statistic heading, you can use the HEADER= option.

The following example uses both the FORMAT= and the HEADER= statements to format the statistics values and to change the Frequency heading to Count. This example also changes the Percent heading to Pct.

proc format;
     picture pctfmt (round) other='009.9%';
run;

proc template; 
     edit Base.Freq.OneWayList;
        edit frequency;  
           header='Count'; 
           format=comma12.;
        end;  

        edit cumfrequency;
           format=comma12.; 
        end;
edit percent;
header='Pct';
format=pctfmt.;
end;
   
edit cumpercent;
format=pctfmt.; 
end; 
end;
run;                                                                                  

data class;
     set sashelp.class;
     wgt=1000;
run;
 
proc freq data=class;
     tables age;
     weight wgt; 
run;

This code generates the following table:

FREQ procedure's one-way tables01

Move the Variable Label

If a variable has a label, it is centered, by default, at the top of the table, and the variable name appears above the column of values.

FREQ procedure's one-way tables02

If you want to use the label instead of the name above the column, you can edit the HEADER= value in PROC TEMPLATE, as shown in the following example:

proc template;
   edit Base.Freq.OneWayList;             
        edit h1; 
                /* Set text to a blank instead of VARLABEL. */ 
             text " ";
        end;                                                
                                                                                                  
        edit FVariable;                                           
           just=varjust;
           style=rowheader;
           id;
           generic;
           header=varlabel;
        end;
     end;
  run;

proc freq data=sashelp.class;
   label age='Age in Years';
   tables age;
run;

This code generates the following table, which replaces the default location of the label with blank text and moves the label so that it is above the column of the variable's values, as shown in this example:

FREQ procedure's one-way tables03

Suppress the Frequency Missing= Row

If a variable has missing values and you do not include the MISSING option in the TABLES statement of PROC FREQ, the output includes the frequency of missing values (Frequency Missing=) below the table with the number of missing values.

The following table shows the default output:

FREQ procedure's one-way tables04

To suppress this line without including missing values in the table output, you can modify the table template, as follows:

data test;
input c1 $;
cards;
a
b
.
;
run;

proc template;
     edit Base.Freq.OneWayFreqs; 

   /* This is the default setting:                     */ 
   /*  define f2;                                      */
   /*     text "Frequency Missing =" fMissing -12.99; */
   /*     print = pfoot2;                              */
   /*  end;                                            */
        edit f2; 
           text " ";
        end;
     end;
run;

proc freq data=test;
   table c1;
run;

The table that is generated by this code does not contain the Frequency Missing= row at the bottom of the table.

FREQ procedure's one-way tables05

Customize the Table of Contents

For a one-way table, the default table of contents looks like this:

FREQ procedure's one-way tables06

To control the first node, The Freq Procedure, you can use the ODS PROCLABEL statement.  You can change the text of this node, but you cannot eliminate the node. You can remove the One-Way Frequencies node using PROC TEMPLATE with the CONTENTS= statement. The following example changes the text of the The Freq Procedure node, and it eliminates the One-Way Frequencies node.

proc template;
   edit Base.Freq.OneWayFreqs;
      contents=off;
   end;
run;

ods listing close;
ods pdf file='test.pdf';

ods proclabel='My One-Way Table';
proc freq data=sashelp.class;
tables age;
run;

ods pdf close;
ods listing;

Here is the modified table of contents:

FREQ procedure's one-way tables07

(For additional customization of the table of contents, you need to consider using the DOCUMENT procedure.  The SAS Global Forum 2011 paper Let's Give 'Em Something to TOC about: Transforming the Table of Contents of Your PDF File, by Bari Lawhorn, contains useful information and examples that illustrate how to use PROC DOCUMENT, PROC TEMPLATE, and other SAS statements and options to customize your TOC.)

To restore the default table template, run the following code:

proc template;
delete Base.Freq.OneWayFreqs;
delete Base.Freq.OneWayList;
run;

The examples above show how you actually can customize PROC FREQ one-way tables, with a little help from PROC TEMPLATE. PROC TEMPLATE offers several methods that make it easy to customize one-way, frequency-count tables that enhance table format and appearance.
tags: Problem Solvers, PROC FREQ, SAS Programmers

Seeing the FREQ procedure's one-way tables in a new light was published on SAS Users.

5月 182016
 

EnvironmentManagerMy previous blog discussed the SAS Environment Manager Report Center, and talked about its organization, and how to start using some of the prompts to get the reports you want. The next step is to learn to use some of the example reports provided to help you design your own, production-level reports.

It’s helpful to distinguish ad-hoc, or exploratory reporting, vs. standardized, production-style reports, which might be run daily, weekly, or any other number of times. You can experiment with many ad-hoc reports and create hundreds of variations and different reports “on the fly,” just by manipulating the various parameters provided within the Report Center interface. Perhaps more importantly, you can use knowledge gained from this process to design more permanent production reports, using some of the provided examples as “templates.”

First, you need to understand the basic structure: All the SAS Environment Manager Reports are SAS Stored Processes available from the Stored Process web application. Most of them accept user prompts that specify many details about the data to include and the way the output should appear. Then, the stored process passes these parameters to a small set of reporting macros, which perform the data manipulations and produce the actual report using well-known SAS Procedures.

In addition to all the pre-defined reports, there are two sets of example reports specifically intended to be used as templates from which you can develop your own custom report. Often, this only requires a few changes to parameters being passed to the reporting macro. These example reports are located here:

Under Products>SAS Environment Manager>Custom
Under System>Applications>SAS Environment Manager>Ad Hoc Reporting

Creating custom reports from the Data Mart01

In the first set, the Example2 Call EV Macros with prompts is likely of most interest. This report demonstrates how to call two of the most useful reporting macros:

%evtmpplate creates a wide variety of plots using PROC TEMPLATE
%evtabrpt  creates a wide variety of data tables using PROC TABULATE

The second set in Ad Hoc Reporting has a series of reports that use many additional reporting macros. Notice that many of the reports include the reporting macro name they showcase, which matches the corresponding stored process filename, in the report name itself. Thus you can quickly locate the most important files being used in the running of the report based on the report’s name. Here’s a table showing the most useful reporting macros:

Creating custom reports from the Data Mart02

Armed with this information, here’s the approach to take to create your own custom reports in the Report Center:

  1. Decide what you want to report on, and what general type of report you want to create (table, crosstab, bar chart, line graph, heat map, etc.)
  2. Try to find existing reports in the Report Center that may already do what you want, or are at least based on the data that you want to use
  3. Determine which Data Mart data set you want to report from, based on Steps 1 and 2
  4. If none of the existing reports provide what you want, try experimenting with all the parameters available in the Ad-Hoc reports section, (ACM, ARTIFACT, KITS libraries only) using the report type you want (plot, chart, tabulate, etc.) to determine the example report to try.  Most of these reports allow you a wide variety of choices, including:
    o  data set
    o  main analysis variables
    o  class variables, or BY variables
    o  type of plot, plus many options and details about the plots
    o  data subsetting by variable (WHERE clause), dates
    o  output formatting
    o  titles/footnotes/ODS style, etc.
  5. When running the Ad Hoc reports, first specify the data set and analysis variable(s) that you want, and then experiment with various other parameters. Open the file containing the macro code that you are using–the reporting macros are fully documented with all options and parameters accepted. Use the table above to help you find the correct file(s).
  6. Once you get the report that you want from the Ad Hoc section, then you can use the information you’ve gained to create a more permanent version. Here’s an overview of the steps to create a permanent custom report:

Caution before proceeding: Be careful not to modify any of the stored process files that are part of the Report Center. If you want to modify one of the stored processes, first make a copy and give it a new name. Then simply refer to the new name in that stored process’ metadata and modify the copy.  Or, you can put the stored process code directly in metadata instead of a file, in which case you don’t have to alter any Report Center files.

  1. Using SAS MC, copy the existing Ad Hoc report to a new metadata location, and rename it.
  2. Make a copy of the stored process file called by the report, and rename it.
  3. Edit the new (copied, renamed) stored process code, and you will see the reporting macro(s) being called.
  4. Modify the parameters on any called reporting macros with the new parameters you want, based on what you learned in the previous steps.  For more guidance, open the *.sas file containing the reporting macro, and follow the detailed documentation that’s included with each macro.
  5. Chances are that it will require a few tries before you get it exactly right, but it’s nice to see how much of the work has already been done for you with the existing examples.

More information on the steps to add a report to Report Center.

 

tags: SAS Administrators, SAS Environment Manager, SAS Professional Services

Creating custom reports from the Data Mart was published on SAS Users.

5月 162016
 

Highlights from SAS Global Forum 2016SAS Global Forum 2016 recently concluded and by all measures it was a huge success. With more than 5,000 attendees, hundreds of crazy-good talks and countless networking and learning opportunities, it very likely was the best SAS users’ conference yet. To put a bow on this year’s event, and help those who were unable to attend enjoy some of the great content, I’ve put together a few highlights from SAS Global Forum 2016, with some helpful links, videos and several additional resources to explore.

Before we get started, though, I wanted to share a big thank you to all the volunteers who made SAS Global Forum 2016 possible. You don’t become the world’s largest analytics conference without substantial support from the user community and an outstanding roster of hard-working and dedicated volunteers.

This year, more than 450 users, led by conference chair Jenn Waller, volunteered countless hours of time and tons of energy to make SAS Global Forum such a success. Within this amazing group, we wanted to highlight the inaugural Global Forum Service Award recipients. These individuals were recognized for their dedication and commitment, not only to this year’s event, but for having served at least five years as a team lead or a section chair at SAS Global Forum. The award recipients are:

  • Maribeth Johnson – 12 years
  • Diana Suhr – 12 years
  • Nancy Brucken – 11 years
  • Pete Lund – 11 years
  • Sue Douglass – 9 years
  • Mary Anne DePesquo – 8 years
  • Christianna Williams – 8 years
  • Stephanie Thompson – 5 years

To learn more about the benefits of volunteering, read SAS Global Forum 2016 Chair Jenn Waller’s blog, Can you help? Volunteer opportunities aplenty at SAS Global Forum 2016. There are a number of opportunities for you to get involved in next year’s conference or one of SAS’ Regional User Group events this fall.

Now for the recap. Here are some highlights from SAS Global Forum and some helpful links and resources in case you weren’t able to attend.

Exciting announcements

The conference began with a bang. At Opening Session, Dr. Goodnight introduced several new offerings from SAS. Perhaps the most exciting was the announcement of SAS Viya, the next-generation high-performance analytics and visualization architecture from SAS. We also introduced other new products, including SAS Analytics for IoT and SAS Customer Intelligence 360.

Check out archived video of SAS Global Forum 2016 Opening Session.

Outstanding speakers

There were hundreds of top-notch session and keynote talks, but there were two in particular that really resonated with me.

The first was Arianna Huffington’s talk, Redefining Success: The Third Metric That Can Benefit Your Bottom Line. In her presentation, the co-founder, president and editor-in-chief of the Huffington Post Media Group, refuted the popular belief that in order to be successful we need to burn ourselves out. Instead, she challenged the audience to take care of themselves and their relationships and watch success follow.

Susan Cain’s keynote presentation was another one that hit the mark for me. The author of Quiet: The Power of Introverts in a World That Can’t Stop Talking, Cain discussed how extroverts can dominate the workplace simply because of their predisposition to speak up, not because of their great ideas. (Yes, sometimes a problem I have!) She had some great advice for how introverts and extroverts could work together more effectively, resolve conflict and how to benefit from each other’s strengths.

View Huffington’s talk in its entirety

View Cain’s talk in its entirety

View other keynote and select breakout presentations on the SAS Video portal

Great networking

For attendees, SAS Global Forum offered a number of awesome opportunities to interact with and learn from other SAS professionals. SAS Global Forum was covered extensively on SAS Blogs (aggregated here), the SAS Support Communities, the SAS Twitter Account (hashtag #SASGF), SAS Users LinkedIn group, and the SAS Users Group Facebook page. But, building a broader network of SAS peers, doesn’t have to end with the conclusion of the event. Whether you attended the conference or not, connections with other SAS users can continue via social media. The channels mentioned above are great follows if you’re looking to get the latest scoop from SAS, start a conversation with your SAS peers, or share tips and tricks.

You might also want to check out this fun blog: Ten SAS Global Forum speakers to follow on Twitter. Now there’s an awesome network for you to follow!

Valuable papers

If you’re a SAS user, there’s nothing better than SAS Global Forum presentations. This year had a record number of presentations from users and SAS subject matter experts. Even if you attended the event, you only got a taste of all that great content. The good news: conference papers (and more) are available on the SAS Global Forum Proceedings webpage. Here you can view all papers or filter by topic, session type, skill level, job role, industry, or product. Heck, you can also download all proceedings if you’d like!

Extensive collection of archived video

During the event, SAS provided dozens of hours of live video streaming. Live coverage included Opening Session, all the keynote talks, select breakouts, Tech Talks, interviews with SAS executives and developers, and so much more. If you didn’t go to SAS Global Forum, or did but missed a few sessions, you can view some of the great content via SAS’ live stream archive.

So there you have it. A few highlights from SAS Global Forum 2016. If you attended SAS Global Forum this year, thank you. If you didn’t make it to Las Vegas, enjoy these resources and we hope you can join us next year in Orlando!

tags: SAS Global Forum

Couldn’t attend SAS Global Forum 2016? Here are a few highlights, resources and thank-yous was published on SAS Users.

5月 142016
 

Trusted data is key to driving accurate reporting and analysis, and ultimately, making the right decision. SAS Data Quality and SAS Data Management are two offerings that help create a trusted, blended view of your data. Both contain DataFlux Data Management Studio, a key component in profiling, enriching monitoring, governing and cleansing your data. Clustering, or grouping similar names or addresses together, is one data quality activity that helps to reduce the number of duplicates in a given data set. You do this by using fuzzy matching to group similar names or addresses together. As an example, a marketing analyst might want to remove duplicate customer names or addresses from a customer list in order to reduce mailing costs.

Have you ever wondered how the cluster results would differ if you changed the match code sensitivity for one of your data columns, or removed a column from one of your cluster conditions or added a new cluster condition? Well, wonder no more!

The Cluster Diff node in DataFlux Data Management Studio compares the results of two different Clustering nodes based on the same input data.  This is useful for comparing the results of different cluster conditions and/or different match code sensitivities.

All records from the input set must be passed to both Clustering nodes and both Clustering nodes must pass out all their data in the same order for this comparison to work. To summarize, in both Clustering nodes, you must select the All clusters output option and you cannot use the Sort output by cluster number option.

DataFlux Data Management Studio with Cluster Comparison

Clustering Node Properties

The results of both Clustering nodes are then fed into the Cluster Diff node.  In order to perform the comparison, the Cluster Diff node must know the unique identifier for the input records (Record ID) and the Cluster number that is returned from the respective Clustering node.

DataFlux Data Management Studio with Cluster Comparison

Cluster Diff Node Properties

The Diff type value describes the type of change when performing the cluster number comparison between the two Clustering nodes. The possible values for Diff type include COMBINE, DIVIDE, NETWORK, and SAME which is represented as a period (.). When comparing the results of the two Clustering nodes the results are reviewed as a Diff set.  Within a Diff set:

  • If the records were in different clusters on the “left table” and in the same cluster on the “right table,” then its Diff type is COMBINE.
  • If the records were in the same cluster on the “left table” and in different clusters on the “right table,” then its Diff type is DIVIDE.
  • If the records were in same cluster on the “left table” and in the same cluster on the “right table,” then its Diff type is “.” (SAME).
  • If when comparing the “left table” cluster to the “right table” clusters at least one record is added to the cluster AND at least one record is removed from the cluster, then its Diff type is NETWORK.
DataFlux Data Management Studio with Cluster Comparison

Cluster Diff Node Results

The Cluster Diff node is not a node that is typically used in a production matching job. However, it is a node that is useful in helping you compare and contrast different match code sensitivities and/or cluster conditions that enable you to achieve the best matching results for your data set.

 

tags: data management, DataFlux Data Management Studio, SAS Professional Services

Improving matching results in DataFlux Data Management Studio with cluster comparison was published on SAS Users.

5月 122016
 

SASMobile_BI_FAQSAS Visual Analytics users view and interact with reports on their desktop computers or laptops. Many, however, have never heard of the SAS Mobile BI app or how it extends the viewing and interactive capabilities of their reports to mobile devices. SAS Mobile BI app is simple to download to a mobile device, and you can immediately view some sample SAS Visual Analytics reports from within the app.

If your organization has deployed SAS Visual Analytics, but is not taking advantage of the ability to view and interact with reports via mobile devices, I urge you to consider it.  Almost every type of interaction that you have with a SAS Visual Analytics report on your desktop – you can do the same with reports viewed in SAS Mobile BI!

It’s worth noting that having SAS Visual Analytics in your organization is not a requirement for downloading this nifty, free app on to your Android or iOS mobile devices.  Once you download the app, you can view and interact with a wide spectrum of sample SAS Visual Analytics reports for different industries.

Like what you see?

If so, talk with the SAS Visual Analytics administrator in your organization and ask them to enable support for viewing your SAS Visual Analytics reports in the SAS Mobile BI app.

To give you a little more guidance, here are some of the most frequently asked questions about SAS Mobile BI.

Do we need a special license to use SAS Mobile BI?

Absolutely not! It is a free app from SAS that is available to anyone who wants to download it on to an Apple or Android device and view reports and dashboards.

Do we need to have SAS Visual Analytics in our organization to use the app?

No, you don’t.

When the app is downloaded to your device, sample charts and reports are made available to you by SAS via a server connection to the SAS Demo Server.

How do we get this free app?

You can download the free app to your Apple or Android device from:

Apple iTunes Store
Google Play

After downloading the app, what are some things we can do with it?

Well, as soon as you download the app and open it, you have a connection established to the SAS Demo Server which hosts a variety of sample reports with data associated with different industries. These reports give you an idea of the various features that are supported by the app. Here is a snapshot of the different folders that contain sample reports for you to browse and view.

 

SASMobileBIFAQ1

Here is the Portfolio where subscribed reports are available in the form of thumbnails that can be tapped to open them.

SASMobileBIFAQ2

If we have reports in our organization that were created with SAS Visual Analytics Designer, can we view those reports in this app?

Absolutely, yes!

The same reports that you view in your web browser on a desktop can be viewed in SAS Mobile BI by using your Apple or Android devices (both tablets and smartphones).  Simply define a connection to your company server and browse for reports. Access to reports from SAS Mobile BI is granted by your SAS Administrator. Report access can be based upon various Capabilities and Permissions.

Live data access requires either a Data or a Wi-Fi connection, and your company may require VPN access or other company-specific security measures. Contact your SAS Visual Analytics Administrator. Also, be sure to mention that there is a comprehensive Help menu within the app that provides answers to queries they might have.

What type of features are available in reports that are viewed from SAS Mobile BI?

Typically, almost all of the features that are available in the desktop-based reports are supported in the app. Drilling, filtering, brushing, display rules, linking, info windows – are just some of the key features available to you when you interact with reports via this app.

I hope you’ve found this blog and the accompanying FAQ helpful. In my next blog I’ll cover some basic security features we’ve built into the app to protect user data.

tags: SAS Mobile BI, SAS R&D, SAS Visual Analytics

SAS Mobile BI FAQ was published on SAS Users.