6月 172020
 

Photo by AbsolutVision on Unsplash

As a SAS user for most of my professional career and SAS employee for eight years, I thought I had a pretty good idea of what SAS offers and how SAS® Analytics can solve problems. Yet even I can experience an "aha" moment when I learn something new about SAS and I wish I had known before.

My most recent "aha" moment came a few months ago, right as the Covid-19 pandemic started to unfold. I learned about the SASEFRED interface engine, a component of SAS/ETS® software that allows you to retrieve a wide range of economic and financial data from the Federal Reserve Economic Data (FRED) site. Hosted by the Federal Reserve Bank of St. Louis, FRED is a treasure trove of about 765,000 US and international times series data reported at the national, state and county levels.

Being able to retrieve data from FRED directly into SAS has been extremely handy as I've developed interactive dashboards to capture key economic and financial indicators, such as unemployment claims, confidence sentiment, stock market index, volatility, etc. My SAS US Public Sector team's focus on this is part of a bigger effort to come alongside customers since the pandemic began. Our goal is to help all levels of government quickly set up analytical environments and provide timely situational awareness and analytical services. We're glad we're positioned to help public officials address public health and economic consequences of the pandemic.

How to get FRED data

SAS FRED documentation is self-explanatory. The first step is to obtain a unique FRED API key on the FRED site: https://api.stlouisfed.org/api_key.html. Once that’s done, you are off to the race.

Below is the snippet of the code that I used to pull unemployment data into SAS with SASEFRED.

options validvarname=any
   sslcalistloc="/opt/sas/viya/config/etc/SASSecurityCertificateFramework/cacerts/trustedcerts.pem";
 
libname _all_ clear;
libname mylib "/opt/sas/viya/config/data/cas/default/public"; /** Folder for final datasets **/
libname fred "/opt/sas/viya/config/data/cas/default/public/fred"; /** Folder for intermediate datasets **/
 
/** Ingest FRED Data **/
 
libname fred sasefred "/opt/sas/viya/config/data/cas/default/public/fred"
   OUTXML=UnemploymentClaims
   AUTOMAP=replace
   APIKEY='XXXXXXXXXXXXXXXXXXXX'  /** please request your API at this site https://api.stlouisfed.org/api_key.html **/
   IDLIST='ICSA,ICNSA,IC4WSA,CCSA,CCNSA'
   START='2008-01-01'
   END='2020-06-30'
   freq='w'
   ;
data mylib.UnemploymentClaims;
   set fred.UnemploymentClaims;
run;
proc print data=mylib.UnemploymentClaims;
run;

The output of the above code execution is the table ‘UnemploymentClaims’ with information on weekly reported initial unemployment claims, four-week moving average claims, and continued claims (see Figure 1 below).

Figure 1

An interactive dashboard with FRED data

The work was done on SAS® Viya, the next generation of SAS Analytics. SAS Viya offers a wide range of robust analytical capabilities, including visual data exploration and reporting. With data ingested into SAS Viya, I am now able to quickly develop an interactive economic dashboard with relevant indicators that will automatically update as the new economic data is reported – all thanks to SASEFRED (see image below).

Check the resources below to learn more. Hope you found this post relevant and useful. Feel free to reach out if you have any questions!

SGF PAPER | Using SAS® Forecast Server and the SASEFRED Engine to Enhance Your Forecast YOUTUBE | Extracting a Common Time Trend in SAS/ETS

How to access Federal Reserve Economic Data (FRED) with SASEFRED in SAS/ETS® software was published on SAS Users.

6月 172020
 

A previous article shows how to use the iml action to read a CAS data table into an IML matrix. This article shows how to write a CAS table from data in an IML matrix. You can read an overview of the iml action, which was introduced in SAS Viya 3.5.

Write a CAS data table from the iml action

Suppose you are running a program in the iml action and you want to write the result of a computation to a CAS data table. The simplest way is to use the the MatrixWriteToCAS subroutine. The syntax of the function is
call MatrixWriteToCAS(matrix, caslib, TableName, colnames);
where

  • matrix is the matrix that you want to save.
  • caslib is a caslib that specifies the location of the table. A blank string means "use the default caslib." Otherwise, specify the name of a caslib. For example, if the CAS table is in your personal caslib, specify 'CASUSER(userName)', where userName is your login name. For more information, see the CAS documentation about caslibs.
  • TableName is a string that specifies the name of the CAS table.
  • colnames is a character vector, whose elements specify the columns in the CAS table.

For simplicity, the following call to the iml action defines a matrix and then writes it to a table. In practice, the matrix would be the result of some computation:

proc cas;
loadactionset 'iml';   /* load action set (once per session) */
source WriteMat;
   corr = {1.0000  0.7874 -0.7095 -0.7173  0.8079,
           0.7874  1.0000 -0.6767 -0.6472  0.6308,
          -0.7095 -0.6767  1.0000  0.9410 -0.7380,
          -0.7173 -0.6472  0.9410  1.0000 -0.7910,
           0.8079  0.6308 -0.7380 -0.7910 1.0000 };
   varNames = {'EngineSize' 'Horsepower' 'MPG_City' 'MPG_Highway' 'Weight'};
   call MatrixWriteToCas(corr, ' ', 'MyCorr', varNames);  /* Write to CAS data table in default caslib */ 
endsource;
iml / code=WriteMat;      /* call the iml action to run the program */
run;

The program writes the matrix to a CAS table named MYCORR. You can call the columnInfo action (which is similar to PROC CONTENTS) to verify that the CAS table exists:

proc cas;
   columnInfo / table='MyCorr';
run;

The output of the columnInfo action shows that the MYCORR data table was created. Notice that in addition to the five specified variables, the CAS table contains a sixth variable, called _ROWID_. This variable is automatically added by the MatrixWriteToCAS subroutine.

How did that variable get there? As explained in the previous article, CAS data tables do not have an inherent row order, but matrices do. The iml action includes the _ROWID_ variable in the output data table in case you need to read the data back into an IML matrix at a later time. If you do, the rows of the new matrix will be in the same order as the rows of the CORR matrix that created the data table. For a correlation matrix, this is important because (by convention) the order of the rows corresponds to the order of the columns. Preserving the row order ensures that the matrix has 1s along the main diagonal.

In summary, you can use the MatrixWriteToCAS subroutine to write a matrix to a CAS table. Not only does the call create a CAS table, but it augments the table with information that can recreate the matrix in the same row order. By the way, if you want to save mixed-type data, you can use the TableWriteToCAS subroutine to write a SAS/IML table to a CAS table.

Further reading

The post Write a CAS data table by using the iml action appeared first on The DO Loop.

6月 172020
 

SAS users help drive innovation, providing feedback on SAS products through various mediums, including customer advisory boards, communities, SASware ballot ideas and SAS Global Forum. The SAS User Feedback Award is presented annually to a user whose recommendations and insights lead to significant improvements to SAS software. This year SAS [...]

SAS User Feedback Award winner helps drive innovation, shape SAS® 360 Engage software was published on SAS Voices by Shadi Shahin

6月 172020
 

It’s official: NASA no longer builds spaceships. They’ve outsourced that task. According to NASA administrator Jim Bridenstine, "We're going with commercial partners. NASA is not purchasing, owning and operating the hardware. We're buying the service." Why? Because NASA needs to focus on exploring space, not building the transportation to get [...]

Are you ready for true cloud native computing? Meet SAS Viya 4 was published on SAS Voices by Keith Renison

6月 172020
 

It’s official: NASA no longer builds spaceships. They’ve outsourced that task. According to NASA administrator Jim Bridenstine, "We're going with commercial partners. NASA is not purchasing, owning and operating the hardware. We're buying the service." Why? Because NASA needs to focus on exploring space, not building the transportation to get [...]

Are you ready for true cloud native computing? Meet SAS Viya 4 was published on SAS Voices by Keith Renison

6月 172020
 

Digital transformation continues to change our relationship with technology. As part of this change, the world is transforming from one of technology-literate people to one of people-literate technology. What do I mean by people-literate technology? I mean smart, automated, reliable, explainable decision systems that operate at scale. Analytics and AI [...]

SAS, Microsoft bring analytics to everyone, everywhere was published on SAS Voices by Oliver Schabenberger

6月 152020
 

A previous article compares a SAS/IML program that runs in PROC IML to the same program that runs in the iml action. (You can read an overview of the iml action.) The example in the previous article was very simple and did not read or write data. This article compares a PROC IML program that reads a SAS data set to a similar program that runs in the iml action and reads CAS tables. The iml action was introduced in SAS Viya 3.5.

PROC IML and SAS data sets

An important task for statistical programmers is reading data from a SAS data set into a SAS/IML matrix. SAS/IML programs are often used to pre-process or post-process data that are created by using other SAS procedures, and SAS data sets enable the output from one procedure to become the input to another procedure.

To simplify matters, let's focus on reading numerical data into a SAS/IML matrix. In PROC IML, the USE and READ statements are used to read data into a matrix. The following program is typical. Several numerical variables from the SasHelp.Cars data are read into the matrix X. The call to the CORR function then computes the correlation matrix for those variables:

proc iml;
varNames = {'EngineSize' 'Horsepower' 'MPG_City' 'MPG_Highway' 'Weight'};
use Sashelp.Cars;              /* specify the libref and data set name */
read all var varNames into X;  /* read all specified variables into columns of X */
close;
 
corr = corr(X);
print corr[r=varNames c=varNames format=7.4];
quit;

The USE and READ statements read SAS data sets. However, there are no SAS data sets in CAS, so these statements are not supported in the iml action. Instead, the iml action supports reading a CAS table into a matrix. Whereas the READ statement in PROC IML reads data sequentially, the iml action reads data from a CAS table in parallel by using multiple threads.

The rows in a CAS table do not have a defined order

Before showing how to read a CAS table into a matrix, let's discuss a characteristic of CAS tables that might be unfamiliar to you. Namely, the order of observations in a CAS table is undefined.

Recall that one purpose of CAS is to process massive amounts of data. Large data tables might be stored in a distributed fashion across multiple nodes in a cluster. When rows of data are read, they are read by using multiple threads that execute in parallel. A consequence of this fact is that CAS data tables do not have an inherent order. If this sounds shocking, realize that the order of the observations does not matter for most data analysis. For example, order does not affect descriptive statistics such as the mean or percentiles. Furthermore, any analysis that uses the sum-of-squares crossproduct matrix (X`*X) is unaffected by reordering the observations. This includes correlation, regression, and a lot of multivariate statistics (for example, principal component analysis).

Of course, for some analyses (such as time series) and for some matrix computations, the order of rows is important. Therefore, the iml action has a way to ensure that the rows of a matrix are in a specific order. If the CAS table has a variable with the name _ROWID_ (or _ROWORDER_), then the data rows are sorted by that variable before the IML matrix is created. This happens automatically when you use the MatrixCreateFromCAS function, which is discussed in a subsequent section. (The _ROWID_ is used only to sort the rows; it does not become a column in the matrix.)

You can use the iml action to read any CAS table. If you read a table that does not contain a _ROWID_ variable, the order of rows in the data matrix might change from run to run.

Upload a data set from SAS to CAS

There are several ways to upload a SAS data set into a CAS table. This example uses the DATA step. By using the DATA step, you can also add a _ROWID_ variable to the data, in case you want the rows of the IML matrix to be in the same order as the data set.

I assume you have already established a CAS session. The following statements use the LIBNAME statement in SAS to create a libref to the active caslib in the current CAS session. You can use this libref to upload a data set into a CAS table:

libname mycaslib cas;    /* libref to the active caslib in the current CAS session */
data mycaslib.cars;      /* create CAS table named CARS in the active caslib */
   set Sashelp.Cars;
   _ROWID_ + 1;          /* add a sort variable */
run;
 
/* Optional: verify that the CAS table includes the _ROWID_ variable */
proc cas;
   columnInfo / table='cars';
run;

The output of the columnInfo action shows that the cars data table was created and includes the _ROWID_ variable.

How to read a CAS data table in the iml action

To read variables from a CAS data table into a matrix, you can use the CreateMatrixFromCAS function. The syntax of the function is
X = MatrixCreateFromCAS(caslib, TableName, options);
where

  • caslib is a caslib that specifies the location of the table. You can specify a blank string if you want to use the default caslib. Or you can specify a caslib as a string. For example, if the CAS table is in your personal caslib, specify 'CASUSER(userName)', where userName is your login name. For more information, see the CAS documentation about caslibs.
  • TableName is a string that specifies the name of the CAS table.
  • options is string that enables you to read only part of the data. The most common option is the KEEP statement, which you can use to specify all numeric values ('KEEP=_NUMERIC_', which is the default), all character variables ('KEEP=_CHARACTER_'), or to specify certain variables ('KEEP=X1 X1 Y Z').

Recall that matrices in the SAS/IML language are either numeric or character. If you want to read mixed-type data, you can use the TableCreateFromCAS function.

Read a CAS table in the iml action

In the previous sections, we created a CAS table ('cars') that contains the Sashelp.Cars data. You can use iml action and the MatrixCreateFromCAS function to read that data table into a matrix, as follows:

proc cas;
   loadactionset 'iml';    /* load the iml action set (only once per session) */
run;
 
proc cas;
source ReadData;
   KeepStmt = 'KEEP=EngineSize Horsepower MPG_City MPG_Highway Weight';
   X = MatrixCreateFromCas(' ', 'cars', KeepStmt);
 
   corr = corr(X);
   varNames = {'EngineSize' 'Horsepower' 'MPG_City' 'MPG_Highway' 'Weight'};
   print corr[r=varNames c=varNames format=7.4];
endsource;
iml / code=ReadData;
run;

The output is identical to the output from PROC IML and is not shown.

The KEEP= option specifies five numeric variables to read. The first argument to the MatrixCreateFromCAS function is a blank string, which means "use the default caslib." Alternatively, you could specify a caslib such as 'CASUSER(userName)'. The matrix X is the data matrix. Because the CARS table contains a _ROWID_ variable, the rows of X are in the same order as the rows of the Sashelp.Cars data set.

After you read the data, you can write standard SAS/IML programs to manipulate or analyze X. For example, I used the CORR function and the PRINT statement to reproduce the results of the previous PROC IML program.

In the next article, I will show how to use the iml action to write a CAS table that contains data in an IML matrix.

Further reading

The post Read a CAS data table by using the iml action appeared first on The DO Loop.

6月 132020
 

I’m going to be honest. When we decided back in March to pivot our largest event of the year from an in-person event to a virtual one, I wasn’t sure what we were getting ourselves into. I didn’t know how we could replace the in-person networking, the unlimited learning opportunities [...]

6 new ways to connect at virtual events was published on SAS Voices by Jenn Chase

6月 122020
 

A previous article provides an introduction and overview of the iml action, which is available in SAS Viya 3.5. The article compares the iml action to PROC IML and states that most PROC IML programs can be modified to run in iml action. This article takes a closer look at what a SAS/IML program looks like in the iml action. If you have an existing PROC IML program, how can you modify it to run in the iml action?

PROC CAS and the SOURCE/ENDSOURCE block

A feature of SAS Viya is that the actions can be called from multiple languages: SAS, Python, R, Lua, etc. I am a SAS programmer, so in my blog, I will use SAS to call actions.

An action (sometimes called a "CAS action") runs by using the SAS Cloud Analytic Services (CAS). The CAS procedure provides a language (called "CASL," for "CAS Language") that enables you to call CAS actions. The documentation for CASL states that "CASL is a scripting language that you use to prepare arguments for execution of CAS actions, submit actions to the CAS server, and then process action results." In short, PROC CAS enables you to "stitch together" a sequence of action calls into a workflow.

PROC CAS supports a SOURCE/ENDSOURCE block, which defines a program that you can submit to an action that supports programming statements. I will use the SOURCE/ENDSOURCE block to define programs for the iml action. To call the iml action by using PROC CAS, you need to do three things:

  1. Load the iml action set. Any CAS action set that is not auto-loaded must be loaded before you can use it. It only needs to be loaded once per session.
  2. Use the SOURCE/ENDSOURCE block to define the IML program.
  3. Call the iml action to run the program. The syntax to call the action is iml / code=ProgramName

In terms of syntax, a typical program in the iml action has the following structure:

PROC CAS;
loadactionset 'iml';        /* load the iml action set */
source ProgramName;
    < put the IML program here >
endsource;
iml / code=ProgramName;     /* run the program in the iml action */
RUN;

Convert a program from PROC IML to the iml action

The following SAS/IML program defines two vectors and computes their inner product. It then computes the variance of the x data vector. Lastly, it centers and scales the data and computes the variance of the standardized data. This program runs in PROC IML:

PROC IML;
   c = {1, 2, 1, 3, 2, 0, 1};   /* weights */
   x = {0, 2, 3, 1, 0, 2, 2};   /* data */
   wtSum = c` * x;              /* inner product (weighted sum) */
   var1 = var(x);               /* variance of original data */
   stdX = (x-mean(x)) / std(x); /* standardize data */
   var2 = var(stdX);            /* variance of standardized data */
   print wtSum var1 var2;
QUIT;

This program does not read or write any data sets, which makes it easy to convert to the iml action. The following statements assume that you have a license for the SAS IML product in Viya and that you have already connected to a CAS server.

/* Example of using PROC CAS in SAS to call the iml action */
PROC CAS;
loadactionset 'iml';            /* load the action set (once) */
source pgm;
   c = {1, 2, 1, 3, 2, 0, 1};   /* weights */
   x = {0, 2, 3, 1, 0, 2, 2};   /* data */
   wtSum = c` * x;              /* inner product (weighted sum) */
   var1 = var(x);               /* variance of original data */
   stdX = (x-mean(x)) / std(x); /* standardize data */
   var2 = var(stdX);            /* variance of standardized data */
   print wtSum var1 var2;
endsource;
iml / code=pgm;                 /* call iml action to run the 'pgm' program */
RUN;

For this example, the SAS/IML statements are exactly the same in both programs. The difference is the way that the program is submitted for execution. For this example, the program is named 'pgm', but you could also have named the program 'MyProgram' or 'StdAnalysis' or 'Robert' or 'Jane'. Whatever identifier you use on the SOURCE statement, use that same identifier for the CODE= parameter when you call in the action.

Calling the iml action from Python

I use PROC CAS in SAS to call CAS actions. However, I know that Python is a popular programming language for some data scientists, so here is how to call the iml action from Python. You first need to install the SAS Scripting Wrapper for Analytics Transfer (SWAT) package. You can then use the following statements to connect to a CAS server and load the action:

# Example of using Python to call the iml action
import swat # load the swat package
s = swat.CAS('myhost', 12345)    # use server='myhost'; port=12345
s.loadactionset('iml')           # load the action set (once)

As mentioned earlier, you only need to load the action set one time per session. After the action set is loaded, you can call the iml action by using the following. In Python, triple quotes enable you to preserve the indention and comments in the IML program.

m = s.iml(code=
"""
   c = {1, 2, 1, 3, 2, 0, 1};   /* weights */
   x = {0, 2, 3, 1, 0, 2, 2};   /* data */
   wtSum = c` * x;              /* inner product (weighted sum) */
   var1 = var(x);               /* variance of original data */
   stdX = (x-mean(x)) / std(x); /* standardize data */
   var2 = var(stdX);            /* variance of standardized data */
   print wtSum var1 var2;
""")

More realistic examples

In this case, the program did not read or write data. However, a typical IML program reads in data, analyzes it, and optionally writes out the results. This is the first (and most common) difference between a program that runs in PROC IML and a similar program that runs in the iml action. A PROC IML program reads and writes SAS data sets, whereas actions read and write CAS data tables.

The next article shows an example of a PROC IML program that reads and writes SAS data sets. It compares the PROC IML program to an analogous program that runs in the iml action and reads and writes CAS tables.

Further reading

The post Getting started with the iml action in SAS Viya appeared first on The DO Loop.