sas programming

8月 022017
 

In my prior posts (Data-driven SAS macro loops, Modifying variable attributes in all datasets of a SAS library, Automating the loading of multiple database tables into SAS tables), I presented various data-driven applications using SAS macro loops.

However, macro loops are not the only tools available in SAS for developing data-driven programs.

CALL EXECUTE is one of them. The CALL EXECUTE routine accepts a single argument that is a character string or character expression. The character expression is usually a concatenation of strings containing SAS code elements to be executed after they have been resolved. Components of the argument expression can be character constants, data step variables, macro variable reference, as well as macro references. CALL EXECUTE dynamically builds SAS code during DATA step iterations; that code executes after the DATA step’s completion outside its boundary. This makes a DATA step iterating through a driver table an effective SAS code generator similar to that of SAS macro loops.

However, the rather peculiar rules of the CALL EXECUTE argument resolution may make its usage somewhat confusing. Let’s straighten things out.

Argument string has no macro or macro variable reference

If an argument string to the CALL EXECUTE contains SAS code without any macro or macro variable references, that code is simply pushed out (of the current DATA step) and appended to a queue after the current DATA step. As the DATA step iterates, the code is appended to the queue as many times as there are iterations of the DATA step. After the DATA step completes, the code in the queue gets executed in the order of its creation (First In First Out).

The beauty of this process is that the argument string can be a concatenation of character constants (in single or double quotes) and SAS variables which get substituted with their values by CALL EXECUTE for each DATA step iteration. This will produce data-driven, dynamically generated SAS code just like an iterative SAS macro loop.

Let’s consider the following example. Say we need to load multiple Oracle tables into SAS tables.

Step 1. Creating a driver table

In order to make our process data-driven, let’s first create a driver table containing a list of the table names that needed to be extracted and loaded:

/* create a list of tables to extract & load */
libname parmdl '/sas/data/parmdata';
data parmdl.tablelist;
        length tname $8;
        input tname;
        datalines;
ADDRESS
ACCOUNT
BENEFIT
FINANCE
HOUSING
;

This program runs just once to create the driver table parmdl.tablelist.

Step 2. Loading multiple tables

Then, you can use the following data-driven program that runs each time you need to reload Oracle tables into SAS:

/* source ORACLE library */
libname oralib oracle path="xxx" schema="yyy" user="uuu"
 	PASSWORD="{SAS002}ABCDEFG12345678RTUR" access=readonly;
 
/* target SAS library */
libname sasdl '/sas/data/appdata';
 
/* driver table SAS library */
libname parmdl '/sas/data/parmdata';
 
data _null_;
   set parmdl.tablelist;
   call execute(cats(
      'data sasdl.',tname,';',
         'set oralib.',tname,';',
      'run;'));
run;

In order to concatenate the components of the CALL EXECUTE argument I used the cats() SAS function which returns a concatenated character string removing leading and trailing blanks.

When this program runs, the SAS log indicates that after the data _null_ step the following statements are added and executed:

NOTE: CALL EXECUTE generated line.
1   + data sasdl.ADDRESS;set oralib.ADDRESS;run;
2   + data sasdl.ACCOUNT;set oralib.ACCOUNT;run;
3   + data sasdl.BENEFIT;set oralib.BENEFIT;run;
4   + data sasdl.FINANCE;set oralib.FINANCE;run;
5   + data sasdl.HOUSING;set oralib.HOUSING;run;

In this example we use data _null_ step to loop through the list of tables (parmdl.tablelist) and for each value of the tname column a new data step gets generated and executed after the data _null_ step. The following diagram illustrates the process:

Diagram explaining CALL EXECUTE for SAS data-driven programming

Argument string has macro variable reference in double quotes

If an argument to the CALL EXECUTE has macro variable references in double quotes, they will be resolved by the SAS macro pre-processor during the DATA step compilation. Nothing unusual. For example, the following code will execute exactly as the above, and macro variable references &olib and &slib will be resolved to oralib and sasdl before CALL EXECUTE takes any actions:

%let olib = oralib;
%let slib = sasdl;
 
data _null_;
   set parmdl.tablelist;
   call execute (
      "data &slib.."!!strip(tname)!!';'
         "set &olib.."!!strip(tname)!!';'!!
      'run;'
   );
run;

Argument string has macro or macro variable reference in single quotes

Here comes the interesting part. If the argument to CALL EXECUTE has macro or macro variable references in single quotes, they still will be resolved before the code is pushed out of the DATA step, but not by the SAS macro pre-processor during the DATA step compilation as it was in the case of double quotation marks. Macro or macro variable references in single quotes will be resolved by CALL EXECUTE itself. For example, the following code will execute exactly as the above, but macro variable references &olib and &slib will be resolved by CALL EXECUTE:

%let olib = oralib;
%let slib = sasdl;
 
data _null_;
   set parmdl.tablelist;
   call execute('data &slib..'!!strip(tname)!!';'!!
                'set &olib..'!!strip(tname)!!';'!!
                'run;'
               );
run;

Timing considerations

CAUTION: If your macro contains some non-macro language constructs for assigning macro variables during run time, such as a CALL SYMPUT or SYMPUTX statement (in a DATA step) or an INTO clause (in PROC SQL), resolving those macro variable references by CALL EXECUTE will happen too soon, before your macro-generated code gets pushed out and executed. This will result in unresolved macro variables. Let’s run the following code that should extract Oracle tables into SAS tables as above, but also re-arrange column positions to be in alphabetical order:

%macro onetable (tblname);
   proc contents data=oralib.&tblname out=one(keep=name) noprint;
   run;
 
   proc sql noprint;
      select name into :varlist separated by ' ' from one;
   quit;
   %put &=varlist;
 
   data sasdl.&tblname;
      retain &varlist;
      set oralib.&tblname end=last nobs=n;
      if last then call symput('n',strip(put(n,best.)));
   run;
   %put Table &tblname has &n observations.;
%mend onetable;
 
data _null_;
   set parmdl.tablelist;
   call execute('%onetable('!!strip(tname)!!');');
run;

Predictably, the SAS log will show unresolved macro variable references, such as:

WARNING: Apparent symbolic reference VARLIST not resolved.
WARNING: Apparent symbolic reference N not resolved.
Table ADDRESS has &n observations.

SOLUTION: To avoid the timing issue when a macro reference gets resolved by CALL EXECUTE too soon, before macro variables have been assigned during macro-generated step execution, we can strip CALL EXECUTE of the macro resolution privilege. In order to do that, we can mask & and % characters using the %nrstr macro function, thus making CALL EXECUTE “macro-blind,” so it will push the macro code out without resolving it. In this case, macro resolution will happen after the DATA step where CALL EXECUTE resides. If an argument to CALL EXECUTE has a macro invocation, then including it in the %nrstr macro function is the way to go. The following code will run just fine:

data _null_;
   set parmdl.tablelist;
   call execute('%nrstr(%onetable('!!strip(tname)!!'));');
run;

When this DATA step runs, the SAS log indicates that the following statements are added and executed:

NOTE: CALL EXECUTE generated line.
1   + %onetable(ADDRESS);
2   + %onetable(ACCOUNT);
3   + %onetable(BENEFIT);
4   + %onetable(FINANCE);
5   + %onetable(HOUSING);

CALL EXECUTE argument is a SAS variable

The argument to CALL EXECUTE does not necessarily have to contain or be a character constant. It can be a SAS variable, a character variable to be precise. In this case, the behavior of CALL EXECUTE is the same as when the argument is a string in single quotes. It means that if a macro reference is part of the argument value it needs to be masked using the %nrstr() macro function in order to avoid the timing issue mentioned above.

In this case, the argument to the CALL EXECUTE may look like this:

arg = '%nrstr(%mymacro(parm1=VAL1,parm2=VAL2))';
call execute(arg);

Making CALL EXECUTE totally data-driven

In the examples above we used the tablelist driver table to retrieve values for a single macro parameter for each data step iteration. However, we can use a driver table not only to dynamically assign values to one or more macro parameters, but also to control which macro to execute in each data step iteration. The following diagram illustrates the process of totally data-driven SAS program:

Diagram explaining using CALL EXECUTE for SAS data-driven programming

Conclusion

CALL EXECUTE is a powerful tool for developing dynamic data-driven SAS applications. Hopefully, this blog post demonstrates its benefits and clearly explains how to avoid its pitfalls and use it efficiently to your advantage. I welcome your comments, and would love to hear your experiences with CALL EXECUTE.

CALL EXECUTE made easy for SAS data-driven programming was published on SAS Users.

7月 262017
 

Trivial Pursuit, Justin Bieber and Timbits. Some pretty great things have come from Canada, eh? Well, you can go ahead and add expert SAS programmers to that impressive list.

In this video, six Canadian SAS programmers, with more than 115 years of SAS programming experience between them, share some of their favorite, little-known SAS programming tips. You're sure to discover a new trick or two.

The video includes the following tips and more:

  • Standardizing and documenting your SAS program.
  • Creating parameter lookup tables.
  • Declaring your macro variables.
  • Using the Characterize Data task in SAS Enterprise Guide.
  • Data exploration best practices for SAS Enterprise Guide.

Looking for more great tips to help bring your SAS programming skills to the next level? Check out these great resources and learn even more from your SAS peers:

  • SAS Support Communities: Peer-to-peer support for SAS users about programming, data analysis, installation and deployment issues, tips and best practices and a whole lot more.
  • SAS Blogs: Connecting you to people, products and ideas from SAS with technical tips, support information and more.
  • SAS Newsletters: The latest news, tips, tricks and resources from SAS, plus advice and industry knowledge gleaned from top experts and other SAS users.

 

Programming tips from experienced SAS users was published on SAS Users.

7月 182017
 

Datasets are rarely ready for analysis, and one of the most prevalent problems is missing data. This post is the first in a short series focusing on how to think about missingness, how JMP13 can help us determine the scope of missing data in a given table, and how to [...]

The post How severe is your missing data problem? appeared first on SAS Learning Post.

7月 032017
 

If you spend a lot of time in SAS Enterprise Guide (as I do), you probably get to know its features pretty well. But we don't always take the time to explore as we should, so there might be a few golden nuggets of editor knowledge that have escaped you so far. Here are 10 program editor features that I've found essential while writing, editing, and debugging SAS programs. How many of these do you already know and use?

1. Turn on the line numbers

We programmers like to count lines of code. The SAS log often uses line numbers to reference problems in WARNINGs and ERRORs. So of course, you should have line numbers displayed in the program editor. But they aren't on by default. Go to Program → Editor Options and select "Show line numbers" to turn them on.

line numbers

2. Get the tabs out (or leave them in)

Tabs or spaces? Your choice here can have a significant effect on your earning potential, and perhaps even on your love life. Most code editors have options that support your choice, regardless of which camp you choose. SAS Enterprise Guide offers these:

  • Tab size - width of a tab character, represented in number of spaces. Default is 4, but I like to use 2 as it makes my program lines less wide.
  • Insert spaces for tabs - when you press the TAB key, don't add a TAB character but instead add the specified number of space characters.
  • Replace tabs with spaces on file open - a perfect passive-aggressive option when working with team members who disagree with your TAB world view. This option will change TAB characters to spaces when you open the program file. If you must retain the TAB characters...well, my main advice is do not rely on TAB characters in your code file. But if you must for some crazy reason, don't select this option and sign a pact with your teammates for the same.

tabs vs spaces

3. Define abbreviations for commonly used code

The most common code snippet that I reuse is a LIBNAME statement that points to my project data. To save on typing and mistakes, I've saved this as an editor abbreviation. When I begin to type the alias I've assigned to the snippet, the program editor offers to complete it for me. The custom abbreviation is presented along side all of the other built-in syntax suggestions.

abbrev animation

See more about editor abbreviations in this article.

4. Let the editor format your code

As shown in the vigorous "TABS vs spaces" debate, programmers care deeply about how their code is formatted. Individuals and teams adopt various standards for line breaks and indenting, and these are usually particular to the programming language. In general, SAS doesn't care how your code is laid out -- statements are delimited by semicolons, and that's the only cue that SAS needs. However, your teammates (and your future YOU, rereading your code) might appreciate something a little more readable.

Press Ctrl+I to format your entire program, applying some reasonable readability rules to indent code lines with conditionals and looping logic. Or select just a portion of the program and press Ctrl+I to affect a smaller part of the program. You can adjust some of the formatting rules by visiting Program → Editor Options, the Indenter tab.

autoformat code

5. Zoom out for the big picture

Some SAS programs are long -- hundreds (or thousands!) of lines of code. Sometimes it's helpful to get a birds-eye view of your code to understand its structure and to help you navigate. The Zoom feature is super helpful for this. Simply press Ctrl+- (control-minus) until you get the view you need. Press Ctrl++ (control-plus) to zoom back in, or press Ctrl+0 to get to the 100% view.

This trick works for SAS logs as well, and also data sets and ODS output (including text listing, which uses the program editor in a special mode for viewing SAS output).

zoom out

6. Change the program editor font

Want to waste an afternoon? Search the Internet for "best font for programmers" and experiment with all of the results that you find. I discovered Consolas (built into Microsoft Windows) a decade ago, and I've yet to find anything better. I use it for all of my "fixed font" needs: programming, terminal windows, command consoles, etc. But you can choose your own favorite -- just don't feel that you're stuck with the default "Courier" that seems to be standard issue.

Change your font in Programs → Editor Options, Appearance tab. You'll find lots of elements that you can tweak for typeface, size and color.

7. Select columns of content with block selection

Even though column block selection -- also known as "Alt+Select" -- is a standard feature in most advanced text editors, many programmers don't know about it. It's the perfect trick for selecting just a few columns of your text without including the content that's on the rest of the line. In SAS programming, this can be handy for selecting columns of values from the text listing output and pasting somewhere else, such as into a DATALINES block. It takes a little practice to master the Alt+Select, but once you do you'll find all sorts of uses for it. To get started, simply hold down the Alt key and click-drag to highlight a vertical column of text within the editor.

column selection animation

8. Find (and replace) using regular expressions

Regular expressions are a powerful, if confusing, method for finding and replacing text that matches certain patterns. The Find/Find and Replace window in SAS Enterprise Guide supports "Regular expression search" as a checkbox option.

Here's an example. Suppose I wanted to find all occurrences of 3 numbers after the thousands separator (comma) at the end of each data line -- and I wanted to turn those digits into zeros. (I don't know why--but just stick with me here.) A regex pattern to match this is ",\d\d\d\n" (comma, followed by 3 occurrences of numeric digits, followed by a line ending). Here's an animation of this in action.

regex replace animation

For more, select Help→SAS Enterprise Guide help and search for "regular expressions". The help topics contain several examples of useful patterns.

9. Scroll just part of your document using a split view

Do you find yourself scrolling back and forth in your program view? Trying to remember what was in that DATA step at the top of your program so you can reference the proper variable in another part of your code? Instead of dealing with "scrolling whiplash", you can split the program editor view to keep one part of your code always visible while you work on another code segment that's hundreds of lines away from it.

split view

There are several ways to split your view of SAS code, log output, and listing. Check out the article here for details.

10. Break out to your other favorite editor

Please don't tell anyone, but I have a secret: SAS Enterprise Guide is not my default application associated with .SAS files. When I double-click on a .SAS file in Windows Explorer, I like to use Notepad++ to provide a quick view of what's in that program file. Don't get me wrong: I use SAS Enterprise Guide for all of my serious SAS programming work. With syntax suggestions, color coding, built-in DATA step debugger, and more -- there just isn't a better, more full-featured environment. (No, I'm not trying to troll you, diehard SAS display manager users -- you keep using what makes you happy.) But Notepad++ has a deep set of text editing features, and sometimes I like to use it for hardcore find/replace functions, deeper inspection of special characters in my files, and more.

You can launch your program into your other favorite editor from SAS Enterprise Guide. Simply right-click on the program node in your process flow, select Open → Open <program name> with Windows Default. And make sure your other editor is registered in Windows as the default "Open with" action for SAS programs. Note: this trick works only with SAS programs that you've saved locally on your Windows file system.

Open with default

More than editing -- this is your workbench

The program editor isn't just about "editing programs." It's also the launchpad for several other programmer-centric features, such as debugging your DATA step, comparing your SAS programs, viewing program history and source control, and more. If you use SAS Enterprise Guide, take the time to learn about all of its programming features -- you'll become a more productive programmer as a result.

The post Ten SAS Enterprise Guide program editor tricks appeared first on The SAS Dummy.

6月 272017
 

In a previous blog, I demonstrated a program and macro that could identify all numeric variables set to a specific value, such as 999. This blog discusses an immensely useful technique that allows you to perform an operation on all numeric or all character variables in a SAS data set. [...]

The post How to perform an operation on all numeric or all character variables in a SAS data set appeared first on SAS Learning Post.

6月 272017
 

When I teach my Data Cleaning course, the last topic I cover in the two-day course is SAS Integrity Constraints.  I find that most of the students, who are usually quite advanced programmers, have never heard of Integrity Constraints (abbreviated ICs).  I decided a short discussion on this topic would [...]

The post Keeping your data set clean: Integrity constraints appeared first on SAS Learning Post.

6月 062017
 

Wait!  Don't close this window.  I understand that regular expressions can be very complicated (yes, there are many books on the subject), but some basic expressions to test patterns such as zip codes or telephone numbers are not that difficult.  In addition, you can sometimes use Google to search for [...]

The post Using regular expressions to verify the pattern of character data appeared first on SAS Learning Post.

5月 312017
 

SAS programs in Excel, finally!When SAS Add-In for Microsoft Office was first created over a decade ago, SAS programmers were told to check their skills at the door. This new product was for non-programmers only. SAS programmers were invited to contribute to the experience by packaging their code in SAS stored processes, which end users would then run using point-and-click menus. But there was no way to write and run your SAS programs directly in Microsoft Excel or Word or PowerPoint.

This was a comfort to many SAS administrators, who wanted to provide SAS analytics to their end users but didn't want them to have to learn to program. Or perhaps to even allow them to program. But, times have changed. Citizen data scientists have been practicing their coding skills, and now they want to mix it up in Microsoft Office. In response to this demand, SAS R&D has added the SAS programming environment -- the parts that make sense, anyway -- into SAS Add-In for Microsoft Office. You can write programs, run them, and drop the results into any part of your Microsoft Office document.

Here's a short screencast of how it works in Microsoft Excel:

SAS programming in AMO

If you want to learn more and see a narrated demo from the principal developer, watch this interview with Tim Beese from SAS R&D. Tim shows the coding feature along with several other cool enhancements to SAS Add-In for Microsoft Office. As Tim explains, SAS administrators still have the final say when it comes to allowing Excel users to let loose with SAS code -- they can enable the feature by role, and so grant this as a privilege at their discretion.

The first few minutes of this video shows some impressive integration with SAS Visual Analytics and Microsoft Excel. The SAS programming demo begins around the 5-minute mark.

The post Create and run SAS code in SAS Add-In for Microsoft Office appeared first on The SAS Dummy.

5月 312017
 

SAS programs in Excel, finally!When SAS Add-In for Microsoft Office was first created over a decade ago, SAS programmers were told to check their skills at the door. This new product was for non-programmers only. SAS programmers were invited to contribute to the experience by packaging their code in SAS stored processes, which end users would then run using point-and-click menus. But there was no way to write and run your SAS programs directly in Microsoft Excel or Word or PowerPoint.

This was a comfort to many SAS administrators, who wanted to provide SAS analytics to their end users but didn't want them to have to learn to program. Or perhaps to even allow them to program. But, times have changed. Citizen data scientists have been practicing their coding skills, and now they want to mix it up in Microsoft Office. In response to this demand, SAS R&D has added the SAS programming environment -- the parts that make sense, anyway -- into SAS Add-In for Microsoft Office. You can write programs, run them, and drop the results into any part of your Microsoft Office document.

Here's a short screencast of how it works in Microsoft Excel:

SAS programming in AMO

If you want to learn more and see a narrated demo from the principal developer, watch this interview with Tim Beese from SAS R&D. Tim shows the coding feature along with several other cool enhancements to SAS Add-In for Microsoft Office. As Tim explains, SAS administrators still have the final say when it comes to allowing Excel users to let loose with SAS code -- they can enable the feature by role, and so grant this as a privilege at their discretion.

The first few minutes of this video shows some impressive integration with SAS Visual Analytics and Microsoft Excel. The SAS programming demo begins around the 5-minute mark.

The post Create and run SAS code in SAS Add-In for Microsoft Office appeared first on The SAS Dummy.

5月 262017
 

Elizabeth is courageous. Scoliosis since birth, corrective spinal surgery replaced her spine with steel, tripping on stairs permanently broke her right ankle. Then she decided to come take yoga with me. To help ease back pain & reduce hip stress, I offered options like bent legs not cross. In class [...]

The post Take that SAS option appeared first on SAS Learning Post.