base sas

6月 082022
 

When generating animation files, the number of frames shown per second is called the frame rate (FPS - Frames per second). Generally, the higher the frame rate, the smoother the animation. Human eyes can process 10 to 12 frames per second and perceive them individually, while higher frame rates are perceived as motion due to "persistence of vision." Modern movies are usually shot and presented at 24 FPS, and there are currently five commonly used frame rates in the film and television industry: 24 FPS, 25 FPS, 30 FPS, 50 FPS and 60 FPS for HDTV.

Animations can present richer time-varying analysis results than static images. It can consider and compare analysis results from different angles to present more details. Animations generated in SAS can be in GIF or SVG format. Various SAS/Graph PROC and SG PROC steps support generating animated GIF files. This short post discusses some timing control issues that authors have found while generating animations for rigorous physics simulations with SAS. Preserving time accuracy is important in physics simulation and some time-sensitive animation generation.

GIF format review

The Graphics Interchange Format (GIF) is a bitmap image format with animation features created by CompuServer in 1987, and GIF89a is version 89a (July 1989) of the format. Its advantages include smaller file sizes and wider support for web browsers and platforms, but the smallest unit of frame duration in the GIF89a file format is hundredths of a second (0.01 seconds) instead of milliseconds (0.001 seconds). Many GIF tools allow specifying the frame duration in milliseconds, even though they will be rounded to hundredths of seconds anyways. They unintendedly mask the limitations of this file format. On the other hand, some GIF-playing software will automatically modify a time interval that is too small. For example, Microsoft's IE rounds up time intervals under 50ms to 100ms. This creates certain challenges for producing time-accurate animations.

Research in 2014[1] indicated that GIF89a does not produce substantial delays when an animation interval is larger than 100ms, regardless of the web browser. Below this threshold, the resulting performance falls considerably. IE9 reinterprets any delay below 100ms as 100ms, so a 50ms interval shows a 50ms delay, while a 16.67ms interval shows around 85ms delay. Firefox 10 and Chrome 17 can execute a GIF89a animation correctly (except for an 85ms delay on 16.67ms intervals), and their mean number of missed frames is smaller and stabilizes over time. The refresh interval is 16.67 milliseconds for a screen with a refresh rate of 60Hz.

Due to the need to control the size of the generated file, 8 FPS (125ms) is enough for most GIF files. When there are many dynamics at play, we can use 15 FPS (66.67ms). The file size for 24 FPS (41.67ms) GIFs without compression will be larger than a standard MP4 video file, so then it would be better to produce the animation as an MP4 instead of a GIF format with a higher FPS.

What is the proper FPS?

When generating animations in SAS, you can specify the duration of a frame through the SAS system option ANIMDURATION. It can accept up to 10 characters in length, that is, there can be 8 significant digits after the decimal point. This commonly leads to programmers incorrectly thinking that the duration of the animation can be specified in milliseconds. The actual test shows that in SAS, the duration of a frame specified via ANIMDURATION option with value 1/FPS, and the actual duration of the animation will be quite different. That is, if the programmer wants to generate relatively time-accurate animation directly in the simulation, they should pay attention when selecting a specific FPS to generate the animation and use the following relationship to specify the target DURATION value:

/*Calculate the accurate ANIMDURATION from/to FPS in SAS*/
 
proc fcmp inlib=work.proto outlib=work.fcmp.animation;
  function fps_animduration(fps);
    dur=1/fps;
    return( floor( dur * 100)/100 ); *Keep the first 2 digits after decimal point instead of round(dur, 0.01);
  endsub;
 
  function animduration_fps(dur);
    return( floor(<strong>1</strong>/dur) ); *Cut the integer part;
  endsub;
run;
 
option cmplib=(work.fcmp);

Accordingly, the following conversion can be used in SAS Macro:

%let DUR=%sysevalf( %sysfunc(floor((1/&FPS)*100 ))/100 );

In a time-accurate physics simulation, the duration or period is the absolute time, so the number of frames that need to be generated is the duration divided by the frame rate. Since the minimum unit of GIF per frame duration mentioned above is one-hundredth of a second, the actual generated frame rate is larger than expected in a generated GIF file, which means that the actual number of frames generated is less than expected for a specified simulation duration. Experiments show that when the FPS is less than or equal to 100, if no error in the number of generated frames is allowed, there are only 19 frame rates that can be used: 1-10, 11, 12, 14, 16, 20, 25, 33, 50 and 100 FPS. If 5% of the generated frames or time error is allowed, you can have an additional 8 frame rates to choose from: 24, 32, 48, 49, 96, 97, 98 and 99 FPS. Based on the background knowledge mentioned earlier, combined with actual needs, GIF file size limitations and playback smoothness requirements, we’d better generally choose a frame rate between 8 and 25 to produce time-accurate animations. 8, 10, 16, 20 and 25 FPS are all good choices. 24 FPS can be chosen if the error rate of frames generated is allowed to be <=5%. The following table lists the error between the actual number of frames generated and the expected number of frames when the duration is 1s under different FPS settings:

Figure1: Frame lost rate (<=5%) and FPS in GIF generation

The relationship between the actual loss rate of generated frames and specified FPS in the figure below reveals which frame rate we should specify when generating time-accurate animations:

Figure2: Frame lost rate and FPS specified in GIF generation

The figure below is a single-cycle animation generated by solving the figure-8 solution of the three-body problem in SAS. The physical time of a single cycle is about 6.3259 seconds, and the actual animation time is also 6.32 seconds with a time scale factor of 1. The duration is 0.04s per frame, for a total of 158 frames. In fact, the problem discussed in this article was discovered when trying to solve the three-body problem with SAS. Of course, in addition to the above frame rate selection, introducing compression techniques to SAS GIF generating is recommended to reduce the file size. Several experiments have shown that third-party tools such as GIF Optimizer can be used to reduce a GIF file's size by 90% without a significant reduction in graphics quality.

Figure3: An animation with galaxy background for the figure-8 solution of three body problem in SAS

Conclusion

When generating GIF animations in SAS that need to be time-accurate such as physics simulation, pay attention to the selection of the ANIMDURATION corresponding to the appropriate frame rate. When programming a GIF animation to display the analysis results, do not choose a random frame rate, or else there will be a large loss of generated frames or a time error. 8, 10, 16, 20 and 25 FPS can be chosen when there is no generated frame loss allowed and 24 FPS can be selected when a generated frame loss rate below 5% is allowed. The limitations discussed here are inherent to the GIF file specification; not due to SAS or third-party tools.

Learn more

Considerations for generating time-accurate animation with SAS was published on SAS Users.

6月 062022
 

This article was co-authored by Deva Kumar and Ali Dixon.

The launch of SAS Analytics Pro Advanced Programming offers key statistical capabilities in a docker container. The product bundles selected executables from SAS Viya to create the container, which eases or streamlines the setup required for fixes and updates to the software.

Current and new users can use the platform for data management, advanced analytics, reporting and batch processing. It includes the products in SAS Analytics Pro, Base SAS, SAS Studio 5.2, SAS/STAT, SAS/GRAPH. Integration with Python is available and includes Proc Python and access to saspy from python clients (ex: JupyterLab). Analytics Pro Advanced Programming offers the addition of SAS/ETS, SAS/IML, SAS/OR and SAS/QC.

For the remainder of this post, a reference to “Analytics Pro” includes the features of both Analytics Pro and Analytics Pro Advanced Programming on Viya.

Easy as 1-2-3 OR Simple is Good

Analytics Pro is delivered as a Docker image. This makes it easy to deploy, use and maintain. The diagram below shows that it takes just 3 steps to begin programming with SAS on your workstation.

Getting Started With APro

For a current Analytics Pro 9.x user, Analytics Pro is the next generation with all the goodness of Analytics Pro 9.x with many improvements from SAS Viya.

What is the same?

Users can run all of their existing programs with Analytics Pro. A few things like having a filename to a C: drive must change, but not the logic of the program. Users can run interactively using SAS Studio or run batch jobs.

Users can continue to program using the SAS Language consisting of DataStep, Procedures and Macros.

What is different?

Analytics Pro is “carved” out of SAS Viya so that it is familiar to current SAS 9.x users. In keeping with this philosophy, Analytics Pro does not include many of the advanced capabilities of SAS Viya.

  • The Cloud Analytic Server (CAS) is not included. However, Analytics Pro programs can connect to CAS.
  • To keep the size small, Analytics Pro does not include the latest version of SAS Studio or any of the other great visual products on SAS Viya.
  • Analytics Pro includes the database engines that are on SAS Viya. There are no additional charges for these.
  • Analytics Pro includes Proc Python from SAS Viya allowing users to include Python code as part of the SAS program.
  • Python programmers can use SASPy to access Analytics Pro from their Python code.
  • Updates are shipped monthly giving users the latest fixes and updates. This will include any new database drivers that are added to SAS Viya and other SAS Viya features like Proc Python.

What if users want to run it on the cloud?

One of the advantages of Analytics Pro is it will fit into the user’s cloud strategy. The container can be in docker on virtual machines – Linux or Windows. See this article for an example. The container can also be deployed on Kubernetes clusters. Users can use the platform that suits their needs.

View a demo of these new features and learn more on the SAS Viya Release Highlights Show.



SAS Analytics Pro Advanced Programming provides a powerful package to explore and solve problems for those seeking new statistical capabilities.

Learn more

Check out these Analytics Pro articles or take a deep dive into this additional Analytics Pro documentation.

Introducing Analytics Pro Advanced Programming: The power of SAS on your desktop was published on SAS Users.

10月 302021
 

In a September 10 post on the SAS Users blog, we announced that SAS Analytics Pro is now available for on-site or containerized cloud-native deployment. For our thousands of SAS Analytics Pro customers, this provides an entry point into SAS Viya.

SAS Analytics Pro consists of three core elements of the SAS system: Base SAS®, SAS/GRAPH® and SAS/STAT®. The containerized deployment option adds the full selection of SAS/ACCESS engines making it even easier to work with data from virtually any source.

Even better, the containerized deployment option now adds new statistical capabilities that are not available in SAS/STAT on SAS9. Thanks to SAS Viya’s continuous delivery approach, we are able to provide this additional functionality so soon after the initial release.

Below are highlights of these additional capabilities (you can find more details by following the links):

Causal Inference Procedures

Bayesian Analysis Procedures

  • Model multinomial data with cumulative probit, cumulative logit, generalized link, or other link functions in PROC BGLIMM.
  • Specify fixed scale values in a generalized linear mixed-effects model, and use an improved CMPTMODEL statement in PROC MCMC and PROC NLMIXED to fit compartment models.

Survey Procedures

Additional Capabilities:

For those SAS customers already on SAS Viya, or those considering the move, SAS Analytics Pro provides one more example of the new powers you will enjoy!

Additional statistical capabilities in the containerized deployment of SAS Analytics Pro was published on SAS Users.

9月 102021
 

As its thousands of users know, SAS Analytics Pro consists of three core elements of the SAS system: Base SAS®, SAS/GRAPH® and SAS/STAT®. It provides the fundamental capabilities of data handling, data visualization, and statistical analysis either through coding or through the SAS Studio interface. For many years, SAS Analytics Pro has been deployed on-site as the entry-level workhorse to the SAS system.

Now, SAS Analytics Pro includes a new option for containerized cloud-native deployment. In addition, the containerized option comes with the full selection of SAS/ACCESS engines making it even easier to work with data from virtually any source. For organizations considering the move to the cloud, or those already there, SAS Analytics Pro provides an exciting new option for cloud deployment.

What is SAS Analytics Pro?

SAS Analytics Pro is an easy-to-use, yet powerful package for accessing, manipulating, analyzing and presenting information. It lets organizations improve productivity with all the tools and methods needed for desktop data analysis – in one package.

  • Organizations can get analysis, reporting and easy-to-understand visualizations from one vendor. Rather than having to piece together niche software packages from different vendors, this consolidated portfolio reduces the cost of licensing, maintenance, training and support – while ensuring that consistent information is available across your enterprise.
  • Innovative statistical techniques are provided with procedures constantly being updated to reflect the latest advances in methodology. Organizations around the world rely on SAS to provide accurate answers to data questions, along with unsurpassed technical support.
  • SAS software integrates into virtually any computing environment, unifying your computing efforts to get a single view of your data, and freeing analysts to focus on analysis rather than data issues.
  • Easily build analytical-style graphs, maps and charts with virtually any style of output that is needed, so you can deliver analytic results where they’re needed most

Why data scientists should care about cloud and containers?

With the new containerized cloud-native deployment option for SAS Analytics Pro, this raises the question of why data scientists should care about cloud and containers? This question was addressed in a SAS Users blog post by SAS R&D director Brent Laster.

This post characterizes a container as a “self-contained environment with all the programs, configuration, initial data, and other supporting pieces to run applications.” The nice thing for data scientists is that this environment can be treated as a stand-alone unit, to turn on and run at any time – sort of a “portable machine.” It provides a complete virtual system configured to run your targeted application.

Using popular container runtime environments (e.g., Docker), containers can be an efficient way for individual users to deploy and manage software applications. This is especially useful for applications like SAS Analytics Pro, which participates in SAS Viya’s continuous delivery approach, releasing updates on a regular basis.

For large, IT-managed environments, containers can call for something like a “data center” to simplify deployment and management of dynamic workloads. The most prominent one today is Kubernetes, which automates key needs around containers including deployment, scaling, scheduling, healing, and monitoring – so the data scientist doesn’t have to.

The combination of containers and cloud environments provides an evolutionary jump in the infrastructure and runtime environments where data scientists run their applications. And this gives them a similar jump in being able to provide the business value their customers demand. Containerized and cloud-native deployment of SAS Analytics Pro provides the automatic optimization of resources and the automatic management of workloads that your organization needs to be competitive.

Note that existing customers can continue programming in SAS in a small footprint environment while availing themselves of the SAS container-based continuous delivery process. And if you aren’t already a SAS customer, cloud deployment gives you one more good reason to start letting SAS Analytics Pro deliver value to your organization.

Learn more

Website: SAS® Analytics Pro
Training: SAS® Global Certification Program

On-Premises Documentation:

Cloud-Native Documentation:

SAS Analytics Pro now available for on-site or containerized cloud-native deployment was published on SAS Users.

9月 072020
 

Locale-specific SAS® format catalogs make reporting in multiple languages more dynamic. It is easy to generate reports in different languages when you use both the LOCALE option in the FORMAT procedure and the LOCALE= system option to create these catalogs. If you are not familiar with the LOCALE= system option, see the "Resources" section below for more information.

This blog post, inspired by my work on this topic with a SAS customer, focuses on how to create and use locale-specific informats to read in numeric values from a Microsoft Excel file and then transform them into SAS character values. I incorporated this step into a macro that transforms ones and zeroes from the Excel file into meaningful information for multilingual readers.

Getting started: Creating the informats

The first step is to submit the LOCALE= system option with the value fr_FR. For the example in this article, I chose the values fr_FR and en_US for French and English from this table of LOCALE= values. (That is because I know how to say “yes” and “no” in both English and French — I need to travel more!)

   options locale=fr_fr;

The following code uses both the INVALUE statement and the LOCALE option in PROC FORMAT to create an informat that is named $PT_SURVEY:

   proc format locale library=work;
      invalue $pt_survey 1='oui' 0='non'; run;

Now, toggle the LOCALE= system option and create a second informat using labels in a different language (in this example, it is English):
options locale=en_us;

   proc format locale library=work;
      invalue $pt_survey 1='yes' 0='no';
   run;

In the screenshot below, which shows the output from the DATASETS procedure, you can see that PROC FORMAT created two format catalogs using the specified locale values, which are preceded by underscore characters. If the format catalogs already exist, PROC FORMAT simply adds the $PT_SURVEY informat entry type to them.

   proc datasets memtype=catalog; 
   quit;

Before you use these informats for a report, you must tell SAS where the informats are located. To do so, specify /LOCALE after the libref name within the FMTSEARCH= system option. If you do not add the /LOCALE specification, you see an error message stating either that the $PT_SURVEY informat does not exist or that it cannot be found. In the next two OPTIONS statements, SAS searches for the locale-specific informat in the FORMATS_FR_FR catalog, which PROC FORMAT created in the WORK library:

   options locale=fr_fr;
   options fmtsearch=(work/locale);

If you toggle the LOCALE= system option to have the en_US locale value, SAS then searches for the informat in the other catalog that was created, which is the FORMATS_EN_US catalog.

Creating the Excel file for this example

For this example, you can create an Excel file by using the ODS EXCEL destination from the REPORT procedure output. Although you can create the Excel file in various ways, the reason that I chose the ODS EXCEL statement was to show you some options that can be helpful in this scenario and are also useful at other times.
Use the ODS EXCEL destination to create a file from PROC REPORT. I specify the TAGATTR= style attribute using “TYPE:NUMBER” for the Q_1 variable:

   %let  path=%sysfunc(getoption(WORK));
   filename temp "&path\surveys.xlsx"; 
   ods excel file=temp;
 
 
   data one;
      infile datalines truncover;
      input ptID Q_1;
      datalines;
   111 0
   112 1
   ;
   run;
 
   proc report data=one;
      define ptID / display style(column)={tagattr="type:String"};
      define Q_1 / style(column)={tagattr="type:Number"};
   run;
 
   ods excel close;

Now you have a file that looks like this screenshot when it is opened in Excel. Note that the data value for the Q_1 column is numeric:

The IMPORT procedure uses the DBSASTYPE= data set option to convert the numeric Excel data into SAS character values. Then I can apply the locale-specific character informat to a character variable.

As you will see below, in the macro, I use DBMS=EXCEL in PROC IMPORT to read the Excel file because my SAS and Microsoft Office versions are both 64-bit. (You might have to use the PCFILES LIBNAME Engine to connect to Excel through the SAS PC Files Server if you are not set up this way.)

Using the informats in a macro to create the multilingual reports

The final step is to run the macro with parameters to produce the two reports in French and English, using the locale-specific catalogs. When the macro is called, depending on the parameter value for the macro variable LOCALE, the LOCALE= system option changes, and the $PT_SURVEY informat from the locale-specific catalog is applied. These two tabular reports are produced:

Here is the full code for the example:

   %let  path=%sysfunc(getoption(WORK));
   filename temp "&path\surveys.xlsx";
   ods excel file=temp;
 
   data one;
      infile datalines truncover;
      input ptID Q_1;
      datalines;
   111 0
   112 1
   ;
   run;
 
   proc report data=one;
      define ptID / display style(column)={tagattr="type:String"};
      define Q_1 / style(column)={tagattr="type:Number"};
   run;
 
   ods excel close;
   options locale=fr_fr;
 
   proc format locale library=work;
      invalue $pt_survey 1='oui' 0='non';
   run;
 
   options locale=en_us;
 
   proc format locale library=work;
      invalue $pt_survey 1='yes' 0='no';
   run;
 
   /* Set the FMTSEARCH option */
   options fmtsearch=(work/locale);
 
   /* Compile the macro */
   %macro survey(locale,out);
      /* Set the LOCALE system option */
      options locale=&locale;
 
      /* Import the Excel file  */
      filename survey "&path\surveys.xlsx";
 
      proc import dbms=excel datafile=survey out=work.&out replace;
         getnames=yes;
         dbdsopts="dbsastype=(Q_1='char(8)')";
      run;
 
      data work.&out;
         set work.&out;
 
         /* Create a new variable for the report whose values are assigned by specifying the locale-specific informat in the INPUT function */
         newvar=input(Q_1, $pt_survey.);
         label newvar='Q_1';
      run;
 
      options missing='0';
 
      /*  Create the tabular report */
      proc tabulate data=&out;
         class ptID newvar;
 
         table ptID='Patient ID', newvar*n=' '/box="&locale";
      run;
 
   %mend survey;
 
   /* Call the macros */
   %survey(fr_fr,fr)
   %survey(en_us,en)

For a different example that does not involve an informat, you can create a format in a locale-specific catalog to print a data set in both English and Romanian. See Example 19: Creating a Locale-Specific Format Catalog in the Base SAS® 9.4 Procedures Guide.

Resources

For more information about the LOCALE option:

For more information about reading and writing Excel files:

For more information about creating macros and using the macro facility in SAS:

Using locale-specific format catalogs to create reports in multiple languages was published on SAS Users.

9月 022020
 

SAS offering free learning resources in celebration of programmers

For more than 40 years, SAS programmers have crafted software and solutions that transform the world. From statistics to data science, to analytics and artificial intelligence, people writing code have architected a new economy with incredible opportunities. SAS Programmer Week honors those people by offering free learning resources available for everyone, from students to early career professionals to SAS veterans.

Running from Sept. 7-11, SAS Programmer Week leads up to the international Day of the Programmer on Saturday, Sept. 12. Training resources will be available for free through a variety of YouTube and video tutorials, webinars, blogs and documentation.

There will be three different tracks for new, experienced and analytics-focused users, with new content released each day. The week culminates with SAS certification prep content that will have participants ready to pursue a valuable SAS credential.

For instance, Tech Republic named SAS as one of 7 data science certifications to boost your resume and salary. CIO Magazine puts SAS among the top 11 big data and analytics certifications for 2020.

Since SAS programmers are busy and may not have all day to engage with the materials, SAS Programmer Week is flexible. Participants can access the material when they want to learn a specific skill related to the day’s topic or consume the material in snippets when they have time.

Interested participants can visit the SAS Programmer Week website to register today, preview the materials and schedule, and jump-start their career journeys.

 

All hail the SAS programmer! was published on SAS Users.

6月 262019
 

"There's a way to do it better - find it." - Thomas A. Edison

Finding a better SAS code

When it comes to SAS coding, this quote by Thomas A. Edison is my best advisor. Time permitting, I love finding better ways of implementing SAS code.

But what code feature means “better” – brevity, clarity or efficiency? It all depends on the purpose of your code. When code is to illustrate a coding concept or technique, clarity is a paramount. However, when processing large data volumes in near real-time, code efficiency becomes critical, not just a luxury or convenience. And brevity won’t hurt in either case. Ideally, your code should be a combination of all three features - brevity, clarity and efficiency.

Parsing a character string

In this blog post we will solve a problem of parsing a character string to find a position of n-th occurrence of a group of characters (substring) in that string.

The closest out-of-box solution to this problem is SAS’ FIND() function. Except this function searches only for a single/first instance of specified substring of characters within a character string. Close enough, and with some do-looping we can easily construct what we want.

After some internet and soul searching to find the Nth occurrence of a substring within a string, I came up with the following DATA STEP code snippet:

   p = 0;
   do i=1 to n until(p=0); 
      p = find(s, x, p+1);
   end;

Here, s is a text string (character variable) to be parsed; x is a character variable holding a group of characters that we are searching for within s; p is a position of x value found within s; n is an instance number.

If there is no n-th instance of x within s found, then the code returns p=0.

In this code, each do-loop iteration searches for x within s starting from position p+1 where p is position found in prior iteration: p = find(s,x,p+1);.

Notice, if there is no prior-to n instance of x within s, the do-loop ends prematurely, based on until(p=0) condition, thus cutting the number of loops to the minimal necessary.

Reverse string search

Since find() function allows for a string search in a reverse direction (from right to left) by making the third augment negative, the above code snippet can be easily modified to do just that: find Nth instance (from right to left) of a group of characters within a string. Here is how you can do that:

   p = length(s) + 1;
   do i=1 to n until(p=0); 
      p = find(s, x, -p+1);
   end;

The difference here is that we start from position length(s)+1 instead of 0, and each iteration searches substring x within string s starting from position –(p-1)=-p+1 from right to left.

Testing SAS code

You can run the following SAS code to test and see how these searches work:

data a;
   s='AB bhdf +BA s Ab fs ABC Nfm AB ';
   x='AB';
   n=3;
 
   /* from left to right */
   p = 0;
   do i=1 to n until(p=0); 
      p = find(s, x, p+1);
   end;
   put p=;
 
   /* from right to left */
   p = length(s) + 1;
   do i=1 to n until(p=0); 
      p = find(s, x, -p+1);
   end;
   put p=;
run;

FINDNTH() function

We can also combine the above left-to-right and right-to-left searches into a single user-defined SAS function by means of SAS Function Compiler (PROC FCMP) procedure:

proc fcmp outlib=sasuser.functions.findnth;
   function findnth(str $, sub $, n);
      p = ifn(n>=0,0,length(str)+1);
      do i=1 to abs(n) until(p=0);
         p = find(str,sub,sign(n)*p+1);
      end;
      return (p);
   endsub;
run;

We conveniently named it findnth() to match the Tableau FINDNTH(string, substring, occurrence) function that returns the position of the nth occurrence of substring within the specified string, where the occurrence argument defines n.

Except our findnth() function allows for both, positive (for left-to-right searches) as well as negative (for right-to-left searches) third argument while Tableau’s function only allows for left-to-right searches.

Here is an example of the findnth() function usage:

options cmplib=sasuser.functions;
data a;
   s='AB bhdf +BA s Ab fs ABC Nfm AB ';
   x='AB';
   n=3;
 
   /* from left to right */
   p=findnth(s,x,n);
   put p=;
 
   /* from right to left */
   p=findnth(s,x,-n);
   put p=;
run;

Using Perl regular expression

As an alternative solution I also implemented SAS code for finding n-th occurrence of a substring within a string using Perl regular expression (regex or prx):

data a;
   s='AB bhdf +BA s Ab fs ABC Nfm AB ';
   x='AB';
   n=3;
 
   /* using regex */
   xid = prxparse('/'||x||'/o');
   p = 0;
   do i=1 to n until(p=0);
      from = p + 1;
      call prxnext(xid, p + 1, length(s), s, p, len);
   end;
   put p=;
run;

However, efficiency benchmarking tests demonstrated that the above solutions using FIND() function or FINDNTH() SAS user-written function run roughly twice faster than this regex solution.

Challenge

Can you come up with an even better solution to the problem of finding Nth instance of a sub-string within a string? Please share your thoughts and solutions with us. Thomas A. Edison would have been proud of you!

Finding n-th instance of a substring within a string was published on SAS Users.

5月 302019
 

Human behavior is fascinating. We come in so many shapes, sizes and backgrounds. Doesn’t it make sense that any tests we write also accommodate our wonderful differences?

This picture is of Miko, a northern rescue and a recent addition to my family. He’s learning to live in an urban household and doing great with some training. He’s going through so many new tests as he adapts to life in the city, which is quite different from being free in the northern territories. Watch for a later post on his training successes.

I’m so happy to share how SAS has been helping candidates by offering a variety of certification credentials geared towards testing for differences and preferences in thought. If you are wondering – I’ve been addicted to psychometrics for a while now, anything human behavior-related interests me. I thought I would begin with sharing some different types of testing roles that I have held in the past.

1. Psychometric testing

Before I joined SAS, I worked at CSI. To answer that unspoken thought dear reader, CSI has been providing financial training and accreditation since 1964 – way before CSI the TV show became popular.

My role as Test Manager was super exciting for someone with a curiosity for analytics and helping people succeed. In a team of four we scored over 200 exams to provide credentials. Psychometrics was the most exciting part of my job analyzing the performance of test takers to constantly innovate our tests. Psychometric tests are used to identify a candidate's skills, knowledge and personality.

2. Multiple-choice testing

While setting multiple choice exam questions, I learned that it was ideal for the four answer choices to be similar in length, and complexity (e.g. if candidates typically chose option A for a question whose right response was B, we would dig deeper to compare the lengths of the options, the language of the options, and then change the option if that was what the review committee agreed upon).

3. Adaptive testing

Prior to CSI, I worked at the test center of Devry Institute of technology. In adaptive testing, the test’s difficulty adapts to candidate performance. A correct response leads into a more complex question. On the flip side, an incorrect response leads to an easier next question. So that, eventually, we could help candidates decide which engineering program would be the right skill fit.

This is where I met the student who asked, “can my boyfriend write my exam?”

4. Performance testing

With SAS at the forefront of analytics, it should come as no surprise that certification exams have evolved to the next level. As a certification candidate you can now try out performance-based testing.

A performance test requires a candidate to actually perform a task, rather than simply answering questions. An example is writing SAS code. Instead of answering a knowledge-level multiple choice exam about SAS code, the candidate is asked to actually write code to arrive at answers.

Certification at SAS

SAS Certified Specialist: Base Programming Using SAS 9.4 is great for those who can demonstrate ease in putting into practice the knowledge learned in the Foundation Programming classes 1 and 2. During this performance-based exam, candidates will access a SAS environment. Coding challenges will be presented, and you will need to write and execute SAS code to determine the correct answers to a series of questions.

SAS® Certified Base Programmer for SAS®9 credential remains, but the exam will be retired in June 2019.

While writing this post I came across this on Wikipedia: it shows how the study of adaptive behavior goes back to Darwin’s time. It’s a good read for anyone intrigued by the science and art of testing.

“Charles Darwin was the inspiration behind Sir Francis Galton who led to the creation of psychometrics. In 1859, Darwin published his book The Origin of Species, which pertained to individual differences in animals. This book discussed how individual members in a species differ and how they possess characteristics that are more adaptive and successful or less adaptive and less successful. Those who are adaptive and successful are the ones that survive and give way to the next generation, who would be just as or more adaptive and successful. This idea, studied previously in animals, led to Galton's interest and study of human beings and how they differ one from another, and more importantly, how to measure those differences.”

Are you fascinated by the science and art of human behavior as it relates to testing? Are you as excited as I am about the possibilities of performance-based testing? I would love to hear your comments below.

New at SAS: Psychometric testing was published on SAS Users.

5月 172019
 
Did you know that you can run Lua code within Base SAS? This functionality has been available since the SAS® 9.4M3 (TS1M3) release. With the LUA procedure, you can submit Lua statements from an external Lua script or just submit the Lua statements using SAS code. In this blog, I will discuss what PROC LUA can do as well as show some examples. I will also talk about a package that provides a Lua interface to SAS® Cloud Analytic Services (CAS).

What Is Lua?

Lua is a lightweight, embeddable scripting language. You can use it in many different applications from gaming to web applications. You might already have written Lua code that you would like to run within SAS, and PROC LUA enables you to do so.
With PROC LUA, you can perform these tasks:

  • run Lua code within a SAS session
  • call most SAS functions within Lua statements
  • call functions that are created using the FCMP procedure within Lua statements
  • submit SAS code from Lua
  • Call CAS actions

PROC LUA Examples

Here is a look at the basic syntax for PROC LUA:

proc lua <infile='file-name'> <restart> <terminate>;

Suppose you have a file called my_lua.lua or my_lua.luc that contains Lua statements, and it is in a directory called /local/lua_scripts. You would like to run those Lua statements within a SAS session. You can use PROC LUA along with the INFILE= option and specify the file name that identifies the Lua source file (in this case, it is my_lua). The Lua file name within your directory must contain the .lua or. luc extension, but do not include the extension within the file name for the INFILE= option. A FILENAME statement must be specified with a LUAPATH fileref that points to the location of the Lua file. Then include the Lua file name for the INFILE= option, as shown here:

filename luapath '/local/lua_scripts';
proc lua infile='my_lua';

This example executes the Lua statements contained within the file my_lua.lua or my_lua.luc from the /local/lua_scripts directory.

If there are multiple directories that contain Lua scripts, you can list them all in one FILENAME statement:

filename luapath ('directory1', 'directory2', 'directory3');

The RESTART option resets the state of Lua code submissions for a SAS session. The TERMINATE option stops maintaining the Lua code state in memory and terminates the Lua state when PROC LUA completes.

The syntax above discusses how to run an external Lua script, but you can also run Lua statements directly in SAS code.

Here are a couple of examples that show how to use Lua statements directly inside PROC LUA:

Example 1

   proc lua; 
   submit; 
      local names= {'Mickey', 'Donald', 'Goofy', 'Minnie'} 
      for i,v in ipairs(names) do 
         print(v) 
   end 
   endsubmit; 
   run;

Here is the log output from Example 1:

NOTE: Lua initialized.
Mickey
Donald
Goofy
Minnie
NOTE: PROCEDURE LUA used (Total process time):
      real time           0.38 seconds
      cpu time            0.10 seconds

Example 2

   proc lua;
   submit;
      dirpath=sas.io.assign("c:\\test")
      dir=dirpath:opendir()
      if dir:has("script.txt") then print ("exists")
      else print("doesn't exist")
      end
   endsubmit;
   run;

Example 2 checks to see whether an external file called script.txt exists in the c:\test directory. Notice that two slashes are needed to specify the backslash in the directory path. One backslash would represent an escape character.

All Lua code must be contained between the SUBMIT and ENDSUBMIT statements.

You can also submit SAS code within PROC LUA by calling the SAS.SUBMIT function. The SAS code must be contained within [[ and ]] brackets. Here is an example:

   proc lua; 
   submit;
      sas.submit [[proc print data=sashelp.cars; run; ]]
   endsubmit;
   run;

Using a Lua Interface with CAS

Available to download is a package called SWAT, which stands for SAS Scripting Wrapper for Analytics Transfer. This is a Lua interface for CAS. After you download this package, you can load data into memory and apply CAS actions to transform, summarize, model, and score your data.

The package can be downloaded from this Downloads page: SAS Lua Client Interface for Viya. After you download the SWAT package, there are some requirements for the client machine to use Lua with CAS:

  1. You must use a 64-bit version of either Lua 5.2 or Lua 5.3 on Linux.

    Note: If your deployment requires newer Lua binaries, visit http://luabinaries.sourceforge.net/.
    Note: Some Linux distributions do not include the required shared library libnuma.so.1. It can be installed with the numactl package supplied by your distribution's package manager.

  2. You must install the third-party package dependencies middleclass (4.0+), csv, and ee5_base64, which are all included with a SAS® Viya® installation.

For more information about configuration, see the Readme file that is included with the SWAT download.

I hope this blog post has helped you understand the possible ways of using Lua with SAS. If you have other SAS issues that you would like me to cover in future blog posts, please comment below.

To learn more about PROC LUA, check out these resources:

Using the Lua programming language within Base SAS® was published on SAS Users.

12月 192014
 

SAS Technical Support Problem SolversIf you haven’t tried them for your web applications and other graphics needs, you’ll want to read further!

Scalable Vector Graphics (SVG) output is vector graphics output you can display with most (if not all) modern web browsers. Because SVG graphic output is scalable, you can zoom in on the graphics output without losing resolution. Unlike bit-mapped images such as PNG or GIF output, they can be resized or transformed without compromising the clarity, eliminating the need to produce multiple versions of the same image! There are other advantages for using Scalable Vector Graphics like their ability to zoom in to view details, their smaller output file size and their usefulness for producing graphics for a range of display sizes and types.

Which SAS products offer SVG graphics?

The SVG family of device drivers has shipped as part of the SAS/GRAPH product since the SAS 9.2 release. Note that you can only use these SVG device drivers with traditional SAS/GRAPH procedures such as PROC GPLOT and PROC GCHART.

Starting with SAS 9.3 version of the Base product, you can also create SVG output with the SAS SG procedures such as SGPLOT and SGPANEL as well as with graphics output created with ODS Graphics. In SAS 9.4, you can also use Scalable Vector Graphics to produce animations.

Typically, when you create SVG graphics, you will want to create the output in one of these ways:

  • a standalone SVG file with a file extension of .svg
  • an HTML output file using the ODS HTML statement
  • an HTML5 output file in SAS 9.4

The output method you choose depends on your application. If you’re creating standalone SVG files, you can use that SVG file in some other document and make reference to it in another HTML page. For example, a common application for this would be creating logos in SVG that can be sized to any space. If you are using SAS 9.4, the HTML5 method is the best when creating an HTML document because the SVG can be embedded directly and there are no additional files to be moved.

In this blog post I’ll show you how to produce each one of these output types using the Base Product or SAS/GRAPH. I’ve also included a list of sample SAS/GRAPH animations that you can try.

Creating Scalable Vector Graphics with Base SAS

In SAS 9.3 and SAS 9.4, you can specify Scalable Vector Graphics output by specifying the OUTPUTFMT=SVG option on the ODS Graphics statement before the procedure step, such as:

ods graphics on / outputfmt=svg; 

The examples in this sections use the sashelp.cars data set shipped with the SAS 9.3 and SAS 9.4 Base product to produce a bubble plot.

svg_sgplot

Stand-alone SVG file. The following sample code uses PROC SGPLOT to write a standalone SVG file with the name sastest.svg to the C:temp directory when running on the Windows operating system:

    ods _all_ close; 
    ods listing gpath='c:temp';

    ods graphics / reset=all outputfmt=svg imagename='sastest'; 
 
    title1 'Plot of MPG City versus Horsepower';  
    proc sgplot data=sashelp.cars; 
      bubble x=horsepower y=mpg_city size=cylinders;
    run;

HTML file. This code uses the same PROC SGPLOT code to write a SVG file along with a corresponding HTML file to C:temp when running on the Windows operating system:

    ods _all_ close; 
    ods html path='c:temp' (url=none) file='svg.html'; 

    ods graphics / outputfmt=svg; 

    title1 'Plot of MPG City versus Horsepower';  
    proc sgplot data=sashelp.cars; 
      bubble x=horsepower y=mpg_city size=cylinders;
    run;

    ods html close; 
    ods listing; 

HTML 5 file. With SAS 9.4 only, you can use PROC SGPLOT with the ODS HTML5 statement to embed the SVG output in an HTML file. Note that with the code below, the SVG output is embedded inside the HTML output via the use of the svg_mode='inline' option on the ODS HTML5 statement.

    ods _all_ close; 
    ods html5 path='c:temp' (url=none) file='svg.html'
                options(svg_mode='inline');

    ods graphics / outputfmt=svg; 

    title1 'Plot of MPG City versus Horsepower';  
    proc sgplot data=sashelp.cars; 
      bubble x=horsepower y=mpg_city size=cylinders;
    run;

    ods html5 close; 
    ods listing; 

Creating Scalable Vector Graphics with SAS/GRAPH

The examples in this sections use PROC GPLOT and the sashelp.class data set to produce a linear plot of weight versus height.

svg_gplot

Stand-alone SVG file. Here is sample SAS code that uses PROC GPLOT to write a standalone SVG file with the name sastest.svg to the Temp directory on your C: drive when running on the Windows operating system:

    ods _all_ close; 
    ods listing;

    filename grafout 'c:tempsastest.svg'; 

    goptions reset=all device=svg gsfname=grafout;  

    symbol1 i=none v=dot c=black h=1.5;
    axis1 minor=none;  
    title1 'Plot of Weight versus Height';
    proc gplot data=sashelp.class;
      plot weight*height / haxis=axis1 vaxis=axis1;
    run;
    quit;  

HTML file. Here’s how to write the same output to a SVG file along with a corresponding HTML file:

    ods _all_ close; 
    ods html path='c:temp' (url=none) file='svg.html';  

    goptions reset=all device=svg;  

    symbol1 i=none v=dot c=black h=1.5;
    axis1 minor=none;  
    title1 'Plot of Weight versus Height';
    proc gplot data=sashelp.class;
      plot weight*height / haxis=axis1 vaxis=axis1;
    run;
    quit;  

    ods html close; 
    ods listing;

HTML 5 file. With SAS 9.4 only, the following sample code uses PROC GPLOT together with the ODS HTML5 statement to embed the SVG output in the resulting HTML file. Note that with the code below, the SVG output is embedded inside the HTML output via the use of the svg_mode='inline' option on the ODS HTML5 statement.

    ods _all_ close; 
    ods html5 path='c:temp' (url=none) file='svg.html'
               options(svg_mode='inline');   

    goptions reset=all device=svg;  

    symbol1 i=none v=dot c=black h=1.5;
    axis1 minor=none;  
    title1 'Plot of Weight versus Height';
    proc gplot data=sashelp.class;
      plot weight*height / haxis=axis1 vaxis=axis1;
    run;
    quit;  

    ods html5 close; 
    ods listing;  

Using Scalable Vector Graphics for animation in SAS/GRAPH

Beginning with SAS 9.4, you can create animated graphs for the web using the SVG device driver together with new options available on the OPTIONS statement. Here are links to sample programs on support.sas.com that demonstrate how to create animated graphs for the web using the SAS 9.4 SVG device driver:

tags: base sas, HTML5, ods, SAS Problem Solvers, sas/graph, Scalable Vector Graphics