Programming

6月 112020
 

Whether you enjoy debugging or hate it, for programmers, debugging is a fact of life. It’s easy to misspell a keyword, scramble your array subscripts, or (heaven forbid!) forget a semicolon. That’s why we include a chapter on debugging in The Little SAS® Book and its companion book, Exercises and Projects for the Little SAS® Book. We believe that learning to debug makes you a better programmer. Once you understand a bug, you will be better prepared to avoid it in the future.

To help hone your debugging skills, here is an example of the type of problems you can find in our book of exercises and projects. See if you can find the bugs.

Programming exercise

  1. A friend tells you that she is learning SAS and wrote the following program. Unfortunately, the program won’t run. Help her improve her programming skills by finding the mistakes.

TITLE Height, Weight, and BMI;
TITLE2 by Sex and Age Group;
PROC CONTENT DATA = SASHELP.class; RUN;
DATA; SET SASHELP.class;
Height_m = Heigth * 0.0254;
Weight_kg = Weight * 0.4536;
BMI = Weight_kg / Height_m**2;
PROC FORMAT; VALUE
$sex 'M' = 'Boys' 'F' = 'Girls';
VALUE agegp 11-12 = 'Preteens
13-16 = 'Teens';
PROC TABULATE;
CLASS Sex Age; VAR Height_m Weight_kg;
TABLES (Height_m Weight_kg BMI)*
MEAN, Sex Age ALL;
FORMAT Sex $sex. Age agegp.;
RUN;
QUIT;

 

  • a. Examine the SAS data set SASHELP.CLASS including variable attributes.
  • b. Clean up the formatting of the program by adding appropriate indention and line spacing to show the structure of the DATA and PROC steps. Make changes as needed to make the program conform to standard best practices.
  • c. Fix any errors in the code so that the program will run correctly.
  • d. Add comments to the revised program for each bug that you fix so that your friend can understand her mistakes.

Solution

In the book, we provide solutions for odd-numbered multiple choice and short answer questions, and hints for the programming exercises. So here is a hint for this exercise:

  1. Hint: This program contains four bugs. It also contains “red herrings” that are unusual for SAS code, but nonetheless do run properly and so are not actual bugs. Be sure you know how SAS handles data set names by default. SAS Enterprise Guide can format code for you; right-click the Program window and select Format Code from the pop-up menu. To format code in SAS Studio, click the Format Code icon at the top of Program window.

For more about The Little SAS Book and its companion book of exercises and projects, check out these blogs:

What's wrong with this code? was published on SAS Users.

4月 282020
 

With increasing interest in Continuous Integration/Continuous Delivery (CI/CD), many SAS Users want to know what can be done for Visual Analytics reports. In this article, I will explain how to use Python and SAS Viya REST APIs to extract a report from a SAS Viya environment and import it into another environment. For those trying to understand the secret behind CI/CD and DevOps, here it is:

What you do tomorrow will be better than what you did yesterday because you gained more experience today!

About Continuous Integration/Continuous Delivery

If you apply this principle to code development, it means that you may update your code every day, test it, deploy it, and start the process all over again the following day. You might feel like Sisyphus rolling his boulder around for eternity. This is where CI/CD can help. In the code deployment process, you have many recurrent tasks that can be automated to reduce repetitiveness and boredom. CI/CD is a paradigm where improvements to code are pushed, tested, validated, and deployed to production in a continuous automated manner.

About ModelOps and AnalyticOps

I hope you now have a better understanding of what CI/CD is. You might now wonder how CI/CD relates to Visual Analytics reports, models, etc. With the success of DevOps which describes the Development Operations for software development, companies have moved to the CI/CD paradigms for operations not related to software development. This is why you hear about ModelOps, AnalyticOps... Wait a second, what is the difference between writing or generating code for a model or report versus writing code for software? You create a model, you test it, you validate it, and finally deploy it. You create a report, you test it, you validate it, and then you deploy it. Essentially, the processes are the same. This is why we apply CI/CD techniques to models, reports, and many other business-related tasks.

About tools

As with many methodologies like CI/CD, tools are developed to help users through the process. There are many tools available and some of them are more widely used. SAS offers SAS Workflow Manager for building workflows to help with ModelOps. Additionally, you have surely heard about Git and maybe even Jenkins.

  • Git is a version control system that is used by many companies with some popular implementations: GitHub, GitLab, BitBucket.
  • Jenkins is an automation program that is designed to build action flows to ease the CI/CD process.

With these tools, you have the needed architecture to start your CI/CD journey.

The steps

With a basic understanding of the CI/CD world; you might ask yourself: How does this apply to reports?

When designing a report in an environment managed by DevOps principles, here are the steps to deploy the report from a development environment to production:

  1. Design the report in development environment.
  2. Validate the report with Business stakeholders.
  3. Export the report from the development environment.
  4. Save a version of the report.
  5. Import the report into the test environment.
  6. Test the report into the test environment.
  7. Import the report into the production environment.
  8. Monitor the report usage and performance.

Note: In some companies, the development and test environments are the same. In this case, steps 4 to 6 are not required.

Walking through the steps, we identify steps 1 and 2 are manual. The other steps can be automated as part of the CI/CD process. I will not explain how to build a pipeline in Jenkins or other tools in this post. I will nevertheless provide you with the Python code to extract, import, and test a report.

The code

To perform the steps described in the previous section, you can use different techniques and programming languages. I’ve chosen to use Python and REST APIs. You might wonder why I've chosen these and not the sas-admin CLI or another language. The reason is quite simple: my Chinese zodiac sign is a snake!

Jokes aside, I opted for Python because:

  • It is easy to read and understand.
  • There is no need to compile.
  • It can run on different operating systems without adaptation.
  • The developer.sas.com site provides examples.

I’ve created four Python files:

  1. getReport.py: to extract the report from an environment.
  2. postReport.py: to create the report in an environment.
  3. testReport.py: to test the report in an environment.
  4. functions.py: contains the functions that are used in the other Python files.

All the files are available on GitHub.

The usage

Before you use the above code, you should configure your SAS Viya environment to enable access to REST APIs. Please follow the first step from this article to register your client.

You should also make sure the environment executing the Python code has Python 3 with the requests package installed. If you're missing the requests package, you will get an error message when executing the Python files.

Get the report

Now that your environment is set up, you can execute the getReport code to extract the report content from your source environment. Below is the command line arguments to pass to execute the code:

python3 getReport.py -a myAdmin -p myAdminPW -sn http://va85.gel.sas.com -an app -as appsecret -rl "/Users/sbxxab/My Folder" -rn CarsReport -o /tmp/CICD/

The parameters are:

  • a - the user that is used to connect to the SAS Viya environment.
  • p - the password of the user.
  • sn - the URL of the SAS Viya environment.
  • an - the name of the application that was defined to enable REST APIs.
  • as - the secret to access the REST APIs.
  • rl - the report location should be between quotes if it contains white spaces.
  • rn - the report name should be between quotes if it contains white spaces.
  • o - the output location that will be used.

The output location should ideally be a Git repository. This allows a commit as the next step in the CI/CD process to keep a history log of any report changes.

The output generated by getReport is a JSON file which has the following structure:

{
"name": "CarsReport",
"location": "/Users/sbxxab/My Folder",
"content": {
"@element": "SASReport",
"xmlns": "http://www.sas.com/sasreportmodel/bird-4.2.4",
"label": "CarsReport",
"dateCreated": "2020-01-14T08:10:31Z",
"createdApplicationName": "SAS Visual Analytics 8.5",
"dateModified": "2020-02-17T15:33:13Z",
"lastModifiedApplicationName": "SAS Visual Analytics 8.5",
"createdVersion": "4.2.4",
"createdLocale": "en",
"nextUniqueNameIndex": 94,
 
...
 
}

From the response:

  • The name value is the report name.
  • The location value is the folder location in the SAS Content.
  • The content value is the BIRD representation of the report.

The generated file is not a result of the direct extraction of the report in the SAS environment. It is a combination of multiple elements to build a file containing the required information to import the report in the target environment.

Version the report

The next step in the process is to commit the file within the Git repository. You can use the git commit command followed by a git push to upload the content to a remote repository. Here are some examples:

# Saving the report in the local repository
git commit -m "Save CarsReport"
 
# Pushing the report to a remote repository after a local commit
git push https://gitlab.sas.com/myRepository.git

Promote the report

As soon as you have saved a version of the report, you can import the report in the target environment (production). This is where the postReport comes into play. Here is a sample command line:

python3 postReport.py -a myAdmin -p myAdminPW -sn http://va85.gel.sas.com -an app -as appsecret -i /tmp/CICD/CarsReport.json

The parameters are:

  • a - the user that is used to connect to the SAS Viya environment.
  • p - the password of the user.
  • sn - the URL of the SAS Viya environment.
  • an - the name of the application that was defined to enable REST APIs.
  • as - the secret to access the REST APIs.
  • i - the input JSON file which contains the output of the getReport.py.

The execution of the code returns nothing except in the case of an error.

Testing the report

You now have access to the report in the production environment. A good practice is to test/validate access to the report. While testing manually in an interface is possible, it's best to automate. Sure, you could validate one report, but what if you had twenty? Use the testReport script to verify the report. Below are the command line arguments to execute the code:

python3 testReport.py -a myAdmin -p myAdminPW -sn http://va85.gel.sas.com -an app -as appsecret -i /tmp/CICD/CarsReport.json

The parameters are:

  • a - the user that is used to connect to the SAS Viya environment.
  • p - the password of the user.
  • sn - the URL of the SAS Viya environment.
  • an - the name of the application that was defined to enable REST APIs.
  • as - the secret to access the REST APIs.
  • i - the input JSON file which contains the output of the getReport.py

The testReport connects to SAS Visual Analytics using REST APIs and generates an image of the first section of the report. The image generation process produces an SVG image of little interest. The most compelling part of the test is the duration of the execution. This gives us an indication of the report's performance.

Throughout the CI/CD process, validation is important and it is also interesting to get a benchmark for the report. This is why the testReport populates a .perf file for the report. The response file has the following structure:

{
"name": "CarsReport",
"location": "/Users/sbxxab/My Folder",
"performance": [
{
"testDate": "2020-03-26T08:55:37.296Z",
"duration": 7.571
},
{
"testDate": "2020-03-26T08:55:56.449Z",
"duration": 8.288
}
]
}

From the response:

  • The name value is the report name.
  • The location value is the folder location in the SAS Content.
  • The performance array contains the date time stamp of the test and the time needed to generate the image.

The file updates for each execution of the testReport code.

Conclusion

CI/CD is important to SAS. Many SAS users need solutions to automate their deployment processes for code, reports, models, etc. This is not something to fear because SAS Viya has the tools required to integrate into CI/CD pipelines. As you have seen in this post, we write code to ease the integration. Even if you don’t have CI/CD tools like Jenkins to orchestrate the deployment, you can execute the different Python files to promote content from one environment to another and test the deployed content.

If you want to get more information about ModelOps, I recommend to have a look at this series.

Continuous Integration/Continuous Delivery – Using Python and REST APIs for SAS Visual Analytics reports was published on SAS Users.

10月 022018
 

I often get asked for programming tips. Here, I share three of my favorite tips for beginners. Tip #1: COUNTC and CATS Functions Together The CATS function concatenates all of its arguments after it strips leading and trailing blanks. The COUNTC function counts characters. Together, they can let you operate [...]

The post Three of My Favorite Programming Tips appeared first on SAS Learning Post.

9月 292016
 

SAS Programming Professionals, SAS & bugs & rock & roll? But, of course! SAS Because of its amazing versatility, SAS is indisputably the greatest software package currently in use anywhere within the Milky Way Galaxy.  Can SAS input every type of flat file imaginable?  Yes!  Can SAS read and write […]

The post SAS & bugs & rock & roll appeared first on SAS Learning Post.

9月 272016
 

Are you ready to broaden your programming skills to land a new job or be a more versatile programmer at your current job? Then this new (and free!) course might be for you. SAS Programming for R Users is a free course developed to allow you to easily transfer your […]

The post Free training course: SAS Programming for R Users appeared first on SAS Learning Post.

6月 072016
 

A reader posed a question in the comments to an earlier Jedi SAS Trick, asking how to write the results of a DS2 DATA _NULL_ program to a text file. It's an interesting question, as DS2 currently has no text file handling statements or capabilities. Take, for example, this traditional […]

The post Jedi SAS Tricks: Writing to Text Files from DS2 appeared first on SAS Learning Post.

5月 022016
 

A recent post, Jedi SAS Tricks: The DATA to DATA Step Macro, engendered a lot of response on Twitter. One of the re-tweets included a call to action - make this a button in Base SAS! Well, Sam, I aim to please! Now, I'm not the guy who builds the SAS Windowing […]

The post Jedi SAS Tricks - Make This a Button in Base SAS appeared first on SAS Learning Post.

3月 122016
 

I was answering questions about SAS in a forum the other day, and it struck me how much easier it is to help folks if they can provide a snippet of data to go along with their program when asking others to help troubleshoot. This makes it easy to run […]

The post Jedi SAS Tricks: The DATA to DATA Step Macro appeared first on SAS Learning Post.

12月 082015
 

Last Christmas I decided to knit a scarf while at a concert in the Bahamas.  A little boy and his sister sitting in front of me watched me very intently. Their eyes grew bigger and bigger as the scarf grew longer and longer. Finally the little boy couldn’t take it […]

The post Top 3 stocking stuffers for coders appeared first on SAS Learning Post.