SAS variables are variables in the statistics sense, not the computer programming sense. SAS has what many computer languages call “variables,” it just calls them “macro variables.” Knowing the difference between SAS variables and SAS macro variables will help you write more flexible and effective code.
The post When a variable is not a variable appeared first on SAS Learning Post.
Plastic pollution in the oceans is becoming a huge problem. And, as with any problem, finding the solution starts with identifying the source of the problem. A recent study estimated that 95% of the plastic pollution in our oceans comes from 10 rivers - let's put some visual analytics to [...]
The post Can you guess which 10 rivers produce 95% of ocean's plastic pollution? appeared first on SAS Learning Post.
You've probably heard about the "80-20 Rule," which describes many natural and manmade phenomena. This rule is sometimes called the "Pareto Principle" because it was discovered by Vilfredo Pareto (1848–1923) who used it to describe the unequal distribution of wealth. Specifically, in his study, 80% of the wealth was held by 20% of the population. The unequal distribution of effort, resources, and other quantities can often be described in terms of the Pareto distribution, which has a parameter that controls the inequity. Whereas some data seem to obey an 80-20 rule, other data are better described as following a "70-30 rule" or a "60-40 rule," and so on. (Although the two numbers do not need to add to 100, it makes the statement pleasingly symmetric.)
I thought about the Pareto distribution recently when I was looking at some statistics for the SAS blogs. I found myself wondering whether 80% of the traffic to a blog is generated by 20% of its posts. (Definition: A blog is a collection of articles. Each individual article is a post.) As you can imagine, some posts are more popular than others. Some posts are timeless. They appear in internet searches when someone searches for a particular statistical or programming technique, even though they were published long ago. Other posts (such as Christmas-themed posts) do not get much traffic from internet searches. They generate traffic for a week or two and then fade into oblivion.
To better understand how various blogs at SAS follow the Pareto Principle, I downloaded data for seven blogs during a particular time period. I then kept only the top 100 posts for each blog.
The following line plot shows the pageviews for one blog. The horizontal axis indicates the posts for this blog, ranked by popularity. (The most popular post is 1, the next most popular post is 2, and so forth.) This blog has four very popular posts that each generated more than 7,000 pageviews during the time period. Another group of four posts were slightly less popular (between 4,000 and 5,000 pageviews). After those eight "blockbuster" posts are the rank-and-file posts that were viewed less than 3,000 times during the time period.
The pageviews for the other blogs look similar. This distribution is also common in book-sales data: most books sell only a few thousand copies whereas the best-sellers (think John Grisham) sell hundreds of millions of copies. Movie revenue is another example that follows this distribution.
The distribution is a long-tailed distribution, a fact that becomes evident if you graph a histogram of the underlying quantity. For the blog, the following histogram shows the distribution of the pageviews for the top 100 posts:
Notice the "power law" nature of the distribution for the first few bars of the histogram. The height of each bar is about 0.4 of the previous height. About 57% of the blog posts had less than 1,000 pageviews. Another 22% had between 1,000 and 2,000 pageviews. The number of rank-and-file posts in each category decreases like a power law, but then the blockbuster posts start to appear. These popular posts give the distribution a very long tail.
Because some blogs (like the one pictured) attract thousands of readers whereas other blogs have fewer readers, you need to standardize the data if you want to compare the distributions for several blogs. Recall that the Pareto Principle is a statement about cumulative percentages. The following graph shows the cumulative percentages of pageviews for seven blogs at SAS (based on the Top 100 posts):
The graph shows a dashed line with slope –1 that is overlaid on the cumulative percentage curves. The places where the dashed line intersects a curve are the "Pareto locations" for which Y% of the pageviews are generated by the first (1-Y)% most popular blog posts. In general, these blogs appear to satisfy a "70-30 rule" because about 70% of the pageviews are generated by the 30 / 100 most popular posts. There is some variation between blogs, with the upper curve following a "72-28 rule" and the lower curve satisfying a "63-37 rule."
All this is approximate and is based on only the top 100 posts for each blog. For a more rigorous analysis, you could use PROC UNIVARIATE or PROC NLMIXED to fit the parameters of the Pareto distribution to the data for each blog. However, I am happy to stop here. In general, blogs at SAS satisfy an approximate "70-30 rule," where 70% of the traffic is generated by the top 30% of the posts.
The post The 80-20 rule for blogs appeared first on The DO Loop.
During SAS Global Forum 2018, SAS instructor Charu Shankar sat down with four SAS users to get their take on what makes a SAS user. Read through to find valuable tips they shared and up your SAS game. I’m sure you will come away inspired, as you discover some universal commonalities in being a SAS user.
The post What makes a SAS user? SAS thinks like me: Dede Schreiber-Gregory appeared first on SAS Learning Post.
During SAS Global Forum 2018, SAS instructor Charu Shankar sat down with four SAS users to get their take on what makes them a SAS user. Read through to find valuable tips they shared and up your SAS game. I’m sure you will come away inspired, as you discover some universal commonalities in being a SAS user.
The post What makes a SAS user? Introverts find their tribe: Richann Watson appeared first on SAS Learning Post.
View more Problem Solvers posts
Have you ever been working in the macro facility and needed a macro function, but you could not locate one that would achieve your task? With the %SYSFUNC macro function, you can access most SAS® functions. In this blog post, I demonstrate how %SYSFUNC can help in your programming needs when a macro function might not exist. I also illustrate the formatting feature that is built in to %SYSFUNC. %SYSFUNC also has a counterpart called %QSYSFUNC that masks the returned value, in case special characters are returned.
%SYSFUNC enables the execution of SAS functions and user-written functions, such as those created with the FCMP procedure. Within the DATA step, arguments to the functions require quotation marks, but because %SYSFUNC is a macro function, you do not enclose the arguments in quotation marks. The examples here demonstrate this.
%SYSFUNC has two possible arguments. The first argument is the SAS function, and the second argument (which is optional) is the format to be applied to the value returned from the function. Suppose you had a report and within the title you wanted to issue today’s date in word format:
title "Today is %sysfunc(today(),worddate20.)"; |
The title appears like this:
"Today is July 4, 2018" |
Because the date is right-justified, there are leading blanks before the date. In this case, you need to introduce another function to remove the blank spaces. Luckily %SYSFUNC enables the nesting of functions, but each function that you use must have its own associated %SYSFUNC. You can rewrite the above example by adding the STRIP function to remove any leading or trailing blanks in the value:
title "Today is %sysfunc(strip(%sysfunc(today(),worddate20.)))"; |
The title now appears like this:
"Today is July 4, 2018" |
The important thing to notice is the use of two separate functions. Each function is contained within its own %SYSFUNC.
Suppose you had a macro variable that contained blank spaces and you wanted to remove them. There is no macro COMPRESS function that removes all blanks. However, with %SYSFUNC, you have access to one. Here is an example:
%let list=a b c; %put %sysfunc(compress(&list)); |
The value that is written to the log is as follows:
abc |
In this last example, I use %SYSFUNC to work with SAS functions where macro functions do not exist.
The example checks to see whether an external file is empty. It uses the following SAS functions: FILEEXIST, FILENAME, FOPEN, FREAD, FGET, and FCLOSE. There are other ways to accomplish this task, but this example illustrates the use of SAS functions within %SYSFUNC.
%macro test(outf); %let filrf=myfile; /* The FILEEXIST function returns a 1 if the file exists; else, a 0 is returned. The macro variable &OUTF resolves to the filename that is passed into the macro. This function is used to determine whether the file exists. In this case you want to find the file that is contained within &OUTF. Notice that there are no quotation marks around the argument, as you will see in all cases below. If the condition is false, the %ELSE portion is executed, and a message is written to the log stating that the file does not exist.*/ %if %sysfunc(fileexist(&outf)) %then %do; /* The FILENAME function returns 0 if the operation was successful; else, a nonzero is returned. This function can assign a fileref for the external file that is located in the &OUTF macro variable. */ %let rc=%sysfunc(filename(filrf,&outf)); /* The FOPEN function returns 0 if the file could not be opened; else, a nonzero is returned. This function is used to open the external file that is associated with the fileref from &FILRF. */ %let fid=%sysfunc(fopen(&filrf)); /* The %IF macro checks to see whether &FID has a value greater than zero, which means that the file opened successfully. If the condition is true, we begin to read the data in the file. */ %if &fid > 0 %then %do; /* The FREAD function returns 0 if the read was successful; else, a nonzero is returned. This function is used to read a record from the file that is contained within &FID. */ %let rc=%sysfunc(fread(&fid)); /* The FGET function returns a 0 if the operation was successful. A returned value of -1 is issued if there are no more records available. This function is used to copy data from the file data buffer and place it into the macro variable, specified as the second argument in the function. In this case, the macro variable is MYSTRING. */ %let rc=%sysfunc(fget(&fid,mystring)); /* If the read was successful, the log will write out the value that is contained within &MYSTRING. If nothing is returned, the %ELSE portion is executed. */ %if &rc = 0 %then %put &mystring; %else %put file is empty; /* The FCLOSE function returns a 0 if the operation was successful; else, a nonzero value is returned. This function is used to close the file that was referenced in the FOPEN function. */ %let rc=%sysfunc(fclose(&fid)); %end; /* The FILENAME function is used here to deassign the fileref FILRF. */ %let rc=%sysfunc(filename(filrf)); %end; %else %put file does not exist; %mend test; %test(c:\testfile.txt) |
There are times when the value that is returned from the function used with %SYSFUNC contains special characters. Those characters then need to be masked. This can be done easily by using %SYSFUNC’s counterpart, %QSYSFUNC. Suppose we run the following example:
%macro test(dte); %put &dte; %mend test; %test(%sysfunc(today(), worddate20.)) |
The above code would generate an error in the log, similar to the following:
1 %macro test(dte); 2 %put &dte; 3 %mend test; 4 5 %test(%sysfunc(today(), worddate20.)) MLOGIC(TEST): Beginning execution. MLOGIC(TEST): Parameter DTE has value July 20 ERROR: More positional parameters found than defined. MLOGIC(TEST): Ending execution. |
The WORDDATE format would return the value like this: July 20, 2017. The comma, to a parameter list, represents a delimiter, so this macro call is pushing two positional parameters. However, the definition contains only one positional parameter. Therefore, an error is generated. To correct this problem, you can rewrite the macro invocation in the following way:
%test(%qsysfunc(today(), worddate20.)) |
The %QSYSFUNC macro function masks the comma in the returned value so that it is seen as text rather than as a delimiter.
For a list of the functions that are not available with %SYSFUNC, see the “How to expand the number of available SAS functions within the macro language was published on SAS Users.
Jim Harris asks: Do you retain and maintain data, or do you have a data retention strategy?
The post Data retention strategy – Part 1 appeared first on The Data Roundtable.
In SAS Visual Analytics 7.4 on 9.4M5 and SAS Visual Analytics 8.2 on SAS Viya, the periodic operators have a new additional parameter that controls how filtering on the date data item used in the calculation affects the calculated values.
The new parameter values are:
These parameter values enable you to improve the appearance of reports based on calculations that use periodic operators. You can have periods that produce missing values for periodic calculations removed from the report, but still available for use in the calculations for later periods. These parameter settings also enable you to provide users with a prompt for choosing the data to display in a report, without having any effect on the calculations themselves.
The following will illustrate the points above, using periodic Revenue calculations based on monthly data from the MEGA_CORP table. New aggregated measures representing Previous Month Revenue (RelativePeriod) and Same Month Last Year (ParallelPeriod) will be displayed as measures in a crosstab. The default _ApplyAllFilters_ is in effect for both, as shown below, but there are no current filters on report or objects.
The Change from Previous Month and Change From Same Month Last Year calculations, respectively, are below:
The resulting report is a crosstab with Date by Month and Product Line in the Row roles, and Revenue, along with the 4 aggregations, in the Column roles. All calculations are accurate, but of course, the calculations result in missing values for the first month (Jan2009) and for the first year (2009).
An improvement to the appearance of the report might be to only show Date by Month values beginning with Jan2010, where there are no non-missing values. Why not apply a filter to the crosstab (shown below), so that the interval shown is Jan2010 to the most recent date?
With the above filter applied to the crosstab, the result is shown below—same problem, different year!
This is where the new parameter on our periodic operators is useful. We would like to have all months used in the calculations, but only the months with non-missing values for both of the periodic calculations shown in the crosstab. So, edit both periodic calculations to change the default _ApplyAllFilters_ to _IgnoreAllTimeFrameFilters_, so that the filters will filter the data in the crosstab, but not for the calculations. When the report is refreshed, only the months with non-missing values are shown:
This periodic operator parameter is also useful if you want to enable users to select a specific month, for viewing only a subset of the crosstab results.
For a selection prompt, add a Drop-Down list to select a MONYY value and define a filter action from the Drop-Down list to the Crosstab. To prevent selection of a month value with missing calculation values, you will also want to apply a filter to the Drop-Down list as you did for the crosstab, displaying months Jan2010 and after in the list.
Now the user can select a month, with all calculations relative to that month displayed, shown in the examples below:
Note that, at this point, since you’ve added the action from the drop-down list to the crosstab, you actually no longer need the filter on the crosstab itself. In addition, if you remove the crosstab filter, then all of your filters will now be from prompts or actions, so you could use the _IgnoreInteractiveTimeFrameFilters_ parameter on your periodic calculations instead of the _IgnoreTimeFrameFilters_ parameter.
You will also notice that, in release 8.2 of SAS Visual Analytics that the performance of the periodic calculations has been greatly improved, with more of the work done in CAS.
Be sure to check out all of the periodic operators, documented here for SAS Visual Analytics 7.4 and SAS Visual Analytics filters on periodic calculations: Apply them or ignore them! was published on SAS Users.
When I was in Denver for SAS Global Forum 2018, one of our customers asked me for three use cases that show strong return on investment for analytics in today's electric utility industry. Since our energy team has identified at least 125 separate use cases that show how analytics can [...]
Three utility use cases show the value of analytics was published on SAS Voices by David Pope
There were 97 e-posters in The Quad demo room at SAS Global Forum this year. And the one that caught my eye was Ted Conway's "Periodic Table of Introductory SAS ODS Graphics Examples." Here's a picture of Ted fielding some questions from an interested user... He created a nice/fun graphic, [...]
The post A periodic table to help you with your SAS ODS graphics! appeared first on SAS Learning Post.