6月 082018
 

SAS Viya provides import and export functionality for user-created content like reports and data plans. Often, in addition to content, an administrator will want to save configuration so that it can be reloaded or updated and applied to a different system. SAS Viya provides the capability to save and reload configuration using the SAS Viya command-line interfaces that are previous blog post.

The

It is possible to save a set of configuration settings and reload them to the same or a different system. This can be useful when you have your configuration established and you wish to keep a backup, or make a selective backup of configuration prior to making a change.

The connection to LDAP is a key early step in a SAS Viya implementation. With the configuration CLI, once you have the SAS Viya LDAP configuration established, you can export it to a file, and then use that file (with any necessary modifications) to stage additional systems, or as a backup prior to making changes to your existing systems configuration.

How to save and reload configuration

As always, when using the command-line interfaces you must

./sas-admin configuration configurations list --definition-name sas.identities.providers.ldap.user  --service identities

 

Next, using the id from the previous step you can list the configuration properties.

./sas-admin configuration configurations show -id b313a5a7-1c73-4f4a-9d3d-bba05b626939

 

Save LDAP Configuration

The save process creates json files. The following steps use the download command to save to json files the connection, user and group configuration instances for the SAS Viya connection to LDAP.

./sas-admin configuration configurations download --target /tmp/ldapconnection.json  --definition-name sas.identities.providers.ldap.connection  --service identities
 
 
./sas-admin configuration configurations download --target /tmp/user.json  --definition-name sas.identities.providers.ldap.user  --service identities
 
 
./sas-admin configuration configurations download --target /tmp/group.json  --definition-name sas.identities.providers.ldap.group  --service identities

 

You should open the json files and check that the correct configuration has been saved. It is possible for the process to complete without errors and return json that is not what you are expecting. This would cause problems with your reload, so checking the saved json is important.

You can keep the JSON file as is, or make changes to key attributes. You may want to do this if you are importing to a different system.

Load the SAS Viya LDAP Configuration

To load you simply use the update command and pass the json file.

./sas-admin configuration configurations update --file /tmp/ldapconnection.json
 
./sas-admin configuration configurations update --file /tmp/user.json
 
./sas-admin configuration configurations update --file /tmp/group.json

 

The impact of isDefault

There is a value, isDefault, stored within the configuration which has an impact on the persistence of changes made to configuration.

isDefault impacts how services treat existing configuration when a service starts. When a service starts a setting of:

  • isDefault=true in the existing configuration means the service will overwrite the configuration object with new defaults.
  • isDefault=false in the existing configuration means the service will NOT overwrite the existing configuration object.

In other words, if the configuration is flagged as “default” then the service is permitted to update or add to the default values.

Objects created by the services at startup always have isDefault set to true. Objects created in Environment Manager always have isDefault set to false. This means changes in Environment Manager are always respected by services on restart, they will not be overwritten.  But services are allowed to overwrite their own defaults at startup.

When using the CLI, the administrator needs to decide what is the appropriate value for isDefault. If you require the configuration change to persist across service restarts then set isDefault=false.

Saving and Reloading Micro-Service Logging Levels

Let’s look at another use case for save and reload of configuration. Updating micro-service logging configuration levels in batch can be very useful. You may want to save your current logging configuration and modify it to raise logging levels. You may create multiple json files with different logging configurations for different scenarios. When debugging an issue in the environment you could load a verbose logging configuration. If you wish to keep the new configuration you would edit the json and set IsDefault=false.

The step below saves all configuration instances created from the logging.level configuration definition. These configuration instances control the logging level for the SAS Viya microservices and servers.

./sas-admin configuration configurations download --definition-name logging.level -target /tmp/default_logging.level.txt

 

If you wish to persist your new logging configuration, edit the file to set metadata.isDefault=false, save the new file and then and update the logging configuration using the update command:

./sas-admin configuration configurations update --file /tmp/new_logging.level.txt

 

When you are done, you can use the original file to reset the logging level back to default values.

In most cases a server restart is not required after a configuration update, find details in the administration guide.

Saving and reloading SAS Viya configuration was published on SAS Users.

6月 082018
 

I recently recorded a short video about the new syntax for specifying and manipulating lists in SAS/IML 14.3. This is a video of my Super Demo at SAS Global Forum 2018. The new syntax supports dynamic arrays, associative arrays ("named lists"), and hierarchical data structures such as lists of lists.


If your browser does not support embedded video, you can go directly to the video on YouTube.

If you prefer to read about lists, the following references provide more information and additional examples of ways to use lists:

The post Video: A new syntax for lists in SAS/IML appeared first on The DO Loop.

6月 082018
 

Who doesn’t love a good makeover story? We know we do at BNL Consulting—and so do our customers. That’s why we’re thrilled with SAS Visual Analytics sophisticated new look and feel as well as its expanded functionality. SAS Visual Analytics takes advantage of the performance and scalability of the SAS Viya platform, providing a Business Intelligence framework that can work with massive amounts of data, bringing forward the powerful analytics that has made SAS the market leader in this space.

With SAS Viya, we get an even more unified platform with a consistent look and feel across applications which have been rewritten as zero client HTML5 web applications. The visualizations that are available in SAS Visual Analytics are embedded within the other tools as well. Now customers—from analysts to business executives—can point and click all from their web browser to get the information they need to make better business decisions right out of the box.

The newest version of SAS Visual Analytics provides direct access to an in-memory set of data as well. The process of loading data that can be made available to SAS Visual Analytics has been streamlined to give users fast access to information to create dashboards, reports and perform ad-hoc analysis. And because the application is so intuitive, users can be up and running in a matter of hours, not days.

In what has become a highly competitive market for business analytics—with BI tools being ubiquitous—SAS Visual Analytics running on SAS Viya provides a formidable means to analyze data, helping organizations tell a story—simply and easily. With SAS Visual Analytics on their side, customers will no longer need to reach outside the platform and make additional purchases. The unified platform is extremely compelling.

New features and benefits

SAS Visual Analytics customers will also benefit from a number of new features, including:

  • A new look and feel that provides drag and drop layout controls to size dashboard objects. New graph, chart, and map objects offer greater control of the properties to make them presentation-ready.
  • Objects on the screen can “talk” to each other, listening in on object events to filter accordingly in unison.
  • With no more LASR server, the process of loading and accessing data is much faster, with even better performance than the previous in-memory LASR server architecture.
  • There is now the ability to create custom HTML5/JavaScript objects, which can be hooked up to the data loaded in memory (data-driven content). HTML/JavaScript/D3 has become the standard in web development, opening up almost countless options for SAS to visualize data and turn it into useful information. Before, custom development was sometimes needed, but SAS Visual Analytics can handle a customer’s needs, all with point and click, drag and drop.

Next steps

When organizations use BI tools, they expect appealing visuals on top of rich data that can tell a story. SAS Visual Analytics on SAS Viya delivers in a big way, ushering in a new era for SAS BI users. We encourage customers to try out SAS Visual Analytics and see for themselves. We have found that a successful path is to start with small-scale proof of concepts, designed to satisfy a few critical uses. As you become more comfortable, expand to include more users and different types of projects.

With all the software options out there—both COTS and Open Source—we want customers to feel confident that SAS Visual Analytics for SAS Viya is the right choice. For more details, you can contact BNL at BNLConsulting.com.

SAS Visual Analytics on the new SAS platform was published on SAS Users.

6月 082018
 

Who doesn’t love a good makeover story? We know we do at BNL Consulting—and so do our customers. That’s why we’re thrilled with SAS Visual Analytics sophisticated new look and feel as well as its expanded functionality. SAS Visual Analytics takes advantage of the performance and scalability of the SAS Viya platform, providing a Business Intelligence framework that can work with massive amounts of data, bringing forward the powerful analytics that has made SAS the market leader in this space.

With SAS Viya, we get an even more unified platform with a consistent look and feel across applications which have been rewritten as zero client HTML5 web applications. The visualizations that are available in SAS Visual Analytics are embedded within the other tools as well. Now customers—from analysts to business executives—can point and click all from their web browser to get the information they need to make better business decisions right out of the box.

The newest version of SAS Visual Analytics provides direct access to an in-memory set of data as well. The process of loading data that can be made available to SAS Visual Analytics has been streamlined to give users fast access to information to create dashboards, reports and perform ad-hoc analysis. And because the application is so intuitive, users can be up and running in a matter of hours, not days.

In what has become a highly competitive market for business analytics—with BI tools being ubiquitous—SAS Visual Analytics running on SAS Viya provides a formidable means to analyze data, helping organizations tell a story—simply and easily. With SAS Visual Analytics on their side, customers will no longer need to reach outside the platform and make additional purchases. The unified platform is extremely compelling.

New features and benefits

SAS Visual Analytics customers will also benefit from a number of new features, including:

  • A new look and feel that provides drag and drop layout controls to size dashboard objects. New graph, chart, and map objects offer greater control of the properties to make them presentation-ready.
  • Objects on the screen can “talk” to each other, listening in on object events to filter accordingly in unison.
  • With no more LASR server, the process of loading and accessing data is much faster, with even better performance than the previous in-memory LASR server architecture.
  • There is now the ability to create custom HTML5/JavaScript objects, which can be hooked up to the data loaded in memory (data-driven content). HTML/JavaScript/D3 has become the standard in web development, opening up almost countless options for SAS to visualize data and turn it into useful information. Before, custom development was sometimes needed, but SAS Visual Analytics can handle a customer’s needs, all with point and click, drag and drop.

Next steps

When organizations use BI tools, they expect appealing visuals on top of rich data that can tell a story. SAS Visual Analytics on SAS Viya delivers in a big way, ushering in a new era for SAS BI users. We encourage customers to try out SAS Visual Analytics and see for themselves. We have found that a successful path is to start with small-scale proof of concepts, designed to satisfy a few critical uses. As you become more comfortable, expand to include more users and different types of projects.

With all the software options out there—both COTS and Open Source—we want customers to feel confident that SAS Visual Analytics for SAS Viya is the right choice. For more details, you can contact BNL at BNLConsulting.com.

SAS Visual Analytics on the new SAS platform was published on SAS Users.

6月 082018
 

Who doesn’t love a good makeover story? We know we do at BNL Consulting—and so do our customers. That’s why we’re thrilled with SAS Visual Analytics sophisticated new look and feel as well as its expanded functionality. SAS Visual Analytics takes advantage of the performance and scalability of the SAS Viya platform, providing a Business Intelligence framework that can work with massive amounts of data, bringing forward the powerful analytics that has made SAS the market leader in this space.

With SAS Viya, we get an even more unified platform with a consistent look and feel across applications which have been rewritten as zero client HTML5 web applications. The visualizations that are available in SAS Visual Analytics are embedded within the other tools as well. Now customers—from analysts to business executives—can point and click all from their web browser to get the information they need to make better business decisions right out of the box.

The newest version of SAS Visual Analytics provides direct access to an in-memory set of data as well. The process of loading data that can be made available to SAS Visual Analytics has been streamlined to give users fast access to information to create dashboards, reports and perform ad-hoc analysis. And because the application is so intuitive, users can be up and running in a matter of hours, not days.

In what has become a highly competitive market for business analytics—with BI tools being ubiquitous—SAS Visual Analytics running on SAS Viya provides a formidable means to analyze data, helping organizations tell a story—simply and easily. With SAS Visual Analytics on their side, customers will no longer need to reach outside the platform and make additional purchases. The unified platform is extremely compelling.

New features and benefits

SAS Visual Analytics customers will also benefit from a number of new features, including:

  • A new look and feel that provides drag and drop layout controls to size dashboard objects. New graph, chart, and map objects offer greater control of the properties to make them presentation-ready.
  • Objects on the screen can “talk” to each other, listening in on object events to filter accordingly in unison.
  • With no more LASR server, the process of loading and accessing data is much faster, with even better performance than the previous in-memory LASR server architecture.
  • There is now the ability to create custom HTML5/JavaScript objects, which can be hooked up to the data loaded in memory (data-driven content). HTML/JavaScript/D3 has become the standard in web development, opening up almost countless options for SAS to visualize data and turn it into useful information. Before, custom development was sometimes needed, but SAS Visual Analytics can handle a customer’s needs, all with point and click, drag and drop.

Next steps

When organizations use BI tools, they expect appealing visuals on top of rich data that can tell a story. SAS Visual Analytics on SAS Viya delivers in a big way, ushering in a new era for SAS BI users. We encourage customers to try out SAS Visual Analytics and see for themselves. We have found that a successful path is to start with small-scale proof of concepts, designed to satisfy a few critical uses. As you become more comfortable, expand to include more users and different types of projects.

With all the software options out there—both COTS and Open Source—we want customers to feel confident that SAS Visual Analytics for SAS Viya is the right choice. For more details, you can contact BNL at BNLConsulting.com.

SAS Visual Analytics on the new SAS platform was published on SAS Users.

6月 072018
 

Light lengthens the day and allows us more time to learn, socialize, contemplate and create. Exploring NASA nighttime satellite images shows how illumination patterns have changed over time. Increases and decreases in illumination show the effects of human civilization on earth. From population collapse and destruction in war zones to economic [...]

The dark side: analyzing global changes in nighttime illumination was published on SAS Voices by Falko Schulz

6月 062018
 

Log Management in SAS ViyaLogs. They can be an administrator’s best friend or a thorn in their side. Who among us hasn’t seen a system choked to death by logs overtaking a filesystem? Thankfully, the chances of that happening with your SAS Viya 3.3 deployment is greatly reduced due to the automatic management of log files by the SAS Operations Infrastructure, which archives log files every day.

With a default installation of SAS Viya 3.3, log files older than three days will be automatically zipped up, deleted, and stored in /opt/sas/viya/config/var/log. This process is managed by the sas-ops-agent process on each machine in your deployment. According to SAS R&D, this results in up to a 95% compression rate on the overall log file requirements.

The task that archives log files is managed by the sas-ops-agent on each machine.  Running ./sas-ops tasks shows that the LogFileArchive process runs daily at 0400:

 {
 "version": 1,
 "taskName": "LogfileArchive",
 "description": "Archive daily",
 "hostType": "linux",
 "runType": "time",
 "frequency": "0000-01-01T04:00:00-05:00",
 "maxRunTime": "2h0m0s",
 "timeOutAction": "restart",
 "errorAction": "cancel",
 "command": "sas-archive",
 "commandType": "sas",
 "publisherType": "none"
 },

While the logs are zipped to reduce size, the zip files are stored locally so additional maintenance may be required to move the zipped files to another location. For example, my test system illustrates that the zip files are retained once created.

[sas@intviya01 log]$ ll
total 928
drwxr-xr-x. 3 sas sas 20 Jan 24 14:19 alert-track
drwxrwxr-x. 3 sas sas 20 Jan 24 16:13 all-services
drwxr-xr-x. 3 sas sas 20 Jan 24 14:23 appregistry
...
drwxr-xr-x. 4 sas sas 31 Jan 24 14:20 evmsvrops
drwxr-xr-x. 3 sas sas 20 Jan 24 14:27 home
drwxr-xr-x. 3 root sas 20 Jan 24 14:18 httpproxy
drwxr-xr-x. 3 sas sas 20 Jan 24 14:35 importvaspk
-rw-r--r--. 1 sas sas 22 Jan 26 04:00 log-20180123090001Z.zip
-rw-r--r--. 1 sas sas 22 Jan 27 04:00 log-20180124090000Z.zip
-rw-r--r--. 1 sas sas 22 Jan 28 04:00 log-20180125090000Z.zip
-rw-r--r--. 1 sas sas 10036 Jan 29 04:00 log-20180126090000Z.zip
-rw-r--r--. 1 sas sas 366303 Mar 6 04:00 log-20180303090000Z.zip
-rw-r--r--. 1 sas sas 432464 Apr 3 04:00 log-20180331080000Z.zip
-rw-r--r--. 1 sas sas 22 Apr 4 04:00 log-20180401080000Z.zip
-rw-r--r--. 1 sas sas 22 Apr 5 04:00 log-20180402080000Z.zip
-rw-r--r--. 1 sas sas 15333 Apr 6 04:00 log-20180403080000Z.zip
-rw-r--r--. 1 sas sas 21173 Apr 7 04:00 log-20180404080000Z.zip
-rw-r--r--. 1 sas sas 22191 Apr 8 04:00 log-20180405080000Z.zip
-rw-r--r--. 1 sas sas 21185 Apr 9 04:00 log-20180406080000Z.zip
-rw-r--r--. 1 sas sas 21405 Apr 10 04:00 log-20180407080000Z.zip
drwxr-xr-x. 3 sas sas 20 Jan 24 14:33 monitoring

If three days is too short of a time to retain logs, you can adjust the default timeframe for archiving logs by modifying the default task list for the sas-ops-agent on each machine.

Edit the tasks.json file to suit your needs and then issue a command to modify the task template for sas-ops-agent processes. For example, this will modify the log archive process to retain seven days of information:

...  
{
 "version": 1,
 "taskName": "LogfileArchive",
 "description": "Archive daily",
 "hostType": "linux",
 "runType": "time",
 "frequency": "0000-01-01T04:00:00-05:00",
 "maxRunTime": "2h0m0s",
 "timeOutAction": "restart",
 "errorAction": "cancel",
 "command": "sas-archive -age 7",
 "commandType": "sas",
 "publisherType": "none"
 },
...
 
$ ./sas-ops-agent import -tasks tasks.jason

Restart the sas-ops-agent processes on each of your machines and you will be good to go.

I hope you found this post helpful.

Additional Resources for Administrators

SAS Administrators Home Page
How-to Videos for Administrators
SAS Administration Community
SAS Administrators Blogs
SAS Administrator Training
SAS Administrator Certification

Log Management in SAS Viya 3.3 was published on SAS Users.

6月 062018
 

The SURVEYSELECT procedure in SAS 9.4M5 supports the OUTRANDOM option, which causes the selected items in a simple random sample to be randomly permuted after they are selected. This article describes several statistical tasks that benefit from this option, including simulating card games, randomly permuting observations in a DATA step, assigning a random ID to patients in a clinical study, and generating bootstrap samples. In each case, the new OUTRANDOM option reduces the number of statements that you need to write. The OUTRANDOM option can also be specified by using OUTORDER=RANDOM.

Sample data with PROC SURVEYSELECT

Often when you draw a random sample (with or without replacement) from a population, the order in which the items were selected is not important. For example, if you have 10 patients in a clinical trial and want to randomly assign five patients to the control group, the control group does not depend on the order in which the patients were selected. Similarly, in simulation studies, many statistics (means, proportions, standard deviations,...) depend only on the sample, not on the order in which the sample was generated. For these reasons, and for efficiency, the SURVEYSELECT procedure in SAS uses a "one-pass" algorithm to select observations in the same order that they appear in the "population" data set.

However, sometimes you might require the output data set from PROC SURVEYSELECT to be in a random order. For example, in a poker simulation, you might want the output of PROC SURVEYSELECT to represent a random shuffling of the 52 cards in a deck. To be specific, the following DATA step generates a deck of 52 cards in order: Aces first, then 2s, and so on up to jacks, queens, and kings. If you use PROC SURVEYSELECT and METHOD=SRS to select 10 cards at random (without replacement), you obtain the following subset:

data CardDeck;
length Face $2 Suit $8;
do Face = 'A','2','3','4','5','6','7','8','9','10','J','Q','K';
   do Suit = 'Clubs', 'Diamonds', 'Hearts', 'Spades';
      CardNumber + 1;
      output;
   end;
end;
run;
 
/* Deal 10 cards. Order is determined by input data */
proc surveyselect data=CardDeck out=Deal noprint
     seed=1234 method=SRS /* sample w/o replacement */
     sampsize=10;         /* number of observations in sample */
run;
 
proc print data=Deal; run;
Random sample without replacement from a card deck. The sample is in the same order as the data.

Notice that the call to PROC SURVEYSELECT did not use the OUTRANDOM option. Consequently, the cards are in the same order as they appear in the input data set. This sample is adequate if you want to simulate dealing hands and estimate probabilities of pairs, straights, flushes, and so on. However, if your simulation requires the cards to be in a random order (for example, you want the first five observations to represent the first player's cards), then clearly this sample is inadequate and needs an additional random permutation of the observations. That is exactly what the OUTRANDOM option provides, as shown by the following call to PROC SURVEYSELECT:

/* Deal 10 cards in random order */
proc surveyselect data=CardDeck out=Deal2 noprint
     seed=1234 method=SRS /* sample w/o replacement */
     sampsize=10          /* number of observations in sample */
     OUTRANDOM;           /* SAS/STAT 14.3: permute order */
run;
 
proc print data=Deal2; run;
Random sample without replacement from a card deck. The sample is then permuted into a random order.

You can use this sample when the output needs to be in a random order. For example, in a poker simulation, you can now assign the first five cards to the first player and the second five cards to a second player.

Permute the observations in a data set

A second application of the OUTRANDOM option is to permute the rows of a SAS data set. If you sample without replacement and request all observations (SAMPRATE=1), you obtain a copy of the original data in random order. For example, the students in the Sashelp.Class data set are listed in alphabetical order by their name. The following statements use the OUTRANDOM option to rearrange the students in a random order:

/* randomly permute order of observations */
proc surveyselect data=Sashelp.Class out=RandOrder noprint
     seed=123 method=SRS /* sample w/o replacement */
     samprate=1          /* proportion of observations in sample */
     OUTRANDOM;          /* SAS/STAT 14.3: permute order */
run;
 
proc print data=RandOrder; run;
Use PROC SURVEYSELECT to produce a random ordering of a SAS data set

There are many other ways to permute the rows of a data set, such as adding a uniform random variable to the data and then sorting. The two methods are equivalent, but the code for the SURVEYSELECT procedure is shorter to write.

Assign unique random IDs to patients in a clinical trial

Another application of the OUTRANDOM option is to assign a unique random ID to participants in an experimental trial. For example, suppose that four-digit integers are used for an ID variable. Some clinical trials assign an ID number sequentially to each patient in the study, but I recently learned from a SAS discussion forum that some companies assign random ID values to subjects. One way to assign random IDs is to sample randomly without replacement from the set of all ID values. The following DATA step generates all four-digit IDs, selects 19 of them in random order, and then merges those IDs with the participants in the study:

data AllIDs;
do ID = 1000 to 9999;  /* create set of four-digit ID values */
   output;
end;
run;
 
/* randomly select 19 unique IDs */
proc surveyselect data=AllIDs out=ClassIDs noprint
     seed=12345 method=SRS  /* sample w/o replacement */
     sampsize=19            /* number of observations in sample */
     OUTRANDOM;             /* SAS/STAT 14.3: permute order */
run;
 
data Class;
   merge ClassIDs Sashelp.Class;   /* merge ID variable and subjects */
run;
 
proc print data=Class; var ID Name Sex Age; run;

Random order for other sampling methods

The OUTRANDOM option also works for other sampling schemes, such as sampling with replacement (METHOD=URS, commonly used for bootstrap sampling) or stratified sampling. If you use the REPS= option to generate multiple samples, each sample is randomly ordered.

It is worth mentioning that the SAMPLE function in SAS/IML also can to perform a post-selection sort. Suppose that X is any vector that contains N elements. Then the syntax SAMPLE(X, k, "NoReplace") generates a random sample of k elements from the set of N. The documentation states that "the elements ... might appear in the same order as in X." This is likely to happen when k is almost equal to N. If you need the sample in random order, you can use the syntax SAMPLE(X, k, "WOR") which adds a random sort after the sample is selected, just like PROC SURVEYSELECT does when you use the OUTRANDOM option.

The post Sample and obtain the results in random order appeared first on The DO Loop.

6月 062018
 

You've probably seen in the news that a volcano erupted in Guatemala recently. But do you really know much about this volcano, or even where it's located? Hopefully this blog post will get you up to speed on your volcanology! Pictures of a recent volcano erupting in Hawaii show slow-moving [...]

The post Where is that volcano in Guatemala? appeared first on SAS Learning Post.