SAS Professional Services

11月 202017
 

Many SAS users have inquired about SAS Cloud Analytic Services’ (CAS) Distributed Network File System (Learn more about CAS.)

The “NFS” in “DNFS”

Let’s start at the beginning. The “NFS” in DNFS stands for “Network File System” and refers to the ability to share files across a network. As the picture below illustrates, a network file system lets numerous remote hosts access another host’s files.

Understanding DNFS

NFS

There are numerous network file system protocols that can be used for file sharing – e.g. CIFS, Ceph, Lustre – but the most common on Linux is NFS. While NFS is the dominant file-sharing protocol, the “NFS” part of the DNFS does not correspond to the NFS protocol. Currently all the DNFS supported file systems are based on NFS, but DNFS may support file systems based on other protocols in the future. So, it’s best to think of the “NFS” part of “DNFS” as a generic “network file system” (clustered file system) and not the specific NFS protocol.

The “D” in “DNFS”

The “D” in DNFS stands for “Distributed” but it does not refer to the network file system. By definition, that is already distributed since the file system is external to the machines accessing it. The “Distributed” in DNFS refers to CAS’ ability to use a network file system in a massively parallel way. With a supported file system mounted identically on each CAS node, CAS can access (both write) the file system’s CSV and SASHDAT files from every worker node in parallel.

This parallel file access is not an attribute of the file system, it is a capability of CAS. By definition, network file systems facilitate access at the file level, not the block level. With DNFS, CAS actively manages parallel block level I/O to the network file system, making sure file seek locations and block I/O operations do not overlap, etc.

DNFS

 

DNFS as CAS Backing Store

Not only can CAS perform multi-machine parallel I/O from network file systems, it can also memory-map NFS SASHDAT files directly into CAS. Thus, SASHDAT files on DNFS act as both the CASlib data source as well as the virtual memory “backing store,” often negating the need for CAS to utilize memory mapping (mmap()).

Note 1: Data transformations on load, such as row filtering and field selection, as well as encryption can trigger CAS_DISK_CACHE usage. Since the data must be transformed (subset and/or decrypted), CAS copies the transformed data into CAS_DISK_CACHE to support CAS processing.

Note 2: It is possible to use DNFS atop an encrypting file system or hardware storage device. Here, the HDAT blocks are stored encrypted but transmitted to the CAS I/O layer decrypted. Assuming no other transformations, CAS_DISK_CACHE will not be used in this scenario.

DNFS Memory Mapping

Performance Considerations

DNFS-based CAS loading will only be as fast as the slowest component involved. The chosen NFS architecture (hardware and CAS connectivity) should support I/O throughput commensurate with the CAS installation and in-line with the implementation’s service level agreements. So, while NetApp ONTAP clustering architecture. A different file system technology might look a little different but the same basic ideas will apply.

DNFS w/ Multi Machine File System

As described earlier, CAS manages the parallel I/O operations. Requests from CAS are sent to the appliance and handled by the NFS metadata server. The storage device implementing the NFS protocol points CAS DNFS to the proper file and block locations on the NFS data servers which pass the data to the CAS worker nodes directly.

Understanding DNFS was published on SAS Users.

11月 162017
 

As a SAS Viya user, you may be wondering whether it is possible to execute data append and data update concurrently to a global Cloud Analytic Services (CAS) table from two or more CAS sessions. (Learn more about CAS.) How would this impact the report view while data append or data update is running on a global CAS table? These questions are even more important for those using the programming interface to load and update data in CAS. This post discusses data append, data update, and concurrency in CAS.

Two or more CAS sessions can simultaneously submit a data append and data update process to a CAS table, but only one process at a time can run against the same CAS table. The multiple append and update processes execute in serial, one after another, never running in a concurrent fashion. Whichever CAS session is first to acquire the write lock on a global CAS table prevails, appending or updating the data first. The other append and update processes must wait in a queue to acquire the write lock.

During the data append process, the appended data is not available to end users or reports until all rows are inserted and committed into the CAS table. While data append is running, users can still render reports against the CAS table using the original data, but excluding the appended rows.

Similarly, during the data update process, the updated data is not available to users or reports until the update process is complete. However, CAS lets you render reports using the original (non-updated) data, as the CAS table is always available for the read process. During the data update process, CAS makes additional copies into memory of the to-be-updated blocks containing rows in order to perform the update statement. Once the update process is complete, the additional and now obsolete copies of blocks, are removed from CAS. Data updates to a global CAS table is an expensive operation in terms of CPU and memory usage. You have to factor in the additional overhead memory or CAS_CACHE space to support the updates. The space requirement depends on the number of rows being affected by the update process.

At any given time, there could be only one active write process (append/update) against a global CAS table. However, there could be many concurrent active read processes against a global CAS table. A global CAS table is always available for read processes, even when an append or update process is running on the same CAS table.

The following log example describes two simultaneous CAS sessions executing data appends to a CAS table. Both append processes were submitted to CAS with a gap of a few seconds. Notice the execution time for the second CAS session MYSESSION1 is double the time that it took the first CAS session to append the same size of data to the CAS table. This shows that both appends were executing one after another. The amount of memory used and the CAS_CACHE location also shows that both processes were running one after another in a serial fashion.

Log from simultaneous CAS session MYSESSION submitting APPEND

58 proc casutil ;
NOTE: The UUID '1411b6f2-e678-f546-b284-42b6907260e9' is connected using session MYSESSION.
59 load data=mydata.big_prdsale
60 outcaslib="caspath" casout="big_PRDSALE" append ;
NOTE: MYDATA.BIG_PRDSALE was successfully added to the "caspath" caslib as "big_PRDSALE".
61 quit ;
NOTE: PROCEDURE CASUTIL used (Total process time):
real time 49.58 seconds
cpu time 5.05 seconds

Log from simultaneous CAS session MYSESSION1 submitting APPEND

58 proc casutil ;
NOTE: The UUID 'a20a246e-e0cc-da4d-8691-c0ff2a222dfd' is connected using session MYSESSION1.
59 load data=mydata.big_prdsale1
60 outcaslib="caspath" casout="big_PRDSALE" append ;
NOTE: MYDATA.BIG_PRDSALE1 was successfully added to the "caspath" caslib as "big_PRDSALE".
61 quit ;
NOTE: PROCEDURE CASUTIL used (Total process time):
real time 1:30.33
cpu time 4.91 seconds

 

When the data append process from MYSESSION1 was submitted alone (no simultaneous process), the execution time is around the same as for the first session MYSESSION. This also shows that when two simultaneous append processes were submitted against the CAS table, one was waiting for the other to finish. At one time, only one process was running the data APPEND action to the CAS table (no concurrent append).

Log from a lone CAS session MYSESSION1 submitting APPEND

58 proc casutil ;
NOTE: The UUID 'a20a246e-e0cc-da4d-8691-c0ff2a222dfd' is connected using session MYSESSION1.
59 load data=mydata.big_prdsale1
60 outcaslib="caspath" casout="big_PRDSALE" append ;
NOTE: MYDATA.BIG_PRDSALE1 was successfully added to the "caspath" caslib as "big_PRDSALE".
61 quit ;
NOTE: PROCEDURE CASUTIL used (Total process time):
real time 47.63 seconds
cpu time 4.94 seconds

 

The following log example describes two simultaneous CAS sessions submitting data updates on a CAS table. Both update processes were submitted to CAS in a span of a few seconds. Notice the execution time for the second CAS session MYSESSION1 is double the time it took the first session to update the same number of rows. The amount of memory used and the CAS_CACHE location also shows that both processes were running one after another in a serial fashion. While the update process was running, memory and CAS_CACHE space increased, which suggests that the update process makes copies of to-be-updated data rows/blocks. Once the update process is complete, the space usage in memory/CAS_CACHE returned to normal.

When the data UPDATE action from MYSESSION1 was submitted alone (no simultaneous process), the execution time is around the same as for the first CAS session.

Log from a simultaneous CAS session MYSESSION submitting UPDATE

58 proc cas ;
59 table.update /
60 set={
61 {var="actual",value="22222"},
62 {var="country",value="'FRANCE'"}
63 },
64 table={
65 caslib="caspath",
66 name="big_prdsale",
67 where="index in(10,20,30,40,50,60,70,80,90,100 )"
68 }
69 ;
70 quit ;
NOTE: Active Session now MYSESSION.
{tableName=BIG_PRDSALE,rowsUpdated=86400}
NOTE: PROCEDURE CAS used (Total process time):
real time 4:37.68
cpu time 0.05 seconds

 

Log from a simultaneous CAS session MYSESSION1 submitting UPDATE

57 proc cas ;
58 table.update /
59 set={
60 {var="actual",value="22222"},
61 {var="country",value="'FRANCE'"}
62 },
63 table={
64 caslib="caspath",
65 name="big_prdsale",
66 where="index in(110,120,130,140,150,160,170,180,190,1100 )"
67 }
68 ;
69 quit ;
NOTE: Active Session now MYSESSION1.
{tableName=BIG_PRDSALE,rowsUpdated=86400}
NOTE: PROCEDURE CAS used (Total process time):
real time 8:56.38
cpu time 0.09 seconds

 

The following memory usage snapshot from one of the CAS nodes describes the usage of memory before and during the CAS table update. Notice the values for “used” and “buff/cache” columns before and during the CAS table update.

Memory usage on a CAS node before starting a CAS table UPDATE

Memory usage on a CAS node during CAS table UDPATE

Summary

When simultaneous data append and data update requests are submitted against a global CAS table from two or more CAS sessions, they execute in a serial fashion (no concurrent process execution). To execute data updates on a CAS table, you need an additional overhead memory/CAS_CACHE space. While the CAS table is going through the data append or data update process, the CAS table is still accessible to rendering reports.

Concurrent data append and update to a global CAS table was published on SAS Users.

11月 142017
 

SAS Visual Analytics 7.4 has added the support for date parameters. Recall from my first post,  Using parameters in SAS Visual Analytics, a parameter is a variable whose value can be changed at any time by the report viewer and referenced by other report objects. These objects can be a calculated item, aggregated measure, filter, rank or display rule. And remember, every time the parameter is changed, the corresponding objects are updated to reflect that change.

Here is my updated table that lists the supported control objects and parameter types for SAS Visual Analytics 7.4. The type of parameter is required to match the type of data that is assigned to the control.
Notice that SAS Visual Analytics 7.4 has also introduced the support for multiple value selection control objects. I’ll address these in another blog.

Using Date Parameters in your SAS Visual Analytics Reports

Let’s look at an example of a SAS Visual Analytics Report using date parameters. In this fictitious report, we have been given the requirements that the user wants to pick two independent date periods for comparison. This is not the same requirement as filtering the report between a start and end date. This report requirement is such that a report user can pick two independent months in the source data to be able to analyze the change in Expense magnitude for different aggregation levels, such as Region, Product Line and Product.

In this example, we will compare two different Month,Year periods. This could easily be two different Quarter,Year or Week,Year periods; depending on the report requirements, these same steps can be applied.

In this high level breakdown, you can see in red I will create two date parameters from data driven drop-down lists. From these parameter values, I will create two calculated data items, shown in purple, and one aggregated measure that will be used in three different report objects, shown in green.

Here are the steps:

1.     Create the date parameters.

2.     Add the control objects to the report and assign roles.

3.     Create the dependent data items, i.e. the calculated data items and aggregated measure.

4.     Add the remaining report objects to the canvas and assign roles.

Step 1: Create the date parameters

First we will need to create the date parameters that will hold the values made by the report viewers. From the Data Pane, use the drop-down menu and select New Parameter….

Then create your first parameter as shown below. Give it a name.

Next, select minimum and maximum values allowed for this parameter. I used the min and max available in my data source, but you could select a more narrow range if you wanted to restrict the users to only have access to portions of the data, just so long as the values are in your data source since, in this example, we will use the data source to populate the available values in the drop-down list.

Then select a current value, this will serve as the default value that will populate when a user first opens the report.

Finally, select the format in which you want your data item to be formatted. I selected the same format as my underlying data item I will be using to populate the drop-down list.

Notice how your new parameters will now be available from your Data Pane.

Step 2: Add the control objects to the report and assign roles

Next, drag and drop the drop-down list control objects onto the report canvas. In this example, we are not using the Report or Section Prompt areas since I do not want to filter the objects in the report or section automatically. Instead, I am using these prompt values and storing them in a parameter. I will then use those values to create new calculated data items and an aggregated measure.

Once your control objects are in the report canvas, then use the Roles Pane to assign the data items to the roles. As you can see from the screenshot, we are using the Date by Month data item to seed the values of the drop-down list by assigning it to the Category role, this data item is in our data source.

Then we are going to assign our newly created parameters, Period1Parameter and Period2Parameter to the Parameter role. This will allow us to use the value selected in our calculations.

Step 3: Create the dependent data items, i.e. the calculated data items and aggregated measure

Now we are free to use our parameters as we like. In this example, I am prompting the report viewer for two values: Period 1 and Period 2 which are the two periods the user would like compared in this report. So, we will need to create two calculated data times from a single column in our source data. Since we want to display these as columns next to each other in a crosstab object and use them for an aggregated measure, this technique can be used.

Calculated Data Item: Period 1 Expenses

From the Data Pane, use the drop-down menu and select New Calculated Item…. Then use the editor to create this expression: If the Date by Month for this data row equals the parameter value selected for Period 1, then return the Expenses; else return 0.

Calculated Data Item: Period 2 Expenses

Repeat this using the Period2Parameter in the expression.

Aggregated Measure: Period Difference

Next, we want to calculate the difference between the two user selected Period Expenses. To do this, we will need to create an aggregated measure which will evaluate based on the report object’s role assignments. In other words, it will be calculated “on-the-fly” based on the visualization.

Similar to the calculated data items, use the Data Pane and from the drop-down menu select New Aggregated Measure…. Use the editor to create this expression. Notice that we are using our newly created calculated data items we defined using the parameter values. This expression does not use the parameter value directly, but indirectly through the calculated data item.

Step 4: Add the remaining report objects to the canvas and assign roles

No that we have:

  • our Control Objects to capture the user input.,
  • the Parameters to store the values.,
  • and the Calculated Data Items and Aggregated Measure created…

we can add our report objects to the canvas and assign our roles.

You can see I used all three new measures in the crosstab object. I used the aggregated measure in the bar chart and treemap but notice the different aggregation levels. There is even a hierarchy assigned to the treemap category role. This Period Difference aggregated measure calculation is done dynamically and will evaluate for each visualization with its unique role assignments, even while navigating up and down the hierarchy.

Here are some additional screenshots of different period selections.

In this first screenshot you can see the parallel period comparison between December 2010 and 2011.

In these next two screenshots, we are looking at the Thanksgiving Black Friday month of November. We are comparing the two years 2010 and 2011 again. Here we see that the Board Product from the Game Product Line is bright blue indicating an increase in magnitude of Expenses in the most recent period, Nov2011.

By double clicking on Board in the treemap, we are taken to the next level of the hierarchy, Product Description, where we see a the largest magnitude of Expenses is coming from Backgammon and Bob Board Games.

In these final two screenshots we are comparing consecutive periods, November 2011 with December 2011. We can see from the bar chart easily the Region and Product Line where there is the greatest increase in Expenses.

I’ve configured a brush interaction between all three visualizations so that when I select the tallest bar it will highlight the corresponding data values in the crosstab and treemap.

Conclusion

Now you can use date parameters in your Visual Analytics Reports. There are several applications of this feature and this is only one way you can use parameters to drive business intelligence. Using this technique to create columns based on a user selected value is great when you need to compare values when your source data isn’t structured in this manner.

Using Date Parameters in your SAS Visual Analytics Reports was published on SAS Users.

11月 022017
 

SAS Visual Investigatorcustom icons and map pin icons in SAS® Visual Investigator has an attractive user interface. One of its most engaging features is the network diagram, which represents related ‘entities’ and the connections between them, allowing an investigator to see and explore relationships in their source data.

For maximum impact, each entity in the network diagram should have an icon – a symbol – which clearly represents the entity. Network diagrams are significantly more readable if the icons used relate to the entities they represent, particularly when a larger numbers of entity types are in the diagram.

SAS Visual Investigator 10.2.1 comes with 56 icons in various colors and 33 white map pins. But if none of these adequately represent the entities you are working with, you can add your own. The catch is that the icons have to be vector images – defined in terms of straight and curved lines – in a file format called SVG, rather than raster images defined in terms of pixels, as GIF, JPEG, PNG files or similar. The vector format greatly improves how good the icons look at different scales. But requiring a vector icon file means you can’t just use any image as a network diagram symbol in Visual Investigator.

So where can you find new icons to better represent your entities? In this post, we’ll see how you can make (or more accurately, find and set the colors of) icons to suit your needs.

For this example, I’m going to suppose that we have an entity called ‘Family,’ representing a family group who are associated with one or more ‘Addresses’ and with one or more individual ‘Persons.’

There are good icons for an ‘Address’ in the default set of 56 provided: I’d choose one of the building icons, probably the house. There are also good icons for a ‘Person’: any of several icons representing people would be fine. The ‘house’ and ‘people’ icons come in a nice variety of colors, and there are white-on-some-background-color versions of each icon for use in Map Pins.

But, there is no icon in the default set which obviously represents the idea of a ‘family.’ So, let’s make one.

We will use the free* library of icons at www.flaticon.com as an example of how to find a source icon image.

*NOTE: While these are free there are some license terms and conditions you must adhere to if you use them. Please read, but the main points to note are that you must give attribution to the designer and site in a specific and short format for the icons you use, and you’re not allowed to redistribute them.

Begin by browsing to https://www.flaticon.com/, and either just browse the icons, or search for a key word which has something to do with the entity for which you need an icon. I’m going to search for ‘Family’:

Notice that many icons have a smaller symbol beneath them, indicating whether they are a ‘premium’ or ‘selection’ icon:

Premium:

Selection:

Premium icons can only be downloaded if you subscribe to the site. Selection icons can be downloaded and used without any subscription so long as you give proper attribution. If you do subscribe, however, you are allowed to use them without attribution. Icons with no symbol beneath them are also free, but always have to be properly attributed, regardless of whether you subscribe or not.

If you are planning on using icons from this site in an image which may be shared with customers, or on a customer project please be sure to comply with company policies and guidelines on using third party content offered under a Creative Commons license. Please do your own research and adhere by their rules.

The icons here are shown at a larger size than they will be in SAS Visual Investigator. For best results, pick an icon with few small details, so that it will still be clear at less than half the size. For consistency with the other icons in your network diagrams and maps, pick a simple black-and-white icon. Several icons in this screenshot are suitable, but I will choose this one named ‘Family silhouette,’ which I think will be clear and recognizable when shrunk:

Icons made by Freepik from www.flaticon.com is licensed by CC 3.0 BY

When you hover over the icon, you’ll see a graphic overlaid on top of it, like this:

If you are working with a number of icons at once, it can be convenient to click the top half of that graphic, to add the icon to a collection. When you have assembled the icons you require, you can then edit their colors together, and download them together. Since I am just working with one icon, I clicked the bottom half of the overlay graphic, to simply view the icon’s details. This is what you see next:

Click the green SVG button to choose a color for your icon – a box with preset colors swings down below the buttons. This offers 7 default colors, which are all okay (white is perfect for map pin icons). But you may prefer to match the colors of existing icons in SAS Visual Investigator – to set your own color, click the multicolor button (highlighted in a red box below):

These are the hex color codes for icons and map pin backgrounds in SAS Visual Investigator 10.2.1. Enter one of these codes in the website’s color picker to match the new icon’s color to one already used in VI:

I chose the green in the center of this table, with hex color code #4d9954:

Then, click Download, and in the popup, copy the link below the button which will allow you to credit the author, then click ‘Free download’ (you must credit the author):

Your SVG icon file is downloaded in your browser. Paste the attribution text/HTML which credits the icon’s author in a text editor.

You must ensure that the files are named so that they contain only the characters A-Z, a-z, 0-9 and ‘_’ (underscore). SAS Visual Investigator doesn’t like SVG filenames containing other characters. In this example, we MUST rename the files to replace the ‘-‘ (dash or hyphen) with something else, e.g. an underscore.

You may also want to rename the SVG file to reflect the color you chose. So, making both these changes, I renamed my file from ‘family-silhouette.svg’ to ‘family_silhouette_green.svg’.

Adding the color to the file name is a good idea when you create multiple copies of the same icon, in different colors. You should also consider creating a white version of each icon for use as a map pin. Doing that, I saved another copy of the same icon in white, and renamed the downloaded file to ‘family_silhouette_white.svg’.

Then, if necessary, copy your SVG file(s) from your local PC to a machine from which you can access your copy of SAS Visual Investigator. To import the icons, open the SAS Visual Investigator: Administration app, and log in as a VI administrator.

When creating or editing an Entity, switch to the Views tab, and click on the Manage icons and map pins button:

In the Manage Icons and Map Pins popup, click Upload…:

Select all of the SVG files you want to upload:

The new icons are uploaded, and if you scroll through the list of icons and map pins you should be able to find them. Change the Type for any white icons you uploaded to Map Pin, and click OK to save your changes to the set of icons and map pins:

You can now use your new icons and map pins for an Entity:

Don’t forget to include a reference to the source for the icon in any materials you produce which accompany the image, if you share it with anyone else. Here’s mine:

Icons made by Freepik from http://www.freepik.com/ is licensed by CC 3.0 BY

See you next time!

Creating and uploading custom icons and map pin icons in SAS® Visual Investigator was published on SAS Users.

11月 022017
 

Managing SAS Configuration Directory SecurityNeed to grant one or more users access to part of your secure SAS configuration directory? You can do it without opening up your SAS configuration directory to everyone.

Most SAS 9.4 Platform deployments on Unix have been done using the SAS Installer account known as sas. The sas account is the owner of the SAS configuration directory. Along with the sas account comes a sas group that out of the box is given generous access to the SAS configuration.

SAS Configuration Directory

The SAS configuration not only includes scripts like sas.servers but it also includes configuration files and logs. It generally includes a lot of control over the SAS environment. Despite locked down security of the SAS configuration on Unix out of the box, there are still valid situations when you need to grant one or more users access to part of the SAS configuration. For example, you might need to enable logging for the workspace server and need to grant write access to the workspace server Logs directory. Or maybe you’re setting up an account to be used for autoloading data into the Public LASR Server. There are many more such examples where you might need to grant one or more users access to part of the SAS configuration.

The sas Group

How do you grant someone access to part of the SAS configuration directory? Why not add the user in question to the sas group? While this may grant your user the access you want, it also introduces the potential for a lot of problems. Keep in mind that the sas group has (and needs) broad access to the SAS configuration. When adding a user to the sas group you are granting that user the same access as the sas group. If the user is administering your SAS environment, that might be okay. If that user is not going to administer your SAS environment, you’ve opened the door for someone to modify or delete any and all of your SAS configuration.

So, what should you do? The short answer is that you should only grant the access needed to the users who need it.

Modifying Security for Workspace Server Logging

Let’s look at the example of enabling the workspace server to produce logs. This is typically done if you need to collect logs for troubleshooting. By default, the workspace server runs under each individual user’s credentials; therefore, each workspace server user would need to be given access to create logs under the workspace server Logs directory. By default, the sas user and sas group are given read, write and execute permission on the workspace server Logs directory. All other users have no access to the workspace server Logs directory. This is a situation where granting all other users read, write and execute access while you need to generate workspace server logs is the recommendation.

Be aware that if logging is enabled for the workspace server, any user who does not have read, write and execute access to the workspace server’s Logs directory will not be able to launch a workspace server session.

The complete steps for enabling workspace server logging can be found in the SAS 9.4 Intelligence Platform: System Administration Guide.

Modifying Security for Autoload

In SAS Visual Analytics, the autoload feature allows periodic synchronization between files on disk and data in memory in a LASR server. The autoload feature for the Public LASR Server is mostly configured out of the box; however, there are a few steps required in order for autoload to be fully enabled. The final step is to schedule the job that will perform the autoload.

By default, the autoload directory for the Public LASR Server is in the SAS configuration directory. It is owned by the sas user and the sas group. The first step in the documentation for how to start autoload is to identify which account will be used to schedule the autoload script. The account you use needs access to both metadata and to the autoload configuration directory. The sas account has access to the autoload configuration directory by default but is not registered in metadata. The ideal answer is to find a balance between overloading an account like sas and not overcomplicating your environment. You could register the sas account in metadata but that would not be my preference. You could also provide metadata server connection information in the autoload script but storing a user id and password in a file is less than ideal. A better solution is the one presented in the Visual Analytics 7.3 – Recommended Post-Install Activities: create an account for the purpose of running autoload, for example, lasradm. The account needs to exist on the operating system (or other authentication provider being used) and in metadata. You would change the ownership of the autoload directory to the lasradm account and to a group other than sas. Creating an operating system group for all of your SAS users is a convenient way to grant permissions or rights to the group of users who will be using SAS. You can create a sasusers group, add lasradm as a member, and make sasusers the group owner of the autoload directory. Now you can schedule the autoload script as lasradm.

SAS Admin Notebook: Managing SAS Configuration Directory Security was published on SAS Users.

10月 192017
 

The goal of this article is to describe the steps needed to configure the bridge that allows SAS Data Integration 4.902, based on SAS 9.4M4, to load data directly into CAS on SAS Viya 3.2.

Of course, SAS 9.4M5 simplifies this process, as a SAS/CONNECT communication will no longer be required, enabled by a direct function within SAS Data Integration to CAS - but for those of you who may not move immediately from SAS 9.4M4 to SAS 9.4M5, this could be helpful.

It is assumed here that SAS/CONNECT has been installed and configured on both environments, SAS 9.4M4 and SAS Viya 3.2.

Validate the connection from SAS 9.4M4 to SAS Viya 3.2

⇒     Check the status of the SAS/CONNECT Spawner on SAS Viya, on the machine where this service is installed.

SAS Viya 3.2 Bridge for SAS Data Integration Studio

⇒     Note the machine and the listening port of the SAS/CONNECT Spawner on SAS Viya.
⇒     Open SAS Studio on SAS 9.4M4 and sign-in.
⇒     Run the following SAS code with your machine details and a valid SAS Viya user account and check the results.

SAS Viya 3.2 Bridge for SAS Data Integration Studio

⇒     If successful, sign-off the SAS/CONNECT session and sign-out from SAS Studio SAS 9.4M4

Setup SAS9.4M4 metadata

⇒     Open SAS Management Console 9.4M4 as sasadm@saspw.
⇒     On the “Server Manager” plugin, add a “New Server…”

  • Server type: “SAS Application Server”
  • Name: “SASViya”
  • Select “Connect Server” as the sub-type of server you want to add to this SAS Application Server
  • Configure the “Connect Server” as shown below (you might have to create a new authentication domain for SAS Viya) and set the values accordingly (server where the SAS/CONNECT Spawner on SAS Viya is listening)

⇒     On the “Server Manager” plugin, add a “New Server…”

  • Server type: “SAS Cloud Analytic Services Server”
  • Name: “CAS Server”
  • Configure the “CAS Server” as shown below and set the values accordingly (CAS controller)

⇒     On the “User Manager” plugin, set a login for the SASViya application server, on a user or group that you will use in SAS Data Integration Studio

⇒     On the “Data Library Manager” plugin, add a “New Library…”

  • Library type: “SAS Cloud Analytic Services Library”
  • Name: CAS_DATA
  • Assign the library to the SASViya server

⇒     Configure the CAS library as shown below and set the values accordingly (the CASLIB must exist in the SAS Viya environment; here CASPATH is the name of an existing CASLIB).

⇒     Specify the server and connection information as shown below:

Build a SAS Data Integration Studio job to load data into CAS

⇒     Open SAS Data Integration Studio 4.902 as a user who holds, directly or not, a login for the ViyaAuth authentication domain.
⇒     Test the CAS_DATA library by “Register(ing) tables…”

  • In SAS Environment Manager on SAS Viya, some tables must have been loaded before into the CASLIB (the one that is pointed on by the CAS library, here CASPATH), so that you can display some tables in the “Register Tables…” wizard.
  • If you see some tables then it looks like you are all set.

⇒     If you want to go further and test the “Cloud Analytic Services Transfer” DI transformation, create the metadata for a target table in CAS with appropriate columns.

⇒     Build a job that loads a source table to this target table using the “Cloud Analytic Services Transfer” (“Access” folder) as shown below:

⇒     The “Cloud Analytic Services Transfer” is basically a “Data Transfer” transformation that fits with CAS; it enables you to provide some CAS options such as COPIES; by default the table is PROMOTED.

⇒     Run it and check if it ran successfully and if the table is available from the SAS Viya world.

SAS/CONNECT on SAS Viya configuration directory

⇒     The SAS/CONNECT on SAS Viya configuration is located by default here /opt/sas/viya/config/etc/connectserver/default

⇒     You might want to customize/clean some of the files here.

  • Check the CASHOST option in autoexec files, sometimes the value is not appropriate.

Normally, options here are sourced from the CONNECTSERVER_CONFIGURATION in vars.yml during the deployment of SAS Viya.

SAS Viya 3.2 Bridge for SAS Data Integration Studio 4.902 (on SAS 9.4M4) configuration steps was published on SAS Users.

10月 172017
 

moving content between SAS Viya environmentsIn a SAS Viya 3.2 environment two types of content can be created: SAS Visual Analytics Reports and Data Plans. For administrators, who may want to manage that content within a folder structure, there are some things to keep in mind. In the current release, both types of content can be moved around in folders, but the objects cannot be copied. In addition, SAS Viya 3.2 supports the promotion of SAS Visual Analytics Reports, but doesn’t support the promotion of Data Plans (support for Plans is coming in SAS Viya 3.3). So, what if I want to copy a report between, say my personal folders, to a production folder?

If you want copy a Report or Data Plan within an environment there is an easy way. When the object is open in edit mode you can do a Save As to save a copy to a different location in the folder structure.

Between environments, Reports can be exported and imported using the SAS Visual Analytics, when you are editing your content (Report or Data Plan) you can access a “diagnostics” window. The diagnostics window will show you the json (or xml) used to render the Report or Plan. To enter the diagnostics window use the keystrokes:

  • ctl+alt+d for SAS Visual Data Builder.
  • ctl+alt+b for SAS Visual Analytics.

In the steps below I will use the diagnostics window to save a Data Plan so that it can be loaded to a different SAS Viya Environment. The steps for a SAS Visual Analytics report are very similar.

In SAS Visual Data Builder when editing your Data Plan select ctl-alt-d to open the SAS Visual Data Builder Diagnostics window. The source tab of the window shows the json that will render the data plan.

Click Save to save the json to a text file and close the dialog. The json file will be saved in the browsers default downloads folder.

Copy the saved text file to a location accessible to the SAS Viya environment where you want to import the plan. In that environment, open Data Builder and click New to open a new Data Plan.

Click ctl-alt-d on the empty data plan and cut and paste the json from your text file replacing the json in the diagnostics window.

Click Parse to check the json.A message should be displayed indicating that the  “plan text was parsed successfully.”  Once you have parsed the text, click Run and the plan is loaded into SAS Visual Data Builder.

In SAS Visual Data Builder, select Save As and save the plan to any location in the folder structure.

The assumption with this approach is that the data is available in the same location in both environments.

You can do much the same with SAS Visual Analytics reports. The key-stroke is ctl-alt-b to open the SAS Visual Analytics Diagnostics window.  You can see the report xml or json on the BIRD tab.

To copy a single report between environments, you can select json and then save the json to a file. In the target environment open a new report, paste the json in the BIRD tab, parse and load and then save the report to a folder. This can be a useful approach if you want to relocate a report to a different location in your target environment. The transfer service currently will only import reports to the same folder location in the target that they are located in the source environment.

I hope you found this tip useful.

A tip for moving content between SAS Viya environments was published on SAS Users.

9月 252017
 

With the fifth maintenance release of SAS 9.4 (SAS 9.4M5) now available, it seems like a good time to get a refresher on some of the ways you can upgrade your existing SAS deployments to the latest release. Among several benefits, SAS 9.4M5 provides closer integration with the CAS in-memory runtime engine in SAS Viya so it’s something you might want to consider.

There are three ways you can upgrade a SAS Deployment, they are illustrated in the diagram below. Let’s take a look at each one:

Upgrading a SAS 9 deployment

Automated Migration

Automated migration consists of creating a new SAS deployment from an existing SAS Deployment using the automated migration tools the SAS Migration Utility and SAS Deployment Wizard. The automated migration tools and process are designed to create a target deployment that preserves the content and configuration information from the source. These tools require an all-at-once approach, and provide limited options for making changes to the deployment during the upgrade. The automated migration tools support a like-for-like transition—the operating system family and the distribution of SAS components must be the same in the source and target environments.

Deploy new and Promote

Promotion is a related concept to migration; promotion is the process of copying selected metadata and associated content within or between SAS Deployments. To upgrade using promotion, create a new out-of-the-box SAS deployment and use the export/import functionality to move the content from the old deployment. The promotion framework is designed to allow you to selectively move content from one deployment to another. Depending on the release of your source deployment there are limits to the content that can be exported using promotion; however, this option offers the most flexibility in changing the deployed topology, operating system and transitioning in stages.

Update-in-Place

Update-in-Place is the process of upgrading an existing SAS Deployment to apply maintenance or add and update SAS products. The update modifies the existing deployment rather than creating a new deployment. Update-in-place is only supported within a major SAS release. For example, you can use update-in-place to update a SAS 9.4 deployment to maintenance 4 or maintenance 5, however you cannot use update-in-place to transition from SAS 9.3 to SAS 9.4.

Considerations when Deciding on an Approach

Automated Migration

  • Automated migration moves a whole system, configuration and content, you cannot move only part of a system.
  • The tools support migration from (n-2), so the earliest release you can migrate from is currently SAS 9.2.
  • Individual SAS Products and solutions may have additional baselines for migration.
  • During a migration you cannot change your:
    • Operating system family.
    • Software Topology (the distribution of the SAS components in the environment).

Deploy new and Promote

  • Deploy New and promote creates a new deployment and moves content into it.
  • The target system can be a different topology and operating system from the source.
  • The tools support promotion from (n-2), so the earliest release you can promote from is currently SAS 9.2.
  • Content can be transitioned in stages.
  • Not all content is supported by the import and export wizards and batch tools (e.g., dashboard content cannot be promoted between releases and portal content requires execution of a command line tool).

Update-in-Place

  • Update-in-Place can only be done within the same major SAS release.
  • Updates operate on a whole deployment, you cannot selectively update products.
  • Maintenance updates do not preserve the source environment because maintenance releases:
    • are applied to the existing environment.
    • are cumulative (including hotfixes).
    • cannot be uninstalled, e.g. you cannot uninstall M2 to get back to M1.
  • Adding products to a Deployment will require a second pass through the SAS Deployment Wizard (update and then add).

Summary

I would encourage you to consider your target SAS 9.4 deployment as a new house.

Using automated migration you would move the house. Take the whole house off its current foundation. Move the house, contents and all, to a new foundation.

  • The Layout of the house remains the same.
  • Everything in the house moves at one time.
  • Anything outside the house does not move.

Using deploy new and promote you can selectively move the content of your house to a whole new house. You can leave your junk behind. Your new house:

  • Must be built before you move in.
  • Can be constructed any way you want.

No matter how hard you try you cannot move all content.

With update-in-place you have a renovation project of your existing house. However, if your house is old, you cannot or may not want to renovate it. Plus, if you mess up the renovation then your existing house is broken.

You can find the documentation on upgrading in a variety of places.

A review of the options available for upgrading a SAS 9 deployment was published on SAS Users.

9月 082017
 

In SAS Visual Analytics 8.1, report creators have the ability to include drive-distance and drive-time in their geographical maps, but only if their site has an Esri ArcGIS Online account and they have valid credentials for the account.

In the user Settings for SAS Visual Analytics Geographic Mapping 8.1 release, there are three choices for selection of a geographic map provider.  The map provider creates the background map for geo maps and for network diagrams that display a map.

The map provider options are:

  • OpenStreet Map service, hosted at SAS.
  • Esri ArGIS Online Services, which only requirFinale acceptance of the terms and conditions.
  • Esri premium services, which requires a credential validation.

If Esri premium services is selected, there is an additional prompt for valid credentials, and you must still accept the Esri ArcGIS Online Services terms in order to select the premium services checkbox.

It’s also worth pointing out here, that even if you have Esri premium credentials, in order for these credentials to be validated in SAS Visual Analytics, you must also be a member of the ESRI Users custom group.  Users can be added to this group in SAS Environment Manager, as shown below.

Note that without the Esri ‘premium’ service and validated credentials, when you right-click and Create geographic selection in your report map, you are only able to select the Distance selection, which displays the radial distance for the selection point.

With premium services in effect, you can also select drive-time or drive-distance.  An example of a drive-time selection is shown here.  Drive-time creates an irregular selection based on the distance that can be driven in the specified amount of time.

A drive-distance example is shown below.  Drive-distance creates an irregular selection based on the driving distance using roads.

When selecting drive-time or drive distance, you can also add breaks to show, as in the example below, the 5-mile distance, the 10-mile distance, and the 15-mile distance on the maps.

It’s also worth pointing out, that if a viewer of the report has not had Esri premium credentials validated, the viewer will be unable to view the drive-distances and drive-time features.  The settings for users of the report viewer are also stored in the Report Viewer Geographic Mapping user settings.

If a user is adding a connection to the server in SAS Mobile BI 8.15 and their account is a member of the Esri Users group, they will be prompted for their Esri premium credentials when adding the server connection:

I hope you’ve found this post helpful.

How do I access the Premium Esri Map Service for my SAS Visual Analytics reports? was published on SAS Users.

8月 182017
 

SAS Viya deployments use credentials for accessing databases and other third-party products that require authentication. In this blog post, I will look at how this sharing of credentials is implemented in SAS Environment Manager.

In SAS Viya, domains are used to store the:

  • Credentials required to access external data sources.
  • Identities that are allowed to use those credentials.

There are three types of domains:

  • Authentication stores credentials that are used to access an external source that can then be associated with a caslib.
  • Connection used when the external database has been set up to require a User ID but no password.
  • Encryption stores an encryption key required to read data at rest in a path assigned to a caslib.

In this blog post we will focus on authentication domains which are typically used to provide access to data in a database management system. It is a pretty simple concept; an authentication domain makes a set of credentials available to a set of users. This allows SAS Viya to seamlessly access a resource. The diagram below shows a logical view of a domain. In this example, the domain PGAuth stores the credentials for a Postgres database, and makes those credentials available to two groups (and their members) and three users.

How does this work when a user accesses data in a database caslib? The following steps are performed:

1.     Log on to SAS Viya using personal credentials: the user’s identity is established including group memberships.

2.     Access a CASLIB for a database: using the user’s identity and the authentication domain of the CASLIB, Viya will look up the credentials associated with that identity in the domain.

3.     Two results are possible. A credential match is:

  • 1.     Found: the credentials are passed to the database authentication provider to determine access to the data.
  • 2.     Not found: no access to the data is provided.

To manage domains in SAS Environment Manager you must be an administrator. In SAS Environment Manager select Security > Domains. There are two views available:  Domains and Credentials. The Domains view lists all defined domains. You can access the credentials for a domain by right-clicking on the domain and selecting Credentials.

The Credentials view lists all credentials defined and the domains for which they are associated.

Whatever way you get to a credential, you can edit it by right-clicking and selecting Edit. In the edit dialog, you can specify the Identities (users and groups) that can use the credential, and the User ID and Password of the credential.  Note that only users who are already listed in the Identities field will be able to edit this field, so make sure you are in this field (directly or through group membership) prior to saving.

To use an authentication domain, you reference it in the CASLIB definition. When defining a non-path based CASLIB you must select a domain to provide user credentials to connect to the database server. This can be done when creating a new CASLIB in SAS Environment Manager in the Data > Libraries area.

If you use code to create or access your caslib, use the authenticationdomain option. In this example, we specify authenticationdomain in the table.addcaslib action.

If a user is not attached to the authentication domain directly, or through a group membership, they will not be able to access the credentials. An error will occur when they attempt to access the data.

This has been a brief look at storing and using credentials to access databases from SAS Viya. You can find  more detail in the SAS Viya Administration Guide in the section titled SAS Viya sharing credentials for database access was published on SAS Users.