sas administration

11月 292017

SAS Viya is an exciting addition to the SAS Platform, allowing you to conduct analysis faster than ever before and providing you the flexibility to utilize open source technologies and generate insights from data in any computing environment. The blog post “Top 12 Advantages of SAS Viya” does a great [...]

The post Learn more about SAS Viya with resources from SAS Education appeared first on SAS Learning Post.

2月 212016

Last week I described how to use PROC IOMOPERATE to list the active SAS sessions that have been spawned in your SAS environment. I promised that I would share a custom task that simplifies the technique. Today I'm sharing that task with you.

How to get the SAS Spawned Processes task

You can download the task from this SAS communities topic, where I included it as an attachment. The instructions for installation are standard for any custom task; the details are included in the README file that is part of the task package.

You can also view and pull the source code for the task from my GitHub repository. I built it using Microsoft .NET and C#.

How to use the SAS Spawned Processes task

Once you have the task installed, you can access it from the Tools->Add-In menu in SAS Enterprise Guide. (By the way, the task should also work in the SAS Add-In for Microsoft Office -- though the installation instructions are a little different.)

The task works by using PROC IOMOPERATE to connect to the SAS Object Spawner. You'll need to provide the connection information (host and port) plus the user/password for an account that has the appropriate permissions (usually a SAS admin account). Note that the port value is that of the Object Spawner operator port (by default, 8581) and not the SAS Metadata Server.

The task shows a list of active SAS processes. Of course, you're using a SAS process to even run the task, so your active process is shown with a yellow highlight. You can select any of the processes in the list and select End Process to stop it. You can drill into more detail for any selected process with the Show Details button. Here's an example of more process details:

Did you try the task? How did it work for you? Let me know here or in the SAS communities.

Custom task features within this example

If you're professionally interested in how to build custom tasks, this example shows several techniques that implement common requirements. Use the source code as a reference to review how these are built (and of course you can always refer to my custom tasks book for more guidance).

  • Submit a SAS program in the background with the SasSubmitter class. There are two examples of this in the task. The first example is an asynchronous submit to get the list of processes, where control returns to the UI and you have the option to cancel if it takes too long. With an asynch submit, there are some slightly tricky threading maneuvers you need to complete to show the results in the task. The second example uses a synchronous submit (SubmitSasProgramAndWait) to stop a selected SAS process.
  • Read a SAS data set. The SAS program that retrieves a list of processes places that result in a SAS data set. This task uses the SAS OLE DB provider to open the data set and read the fields in each row, so it can populate the list view within the task.
  • Detect errors and show the SAS log. If the SAS programs used by the task generate any errors (for example, if you supply the wrong credentials), the task uses a simple control (SAS.Tasks.Toolkit.Controls.SASLogViewDialog) to show the SAS log -- color-coded so the error is easy to spot.
  • Retrieve the value of a SAS macro variable by using SasServer.GetSasMacroValue("SYSJOBID"). This pulls the process ID for your active SAS session, so I can compare it to those retrieved by PROC IOMOPERATE. That's how I know which list item to highlight in yellow.
  • Save and restore settings between uses. Entering credentials is a drag, so the task uses a helper class (SAS.Tasks.Toolkit.Helpers.TaskUserSettings) to save your host/port/user information to a local file in your Windows profile. When you use the task again, the saved values are placed into the fields for you. I don't save the password -- I'm sure that I'd get complaints if I did that, even if I encoded it.
tags: IOMOPERATE, sas administration, SAS custom tasks, SAS Enterprise Guide

The post A custom task to list and stop your SAS sessions appeared first on The SAS Dummy.

2月 162016

If you're a SAS administrator, you probably know that you can use SAS Management Console to view active SAS processes. These are the SAS sessions that have been spawned by clients such as SAS Enterprise Guide or SAS Add-In for Microsoft Office, or those running SAS stored processes. But did you know that you can generate a list of these processes with SAS code? It's possible with the IOMOPERATE procedure.

To use PROC IOMOPERATE, you need to know the connection information for the SAS Object Spawner: the host, port, and credentials that are valid for connecting to the Object Spawner operator port. You plug this information into a URI scheme like the following:


Here's an example:


Are you squeamish about clear-text passwords? Good for you! You can also use PROC PWENCODE to obscure the password and replace its value in the URI, like this:


Getting useful information from PROC IOMOPERATE is an iterative process. First, you use the LIST SPAWNED command to show all of the spawned SAS processes:

%let connection=
/* Get a list of processes */
proc iomoperate uri=&connection.;
    list spawned out=spawned;

Example output:
You can retrieve more details about each process by running subsequent IOMOPERATE steps with the LIST ATTRS command. This can get tedious if you have a long list of spawned sessions. I've wrapped the whole shebang into a SAS program that discovers the processes and iterates through the list for you.

%let connection=
/* Get a list of processes */
proc iomoperate uri=&connection.;
    list spawned out=spawned;
/* Use DOSUBL to submit a PROC IOMOPERATE step for   */
/* each SAS process to get details                   */
/* Then use PROC TRANSPOSE to get a row-wise version */
data _null_;
    set spawned;
    /* number each output data set */
    /* for easier appending later  */
    /* TPIDS001, TPIDS002, etc.    */
    length y $ 3;
    y = put(_n_,z3.);
    x = dosubl("
    proc iomoperate uri=&connection. launched='" || serverid || "';
    list attrs cat='Information' out=pids" || y || ";
    data pids" || y || ";
    set pids" || y || ";
    length sname $30;
    sname = substr(name,find(name,'.')+1);
    proc transpose data=work.pids" || y || "
    out=work.tpids" || y || "
    id sname;
    var value;
/* Append all transposed details together */
data allpids;
    set tpids:;
    /* calculate a legit datetime value */
    length StartTime 8;
    format StartTime datetime20.;
    starttime = input(UpTime,anydtdtm19.);
/* Clean up */
proc datasets lib=work nolist;
delete tpids:;
delete spawned;

The output details include "up time" (when the process was launched), the process ID (a.k.a. PID), the owner account, the SAS version, and more. Here's a snippet of some example output:

You can use this information to stop a process, if you want. That's right: from a SAS program, you can end any (or all) of the spawned SAS processes within your SAS environment. That's a handy addition to the SAS administrator toolbox, though it should be used carefully! If you stop a process that's in active use, an unsuspecting SAS Enterprise Guide user might lose work. And he won't thank you for that!

To end (kill) a SAS process, you need to reference it by its unique identifier. In this case, that's not the PID -- it's the UUID that the LIST DETAILS command provided. Here's an example of the STOP command:

/* To STOP a process */
    proc iomoperate uri=&connection.;                                  
        STOP spawned server 
/* ID = value is the UniqueIdentifier (UUID)      */
/*  Not the process ID (PID)                      */

It seemed to me that this entire process could be made easier with a SAS Enterprise Guide custom task, so I've built one! I'll share the details of that within my next blog post.

tags: IOMOPERATE, sas administration

The post Using PROC IOMOPERATE to list and stop your SAS sessions appeared first on The SAS Dummy.

7月 032013

When I work on SAS projects that create lots of files as results, it's often a requirement that those files be organized in a certain folder structure. The exact structure depends on the project, but here's an example:

   |__ html
       |__ images
   |__ xls
   |__ data

Before you can have SAS populate these file folders, the folders have to actually exist. Traditionally, SAS programmers have handled this by doing one of the following:

  • Simply require that the folders exist before you run through the project. (This is the SEP method: Somebody Else's Problem.)
  • Use SAS statements and shell commands (via SYSTASK or other method) to create the folders as needed. The SAS-related archives are full of examples of this. It can get complex when you have to account for operating system differences, and whether operating system commands are even permitted (NOXCMD system option).

In SAS 9.3 there is a new system option that simplifies this: DLCREATEDIR. When this option is in effect, a LIBNAME statement that points to a non-existent folder will take matters into its own hands and create that folder.

Here's a simple example, along with the log messages:

options dlcreatedir;
libname newdir "/u/sascrh/brand_new_folder";

NOTE: Library NEWDIR was created.
NOTE: Libref NEWDIR was successfully assigned as follows: 
      Engine:        V9 
      Physical Name: /u/sascrh/brand_new_folder

You might be thinking, "Hey, SAS libraries are for data, not for other junk like ODS results." Listen: we've just tricked the LIBNAME statement into making a folder for you -- you can use it for whatever you want. I won't tell.

In order to create a series of nested folders, you'll have to create each folder level in top-down order. For example, if you need a "results" and a "results/images" folder, you can do this:

%let outdir=%sysfunc(getoption(work));
/* create a results folder in the WORK area, with images subfolder */
options dlcreatedir;
libname res "&outdir./results";
libname img "&outdir./results/images";
/* clear the librefs - don't need them */
libname res clear;
libname img clear;

Or (and this is a neat trick) you can use a single concatenated LIBNAME statement to do the job:

libname res ("&outdir./results", "&outdir./results/images");
libname res clear;

NOTE: Libref RES was successfully assigned as follows: 
      Levels:           2
      Engine(1):        V9 
      Physical Name(1): /saswork/SAS_workC1960000554D_gsf0/results
      Engine(2):        V9 
      Physical Name(2): /saswork/SAS_workC1960000554D_gsf0/results/images

If you feel that folder creation is best left to the card-carrying professionals, don't worry! It is possible for a SAS admin to restrict use of the DLCREATEDIR option. That means that an admin can set the option (perhaps to NODLCREATEDIR to prohibit willy-nilly folder-making) and prevent end users from changing it. Just let them try, and they'll see:

13         options dlcreatedir;
WARNING 36-12: SAS option DLCREATEDIR is restricted by your Site 
Administrator and cannot be updated.

That's right -- DENIED! Mordac the Preventer would be proud. Job security: achieved!

Exact documentation for how to establish Restricted Options can be a challenge to find. You'll find it within the Configuration Guides for each platform in the Install Center. Here are quick links for SAS 9.3: Windows x64, Windows x86, and UNIX.

tags: DLCREATEDIR, Restricted Options, sas administration, SAS libraries, SAS programming, SAS tips
2月 232013

SAS Integration Technologies provides a flexible platform to create all types of apps, from simple utilities to full-blown applications. As part of the research for my SAS Global Forum 2013 paper (Create Your Own Client Apps Using SAS Integration Technologies), I've been trying to invent some useful examples that you can run from your Windows desktop. In this post I'll cover how you can use Windows PowerShell plus the SAS Integration Technologies client to connect to a SAS Metadata Server.

Creating objects with SAS Object Manager

When working with the SAS Integration Technologies client, you need a way to create the objects that represent the connections to the SAS services. For that, you must use the SAS Object Manager.

The SAS Object Manager includes a class named ObjectFactory. As the name implies, the "object factory" class is where your subsequent objects will be created. In our examples, we will use the ObjectFactoryMulti2 class to create the connection to the SAS server for use in our applications. After creating that connection, you can use methods on the connection object to get to the other services we need.

To get started with the SAS Object Manager in Windows PowerShell, use the New-Object -ComObject command.

$objFactory = New-Object -ComObject SASObjectManager.ObjectFactoryMulti2

Before you can connect to a SAS server, you must define its attributes to SAS Object Manager. A SAS server has several attributes: a host name, TCP port number, and a Class Identifier. The Class Identifier is a 32-character GUID (unique ID) that indicates the type of SAS server that you expect to connect to.

How to Find the Correct Class Identifier
If you search you may be able to find a "lookup" table for the Class Identifier values to map to the specific types of SAS servers. However, the most reliable source for these values, and usually the easiest to access, can come from SAS itself by way of PROC IOMOPERATE. Here is an example program:

proc iomoperate;
   list types;

Here's an excerpt of the SAS log, which contains the values:

SAS Metadata Server 
    Short type name  : Metadata 
    Class identifier : 0217e202-b560-11db-ad91-001083ff6836

SAS Stored Process Server 
    Short type name  : StoredProcess 
    Class identifier : 15931e31-667f-11d5-8804-00c04f35ac8c

SAS Workspace Server 
    Short type name  : Workspace 
    Class identifier : 440196d4-90f0-11d0-9f41-00a024bb830c

You use the SAS Object Manager to create a ServerDef object, and then use the CreateObjectByServer method to establish the connection. You name the SAS Metadata Server port (which is 8561 in a default installation) and the SAS Metadata Server value for the ClassIdentifier. Here is an example:
$objFactory   = New-Object -ComObject SASObjectManager.ObjectFactoryMulti2
$objServerDef = New-Object -ComObject SASObjectManager.ServerDef
# assign the attributes of your metadata server
$objServerDef.MachineDNSName  = ""
$objServerDef.Port            = 8561  # metadata server port
$objServerDef.Protocol        = 2     # 2 = IOM protocol
# Class Identifier for SAS Metadata Server
$objServerDef.ClassIdentifier = "0217E202-B560-11DB-AD91-001083FF6836"
# connect to the server
# we'll get back an OMI handle (Open Metadata Interface)
$objOMI = $objFactory.CreateObjectByServer(
        "sasdemo",  # metadata user ID
        "Password1" # password
Write-Host "Connected to " $objServerDef.MachineDNSName 
catch [system.exception]
  Write-Host "Could not connect to SAS metadata server: " $_.Exception
  exit -1

CreateObjectByServer returns a connection to the SAS Metadata Server, sometimes called OMI (which stands for "Open Metadata Interface" and is easily confused with IOM). In the above example program, the connection is in the PowerShell variable named $objOMI.

Most metadata operations require that you know the metadata ID for the repository in which your metadata resides. For most applications that's the Foundation repository. Even though it has a standard name ("Foundation"), the ID value can differ in each installation. So the next step is to use the GetRepositories method to find the ID value for the Foundation repository.

The GetRepositories method returns information within an XML-formatted structure, which you must then parse to get the information you need. Here is an example result from the GetRepositories method:

 <Repository Id="A0000001.A5B4FV3C" 
    Name="Foundation" Desc="" DefaultNS="SAS"/>
 <Repository Id="A0000001.A5Q23NT1" 
    Name="BILineage" Desc="BILineage" DefaultNS="SAS"/>

The nugget of information that you need from this example is A0000001.A5B4FV3C, which is the ID for the Foundation repository in this installation. Fortunately, Windows PowerShell provides an XML data type that makes it easy to filter and parse. The following code segment does the job:
# get list of repositories
$reps="" # this is an "out" param we need to define
$rc = $objOMI.GetRepositories([ref]$reps,0,"")

# parse the results as XML
[xml]$result = $reps

# filter down to "Foundation" repository
$foundationNode = $result.Repositories.Repository | ? {$_.Name -match "Foundation"} 
$foundationId = $foundationNode.Id

Write-Host  "Foundation ID is $foundationId"  

There! The connection is made, and you know the Foundation ID. The next step is to use the metadata API methods (such as GetMetadataObjects) to retrieve useful information from the SAS Metadata Server.

I'll describe that in my next post, but if you want a sneek peak you can see the complete code for the example here (hosted on GitHub).

Related links

Using Windows PowerShell to find registered tables and columns in SAS metadata
Running Windows PowerShell scripts
Build your own SAS data set viewer using Windows PowerShell
Example of using Windows PowerShell to query SAS table metadata from SAS Metadata Server (GitHub)

tags: PowerShell, sas administration, SAS Integration Technologies, sasgf13
1月 082013

I've bragged about how easy it is to install custom tasks for SAS Enterprise Guide. It's simple: you copy the .NET assembly (usually a single DLL file) into one of the designated folders in your SAS Enterprise Guide installation, and your task just shows up in the menu. Since I have made several custom tasks available for download from this blog, it's possible for users to spend just a few minutes to acquire capabilities that they didn't have before. (By the way, this also applies to the SAS Add-In for Microsoft Office.)

This can trigger anxiety for some SAS administrators. A custom task, despite it's usefulness, might work against the policies of those who maintain a SAS environment. For example, the Copy Files task makes it very easy to move files between your SAS server and your local PC. Some organizations don't want server-based files to leave their controlled server environment, so this type of custom task is a thorn in their side.

Fortunately, SAS administrators can control access to these capabilities by defining roles in SAS metadata, and then associating those roles with users or groups of users. Each role has a list of capabilities that are permitted. If a user's role doesn't permit a certain capability, then that feature simply doesn't appear in the SAS Enterprise Guide user interface.

Custom tasks are a special case for administration. A custom task is a new capability, and so by definition it's not known to SAS metadata. If it's not known, how can you control it? You accomplish it by following these general steps:

  1. First, examine the capability named Access Unregistered Custom Tasks. Do you want your users to be able to add their own tasks? Even if you allow this, you can still "disallow" certain custom tasks if you register them in metadata.
  2. Next, register the custom tasks that you want to control into SAS metadata. SAS Enterprise Guide provides a tool to do this. (It's not in SAS Management Console.)
  3. With the task now listed in SAS metadata, use SAS Management Console to control which roles (and therefore, which users) have the permission to run the task.

    In the remainder of this post, we'll explore these pictures.

    In SAS Enterprise Guide, select Tools->Add-In Manager to register the task on one machine

    It's important to use Add-In Manager (not the easier "drop-in" deployment) for this step, as it updates a version of the local SAS Enterprise Guide task list so that it contains information about the task.

    In SAS Enterprise Guide, select Tools->SAS Enterprise Guide Explorer and use the Task Import Wizard

    SAS Enterprise Guide Explorer provides this special tool that can examine the local task list (which now contains your custom task) and identify the "new" tasks that are not yet part of the SAS Metadata Server. This tool allows you to add those new tasks into the metadata-controlled list.

    In SAS Management Console, edit role to disallow unregistered custom tasks

    This step is optional -- if you don't want your end users to add any custom tasks on their own. You can restrict access to any custom tasks that you add to SAS metadata.

    In SAS Management Console, edit role to allow desired custom tasks for appropriate groups

    The new tasks show up under an "Imported Tasks" group. For any given role you can decide whether the task is allowed.

    Result: End user sees "restricted" functions

    Once the settings are in effect, the end user will see that his/her functions are restricted -- as indicated in the SAS Enterprise Guide status area on the bottom of the window.

    The user can click the "Functions" link to see the full list of features that are available in the product, along with an indication as to which are disallowed for the user. The user can use this information to "petition" the SAS administrator for access to additional capabilities, if desired. (Ah, the power of a SAS administrator!)

    tags: SAS Add-In for Microsoft Office, sas administration, SAS custom tasks, SAS Enterprise Guide
    9月 152011

    As I mentioned in my introductory post about Windows PowerShell, you can use PowerShell commands as a simple and concise method to collect data from your Windows systems -- information that is ripe for analysis within SAS.

    In this example, I'll show a technique for using a SAS program to call PowerShell and funnel the results directly into your SAS session for analysis. This example uses the Get-Process cmdlet (pronounced /command-let/) to collect information about running processes on a given Windows machine.

    I built this example in pieces, making sure that each part worked before putting it all together:

    • I ran the powershell command from a command prompt console, and redirected the output to a text file. For example:
      powershell -Command "Get-Process -ComputerName L73391" 
        > c:\temp\outProcesses.txt
    • I worked with the text file in SAS and SAS Enterprise Guide to develop a DATA step that could read/interpret it
    • I combined the command and DATA step into a program using FILENAME PIPE syntax
    • I tested the program on different machines, just to make sure that the behavior of the command and the text file didn't change across machines.

    In order to run this example, you need the following conditions to be true:

     /* name of a machine on my network where I have admin rights */
    %let machine = L73391; 
    filename process pipe
      "powershell -Command ""Get-Process -ComputerName &machine.""";
    data processes;
    	infile process firstobs=4;
    		Handles 8
    		WorkingSet 8
    		CPU 8
    		ID $ 8
    		ProcessName $ 255;
    		WorkingSet comma12.
    		Handles comma6.;
    	input @1  Handles
    		@25 WorkingSet
    		@42 CPU 42-50
    		@51 ID
    		@59 ProcessName;
    title "Processes on &machine";
    /* with an ODS destination active, this will create a report */
    proc sql;
    	select * from processes order by WorkingSet desc;
    proc means data=processes;
    	var Handles WorkingSet;

    This produces a simple report of the active processes on the given machine. Here is a partial example of the output, in descending order of "Working Set" (which is one measurement of the amount of system memory in use by a process).

    tags: filename pipe, PowerShell, sas administration, SAS tips