Tech

11月 072018
 

Migration, version road maps and configurations were the themes of several questions that came up in a recent webinar about combining SAS Grid Manager and SAS Viya. You’ll see in this blog post that we were ready to get into the nitty-gritty details in our answers below – just as we did in the previous FAQs post. We hope you find them useful in your work using SAS Grid Manager and SAS Viya together.

1. Can we migrate SAS programs that are currently on SAS PC environments into the SAS Grid environment – or do we need to rewrite the programs for SAS Grid Manager?

No, you don’t need to rewrite your SAS programs to run on a SAS Grid environment. Many customers migrate their code from other environments (like PCs or servers) and submit them to SAS Grid Manager from SAS Display Manager, SAS Studio or any other application of their choice.

If you already use SAS Enterprise Guide to run jobs on a remote server, the process may be as simple as changing your server configuration to use a grid-launched workspace server (information that your SAS Administrator would provide) and continuing to work in much the same way as always, requiring no changes to your code.

Depending on other changes that take place at the same time SAS Grid Manager is implemented, there may need to be some small adjustments to your programs.  For example, if your organization consolidates source data onto new storage, you may need to change paths associated with your LIBNAME statements.  These should be housekeeping items rather than significant rewrites of the logic in your SAS code.

If you plan to continue to use the programming environment provided by BASE SAS itself (DMS) and have been using SAS/CONNECT, you will need to add the SIGNON statement to start a session on the SAS Grid Manager

  • ENDRSUBMIT statement to end the block of code to be run on the grid
  • Divide and Conquer – Writing Parallel SAS Code to Speed Up Your SAS Program.

    2. Is there a version of SAS Grid Manager that runs on the SAS Viya architecture?

    The SAS Grid Manager roadmap includes a release of SAS Grid Manager on the SAS Viya Architecture late in 2019.

    3. Will I be able to migrate my SAS Grid Manager configuration and jobs from SAS 9.4 to the SAS Viya-based release of SAS Grid Manager?

     The plan to deliver SAS Grid Manager on the SAS Viya architecture includes automation to migrate jobs, flows, and schedule information from your SAS 9.4 environment to your SAS Viya environment.  It is our goal to make this transition as straightforward and easy as possible – especially where there is feature parity between SAS 9 based and SAS Viya-based solutions.  Since each product delivers solution-specific PROCs and other functionality that can be used within a job executed by SAS Grid Manager, each customer should work with their SAS team to understand which jobs can be migrated and which jobs may need to continue to run against your SAS 9.4 environment.  

    * * *

    These were all great questions that we thought deserved more detail than we could offer in a webinar.  If you have more questions that weren’t covered here, or in our previous post on this topic, just post them in the comments section.  We’ll answer them quickly.  Thanks for your interest!

    3 questions about implementing SAS Grid Manager and SAS Viya was published on SAS Users.

  • 11月 062018
     

    This post was also written by SAS' Xiangxiang Meng.

    You can communicate with various clients (SAS, Python, Lua, Java, and REST) in the same place using Pandas Data Analysis Library, CAS actions should come naturally. CAS enables you to subset tables using Python expressions. Using Python, you can create conditions that are based on the data pulled, instead of creating the conditions yourself. SAS® will use the information you want pulled to determine which rows to select.

    For example, rather than using fixed values of rows and columns to select data, SAS can create conditions based on the data in the table to determine which rows to select. This is done using the same syntax as DataFrames. CASColumn objects support Python’s various comparison operators and builds a filter that subsets the rows in the table. You can then use the result of that comparison to index into a CASTable. It sounds much more complicated than it is, so let’s look at an example.

    The examples below are from the Iris flower data set, which is available in the SASHELP library, in all distributions of SAS. The listed code and output are produced using the IPython interface but can be employed with Jupyter Notebook just as easily.

    If we want to get a CASTable that only contains values where petal_length is greater than 7, we can use the following expression to create our filter.


    Behind the scenes, this expression creates a computed column that is used in a WHERE expression on the CASTable. This expression can then be used as an index value for a CASTable. Indexing this way essentially creates a boolean mask. Wherever the expression values are true, the rows of the table are returned. Wherever the expression is false, the rows are filtered out.

    These two steps are more commonly done in one line.


    We can further filter rows out by indexing another comparison.

    Comparisons can be joined using the bitwise comparison operators & (and) and | (or). You do have to be careful with these though due to the operator precedence. Bitwise comparison has a higher precedence than comparisons such as greater-than and less-than, so you need to wrap your comparisons in parentheses.


    In all cases, we are not changing anything about the underlying data in CAS. We are simply constructing a query that is executed with the CASTable when it is used as the parameter in a CAS action. You can see what is happening behind the scenes by displaying the resulting CASTable objects.


    You can also do mathematical operations on columns with constants or other columns within your comparisons.

    The list of supported operations is shown in the table below.

    The supported comparison and operators are shown in the following table.

    As you can see in the tables above, it is possible to do comparisons on character columns as well. This includes using many of Python’s string methods on the column values. These are accessed using the str attribute of the column, just like in DataFrames.

    This easy syntax allows the Python client to manipulate data much easier when working in SAS Viya.

    Another great tip? The Python client allows you to manipulate data on the fly, without moving or copying the data to another location. Creating computed columns allows you to speed up the wrangling of data, while giving you options for how want to get there.

    Want to learn more great tips about integrating Python with SAS Viya? Check out Kevin Smith and Xiangxiang Meng’s SAS Viya: The Python Perspective to learn how Python can be intergraded into SAS® Viya® —and help you manipulate data with ease.

    Great tip for dynamic data selection using SAS Viya and Python was published on SAS Users.

    11月 032018
     

    When you begin to work within the SAS Viya ecosystem, you learn that the central piece is SAS Cloud Analytic Services (CAS). CAS allows all clients in the SAS Viya ecosystem to communicate and run analytic methods. The great part about SAS Viya is that the R client can drive CAS directly using familiar objects and constructs for R programmers.

    The SAS Scripting Wrapper for Analytics Transfer (SWAT) package is an R interface to CAS. With this package, you can load data into memory and apply CAS actions to transform, summarize, model and score the data. You can still retain the ease-of-use of R on the client side to further post process CAS result tables.

    But before you can do any analysis in CAS, you need some data to work with and a way to get to it. There are two data access components in CAS:

    1. Caslibs, definitions that give access to a resource that contains data.
    2. CASTables, for analyzing data from a caslib resource. You load the data into a CASTable, which contains information about the data in the columns.

    Other references you may find of interest include this GitHub repository where you can find more information on installing and configuring CAS and SWAT. Also available is this article on using RStudio with SAS Viya.

    The following excerpt from SAS® Viya® : the R Perspective, the book I co-authored with my SAS colleague Xiangxiang Meng, demonstrates the way the R client in SAS Viya allows you to select data with precision. The examples come from the iris flower data set, which is available in the SASHELP library, in all distributions of SAS. The CASTable object sorttbl is sorted by the Sepal.Width column.

    Rather than using fixed values of rows and columns to select data, we can create conditions that are based on the data in the table to determine which rows to select. The specification of conditions is done using the same syntax as that used by data.frame objectsCASTable objects support R’s various comparison operators and build a filter that subsets the rows in the table. You can then use the result of that comparison to index into a CASTableIt sounds much more complicated than it is, so let’s look at an example.

    This expression creates a computed column that is used in a where expression on the CASTable. This expression can then be used as an index value for a CASTable. Indexing this way essentially creates a Boolean mask. Wherever the expression values are true, the rows of the table are returned. Wherever the expression is false, the rows are filtered out.

    > newtbl <- sorttbl[expr,] > head(newtbl) 
     
      Sepal.Length Sepal.Width Petal.Length Petal.Width   Species 
    1          7.7         2.6          6.9         2.3 virginica 
    2          7.7         2.8          6.7         2.0 virginica 
    3          7.6         3.0          6.6         2.1 virginica 
    4          7.7         3.8          6.7         2.2 virginica

    These two steps are commonly entered on one line.

    > newtbl <- sorttbl[sorttbl$Petal.Length > 6.5,]
    > head(newtbl) 
     
      Sepal.Length Sepal.Width Petal.Length Petal.Width   Species 
    1          7.7         2.6          6.9         2.3 virginica 
    2          7.7         2.8          6.7         2.0 virginica 
    3          7.6         3.0          6.6         2.1 virginica 
    4          7.7         3.8          6.7         2.2 virginica

    We can further filter rows out by indexing another comparison expression.

    > newtbl2 <- newtbl[newtbl$Petal.Width < 2.2,] > head(newtbl2) 
     
      Sepal.Length Sepal.Width Petal.Length Petal.Width   Species 
    1          7.7         2.8          6.7         2.0 virginica 
    2          7.6         3.0          6.6         2.1 virginica

    Comparisons can be joined using the bitwise comparison operators & (and) and | (or). You must be careful with these operators though due to operator precedence. Bitwise comparison has a lower precedence than comparisons such as greater-than and less-than, but it is still safer to enclose your comparisons in parentheses.

    > newtbl3 <- sorttbl[(sorttbl$Petal.Length > 6.5) & (sorttbl$Petal.Width < 2.2),] > head(newtbl3) 
     
      Sepal.Length Sepal.Width Petal.Length Petal.Width   Species 
    1          7.7         2.8          6.7         2.0 virginica 
    2          7.6         3.0          6.6         2.1 virginica

    In all cases, we are not changing anything about the underlying data in CAS. We are simply constructing a query that is executed with the CASTable when it is used as the parameter in a CAS action. You can see what is happening behind the scenes by displaying the attributes of the resulting CASTable objects.

    > attributes(newtbl3) 
     
    $conn 
    CAS(hostname=server-name.mycompany.com, port=8777, username=username, session=11ed56e2-f9dd-9346-8d01-44a496e68880, protocol=http) 
     
    $tname
    [1] "iris" 
     
    $caslib 
    [1] "" 
     
    $where 
    [1] "((\"Petal.Length\"n > 6.5) AND (\"Petal.Width\"n < 2.2))" 
     
    $orderby 
    [1] "Sepal.Width" 
     
    $groupby 
    [1] "" 
     
    $gbmode 
    [1] "" 
     
    $computedOnDemand 
    [1] FALSE 
     
    $computedVars 
    [1] "" 
     
    $computedVarsProgram 
    [1] "" 
     
    $XcomputedVarsProgram 
    [1] "" 
     
    $XcomputedVars 
    [1] "" 
     
    $names 
    [1] "Sepal.Length" "Sepal.Width"  "Petal.Length" "Petal.Width"  
    [5] "Species"      
     
    $class 
    [1] "CASTable" 
    attr(,"package") 
    [1] "swat"

    You can also do mathematical operations on columns with constants or on other columns within your comparisons.

    > iris[(iris$Petal.Length + iris$Petal.Width) * 2 > 17.5,] 
     
        Sepal.Length Sepal.Width Petal.Length Petal.Width   Species 
    118          7.7         3.8          6.7         2.2 virginica 
    119          7.7         2.6          6.9         2.3 virginica

    The list of supported operators is shown in the following table:

    Operator Numeric Data Character Data
    + (add) ✔
    - (subtract) ✔
    * (multiply) ✔
    / (divide) ✔
    %% (modulo) ✔
    %/% (integer division) ✔
    ^ (power) [✔

    The supported comparison operators are shown in the following table.

    Operator Numeric Data Character Data
    == (equality) ✔ ✔
    != (inequality) ✔ ✔
    < (less than) ✔ ✔
    > (greater than) ✔ ✔
    <= (less than or equal to) ✔ ✔
    >= (greater than or equal to) ✔ ✔

     

    As you can see in the preceding tables, you can do comparisons on character columns as well. In the following example, all of the rows in which Species is equal to "virginica" are selected and saved to a new CASTable object virginica. Note that in this case, data is still not duplicated.

    > tbl <- defCasTable(conn, 'iris') > virginica <- tbl[tbl$Species == 'virginica',] > dim(virginica) 
     
    [1] 50  5 
     
    > head(virginica) 
     
      Sepal.Length Sepal.Width Petal.Length Petal.Width   Species 
    1          7.7         3.0          6.1         2.3 virginica 
    2          6.3         3.4          5.6         2.4 virginica 
    3          6.4         3.1          5.5         1.8 virginica 
    4          6.0         3.0          4.8         1.8 virginica 
    5          6.9         3.1          5.4         2.1 virginica 
    6          6.7         3.1          5.6         2.4 virginica

    It’s easy to create powerful filters that are executed in CAS while still using the R syntax. However, the similarities to dataframe don’t end there. CASTable objects can also create computed columns and by groups using similar techniques.

    Want to learn more? Get your copy of SAS Viya: The R Perspective

    How to use SAS® Viya® and R for dynamic data selection was published on SAS Users.

    11月 012018
     

    This blog post was also written by SAS' Bari Lawhorn.

    We have had several requests from customers who want to use SAS® software to automate the download of data from a website when there is no application programming interface (API) to do it. As an example, the Yahoo Finance website recently changed their service to decommission their API, and this generated an interesting challenge for one of our customers. This SAS programmer wanted to download historical stock price data "unattended," without having to click through a web page. While working on this case, we discovered that the Yahoo Finance website requires a cookie-crumb combination to download. To help you automate downloads from websites that do not have an API, this blog post takes you through how we used the DEBUG feature of PROC HTTP to achieve partial automation, and eventually full automation with this case.

    Partial automation

    To access the historical data for Apple stock (symbol: AAPL) on the Yahoo Finance website, we use this URL: https://finance.yahoo.com/quote/AAPL/history?p=AAPL

    We click Historical Data --> Download Data and get a CSV file with historical stock price data for Apple. We could save this CSV file and read it into SAS. But, we want a process that does not require us to click in the browser.

    Because we know the HTTP procedure, we right-click Download Data and then select Copy link address as shown from a screen shot using the Google Chrome browser below:

    Note: The context menu that contains Copy link address looks different in each browser.

    Using this link address, we expect to get a direct download of the data into a CSV file (note that your crumb= will differ from ours):

    filename out "c:\temp\aapl.csv";
     
    proc http
     url='https://query1.finance.yahoo.com/v7/finance/download/AAPL?period1=1535399845&period2=1538078245&interval=1d&events=history&crumb=hKubrf50i1P'
     method="get" out=out;
    run;

    However, the above code results in the following log message:

    NOTE: PROCEDURE HTTP used (Total process time):
    real time           0.25 seconds
    cpu time            0.14 seconds
     
    NOTE: 401 Unauthorized

    When we see this note, we know that the investigation needs to go further.

    filename out "c:\temp\aapl.csv";
    proc http
     url='https://query1.finance.yahoo.com/v7/finance/download/AAPL?period1=1535399845&period2=1538078245&interval=1d&events=history&crumb=hKubrf50i1P'
     method="get" out=out;
     debug level=3;
    run;

    When we run the code, here's what we see in the log (snipped for convenience):

    > GET
    /v7/finance/download/AAPL?period1=1535399845&period2=1538078245&interval=1d&events=history&crumb=h
    Kubrf50i1P HTTP/1.1
     
    > User-Agent: SAS/9
    > Host: query1.finance.yahoo.com
    > Accept: */*
    > Connection: Keep-Alive
    > Cookie: B=fpd0km1dqnqg3&b=3&s=ug
    > 
    < HTTP/1.1 401 Unauthorized
    < WWW-Authenticate: crumb
    < Content-Type: application/json;charset=utf-8
     
    …more output…
     
    < Strict-Transport-Security: max-age=15552000
     
    …more output…
     
    {    "finance": {        "error": {            "code": "Unauthorized",
    "description": "Invalid cookie"        }    }}
    NOTE: PROCEDURE HTTP used (Total process time):
          real time           0.27 seconds
          cpu time            0.15 seconds
     
    NOTE: 401 Unauthorized

    The log snippet reveals that we did not provide the Yahoo Finance website with a valid cookie. It is important to note that the response header for the URL shows crumb for the authentication method (the line that shows WWW-Authenticate: crumb. A little web research helps us determine that the Yahoo site wants a cookie-crumb combination, so we need to also provide the cookie. But, why did we not need this step when we were using the browser? We used a tool called Fiddler to examine the HTTP traffic and discovered that the cookie was cached when we first clicked in the browser on the Yahoo Finance website:

    Luckily, starting in SAS® 9.4M3 (TS1M3), PROC HTTP will set cookies and save them across HTTP steps if the response contains a "set-cookie:  <some cookie>" header when it successfully connects to a URL. So, we try this download in two steps. The first step does two things:

    • PROC HTTP sets the cookie for the Yahoo Finance website.
    • Adds the DEBUG statement so that we can obtain the crumb value from the log.
    filename out "c:\temp\Output.txt";
     
    filename hdrout "c:\temp\Response.txt";
     
    proc http
     out=out
     headerout=hdrout
     url="https://finance.yahoo.com/quote/AAPL/history?p=AAPL"
     method="get";
     debug level=3;
    run;

    Here's our log snippet showing the set-cookie header and the crumb we copy and use in our next PROC HTTP step:

    …more output…
    < set-cookie: B=2ehn8rhdsf5r2&b=3&s=fe; expires=Wed, 17-Oct-2019 20:11:14 GMT; path=/;
    domain=.yahoo.com
     
    …more output…
     
    Initialized"},"account-switch-uh-0-AccountSwitch":{"status":"initialized"}}},"CrumbStore":{"crumb":
    "4fKG9lnt5jw"},"UserStore":{"guid":"","login":"","alias":"","firstName":"","comscoreC14":-1,"isSig

    The second step uses the cached cookie from Yahoo Finance (indicated in the "CrumbStore" value), and in combination with the full link that includes the appropriate crumb value, downloads the CSV file into our c:\temp directory.

    filename out "c:\temp\aapl.csv";
     
    proc http
     url='https://query1.finance.yahoo.com/v7/finance/download/AAPL?period1=1535399845&period2=1538078245&interval=1d&events=history&crumb=4fKG9lnt5jw'
     method="get"
     out=out;
    run;

    With the cookie value in place, our download attempt succeeds!

    Here is our log snippet:

    31
    32   proc http
    33       out=data
    34       headerout=hdrout2
    35       url='https://query1.finance.yahoo.com/v7/finance/download/AAPL?period1=1534602937&peri
    35 ! od2=1537281337&interval=1d&events=history&crumb=4fKG9lnt5jw'
    36       method="get";
    37   run;
     
    NOTE: PROCEDURE HTTP used (Total process time):
          real time           0.37 seconds
          cpu time            0.17 seconds
     
    NOTE: 200 OK

    Full automation

    This partial automation requires us to visit the website and right-click on the download link to get the URL. There’s nothing streamlined about that, and SAS programmers want full automation!

    So, how can we fully automate the process? In this section, we'll share a "recipe" for how to get the crumb value -- a value that changes with each transaction. To get the current crumb, we use the first PROC HTTP statement to "screen scrape" the URL and to cache the cookie value that comes back in the response. In this example, we store the first response in the Output.txt file, which contains all the content from the page:

    filename out "c:\temp\Output.txt";
    filename hdrout "c:\temp\Response.txt";
     
    proc http 
        out=out 
        headerout=hdrout
        url="https://finance.yahoo.com/quote/AAPL/history?p=AAPL" 
        method="get";
    run;

    It is a little overwhelming to examine the web page in its entirety. And the HTML page contains some very long lines, some of them over 200,000 characters long! However, we can still use the SAS DATA step to parse the file and retrieve the text or information that might change on a regular basis, such as the crumb value.

    In this DATA step we read chunks of the text data and scan the buffer for the "CrumbStore" keyword. Once found, we're able to apply what we know about the text pattern to extract the crumb value.

    data crumb (keep=crumb);
      infile out  recfm=n lrecl=32767;
      /* the @@ directive says DON'T advance pointer to next line */
      input txt: $32767. @@;
      pos = find(txt,"CrumbStore");
      if (pos>0) then
        do;
          crumb = dequote(scan(substr(txt,pos),3,':{}'));
          /* cookie value can have unicode characters, so must URLENCODE */
          call symputx('getCrumb',urlencode(trim(crumb)));
          output;
        end;
    run;
     
    %put &=getCrumb.;

    Example result:

     102        %put &=getCrumb.;
     GETCRUMB=PWDb1Ve5.WD
    

    We feel so good about finding the crumb, we're going to treat ourselves to a whole cookie. Anybody care for a glass of milk?

    Complete Code for Full Automation
    The following code brings it all together. We also added a PROC IMPORT step and a bonus highlow plot to visualize the results. We've adjusted the file paths so that the code works just as well on SAS for Windows or Unix/Linux systems.

    /* use WORK location to store our temp files */
    filename out "%sysfunc(getoption(WORK))/output.txt";
    filename hdrout "%sysfunc(getoption(WORK))/response1.txt";
     
    /* This PROC step caches the cookie for the website finance.yahoo.com */
    /* and captures the web page for parsing later                        */
    proc http 
      out=out
      headerout=hdrout
      url="https://finance.yahoo.com/quote/AAPL/history?p=AAPL" 
      method="get";
    run;
     
    /* Read the response and capture the cookie value from     */
    /* the CrumbStore field.                                   */
    /* The file has very long lines, longer than SAS can       */
    /* store in a single variable.  So we read in <32k chunks. */
    data crumb (keep=crumb);
      infile out  recfm=n lrecl=32767;
      /* the @@ directive says DON'T advance pointer to next line */
      input txt: $32767. @@;
      pos = find(txt,"CrumbStore");
      if (pos>0) then
        do;
          crumb = dequote(scan(substr(txt,pos),3,':{}'));
          /* cookie value can have unicode characters, so must URLENCODE */
          call symputx('getCrumb',urlencode(trim(crumb)));
          output;
        end;
    run;
     
    %put &=getCrumb.;
     
    filename data "%sysfunc(getoption(WORK))/data.csv";
    filename hdrout2 "%sysfunc(getoption(WORK))/response2.txt";
     
    proc http 
        out=data 
        headerout=hdrout2
        url="https://query1.finance.yahoo.com/v7/finance/download/AAPL?period1=1535835578%str(&)period2=1538427578%str(&)interval=1d%str(&)events=history%str(&)crumb=&getCrumb."
        method="get";
    run;
     
    proc import
     file=data
     out=history
     dbms=csv
     replace;
    run;
     
    proc sgplot data=history;
      highlow x=date high=high low=low / open=open close=close;
      xaxis display=(nolabel) minor;
      yaxis display=(nolabel);
    run;


    Disclaimer: As we've seen, Yahoo Finance could change their website at any time, so the URLs in this blog post might not be accurate at a later date. Note that, as of the time of this writing, the above code runs error-free with Base SAS 9.4M5. And it also works in SAS University Edition and SAS OnDemand for Academics!

    How to automate a data download with PROC HTTP was published on SAS Users.

    10月 312018
     

    This article is the first in a series of three posts to address REST APIs and their use in, and with, SAS. Today, I'll present a basic example using SAS Viya REST APIs to download an image from a report in SAS Visual Analytics.

    The second article will show an example of the Cloud Analytics Services (CAS) REST APIs. My third planned article will outline show a simple application that accesses SAS Viya using both sets of REST APIs.

    The inspiration for this example: a visualization of air traffic data

    I ran across a great post from Mike Drutar: How to create animated line charts that "grow" in SAS Visual Analytics. I followed the steps in Mike's example, which creates a visualization of airline traffic. The result was an animated line chart. For this article, I removed the animation, as it will serve me better in my use case.

    SAS Viya APIs and CAS APIs: Two entry points into SAS Viya

    The first thing I'd like to cover is why SAS Viya offers two sets of REST APIs. Let's consider who is using the APIs, and what they are trying to accomplish? SAS Viya APIs target enterprise application developers (who may or may not be versed in analytics), who intend to build on the work of model builders and data scientists. These developers want to deliver apps based on SAS Viya technology -- for example, to call an analytical model to score data. On the other hand, the CAS REST API is used by data scientists and programmers (who are decidedly adept at analytics) and administrators, who need to interact with CAS directly and are knowledgeable about CAS actions. CAS actions are the building blocks of analytical work in SAS Viya.

    How to get started with SAS Viya REST APIs

    The best place to start working with SAS Viya REST APIs is on the SAS Developer's web site. There, you will find links to the API documentation.

    The REST APIs are written to make it easy to integrate the capabilities of SAS Viya to help build applications or create scripts. The APIs are based on URLs, using HTTP Authentication, and HTTP verbs. The API documentation page is split into multiple categories. The following table outlines the breakdown:

    API Category Description
    Visualization Provide access to reports and report images
    Compute Act on SAS compute and analytic servers, including Cloud Analytic Services (CAS)
    Text Analytics Provide analysis and categorization of text documents
    Data Management Enable data manipulation and data quality operations on data sources
    Decision Management Provide access to machine scoring and business rules
    Core Services Provide operations for shared resources such as files and folders

     

    The REST API documentation page is divided into multiple sections.

    SAS Viya REST API doc

    1. The categories are listed in the upper-left side.
    2. Once a you select a category, related services and functions are listed in the lower left pane.
    3. The service appears in the center pane with a description, parameters, responses, and error codes.
    4. The right pane displays how to form a sample request, any optional or required body text, and sample response code.

    The REST API call process

    The example outlined in this article shows how to access a report image from SAS Visual Analytics. To try this out yourself, you will need: a SAS Viya environment (with SAS Visual Analytics configured), an access token, and a REST client. The REST client can be cURL (command line), Postman (a popular REST API environment), or Atom with the rest-client plugin -- or any other scripting language of your choice. Even if you do not have access to an environment right now, read on! As a SAS developer, you're going to want to be aware of these capabilities.

    Get a list of reports from SAS Visual Analytics

    Run the following curl command to get a list of reports on the SAS server:

    curl -X GET http://sasserver.demo.sas.com/reports/reports\
      -H 'Authorization: Bearer &lt;access-token-goes-here&gt;' \
      -H 'Accept: application/vnd.sas.table.column+json'

    Alternatively, use Postman to enter the command and parameters:

    GET Report List API call from Postman

    From the JSON response, find the report object and grab the id of the desired report:

    GET Report List Response

    Create a job

    The next step is to create an asynchronous job to generate the SVG image from the report. I use the following HTTP POST with the /jobs verb:

    curl -X POST <a href="http://sasserver.demo.sas.com/reportImages/jobs/">http://sasserver.demo.sas.com/reportImages/jobs\
      -H 'Authorization: Bearer &lt;access-token-goes-here&gt;' \
      -H 'Accept = application/vnd.sas.report.images.job+json'\
      -H 'Content-Type = application/vnd.sas.report.images.job.request+json'

    Using the following sample Body text

    {
      "reportUri" : "/reports/reports/b555ea27-f204-4d67-9c74-885311220d45",
      "layoutType" : "entireSection",
      "selectionType" : "report",
      "size" : "400x300",
      "version" : 1
    }

    Here is the sample response:

    POST Job Creation Response

    The job creation kicks off an asynchronous action. The response indicates whether the job is completed at response time, or whether it's still pending. As you can see from the above response, our job is still in a 'running' state. The next step is to poll the server for job completion.

    Poll for job completion

    Using the 'id' value from the job creation POST, the command to poll is:

    curl -X GET http://sasserver.demo.sas.com/reportImages/jobs/f7a12533-ac40-4acd-acda-e0c902c6c2c1\
      -H 'Authorization: Bearer ' \ 
      -H ‘Accept = application/vnd.sas.report.images.job+json’

    And the response:

    GET Poll Job Creation Response

    Once the job comes back with a 'completed' state, the response will contain the information we need to fetch the report image.

    Get the image

    I am now ready to get the image. Using the image file name (href field) from the response above, I run the following command:

    curl -X GET http://sasserver.demo.sas.com/reportImages/images/K1870020424B498241567.svg\
      -H 'Authorization: Bearer ' \ 
      -H ‘'Accept: image/svg+xml'

    Postman automatically interprets the response as as an image. If you use the curl command, you'll need to redirect the output to a file.

    SAS Visual Analytics Graph for Air Traffic

    What's Next?

    SAS Visual Analytics is usually considered an interactive, point-and-click application. With these REST APIs we can automate parts of SAS Visual Analytics from a web application, a service, or a script. This opens tremendous opportunities for us to extend SAS Visual Analytics report content outside the bounds of the SAS Visual Analytics app.

    I'll cover more in my next articles. In the meantime, check out the Visualization APIs documentation to see what's possible. Have questions? Post in the comments and I'll try to address in future posts.

    Using SAS Viya REST APIs to access images from SAS Visual Analytics was published on SAS Users.

    10月 312018
     

    This article is the first in a series of three posts to address REST APIs and their use in, and with, SAS. Today, I'll present a basic example using SAS Viya REST APIs to download an image from a report in SAS Visual Analytics.

    The second article will show an example of the Cloud Analytics Services (CAS) REST APIs. My third planned article will outline show a simple application that accesses SAS Viya using both sets of REST APIs.

    The inspiration for this example: a visualization of air traffic data

    I ran across a great post from Mike Drutar: How to create animated line charts that "grow" in SAS Visual Analytics. I followed the steps in Mike's example, which creates a visualization of airline traffic. The result was an animated line chart. For this article, I removed the animation, as it will serve me better in my use case.

    SAS Viya APIs and CAS APIs: Two entry points into SAS Viya

    The first thing I'd like to cover is why SAS Viya offers two sets of REST APIs. Let's consider who is using the APIs, and what they are trying to accomplish? SAS Viya APIs target enterprise application developers (who may or may not be versed in analytics), who intend to build on the work of model builders and data scientists. These developers want to deliver apps based on SAS Viya technology -- for example, to call an analytical model to score data. On the other hand, the CAS REST API is used by data scientists and programmers (who are decidedly adept at analytics) and administrators, who need to interact with CAS directly and are knowledgeable about CAS actions. CAS actions are the building blocks of analytical work in SAS Viya.

    How to get started with SAS Viya REST APIs

    The best place to start working with SAS Viya REST APIs is on the SAS Developer's web site. There, you will find links to the API documentation.

    The REST APIs are written to make it easy to integrate the capabilities of SAS Viya to help build applications or create scripts. The APIs are based on URLs, using HTTP Authentication, and HTTP verbs. The API documentation page is split into multiple categories. The following table outlines the breakdown:

    API Category Description
    Visualization Provide access to reports and report images
    Compute Act on SAS compute and analytic servers, including Cloud Analytic Services (CAS)
    Text Analytics Provide analysis and categorization of text documents
    Data Management Enable data manipulation and data quality operations on data sources
    Decision Management Provide access to machine scoring and business rules
    Core Services Provide operations for shared resources such as files and folders

     

    The REST API documentation page is divided into multiple sections.

    SAS Viya REST API doc

    1. The categories are listed in the upper-left side.
    2. Once a you select a category, related services and functions are listed in the lower left pane.
    3. The service appears in the center pane with a description, parameters, responses, and error codes.
    4. The right pane displays how to form a sample request, any optional or required body text, and sample response code.

    The REST API call process

    The example outlined in this article shows how to access a report image from SAS Visual Analytics. To try this out yourself, you will need: a SAS Viya environment (with SAS Visual Analytics configured), an access token, and a REST client. The REST client can be cURL (command line), Postman (a popular REST API environment), or Atom with the rest-client plugin -- or any other scripting language of your choice. Even if you do not have access to an environment right now, read on! As a SAS developer, you're going to want to be aware of these capabilities.

    Get a list of reports from SAS Visual Analytics

    Run the following curl command to get a list of reports on the SAS server:

    curl -X GET http://sasserver.demo.sas.com/reports/reports\
      -H 'Authorization: Bearer &lt;access-token-goes-here&gt;' \
      -H 'Accept: application/vnd.sas.table.column+json'

    Alternatively, use Postman to enter the command and parameters:

    GET Report List API call from Postman

    From the JSON response, find the report object and grab the id of the desired report:

    GET Report List Response

    Create a job

    The next step is to create an asynchronous job to generate the SVG image from the report. I use the following HTTP POST with the /jobs verb:

    curl -X POST <a href="http://sasserver.demo.sas.com/reportImages/jobs/">http://sasserver.demo.sas.com/reportImages/jobs\
      -H 'Authorization: Bearer &lt;access-token-goes-here&gt;' \
      -H 'Accept = application/vnd.sas.report.images.job+json'\
      -H 'Content-Type = application/vnd.sas.report.images.job.request+json'

    Using the following sample Body text

    {
      "reportUri" : "/reports/reports/b555ea27-f204-4d67-9c74-885311220d45",
      "layoutType" : "entireSection",
      "selectionType" : "report",
      "size" : "400x300",
      "version" : 1
    }

    Here is the sample response:

    POST Job Creation Response

    The job creation kicks off an asynchronous action. The response indicates whether the job is completed at response time, or whether it's still pending. As you can see from the above response, our job is still in a 'running' state. The next step is to poll the server for job completion.

    Poll for job completion

    Using the 'id' value from the job creation POST, the command to poll is:

    curl -X GET http://sasserver.demo.sas.com/reportImages/jobs/f7a12533-ac40-4acd-acda-e0c902c6c2c1\
      -H 'Authorization: Bearer ' \ 
      -H ‘Accept = application/vnd.sas.report.images.job+json’

    And the response:

    GET Poll Job Creation Response

    Once the job comes back with a 'completed' state, the response will contain the information we need to fetch the report image.

    Get the image

    I am now ready to get the image. Using the image file name (href field) from the response above, I run the following command:

    curl -X GET http://sasserver.demo.sas.com/reportImages/images/K1870020424B498241567.svg\
      -H 'Authorization: Bearer ' \ 
      -H ‘'Accept: image/svg+xml'

    Postman automatically interprets the response as as an image. If you use the curl command, you'll need to redirect the output to a file.

    SAS Visual Analytics Graph for Air Traffic

    What's Next?

    SAS Visual Analytics is usually considered an interactive, point-and-click application. With these REST APIs we can automate parts of SAS Visual Analytics from a web application, a service, or a script. This opens tremendous opportunities for us to extend SAS Visual Analytics report content outside the bounds of the SAS Visual Analytics app.

    I'll cover more in my next articles. In the meantime, check out the Visualization APIs documentation to see what's possible. Have questions? Post in the comments and I'll try to address in future posts.

    Using SAS Viya REST APIs to access images from SAS Visual Analytics was published on SAS Users.

    10月 312018
     

    An important step of every analytics project is exploring and preprocessing the data.  This transforms the raw data to make it useful and quality.  It might be necessary, for example, to reduce the size of the data or to eliminate some columns. All these actions accelerate the analytical project that comes right after.  But equally important is how you "productionize" your data science project.  In other words, how you deploy your model so that the business processes can make use of it.

    SAS Viya can help with that.  Several SAS Viya applications have been engineered to directly add models to a model repository including SAS® Visual Data Mining and Machine Learning, SAS® Visual Text Analytics, and SAS® Studio. While the recent post on publishing and running models in Hadoop on SAS Viya outlined how to build models, this post will focus on the process to deploy your models with SAS Model Manager to Hadoop.

    SAS Visual Data Mining and Machine Learning on SAS Viya contains a pipeline interface to assist data scientists in finding the most accurate model.  In that pipeline interface, you can do several tasks such as import score code, score your data, download score API code or download SAS/BASE scoring code.  Or you may decide – once you have a version ready - to store the model out of the development environment by registering your analytical model in a model repository.

    Registered models will show up in SAS Model Manager and are copied to the model repository.   That repository provides long-term storage and includes version control.  It's a powerful tool for managing and governing your analytical models.  A registered version of your model will never get lost, even it's deleted from your development environment.   SAS models are not the only kind of models that SAS Model Manager can handle:  Python, R, Matlab models can also be imported.

    SAS Model Manager can read, write, and manage the model repository and provide actions for model editing, comparing, testing, publishing, validating, monitoring, lineage, and history of the models.  It also allows you to easily demonstrate your compliance with regulations and policies. You can organize models into different projects.   Within a project it's feasible to test, deploy and monitor the performance of the registered models.

    Deploying your models

    Deploying, a key step for any data scientist and model manager, can assist in bringing the models into production processes. Kick off deployment by publishing your models.  SAS Model Manager can publish models to systems being used for batch processing or publish to applications where real-time execution of the models is required.   Let's have a look at how to publish the analytical model to a Hadoop cluster and run the model into the Hadoop cluster.  In doing so, you can score the data where it resides and avoid any data movement.

    1. Create the Hadoop public destination.

    The easiest way to do this is via the Visual Interface.  Go to SAS Environment Manager and click on the Publish destinations icon:

    Click on the new destination icon:

    Important:

    10月 292018
     

    CASL is a language specification that can be used by the SAS client to interact with and provide easy access to Cloud Analytic Services (CAS).  CASL is a statement-based scripting language with many uses and strengths including:

    • Specifying CAS actions to submit requests to the CAS server to perform work and return results.
    • Evaluating and manipulating the results returned by an action.
    • Creating user-defined actions and functions and creating the arguments to an action.
    • Developing analytic pipelines.

    CASL uses PROC CAS which enables you to program and execute CAS actions from the SAS client and use the results to prepare the parameters for a subsequent action.  A single PROC CAS statement can contain several CASL programs.  With the CAS procedure you can run any CAS action supported by the server, load new action sets into the server, use multiple sessions to perform asynchronous execution and operate on parameters and results as variables using the function expression parser.

    CASL, and the CAS actions, provide the most control, flexibility and options when interacting with CAS.  One can use DATA Step, CAS-enabled PROCS and CASL for optimal flexibility and control.  CASL works well with traditional SAS interfaces and the Base SAS language.

    Each CAS action belongs to an action set.  Each action set is further categorized by product (i.e. VA, VS, VDMML, etc.).  In addition to the many CAS actions supplied by SAS, as of SAS® Viya™ 3.4, you can create your own actions using CASL.  Developing and utilizing your own CAS actions allows you to further customize your code and increase your ability to work with CAS in a manner that best suits you and your organization.

    About user-defined action sets

    Developing a CASL program that is stored on the CAS server for processing is defined as a user-defined action set.  Since the action set is stored on the CAS server, the CASL statements can be written once and executed by many users. This can reduce the need to exchange files between users that store common code.  Note that you cannot add, remove, or modify a single user-defined action. You must redefine the entire action set.

    Before creating any user-defined actions, test your routines and functions first to ensure they execute successfully in CAS when submitted from the programming client.  To create user-defined actions, use the defineActionSet action in the builtins action set and add your code.  You also need to modify your code to use CASL functions such as SEND_RESPONSE, so the resulting objects on the server are returned to the client.

    Developing new actions by combining SAS-provided CAS actions

    One method for creating user-defined CAS actions is to combine one or more SAS provided CAS actions into a user-defined CAS action.  This allows you to execute just one PROC CAS statement and call all user-defined CAS actions.  This is beneficial if you repeatedly run many of the same actions against a CAS table.  An example of this is shown below. If you would like copy of the actual code, feel free to leave a reply below.

    In this example, four user-defined CAS actions named listTableInfo, simplefreq, detailfreq, and corr have been created by using the corresponding SAS-provided CAS actions tableInfo, freq, freqTab, and correlation.  These four actions return information about a CAS table, simple frequency information, detailed frequency and tabulate information, and Pearson correlation coefficients respectively.  These four actions are now part of the newly created user-defined action set myActionSet.  When this code is executed, the log will display a note that the new action set has been added.

    Once the new action set and actions have been created, you can call all four or any combination of them via a PROC CAS statement.  Specify the user-defined action set, user-defined action(s), and parameters for each.

    Developing new actions by writing your own code

    Another way to create user-defined CAS actions is to apply user-defined code, functions, and statements instead of SAS-provided CAS actions.

    In this example, two user-defined CAS actions have been created, bdayPct and sos.  These actions belong to the new user-defined action set myFunctionSet.

    To call one or both actions, specify the user-defined action set, user-defined action(s), and parameters for each.

    The results for each action are shown in the log.

    Save and load custom actions across CAS sessions

    User-defined action sets only exist in the current CAS session.  If the current CAS session is terminated, the program to create the user-defined action set must be executed again unless an in-memory table is created from the action set and the in-memory table is subsequently persisted to a SASHDAT file.  Note: SASHDAT files can only be saved to path-based caslibs such as Path, DNFS, HDFS, etc.  To create an in-memory table and persist it to a SASHDAT file, use the actionSetToTable and save CAS actions.

    To use the user-defined action set, it needs to be restored from the saved SASHDAT file.  This is done with the actionSetFromTable action.

    More about CASL programming and CAS actions

    Check out these resources for further information on programming in the CASL language and running actions with CASL.

    How to use CASL to develop and work with user-defined CAS actions was published on SAS Users.

    10月 222018
     

    This blog post was also written by Reece Clifford.

    Who’s responsible for x, y, z sales territory? What’s the most amount of people they engaged with in a month? What type of location leads to the best response from the meeting?

    To get the complete answer to these sales team-related questions, you need to trust your data. You need to be able to cut and slice high-quality data to prepare for analytics to drive innovation in your company. With SAS Data Preparation alongside SAS Decision Manager, you can do all this. Its many features allow you to perform out-of-the-box column and row transformations to increase your data quality and build the foundations for data-driven innovation.

    This blog will discuss how you can leverage SAS Decision Manager to enrich data when preparing it through SAS Data Preparation.

    The use case

    As posed above, we want to create a SAS Data Preparation plan to map a sales person to a postcode area. We use a SAS Decision Manager rule to find the sales person for a postcode area and map the person to the address. To trigger the rule, we are going to call it from SAS Data Preparation.

    In SAS Decision Manager we import a csv file to create a Lookup Table mapping a sales person to a postcode area. Lookup Tables are tables of key-value pairs and provide the ability to access reference data from business rules.

    Next, we create a rule to map a postcode and sales person. A rule specifies conditions to be evaluated and actions to be taken if those conditions are satisfied. Rules are grouped together into rule sets. Most rules correspond to the form:

    if condition_expressions then action_expressions

    For our rule, we are going to have an incoming postcode plus a record id. The postcode is assumed to be a UK postcode. We are extracting the first two characters of the postcode and lookup the sales person from the Lookup Table that we have just imported.

    The rule outputs the sales person (representative) and the record ID. When we have tested and published the rule, it's ready to be used in a SAS Data Preparation Plan.

    In SAS Data Preparation, we load a table with address data that we want to enrich by the appropriate sales person.

    1. We need to make sure the table column names and rule input parameter names match. Therefore, we are renaming the field ADDRESS_ID to ID, as ID is the rule input name. The second rule input parameter is Postcode which is the same as in the table, therefore no action is needed.

    1. We can then call the previously-created rule in SAS Decision Manager to map a sales person to an area. This will be done by adding some CASL code to the Code node in the SAS Data Preparation plan. This is featured below with a brief explanation of the functions.
      As the rule has two output parameters, we receive only two columns when executing the code step.

    CASL Code

    loadactionset "ds2";
    action runModel submit / 
    	modelTable={name="MONITORRULES", caslib="DCMRULES"}
    	modelName="Mon_Person"
    	table= {name= _dp_inputTable, caslib=_dp_inputCaslib}
    	casout= {name= _dp_outputTable, caslib=_dp_outputCaslib};

    Parameters settings for CASL call

    modelTable name Name of the table where the rule set was published to.
    modelTable caslib Name of the caslib where the rule set was published to.
    modelName Name of the decision flow to execute.
    table name Table name of the decision flow input data.

    (Set to _dp_inputTable)

    table caslib caslib name of the table input data.

    (Set to _dp_inputCaslib)

    casout name Table name of the decision flow output data.

    (Set to _dp_outputTable)

    casout caslib caslib name of the table output data.

    (Set to _dp_outputCaslib)

     

    Decision Manager Publishing Dialogue

     

    1. We then wanted to bring back the columns from the input table. We do this through joining the table in the SAS Data Preparation Plan to the original table (again) on the rule output field ID and the tables field ADDRESS_ID.

    Conclusion

    We have answered our initial question of which sales person is mapped to which region by enriching our data in a user-friendly, efficient process in SAS Data Preparation. We can now begin to gain further insight from our data to answer more of the questions posed at the beginning of the blog to help drive innovation. This can be done through additional insight using SAS Decision Manager or functions in SAS Data Preparation in the current plan or use the output table in another plan. Ultimately, this will facilitate data-driven Innovation via reporting or advanced analytics in your organisation.

    Using SAS Decision Manager to enrich the data prep process was published on SAS Users.

    10月 192018
     

    Did you know that you can now chat with SAS Technical Support? Technical Chat enables you to quickly engage with a knowledgeable consultant when you have a SAS question or need help with troubleshooting an issue.

    Technical Chat is a great tool for quick questions like these:

    • “What does this error mean?”
    • “How do I apply my new license?”
    • “What release of the operating system is supported on SAS® 9.4?”
    • “How do I activate JMP® without an internet connection?”
    • “Which function can I use to obtain the antilog of a value?”
    • “What is the status of my track?”

    How to start a Technical Chat

    To get started, click the orange Technical Chat button on select Technical Support web pages. Technical Chat is currently available in the United States and Canada, Monday–Friday from 9 a.m. to 6 p.m. Eastern Time. If the button is not available, technical assistance through other channels is listed on this SAS Technical Support page.  (And don't forget about SAS Support Communities -- peer-to-peer support that's available 24/7.)

    Having trouble with a DATA step program or an ODS statement? Specialists in the areas of SAS programming, SAS Studio, and graphics might be available during select afternoon hours. When these specialists are available, you can request their assistance as soon as you click the Technical Chat button. When prompted with What is the nature of your inquiry?, select Usage of Base SAS, SAS/Studio or graphics. For all other questions, select All other products and/or tasks. Generalists are available throughout the day to answer questions.

    Ensure that you have the following information available:

    • Site number
    • Operating system
    • Release of SAS, including the maintenance level

    When the chat begins, the chat agent’s name appears at the top of the window, as in the following example:

    Although every effort is made to resolve your question during the chat, sometimes the chat agent needs to open a track with a subject matter expert. Your question will still be addressed with the same urgency and professionalism that you are accustomed to when working with SAS Technical Support!

    Your feedback counts!

    When the chat is complete, you can request an emailed copy of the chat transcript. You can also rate your chat experience and provide feedback. Your responses are important to us as we continue to evaluate and improve our Technical Chat services.

    Try it out. . . . Chat with SAS Technical Support!

    Solve your SAS questions with Technical Chat was published on SAS Users.