SAS administrators

8月 302022
 

A key tool for SAS Viya admins and many users is the command line interface (CLI). You can think of it in terms of other CLIs, but with direct access to SAS Viya. The CLI facilitates automation of tasks as well as other administrative functions. You can find information about installing the CLI and its plugins SAS Users YouTube tutorial.

SAS Viya Workload Management Environment. SAS Viya Workload Management was developed by SAS, starting with the cloud-native version of SAS Viya and relies on Kubernetes. Kubernetes comes with capabilities to balance workload of jobs; however, SAS Viya Workload Orchestrator is designed to enhance Kubernetes’ workload capabilities by adding prioritization for jobs, queues management, etc.

Now, I am predicting you know what a plug-in is but just in case you don’t, a plug-in manages and monitors the service for which you are seeking or providing information. Before you use you any SAS Viya plug-in via the CLI, you must create a profile and authenticate. Here is the documentation which details those steps. You may also refer to the YouTube tutorial linked earlier.

Workload orchestrator plugin

In the example in Figure 1, I am using a command prompt for the CLI to connect to my SAS Viya environment. Make sure you read the links above on how to create a profile and authenticate. In the shell, navigate to the directory with the CLI executable. SAS Viya commands take the form:

> sas-viya --profile <profile name> command [command options] [arguments]

Now let’s look at some basic commands. We’ll increase the complexity of the command as we go along.

List command options

The most generic command for workload-orchestrator is listed in Figure 1.

Figure 1: List of command Line Options for workload orchestrator

The command returns the name of the CLI, the usage syntax and commands at my disposal.

Config command

Next, in Figure 2, let’s run the config command.

Figure 2: The config command displays granular details for the grid or tenant

The response displays details for additional commands for the grid or tenant configuration.

Extending the config code from above, we’ll use the config grid (and tenant) commands. With these commands you have the options get and set. Get returns the current grid information and set applies a new configuration to the grid.

Figure 3: The config command followed by grid, tenant and get

As you can see in Figure 3, I used the get command for both grid and tenant to display the configurations.

Hosts command

Now let’s look at the hosts command. In Figure 4, we use the hosts command to display the additional commands close, list and open hosts.

Figure 4: The hosts command

Figure 5 uses the list command and displays information about the current SAS Viya host.

Figure 5: The hosts command with the list command

Info command

Moving on to the info command in Figure 6, we see the available commands are grid, manager and tenant.

Figure 6: The info command with the list commands

In Figure 7, we use the three commands.

Figure 7: The info command with the grid, manager and tenant commands

The grid command lists information about the product. Manager lists information about the current usage of the manager. Finally, the tenant command lists information about the tenant.

Jobs command

Moving on to the jobs command in Figure 8.

Figure 8: The jobs command with the various commands

We see the results with the available commands including cancelling, listing, resume processing and suspending jobs.
We have a couple of commands remaining. Let’s move on.

Logs command

The logs command, as seen in Figure 9, allows you to set the log level or list specified logs.

Figure 9: The logs command with the various commands

The levels command also has get and set sub-commands. Get displays the logger levels for a specified host or all hosts. The set command sets a specified level for one more logger.

Figure 10 displays the logs command with the list command, and it shows info and error message collected.

Figure 10: The logs command with the list command

Queues command

The final command we’ll look at is the queues command. In Figure 11, you can see there are multiple commands available for queues.

Figure 11: The queues command with the various commands

Figure 12 is the list command of the queues command.

Figure 12: The queues command with the list command

Feel free to try out the other queues commands on your own.

Finally

Fairly straight forward, eh? I hope this helps you understand how you can use the workload-orchestration SAS Viya CLI plug to retrieve and configure various settings of the SAS Viya Workload Management Environment.

SAS Viya Workload Orchestrator CLI Plug-in was published on SAS Users.

7月 282022
 

SAS Grid Manager for Platform, SAS Grid Manager, and SAS Viya Workload Management all manage and balance job loads, right? So, are there three SAS products providing the same functionality? Let’s explore, as I am curious in your answer after you read this article.

Each of the three applications has a Graphical User Interface (GUI), which provides basic functionality like monitoring and managing jobs, queues, and hosts. The GUIs differ slightly between applications, but let’s save that detailed conversation for another post. For now, let’s take a look at the applications.

Interfaces

SAS Grid Manager for Platform

SAS Grid Manager for Platform is developed by IBM and provided as an OEM license through SAS. This product was first made available in SAS 9.4 M2. The GUI has matured over time and Figure 1 represents the latest iteration.

Figure 1: Visual of the user interface for SAS Grid Manager for Platform

SAS Grid Manager for Platform takes advantage of the Load Sharing Facility (LSF), Process Manager (PM), and Platform Web Services (PWS) features. Together these features provide services for job execution, job scheduling and host information. You launch the GUI for SAS Grid Manager for Platform from a separate URL and it is also listed in the Instructions.html file generated during the install process. Most SAS customers use Platform Suite for SAS. It is our legacy product.

SAS Grid Manager

In SAS 9.4 M6, SAS released SAS Grid Manager which balances workloads, supports high availability, and has multiprocessing capabilities. See Figure 2 for a detailed view.

Figure 2: Visual of the user interface for SAS Grid Manager, which is called SAS Workload Orchestrator

The Balance Workload feature ensures the spread of resources across machines and they are not overloaded with running jobs. High Availability ensures resource availability in case of machine failure and insures jobs still run and execute. Multiprocessing capabilities breaks down jobs, so they run across machines.
The GUI for SAS Grid Manager goes by the name SAS Workload Orchestrator (SWO). You launch the SWO from a separate URL and it is also listed in the Instructions.html, which generates during the install of SAS. Since it is built with the SAS threaded kernel, SAS Grid Manager takes great advantages of the SAS features and capabilities.

SAS Viya Workload Management

SAS developed the SAS Viya Workload Management for the newest version of SAS Viya. The GUI for SAS Viya Workload Management goes by the name Workload Orchestrator (hmmm, sound familiar? 😉). The plug-in appears as a menu item in the left-hand navigation pane from SAS Environment Manager, as seen in Figure 3 below.

Figure 3: Visual of the user interface for SAS Viya Workload Management

SAS Viya is now cloud native and relies on Kubernetes and its capabilities for balancing job workloads. The SAS Viya Workload Orchestrator is designed to enhance Kubernetes’ workload proficiencies by adding prioritization for jobs, queue management, etc.

Functionality

Now that we've seen some of the differences in the user interfaces, let’s take a look at functionality.

SAS Grid Manager for Platform

SAS Grid Manager for Platform has a Grid Controller and Nodes. As seen in Diagram 1, the SAS Grid Controller machine has Process Manager installed on the machine.

Diagram 1: SAS Grid Manager for Platform

The Process Manager schedules the job and gives the job to the Load Sharing Facility (LSF). LSF executes the job and then submits the results back to Process Manager.

SAS Grid Manager

The SAS Workload Orchestrator (SWO) is the “brains” that acts as a coordination engine for SAS Grid Manager. As depicted in Diagram 2, the SWO keeps track of all of the entry points for jobs, including jobs that come in through the Metadata Server (e.g., Enterprise Guide sessions), Mid-Tier (e.g., BI Server sessions), and through batch submission (e.g., usage of gsub utility).

Diagram 2: SAS Grid Manager

These entry points then utilize compute resources; those resources may be managed through the Workspace Server, Stored Process server, or may be spawned directly through batch submission.

SAS Viya Workload Management

SAS Viya Workload Management extends the Kubernetes capabilities. SAS Compute talks to the Launcher Service, followed by the Workload Orchestrator Manager, which talks to the Workload Orchestrator Server. This flow is depicted in Diagram 3 below.

Diagram 3: SAS Viya Workload Management

The Kubernetes API Server talks to the Workload Orchestrator Manager and Server. It determines which nodes can start a pod for jobs. SAS Viya Workload Management adds queue prioritization and other features as well.

Side-by-side comparison

The following table summarizes the features of each product we’ve discussed in this article.

Finally

So, what do you think? Are SAS Grid Manager, SAS Grid for Platform and SAS Viya Workload Management alike or different? Can we agree on both? In my opinion, there are distant differences but there are some similarities, also..:-)

To summarize, all three products manage jobs and processes. SAS Grid Manager and SAS Grid Manager for Platform are similar running on physical hardware in the older version of SAS with multiple applications to manage. While SAS Viya Workload Management is running on the newest release of SAS in a cloud-based environment taking advantage of the Kubernetes technology in a one stop shop of SAS Environment Manager.

Manage and Balance Workloads in SAS was published on SAS Users.

5月 172022
 

Here at SAS, we understand the importance of having access to cutting-edge professional resources. That’s why, for more than 40 years, we’ve provided individuals in programming, data management and analytics fields with low-cost and no-cost materials that promote success in their educational and professional journeys. And today, as the demand for employees with advanced skill sets and global certifications grows, we get it – having the ability to easily access the tools you need to succeed is more important than ever.

We’ve got you covered.

As part of our ongoing commitment to helping individuals enhance their skills, further their careers and increase their chance of success in the field, we’re now offering SAS Certification Practice Exams for free. Yes, free.

Over the years, candidates who have taken advantage of our practice exams have found them to be a valuable, effective tool for gauging their preparedness for SAS Certification Exams. When combined with other SAS training resources – like webinars, content guides, training courses and web tutorials – these free exams greatly increase candidates’ chances of success. Exams are currently available in:

    • Programming
    • Advanced Analytics
    • Data Management
    • Visual Analytics
    • Administration

Representative of the live exams, our online practice exams go through the same rigorous development process and are designed to give candidates an idea of what they should expect in the actual test questions. SAS practice exams also provide the rationale behind correct and incorrect answers, giving participants even more insight and opportunity for exam success.

And the numbers speak for themselves.*

    • Those who passed the practice exam had a 17.5% higher pass rate on the live exam than those who did not take or pass the practice version.
    • Those who took the practice exam – regardless of score – had an 8.15% higher pass rate on the live exam than those who opted not to take it.

Not to mention taking a practice test just might make the difference between passing the SAS Certification Exam on the first try or having to retake it. We’ve found that many who don’t pass the Certification Exam miss the mark by only a few questions, which they could have avoided with a bit more preparation.

Keep in mind, in order to make the best use of this resource, we recommend taking a practice exam as your final method of study in order to test your preparedness before diving into the actual exam. If you are curious about the types of questions typically on the certification exams, we have sample questions available for you to review.

So, what are you waiting for? A free resource to prepare you for a globally-recognized certification and make your resume stand out from the rest? That’s a no-brainer.

For more information about free SAS Certification Practice Exams, visit: https://www.sas.com/en_us/certification/resources/sas-practice-exams.html

*Based on data from 20,000 practice exams taken since 2020.

Access for success: SAS Certification practice exams now offered for free was published on SAS Users.

5月 052022
 

The SAS Viya LTS 2021.2 release supports application multi-tenancy. This 3-part series reviews how authentication can be configured.

SAS Viya 2021.2 OIDC with Multi-Tenancy

In this post, we look at configuring OpenID Connect (OIDC) authentication in a multi-tenant (M-T) environment. The SAS Viya configuration of OIDC authentication could be performed in the M-T provider and apply to all tenants or could be configured in one or more individual tenants and only apply to that tenant. For this post, we configure ODIC for the M-T provider and the marketing tenant.

The third-party OIDC Identity Provider, such as Azure Active Directory, has three key properties. These are:

  1. The Relying Party ID or Client ID – which is a unique name for the SAS Viya instance in the third-party OIDC Identity Provider.
  2. The Relying Party Secret or Client Secret – which is a shared secret between the SAS Viya instance and the third-party OIDC Identity Provider used to authenticate SAS Viya to the OIDC Identity Provider.
  3. One or more Redirect URIs – these are normally fully qualified URLs registered with the third-party OIDC Identity Provider. Only authentication requests coming from these registered URLs are valid.

SAS Viya 2021.2 SAML with Multi-Tenancy

In this post, we look at configuring SAML authentication in a multi-tenant (M-T) environment. There are two parts to the SAS Viya SAML configuration. The first part configures SAS Logon Manager as a SAML Service Provider and is configured with the settings under sas.logon.saml. The SAML Service Provider settings are configured only in the M-T provider.

The second part of the SAS Viya SAML configuration is the link to the third-party SAML Identity Provider, and is configured with the settings under sas.logon.saml.providers. The sas.logon.saml.providers settings can be configured for the M-T provider and apply to the provider and all tenants. Or the sas.logon.saml.providers settings can be configured for an individual tenant and apply to that tenant only.

For this post, we configure the sas.logon.saml settings in the M-T provider. Then we configure the sas.logon.saml.providers settings in both the M-T provider and the marketing tenant.

SAS Viya 2021.2.4 Bypass SAS Logon Manager

Prior to the SAS Viya 2021.2.4 release, SAS provided a sample patch transformer to update the SAS Logon Manager Ingress definition. This patch transformer used a server snippet annotation to insert a login_hint into requests. Adding the login_hint allowed customers using SAML or OpenID Connect to bypass SAS Logon Manager, meaning that end-users would automatically get redirected to the third-party SAML or OIDC Identity Provider (IdP). However, an issue with custom snippets was discovered, as documented in CVE-2021-25742. With SAS Viya 2021.2.4 the sample patch transformer has been removed and SAS is providing a configuration option to enable bypassing SAS Logon Manager. In this post, we discuss this updated approach for bypassing SAS Logon Manager.

Configuring authentication for SAS Viya multi-tenancy: A 3-part series was published on SAS Users.

12月 072021
 

This post is written in the hopes of easing the SAS Viya deployment process for novices like me. Firstly, deploying SAS Viya, like most enterprise software packages, isn't a skill we're innately born with. We're going to need a little help, some good documentation, and time to absorb the intricoes of the task.

There are many parts and pieces to standing up SAS Viya, depending on what you’re trying to accomplish and how you’d like to go about doing it. Know that the documentation and process can seem colossal and overwhelming, so take your time and don’t rush things. You got this.

Scope of the post

What this blog is and is not

This post will not walk you through the entirety of a deployment. Instead, it’ll point you to the right resources, guide you away from pitfalls, and show you how to accomplish certain tasks the documentation may not entirely cover. Many of these nuances were hard-earned lessons either by me or by people who have been kind enough to show me the way.

Please note the following

  • my experience is limited, and mostly pertains to AWS and Azure
  • the information is current at the time of this writing (December, 2021)

Please feel free to reach out to me if you have any suggestions, comments, or spot any mistakes. Many thanks!

Santa’s Workshop

Deploying SAS Viya is akin to creating toy trains in Santa’s workshop.

At its core, each toy train requires an engine, several cars, and a track. Likewise, each SAS Viya deployment requires a CAS engine that lives on a Kubernetes cluster together with several other servers (e.g., Compute, Connect, Stateful/Stateless), and storage.

Each toy train can be modified in numerous ways depending on the person’s preferences, whether it’s a steam locomotive or a bullet train. Or maybe it’s something more trivial, like merely the color of the train. Regardless of the need, Santa’s workshop contains a plethora of tools, materials, and plenty of knowledgeable elves who have different expertise and insights to customize the pipelines and trains.

Once again, each instance of SAS Viya can also be modified greatly depending on the customer’s needs. There are many hosts, flavors of servers, storage options, and common customizations. A SAS Viya deployment has its own kitchen sink full of tools, pipelines, and methods. And just like in Santa’s workshop, there are plenty of people who are experienced with deploying SAS Viya (and have specialties in different aspects of the deployment) who will assist if you run into issues.

Links Galore

There’s never a shortage of links required to complete deployments. I find myself with multiple windows filled with tabs (for referencing info) while I’m deploying so here’s a list of some I have found helpful.

      1. Setting up SAS Viya Monitoring for K8S
      2. Azure
      3. AWS
      4. GCP
      5. SAS Viya 4 Resource Guide

      Setting Up Your Environment

      There are several required tools for deployment. These include but are not necessarily limited to:

      • kubectl v1.19.9
      • kustomize 3.7.0
      • Docker

      Ensure your environment is set up precisely the way the docs recommend. For example, if you’re going the Terraform route from Viya4-IaC-AWS, you’re going to need this:

      • Terraform v1.0.0
      • kubectl v1.19.9
      • jq v1.6
      • AWS CLI v2.1.29

      The documentation is rather specific in terms of the required version, so please read carefully.

      Starting Off

      To start off, there are a few required readings to get a better understanding of SAS Viya architecture and requirements. Please review these webpages as often as you’d like to ensure understanding and avoid missing any steps.

      1. Getting Started portion of the Viya Operations documentation (linked previously).
      2. In the System Requirements section, please pay special attention to the “Kubernetes Client Machine Requirements” (under Virtual Infrastructure Requirements) to ensure you have the right tools and versions installed.
      3. When you’re done reading the above, it’s time to set up the IaC.

      Choose the corresponding link for “Help with Cluster Setup” (under Virtual Infrastructure Requirements) based on your cloud host of choice.

      IaC

      IaC stands for Infrastructure as Code. These are essentially scripts allowing you to build your cloud infrastructure and provision them through code instead of through the GUI. Several things to note here:

      1. I prefer cloning the IaC repo alongside the other folders, not within them so they’re better organized. It looks something like this:
        Viya4 <– Parent directory
        |– IaC
        |– Deploy
      2. Grab a sample .tfvars file under /examples and paste it into the root IaC directory. I recommend the “sample-input-minimal.tfvars” file if you’re just practicing.
        • Rename this file to “terraform.tfvars” (or preferred name, just be aware that the doc’s instructions assume that you have named it “terraform.tfvars”)
        • This file has several important values to keep in mind / input.
          1. This file contains the cluster configuration and details what all will be created
          2. “prefix” is essentially the name given to all your resources
          3. “default_public_access_cidrs” are CIDRs that you’d like to allow access to your cluster.
          4. “tags” you should include are {“resourceowner”=”your_Email”} (this is to ensure that people will be able to tell who owns the resource. Also, note that the preferred syntax is dependent on the cloud provider, please check the docs to be sure)
          5. “postgres_servers” should only be uncommented if you require an external db server (more expensive), if you don’t and you’re just practicing, leave it commented and it should create an internal one
      3. I highly recommend going the Docker route instead of Terraform (I have personally run into fewer problems through Docker, especially the tearing down process as compared to Terraform).
      4. It takes a while to create the cloud resources so have patience (takes about 15 mins at most).
      5. Once the resources exist, ensure you copy the [prefix]-eks-kubeconfig.conf file into your $(pwd) as well as your ~/.kube/config file if you’d like to keep it. The command to copy the conf file to your ~/.kube location is cp &lt;.conf file&gt; ~/.kube/config
      6. After you’re done with the above, make sure you run export KUBECONFIG=&lt;.conf file&gt;
      7. Test that your deployment is actually up: kubectl get nodes

      Post-IaC

      The next section covers additional SAS Viya requirements for the cluster after standing it up. There are a few things I’d recommend building after ensuring the deployment is up.

      • Ingress Controller
        kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.44.0/deploy/static/provider/cloud/deploy.yaml
      • Cert Manager
        kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.2.0/cert-manager.yaml
      • Helm/nfs-provisioner (this part is specifically for AWS)
        What’s happening here is that we’re getting the elastic load balancer URL from the ingress-nginx, the EFS ID, and installing the NFS server provisioner

        kubectl get service -n ingress-nginx
         
        ELBURL=$(kubectl get svc -n ingress-nginx ingress-nginx-controller --output jsonpath='{.status.loadBalancer.ingress[0].hostname}')
        echo $ELBURL
         
        EFSFSID=$(aws efs describe-file-systems --region $AWS_DEFAULT_REGION --query "FileSystems[*].FileSystemId" --region $AWS_DEFAULT_REGION --output text)
        echo $EFSFSID
         
        helm repo add stable  --force-update
        helm install stable/nfs-server-provisioner
         
        kubectl get storageclass # to check if the NFS server is up
      • Create a namespace where your SAS Viya deployment lives in the cluster – kubectl create ns . It is critical to go through the System Requirements entirely to ensure you don’t miss any steps (Just be sure that you’re following the portions meant for your cloud host). Examples in the Hardware and Resource Requirements page:
        • Azure – There’s an “Additional PVC Requirements for Microsoft Azure”: a link for “Specify PersistentVolumeClaims to Use ReadWriteMany StorageClass” where you’re required to add a file in the /site-config directory and an additional portion under “transformers” in the kustomization.yaml file
        • AWS – Under “File System and Shared Storage Requirements” refer to the notes on installing a provisioner for the EBS volumes. (The instructions are in the code block above)

      Installation

      This section sets up the parameters and additional customizations to included in the $deploy folder. It falls specifically under the Deployment tab of the SAS Viya Operations documentation.
      After retrieving the required files (under the desired version!), the certificates, license, and all assets and untarring them, take a good look at the section named "Directory Structure" so you have an understanding your desired file structure.

      Under “Installation -> Initial kustomization.yaml file”, once you’ve created your kustomization.yaml file, there are a few things of note here to change:

      1. {{ NAME-OF-NAMESPACE }}
        • If you haven’t already created the namespace where SAS Viya will live, do so now (instructions above #4)
        • Once you have a namespace, replace the entire thing including the {{}} with the name you have chosen.
        • You can always check what namespaces your cluster has by running kubectl get ns
      2. {{ NAME-OF-INGRESS-HOST }} and {{ PORT }} (note that there are multiple references in the kustomization.yaml file)
        • Use kubectl get service -n ingress-nginx and use the external-ip of the output
        • port is 80

      There are plenty of instructions beneath the kustomization.yaml file example, be sure to read through them and follow their instructions thoroughly.

      Additionally, Configure TLS

    1. kustomize build -o site.yaml
      kubectl apply --kubeconfig=kubeconfig-file --selector="sas.com/admin=cluster-wide" -f site.yaml
      kubectl wait --kubeconfig=kubeconfig-file --for condition=established --timeout=60s -l "sas.com/
      admin=cluster-wide" crd
      kubectl apply --kubeconfig=kubeconfig-file --selector="sas.com/admin=cluster-local" -f site.yaml --prune
      kubectl apply --kubeconfig=namespace-kubeconfig-file --selector="sas.com/admin=namespace" -f site.yaml --prune

OR
kustomize build . | kubectl apply -f -
(Note that this is the shortcut of building and piping the results to be applied in kubectl. It does not output a site.yaml file.)

There are a few false-positive errors that may appear during the process (the documentation outlines them pretty clearly).

Post-Deployment

You may run the readiness service to check for when your deployment is ready. Note that this process is lengthy and the fastest I’ve seen a deployment go up is about 15-20 mins. (Now’s a good time to go for a walk or get a cup of coffee).

I highly recommend using Lens to visualize the deployment process and to take a look at the pods and their logs (mini section below).

While all of these steps are possible in Lens, it’s good to know the commands required to inspect and manipulate pods.

kubectl get pods -n  # Take a look at all the pods, add a -W flag to watch them as they update
kubectl describe pod  -n  # To describe specific pods
kubectl logs  -n  # To see the logs of specific pods
kubectl delete pods  -grace-period=0 --force # To force deletion of pods, pods will automatically restart after being deleted.

Important pods to look at:

  • Logon
  • Consul
  • Cache

These pods are pre-requisites for many other pods to come up. If they’re stuck, go ahead and delete them to initiate a restart. This seems to work frequently.

If the pods look good, try going to this website: www.name-of-ingress-host:port/SASDrive. You should see a blue background and a SAS login screen.

Hooray! Now you just have to follow the Sign In as the sasboot User instructions and complete other post deployment tasks (Post-Installation Tasks, Validating the Deployment, etc.” that are pertinent to your use case.

Quick aside: Lens

K8s Lens is an incredibly useful IDE to visualize what is going on in your Kubernetes cluster.
Here are two quick screenshots to help you get situated when you’re looking at pods.

First, you need your .conf file to connect to your cluster. Upon entry, click on Workloads -> Pods to look at the pods. Also click on your namespace for all of the pods for the SAS Viya Deployment to show up.

There are times where you’ll see a yellow triangle with an exclamation mark. While this is technically a warning, it may be an indicator of an error your pod is suffering from. (If you see a HTTP 503 Readiness Probe error, it may just mean that the pod is starting up)

Click on the pod and the lines on the top right in order to see the logs for the chosen pod.

Conclusion

Hopefully this post was helpful for your start in deploying SAS Viya.

Please remember there’s a lot more to it than is covered here. Don’t be disheartened if this wasn’t particularly easy, it certainly wasn’t for me.
Know there are plenty of customizations as well as a constant stream of changes (updates, product related etc.), new methods, and places to deploy.
So there’s always plenty to learn.

Please feel free to reach out and let me know if you have any questions or suggestions for this post.

Acknowledgements

Many thanks to my colleagues Ali Aiello and Jacob Braswell for answering my incessant questions and helping me on this journey!

A Novice Perspective on SAS Viya Deployment was published on SAS Users.

12月 072021
 

This post is written in the hopes of easing the SAS Viya deployment process for novices like me. Firstly, deploying SAS Viya, like most enterprise software packages, isn't a skill we're innately born with. We're going to need a little help, some good documentation, and time to absorb the intricoes of the task.

There are many parts and pieces to standing up SAS Viya, depending on what you’re trying to accomplish and how you’d like to go about doing it. Know that the documentation and process can seem colossal and overwhelming, so take your time and don’t rush things. You got this.

Scope of the post

What this blog is and is not

This post will not walk you through the entirety of a deployment. Instead, it’ll point you to the right resources, guide you away from pitfalls, and show you how to accomplish certain tasks the documentation may not entirely cover. Many of these nuances were hard-earned lessons either by me or by people who have been kind enough to show me the way.

Please note the following

  • my experience is limited, and mostly pertains to AWS and Azure
  • the information is current at the time of this writing (December, 2021)

Please feel free to reach out to me if you have any suggestions, comments, or spot any mistakes. Many thanks!

Santa’s Workshop

Deploying SAS Viya is akin to creating toy trains in Santa’s workshop.

At its core, each toy train requires an engine, several cars, and a track. Likewise, each SAS Viya deployment requires a CAS engine that lives on a Kubernetes cluster together with several other servers (e.g., Compute, Connect, Stateful/Stateless), and storage.

Each toy train can be modified in numerous ways depending on the person’s preferences, whether it’s a steam locomotive or a bullet train. Or maybe it’s something more trivial, like merely the color of the train. Regardless of the need, Santa’s workshop contains a plethora of tools, materials, and plenty of knowledgeable elves who have different expertise and insights to customize the pipelines and trains.

Once again, each instance of SAS Viya can also be modified greatly depending on the customer’s needs. There are many hosts, flavors of servers, storage options, and common customizations. A SAS Viya deployment has its own kitchen sink full of tools, pipelines, and methods. And just like in Santa’s workshop, there are plenty of people who are experienced with deploying SAS Viya (and have specialties in different aspects of the deployment) who will assist if you run into issues.

Links Galore

There’s never a shortage of links required to complete deployments. I find myself with multiple windows filled with tabs (for referencing info) while I’m deploying so here’s a list of some I have found helpful.

      1. Setting up SAS Viya Monitoring for K8S
      2. Azure
      3. AWS
      4. GCP
      5. SAS Viya 4 Resource Guide

      Setting Up Your Environment

      There are several required tools for deployment. These include but are not necessarily limited to:

      • kubectl v1.19.9
      • kustomize 3.7.0
      • Docker

      Ensure your environment is set up precisely the way the docs recommend. For example, if you’re going the Terraform route from Viya4-IaC-AWS, you’re going to need this:

      • Terraform v1.0.0
      • kubectl v1.19.9
      • jq v1.6
      • AWS CLI v2.1.29

      The documentation is rather specific in terms of the required version, so please read carefully.

      Starting Off

      To start off, there are a few required readings to get a better understanding of SAS Viya architecture and requirements. Please review these webpages as often as you’d like to ensure understanding and avoid missing any steps.

      1. Getting Started portion of the Viya Operations documentation (linked previously).
      2. In the System Requirements section, please pay special attention to the “Kubernetes Client Machine Requirements” (under Virtual Infrastructure Requirements) to ensure you have the right tools and versions installed.
      3. When you’re done reading the above, it’s time to set up the IaC.

      Choose the corresponding link for “Help with Cluster Setup” (under Virtual Infrastructure Requirements) based on your cloud host of choice.

      IaC

      IaC stands for Infrastructure as Code. These are essentially scripts allowing you to build your cloud infrastructure and provision them through code instead of through the GUI. Several things to note here:

      1. I prefer cloning the IaC repo alongside the other folders, not within them so they’re better organized. It looks something like this:
        Viya4 <– Parent directory
        |– IaC
        |– Deploy
      2. Grab a sample .tfvars file under /examples and paste it into the root IaC directory. I recommend the “sample-input-minimal.tfvars” file if you’re just practicing.
        • Rename this file to “terraform.tfvars” (or preferred name, just be aware that the doc’s instructions assume that you have named it “terraform.tfvars”)
        • This file has several important values to keep in mind / input.
          1. This file contains the cluster configuration and details what all will be created
          2. “prefix” is essentially the name given to all your resources
          3. “default_public_access_cidrs” are CIDRs that you’d like to allow access to your cluster.
          4. “tags” you should include are {“resourceowner”=”your_Email”} (this is to ensure that people will be able to tell who owns the resource. Also, note that the preferred syntax is dependent on the cloud provider, please check the docs to be sure)
          5. “postgres_servers” should only be uncommented if you require an external db server (more expensive), if you don’t and you’re just practicing, leave it commented and it should create an internal one
      3. I highly recommend going the Docker route instead of Terraform (I have personally run into fewer problems through Docker, especially the tearing down process as compared to Terraform).
      4. It takes a while to create the cloud resources so have patience (takes about 15 mins at most).
      5. Once the resources exist, ensure you copy the [prefix]-eks-kubeconfig.conf file into your $(pwd) as well as your ~/.kube/config file if you’d like to keep it. The command to copy the conf file to your ~/.kube location is cp &lt;.conf file&gt; ~/.kube/config
      6. After you’re done with the above, make sure you run export KUBECONFIG=&lt;.conf file&gt;
      7. Test that your deployment is actually up: kubectl get nodes

      Post-IaC

      The next section covers additional SAS Viya requirements for the cluster after standing it up. There are a few things I’d recommend building after ensuring the deployment is up.

      • Ingress Controller
        kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.44.0/deploy/static/provider/cloud/deploy.yaml
      • Cert Manager
        kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.2.0/cert-manager.yaml
      • Helm/nfs-provisioner (this part is specifically for AWS)
        What’s happening here is that we’re getting the elastic load balancer URL from the ingress-nginx, the EFS ID, and installing the NFS server provisioner

        kubectl get service -n ingress-nginx
         
        ELBURL=$(kubectl get svc -n ingress-nginx ingress-nginx-controller --output jsonpath='{.status.loadBalancer.ingress[0].hostname}')
        echo $ELBURL
         
        EFSFSID=$(aws efs describe-file-systems --region $AWS_DEFAULT_REGION --query "FileSystems[*].FileSystemId" --region $AWS_DEFAULT_REGION --output text)
        echo $EFSFSID
         
        helm repo add stable  --force-update
        helm install stable/nfs-server-provisioner
         
        kubectl get storageclass # to check if the NFS server is up
      • Create a namespace where your SAS Viya deployment lives in the cluster – kubectl create ns . It is critical to go through the System Requirements entirely to ensure you don’t miss any steps (Just be sure that you’re following the portions meant for your cloud host). Examples in the Hardware and Resource Requirements page:
        • Azure – There’s an “Additional PVC Requirements for Microsoft Azure”: a link for “Specify PersistentVolumeClaims to Use ReadWriteMany StorageClass” where you’re required to add a file in the /site-config directory and an additional portion under “transformers” in the kustomization.yaml file
        • AWS – Under “File System and Shared Storage Requirements” refer to the notes on installing a provisioner for the EBS volumes. (The instructions are in the code block above)

      Installation

      This section sets up the parameters and additional customizations to included in the $deploy folder. It falls specifically under the Deployment tab of the SAS Viya Operations documentation.
      After retrieving the required files (under the desired version!), the certificates, license, and all assets and untarring them, take a good look at the section named "Directory Structure" so you have an understanding your desired file structure.

      Under “Installation -> Initial kustomization.yaml file”, once you’ve created your kustomization.yaml file, there are a few things of note here to change:

      1. {{ NAME-OF-NAMESPACE }}
        • If you haven’t already created the namespace where SAS Viya will live, do so now (instructions above #4)
        • Once you have a namespace, replace the entire thing including the {{}} with the name you have chosen.
        • You can always check what namespaces your cluster has by running kubectl get ns
      2. {{ NAME-OF-INGRESS-HOST }} and {{ PORT }} (note that there are multiple references in the kustomization.yaml file)
        • Use kubectl get service -n ingress-nginx and use the external-ip of the output
        • port is 80

      There are plenty of instructions beneath the kustomization.yaml file example, be sure to read through them and follow their instructions thoroughly.

      Additionally, Configure TLS

    1. kustomize build -o site.yaml
      kubectl apply --kubeconfig=kubeconfig-file --selector="sas.com/admin=cluster-wide" -f site.yaml
      kubectl wait --kubeconfig=kubeconfig-file --for condition=established --timeout=60s -l "sas.com/
      admin=cluster-wide" crd
      kubectl apply --kubeconfig=kubeconfig-file --selector="sas.com/admin=cluster-local" -f site.yaml --prune
      kubectl apply --kubeconfig=namespace-kubeconfig-file --selector="sas.com/admin=namespace" -f site.yaml --prune

OR
kustomize build . | kubectl apply -f -
(Note that this is the shortcut of building and piping the results to be applied in kubectl. It does not output a site.yaml file.)

There are a few false-positive errors that may appear during the process (the documentation outlines them pretty clearly).

Post-Deployment

You may run the readiness service to check for when your deployment is ready. Note that this process is lengthy and the fastest I’ve seen a deployment go up is about 15-20 mins. (Now’s a good time to go for a walk or get a cup of coffee).

I highly recommend using Lens to visualize the deployment process and to take a look at the pods and their logs (mini section below).

While all of these steps are possible in Lens, it’s good to know the commands required to inspect and manipulate pods.

kubectl get pods -n  # Take a look at all the pods, add a -W flag to watch them as they update
kubectl describe pod  -n  # To describe specific pods
kubectl logs  -n  # To see the logs of specific pods
kubectl delete pods  -grace-period=0 --force # To force deletion of pods, pods will automatically restart after being deleted.

Important pods to look at:

  • Logon
  • Consul
  • Cache

These pods are pre-requisites for many other pods to come up. If they’re stuck, go ahead and delete them to initiate a restart. This seems to work frequently.

If the pods look good, try going to this website: www.name-of-ingress-host:port/SASDrive. You should see a blue background and a SAS login screen.

Hooray! Now you just have to follow the Sign In as the sasboot User instructions and complete other post deployment tasks (Post-Installation Tasks, Validating the Deployment, etc.” that are pertinent to your use case.

Quick aside: Lens

K8s Lens is an incredibly useful IDE to visualize what is going on in your Kubernetes cluster.
Here are two quick screenshots to help you get situated when you’re looking at pods.

First, you need your .conf file to connect to your cluster. Upon entry, click on Workloads -> Pods to look at the pods. Also click on your namespace for all of the pods for the SAS Viya Deployment to show up.

There are times where you’ll see a yellow triangle with an exclamation mark. While this is technically a warning, it may be an indicator of an error your pod is suffering from. (If you see a HTTP 503 Readiness Probe error, it may just mean that the pod is starting up)

Click on the pod and the lines on the top right in order to see the logs for the chosen pod.

Conclusion

Hopefully this post was helpful for your start in deploying SAS Viya.

Please remember there’s a lot more to it than is covered here. Don’t be disheartened if this wasn’t particularly easy, it certainly wasn’t for me.
Know there are plenty of customizations as well as a constant stream of changes (updates, product related etc.), new methods, and places to deploy.
So there’s always plenty to learn.

Please feel free to reach out and let me know if you have any questions or suggestions for this post.

Acknowledgements

Many thanks to my colleagues Ali Aiello and Jacob Braswell for answering my incessant questions and helping me on this journey!

A Novice Perspective on SAS Viya Deployment was published on SAS Users.

2月 252021
 

The people, the energy, the quality of the content, the demos, the networking opportunities…whew, all of these things combine to make SAS Global Forum great every year. And that is no exception this year.

Preparations are in full swing for an unforgettable conference. I hope you’ve seen the notifications that we set the date, actually multiple dates around the world so that you can enjoy the content in your region and in your time zone. No one needs to set their alarm for 1:00am to attend the conference!

Go ahead and save the date(s)…you don’t want to miss this event!

Content, content, content

We are working hard to replicate the energy and excitement of a live conference in the virtual world. But we know content is king, so we have some amazing speakers and content lined up to make the conference relevant for you. There will be more than 150 breakout sessions for business leaders and SAS users, plus the demos will allow you to see firsthand the innovative solutions from SAS, and the people who make them. I, for one, am looking forward to attending live sessions that will allow attendees the opportunity to ask presenters questions and have them respond in real time.

Our keynote speakers, while still under wraps for now, will have you on the edge of your seats (or couches…no judgement here!).

Networking and entertainment

You read that correctly. We will have live entertainment that'll have you glued to the screen. And you’ll be able to network with SAS experts and peers alike. But you don’t have to wait until the conference begins to network, the SAS Global Forum virtual community is up and running. Join the group to start engaging with other attendees, and maybe take a guess or two at who the live entertainment might be.

A big thank you

We are working hard to bring you the best conference possible, but this isn’t a one-woman show. It takes a team, so I would like to introduce and thank the conference teams for 2021. The Content Advisory Team ensures the Users Program sessions meet the needs of our diverse global audience. The Content Delivery Team ensures that conference presenters and authors have the tools and resources needed to provide high-quality presentations and papers. And, finally, the SAS Advisers help us in a multitude of ways. Thank you all for your time and effort so far!

Registration opens in April, so stay tuned for that announcement. I look forward to “seeing” you all in May.

What makes SAS Global Forum great? was published on SAS Users.

1月 272021
 

SASensei logoSAS offers myriad ways to level up your SAS skills (scroll to the bottom to see a list of SAS-provided learning paths and channels). In this post, I introduce you to SASensei, an independent, third-party online SAS learning resource that I enjoy a lot.

Learning: dull or fun?

Learning is not always associated with fun. Sometimes it feels difficult and exhausting. New concepts often contradict our prior knowledge and experience, compelling us to rethink, adjust, change and adapt to new paradigms.

Learning new ideas, skills and technologies can be intimidating, challenging and demanding. While learning, you are stretching out of your comfort zone. But that feeling is only transient. As a matter of attitude, learning is not about pushing yourself out of your comfort zone, it’s about expanding your comfort zone. And that is long lasting. The more you learn, the more comfortable and self-confident you become.

Learning does not have to be tedious. Look at pre-school kids. They learn basic life skills like walking (rolling, crawling), talking (in one or more languages), asking questions (a lot) – all without taking classes, just through their natural curiosity and ... playing games.

What is SASensei? Gamified SAS learning

When I first discovered the SASensei online SAS learning game/application I was pleasantly surprised by its non-traditional approach to learning such a serious and well-established platform as SAS.

As stated on its website, “Sasensei is a question based learning system. You must demonstrate your command of SAS® to earn Tokens - which should be wisely invested, to enable you to unlock new levels within the game...”

The following screenshot shows the main page of the SASensei website that displays a dashboard of the top players (they call it leaderboard). You can filter it geographically - by Country, Continent, or World, as well as by the timeline – by Past Month, Past Year, or All Time.

SASensei leaderboard

Privacy or prominence

Users have full control of their privacy or prominence. As you can see in the screenshot above, registered users are displayed by their screen names. This allows the users to either remain anonymous by selecting some fictitious obscure screen name or use their real name. Users can change their screen name at any time.

Rules of the game

In this blog post I provide just an overview of the main functionality and features of the SASensei learning platform. For detailed rules of the game, see SASensei Documentation.

Play and learn

Users are offered a variety of learning activities:

  • Viewing, reviewing and submitting SAS-learning flashcards;
  • Playing, reviewing and submitting questions by different SAS-related topics;
  • Taking and creating public, private, multi-player and custom quizzes;
  • Providing feedback on questions and flashcards by voting and commenting.

Users can challenge themselves by delving into different topics. Your successes and failures will provide you an honest and objective estimation of your SAS strengths as well as weaknesses. A healthy competition with other users encourages you to learn more and hone your SAS skills. When you fail a question, you can review the explanation of the correct answer and thus learn why you failed and acquire new knowledge, tips and tricks quickly and efficiently.

Invest, score, win and build a reputation

To play you will need to earn and spend tokens which are essentially the game’s currency. To motivate you further, you also earn reputation points, which is your ultimate score, a level of achievement in demonstrating SAS skills. Your reputation score is prominently displayed in your public profile. As you progress in the game and your reputation grows, additional functionality unlocks and becomes available to you. Your reputation score determines your SASensei standing level which is derived from those used in martial arts:

Sasensei title Sasamurai title

  • White Belt (new players)
  • Yellow Belt (reputation ≥ 50)
  • Green Belt (reputation ≥ 100)
  • Black Belt (reputation ≥ 200)
  • Sasamurai (reputation ≥ 500)
  • Assassin (reputation ≥ 1000)
  • Sasensei (reputation ≥ 5000)

Sample SASensei question

When you play a question, you select a topic, and then you are presented with a randomly selected multiple-choice question of a specified time limit (30, 60, 90 or 120 seconds). Here is a sample of such question:

Question:

What is wrong with the following LIBNAME statement?
libname fruits (apples oranges tomatoes);

Answers:

  • Incorrect syntax
  • You cannot mix apples and oranges in LIBNAME statement
  • Nothing is wrong, valid LIBNAME statement
  • Tomatoes are not fruits, therefore the statement is not correct

Correct answer:

Nothing is wrong, valid LIBNAME statement

Explanation:

According to Combine and conquer with SAS for examples of usage.

Question
Try tackling a question on your own in the SASensei environment to get real life experience: Sample Question.

Take a SASensei sample quiz

There are various quizzes available at SASensei: public quizzes, multiplayer quiz games, private quizzes (tests) for students.

A public quiz contains 12 questions with a total time cap of 12 minutes, and costs eight tokens to play. You can choose a single topic (sas statements, sas macro, procedures, options, etc.), and if you pass (75% or more) you get 12 tokens back, plus 20 point to your reputation. If you get 100%, you get 30 reputation points plus Top Student badge. A count of passed sessions (by topic) is displayed on your public profile.

QuizAlthough public quizzes are unlocked at the SASamurai level, for the readers of this blog, I have created a special custom quiz sample so you can experience it firsthand right here, right now. Just click on this link, Sample Quiz, register, and enjoy your ride.

See you at the top of the SASensei dashboard!

Credit

Big THANKS to Allan Bowe (United Kingdom) – SAS innovator and entrepreneur who created and founded SASensei learning platform.

Other SAS learning resources

Game on! SASensei: a fun way to learn SAS was published on SAS Users.

12月 172020
 

There’s nothing worse than being in the middle of a task and getting stuck. Being able to find quick tips and tricks to help you solve the task at hand, or simply entertain your curiosity, is key to maintaining your efficiency and building everyday skills. But how do you get quick information that’s ALSO engaging? By adding some personality to traditionally routine tutorials, you can learn and may even have fun at the same time. Cue the SAS Users YouTube channel.

With more than 50 videos that show personality published to-date and over 10,000 hours watched, there’s no shortage of learning going on. Our team of experts love to share their knowledge and passion (with personal flavor!) to give you solutions to those everyday tasks.

What better way to round out the year than provide a roundup of our most popular videos from 2020? Check out these crowd favorites:

Most viewed

  1. How to convert character to numeric in SAS
  2. How to import data from Excel to SAS
  3. How to export SAS data to Excel

Most hours watched

  1. How to import data from Excel to SAS
  2. How to convert character to numeric in SAS
  3. Simple Linear Regression in SAS
  4. How to export SAS data to Excel
  5. How to Create Macro Variables and Use Macro Functions
  6. The SAS Exam Experience | See a Performance-Based Question in Action
  7. How it Import CSV files into SAS
  8. SAS Certification Exam: 4 tips for success
  9. SAS Date Functions FAQs
  10. Merging Data Sets in SAS Using SQL

Latest hits

  1. Combining Data in SAS: DATA Step vs SQL
  2. How to Concatenate Values in SAS
  3. How to Market to Customers Based on Online Behavior
  4. How to Plan an Optimal Tour of London Using Network Optimization
  5. Multiple Linear Regression in SAS
  6. How to Build Customized Object Detection Models

Looking forward to 2021

We’ve got you covered! SAS will continue to publish videos throughout 2021. Subscribe now to the SAS Users YouTube channel, so you can be notified when we’re publishing new videos. Be on the lookout for some of the following topics:

  • Transforming variables in SAS
  • Tips for working with SAS Technical Support
  • How to use Git with SAS

2020 roundup: SAS Users YouTube channel how to tutorials was published on SAS Users.