Chris Hemedinger

5月 032023

"When will we have SAS Global Forum in person again?"

In-person connections are great; arm-wrestling is optional
This is the top question I hear from SAS users: in person, on the community, in emails and social media. After a long hiatus from in-person events, these people are seeking the return of SAS connections for all of these reasons:

  • They have SAS tips and experiences that they want to share with their peers.
  • They miss their SAS friends -- fellow users and SAS staff who they have come to know at past conferences.
  • They are a little fatigued (maybe more than a little) with virtual events -- nothing beats an in-person connection!

The great news is that we will have a live, in-person conference for SAS users in September 2023. It's called SAS Explore. Thousands of you have attended the virtual version of SAS Explore in 2022; now it's time to plan to meet in person (Sept 11-14 in Las Vegas)!

SAS Explore is for SAS users

SAS Explore 2023 is like SAS Global Forum events of the past in the most important ways:

  • This event is focused on users: data scientists, developers, SAS programmers. If your hands are on the keyboard making SAS work for your organization, this conference is for you.
  • This conference needs your contributions! Many of you have experiences, expertise and tips that your fellow SAS users can learn from.

Like past conferences, the call for content invites your presentations about any topic of interest to your fellow SAS users: SAS programming, analytics and AI, working with open source, APIs, administration, industry solutions and more. And topics that apply to any SAS platform are fair game: SAS 9, SAS Viya, SAS in the cloud or on-premise.

We have a SAS Explore Advisory Board made up of SAS community members (including some past SAS Global Forum leaders) and experts at SAS who will review ALL submissions and help to set the conference agenda. We are eager to see your ideas!

SAS Explore 2023 does have some notable differences when compared to past conferences:

  • This conference is focused on SAS users, and is not combined with an "executive" event. We have a different event called SAS Innovate (happening next week!) that is focused on business leaders and industry networking.
  • Accepted presenters will not be asked to write a conference paper. Like other technology conferences in our industry, we are focused on the sharing of expertise in person. Of course you may have materials to share (presentation deck, code and other assets), and we will host these materials on the SAS Explore hub as we did last year.

For SAS users in the US, there are regional events to consider too! WUSS (Western Users of SAS Software) and SESUG (Southeast SAS Users Group) are both recruiting for content. And we are okay with you submitting the same idea/proposal for multiple conferences: we're all friends here!

Submit your proposals today!

Obviously, we want your content ideas and your attendance for SAS Explore. But what's in it for you (aside from the glory and fame of appearing on the agenda)? Accepted presenters will earn free registration for the event! (You'll still need to get yourself to Las Vegas to present.)

Submit your content proposal here.

Call for content is open until May 31.

Reminder: we are also inviting SAS customers to nominate themselves for SAS Customer Recognition awards! There are several categories to consider, and first-place winners will earn free attendance to SAS Explore 2023 (including registration and travel allowance). See the SAS Customer Recognition Awards site for all of the details.

Bring your expertise to SAS Explore 2023 was published on SAS Users.

3月 312023

The SAS Customer Recognition Awards were created to recognize and reward SAS customers and partners around the globe who help us change the world through analytics. The awards will recognize customers and partners for their stand-out contributions over the past year. Not only do the awards celebrate our customers and partners, but they also highlight all the fantastic work they have accomplished. The winners will be announced at SAS Explore 2023 in Las Vegas, NV and each first-place winner will receive one trip to the event*, a SAS trophy, SAS swag and a social tile to promote their win. We will ask each first-place winner to create a short video to describe their winning submission.

*Please visit the SAS Customer Recognition Awards site for all the program details, rules and to submit an entry. 

Contest details 

The SAS Customer Recognition Awards live on the SAS Support Community, which allows customers to submit photos, documents and videos with their entry and allows us to do online voting. Winners will be chosen based on a combination of public votes and judging panel votes.

Submit Entry Dates: April 3, 2023 – June 2, 2023 

Voting Dates: June 12, 2023 – June 23, 2023 

Panel Judging Dates: June 26, 2023 – June 30, 2023  

Winners Notified: July 24, 2023 

First Place Winners Announced: September 14, 2023 at SAS Explore (complete winner list posted in the SAS Support Community.)

Winners: First place, second place, third place (pending quantity of entries) 

Categories that accept nominations

(Votes by customers combined with scores from a judging panel determine winners.)

Distinguished Partner: Awarded to a SAS partner that exceeded the customer's expectations and was essential in the customer's success.

SAS Analytics Explorers Advocate: Awarded to a SAS customer who is leveraging the SAS Analytics Explorers program to grow their skills, their career and/or their network.

Community Uplift Award: Awarded to a SAS customer who has made an impact in their community at large using SAS products.

Innovative Problem Solver: Awarded to a SAS customer who uses SAS in innovative ways to solve a business problem.

Professional Growth Award: Awarded to a SAS customer who leverages SAS resources for personal professional development or for developing their team.

Rookie of the Year: Awarded to a new SAS customer (within the last 5 years) that is using SAS to make a positive impact at their organization. 

ROI Rock Star: Awarded to a SAS customer demonstrating the greatest business benefit and return on investment (ROI) using SAS products. 

Curious Thinker: Awarded to a SAS customer who embodies curiosity, one of SAS' guiding principles. 

Visual Storyteller: Awarded to a SAS customer who has mastered the art of using data visualizations to help educate their company and tell a story about their data. 


Categories where winners are selected by SAS

Customer Impact Award – Public Sector: Awarded to a public sector customer who has had the most impact through a willingness to share their analytics journey, successes and lessons learned with others 

Customer Impact Award – Private Sector: Awarded to a private sector customer who has had the most impact through a willingness to share their analytics journey, successes and lessons learned with others 

SAS Support Community Hero: A customer that answers a lot of questions in Communities and goes above and beyond helping other users on the SAS Support Community.  

Regional User Group MVP: Awarded to a Regional User Group leader demonstrating a dedicated passion for the success of user group members 

User Feedback Award: Awarded to a customer who provides valuable feedback on SAS products and has been an essential reference for product improvements. Their feedback is high-quality, provides examples and is reliable. 

Valuable Voice: Awarded to a customer who has consistently engaged in our Inner Circle surveys over the last year and provided value feedback on our products and services 

 Please reach out to Wendy Reeves with any questions. 

Announcing the SAS Customer Recognition Awards was published on SAS Users.

2月 212023

Some months ago I saw Scott Hanselman (a technology influencer from Microsoft) wearing a t-shirt with "#include <everyone>;" printed with a rainbow gradient. (Here's a picture of Scott in the shirt.) It's a clever syntactic wordplay that reminds us to value diversity and inclusiveness in our technology communities. The syntax is from C#, a programming language that I've used more than any other in my career -- except for SAS.

I loved the message, and couldn't stop thinking about how the SAS programming language also features an INCLUDE statement (and it predates C# by decades). In the spirit of "SAS coders can have cool geek shirts too", I made this simple design and put it on a shirt:

"Include everyone" shirt
A shirt for SAS nerds

I published the design to a t-shirt marketplace for community creations. I ordered some for myself (a shirt and stickers) and shared the page with my connections on LinkedIn. (The site pays a small commission for each item ordered, but I'm not interested in making money on this endeavor. To offset my anticipated "profit," I made a donation to The Trevor Project.)

Since sharing this on LinkedIn and within SAS, the response has been tremendous. I've had to double my initial donation to stay ahead of the orders, which are still rolling in. Around campus I've been stopped by people who see me wearing the shirt (I have 2 in my rotation); they either to ask me how to get one or they tell me that they already ordered their own (or some for friends and family). I even changed the marketplace entry to allow for kids sizes and onesies -- because people asked.

The importance of inclusiveness in tech and community

I wish I could say "it goes without saying that we value diversity on our SAS community" -- but it doesn't..."go without saying." That's why I think it's important to say it as often as possible, and to wear it on our sleeves (so to speak) when we can.

I'm proud that SAS has a strong diversity & inclusion message and backs it up with action -- internal and external. As a developer, I had a front-row seat to the efforts behind making our software usable by people of many languages and cultures, and to make it accessible to people of all abilities. I've seen our language evolve with our awareness of how important it is to behave "with inclusivity" in mind. And I personally have close friends and family members who need to see us making this effort to lessen the impact of being marginalized in this noisy world that sometimes has tremendous cultural inertia.

Super FREQ and PROC Star - bonus designs

I enjoyed making this design and didn't want to stop. TeePublic, the marketplace site, hosts thousands of designs and it can be quite addicting to shop there. But as many geek themes as I found, there weren't many that are specific to SAS. I decided to fill the gap.

I made two additional designs: "Super FREQ" and "PROC Star". Some of you might recognize these as special ranks that we award on the SAS Community for top contributors. I don't want to take away from the specialness of that, but there is nothing to prevent you, a SAS enthusiast, from proudly declaring your SAS nerd persona by wearing one (or all) of these shirts. I promise to add any additional proceeds to my donations to The Trevor Project.

The complete list of designs (so far) with links:

If you order any or all of these, leave me a comment here or tweet a pic to @cjdinger, or share on LinkedIn and mention me!

Notes and disclaimers

For now, this is my effort and are not official designs from SAS. It's something I wanted to have for myself and was inspired to share. I was careful to steer clear of copyrighted text and images/logos. I'm friendly with our Legal department -- I want to keep it that way.

TeePublic offers a very good satisfaction promise. I've had people tell me they had to exchange their shirts for a different size -- and it was easy to do at no cost. Note that I don't know if it's possible to order from all countries, or what an international shipping cost might be.

If ordering a shirt, I recommend you consider a size larger than your usual, and two sizes up if selecting a "women's cut." To their credit TeePublic advises the same -- I'm just telling you to heed that advice.

The post Inclusive language: in coding, in your wardrobe, in life appeared first on The SAS Dummy.

1月 312023

SAS has released SAS 9.4 Maintenance 8, a major update to SAS 9.4.

Security is the primary focus of the Maintenance 8 update. This release contains updates for many of the third-party technologies that are used by the platform, including the Java runtime environment (JRE) and many of the third-party JAR files. This release also adds support for major releases of supported operating systems, while limiting support for operating systems that are no longer supported by their respective suppliers. My colleague Margaret Crevar has summarized these changes in this SAS Communities post.

As with all SAS maintenance releases, this release "rolls up" the hotfixes and enhancements delivered since the last major update (SAS 9.4 Maintenance 7). Most SAS platform products and solutions have also been updated to remain compatible with this release and take advantage of enhancements. However, there are some products and solutions that will not be available immediately, or that will not deliver support for SAS 9.4 Maintenance 8.

Because SAS 9.4 Maintenance 8 is a major software release for SAS 9.4, it is covered by the SAS Support policy for the "Standard Support" timeframe according to its general availability date: Jan 31, 2023. (Current policy offers Standard Support for 5 years from the GA date.)

While this maintenance release doesn't contain new features, it does demonstrate the commitment of SAS to support users of the SAS 9.4 platform for many years to come. (See "Your platform, your way" in "Your analytics, your way" from Shadi Shahin.) New data and analytics capabilities are delivered in the SAS Viya platform, which offers monthly cadence releases via its continuous delivery model.

For an overview of all product changes and updates in SAS 9.4 Maintenance 8, see the What's New topic in the SAS documentation.

SAS 9.4 Maintenance 8 is available was published on SAS Users.

3月 172022

I created this project as a fun exercise to emulate the popular game Wordle in the SAS language. I was inspired by this story about a GitHub user who implemented Wordle in a Bash shell. (See the Bash script here. Read the comments -- it's an amazing stream of improved versions, takes in different programming languages, and "code golf" to reduce the lines-of-code.)

While developing my SAS solution, I created a Twitter poll to ask how other SAS programmers might approach it.

twitter poll
For me it was always going to be arrays, since that's what I know best. I'd love to be able to dash out a hash object approach or even use SAS/IML, but it would take me too long to wrap my brain around these. The PRX* regex function choice is bit of a feint -- regular expressions seem like a natural fit (pattern matching!), but Wordle play has nuances that makes regex less elegant than you might guess. Prove me wrong!

Two SAS users from Japan, apparently inspired by my poll, each implemented their own games! I've added links to their work at the end of this article.

How to code Wordle in SAS

You can see my complete SAS code for the game here: sascommunities/wordle-sas.

The interesting aspects of my version of the game include:

  • Uses the "official" word lists from New York Times as curated by cfreshman. I found these while examining the Bash script version. I used PROC HTTP to pull this list dynamically.
  • Also verifies guesses as "valid" using the list of allowed guesses, again curated by cfreshman. You know you can't submit just any 5 characters as a guess, right? That's an important part of the game.
  • Uses DATA step array to verify guesses against solution word.
  • Use DATA step object method to create a gridded output of the game board. Credit to my SAS friends in Japan for this idea!

I've added a screenshot of example game play below. This was captured in SAS Studio running in SAS Viya.

Example game with output

Example game play for my Wordle in SAS

How to play Wordle in SAS

To play:

  1. Submit the program in your SAS session. This program should work in PC SAS, SAS OnDemand for Academics, SAS Enterprise Guide, and SAS Viya.

    The program will fetch word lists from GitHub and populate into data sets. It will also define two macros you will use to play the game.

  2. Start a game by running:

    This will select a random word from the word list as the "puzzle" word and store it in a SAS macro variable (don't peek!)
  3. Optionally seed a game with a known word by using and optional 5-character word parameter:

    This will seed the puzzle word ("crane" in this example). It's useful for testing. See a battery of test "sessions" in
  4. Submit a first guess by running:

    This will check the guess against the puzzle word, and it will output a report with the familiar "status" - letters that appear in the word (yellow) and that are in the correct position (green). It will also report if the guess is not a valid guess word, and it won't count that against you as one of your 6 permitted guesses.

Use the %guess() macro (one at a time) to submit additional guesses. The program keeps track of your guesses and when you solve it, it shares the familiar congratulatory message that marks the end of a Wordle session. Ready to play again? Use the %startGame macro to reset.

Start a fresh game using Git functions

If you don't want to look at or copy/paste the game code, you can use Git functions in SAS to bring the program into your SAS session and play. (These Git functions require at least SAS 9.4 Maint 6 or SAS Viya.)

options dlcreatedir;
%let repopath=%sysfunc(getoption(WORK))/wordle-sas;
libname repo "&repopath.";
data _null_;
    rc = gitfn_clone( 
    put 'Git repo cloned ' rc=; 
%include "&repopath./";
/* start a game and submit first guess */


I know that my program could be more efficient...perhaps at the cost of readability. Also it's possible that I have some lingering bugs, although I did quite a bit of testing and bug-fixing along the way. Pro tip: The DATA step debugger (available in SAS Enterprise Guide and in SAS Viya version of SAS Studio) was a very useful tool for me!

Your feedback/improvements are welcome. Feel free to comment here or on the GitHub project.

Wordle games in SAS by other people

SAS users in Japan quickly implemented their own versions of Wordle-like games. Check them out:

The post Programming the Wordle game in SAS appeared first on The SAS Dummy.

7月 072021

When I was a computer science student in the 1980s, our digital alphabet was simple and small. We could express ourselves with the letters A..Z (and lowercase a..z) and numbers (0..9) and a handful of punctuation and symbols. Thanks to the ASCII standard, we could represent any of these characters in a single byte (actually just 7 bits). This allowed for a generous 128 different characters, and we had character slots to spare. (Of course for non-English and especially non-latin characters we had to resort to different code pages...but that was before the Internet forced us to work together. Before Unicode, we lived in a digital Tower of Babel.)

Even with the limited character set, pictorial communication was possible with ASCII through the fun medium of "ASCII art." ASCII art is basically the stone-age version of emojis. For example, consider the shrug emoji: 🤷

Its ASCII-art ancestor is this: ¯\_(ツ)_/¯ While ASCII art currently enjoys a retro renaissance, the emoji has become indispensable in our daily communications.

Emojis before Unicode

Given the ubiquity of emojis in every communication channel, it's sometimes difficult to remember that just a few years ago emoji characters were devised and implemented in vendor-specific offerings. As the sole Android phone user in my house, I remember a time when my iPhone-happy family could express themselves in emojis that I couldn't read in the family group chat. Apple would release new emojis for their users, and then Android (Google) would leap frog with another set of their own fun symbols. But if you weren't trading messages with users of the same technology, then chunks of your text would be lost in translation.

Enter Unicode. A standard system for encoding characters that allows for multiple bytes of storage, Unicode has seemingly endless runway for adding new characters. More importantly, there is a standards body that sets revisions for Unicode characters periodically so everyone can use the same huge alphabet. In 2015, emoji characters were added into Unicode and have been revised steadily with universal agreement.

This standardization has helped to propel emojis as a main component of communication in every channel. Text messages, Twitter threads, Venmo payments, Facebook messages, Slack messages, GitHub comments -- everything accepts emojis. (Emojis are so ingrained and expected that if you send a Venmo payment without using an emoji and just use plain text, it could be interpreted as a slight or at the least as a miscue.)

For more background about emojis, read How Emjois Work (source: How Stuff Works).

Unicode is essential for emojis. In SAS, the use of Unicode is possible by way of UTF-8 encoding. If you work in a modern SAS environment with a diverse set of data, you should already be using ENCODING=UTF8 as your SAS session encoding. If you use SAS OnDemand for Academics (the free environment for any learner), this is already set for you. And SAS Viya offers only UTF-8 -- which makes sense, because it's the best for most data and it's how most apps work these days.

Emojis as data and processing in SAS

Emojis are everywhere, and their presence can enrich (and complicate) the way that we analyze text data. For example, emojis are often useful cues for sentiment (smiley face! laughing-with-tears face! grimace face! poop!). It's not unusual for a text message to be ALL emojis with no "traditional" words.

The website maintains the complete compendium of emojis as defined in the latest standards. They also provide the emoji definitions as data files, which we can easily read into SAS. This program reads all of the data as published and adds features for just the "basic" emojis:

/* MUST be running with ENCODING=UTF8 */
filename raw temp;
proc http
ods escapechar='~';
data emojis (drop=line);
length line $ 1000 codepoint_range $ 45 val_start 8 val_end 8 type $ 30 comments $ 65 saschar $ 20 htmlchar $ 25;
infile raw ;
line = _infile_;
if substr(line,1,1)^='#' and line ^= ' ' then do;
 /* read the raw codepoint value - could be single, a range, or a combo of several */
 codepoint_range = scan(line,1,';');
 /* read the type field */
 type = compress(scan(line,2,';'));
 /* text description of this emoji */
 comments = scan(line,3,'#;');
 /* for those emojis that have a range of values */
 val_start = input(scan(codepoint_range,1,'. '), hex.);
 if find(codepoint_range,'..') > 0 then do;
  val_end = input(scan(codepoint_range,2,'.'), hex.);
 else val_end=val_start;
 if type = "Basic_Emoji" then do;
  saschar = cat('~{Unicode ',scan(codepoint_range,1,' .'),'}');
  htmlchar = cats('<span>&#x',scan(codepoint_range,1,' .'),';</span>');
proc print data=emojis; run;

(As usual, all of the SAS code in this article is available on GitHub.)

The "features" I added include the Unicode representation for an emoji character in SAS, which could then be used in any SAS report in ODS or any graphics produced in the SG procedures. I also added the HTML-encoded representation of the emoji, which uses the form &#xNNNN; where NNNN is the Unicode value for the character. Here's the raw data view:

When you PROC PRINT to an HTML destination, here's the view in the results browser:

In search of structured emoji data

The site can serve up the emoji definitions and codes, but this data isn't exactly ready for use within applications. One could work through the list of emojis (thousands of them!) and tag these with descriptive words and meanings. That could take a long time and to be honest, I'm not sure I could accurately interpret many of the emojis myself. So I began the hunt for data files that had this work already completed.

I found the GitHub/gemoji project, a Ruby-language code repository that contains a structured JSON file that describes a recent collection of emojis. From all of the files in the project, I need only one JSON file. Here's a SAS program that downloads the file with PROC HTTP and reads the data with the JSON libname engine:

filename rawj temp;
 proc http
libname emoji json fileref=rawj;

Upon reading these data, I quickly realized the JSON text contains the actual Unicode character for the emoji, and not the decimal or hex value that we might need for using it later in SAS.

I wanted to convert the emoji character to its numeric code. That's when I discovered the UNICODEC function, which can "decode" the Unicode sequence into its numeric values. (Note that some characters use more than one value in a sequence).

Here's my complete program, which includes some reworking of the tags and aliases attributes so I can have one record per emoji:

filename rawj temp;
 proc http
libname emoji json fileref=rawj;
/* reformat the tags and aliases data for inclusion in a single data set */
data tags;
 length ordinal_root 8 tags $ 60;
 set emoji.tags;
 tags = catx(', ',of tags:);
 keep ordinal_root tags;
data aliases;
 length ordinal_root 8 aliases $ 60;
 set emoji.aliases;
 aliases = catx(', ',of aliases:);
 keep ordinal_root aliases;
/* Join together in one record per emoji */
proc sql;
 create table full_emoji as 
 select  t1.emoji as emoji_char, 
    unicodec(t1.emoji,'esc') as emoji_code, 
    t1.description, t1.category, t1.unicode_version, 
     when t1.skin_tones = 1 then  t1.skin_tones
	 else 0
	end as has_skin_tones,
    t2.tags, t3.aliases
  from emoji.root t1
  left join tags t2 on (t1.ordinal_root = t2.ordinal_root)
  left join aliases t3 on (t1.ordinal_root = t3.ordinal_root)
proc print data=full_emoji; run;

Here's a snippet of the report that includes some of the more interesting sequences:

The diversity and inclusion aspect of emoji glyphs is ever-expanding. For example, consider the emoji for "family":

  • The basic family emoji code is \u0001F46A (👪)
  • But since families come in all shapes and sizes, you can find a family that better represents you. For example, how about "family: man, man, girl, girl"? The code is \u0001F468\u200D\u0001F468\u200D\u0001F467\u200D\u0001F467, which includes the codes for each component "member" all smooshed together with a "zero-width joiner" (ZWJ) code in between (👨‍👨‍👧‍👧)
  • All of the above, but with a dark-skin-tone modifier (\u0001F3FF) for 2 of the family members: \u0001F468\u0001F3FF\u200D\u0001F468\u200D\u0001F467\u200D\u0001F467\u0001F3FF (👨🏿‍👨‍👧‍👧🏿)

Conclusion: Emojis reflect society, and society adapts to emojis

As you might have noticed from that last sequence I shared, a single concept can call for many different emojis. As our society becomes more inclusive around gender, skin color, and differently capable people, emojis are keeping up. Everyone can express the concept in the way that is most meaningful for them. This is just one way that the language of emojis enriches our communication, and in turn our experience feeds back into the process and grows the emoji collection even more.

As emoji-rich data is used for reporting and for training of AI models, it's important for our understanding of emoji context and meaning to keep up with the times. Already we know that emoji use differs among different age generations and across other demographic groups. The use and application of emojis -- separate from the definition of emoji codes -- is yet another dimension to the data.

Our task as data scientists is to bring all of this intelligence and context into the process when we parse, interpret and build training data sets. The mechanics of parsing and producing emoji-rich data is just the start.

If you're encountering emojis in your data and considering them in your reporting and analytics, please let me know how! I'd love to hear from you in the comments.

The post How to work with emojis in SAS appeared first on The SAS Dummy.

1月 182021

Recommended soundtrack for this blog post: Netflix Trip by AJR.

This week's news confirms what I already knew: The Office was the most-streamed television show of 2020. According to reports that I've seen, the show was streamed for 57 billion minutes during this extraordinary year. I'm guessing that's in part because we've all been shut in and working from home; we crave our missing office interactions. We lived vicariously (and perhaps dysfunctionally) through watching Dunder Mifflin staff. But another major factor was the looming deadline of the departure of The Office from Netflix as of January 1, 2021. It was a well-publicized event, so Netflix viewers had to get their binge on while they could.

People in my house are fans of the show, and they account for nearly 6,000 of those 57 billion streaming minutes. I can be this precise (nerd alert!) because I'm in the habit of analyzing our Netflix activity by using SAS. In fact, I can tell you that since late 2017, we've streamed 576 episodes of The Office. We streamed 297 episodes in 2020. (Since the show has only 201 episodes we clearly we have a few repeats in there.)

I built a heatmap that shows the frequency and intensity of our streaming of this popular show. In this graph each row is a month, each square is a day. White squares are Office-free. A square with any red indicates at least one virtual visit with the Scranton crew; the darker the shade, the more episodes streamed during that day. You can see that Sept 15, 2020 was a particular big binge with 17 episodes. (Each episode is about 20-21 minutes, so it's definitely achievable.)

netflix trip through The Office

Heatmap of our household streaming of The Office

How to build the heatmap

To build this heatmap, I started with my Netflix viewing history (downloaded from my Netflix account as CSV files). I filtered to just "The Office (U.S.)" titles, and then merged with a complete "calendar" of dates between late 2017 and the start of 2021. Summarized and merged, the data looks something like this:

With all of the data summarized in this way such that there is only one observation per X and Y value, I can use the HEATMAPPARM statement in PROC SGPLOT to visualize it. (If I needed the procedure to summarize/bin the data for me, I would use the HEATMAP statement. Thanks to Rick Wicklin for this tip!)

proc sgplot data=ofc_viewing;
 title height=2.5 "The Office - a Netflix Journey";
 title2 height=2 "&episodes. episodes streamed on &days. days, over 3 years";
 label Episodes="Episodes per day";
 format monyear monyy7.;
 heatmapparm x=day y=monyear 
   colorresponse=episodes / x2axis
   colormodel=(white  CXfcae91 CXfb6a4a CXde2d26 CXa50f15) ;
 yaxis  minor reverse display=(nolabel) 
 x2axis values=(1 to 31 by 1) 
   display=(nolabel)  ;

You can see the full code -- with all of the data prep -- on my GitHub repository here. You may even run the code in your own SAS environment -- it will fetch my Netflix viewing data from another GitHub location where I've stashed it.

Distribution of Seasons (not "seasonal distribution")

If you examine the heatmap I produced, you can almost see our Office enthusiasm in three different bursts. These relate directly to our 3 children and the moments they discovered the show. First was early 2018 (middle child), then late 2019 (youngest child), then late 2020 (oldest child, now 22 years old, striving to catch up).

The Office ran for 9 seasons, and our kids have their favorite seasons and episodes -- hence the repeated viewings. I used PROC FREQ to show the distribution of episode views across the seasons:

Season 1 is remarkably low for two reasons. First and most importantly, it contains the fewest episodes. Second, many viewers agree that Season 1 is the "cringiest" content, and can be uncomfortable to watch. (This Reddit user leaned into the cringe with his data visualization of "that's what she said" jokes.)

From the data (and from listening to my kids), I know that Season 2 is a favorite. Of the 60 episodes we streamed at least 4 times, 19 of them were in Season 2.

More than streaming, it's an Office lifestyle

Office fandom goes beyond just watching the show. Our kids continue to embrace The Office in other mediums as well. We have t-shirts depicting the memes for "FALSE." and "Schrute Farms." We listen to The Office Ladies podcast, hosted by two stars of the show. In 2018 our daughter's Odyssey of the Mind team created a parody skit based on The Office (a weather-based office named Thunder Mifflin) -- and advanced to world finals.

Rarely does a day go by without some reference to an iconic phrase or life lesson that we gleaned from The Office. We're grateful for the shared experience, and we'll miss our friends from the Dunder Mifflin Paper Company.

The post Visualizing our Netflix Trip through <em>The Office</em> appeared first on The SAS Dummy.

11月 102020

The code and data that drive analytics projects are important assets to the organizations that sponsor them. As such, there is a growing trend to manage these items in the source management systems of record. For most companies these days, that means Git. The specific system might be GitHub Enterprise, GitLab, or Bitbucket -- all platforms that are based on Git.

Many SAS products support direct integration with Git. This includes SAS Studio, SAS Enterprise Guide, and the SAS programming language. (That last one checks a lot of boxes for ways to use Git and SAS together.) While we have good documentation and videos to help you learn about Git and SAS, we often get questions around "best practices" -- what is the best/correct way to organize your SAS projects in Git?

In this article I'll dodge that question, but I'll still try to provide some helpful advice in the process.

Ask the Expert resource: Using SAS® With Git: Bring a DevOps Mindset to Your SAS® Code

Guidelines for managing SAS projects in Git

It’s difficult for us to prescribe exactly how to organize project repositories in source control. Your best approach will depend so much on the type of work, the company organization, and the culture of collaboration. But I can provide some guidance -- mainly things to do and things to avoid -- based on experience.

Do not create one huge repository

DO NOT build one huge repository that contains everything you currently maintain. Your work only grows over time and you'll come to regret/revisit the internal organization of a huge project. Once established, it can be tricky to change the folder structure and organization. If you later try to break a large project into smaller pieces, it can be difficult or impossible to maintain the integrity of source management benefits like file histories and differences.

Design with collaboration in mind

DO NOT organize projects based only on the teams that maintain them. And of course, don't organize projects based on individual team members.

  • Good repo names: risk-adjustment-model, engagement-campaigns
  • Bad repo names: joes-code, claims-dept

All teams reorganize over time, and you don't want to have to reorganize all of your code each time that happens. And code projects change hands, so keep the structure personnel-agnostic if you can. Major refactoring of code can introduce errors, and you don't want to risk that just because you got a new VP or someone changed departments.

Instead, DO organize projects based on function/work that the code accomplishes. Think modular...but don't make projects too granular (or you'll have a million projects). I personally maintain several SAS code projects. The one thing they have in common is that I'm the main contributor -- but I organize them into functional repos that theoretically (oh please oh please) someone else could step in to take over.

The Git view of my YouTube API project in SAS Enterprise Guide

Up with reuse, down with ownership

This might seem a bit communist, but collaboration works best when we don't regard code that we write as "our turf." DO NOT cling to notions of code "ownership." It makes sense for teams/subject-matter experts to have primary responsibility for a project, but systems like Git are designed to help with transparency and collaboration. Be open to another team member suggesting and merging (with review and approval) a change that improves things. GitHub, GitLab, and Bitbucket all support mechanisms for issue tracking and merge requests. These allow changes to be suggested, submitted, revised, and approved in an efficient, transparent way.

DO use source control to enable code reuse. Many teams have foundational "shared code" for standard operations, coded in SAS macros or shared statements. Consider placing these into their own project that other projects and teams can import. You can even use Git functions within SAS to fetch and include this code directly from your Git repository:

/* create a temp folder to hold the shared code */
options dlcreatedir;
%let repoPath = %sysfunc(getoption(WORK))/shared-code;
libname repo "&repoPath.";
libname repo clear;
/* Fetch latest code from Git */
data _null_;
 rc = git_clone( 
options source2;
/* run the code in this session */
%include "&repoPath./";

If you rely on a repository for shared code and components, make sure that tests are in place so changes can be validated and will not break downstream systems. You can even automate tests with continuous integration tools like Jenkins.

DO document how projects relate to each other, dependencies, and prepare guidance for new team members to get started quickly. For most of us, we feel more accountable when we know that our code will be placed in central repositories visible to our peers. It may inspire cleaner code, more complete documentation, and a robust on-boarding process for new team members. Use the Markdown files ( and others) in a repository to keep your documentation close to the code.

My SAS code to check Pagespeed Insights, with documentation

Work with Git features (and not against them)

Once your project files are in a Git repository, you might need to change your way of working so that you aren't going against the grain of Git benefits.

DO NOT work on code changes in a shared directory with multiple team members –- you'll step on each other. The advantage of Git is that it's a distributed workflow and each developer can work with their own copy of the repository, and merge/accept changes from others at their own pace.

DO use Git branching to organize and isolate changes until you are ready to merge them with the main branch. It takes a little bit of learning and practice, but when you adopt a branching approach you'll find it much easier to manage -- it beats keeping multiple copies of your code with slightly different file and folder names to mark "works in progress."

DO consider learning and using Git tools such as Git Bash (command line), Git GUI, and a code IDE like VS Code. These don't replace the SAS-provided coding tools with their Git integration, but they can supplement your workflow and make it easier to manage content among several projects.

Learning more

When you're ready to learn more about working with Git and SAS, we have many webinars, videos, and documentation resources:

The post How to organize your SAS projects in Git appeared first on The SAS Dummy.

9月 062019

A few years ago I shared a method to publish content from SAS to a Slack channel. Since that time, our teams at SAS have gone "all in" on collaboration with Microsoft Office 365, including Microsoft Teams. Microsoft Teams is the Office suite's answer to Slack, and it's not a coincidence that it works in nearly the same way.

The lazy method: send e-mail to the channel

Before I cover the "deluxe" method for sending content to a Microsoft Teams channel, I want to make sure you know that there is a simple method that involves no coding, and no need for APIs. The message experience isn't as nice, but it does the job. You can simply "send e-mail" to the channel. If you're automating output from SAS, it's a simple, well-documented process to send e-mail from a SAS program. (Here's an example from me, using FILENAME EMAIL.)

When you send e-mail to a Microsoft Teams channel, the message notice includes the message subject line, sender, and the first bit of the message content. To see the entire message, you must click on the "View original e-mail" link in the notice. This "downloads" the message to your device so that you can open it with a local tool (such as your e-mail reader, Microsoft Outlook). My team uses this method to receive certain alerts from our platform. Here's an example:

To get the unique e-mail address for a channel, right-click on the channel name and select Get email address. Any message that you send to that e-mail address will be distributed to the team.

Getting started with a Microsoft Teams webhook

In order to provide a richer, more integrated experience with Microsoft Teams, you can publish content using a webhook. A webhook is a REST API endpoint that allows you to post messages and notifications with more control over the appearance and interactive options within the messages. In SAS, you can publish to a webhook by using PROC HTTP.

To get started, you need to add and configure a webhook for your Microsoft Teams channel:

  1. Right-click on the channel name and select Connectors.
  2. Microsoft Teams offers built-in connectors for many different applications. To find the connector for Incoming Webhook, use the search field to narrow the list. Then click Add to add the connector to the channel.
  3. You must grant certain permissions to the connector to interact with your channel. In this case, you need to allow the webhook to send messages and notifications. Review the permissions and click Install.
  4. On the Configuration page, assign a name to this connector and optionally customize the image. The image will be the avatar that's used when the connector posts content to the channel. When you've completed these changes, select Create.
  5. The connector generates a unique (and very long) URL that serves as the REST API endpoint. You can copy the URL from this field -- you will need it later in your SAS program. You can always come back to these configuration settings to change the connector avatar or re-copy the URL.

    At this point, it's a good idea to test that you can publish a basic message from SAS. The "payload" for a Teams message is a JSON-formatted structure, and you can find examples in the Microsoft Teams reference doc. Here's a SAS program that publishes the simplest message. Add your webhook URL and run the code to verify the connector is working for your channel.

    filename resp temp;
    options noquotelenmax;
    proc http
      /* Substitute your webhook URL here */
          "$schema": "",
          "type": "AdaptiveCard",
          "version": "1.0",
          "summary": "Test message from SAS",
          "text": "This message was sent by **SAS**!"

    If successful, this step will post a simple message to your Teams channel:

    Design a message card for Microsoft Teams

    Now that we have the basic plumbing working, it's time to add some bells and whistles. Microsoft Teams calls these notifications "message cards", which are messages that can include interactive features such as images, data, action buttons, and more.

    Designing a simple message

    Microsoft Teams supports a large palette of building blocks (expressed in JSON) to create different card experiences. You can experiment with these cards in the MessageCard Playground that Microsoft hosts. The tool provides templates for several card varieties, and you can edit the JSON definitions to tweak and design your own.

    For one of my use cases, I designed a simple card to show the status of our recommendation engine on SAS Support Communities. (Read this article for more information about how we built and monitor the recommendation engine.) The engine runs as a service and is accessed with its own API. I wanted a periodic "health check" to post to our internal team that would alert us to any problems. Here's the JSON that I used in the MessageCard Playground to design it.

    Much of the JSON is boilerplate for the message. I drew the green blocks to indicate the areas that need to be dynamic -- that is, replaced with values from the real-time API call. Here's what the card looks like when rendered in the Microsoft Teams channel.

    Since my API call to the recommendation engine service creates a data set, I can run that data through PROC JSON to create the JSON segment I need:

    /* reading the results from my API call to the engine */
    libname results json fileref=resp;
    /* Prep a simple name-value data set with the results */
    data segment (keep=name value);
     set results.root;
     name="Score data updated (UTC)";
     value= astore_creation;
     name="Topics scored";
     name="Number of users";
     value= left(num_users);
     name="Process time";
     value= process_time;
    /* use PROC JSON to create the segment */
    filename segment temp;
    proc json out=segment nosastags pretty;
     export segment;

    I shared a version of the complete program on GitHub. It should run as is -- but you would need to supply your own webhook endpoint for a channel that you can publish to.

    Design a message with actions

    I also use Microsoft Teams to share updates about the SAS Software GitHub organization. In a previous article I discussed how I use GitHub APIs to gather data from the GitHub service. Each day, my program summarizes the recent activity from and publishes a message card to the team. Here's an example of a daily update:

    This card is fancier than my first example. I added action buttons that can direct the team members to the internal reports for more details and to the GitHub site itself. I used the Microsoft Teams documentation and the MessageCard Playground to design the experience:

    Messaging apps as part of a DevOps strategy

    Like many organizations, we (SAS) invest a considerable amount of time and energy into gathering metrics and building reports about our operations. However, reports are useful only when the intended audience is tuned in and refers to them regularly. With a small additional step, you can use SAS to bring your most interesting data forward to your team -- automatically.

    Whether you use Microsoft Teams or Slack, automated alerting and updates are a great opportunity to keep your teams informed. Each of these tools offers fit-for-purpose connectors that can tie in with information from other popular operational systems (Salesforce, GitHub, Yammer, JIRA, and many more). For cases where a built-in connector is not available, the webhook approach allows you to easily create your own.

The post How to publish to a Microsoft Teams channel using SAS appeared first on The SAS Dummy.