Being a SAS consultant is about solving problems. In our day-to-day work we solve myriads of all sorts of problems – technical problems, data problems, programming problems, optimization problems – you name it. And in the grand scheme of things we solve business problems.
But without a well-defined business problem, all our problem-solving efforts become irrelevant. What is the benefit of optimizing a SAS program so it runs 2 seconds instead of 20? Well, you can claim a ten-fold improvement, but “so what?” if that program is intended to run just once! Given the number of hours you spent on such an optimization, you were definitely solving the wrong problem.
The ice cream maker
There was one event early in my life that made an unforgettable impression on me and forever changed my problem-solving mindset. When I was a teenager, my father and I bought Mom a present for her birthday – an ice cream maker – a little bowl with a slow electrical mixer that you place into a fridge. Yes, it was a cleverly self-serving gift on our part, but, hey, that was what she really wanted!
Unfortunately for all of us, the gift turned out to be totally useless, as our old refrigerator had such a construction that its door’s rigid rubber-covered-metal gasket would just cut off the ice cream maker’s wire, which was supposed to be plugged in outside of the fridge. Obviously, we were all pretty upset about it. My father was contemplating making a hole somewhere in the refrigerator to accommodate the wire. Mom pretended she never really cared about the ice cream maker in the first place. The whole gift situation was just terrible.
A couple weeks later we discovered a lottery ticket amongst the ice cream maker documentation – you know, one of those worthless no-chance promotional lottery tickets no one ever wins with.
Never in my life have I won anything with a lottery… except that time. No, we didn’t win a cash jackpot. Nor had we won a book titled “Converting an ice cream maker into a wireless device for dummies.” The win was much more meaningful. We had won a new refrigerator. It had a nice, soft rubber gasket around its door that perfectly allowed for the ice cream maker’s wire. The beauty of this win was not just that it was the perfect solution to our problem, but that, in fact, the solution (the winning lottery ticket) came packaged with the source of our problem (the ice cream maker)!
I can’t tell you how much that lucky event shaped my professional and overall attitude. From then on, whenever I faced a problem, I would always search for that little “lottery ticket.” I know it’s there; I just need to find it, and that will turn the problem on its head. From then on, I saw the opportunity in any problem, and the satisfaction of solving the right one.
Problem-driven or purpose-driven solution?
The customer was straight to the point. They had a huge SAS data table that needed to be updated on a daily basis, at about 10 am when all the primary data sources were finished refreshing. The problem was that updating that SAS table was taking about 30 minutes, during the entirety of which the table was not available to their business analysts, as it was locked by the updating process. This created a 30 minute downtime for the business analysts. The customer formulated the problem as “too long a data table update,” with the suggested solution being “to optimize the process of updating the table”.
Sounds logical, doesn’t it? But we should never take for granted the customer’s vision of the problem, especially when it’s accompanied by a suggested solution. In most cases, this will lead to solving the wrong problem. Just keep your mind open and help your customers to keep their minds open too.
Problems do not exist by themselves without a purpose, a goal. The customer is the ultimate authority of setting those goals and purposes. Get that purpose to be articulated first before jumping to formulating a problem. Picture this dialogue:
- What is your goal - to minimize the data table refresh time or to minimize the business analysts’ downtime?
- Aren’t those the same?
- Let’s assume not.
- But how?
- Let’s not ask “how” just yet. If I reduce your business analysts’ downtime to zero, would you care if your data table is being updated 30 minutes or 15 minutes?
- No. Well, then our goal is to minimize the business analysts’ downtime.
We just made a huge leap – from an imaginary problem to a purpose. In fact, the duration of the table update is not important as long as the analysts’ downtime is minimized. So the real problem is the downtime, which we need to reduce or, ideally, eliminate.
When the real problem is well-defined, finding a solution is a matter of practice. As Albert Einstein once said, “If I were given one hour to save the planet, I would spend 59 minutes defining the problem and one minute resolving it.” It makes a lot of sense, since spending all your time on solving the wrong problem does not move you an iota in the right direction.
There are many proven methods and techniques of creative problem-solving. One of them is using analogies. To arrive at “how” (the solution), let’s consider the following analogy. Suppose we want to renovate a restaurant but we don’t want to stop its operations and lose its patrons during that renovation. What would we do? An ideal solution would be to build a new restaurant building next to the old one. After we finish the construction, we would simply take the sign from the old building and affix it to the new one. Problem solved. We can demolish or reuse the old building.
For SAS data tables that would translate into this. Let’s say our data table is called ANALYTICS. Instead of rebuilding it “in place,” we build a new table, let’s call it ANALYTICS_NEW. We can spend 30 minutes or 15 minutes doing this – it does not matter. After we finish table-building we can just make two quick renames:
- Rename ANALYTICS table to ANALYTICS_OLD;
- Rename ANALYTICS_NEW table to ANALYTICS.
Using CHANGE statement of PROC DATASETS these 2 renames would take about 0 seconds – that is our new downtime. After that we can delete the ANALYTICS_OLD table. Looks like we just found an ideal purpose-driven solution to effectively eliminate (not just reduce) the downtime.
I can foresee some inquisitive minds asking why we can’t do the table update in place without locking it out by using SAS/SHARE software. Yes, we can, and that would be a “refrigerator” solution to an “ice-cream maker” problem. Unfortunately, not all the refrigerators are won in a lottery.