To get a high-performing analytics team producing insights that matter, you need great people, powerful software and a culture of experimentation and innovation. Three simple ingredients, but getting there is far from easy. In this post, I’d like to get you thinking about how to organize for success by building [...]
SAS® Viya™ 3.1 represents the third generation of high performance computing from SAS. Our journey started a long time ago and, along the way, we have introduced a number of high performance technologies into the SAS software platform:
- In-Database processing where SAS data quality and analytical processing occur within the data source, minimizing data movement and leveraging the native language of the data source.
- Grid computing to distribute processing over a number of computing nodes in a cluster.
- High Performance Analytics where in-memory calculations are processed across the nodes of a cluster including High Performance Risk and High Performance Data Mining.
- In-memory Visual Analytics and Visual Statistics powered by the SAS LASR analytic server.
Introducing Cloud Analytic Services (CAS)
SAS Viya introduces Cloud Analytic Services (CAS) and continues this story of high performance computing. CAS is the runtime engine and microservices environment for data management and analytics in SAS Viya and introduces some new and interesting innovations for customers. CAS is an in-memory technology and is designed for scale and speed. Whilst it can be set up on a single machine, it is more commonly deployed across a number of nodes in a cluster of computers for massively parallel processing (MPP). The parallelism is further increased when we consider using all the cores within each node of the cluster for multi-threaded, analytic workload execution. In a MPP environment, just because there are a number of nodes, it doesn’t mean that using all of them is always the most efficient for analytic processing. CAS maintains node-to-node communication in the cluster and uses an internal algorithm to determine the optimal distribution and number of nodes to run a given process.
However, processing in-memory can be expensive, so what happens if your data doesn’t fit into memory? Well CAS, has that covered. CAS will automatically spill data to disk in such a way that only the data that are required for processing are loaded into the memory of the system. The rest of the data are memory-mapped to the filesystem in an efficient way for loading into memory when required. This way of working means that CAS can handle data that are larger than the available memory that has been assigned.
The CAS in-memory engine is made up of a number of components - namely the CAS controller and, in an MPP distributed environment, CAS worker nodes. Depending on your deployment architecture and data sources, data can be read into CAS either in serial or parallel.
What about resilience to data loss if a node in an MPP cluster becomes unavailable? Well CAS has that covered too. CAS maintains a replicate of the data within the environment. The number of replicates can be configured but the default is to maintain one extra copy of the data within the environment. This is done efficiently by having the replicate data blocks cached to disk as opposed to consuming resident memory.
One of the most interesting developments with the introduction of CAS is the way that an end user can interact with SAS Viya. CAS actions are a new programming construct and with CAS, if you are a Python, Java, SAS or Lua developer you can communicate with CAS using an interactive computing environment such as a Jupyter Notebook. One of the benefits of this is that a Python developer, for example, can utilize SAS analytics on a high performance, in-memory distributed architecture, all from their Python programming interface. In addition, we have introduced open REST APIs which means you can call native CAS actions and submit code to the CAS server directly from a Web application or other programs written in any language that supports REST.
Whilst CAS represents the most recent step in our high performance journey, SAS Viya does not replace SAS 9. These two platforms can co-exist, even on the same hardware, and indeed can communicate with one another to leverage the full range of technology and innovations from SAS. To find out more about CAS, take a look at the early preview trial. Or, if you would like to explore the capabilities of SAS Viya with respect to your current environment and business objectives speak to your local SAS representative about arranging a ‘Path to SAS Viya workshop’ with SAS.
Who cares about sports and data? Not just athletes, coaches and fans. It turns out that many companies outside of sporting organisations are also associated with the sports industry. For example, financial services organisations are actively involved in sports sponsorships. Retailers sell fan merchandise. Telcos build social engagement strategies around […]
It was John Allen Paulos who said, “Data, data everywhere, but not a thought to think.” That rings true more than ever before. Companies are struggling with the deluge of data coming at them from multiple channels. But traditional data channels are just the beginning. Companies also are facing an […]
I moved to Australia from Belgium two months ago for a short-term assignment. I am very concerned by the exchange rate. My dollars have lost over 15% of their value in euros and I share my frustration around me. People tell me, "Just wait, it cannot stay so low, the rate will go up again". So I keep waiting.
Actually, this intuition is against all economic principles and historic observation. If there was information on the market indicating the value of the dollar is underestimated and will go up again, it would be immediately incorporated in its price. Making the intuitive assumption that the dollar will come back to its historical value is like buying a lottery ticket. You are only buying a dream. It has the exact same likelihood to go up or to go down again.
Yet, I am taking the decision to wait. I don’t need the money now, and I buy the dream that the value of the dollar will rise again, with an eye on the next iPhone I could buy with it.
Many organizations have to take the exact same decision as me.
If you were an executive in a company with Australian dollars in the bank, shareholders in Europe and no local investments in sight, would you wait? Would you consider it a rational decision? Can you afford the same subjectivity as me? Can you buy the dream the value will rise again?
Doing business is taking decisions and companies cannot make subjective decisions. All business decisions must be treated with a maximum level of objectivity incorporating all available information. Whether these decisions are strategic, tactical or operational is irrelevant to this principle. So, organizations cannot buy the dream the dollars will rise again.
And it is a challenge for many organizations to get rid of that subjective bias. They must engineer their processes in order to take optimal decisions, requiring the right information at the right time. This information consists of analytical insights, patterns, sentiments and anomalies hidden somewhere in data. It will require to read, store, crunch and process tons of records, texts, trades, news and databases at extreme speed to meet the decision timeframe.
This process must start by defining the decisions that the company want to objectivize. The price of an airline ticket, the purchase of additional ships for oil prospection or the insurance premium of a policy holder.
These decisions define the insights needed for improvements. It may be simply surfacing the most recent update about a customer. It may require some probabilistic computations of oil prices or sentiment analysis based on previous phone conversations.
And only those expected insights will ensure the organization put up the right data requirements. What data is needed to provide these insights? Comment fields, sonar echoes, phone calls, pictures? Is the data to be stored or only processed? Where and how does it have to be stored? Etc.
Organizations must aim at decreasing the subjectivity of decisions and this process must drive the requirements for technologies. As such, decisions will define the analytical strategy, and analytics will define the data strategy.
After doing some recent research with IDC®, I got to thinking again about the reasons that organizations of all sizes in all industries are so slow at adopting analytics as part of their ‘business as usual’ operations.
While I have no hard statistics on who is and who isn’t adopting analytics, the research shows that organizations that do leverage analytics are more successful on average than those that don’t. What we need is a new analytics experience, an experience where organizations can:
- Make confident decisions
- Analyze all their data where it exists
- Seize new opportunities with analytics
- Remove restrictions for data scientists
IDC states that “50.6% of Asia Pacific enterprises want to monetize their data in the next 18 months”. Are you one of them or are you going to let your competition get the jump on you?
Big data (or more specifically how to actually gain some sort of competitive advantage from it) is top of mind for forward-looking businesses.
Our research with IDC gives us a few clues on where to head when it comes to the monetization discussion.
In the recent Monetizing Your Data infographic (PDF) created by IDC and SAS, three key approaches to monetizing big data emerged:
- Data decisioning, where insights derived from big data can be used to enhance business processes;
- Data products, where new innovative data products can be created and sold;
- Data partnerships, where organizations sell or share core analytics capabilities with partners.
Organizations that adopt and combine all three key approaches to leverage analytics are twice as likely to outperform their peers1.
If you’re looking to truly create value from the stores of data you have then you need to look at deploying analytics.
1 IDC APEJ Big Data MaturityScape Benchmark Survey 2014 (n=1255) IDC APEJ Big Data Pulse 2014 (n = 854)
Are you missing out when it comes to data monetization? was published on Left of the Date Line.
Insurance relies on the ability to predict future claims or loss exposure based on historical information and experience. However, insurers face an uncertain future due to spiraling operational costs, escalating regulatory pressures, increasing competition and greater customer expectations. More than ever, insurance companies need to optimize their business processes. But […]
In my last blog post, I introduced SAS Visual Analytics for UN Comtrade, which helps anyone glean answers from the most comprehensive collection of international trade data in the world. I’d like to share some of what we learned in the development process as it pertains to big data, high […]
The post Big data lessons learned from visualizing 27 years of international trade data appeared first on SAS Voices.