text analytics

5月 252012
Wonderful news!  My Viennese  colleague, Gerhard Svolba – has completed his new book, “Data Quality for Analytics Using SAS”, now available from Amazon, as well as via the SAS Bookstore.  Cheers!  With celebrations all around. In the text Gerhard addresses the essential fact that analytics has special data   requirements – [...]
4月 272012

As always, SAS Global Forum holds a wealth of inspiration. The conversations that I have with you guys while I'm there almost always start with, "I just heard/saw/read the coolest thing. I can't wait to get home and get started using this!" For those of you who missed this year's big dance, we missed you! So, my colleagues and I have tried to collect as much of the inspiration and spirit as possible. We'll be putting it on the SAS Software YouTube channel, and our social channels. You can also read all of the papers online.

Today's fraudsters always seem to be one step ahead of investigators, so John York, Doris Wong and Dan Zaratsian from SAS wrote Becoming the Smartest Guys in the Room: An Analysis of the Enron Emails Using an Integration of Text Analytics and Case Management. They wrote the paper to show that fraud investigators can gain strong advantage by combining text analytics with case management software.

Here's some additional insight from John York in an interview with Anna Brown from Inside SAS Global Forum.


tags: case management, Enron, Friday's Innovation Inspiration, Inside SAS Global Forum, papers & presentations, SAS Global Forum, text analytics
4月 202012
Most people my age were obsessed with N*SYNC or the Denver Broncos when we were teenagers. I, however, was really into Shakespeare. Three different editions of Hamlet hold places of honor on my bookshelf. Surrounding them are hardbacks on Shakespeare's contributions to the vernacular and management style. In high school [...]
4月 032012

What do your online conversations really say? Let text analytics tell the storyLet us start with a brief text analytics history lesson.  Australia’s first postal services began with the early settlers in 1809 - communication was hard, and they would wait for mail for months on end.  Moving forward in time (approximately four decades), recognising the communication needs of people became the focus.

This is when the post office took control of what was the most modern means of communicating - the telegraph.  There seem to be no public records available on the number of messages created and delivered at the time, however I am sure I can count on one hand the telegraphs delivered in a day in the 1850s. I am also guessing that the content of the communication expressed sentiment,  historical events, current happenings and future wishes in approximately eight hand-written pages or a thousand plus words.  The level of detail allowed the reader to interpret the meaning and context.

Fast forward 200 years to 2012!  BOOM!
My neighbour, George, is also a postmaster in some ways - in that he delivers communications from  his home. For George is a 'serial tweeter' and he is not alone.  In March 2012, Twitter announced that it had 140 million active users, sending 340 million tweets per day. That is a lot of ‘letters’ – 140 characters at a time – being sent worldwide to anyone and everyone every second of the day. The ‘Noughties’ version of the pen pal. There is even a new language that has its roots in Tweets and text messages: 'Tweetish' … LOL, OMG, think I cre8ted a nu word – SMS speak is now so pervasive (used in chat, on Twitter, in SMS messages) that we even have the SMS Dictionary.

If a picture tells a thousand words, do a thousand words give us a picture?
With the millions of words communicated in text conversation today, we can analyse these words and phrases to provide a good understanding of the hot topics of discussion, as well as society’s sentiment, from all around the world.

Analysis of social media using SAS shows increases in chatter about certain topics that are leading and lagging indicators of a spike in unemployment.

Analysis of social media using SAS shows increases in chatter about certain topics that are leading and lagging indicators of a spike in unemployment.

In a unique project recently, SAS teamed up with the United Nations Global Pulse and partnered on a research project entitled ‘Unemployment through the Lens of Social Media’.

This project investigates how social media and online user-generated content can be used to enrich the understanding of the changing job conditions in the US and Ireland by analyzing the moods and topics present in unemployment-related conversations from the open social web and relating them to official unemployment statistics.

It is fascinating research and I recommend you take a look – we have had a lot of interest across Asia in this project. People today are talking much more than they ever did and to everyone in the world about everything in the world.  The next steps are to make sense of the data and turn it into information.

Why is this so important?  Marketing, fraud specialists, risk advisors, journalists, and advertising agencies could all use text analytics to gain competitive advantage and understand the consumer voice.  If my health insurance company analysed my last conversation I had with them a week ago, they would be worried.  My last words to them were “It’s taking you three days to issue me a new policy quote.  I am not happy with your pricing on the policy package, so I will look into other insurers.  Goodbye!”

Question: Think about the online conversations you have had recently. What would sentiment analysis reveal?

tags: sentiment, social media, text analytics, texting, words
2月 092012
Wow what a game! But for people like me, of course, the ads are the real super bowl snacks.  I spent a lot of time thinking about the best way to analyze the super bowl ads. It struck me that most ads are trying to make a statement – to [...]
1月 142012
It's true. "Big data" can be a problem and an opportunity. Many organizations have struggled to manage, much less profit from, the deluge. In 2012, look for big data to spur demand for big data analytics. New developments in high-performance computing as well as increased demand for visualization and text [...]