Hyperparameters

8月 112017
 

How can you tell if your marketing is working? How can you determine the cost and return of your campaigns? How can you decide what to do next? An effective way to answer these questions is to monitor a set of key performance indicators, or KPIs.

KPIs are the basic statistics that give you a clear idea of how your website (or app) is performing. KPIs vary by predetermined business objectives, and measure progress towards those specific objectives. In the famous words of Avinash Kaushik, KPIs should be:

  • Uncomplex.
  • Relevant.
  • Timely.
  • Instantly useful.

An example that fits this description, with applicability to profit, nonprofit, and e-commerce business models, would be the almighty conversion rate.  In digital analytics this metric is interpreted as the proportion of visitors to a website or app who take action to go beyond a casual content view or site visit, as a result of subtle or direct requests from marketers, advertisers, and content creators.

{\mathrm {Conversion\ rate}}={\frac {{\mathrm {Number\ of\ Goal\ Achievements}}}{{\mathrm {Visitors}}}}

Although successful conversions can be defined differently based on your use case, it is easy to see why this KPI is uncomplex, relevant, timely, and useful. We can even splinter this metric into two types:

Macro conversion – Someone completes an action that is important to your business (like making you some money).

Micro conversion – An indicator that a visitor is moving towards a macro conversion (like progressing through a multi-step sales funnel to eventually make you some money)

Regardless of the conversion type, I have always found that reporting on this KPI is a popular request for analysts from middle management and executives. However, it isn't difficult to anticipate what is coming next from the most important person in your business world:

"How can we improve our conversion rate going forward?"

You can report, slice, dice, and segment away in your web analytics platform, but needles in haystacks are not easily discovered unless we adapt. I know change can be difficult, but allow me to make the case for machine learning and hyperparameters within the discipline of digital analytics. A trendy subject for some, a scary subject for others, but my intent is to lend a practitioner's viewpoint. Analytical decision trees are an excellent way to begin because of their frequent usage within marketing applications, primarily due to their approachability, and ease of interpretation.

Whether your use case is for supervised segmentation, or propensity scoring, this form of predictive analytics can be labeled as machine learning due to algorithm's approach to analyzing data. Have you ever researched how trees actually learn before arriving to a final result? It's beautiful math. However, it doesn't end there. We are living in a moment where more sophisticated machine learning algorithms have emerged that can comparatively increase predictive accuracy, precision, and most importantly – marketing-centric KPIs, while being just as easy to construct.

Using the same data inputs across different analysis types like Forests, Gradient Boosting, and Neural Networks, analysts can compare model fit statistics to determine which approach will have the most meaningful impact on your organization's objectives. Terms like cumulative lift or misclassification may not mean much to you, but they are the keys to selecting the math that best answers how conversion rate can be improved by transparently disclosing accurate views of variable importance.

So is that it? I can just drag and drop my way through the world of visual analytics to optimize against KPIs. Well, there is a tradeoff to discuss here. For some organizations, simply using a machine learning algorithm enabled by an easy-to-use software interface will help improve conversion rate tactics on a mobile app screen experience as compared to not using an analytic method. But an algorithm cannot be expected to perform well as a one size fits all approach for every type of business problem. It is a reasonable question to ask oneself if opportunity is being left on the table to motivate analysts to refine the math to the use case. Learning to improve how an algorithm arrives at a final result should not be scary because it can get a little technical. It's actually quite the opposite, and I love learning how machine learning can be elegant. This is why I want to talk about hyperparameters!

Anyone who has ever built a predictive model understands the iterative nature of adjusting various property settings of an algorithm in an effort to optimize the analysis results. As we endlessly try to improve the predictive accuracy, the process becomes painfully repetitive and manual. Due to the typical length of time an analyst can spend on this task alone - from hours, days, or longer - the approach defies our ability as humans to practically arrive at an optimized final solution. Sometimes referred to as auto tuning, hyperparameters address this issue by exploring different combinations of algorithm options, training a model for each option in an effort to find the best model. Imagine running 1000s of iterations of a website conversion propensity model across different property threshold ranges in a single execution. As a result, these models can improve significantly across important fit statistics that relate directly to your KPIs.

At the end of running an analysis with hyperparameters, the best recipe will be identified. Just like any other modeling project, the ability to action off of the insight is no different, from traditional model score code to next-best-action recommendations infused into your mobile app's personalization technology. That's genuinely exciting, courtesy of recent innovations in distributed analytical engines with feature-rich building blocks for machine-learning activities.

If the subject of hyperparameters is new to you, I encourage you to watch this short video.

This will be one of the main themes of my presentations at Analytics Experience 2017 in Washington DC. Using digital data collected by SAS Customer Intelligence 360 and analyzing it with SAS Visual Data Mining & Machine Learning on VIYA, I want to share the excitement I am feeling about digital intelligence and predictive personalization. I hope you'll consider joining the SAS family for an awesome agenda between September 18th-20th in our nation's capital.

Hyperparameters, digital analytics, and key performance indicators was published on Customer Intelligence Blog.