Predictive analytics, five years later

This week I am at Predictive Analytics World Boston to learn from leaders in statistics, data mining, and related analytical disciplines. Much has changed since I attended the inaugural PAW conference over five years ago but many of the core challenges and techniques remain the same.

While it’s clear that demand for predictive analytics has increased significantly over the last several years—measured in part by how much conference attendance has grown!—there’s still a long way to go in terms of real-world adoption. While relatively few organizations are applying predictive analytics to their business my sense is that analytics is being applied to new areas of business—for example, workforce analytics and HR—that previously were managed but not measured.

Some highlights from this week’s conference:

  • The expert panel (Necessary Skills of a Quant) touched on both the technical and soft skills needed to succeed in analytics, citing data from McKinsey on analytical talent shortages: while there’s a 200K gap in analytical experts, there’s an even larger gap of 1.5M managers who can understand and act on analytics.
  • John Elder discussed the power and peril of statistics, recommending a method he calls target shuffling to better understand and communicate the predictive power of models versus traditional techniques while also reducing the risk of false positives.
  • Jack Levis from UPS stressed the importance of analytics in the context of business value, describing his organization’s growth from purely descriptive and retrospective analytics—which still have value and a role to play—to prescriptive analytics like their ORION system to more efficiently route drivers and packages. This growth is summarized in the chart below and Jack overlaid data showing adoption of each flavor decreasing from 70% to 30% to 16% to 3% as you work toward prescriptive.