Measuring marketing value through incrementality experiments

Last week Google released CausalImpact, an open-source r package that both improves and simplifies incrementality analysis of time series data. For marketers who do not always have the luxury of A/B tests but still want to be data-driven, this package should be a core part of the toolkit. Based on original research, the model establishes what’s called a…
Read more

Selecting success metrics in web experiments

Ron Kohavi is one of the most respected names in analytics and currently leads Microsoft’s Experimentation Platform. His publications, especially those since ~2001, are required reading if you’re in the business of data. I am constantly referencing his work, such as Online Experiments: Lessons Learned, an article with Roger Longbotham published in 2007: A common pitfall in…
Read more

In God we trust; all others, bring data

Several months back, Wired posted an essay on the rise of A/B testing within tech companies. Nothing groundbreaking for analytics practitioners, but hearing war stories—both successes and failures—never gets old. On culture and HiPPOs: This was the culture that he had come from at Google, what you might call a democracy of data… Siroker explains,…
Read more