Measuring marketing value through incrementality experiments

Last week Google released CausalImpact, an open-source r package that both improves and simplifies incrementality analysis of time series data. For marketers who do not always have the luxury of A/B tests but still want to be data-driven, this package should be a core part of the toolkit. Based on original research, the model establishes what’s called a…
Read more

Choose your flavor of analytics

Most startups can bucket their analytics into three groups: Experimental analytics is about testing assumptions and hypothesis. If you can remember your high school science classes—where problems take the form if this, then that—then you understand experimental analytics. Product, marketing and sales can all benefit from this approach. Throw out the subjective and let data speak for…
Read more

Selecting success metrics in web experiments

Ron Kohavi is one of the most respected names in analytics and currently leads Microsoft’s Experimentation Platform. His publications, especially those since ~2001, are required reading if you’re in the business of data. I am constantly referencing his work, such as Online Experiments: Lessons Learned, an article with Roger Longbotham published in 2007: A common pitfall in…
Read more

In God we trust; all others, bring data

Several months back, Wired posted an essay on the rise of A/B testing within tech companies. Nothing groundbreaking for analytics practitioners, but hearing war stories—both successes and failures—never gets old. On culture and HiPPOs: This was the culture that he had come from at Google, what you might call a democracy of data… Siroker explains,…
Read more

Price is what you pay, value is what you get

Classic Ben Graham quote, by way of Warren Buffett. ConversionXL walks through this concept, hitting on several challenges of effective pricing strategy: Asking people what they’d pay for and how much rarely works. For one thing people will tell you what they WANT to pay—which is obviously much less than what your product or service is actually…
Read more