What Marketers are doing Wrong with Data Analytics

Interesting podcast piece in K@W. Some useful cautions. The notorious p-hack is brought up again.  Its often used because is so easy to apply.  Simplicity Bias  It cannot be used alone.

What Marketers Are Doing Wrong in Data Analytics
Podcasts Research  North America leveraging-customer-analytics-featured-image

Wharton's Ron Berman explains why most marketers 'p-hack' and why it could lead to wrong results.

(Podcast at the link)

Companies gather and analyze data to fine-tune their operations, whether it’s to help them figure out which webpage design works best for customers or what features to include in their product or service to boost sales. Marketers, in particular, use data analytics to answer questions like this: To put people in a shopping mood, is it better to make the webpage banner blue or yellow? Or do these colors not matter? Getting the answer right could mean the difference between higher sales or losing to the competition.

But new Wharton research shows that 57% of marketers are incorrectly crunching the data and potentially getting the wrong answer — and perhaps costing companies a lot of money. “We expected business experimenters [to make this error], but I was nevertheless surprised that so many of them do so,” said Wharton marketing professor Christophe Van den Bulte, who coauthored the study. Wharton marketing professor Ron Berman, another of the study’s authors, agreed: “This was a pretty common phenomenon that we observed.” (Listen to a podcast interview with Berman about the research at the top of this page.)

Their paper, “p-Hacking and False Discovery in A/B Testing,” which was popularly downloaded and widely cited in social media, looked at the A/B testing practices of marketers who used the online platform Optimizely before the platform added safeguards against potential mistakes. In A/B testing, two or more versions of a webpage are tested to see which one resonates more with users. For example, half of a company’s customers would see webpage version A and the other half version B. “Imagine one version says something about the brand of your product and the other version says something about the technical abilities of your product,” Berman said. “You want to determine which one makes consumers respond better, to buy more of your products.” ... "