Qualitative Analysis

Early on, long before machine learning, we did content analysis.  We still did quantitative analytics, but only after we had structured our problem with qualitative methods.  In fact you always start with qualitative.  Here a good intro:

Qualitative before Quantitative: How Qualitative Methods Support Better Data Science Co-authored with Vicky Zhang in Medium.

Have you ever been embarrassed by the first iteration of one of your machine learning projects, where you didn’t include obvious and important features? In the practical hustle and bustle of trying to build models, we can often forget about the observation step in the scientific method and jump straight to hypothesis testing.

Data scientists and their models can benefit greatly from qualitative methods. Without doing qualitative research, data scientists risk making assumptions about how users behave. These assumptions could lead to:

Neglecting critical parameters,
missing a vital opportunity to empathize with those using our products, or
misinterpreting data.

In this post, we’ll explore how qualitative methods can help all data scientists build better models, using a case study of Indeed’s new lead routing machine learning model which ultimately generated several million dollars in revenue.

What are qualitative methods and how are they different from quantitative methods?
Few data scientists are formally trained in qualitative methods. They’re more deeply familiar with quantitative methods like A/B testing, surveys, and regressions. Quantitative methods are great for answering questions like “How much does the average small business spend on a job posting?”, “What are the skills that make someone a data scientist?”, or even “How many licks does it take to get to the center of a Tootsie roll pop?” (The answer is 3. Three licks.)

But there are some questions that quantitative methods can’t answer, such as “Why do account executives reach out to this lead instead of that lead?” or “How do small businesses make the decision to sponsor or not sponsor a job?” Or the truly deep question: “Why do you want to get to center of the Tootsie roll pop?”

To answer these questions, qualitative researchers rely on methods like in-depth interviews, participant observation, content analysis and usability studies. These methods involve more direct contact with who and what you’re studying. They allow you to better understand how and why people do what they do, and what kinds of meaning they ascribe to different behaviors.

Put another way, quantitative methods can tell you the “what”, the “how much”, or “how often”; qualitative methods can tell you the “why” or the “how”. ... "