“Not everything that can be counted counts, and not everything that counts can be counted.” – William Cameron, Professor of Sociology

“It’s not the men in my life that count, it’s the life in my men.” – Mae West, actress

As I started doing design work within my org, I’ve come to realize that many people do not really understand the difference or purpose of qualitative vs quantitative data.
Quantitative data and its standards are much more well known, so people talk about ‘statistical significance’, ‘standard deviation’, etc. Part of this may have to do with the prevalence of statistical techniques being used around the different domains of business, whether sales, engineering, marketing, etc. Qualitative research is much less well-known or understood, and is often seen as “just subjective opinion (of the respondent)”.
The prevailing paradigm is of quantification. As a consequence, I have frequently encountered questions of “is this finding statistically valid? how many datapoints do we have for this?” etc. even for qualitative research findings.

I always feel very tempted to ask back in reply, “how much do you love your parents/spouse/partner/child on a scale of 1 to 10? And how many datapoints do you have for that rating, and is this rating statistically valid?” But I (just) always manage to hold my tongue…

The key thing that seems to elude most people who ask quant-paradigm questions is that, they are asking from the tool something that it is not meant for. Like using a scalpel to butcher a leg of lamb or a saw for shucking an oyster. You can probably do the job, but it is probably better to select the right tool beforehand.

Quantitative methods are great at showing phenomenon: proving or showing that “this phenomenon exists”. In some cases, you will be able to show causation (usually in the physical sciences), but frequently it is showing correlation and not causation. One thing that quantitative methods can’t really do, is to give clues on motivation and emotion, which are the primary drivers for human behaviour. Humans don’t behave like atoms. With quant methods, you have very little idea why people behave the way they do. Part of that is because of context, which is almost impossible to tease out from quantitative methods, because there are so many degrees of freedom that context is almost impossible to compute.

To illustrate, you could model humans mingling in a crowd pretty accurately using a Boltzmann or perfect gas atom model (at normal room temperatures and pressures) distributions. If a fire suddenly breaks out, everyone will crowd around the nearest fire exit. That is as though you have a gas that fills up a balloon normally, and suddenly all the atoms spontaneously head out of the balloon mouth. Not easy to model. But qualitatively, it is simply just an observation that a fire happened.

So qualitative observations often can help one tease out the contextual phenomena one is seeking, especially in understanding human behaviours (and seeing what they do in context).

The other reason is much more basic and fundamental: it’s still impossible to read people’s minds. So to find out why they did something, you have to rely on the cutting-edge technology of observing and asking them. Of course, there are also tools you can use to help you better figure out people’s motivations, like observing how people behave in certain contexts, and getting them to use prototypes. But the starting point is the same one, i.e. to understand what people think, one has to open one’s mouth to ask.

A similar use of this cutting-edge tech is in the managerial tool called ‘the 1-1’. So I guess a question to ask managers is, would you run a statistical analysis of feedback and comments from your 1-1s and solely rely on that? And what do you miss out when you skip your 1-1s? Quite likely, it will be about missing out on data points around mood, morale, motivation, etc.

Switch the word subordinates/employees with customers, and that is the same value that qualitative research brings. It isn’t something quantifiable, and yet it generates the insight, which is the driver for value.

In my ideal world, we would have huge amounts of quantitative data that are statistically significant, which can then give clues for where we can do qualitative research. You need both, in the same way it’s not apples or oranges but apples AND oranges to have a more balanced diet.

So maybe the above helps give a better understanding of how qualitative data hugely complements quantitative data, and how to use both.

Date 10 Aug 2020