The Future of Consumer Insights

Most mid-level managers, with ~8-10 years’ experience in traditional Market Research firms would
be clued in on the fact that “Analytics” is no longer a vague term when they started out their
careers, and it is definitely the future for the research industry, at least for the Quantitative side of
the business.

The initial vagueness in this field of analytics has slowly been disappearing with recruiters looking at
specific skill sets like data mining, warehousing and experts in AI/ML – this was not the scenario in
the early part of the decade which we are getting done with now.

So, what is happening in the insights industry currently?

In the early part of the decade, most traditional market research firms started moving away from the
usual Pen-and-paper (PAPI) surveys to Computer-aided (CAPI) surveys. This was a switch which led
to a tremendous improvement in the quality of data being collected on-field, as there were multiple
quality measures which could be plugged in when using CAPI (like jumping directly to questions
based on specific conditions, GPS tagging of surveys, control based on time taken to complete a
survey, etc.

Then in the later part of the decade, the increased usage of “R”, as a language pointed towards the
gradual shift in the industry – researchers cannot afford to be stuck with just knowing how to create
a questionnaire, sampling techniques, data analysis using Excel / SPSS, but need to start learning
newer tools and skills to cope up with the advent of Analytics.

The new competitors for Nielsen, Kantar, Ipsos, etc. are going to be businesses which have tons of
data with no proper structure per se. Hence companies like Amazon, Myntra, PepperFry, etc. would
want to get traditional research done but might also stop using traditional research for studies like
Consumer Profiling, Segmentation based on behaviour, etc. as they already have enough tracking
data of consumers from across different geographies, demographics and also their behaviour in
terms of what they have searched for, what categories they are interested in, etc.

But to delve into this amount of consumer data requires the following,

 Knowledge of Statistics
 Knowledge of coding in SQL or Python or any other language which is amenable to write
code for in analytics

Knowledge of Statistics cannot be superficial (which is usually the case with most traditional
researchers where things mostly don’t go beyond correlation, linear regression, Jaccards and
ANOVA) but in-depth i.e. one should not only be able to run a Logit Regression with data but also
understand if the model is a good fit, how can AI/ML be used to train the model and what insights
can be derived from the outputs.

The knowledge of a programming language will probably shift the title from “quantitative
researcher” to “data specialist” or “insights specialist” as they would know have an advanced skillset
of statistics, which will tell them which data points to look at and how to cull out the insights using
the required code.

The key challenge faced can be summed up as follows: how do we integrate big data with
“traditional” research, in order to provide a thorough understanding of consumers? Whether one is
conducting the research or is the end user of the same, the results must be actionable. As an insights
provider, the goal must not be to just collect relevant data, but to add a human-touch to the same;
then clients can use the results to achieve their business objectives. This is critical as artificial
intelligence, and the commodification of research services continues to grow and competitive
differentiation can arise only from delivering usable insights and solutions to marketing problems.

Does this mean that Traditional Research is sinking?

There are certain kinds of researches that cannot be done away with, like Opinion Polls, Mystery
Audits, etc. However, these kinds of researches are quite narrow in scope and very specialized. Most
FMCG firms are also doing away with regular Brand Track exercises as they are leaning more on
direct retail data itself – however certain soft but key metrics like Awareness levels for brands,
future disposition towards brands, etc. cannot be mined from existing data sets but need to be
captured on-field using a survey tool designed specifically for that.

That leaves us with the elephant-in-the room – Qualitative Research.

Qualitative research is seeing a huge upswing in terms of the volume of Depth Interviews, Focus
Groups, ethnographies, etc. There are multiple possible reasons for this, and one cannot really point
out to one specific reason for the same. At a personal level, I have observed that the current “Start-
up culture” has ensured that most firms do not have a specific “Consumer Insights” division but the
marketing team takes care of research for the firm. And a fear of statistics holds sway over most
young marketers and they feel quite comfortable with one-on-one depth interviews with open-
ended data rather than hardcoded responses with percentages.

However, this is not to take away anything from the value-add which a well-designed Qualitative
research can provide to marketers! Any customized research, especially in categories where the
client has minimal knowledge about, needs to start with a detailed Qualitative study which will
throw up many hypotheses to be checked using Quantitative research.

Perhaps going forward, the new mantra would be for marketers to conduct a detailed Qualitative
research in the traditional manner and then address the hypotheses which arise from that using
analytics itself instead of traditional quantitative research.

In this scenario, one can definitely say that “Consumer Insights” is not going to disappear however
the way in which the insights are culled will definitely move towards analytics in a more aggressive
manner.

So what does the new generation of young researchers need to do, to have a successful career in
Insights? Learn as much as possible on the following subjects: Statistics – Basic as well as Advanced,
and languages like R, SQL & Python. There is ample material available on the Internet with regard to
these subjects and there is always more and more to learn.

When armed with both Statistics and Coding, the data insights world is your oyster to have!