I’ve been stewing on my argument for a couple of weeks now, to follow up my post Myth: #BigData Will Kill Online Access Panels. Of course, I think that if access panels exist, even if they offer big data resources for their opt-in panelists, they will continue to be a conduit for survey research to fill in the quantitative gap that the data sources leave.
The reasons that I believe survey research will survive extend the point made by Clive Humby of the data analytics firm Dunnhumby. Humby says that big data will answer the questions “how many” and “how much”, but that research will remain in order to answer the “why” questions. I think the gap between research and big data analytics also include:
- The research process is not included in big data analytics; and
- Big data will have a hard time measuring attitudes and motivations.
As for the first point, I think the research process is too important to abandon. In fact, a recent best-of-the-week blog post from Annie Petit of Conversition Strategies lists “not starting with a research objective” to be the number one among the 6 Worst Market Research Mistakes. The problem that I see with the incompatibilist view (that big data will replace research) is that in the case of big data, the data precludes the research objective. In the process of research, the research question/objective is determined prior to data collection, and I just see the data as being a constraint to the types of questions that can be asked.
For the second point, big data tends to be transactional, regarding activities that a person has engaged in. But will there be a way to connect those actions to a person’s attitudes and motivations? I’m having a hard time imagining how that could be done, but I hold this point as one that could be debated. Neuromarketing may be one answer, but it has a number of hurdles to overcome before it becomes a reality in the way it needs to be to solve this problem. That is, how can you match it to big data if it’s not “big neuro”? And that “big neuro” part is what I think will have a hard time gaining traction.
Personally, I am a quantitative researcher and I’m excited about big data. I love data, no matter what the source, so I’m not trying to defend my job or livelihood by backing the future of market research (as some real skeptics like to say). My opinion is that I don’t see clients abandoning the research process when it allows so much control over the data that is being collected. Survey research is not the answer to everything, and there are clearly topics that will be better addressed by other resources (e.g. usage behavior, shopping behavior, customer lifetime value, etc.). If collecting data by talking to the consumer goes away, then we’ll need to have data sources that:
- Can be matched up to other data sources at the individual level;
- Collect data on attitudes and motivations;
- Can provide insight on hypothetical products and situations;
- And more…
I can’t wait to be proven wrong.
Dale Gilliam is the founder and principal analyst for Troubadour Research & Consulting, a market research services firm providing analytics and data services for the research and consulting community. Dale can be found on Twitter @troubadourrc and LinkedIn.
image credit: xkcd