Just as a reminder, due to the length of some interviews, I recommend you set aside 10-minutes or bookmark this page if you don’t have the time to read it all at once.
Today’s interview was conducted over email with Annie Pettit, Chief Research Officer at Conversition.
“With over 15 years of market research experience, Pettit was previously the VP of Online Panel Analytics at Ipsos Interactive Services, as well as a private consultant in the area of research analytics. She holds a PhD in Experimental Psychology, was previously licensed as an Industrial/Organizational Psychologist, and has specialized in the area of online survey data quality. Pettit is also known under the LoveStats moniker as an avid market research blogger (http://lovestats.wordpress.com/) and tweeter (@LoveStats).” – Conversition
An Exciting Future for Market Research:
“I can’t say I’m excited about this possibility from a personal privacy standpoint, but from a researcher’s perspective, I can see that 3rd party aggregators are going to turn demographic data gathering into an art-form. Many people have multiple online accounts such as Facebook and Twitter and Youtube and Flickr and Linkedin. It’s easy enough for an aggregator to visit all of these sites and gather demos like age, gender, income, household size, race, religion, political views, hobbies etc and create an accurate profile of every person. Imagine the analytics that could be done with such detailed data! If you’re a survey provider, you’d never have to waste ten minutes asking those questions again. If you’re a social media research provider, you wouldn’t have to wonder how to get quality data. As I said however, the privacy and ethical considerations for this process are scary. At what point is collecting data invading privacy and simply stalking?” – Annie
“You’ve brought up an interesting view on where market research will go with social media research. In a short article by Andrew Jeavons titled “Facebook says use us for opinion research…“, Cameron Marlow, the head of the Facebook data science team, has recently mentioned the capability and interest of doing research with the Facebook database.”
[Given that there’s a population of 250 million people or more logging every day, it seems ridiculous that you would try to get in touch with them through a phone…so it seems like it’s improbable that social media won’t be the way that we acquire opinion research.]
“The key words here are “opinion research”, which I’ve assumed means a more traditional opt-in type of research rather than aggregation from posts, profiles, photos, etc. With strict privacy regulation being pushed by government and consumers it is difficult to imagine that demographic data collection from social media will become a valid possibility in the near future.”
“An example of the recent online privacy movement is highlighted in The Wall Street Journal article titled “FTC Backs Do-Not-Track System for Web“. This WSJ article may not focus on aggregation, but it definitely adds to the trend of more consumer privacy online.”
“If market researchers can’t aggregate data online, at least without specific permission from the online users, then do you still see this type of social media data collection as becoming a valid method?” – Sean
“Aggregating data can take place in different ways. It can be aggregating demographic information such as age, gender, income, etc. People see the collection of these data points as extremely intrusive and we know this from our difficulties in collecting the information via traditional phone/panel methods.”
“Aggregating blog posts and forum discussions from public websites without demographic datapoints has a very different kind of privacy issue. In this case, it is very easy for us to respect the privacy of individuals – we don’t have their demographic information and we don’t need to publish the exact web links for every comment. There are lots of companies that don’t understand why it’s not appropriate to publish this ‘public’ information, but market researchers do. Market researchers will once again be the beneficiaries of privileges such as what we have seen with do-not-call lists – because we self regulate and respect privacy, we will have more options available to us than most other types of online data users.” – Annie
A View on Social Media Monitoring:
“Social media monitoring isn’t the most useful things for clients. It’s just not enough to know when the mentions of a brand name increase or decrease. You need to know exactly why the change happened and how much it affected perceptions of the brand. And, this needs to happen in a way that is valid and reliable. This is what social media research provides.”
“As researchers, we know that even the tiniest process can completely change research results – if more than half of your data comes from Twitter, even though only 8% of people use Twitter, your results will not generalize to any group of people but tweeple. So, clients wanting to take full advantage of social media data should use social media research. This method incorporates sampling, weighting, scaling, norms, and other key research processes to ensure results can be generalized beyond Twitter and beyond other unrepresentative websites. Once these processes are in place, social media research is like getting a million responses to a ten hour survey in an age when we get barely get 300 completes to a 20 minute survey. Given that the SMR results can be used either qualitatively or quantitatively in a dataset that looks surprisingly like a survey dataset, it’s no wonder clients are keen to get going with SMR.” – Annie
“You’re absolutely right! Social media research needs to be approached with a critical eye by researchers to ensure that the insights drawn from any findings are valid.” – Sean
Social media measurement: “The tracking of various social media content such as blogs, wikis, micro-blogs, social networking sites, video/photo sharing websites, forums, message boards, and user-generated content in general as a way for marketers to determine the volume and sentiment around a brand or topic in social media.” – Wikipedia
“I still would not say that I do measurement. Research is not simply measurement. Research includes standardized processes such as what I mentioned above which create validity and reliability. You can measure anything but if you don’t include the scientific processes, you’re not measuring anything useful. Social media research is the application of scientific market research principles to the collection and analysis of social media data.” – Annie
Research Insights through Implicit Learning:
“True insight of the light-bulb caliber by definition is few and far between. I think instead I’ll share one of my favourite learnings. Coming from a psychology background, I’ve studied many different phenomena which affect how people go about their daily lives. Of critical importance to market research is something called “implicit learning.” Basically, it refers to things we learn without knowing we’ve learned them. This is a major reason why marketing campaigns work. Even though we might want to think that commercials and banners and billboards have no effect on us, even when we think we aren’t paying attention to them or letting them get to us, we suck ideas and emotions into our brains without realizing it. You cannot prevent it. The money we spend researching the best song, the best colour, or the best spokesperson for a brand isn’t silly. It’s taking advantage of implicit learning and unconscious processes.” – Annie
“This “implicit learning” is something that advertisers have known about for a long time, but have only recently thought about when it comes to online marketing. I recently read an article about how online advertisers need to start focusing on how online ads are influencing people without them knowing it. The article highlights how traditional online measurement like click-through rate is not an accurate representation of an ad’s overall influence (effectiveness) on it’s audience (“Moving Beyond The Click – How Advertising (Should) Work” – comScore Voices).”
Implicit learning: “Learning of complex information in an incidental manner, without awareness of what has been learned.” – Wikipedia
“Do you have a specific example where implicit learning was measured for an online campaign?” – Sean
“I really can’t talk about specific cases but every single advertising campaign, whether they intend to, is taking advantage of implicit learning. It also means that we need to really think about our traditional questionnaire design. We expect people to be aware of all of their senses and be able to answer questions about exactly how and why and where and when they heard about something. But chances are they don’t really know except for the most salient occasion – which was likely not the first occasion. We prefer to think that surveys responders are lazy or not trying when in fact the information simply isn’t available to them.” – Annie
Analytics Integrated into Traditional Market Research:
“If by analytics you mean social media research, we’ve already begun the task of integrating SMR with survey data. Given how we’ve already learned how to integrate online and offline surveys, mobile surveys and online surveys, focus groups and online survey, it’s just a natural next step to integrate SMR with online surveys.”
“Social media research can be designed to match up nicely with surveys. For example, a typical survey asks about purchasing, recommendations, and new/different. Well, social media research can include those exact same variables. The qualitative data can be fed through a process called “content analysis” which creates those matching variables. This process can occur across an entire survey – what competitive brands are used, what are the personality characteristics of the brand, what features of the brand are most important. All of these questions can be paired up between the two methods allowing researchers to examine similarities and differences among people who choose to answer survey and people who choose to participate in social media. In a sense, this integration allows you to listen to people who have never been listened to before while at the same time allowing you to maintain and monitor your ongoing baselines.” – Annie
“By analytics I was specifically referring to text analytics for social media research. I think what you’re referring to is the first step to the integration process, but it isn’t the entire story. In a blog article I wrote titled “How to Avoid Killing Survey Respondent Engagement with New Market Research“, I explain how I see the future of social media research and traditional market research integration. My guess is that people who participate in traditional research are very likely to also participate in social media, but unfortunately I don’t yet have any statistics to confirm this. If this is true, then we can go much further than just comparing the findings between social media and traditional research. We could use social media integration to tailor research to the appropriate people and use methodologies that are most applicable to them.” – Sean
“I don’t have the stats in front of me, but people like to talk about the Comscore finding from a few years back where 1% of people are answering X% of surveys. Whatever the number was, it was an extremely tiny number. Now, we know that 8% of online people use Twitter. That’s already far more than the number of people answering surveys. How many people use facebook? 20% of the WORLD, including people who’ve never seen or heard of a computer. So, even if every single person answering surveys is participating in social media, they are totally overwhelmed by the rest of the non-survey answering world participating in social media. Social media lets us listen to far more people than we could every dream of listening to via surveys.”
“My end goal is not to compare survey results with social media research results. We know that we’re talking to different kinds of people who are participating for different reasons. This step of comparing survey results and social media results is an interim step for people to see what social media data looks like and to understand how they can use it in their own business. My end goal is for people to see social media research as another option in their entire set of options, an option that provides answers that they may not have been able to acquire previously.” – Annie
Choosing The Best Research Method:
“I must protest! The twittersphere is rife with qual vs quant and online WOM vs offline WOM and this vs that … I don’t believe any method is any better than any other method, and we really need to promote a breadth of skill and interest for every method. Every method has strengths and weaknesses, and a good researcher knows how to select the right method for the right job.”
“My favourite method for gathering on the spot retailer decisions is the mobile survey. My favourite method for gathering quantitative data like frequencies and volumes is the good old fashioned survey. My favourite method for learning serendipitous knowledge and going beyond the 20 minute survey barrier is the social media research. If I must choose one method, I would choose any method developed with reason, skill, validity, and reliability.” – Annie
“Said like a true researcher. Alternatively, what methodology or analysis was the most fun for you to use most recently?” – Sean
“I’m actually quite keen on eye-tracking and neuro techniques. Given that people (even me) really don’t understand why they do the things they do or why they learn certain things, it makes a ton of sense to research human processes that are normally hidden from us. Behavioural methods get right to the heart of where our thoughts are coming from.” – Annie
Once again I’d like to thank Annie Pettit for all of the time she spent working on this interview with me. I encourage you to follow Annie through your favorite social media site and also comment below.
Thanks again for reading my latest blog post,
Sean Copeland.
