fbpx

Category: Methodology

Multicultural Research: Top 3 Things You Should Know

Multicultural Research: Top 3 Things You Should Know

According to a 2017 American Community Survey, one in seven US residents are foreign born. In Canada, that number is even higher. At the time of the 2011 National Household Survey, 1 in 5 Canadians were born outside Canada. This large influx of new US and Canadian citizens represents a prime marketing and sales opportunity for North American brands looking to grow their market share in an established and at times crowded market.

The increased importance of brand visibility and awareness amongst newcomers has turned multicultural marketing into somewhat of an arms race. Brands now seek out new advertising and marketing channels that provide high levels of exposure to newly landed or soon to arrive immigrants. Airports, Visa offices, and even partnerships with brands abroad are all opportunities that have been pursued in recent years to varying levels of success.

Brands are now understanding the importance of being first to market, and that making an impression with newly landed immigrants can have a noticeable impact on market share. However, what’s being lost in the mix is messaging and value.  Does the message of your advertising correlate with newcomers’ values and belief systems? Is the messaging relatable and does it speak culturally to those viewing the ad?

In order to create a great marketing campaign, you need the insights to support it.  That’s why multicultural research has become more important than ever.  Here are 3 things to consider when designing and executing your research campaign.
Aerial view of supported hands out together

Consider your sample makeup.

We tend to think of newcomers holistically, but every immigrant has a story, a different path to North America and background with his or her own set of beliefs and cultural shopping behaviours.  That’s why it’s important that your sample set is as representative of the newcomer population as possible.  Things to consider include tenure and acculturation levels, country of origin and regional segmentations within country.  The closer you mirror your sample to representation, the more accurate your results.

multicultural
Think about regional and cultural biases

Each country is unique and even within country there are regional and cultural biases.  The opinions and cultural beliefs of someone from the North of India is different than someone from the south, and someone who speaks Mandarin from mainland China will have vastly differently shopping behaviours than a Cantonese speaker from Hong Kong or Macau.  It’s important to take this into account when designing your questionnaire, ensuring that you provide different language options for those taking part in the study and that you’re tailoring parts of your questionnaire to speak to respondents on a regional basis.

Traditional Roles are Localized

In North America, we tend to view traditional gender roles as a thing of the past.  It’s now common place for both heads of the household to have full time jobs and to help around the house.  Although we view this to be true in North America, the same can’t be said for some other countries around the world.  It’s important to give consideration to who your intended purchaser is and how the roles of ultimate purchaser and decision maker vary from region to region.  For example, in most cases traditionally the man of the house has been the breadwinner for South Asian households, but the female is the ultimate decision maker for food prep and purchase.

A well thought out questionnaire and sample plan will go a long way in ensuring your study is as representative as possible, and that you’re gaining meaningful insights into your target audience.  As newcomers to North American continue to grow in size and they become a larger part of our population, we as market researchers need to do a better job of tailoring our studies to their individuality and to think about them more as specific sub segments as opposed to one large conglomerated audience.  Doing this will create opportunities for additional products and services to benefit those who are currently underserved, and in turn will aid brands in growing their market share in a crowded marketplace.

CATI IN 2019: Battling Declining Response Rates

The transition away from telephone interviewing is well underway. Online polls now make up the principal source of data on consumer opinion surveys, with CATI projects largely being relegated to B2B audiences, historic tracking studies and geographic areas that lack online representation.

online polling Although online has now become the preferred methodology for most due to its attractive price point, it still poses a lot of questions in the subjects of quality and representation. The absence of a clear set of standards for online research, in particular online polling, has led to many questions about the reliability of the results that are shared.

Whilst journalists are now more likely to cite online polls as a reliable source of data, it is still unclear in many cases how these polls are fielded, and if the sample composition is entirely dependable.

Of course, traditional live-interviewing telephone polls have been facing their own challenges as of late. As the cost of labor has increased, so too has the average cost per telephone survey, and as consumers turn away from landline phones in favour of cell phones, response rates continue to decline. To further complicate matters, phone response can vary greatly within different demographic groups.

phone survey There is substantial evidence that phone interviewing is inherently less biased than opt-in panels. For one, randomly dialling both landline and cell phones puts you in touch with a consumer that may or may not know about market research and hasn’t voluntarily opted into a research-based panel. Secondly, a phone approach provides access to segments of the population that are traditionally under-represented on online panels including younger males, visible minorities and senior citizens.

As consumers continue the trend of ditching their landline in favour of a cell phone only household, sample and interviewing techniques must keep pace in order to ensure that a CATI methodology remains representative and that response rates remain high.

Here are three things to consider when conducting your CATI project.

 

Cell Phone vs Landline

An RDD (Random digit dialling) approach is effective in some cases, but as households move away from having a landline in their home it’s now more important than ever to ensure cell phone sample is included in your sample mix. Based on the latest data, a 60% land line, 40% cell phone sample split is ideal. This blend will ensure that you’re reaching cell phone only households which tend to skew younger in age.

 

Maximizing Your Success with Targeted Audiences

Does your project contain visible minorities, or perhaps those who are considered to be affluent? You can dial blindly amongst the general population in the hopes that these individuals will fall out naturally. However, chances are that these audiences are more likely to have unlisted numbers which would not be included in general landline dialling. For these groups a targeted sample list approach might be best. Targeted sample lists contain appended demographic data which is sourced from 3rd party consumer lists and databases. This increases your chances of connecting with your desired audience and you can boost your overall target numbers.

 

Making Your Survey Relatable and User Friendly

In addition to targeted sample, it’s also important to consider the overall framework of your project to ensure that you are making it as easy as possible for participants to take part. If you’re dialling visible minorities are you using interviewers that speak the language? Are you dialling from a local number as opposed to a toll free number. Is the survey itself accessible?  The more relatable you make your survey to participants the higher participation and cooperation rates will be.

 

Some people say phone is dying. I prefer to look at is as a metamorphosis.  As consumers habits change, so too must our research practices and approach. At the end of the day, the goal is to ensure high quality response rates and by following the 3 steps above you’ll ensure that your phone projects continue to be well rounded and balanced.

 


About Chris Connolly

2018 Color headshot Chris Connolly, Vice President of Research Services at The Logit Group is persistently focused on providing superior value and experience for each client relationship. With over 20 years of industry experience serving in leadership roles for both technical groups and project management teams, he has a proven track record of success.

Top Three Research Lessons From Election Season Polling

With the Canadian Federal Election taking place on October 21st and the US Presidential election slated for 2020, it’s a perfect time for us researchers to reflect upon sampling, declining response rates, margins of error, and questionnaire wording for federal election polling surveys.

Those of you who are not researchers or analysts have probably been wondering, how truly representative are election polls, and can they still lend credibility to predicted election results in a world of declining response rates?

Looking back on previous elections and their results, it’s clear that there are important takeaways and learnings from how election polls are conducted.  These takeaways not only ensure that future election work is as representative as possible, but they also lend themselves to takeaways that can be transposed to other research projects.

ballot choice

  1. Ask the right questions to the right audience.

Political polling highlights the importance of asking the right questions. The wording of both the questions and answers will affect the outcome in different ways. For example, here are some considerations:

  • Should you ask respondents for whom they will most likely vote for when the election takes place? Or who they would vote for if the election was held tomorrow?
    • Should your question use the first names of the candidates, as they will appear on the ballots, and/or should you refer to each candidate’s party affiliation?
    • Who are the intended respondents? (All registered voters? All adults of voting age? Anyone intending to vote? Only registered voters who voted in the past election?)

To answer any of these questions, you need to have some clear goals and objectives of why you are conducting the research. There are many types of political polls, and multiple reasons for conducting them. Trying to measure consumer or voter attitudes is different than trying to predict behaviour, which in turn is different from trying to forecast voter turnout.

phone interview

  1. First know the why, then determine which method will get you there.
    Most political polls are subjected to a huge amount of methodological analysis and critique. Was the poll conducted online or by phone? Did the pollster use live interviewers or automated dialing with interactive voice recordings? Was the sample selected through random digit dial (RDD) or targeted listings?

    Questions like these offer insights into the most important methodological challenges facing the research industry in a time of rapid technological and social change. It reminds everyone that the method is so important. The challenges are not just about qualitative versus quantitative or surveys versus focus groups, but also about design, fieldwork, respondent recruitment, data collection, statistical analysis, and interpretation.

people in rally

  1. Think about the story rather than the numbers.
    Search online for “elections results” and you’ll find a multitude of different polls released on a weekly basis. Most of the results are in alignment with each other, but a handful are notable in their approach because they make the story just as important as the outcomes.

    In those cases, the content is relevant, interest is high, and the stakes are made to feel huge. The story is moving, and it feels like you are one of those individuals they were reporting on. While every poll has the support from PR, marketing, and communication campaigns, it is always nice to see those select few polls that report on things, or from angles, that nobody else has explored yet.

    Conclusion

    When it comes to political polling, the aims and goals are so important that you need to know your approach before you even begin to start thinking about the method and the potential questions. However, once you have your direction, it becomes critical to do more than show the numbers—you must also show the story behind those numbers to make your content relevant and interesting… and to really resonate with your audience.


About Jake

Jake-Pryszlak_avatar_1546770824-400x400 Jake Pryszlak, commonly known as the Research Geek, is a 3-time award-winning market researcher, blogger and speaker. He’s a current Forbes columnist who is active across a plethora of social media channels. His aim is to share his market research knowledge with others in the industry. You can find his blog and social media channels here.

Things to Think About When Conducting DIY Research

Smaller companies, especially start-ups, often face challenges because they lack financial security or specialist resources such as research leads. To make matters worse, when it comes to cutting costs, it’s the marketing and research budgets that are frequently the first to be reduced. Fortunately, however, the range of cost-effective tools now available should mean a lack of money and experience is no longer an excuse for even the smallest company to not conduct market research.

Across this industry, DIY (i.e. do it yourself) research tools are accessible, widespread, and sometimes even free! From designing a survey for your customers to programs for data report visualizations, there are automated programs online for you to use. However, as with any DIY project, there is still a number of things you need to think about when conducting your own market research.

Quick Results

With the right DIY tools, you don’t need to hire a research company or wait several months for results. Online research platforms mean the various tools and resources are at your fingertips—all you require is a bit of patience, time, and effort to create your own research. Thankfully, there are loads of different examples to help you to get started.

cash-cent-child-1246954

Further, many organisations who work in the DIY research space are now moving into a SaaS (i.e. software as a service) type of model. This means they can offer their services and tools at cost-effective prices and speed.

Some things are ready for you already
Many survey tools now provide ready-made templates for different industries and scenarios, as well as automated reporting; they can also offer preselected panels for which you can pay online. All these companies have also invested a lot of money into the user experience, so it is simple and easy to conduct your own research in one place.

It can be cheaper (sometimes)
Did you notice the “sometimes” in brackets above? This qualifier is because you do have to take your time with DIY research, and you also have to think about what you really want to understand from your target audience. Without this patience or nuance, you could end up rushing the process and ultimately needing to spend more money to fix the issues you caused.

analytics-chart-charts-920116

When organizations or individuals know what they would like to understand, DIY market research can actually provide plenty of opportunities to be creative at a low price. The question is: Do you truly know what you’re looking for?

Being a step in front of your competitors
Not conducting proper market research is one of the main reasons why so many start-ups fail in their first five years. This is because they haven’t bothered to truly understand the audience to whom they wish to sell. Of course, the fact that not everyone conducts market research can be an advantage for you. To be honest, even thinking about conducting market research puts you a step forward of your competitors who don’t.

The information and insights you can gain from conducting your own market research will help you more even more steps ahead of your competition, if you frame your approach correctly. It’s the difference between a company that conducts market research to understand consumer thoughts and opinions… and a company that just goes off on a hunch or a rough idea.

Overall
More and more vendor partnerships are now being brokered in the DIY space, and the possibilities continue to be exciting as services and tools evolve.

For small companies (or even larger ones with limited budgets), it is important to remember that just because many tools are now automated, you need not rush through the process to find your answer. In fact, it actually means you need to take the same amount of time as you would have done if there was no automation—the speed will come in to play when you are collecting the data and then finding out the results that matter to you.

However, despite all the possibilities associated with DIY research, it should still ultimately be viewed as a complement to traditional full-service research. DIY is a great starting point and offers all the advantages mentioned above, but make sure you do not lose sight of your ultimate goal in exchange for using DIY methods.



About Jake

Jake-Pryszlak_avatar_1546770824-400x400  Jake Pryszlak, commonly known as the Research Geek, is a 3-time award-winning market researcher, blogger and speaker. He’s a current Forbes columnist who is active across a plethora of social media channels. His aim is to share his market research knowledge with others in the industry. You can find his blog and social media channels here.

Getting the Best Out of Your Customer Satisfaction Program

In the last decade, there have been significant changes to how researchers define “customer satisfaction,” as well as how they use this metric.

Also known as CSAT, customer satisfaction measurement has evolved over time, largely spurred on by technology. It has moved from point-in-time to real-time, from anonymous to linked, and from brick-and-mortar to multi-channel. Throughout these changes, the basics behind a customer satisfaction program have remained essential—gather data to help a client turn opinions into actionable learnings and insight.

At Logit, we collect data in different ways, depending on the client’s customer database or research requirements. We offer the capabilities to execute different methodologies to reach different customer audiences, including phone interviews, online surveys, onsite interviews, and mail surveys.

When considering customer satisfaction surveys, you first have to think about the customer journey and put yourselves in their shoes. For example, how would you like to receive a survey? When would you like to complete a survey?

woman-working-on-ss-designer-1200x670

The telephone interview has always been an in-demand service from clients who have contact lists. Nevertheless, we see declining participation rates. Fewer people want to take part in comparison to a number of years ago.

On the other hand, online surveys are rapidly gaining momentum. Depending on the survey length, this methodology can be relatively quick and error-free for the client and the participants. It’s similar when thinking about onsite interviews—we always recommend the survey length be no longer than five minutes. These types of interviews are great for clients who may not have a customer list, or for clients who want to understand the opinions of consumers who may not actually purchase a product from the store.

Depending on your data collection instrument (i.e. phone or on-site), it is always important to think carefully about the identity of the client and the values of the brand. At Logit Group, we ensure all interviewers are trained to represent the brand well.

Making it work
Once you have decided on your methodology, you still need to ensure that it is actionable for your customer satisfaction program.

Connect the dots
Consumer responses and their data must be connected to the specific transaction, if one was made. This means each function of the business can receive specific feedback.

Ask yourself: Are you being clear?
When I look at reports, I always think: “Is this data actionable, and is it written in language that easily understood?”

Data and reporting should be clear and simple to understand. Many clients actually provide real-time shared customer experience information to their internal staff because experiences can change from day to day, month to month, or season to season.

Pause for reflection
A customer satisfaction program should not be left alone for years but reviewed every six to 12 months to ensure it is generating ROI and actionability across the entire organization. You need to ask your internal stakeholders what they think of the tools and the dashboards offered. Their feedback allows you to make effective changes to your approach, making certain it is always relevant to the current state of business.

Things to think about…
Almost all organizations have a customer satisfaction program. From my experience, no two are the same and the ideal approach will be unique to each company and its stakeholders, both internal and external.

Once the CSAT program is in place, the data is used to help evolve your products or services, you still need to ensure your customers understand what is being changed and why. Again: Never forget the customer journey. They are taking time out of their day to help you, so if you have altered something because of their feedback, then you need to make sure you not only tell them what actions have taken place because of their opinions and close the loop, but also thank them for their participation and feedback.

 


About Oscar
oscar

Oscar Fernandes serves as the VP of Sales & Client Services at Logit. For over 25 years he has helped his clients execute successful CSAT programs, both online and over the phone.

Top Four Tips for Boosting Sampling Response Rates

It might sound obvious, but your sample is the most important part of your market research project.

Too often, it seems like the survey participants’ experiences and opinions of market research are somewhat overlooked. However, our industry relies heavily on individuals giving up their own time and effort to respond to long questionnaires. If they don’t enjoy the experience or gain any benefit, then why should they bother participating?

Businesses rely on customer data to guide their decision making and provide a sense of direction when making a change in terms of a product enhancement, service overview, or even a new product range. Therefore, reduced response rates ultimately mean less insight or fewer data-driven outcomes.

How can you help your participants enjoy the experience of giving you feedback?

 

1. Treat people the way you would want to be treated

It is important to ensure your research invitations and reminders clearly outline what you are asking. This may include information on why you are conducting the research, incentives on offer (e.g. gift cards), and an explanation why their feedback will be so valuable.

You should try to personalize communication to an individual as far as possible with the resources you have available. For example, most email marketing tools allow you to directly customize how you address emails to individuals rather than impersonal form letters.

Far too often, researchers leave participant communication to the bottom of their list of priorities. I think this is totally wrong. Ask yourself whether you would complete a particular survey if you yourself received the email you’re about to send.

jakeblog2.image2

2. Go mobile

So many people in the industry mention the use of mobile surveys that it must get boring to always read about it! Still, the reason we all say it so much is because we still continually find surveys that have not been mobile-optimized and are not responsive to being answered on a phone or tablet. It can be challenging to get participants to complete a survey while they are watching TV, and an even bigger task to convince them to answer your questions when they are hard to read on a cellphone screen.

We know a high proportion of individuals are “second-screen watchers,” which means they may be watching TV while also texting on their phone. By making a survey mobile-optimized, you increase the likelihood of someone completing it as a second-screen experience instead of never bothering to take part.

jakeblog2.image9

3. Never be boring

Can you remember the last time you wanted to complete a survey that consisted of 40 questions? I can’t… and I am sure your participants feel the same way.

Neither researcher nor participant benefits from excessively lengthy and tedious questioning in either qual or quant research. When survey participants are bored, they are more likely to flip through the survey questions, rush and give false answers just to complete it. Having a seemingly endless list of questions also increases the likelihood of dropouts throughout the survey, negatively affecting your representative sample.

You should be developing short and lean surveys that take participants less than five minutes to complete. This can give you the essential information you require while also increasing the likelihood of a large sample size because of the short length.

jakeblog2.image10

4. Don’t sit on your feedback

After completing a quantitative survey that has a sample size of 1,000+, the worst thing you could do is just ignore all that feedback and not act on any of the new intelligence.

Participants want to feel valued—not just from a gift or reward point of view, but also emotionally. They want to know whether or not their feedback has truly helped, and they really want to see what you, as a brand, will do with the insight and opinions they shared. Offering participants feedback allows them to see the true value of completing a survey or a piece of research for you. It means they will be far more likely to take five or 10 minutes of their own time to complete something for you again.

jakeblog2.image7

Conclusion

By making surveys short, sharp and to the point, you give participants less work to do and your business still gains valuable data and information. The four tips outlined in this article are only a handful of ways to boost response rates. However, implementing even one of these suggestions will help improve the research experience for your participants. Happy and rewarded participants mean quality data outcomes for you that can lead to data-driven decision making.


About Jake

Jake-Pryszlak_avatar_1546770824-400x400 Jake Pryszlak, commonly known as the Research Geek, is a 3-time award-winning market researcher, blogger and speaker. He’s a current Forbes columnist who is active across a plethora of social media channels. His aim is to share his market research knowledge with others in the industry. You can find his blog and social media channels here.

5 Top Ways To Build An Effective Online Research Sample!

So you have googled different research methodologies available and have decided that online research is your chosen golden nugget. In particular, you are interested in using an online sample because you wish to ask the same or similar individuals questions about your business and products.

Online sampling can be a crazy world and very difficult to start if you don’t have a set process to follow. This is why I have put together my top 5 things you should look out for when creating or using an online sample!

My top 5 tips are for those who want to build a unique online market research sample, because like I said, it can be a tricky task to even start. I’m hoping these 5 takeaways will help you to breakdown the process so its much more manageable.

  1. What Do You Want to Understand?

The first question you must ask yourself is – What would you like to understand from your research? You need to first define your research objectives. Your objectives will affect what research sample you wish to create and promote. Especially if you’re focusing questions to a set persona or type of individual.

For example, if you are looking to change some your core products, or add to your existing product range. You will need to understand which groups this will affect and how you want them to be represented in your research. Customers, potential customers, mar-comm audiences and stakeholders all need to be represented in a way which reflects their opinion.

On the other hand, let’s say you have a targeted marketing campaign that is on Facebook and other social media platforms, then you would actually what to understand the opinions and thoughts from that specific target audience who you have targeted.

blog image 1

  1. Leverage Existing Networks

Recruiting participants for research can actually become very expensive especially when taking into account the size of your sample. Yet, the best place to start for any size or scale of research is your own networks. Whether that is using LinkedIn, social media and even your own email lists, customer databases and any other existing connections you have built. People on these lists will be those most valuable to you which means they will also have an opinion.

Current customer opinions is crucial and actually more relevant in some ways than a panel because they are familiar with your brand.

They will be motivated by a desire to improve the brand and experience, rather than the financial incentive.

African Woman Working Design Creative Concept

  1. Get Yourself on Social!

There has been a lot of buzz recently about social market research with the likes of Brandwatch and SproutSocial dominating the space. Social media listening tools have driven this discussion and market researchers have been quick to adopt such processes. Whilst it would be difficult to use social media from a sample perspective, it is still important to think about social in its broadest sense.

Social media can complement market research through the entire process, from introducing community-based elements during the project, to driving participant recruitment. A subset of snowball sampling methodologies, social media recruitment leverages the personal connections of individuals to reach a wider potential audience. By combining this with your organization’s own networks, it is possible to build a large (and representative) sample in a short space of time.

Then you can think about social media influencers in your area of work to help generate interest and spark a conversation about your new sample. With the use of social media, you can grow your sample size as well as understand what your target audience are actually talking about online. Which will help when creating topics, tasks and surveys for your participants to answer during your sample journey.

blog image 3

Screen Participants

When you promote a sample on social media the danger is sample quality. There are many dangers to look out for, including speeders, professional research participants, no-shows and more. The best way to eliminate these is through an invitation questionnaire to understand who the individual is and whether they are a best fit for the sample you need.

This will then serve two things. The first is to ensure that your sample fits the profile you are looking for, as there is no point in sending questions to a group of individuals who may not even know what you are on about. The second is to drop participants that would reduce your data quality. Speeders are participants that complete research tasks as quickly as possible and do the bare minimum. Their responses are not always reflective of their own thoughts, often writing the first thing that comes to mind.

The easiest way to catch a speeder out is by asking them a particular question which you ask them to select a particular answer. Speeders will unlikely even read the question before making a choice.

blog image 4

  1. Manage Your Lists and Participants

At the end of the day, your list are your participants and potential customers. So treat them how you would like to be treated. Over time, some participants will drop out, it is only natural. As more and more drop out this can have an overall negative impact on your research results and sampling quality. To ensure your research doesn’t suffer, you should regularly monitor active and inactive participants and also those on the verge of leaving. The latter could be sent some new information or you could seek to understand how you could help them from not leaving.

 

So by following these quick 5 steps you will be on your way to creating a high-quality panel of research participants. Obviously there are pro’s and con’s to using samples, however, by controlling the different processes that are there to see, it is possible to create a high-quality sample that will help your business in the short and long term.


About Jake

Jake-Pryszlak_avatar_1546770824-400x400 Jake Pryszlak, commonly known as the Research Geek, is a 3-time award-winning market researcher, blogger and speaker. He’s a current Forbes columnist who is active across a plethora of social media channels. His aim is to share his market research knowledge with others in the industry. You can find his blog and social media channels here.

Trust, But Verify: Using Online Panels for B2B Research

Conducting B2B research via online panels is an increasingly attractive option. Its more efficient cost model translates to roughly 30% of the price of running the same project by telephone. Incentives are lower online and you’re able to cast a much wider net to accomplish goals far more quickly.

While this may sound great, serious questions can arise over how respondents are recruited… and to how to ensure those answering your surveys are in fact qualified to do so (and are who they say they are).

The survey-taking experience is recruited under the guise that it is completely anonymous. As such, respondents aren’t recruited with a phone number and can’t be validated with telephone verification. This means you need to have a certain level of faith in your panel-of-choice company, and this trust needs to be cemented by a history of successful projects together and the power of their name within the industry.

While trust is important, so is a little common sense. Regardless of past performance, there are additional steps to take across each survey as safeguards to help ensure the content of the report comes from qualified B2B online respondents.

recruiting

Picking the right partner

Client relationships are carefully constructed; they need care, attention, and acknowledgement that years of hard work have taken place prior. It’s important to pick a partner that not only respects this philosophy, but also has the experience and courage to tell you the possible pitfalls, preparing you for the reality of the project at hand. This enables you to more reasonably predict the end result, using experience and asking the right clarifying questions to give everyone confidence and a platform from which to build.

With data collection, an account manager will often work on the viability/feasibility and costs for a project, but then passes it along to a project team for execution. Effective B2B research is accomplished when the account manager is tethered to the project from start to finish, and can frame expectations, ensure the team is on target, and work with the client on the fly if necessary to adjust and implement backup plans. B2B research can be nuanced and fraught with challenges that require foresight, experience, and the ability to jump in, correct, and sometimes change tact.

 

b2b

Pre-screening and ensuring B2B panelists are who they say

 “You’re only as good as your last book” is a smart adage to adopt when working with panel sources.

Panels are expected to adhere to the ESOMAR/GRBN Guideline on Online Sample Quality, which sets out best practices in:

  • Research participant validation, to ensure the respondent falls within the description of the research sample;
  • Survey fraud prevention, to ensure the same person doesn’t try to receive more incentives by completing a survey more than once;
  • Survey engagement, to ensure that the respondent is paying sufficient attention;
  • Category and other types of exclusions, to ensure the sample does not include respondents who might bias the results; and
  • Sampling (including sample selection, sample blending, weighting, survey routers, profiling, and screening) to provide transparency.

While these are the cornerstones of panel sampling businesses, it’s important to ensure they do this and acknowledge that respondent profiling isn’t as advanced as it needs to be in B2B sampling.

B2B profilers are sent out, of course, but the completion rates are low and panel companies will often steer consumer respondent traffic that they know is employed within a general business sector.

Most proprietary panel companies have partner sources they introduce. Although vetted appropriately, new sources in the mix can increase the probability for errors based on each source’s ability to control the fraudulent behaviour appearing from time to time. Some of these partner sources can also skew results, with the base of answers really off the expected norms or what other sources in aggregate are showing.

To mitigate this, pre-screening becomes very important even among panel sources that have sufficient profiling for B2B respondents in place. Screening questions for the targeted respondent to go through before entering your survey are ideal for ensuring a respondent is truly qualified to participate.

About half of the incoming panel traffic fails for some reason or another, but this is still an important piece to put in place to ensure that those entering the survey are in fact who you need to answer the survey.

Trust, but verify

“Trust, but verify,” is a useful way to describe how best to manage and monitor a B2B market research panel project and ensure a high-quality data set.

Given the absence of exact profiling, many panels sources need to be tethered together to accomplish ambitious goals or to look for a subsection of respondents within a certain industry.

Whether or not there has been that additional layer of pre-screening, it is critical to embed security conditions (e.g. time to complete, straight-lining) and pepper red herring questions into the survey. (These can be monitored in your daily field disposition, with fails tied to the panel source). Reviewing verbatim for gibberish is another measure for discarding cases that don’t meet quality criteria.

When blending multiple panel sources, it is important to measure the sources against each other and focus on the “quality fails” that arise from the security conditions set, the red herrings, and verbatim review to arrive at pass-back rate percentage by panel. Additionally, you should review responses by panel across each other to identify blips and skews in data. If any are present, they should be isolated and removed from the data set, and passed back to the panel for replacement at no charge. Further, after a pre-test of 10% of the quota is completed, the panel source(s) showing pass-back rates higher than 30 to 40% should be investigated for legitimacy. If necessary, they should be removed from the sampling, forwarded, and removed from the data set.

While all these quality review metrics are important, they must be reasonable—typical pass-back rates on security fails in the industry range between 10 and 20%. (With a pre-screener employed, it tends to be much less). When it is above 20%, there is either a quality source issue or it is overly stringent and the project at hand may not be appropriate for the online methodology. It is important to investigate both possibilities.

Conclusion

Human beings are creatures of comfort, and we prefer to put a lot of faith and trust in proven panel providers. While I think trust is key, it is also important to be vigilant and to employ your own reasonable security metrics that make sense each time. You also need to understand that with panel sources, issues with respondent quality can arise and fraudulent sources (e.g. bots) can break though. With these extra steps and an experienced partner, you’re able to avoid issues and ensure that your report is based purely on respondents that belong.


John Wulff started at Logit Group as its first salesperson in 2008 and has a 30-year career focused on B2B/B2C online, telephone, and onsite data collection. He has held senior positions representing some of the largest and best quantitative phone and panel companies with operations based in North/Central America, Europe, and Asia.

John’s areas of expertise within B2B/B2C data collection are focused on financial, automotive, health-care, entertainment, and information technology segments. In addition to data collection business development efforts for Logit, he leads business development for Logit Group’s technology company—QFI Solutions, a survey software programming/reporting platform.

Do Try This at Home: Understanding the Benefits of IHUT

For more than 25 years, Logit Group has worked in quantitative and qualitative research, but our areas of expertise go beyond traditional online sampling to areas like in-home usage testing (IHUT). If you’re not familiar with this method, it’s a really cost-effective way to test your product with real people before moving forward with a full-scale product launch.

Testing The Market Before You Hit The Market
There are risks in creating a brand-new product without testing anything. When using IHUT, you have the opportunity to ship products to participants to use at home before you hit the traditional markets. Their feedback is gathered using various means such as follow up telephone or online surveys, or even in-person interviews. This way, individuals are fully engaged in the whole process from start to finish. Since IHUT relies on a real-life environment rather than a controlled market research scenario, it is more likely to result in actual outcomes on product satisfaction, usage, and potential improvement areas.

applying makeup

How Logit Conducts IHUT

When we do IHUT work with a client, we recruit respondents via an online panel that fits a particular desired market segment—the target audience. After respondent selection, the products are then sent to the participants. At different times during the usage period, participants receive an invitation to fill out an online questionnaire or a different type of feedback tool.

One of the surveys may capture the participants’ first impressions and experiences with the product after the first week. A final questionnaire can help determine the experiences and satisfaction with the product in detail, as well as help identify areas for improvement. We recently conducted IHUT work for a national Dairy company company that wanted to understand what consumers thought of its packaging and product taste. The research included two different phases—an eye-tracking exercise and a packaging assessment. We invited 500 product users to participate in the 25 minute in-person test. 375 concept acceptors were given product to take home to understand any changes in their satisfaction and acceptance of the product.

Using the IHUT methodology, Logit Group was able to understand participants’ first impressions, appeal, and purchase intent. As part of some performance accept/reject analysis, we could track results over time to understand changes in consumer preferences and opinions in relation to how long they have been using the product.

data

The Uses of IHUT In Further Research

Overall satisfaction, as well as more pinpointed satisfaction on specific product features, could be measured to provide first insights into potential areas of improvement. In the case of a longer usage period and multiple questionnaires, levels of satisfaction can be measured over time. To focus on potential areas for improvements, I always like to recommend open-ended questions to then really dive deep into what the participant is thinking about a product.

Throughout the IHUT process, data is being analyzed to provide clear, relevant results and recommendations. By the end of the research project, the goal is to know exactly whether or not your product is truly ready for full market launch or requires additional improvement, as well as its potential in terms of acceptance and how to best position it within the market space. IHUT can also help you understand whether there are any geographic differences that you need to think about when marketing and launching your product.

When working with highly skilled analysts like those at Logit Group, IHUT allows you to understand some of the most important checks on your product before launch by means of real-life opinions, comments, and data.


About Aref

Aref Munshi As Vice President, Sales & Research Services for The Logit Group, Aref Munshi’s main responsibilities include, managing existing clients. He has been providing qualitative and quantitative support services to clients across the healthcare, consumer & business industries. With over 30 years of data collection experience, Aref’s strength is his holistic market research skill set. From client services, to operations, Aref is the perfect client advocate and research problem solver. He has held senior management roles at two of the larger data collection companies in Canada. First 25 years at RIS Christie and the last 7 with The Logit Group.

Should you combine Quantitative and Qualitative research?

When undertaking a research project, the first question people often ask is whether to conduct quantitative or qualitative research. However, both of these methodologies actually complement each other! In other words, while it can be useful to think of them as single approaches, there are also times when you should combine them for even clearer data.
Difference-between-Quantitative-and-Qualitative-Data

 

What is Qualitative Research? 

Qualitative research, or “qual,” seeks in-depth, freeform answers from respondents either in person or via open-ended responses. As recently as only five years ago, I would have said this was usually carried out with small groups in the form of in-person focus groups, telephone interviews or detailed surveys with free-text responses. Things have changed. While qualitative can still be conducted in person, the vast majority of qual is now done online. (As I’ll explain later in this article, this allows someone to conduct qual research at a quantitative scale.) 

 

images Qual research is often a go-to method for an insight department trying to gather anecdotal views and opinions. It offers a deeper understanding, with the ability to explore topics in more detail., which would usually come about via unprompted feedback. Qual can be a great approach for anyone looking to expand or start a brand-new product line because it allows you to gain honest responses and comments from your target audience.

 

Three to five years ago, you could argue that qualitative research was hard to measure, but there are so many different tools out there to analyze open-ended comments (as well as video responses!) that clients can receive results within hours of their research project getting underway.

 

The only thing I do find potentially difficult when conducting qualitative research is its statistical robustness. This is because you are generalizing to your broader audience, rather than having thousands of data points to analyze.

 

What is Quantitative Research? 

 

On the flip side, quantitative research is, as the name suggests, all about the numbers! It tends to involve a large group of people (usually at least several hundred, but often thousands) completing a survey. While the approach is heavily numerical, this also means the results are clear and are harder to misinterpret. The survey can also be easily repeated and you can reliably track changes over time, such as in a tracker study. Then comes the analysis of your data—as you are asking closed questions, your data can be collected more quickly. 

images (1) However, dealing with numbers means you need a large sample of the population to deliver reliable results. The larger the sample of people, the more statistically accurate the outputs will be.

 

I always like to remind researchers that when they are creating online surveys, the wording is crucial. To be confident in the results of quantitative surveys, you have to be confident that you’re asking the right questions, in the right way, with the correct answer-options included.

 

Final Thoughts

Quantitative and qualitative research both have their place in market research, and a blended approach should be carried out whenever you’re extending product lines or launching something new because it can give you a holistic viewpoint on what your customers are thinking, rather than just from one data point. Both methods can work hand-in-hand; brands can use qualitative research for developing concepts and theories, and quantitative for testing pre-existing ones.

 

You can also use freeform qualitative research to guide the creation of more structured quantitative surveys. Following quantitative surveys, turn to qualitative to better understand the context of the responses! There are so many opportunities!

 

Technology advances like open-response video and heat mapping also mean that qualitative research is getting closer to the quantitative price point. As a result, more and more organizations are able to use both approaches during their work.

 

You no longer have to choose a methodology based on cost because there is now value for the money within the sector. The real question you should be asking is not over which approach but rather how to best use both to provide insights that can have an impact on the bottom line.


About Jake

Jake-Pryszlak_avatar_1546770824-400x400 Jake Pryslak, commonly known as the Research Geek, is a 3-time award-winning market researcher, blogger and speaker. He’s a current Forbes columnist who is active across a plethora of social media channels. His aim is to share his market research knowledge with others in the industry. You can find his blog and social media channels here.