fbpx

Archive: August 29, 2019

5 Top Ways To Build An Effective Online Research Sample!

So you have googled different research methodologies available and have decided that online research is your chosen golden nugget. In particular, you are interested in using an online sample because you wish to ask the same or similar individuals questions about your business and products.

Online sampling can be a crazy world and very difficult to start if you don’t have a set process to follow. This is why I have put together my top 5 things you should look out for when creating or using an online sample!

My top 5 tips are for those who want to build a unique online market research sample, because like I said, it can be a tricky task to even start. I’m hoping these 5 takeaways will help you to breakdown the process so its much more manageable.

  1. What Do You Want to Understand?

The first question you must ask yourself is – What would you like to understand from your research? You need to first define your research objectives. Your objectives will affect what research sample you wish to create and promote. Especially if you’re focusing questions to a set persona or type of individual.

For example, if you are looking to change some your core products, or add to your existing product range. You will need to understand which groups this will affect and how you want them to be represented in your research. Customers, potential customers, mar-comm audiences and stakeholders all need to be represented in a way which reflects their opinion.

On the other hand, let’s say you have a targeted marketing campaign that is on Facebook and other social media platforms, then you would actually what to understand the opinions and thoughts from that specific target audience who you have targeted.

blog image 1

  1. Leverage Existing Networks

Recruiting participants for research can actually become very expensive especially when taking into account the size of your sample. Yet, the best place to start for any size or scale of research is your own networks. Whether that is using LinkedIn, social media and even your own email lists, customer databases and any other existing connections you have built. People on these lists will be those most valuable to you which means they will also have an opinion.

Current customer opinions is crucial and actually more relevant in some ways than a panel because they are familiar with your brand.

They will be motivated by a desire to improve the brand and experience, rather than the financial incentive.

African Woman Working Design Creative Concept

  1. Get Yourself on Social!

There has been a lot of buzz recently about social market research with the likes of Brandwatch and SproutSocial dominating the space. Social media listening tools have driven this discussion and market researchers have been quick to adopt such processes. Whilst it would be difficult to use social media from a sample perspective, it is still important to think about social in its broadest sense.

Social media can complement market research through the entire process, from introducing community-based elements during the project, to driving participant recruitment. A subset of snowball sampling methodologies, social media recruitment leverages the personal connections of individuals to reach a wider potential audience. By combining this with your organization’s own networks, it is possible to build a large (and representative) sample in a short space of time.

Then you can think about social media influencers in your area of work to help generate interest and spark a conversation about your new sample. With the use of social media, you can grow your sample size as well as understand what your target audience are actually talking about online. Which will help when creating topics, tasks and surveys for your participants to answer during your sample journey.

blog image 3

Screen Participants

When you promote a sample on social media the danger is sample quality. There are many dangers to look out for, including speeders, professional research participants, no-shows and more. The best way to eliminate these is through an invitation questionnaire to understand who the individual is and whether they are a best fit for the sample you need.

This will then serve two things. The first is to ensure that your sample fits the profile you are looking for, as there is no point in sending questions to a group of individuals who may not even know what you are on about. The second is to drop participants that would reduce your data quality. Speeders are participants that complete research tasks as quickly as possible and do the bare minimum. Their responses are not always reflective of their own thoughts, often writing the first thing that comes to mind.

The easiest way to catch a speeder out is by asking them a particular question which you ask them to select a particular answer. Speeders will unlikely even read the question before making a choice.

blog image 4

  1. Manage Your Lists and Participants

At the end of the day, your list are your participants and potential customers. So treat them how you would like to be treated. Over time, some participants will drop out, it is only natural. As more and more drop out this can have an overall negative impact on your research results and sampling quality. To ensure your research doesn’t suffer, you should regularly monitor active and inactive participants and also those on the verge of leaving. The latter could be sent some new information or you could seek to understand how you could help them from not leaving.

 

So by following these quick 5 steps you will be on your way to creating a high-quality panel of research participants. Obviously there are pro’s and con’s to using samples, however, by controlling the different processes that are there to see, it is possible to create a high-quality sample that will help your business in the short and long term.


About Jake

Jake-Pryszlak_avatar_1546770824-400x400 Jake Pryszlak, commonly known as the Research Geek, is a 3-time award-winning market researcher, blogger and speaker. He’s a current Forbes columnist who is active across a plethora of social media channels. His aim is to share his market research knowledge with others in the industry. You can find his blog and social media channels here.

Trust, But Verify: Using Online Panels for B2B Research

Conducting B2B research via online panels is an increasingly attractive option. Its more efficient cost model translates to roughly 30% of the price of running the same project by telephone. Incentives are lower online and you’re able to cast a much wider net to accomplish goals far more quickly.

While this may sound great, serious questions can arise over how respondents are recruited… and to how to ensure those answering your surveys are in fact qualified to do so (and are who they say they are).

The survey-taking experience is recruited under the guise that it is completely anonymous. As such, respondents aren’t recruited with a phone number and can’t be validated with telephone verification. This means you need to have a certain level of faith in your panel-of-choice company, and this trust needs to be cemented by a history of successful projects together and the power of their name within the industry.

While trust is important, so is a little common sense. Regardless of past performance, there are additional steps to take across each survey as safeguards to help ensure the content of the report comes from qualified B2B online respondents.

recruiting

Picking the right partner

Client relationships are carefully constructed; they need care, attention, and acknowledgement that years of hard work have taken place prior. It’s important to pick a partner that not only respects this philosophy, but also has the experience and courage to tell you the possible pitfalls, preparing you for the reality of the project at hand. This enables you to more reasonably predict the end result, using experience and asking the right clarifying questions to give everyone confidence and a platform from which to build.

With data collection, an account manager will often work on the viability/feasibility and costs for a project, but then passes it along to a project team for execution. Effective B2B research is accomplished when the account manager is tethered to the project from start to finish, and can frame expectations, ensure the team is on target, and work with the client on the fly if necessary to adjust and implement backup plans. B2B research can be nuanced and fraught with challenges that require foresight, experience, and the ability to jump in, correct, and sometimes change tact.

 

b2b

Pre-screening and ensuring B2B panelists are who they say

 “You’re only as good as your last book” is a smart adage to adopt when working with panel sources.

Panels are expected to adhere to the ESOMAR/GRBN Guideline on Online Sample Quality, which sets out best practices in:

  • Research participant validation, to ensure the respondent falls within the description of the research sample;
  • Survey fraud prevention, to ensure the same person doesn’t try to receive more incentives by completing a survey more than once;
  • Survey engagement, to ensure that the respondent is paying sufficient attention;
  • Category and other types of exclusions, to ensure the sample does not include respondents who might bias the results; and
  • Sampling (including sample selection, sample blending, weighting, survey routers, profiling, and screening) to provide transparency.

While these are the cornerstones of panel sampling businesses, it’s important to ensure they do this and acknowledge that respondent profiling isn’t as advanced as it needs to be in B2B sampling.

B2B profilers are sent out, of course, but the completion rates are low and panel companies will often steer consumer respondent traffic that they know is employed within a general business sector.

Most proprietary panel companies have partner sources they introduce. Although vetted appropriately, new sources in the mix can increase the probability for errors based on each source’s ability to control the fraudulent behaviour appearing from time to time. Some of these partner sources can also skew results, with the base of answers really off the expected norms or what other sources in aggregate are showing.

To mitigate this, pre-screening becomes very important even among panel sources that have sufficient profiling for B2B respondents in place. Screening questions for the targeted respondent to go through before entering your survey are ideal for ensuring a respondent is truly qualified to participate.

About half of the incoming panel traffic fails for some reason or another, but this is still an important piece to put in place to ensure that those entering the survey are in fact who you need to answer the survey.

Trust, but verify

“Trust, but verify,” is a useful way to describe how best to manage and monitor a B2B market research panel project and ensure a high-quality data set.

Given the absence of exact profiling, many panels sources need to be tethered together to accomplish ambitious goals or to look for a subsection of respondents within a certain industry.

Whether or not there has been that additional layer of pre-screening, it is critical to embed security conditions (e.g. time to complete, straight-lining) and pepper red herring questions into the survey. (These can be monitored in your daily field disposition, with fails tied to the panel source). Reviewing verbatim for gibberish is another measure for discarding cases that don’t meet quality criteria.

When blending multiple panel sources, it is important to measure the sources against each other and focus on the “quality fails” that arise from the security conditions set, the red herrings, and verbatim review to arrive at pass-back rate percentage by panel. Additionally, you should review responses by panel across each other to identify blips and skews in data. If any are present, they should be isolated and removed from the data set, and passed back to the panel for replacement at no charge. Further, after a pre-test of 10% of the quota is completed, the panel source(s) showing pass-back rates higher than 30 to 40% should be investigated for legitimacy. If necessary, they should be removed from the sampling, forwarded, and removed from the data set.

While all these quality review metrics are important, they must be reasonable—typical pass-back rates on security fails in the industry range between 10 and 20%. (With a pre-screener employed, it tends to be much less). When it is above 20%, there is either a quality source issue or it is overly stringent and the project at hand may not be appropriate for the online methodology. It is important to investigate both possibilities.

Conclusion

Human beings are creatures of comfort, and we prefer to put a lot of faith and trust in proven panel providers. While I think trust is key, it is also important to be vigilant and to employ your own reasonable security metrics that make sense each time. You also need to understand that with panel sources, issues with respondent quality can arise and fraudulent sources (e.g. bots) can break though. With these extra steps and an experienced partner, you’re able to avoid issues and ensure that your report is based purely on respondents that belong.


John Wulff started at Logit Group as its first salesperson in 2008 and has a 30-year career focused on B2B/B2C online, telephone, and onsite data collection. He has held senior positions representing some of the largest and best quantitative phone and panel companies with operations based in North/Central America, Europe, and Asia.

John’s areas of expertise within B2B/B2C data collection are focused on financial, automotive, health-care, entertainment, and information technology segments. In addition to data collection business development efforts for Logit, he leads business development for Logit Group’s technology company—QFI Solutions, a survey software programming/reporting platform.