fbpx

Archive: October 30, 2019

CATI IN 2019: Battling Declining Response Rates

The transition away from telephone interviewing is well underway. Online polls now make up the principal source of data on consumer opinion surveys, with CATI projects largely being relegated to B2B audiences, historic tracking studies and geographic areas that lack online representation.

online polling Although online has now become the preferred methodology for most due to its attractive price point, it still poses a lot of questions in the subjects of quality and representation. The absence of a clear set of standards for online research, in particular online polling, has led to many questions about the reliability of the results that are shared.

Whilst journalists are now more likely to cite online polls as a reliable source of data, it is still unclear in many cases how these polls are fielded, and if the sample composition is entirely dependable.

Of course, traditional live-interviewing telephone polls have been facing their own challenges as of late. As the cost of labor has increased, so too has the average cost per telephone survey, and as consumers turn away from landline phones in favour of cell phones, response rates continue to decline. To further complicate matters, phone response can vary greatly within different demographic groups.

phone survey There is substantial evidence that phone interviewing is inherently less biased than opt-in panels. For one, randomly dialling both landline and cell phones puts you in touch with a consumer that may or may not know about market research and hasn’t voluntarily opted into a research-based panel. Secondly, a phone approach provides access to segments of the population that are traditionally under-represented on online panels including younger males, visible minorities and senior citizens.

As consumers continue the trend of ditching their landline in favour of a cell phone only household, sample and interviewing techniques must keep pace in order to ensure that a CATI methodology remains representative and that response rates remain high.

Here are three things to consider when conducting your CATI project.

 

Cell Phone vs Landline

An RDD (Random digit dialling) approach is effective in some cases, but as households move away from having a landline in their home it’s now more important than ever to ensure cell phone sample is included in your sample mix. Based on the latest data, a 60% land line, 40% cell phone sample split is ideal. This blend will ensure that you’re reaching cell phone only households which tend to skew younger in age.

 

Maximizing Your Success with Targeted Audiences

Does your project contain visible minorities, or perhaps those who are considered to be affluent? You can dial blindly amongst the general population in the hopes that these individuals will fall out naturally. However, chances are that these audiences are more likely to have unlisted numbers which would not be included in general landline dialling. For these groups a targeted sample list approach might be best. Targeted sample lists contain appended demographic data which is sourced from 3rd party consumer lists and databases. This increases your chances of connecting with your desired audience and you can boost your overall target numbers.

 

Making Your Survey Relatable and User Friendly

In addition to targeted sample, it’s also important to consider the overall framework of your project to ensure that you are making it as easy as possible for participants to take part. If you’re dialling visible minorities are you using interviewers that speak the language? Are you dialling from a local number as opposed to a toll free number. Is the survey itself accessible?  The more relatable you make your survey to participants the higher participation and cooperation rates will be.

 

Some people say phone is dying. I prefer to look at is as a metamorphosis.  As consumers habits change, so too must our research practices and approach. At the end of the day, the goal is to ensure high quality response rates and by following the 3 steps above you’ll ensure that your phone projects continue to be well rounded and balanced.

 


About Chris Connolly

2018 Color headshot Chris Connolly, Vice President of Research Services at The Logit Group is persistently focused on providing superior value and experience for each client relationship. With over 20 years of industry experience serving in leadership roles for both technical groups and project management teams, he has a proven track record of success.

Top Three Research Lessons From Election Season Polling

With the Canadian Federal Election taking place on October 21st and the US Presidential election slated for 2020, it’s a perfect time for us researchers to reflect upon sampling, declining response rates, margins of error, and questionnaire wording for federal election polling surveys.

Those of you who are not researchers or analysts have probably been wondering, how truly representative are election polls, and can they still lend credibility to predicted election results in a world of declining response rates?

Looking back on previous elections and their results, it’s clear that there are important takeaways and learnings from how election polls are conducted.  These takeaways not only ensure that future election work is as representative as possible, but they also lend themselves to takeaways that can be transposed to other research projects.

ballot choice

  1. Ask the right questions to the right audience.

Political polling highlights the importance of asking the right questions. The wording of both the questions and answers will affect the outcome in different ways. For example, here are some considerations:

  • Should you ask respondents for whom they will most likely vote for when the election takes place? Or who they would vote for if the election was held tomorrow?
    • Should your question use the first names of the candidates, as they will appear on the ballots, and/or should you refer to each candidate’s party affiliation?
    • Who are the intended respondents? (All registered voters? All adults of voting age? Anyone intending to vote? Only registered voters who voted in the past election?)

To answer any of these questions, you need to have some clear goals and objectives of why you are conducting the research. There are many types of political polls, and multiple reasons for conducting them. Trying to measure consumer or voter attitudes is different than trying to predict behaviour, which in turn is different from trying to forecast voter turnout.

phone interview

  1. First know the why, then determine which method will get you there.
    Most political polls are subjected to a huge amount of methodological analysis and critique. Was the poll conducted online or by phone? Did the pollster use live interviewers or automated dialing with interactive voice recordings? Was the sample selected through random digit dial (RDD) or targeted listings?

    Questions like these offer insights into the most important methodological challenges facing the research industry in a time of rapid technological and social change. It reminds everyone that the method is so important. The challenges are not just about qualitative versus quantitative or surveys versus focus groups, but also about design, fieldwork, respondent recruitment, data collection, statistical analysis, and interpretation.

people in rally

  1. Think about the story rather than the numbers.
    Search online for “elections results” and you’ll find a multitude of different polls released on a weekly basis. Most of the results are in alignment with each other, but a handful are notable in their approach because they make the story just as important as the outcomes.

    In those cases, the content is relevant, interest is high, and the stakes are made to feel huge. The story is moving, and it feels like you are one of those individuals they were reporting on. While every poll has the support from PR, marketing, and communication campaigns, it is always nice to see those select few polls that report on things, or from angles, that nobody else has explored yet.

    Conclusion

    When it comes to political polling, the aims and goals are so important that you need to know your approach before you even begin to start thinking about the method and the potential questions. However, once you have your direction, it becomes critical to do more than show the numbers—you must also show the story behind those numbers to make your content relevant and interesting… and to really resonate with your audience.


About Jake

Jake-Pryszlak_avatar_1546770824-400x400 Jake Pryszlak, commonly known as the Research Geek, is a 3-time award-winning market researcher, blogger and speaker. He’s a current Forbes columnist who is active across a plethora of social media channels. His aim is to share his market research knowledge with others in the industry. You can find his blog and social media channels here.