ESOMAR 28 QUESTIONS TO HELP BUYERS OF ONLINE SAMPLES
CROWDOLOGY UK AND USA
Below, Redshift have answered ESOMAR’s (The European Society for Opinion and Market Research) 28 questions that cover important points that should be considered when purchasing online ssurveys.
CROWDOLGY COMPANY PROFILE
- What experience does your company have in providing online samples for market research?
Redshift Research was founded in 2007, becoming a part of the marketing services group Next 15. In 2009 Redshift took the need for audience understanding one stage further and developed its own opinion panel, Crowdology.
SAMPLE SOURCES AND RECRUITMENT
- Please describe and explain the type(s) of online sample sources from which you get respondents. Are these databases? Actively managed research panels? Direct marketing lists? Social networks? Web intercept (also known as river) samples?
We have two actively managed panels owned and operated by Redshift, under the brand name Crowdology. We also have access through partnerships with other agencies to over 11 million people across 50 countries.
- If you provide samples from more than one source: How are the different sample sources blended together to ensure validity? How can this be replicated over time to provide reliability? How do you deal with the possibility of duplication of respondents across sources?
We use a variety of deduplication techniques, including IP monitoring at the point of survey entry rather than at the back end of the process.
- Are your sample source(s) used solely for market research? If not, what other purposes are they used for?
Our panellists are exclusively used for Market Research. Crowdology™ follows the Data Protection Act 1998 and current personal privacy legislation, so our panelists can rest assured that their information will not be abused in any way. Crowdology is a division of Redshift Research – a Market Research Society Company Partner which undertakes all research in accordance with the Market Research Society Code of Conduct and ESOMAR guidelines. Redshift Research is also registered with the Data Protection Agency – registration number Z1751697.
- How do you source groups that may be hard to reach on the internet?
Strict quota control for age gender and region are applied where representation is an issue. As a member of the CINT exchange we also have access to millions of people across the globe which often means we can achieve a sufficient sample of hard to reach audiences. Where necessary a blended research approach with phone or face to face interviewing will be used.
- If, on a particular project, you need to supplement your sample(s) with sample(s) from other providers, how do you select those partners? Is it your policy to notify a client in advance when using a third party provider?
Like many sample providers we do on occasion work with third parties. We are a member of the CINT exchange which means all panellist are subject to the same strict sign up controls, and we have visibility of each of the panels recruitment and retention methods.
SAMPLING AND PROJECT MANAGEMENT
- What steps do you take to achieve a representative sample of the target population?
In order to achieve a representative sample, we will firstly send out survey invites that are representative of age, gender, and region in the specific country/area. We also do secondary research into our target audiences, and find their demographic nature (e.g. Age, Gender, Region, Income etc). From this we can replicate the required audience in our sample.
- Do you employ a survey router?
We employ the Cint Optimizer, an integrated route solution which has a profiling engine at its core.
- If you use a router: Please describe the allocation process within your router. How do you decide which surveys might be considered for a respondent? On what priority basis are respondents allocated to surveys?
Cint’s Optimizer uses already stored profiling information to decide which surveys are relevant to respondents. This minimises the number of qualifying questions and time in the routing process for the panel members. Survey allocation is randomised.
- If you use a router: What measures do you take to guard against, or mitigate, any bias arising from employing a router? How do you measure and report any bias?
The Optimizer is only used on studies that will not likely be affected by the routed participants. The randomisation element used within the router is key to guarding against and mitigating bias.
- If you use a router: Who in your company sets the parameters of the router? Is it a dedicated team or individual project managers?
The project manager will decide whether the project is suitable to use the Optimizer with. The routing parameters are set by Cint’s operations team where a selected number of administrators have access to view or change the parameters.
- What profiling data is held on respondents? How is it done? How does this differ across sample sources? How is it kept up-to-date? If no relevant profiling data is held, how are low incidence projects dealt with?
We hold over 900 points of data on each of our panellists, from demographics to hobbies, medical conditions and B2B details. Panelists are paid in cash for the majorty of work, however vouchers are sometimes offered in addition for lengthy in home testing projects or unusual research designs
- Please describe your survey invitation process. What is the proposition that people are offered to take part in individual surveys? What information about the project itself is given in the process? Apart from direct invitations to specific surveys (or to a router), what other means of invitation to surveys are respondents exposed to? You should note that not all invitations to participate take the form of emails.
Most invitations to complete surveys come in the form of email, however we also have an active Facebook page of panelists where unusual projects such as in home testings may be advertised.
- Please describe the incentives that respondents are offered for taking part in your surveys. How does this differ by sample source, by interview length, by respondent characteristics?
Panellist are rewarded dependant on the length of the interview you complete, but most rewards vary from $0.40/ £0.40p up to $10/ £10. The cash reward for the survey is clearly stated before they participate. All respondents are required to have a UK or USA bank account and a PayPal account to ensure residency.
- What information about a project do you need in order to give an accurate estimate of feasibility using your own resources?
Size of sample required, screening questions and turnaround time. If a client can give an estimated incidence rate this is also very helpful. If not, we can always do secondary research and find the incidence rate ourselves. (For example, if the target audience were smokers, we could find the % of people who smoke using national statistics)
- Do you measure respondent satisfaction? Is this information made available to clients?
We have the facility to ask all or some of our respondents for feedback on surveys where deemed beneficial to the research design.
- What information do you provide to debrief your client after the project has finished?
We can provide a full sample provider debrief report, including gross sample, start rate, participation rate, drop-out rate, the invitation/contact text, a description of the field work process, and so on upon request
DATA QUALITY AND VALIDATION
- Who is responsible for data quality checks? If it is you, do you have in place procedures to reduce or eliminate undesired within survey behaviours, such as (a) random responding, (b) illogical or inconsistent responding, (c) overuse of item non-response (e.g. “Don’t Know”) or (d) speeding (too rapid survey completion)? Please describe these procedures.
We run regular trap surveys for speeding, straight lining, neutral/opt out answers and inconsistent answers.
- How often can the same individual be contacted to take part in a survey within a specified period whether they respond to the contact or not? How does this vary across your sample sources?
Respondents can be invited to take the same survey just once – they receive the initial invite and if they do not respond to this a reminder is sent out two days after the original. We enforce a minimum quarantine period of 3 days between contacts to avoid fatigue. Respondents have the option to increase this to a longer period.
- How often can the same individual take part in a survey within a specified period? How does this vary across your sample sources? How do you manage this within categories and/or time periods?
We enforce a minimum quarantine period of 3 days between surveys to avoid fatigue. Respondents have the option to increase this to a longer period.
- Do you maintain individual level data such as recent participation history, date of entry, source, etc., on your survey respondents? Are you able to supply your client with a project analysis of such individual level data?
- Do you have a confirmation of respondent identity procedure? Do you have procedures to detect fraudulent respondents? Please describe these procedures as they are implemented at sample source registration and/or at the point of entry to a survey or router. If you offer B2B samples what are the procedures in place, if any?
We require respondents to have a UK or US bank account and corresponding PayPal account. IP access points are checked and if a respondent is suspected to be located abroad they are required to go through a postal validation check to confirm residency.
POLICIES AND COMPLIANCE
- Please describe the ‘opt-in for market research’ processes for all your online sample sources.
All our panellists are recruited with a double opt-in process to check & confirm that they wish to join Crowdology
- Please describe the measures you take to ensure data protection and data security.
Redshift Research is also registered with the Data Protection Agency – registration number Z1751697.
- What practices do you follow to decide whether online research should be used to present commercially sensitive client data or materials to survey respondents?
This varies on a project by project basis, on occasion respondent confidentiality agreements are required
- Are you certified to any specific quality system? If so, which one(s)?
We are an MRS company partner
- Do you conduct online surveys with children and young people? If so, do you adhere to the standards that ESOMAR provides? What other rules or standards, for example COPPA in the United States, do you comply with?