This year (2015) we were once again given the opportunity by panel review website SurveyPolice to conduct a question and answer forum. The aim of this was primarily to allow panellists to ask questions about online panels, surveys and market research and to provide them with insight from the perspective of the panel owners (us!). However, we also recognised this as a fantastic opportunity to understand the needs and concerns of panellists, which we hoped would enable us to improve our surveys and service.
So, what did we learn?
- Panellists dread grid questions
- Screening questions are not always placed appropriately
- Panellists are genuinely interested in the companies behind the panels
- There is growing discontent with the mobile accessibility of surveys
1. Grid Questions
OK, so this wasn’t really new, we know panellists dislike grid questions, but I don’t think we fully understood the dread that these can foster in people. Common complaints were:
- Difficulty in distinguishing between the lines
- Not being able to see the option headers
- Length of grids
Several mentioned that if they were faced with a long grid they would quit the survey, while others mentioned they are aware that they don’t always give the best quality answers due to their frustrations and the inaccessibility.
During the week this was probably the most mentioned topic/issue, and as such we turned the Q&A on its head and asked two questions ourselves:
- Which style of question is preferred?
- How might grid questions be improved?
We were incredibly appreciative of how forthcoming and detailed responses were and have picked out some of the suggestions here. Not only will Redshift Research be paying heed to these, but we will also be actively passing on this knowledge to those that we work with.
General improvements for all questions:
- Ensure instructions are repeated if continued onto a new page
- Character counters for open ended questions
- Example responses for open ended questions
- Alert respondents at the beginning if they will be required to view videos
Suggestions for grid improvement:
- Alternating row colours
- Column headings after every few rows
- For very long grids – split across pages and indicate how many pages of grids there are
- Use alternative question styles:
Clearly grids are a big concern for panellists and therefore should be so for market researchers due to their potential of increasing drop out rates and affecting data quality.
2. Screening Question Placement
A common complaint was that panellists are often screened out of surveys at a point that they find unacceptable. Some quoted 40-95% of the way through, others mentioned after 20 minutes of question answering. The lesson here is to double check where the screeners are – if they aren’t at the front either move or remove.
3. Interest in Market Research Companies
Perhaps we are just naïve, but we were quite surprised by the genuine interest in the market research industry, such as trends, qualifications, and the companies within it.
There seemed to be particular interest around the relationships between panels and market research companies (which can be as complicated as an East End soap!).
Time for more transparency perhaps?
4. Mobile Surveys
We all know about (and most likely are experiencing) the rise of the smartphone and how it has taken on the challenge of providing everything we need in one small, pocket-sized (or not so much) device. This extends to survey taking as well, as you might expect. But it seems that survey software isn’t always keeping up. Nor, in fact, are survey lengths.
What is needed to facilitate survey taking on mobile devices?
- Fewer (or more accessible) grid questions
- Shorter surveys
- Responsive design
These are perhaps topics for another time but with regards to survey length there seems to be small currents moving in the right direction but we are still waiting for the tidal wave.
Finally, we would like to thank Survey Police for facilitating, and all those who participated in the forum. Here’s to the next one!