Advertisement

Quality & Quantity

, Volume 50, Issue 6, pp 2495–2513 | Cite as

Open narrative questions in PC and smartphones: is the device playing a role?

  • Melanie RevillaEmail author
  • Carlos Ochoa
Article

Abstract

Most survey questions are closed questions, where respondents have to select an answer from a proposed set of alternatives. However, a lot of surveys also include, at least occasionally, some open questions. Open questions that call for elaborated and developed answers, called “open narrative questions”, are used when the researchers want to go deeper into what the respondents think. This paper compares the answers to open narrative questions when the respondent is participating in a PC survey, in a smartphone-not-optimised survey or in a smartphone-optimised survey. The experiment was carried out in Spain using data collected by the Netquest online access panel. Respondents were assigned randomly to each type of device and survey format, in two successive waves. Because respondents have to type in their answer, we expect differences between devices, linked with the size and the kind of keyboards (i.e. physical versus digital, touch-screen or not). Differences are observed between answers that come from PCs and smartphones for the response time per written character, for the number of total characters and for the use of abbreviations, but not for the non-answer and non-substantive responses. No differences are observed between optimised and not optimised versions for smartphones, except for the response time per character written.

Keywords

Web surveys Smartphones Mobile optimised questionnaires Open narrative questions Response time 

Notes

Acknowledgments

We are very grateful to the Netquest team for the support in planning and collecting the necessary data. We would also like to thank Daniele Toninelli for his help in designing the experiment and his very useful comments on a previous draft of this paper, and the anonymous reviewers who gave us many valuable suggestions.

References

  1. Buskirk, T.D., Andrus C.H.: Smart Surveys for Smartphone: Exploring Various Approaches for Conducting Online Mobile Surveys Via Smartphones. Survey Practice. http://surveypractice.wordpress.com/2012/02/21/smart-surveys-for-smart-phones/ (2012)
  2. Buskirk, T.D., Andrus, C.H.: Making mobile browser surveys smarter: results from a randomized experiment comparing online surveys completed via computer or smartphone. Field Methods 26(4), 322–342 (2014). doi: 10.1177/1525822X14526146 CrossRefGoogle Scholar
  3. Christian, L.M., Dillman, D.A.: The influence of graphical and symbolic language manipulations on responses to self-administered questions. Public Opin. Q. 68(1), 57–80 (2004)CrossRefGoogle Scholar
  4. Couper, M.P., Kennedy, C., Conrad, F.G., Tourangeau, R.: Designing input fields for non-narrative open-ended responses in web surveys. J. Off. Stat. 27(1), 65–85 (2011)Google Scholar
  5. Couper, M, Peterson G. : Exploring Why Mobile Web Surveys Take Longer. Presented at the General Online Research conference, 18 to 20 March 2015 in Cologne. https://conftool.gor.de/conftool15/index.php?page=browseSessions&form_session=26&presentations=show (2015)
  6. de Bruijne, M, Wijnant A.: Can mobile web surveys be taken on computers? a discussion on a multi-device survey design. Surv. Pract. 6(4) (2013)Google Scholar
  7. de Bruijne, M., Wijnant, A.: Mobile response in web panels. Soc. Sci. Comput. Rev. 32(6), 728–742 (2014a)CrossRefGoogle Scholar
  8. de Bruijne, M., Wijnant, A.: Improving response rates and questionnaire design for mobile web surveys. Public Opin. Q. 78(4), 951–962 (2014b). doi: 10.1093/poq/nfu046 CrossRefGoogle Scholar
  9. Emde, M., Fuchs M.: Using adaptive questionnaire design in open-ended questions: a field experiment. In: JSM Proceedings, Statistical Computing Section. American Statistical Association, Alexandria (2012)Google Scholar
  10. Emde, M., Fuchs, M.: Using interactive feedback to enhance response quality in Web surveys: The case of open-ended questions. Presentación GOR conferencia 2013 (Mannheim): http://conftool.gor.de/conftool13/index.php?page=browseSessions&presentations=show&form_session=14 (2013)
  11. Israel, G.D.: Effects of answer space size on responses to open-ended questions in mail surveys. J. Off. Stat. 26(2), 271–285 (2010)Google Scholar
  12. Kaikkonen, A.: Mobile internet: past, present, and the future. Int. J. Mob. Hum. Comput. Interact. 1, 29–44 (2009)CrossRefGoogle Scholar
  13. Lambert, A.D., Miller, A.L.: Living with smartphones: does completion device affect survey responses? Res. High Educ. 56, 166–177 (2015). doi: 10.1007/s11162-014-9354-7 CrossRefGoogle Scholar
  14. Mavletova, A.: Data quality in PC and mobile web surveys. Soc. Sci. Comput. Rev. 31(4), 725–743 (2013)CrossRefGoogle Scholar
  15. Mavletova, A., Couper M.P.: Sensitive topics in pc web and mobile web surveys: is there a difference? Surv. Res. Methods 7(3):191–205. https://ojs.ub.uni-konstanz.de/srm/article/view/5751 (2013)
  16. Occhipinti, S.: Mixed modelling using stata. Workshop for GSBRC. http://www.griffith.edu.au/__data/assets/pdf_file/0011/439346/Stata_mixed_intro-1.pdf (2012)
  17. Peterson, G.: Unintended mobile respondents. Paper presented at CASRO technology conference, New York. http://c.ymcdn.com/sites/www.casro.org/resource/collection/D0686718-163A-4AF4-A0BB-8F599F573714/Gregg_Peterson_-_Market_Strategies.pdf (2012)
  18. Peytchev, A., Hill, C.A.: Experiments in mobile web survey design: similarities to other modes and unique considerations. Soc. Sci. Comput. Rev. 28(3), 319–335 (2010)CrossRefGoogle Scholar
  19. Revilla, M., Toninelli, D., Ochoa, C., Loewe, G.: Do online access panels really need to allow and adapt surveys to mobile devices? Internet Res. (fothcoming)Google Scholar
  20. Smyth, J.D., Dillman, D.A., Christian, L.M., Mcbride, M.: Open-ended questions in web surveys. Can increasing the size of answer boxes and providing extra verbal instructions improve response quality? Public Opin. Q. 73(2), 325–337 (2009)CrossRefGoogle Scholar
  21. Toepoel, V., Lugtig, P.: What happens if you offer a mobile option to your web panel? Evidence from a probability-based panel of Internet users. Soc. Sci. Comput. Rev. 32(4), 1–17 (2014)CrossRefGoogle Scholar
  22. Wells, T., Bailey, J.T., Link, M.W.: Filling the void: Gaining a better understanding of tablet-based surveys. Surv. Pract. 6 (2013)Google Scholar
  23. Wells, T., Bailey, J.T., Link, M.W.: Comparison of smartphone and online computer survey administration. Soc. Sci. Comput. Rev. 32(2), 238–255 (2014)CrossRefGoogle Scholar
  24. Zahariev, M., Ferneyhough, C., Ryan, C.: Best practices in mobile research. Paper presented at ESOMAR Online Research, Chicago, Oct 26–28 (2009)Google Scholar
  25. Zuell, C., Menold, N., Körber, S.: The influence of the answer box size on item nonresponse to open-ended questions in a web survey. Soc. Sci. Comput. Rev. 33(1), 115–122 (2015). doi: 10.1177/0894439314528091 CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2015

Authors and Affiliations

  1. 1.RECSM-Universitat Pompeu FabraBarcelonaSpain
  2. 2.NetquestBarcelonaSpain

Personalised recommendations