Experience Sampling Response Modes: Comparing Voice and Online Surveys
When conducting experience sampling studies, one important decision that researchers must make is the method by which surveys are administered. Dozens of reviews and recommendations cover types of response modes, including paper and pencil, online survey, and interactive voice response. However, few studies have empirically tested differences across response methods, and no studies have compared online surveys to interactive voice response surveys. Using a time-based experience sampling design, the present study investigates differences in compliance, data quality, and participant burden when using interactive voice response and online surveys. Results indicate no differences in terms of compliance rates and number of responses between the two methods. Interactive voice response produced lengthier qualitative responses, although there were no differences in the clarity of qualitative responses. Finally, online surveys may alleviate time burden, particularly as the number of items increases.
KeywordsExperience sampling Ecological momentary assessment Interactive voice response Electronic communication Online survey
The authors would like to acknowledge funding sources for this work: The University of South Florida Office for Undergraduate Research’s Research Scholarship, and the University of South Florida Psychology Department’s Walvoord Verizon Wireless Work-Family Research Endowment. This work was also supported in part by the Sunshine Education and Research Center at the University of South Florida. The Center is supported by Training Grant No. T42-OH008438 from the Centers for Disease Control and Prevention/National Institute for Occupational Safety and Health (NIOSH). Additionally, we would like to acknowledge Pamela Smith for her leadership in directing recruitment and data collection, as well as Andrea Bevill, LeVonte Brooks, Jake Mathwich, Dylan Reeves, Maryam Romagosa, Neal Vogel, and Hyeyoung Yoon for their assistance with recruitment and data collection, and Ryan Nguyen for his help with coding. Finally, we thank the editor and two anonymous reviewers for their constructive suggestions.
- American Heart Association. (2017). Blood pressure vs. heart rate (pulse). Retrieved from http://www.heart.org/HEARTORG/Conditions/HighBloodPressure/GettheFactsAboutHighBloodPressure/Blood-Pressure-vs-Heart-Rate-Pulse_UCM_301804_Article.jsp#.WuKHMsgvxPY.
- Bliese, P. (2016). Multilevel: multilevel functions. R package version 2.6. https://CRAN.R-project.org/package=multilevel.
- Bolger, N., Davis, A., & Rafaeli, E. (2003). Diary methods: capturing life as it is lived. Annual Review of Psychology, 54, 579–616. https://doi.org/10.1146/annurev.psych.54.101601.145030.CrossRefPubMedGoogle Scholar
- Broderick, J. E., & Stone, A. A. (2006). Paper and electronic diaries: too early for conclusions on compliance rates and their effects--comment on Green, Rafaeli, Bolger, Shrout, and Reis (2006). Psychological Methods, 11, 106–111. https://doi.org/10.1037/1082-989X.11.1.106.CrossRefPubMedGoogle Scholar
- Collins, R. L., Kashdan, T. B., & Gollnisch, G. (2003). The feasibility of using cellular phones to collect ecological momentary assessment data: application to alcohol consumption. Experimental and Clinical Psychopharmacology, 11, 73–78. https://doi.org/10.1037/1064-12184.108.40.206.CrossRefPubMedGoogle Scholar
- Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: applications and data analysis methods (2nd ed.). Thousand Oaks: Sage Publications, Inc..Google Scholar
- Shiffman, S. (2007). Designing protocols for ecological momentary assessment. In A. Stone, S. Shiffman, A. Atienza, & L. Nebeling (Eds.), The science of real-time data capture: self-reports in health research (pp. 27–53). New York: Oxford University Press.Google Scholar
- Shiffman, S., Stone, A. A., & Hufford, M. R. (2008). Ecological momentary assessment. Annual Review of Clinical Psychology, 4, 1–32. https://doi.org/10.1146/annurev.clinpsy.3.022806.091415.CrossRefPubMedGoogle Scholar
- Weiss, H. M., & Rupp, D. E. (2011). Experiencing work: an essay on a person-centric work psychology. Industrial and Organizational Psychology, 4, 83–97.Google Scholar