Abstract
A progress indicator (PI) is often used to inform respondents of the task completion status of online surveys. When researchers conduct online surveys, reducing the dropout rate and the participants’ cognitive loads is important for improving the surveys’ efficiency. Although many studies have investigated the effect of using PIs with online surveys, it is unclear which PI design should be used to reduce the dropout rate and the participants’ perceived task load. Moreover, even in the Web content accessibility guidelines (WCAG), which should be followed in the building of websites to improve accessibility, the design guidelines for PIs are vague compared to those for other graphic elements. We noted that PIs have various designs and then created 25 types of PIs by combining design factors, labels, and bar graphs. We conducted an online survey through mTurk and collected 1,948 participants’ data. The online survey study results showed that the PI’s label design had a significant effect on the number of items the participants answered. In addition, we found that the respondents’ mental load changed significantly according to the combination of bar graph and label designs of the PI. This study provides strong support for designing the way a PI communicates the task process without reducing the speed of progress, which encourages respondents to answer more items and decreases their mental loads in online survey environments.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Becker, L.J.: Joint effect of feedback and goal setting on performance: A field study of residential energy conservation. J. Appl. Psychol. 63(4), 428 (1978)
Buhrmester, M., Kwang, T., Gosling, S.D.: Amazon’s mechanical turk: a new source of inexpensive, yet high-quality data? (2016)
Chang, J.C., Amershi, S., Kamar, E.: Revolt: collaborative crowdsourcing for labeling machine learning datasets. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. CHI 2017, pp. 2334–2346. Association for Computing Machinery, Denver, Colorado, USA (2017). https://doi.org/10.1145/3025453.3026044, isbn: 9781450346559
Clark, J.M., Paivio, A.: Dual coding theory and education. Educ. Psychol. Rev. 3(3), 149–210 (1991). https://doi.org/10.1007/BF01320076
Conrad, F., et al.: Impact of progress feedback on task completion: first impressions matter. In: CHI 2005 Extended Abstracts on Human Factors in Computing Systems. CHI EA 2005, pp. 1921–1924. Association for Computing Machinery, Portland, OR, USA (2005). https://doi.org/10.1145/1056808.1057057, isbn: 1595930027
Conrad, F.G., et al.: The impact of progress indicators on task completion. Interact. Comput. 22(5), 417–427 (2010)
Couper, M.P., Traugott, M.W., Lamias, M.J.: Web survey design and administration*. Public Opin. Q. 65(2), 230–253 (2001). https://doi.org/10.1086/322199. eprint: https://academic.oup.com/poq/article-pdf/65/2/230/5307018/650230.pdf. issn: 0033–362X
Couper, M.P.: Discussion: what can CAI learn from HCI. In: Proceedings of the Seminar on New Directions in Statistical Methodology, pp. 363–377 (1994)
Crawford, S.D., Couper, M.P., Mark, Lamias, J.: Web surveys: perceptions of Burden. Soc. Sci. Comput. Rev. 19(2), 146–162 (2001). https://doi.org/10.1177/089443930101900202
Dillman, D.A., Smyth, J.D., Christian, L.M.: Internet, Phone, Mail, and Mixed-Mode Surveys: the Tailored Design Method. Wiley, Hoboken (2014)
Dillman, D.A., Tortora, R.D., Bowker, D.: Principles for constructing web surveys. J. Meetings Am. Stat. Assoc. 64, 1–16 (1998)
Drutsa, A., et al.: Crowdsourcing practice for efficient data labeling: aggregation, incremental relabeling, and pricing. In: Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data. SIGMOD 2020, pp. 2623–2627. Association for Computing Machinery, Portland, OR, USA (2020). https://doi.org/10.1145/3318464.3383127, isbn: 9781450367356
Egeln, L.S., Joseph, J.A.: Shopping cart abandonment in online shopping. Atlantic Mark. J. 1(1), 1 (2012)
Galesic, M.: Dropouts on the web: effects of interest and burden experienced during an online survey. J. Official Stat. 22(2), 313 (2006)
Gleicher, M., et al.: Perception of average value in multiclass scatterplots. IEEE Trans. Vis. Comput. Graph. 19(12), 2316–2325 (2013)
Google. Google Form. https://www.google.com/forms/about/. Accessed 2020[a]
. Google material design guideline for progress indicators. https://material.io/components/progress-indicators. Accessed 2020[b]
Hart, S.G.: Nasa-task load index (NASA-TLX); 20 years later. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 50, no. 9, pp. 904–908 (2006). https://doi.org/10.1177/154193120605000909
Healey, B., Macpherson, T., Kuijten, B.: An empirical evaluation of three web survey design principles. Mark. Bull. 16 (2005)
Heerwegh, D.: Using progress indicators in web surveys. In: 59th AAPOR conference, Phoenix, Arizona (2004)
Heerwegh, D., Loosveldt, G.: An experimental study on the effects of personalization, survey length statements, progress indicators, and survey sponsor logos in web surveys. J. Official Stat. 22(2), 191 (2006)
Kaczmirek, L.: Human-survey interaction : usability and nonresponse in online surveys. English. an enhanced book version of this work is published by Herbert von Halem Verlag. Ph.D. thesis (2008). http://www.kaczmirek.de/book2008, https://madoc.bib.uni-mannheim.de/2150/
Soomin, K., Lee, J., Gweon, G.: Comparing data from Chatbot and web surveys: effects of platform and conversational style on survey response quality. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. CHI 2019, pp. 1–12. Association for Computing Machinery, Glasgow, Scotland, UK (2019). https://doi.org/10.1145/3290605.3300316, isbn: 9781450359702
Komatsu, T., Yamada, S.: Exploring auditory information to change users’ perception of time passing as shorter. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. CHI 2020, pp. 1–12. Association for Computing Machinery, Honolulu, HI, USA (2020). https://doi.org/10.1145/3313831.3376157, isbn: 9781450367080
Kukar-Kinney, M., Close, A.G.: The determinants of consumers’ online shopping cart abandonment. J. Acad. Mark. Sci. 38(2), 240–250 (2010). https://doi.org/10.1007/s11747-009-0141-5
Liu, M., Wronski, L.: Examining completion rates in web surveys via over 25,000 real-world surveys. Soc. Sci. Comput. Rev. 36(1), 116–124 (2018). https://doi.org/10.1177/0894439317695581
Locke, E.A.: The motivation to work: what we know. In: Advances in motivation and achievement, vol. 10, no. 2, pp. 375–412 (1997)
Locke, E.A., Latham, G.P.: Work motivation and satisfaction: light at the end of the tunnel. Psychol. Sci. 1(4), 240–246 (1990). https://doi.org/10.1111/j.1467-9280.1990.tb00207.x
Matejka, J., et al.: The effect of visual appearance on the performance of continuous sliders and visual analogue scales. In: CHI 2016, pp. 5421–5432. Association for Computing Machinery, San Jose, California, USA (2016). https://doi.org/10.1145/2858036.2858063, isbn: 9781450333627
Matzat, U., Snijders, C., van der Horst, W.: Effects of different types of progress indicators on dropout rates in web surveys. Soc. Psychol. 40(1), 43–52 (2009)
Monkey, Survey. Survey Monkey question examples (2020). https://www.surveymonkey.com/mp/survey-question-examples/. Accessed 2020
Müller, H., Sedley, A.: Designing surveys for HCI research. In: CHI EA 2015, pp. 2485–2486. Association for Computing Machinery, Seoul, Republic of Korea (2015). https://doi.org/10.1145/2702613.2706683
Müller, H., Sedley, A., Ferrall-Nunge, E.: Survey research in HCI. In: Olson, J.S., Kellogg, W.A. (eds.) Ways of Knowing in HCI, pp. 229–266. Springer, New York (2014). https://doi.org/10.1007/978-1-4939-0378-8_10
Myers, B.A.: The importance of percent-done progress indicators for computer-human interfaces. SIGCHI Bull. 16(4), 11–17 (1985). https://doi.org/10.1145/1165385.317459
Nielsen, J.: Usability Engineering. Morgan Kaufmann, Burlington (1994)
Nielsen, J., Rolf, M.: Heuristic evaluation of user inter-faces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI 1990, pp. 249–256. Association for Computing Machinery Seattle, Washington, USA (1990).https://doi.org/10.1145/97243.97281, isbn: 0201509326
Oviatt, S.: Human-centered design meets cognitive load theory: designing interfaces that help people think. In: Proceedings of the 14thACM International Conference on Multimedia. MM 2006, pp. 871–880. Association for Computing Machinery, Santa Barbara, CA, USA (2006). https://doi.org/10.1145/1180639.1180831, isbn: 1595934472
Paolacci, G., Chandler, J., Ipeirotis, P.G.: Running experiments on amazon mechanical Turk. Judgment Decis. Making 5(5), 411–419 (2010)
Ross, J., et al.: Who are the Crowdworkers? Shifting demographics in mechanical turk. In: CHI 2010 Extended Abstracts on Human Factors in Computing Systems. CHI EA 2010, pp. 2863–2872. Association for Computing Machinery, Atlanta, Georgia, USA (2010). https://doi.org/10.1145/1753846.1753873, isbn: 9781605589305
Sarraf, S., Tukibayeva, M.: Survey page length and progress indicators: what are their relationships to item nonresponse? New Dir. Inst. Res. 161, 83–97 (2014)
Smyth, J.D., et al.: Effects of using visual design principles to group response options in web surveys (2006)
Strang, H.R., Lawrence, E.C., Fowler, P.C.: Effects of assigned goal level and knowledge of results on arithmetic computation: a laboratory study. J. Appl. Psychol. 63(4), 446 (1978)
Streeter, S.B., et al.: Patient and plan characteristics affecting abandonment of oral oncolytic prescriptions. J. Oncol. Pract. 7(3S), 46s–51s (2011). https://doi.org/10.1200/JOP.2011.000316, PMID: 21886519
Tijdens, K.: Dropout rates and response times of an occupation search tree in a web survey. J. Official Stat. 30(1), 23–43 (2014). https://doi.org/10.2478/jos-2014-0002, https://content.sciendo.com/view/journals/jos/30/1/article-p23.xml
Tourangeau, R., Conrad, F.G., Couper, M.P.: The Science of Web Surveys. Oxford University Press, Oxford (2013)
Tukibayeva, M., Sarraf, S.: The relationships between survey page length, progress indicators, and item completion rates. Am. Educ. Res. Assoc. Annu. Meeting (2012)
Villar, A., Callegaro, M., Yang, Y.: Where am I? a meta-analysis of experiments on the effects of progress indicators for web surveys. Soc. Sci. Comput. Rev. 31(6), 744–762 (2013). https://doi.org/10.1177/0894439313497468
W3C. Web Content Accessibility Guidelines. https://www.w3.org/WAI/standards-guidelines/wcag/. Accessed 2020
Yentes, R.D., et al.: Effects of survey progress bars on data quality and enjoyment. In: 27th annual meeting of the Society for Industrial and Organizational Psychology (2012)
Yentes, R.D., et al.: Attention and data quality in online surveys: the role of survey length, progress bars, and time disclosure (2015)
Zhang, J., Wu, X., Sheng, V.S.: Learning from crowdsourced labeled data: a survey. Artif. Intell. Rev. 46(4), 543–576 (2016). https://doi.org/10.1007/s10462-016-9491-9
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Park, W., Lee, J., Lee, S. (2022). Understanding the Design Effects of Progress Indicators on Online Surveys. In: Bruyns, G., Wei, H. (eds) [ ] With Design: Reinventing Design Modes. IASDR 2021. Springer, Singapore. https://doi.org/10.1007/978-981-19-4472-7_92
Download citation
DOI: https://doi.org/10.1007/978-981-19-4472-7_92
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-4471-0
Online ISBN: 978-981-19-4472-7
eBook Packages: Literature, Cultural and Media StudiesLiterature, Cultural and Media Studies (R0)