Abstract
In a study of the American Community Survey online instrument, we assessed how people answered questions about themselves and other individuals living in their household using eye-tracking data and other qualitative measures. This paper focuses on the number of fixations (whether participants looked at specific areas of the screen), fixation duration (how long participants looked at the questions and answers), and number of unique visits (whether participants rechecked the question and answer options). Results showed that for age, date of birth and employment duty questions participants had more fixations and unique visit counts, and spent more time on the screen when answering about unrelated members of their household than when answering about themselves. Differing eye movements for proxy reporting suggest that answering some survey questions for other unrelated people poses more burden on respondents than answering about oneself. However, not all questions showed this tendency, so eye tracking alone is not enough to detect burden.
Chapter PDF
Similar content being viewed by others
References
Redline, C., Lankford, C.: Eye Movement Analysis: A New Tool for Evaluation the design of visually administered instruments (paper and web). In: Proceedings from the American Association for Public Opinion Research Annual Conference, Montreal (2001)
Olmsted-Hawala, E., Holland, T., Quach, V.: Usability testing. In: Romano Bergstrom, J., Schall, A. (eds.) Eye Tracking in User Experience Design. Elsevier Press (2014)
Galesic, M., Tourangeau, R., Couper, M.P., Conrad, F.G.: New insights on response order effects and other cognitive shortcuts in survey responding. Public Opinion Quarterly 72(5), 892–913 (2008)
Jarrett, C., Romano Bergstrom, J.: Forms and Surveys. In: Romano Bergstrom, J., Schall, A. (eds.) Eye Tracking in User Experience Design. Elsevier Press (2014)
Moore, J.: Self/Proxy response status and survey response quality. Journal of Official Statistics 4(2), 155–172 (1998)
Hays, R., Vickery, B., Hermann, K., Perrine, J., Cramer, K., Meador, K., Spritzer, K., Devinsky, O.: Agreement between self reports and proxy reports of quality of life in epilepsy patients. Quality of Life Research 4(2), 159–168 (1995)
Palmer, L., Johnston, S., Rousculp, M., Chu, B., Nichol, K., Mahadevia, J.: Agreement between Internet-Based Self- and Proxy-Reported Health Care Resource Utilization and Administrative Health Care Claims. Value in Health 15(3), 458–465 (2012)
Kojetin, B., Mathiowetz, N.: The effects of self and proxy response status on the reporting of race and ethnicity. In: Proceedings from the American Association for Public Opinion Research Annual Conference (1998)
Bassett, S.S., Magaziner, J., Hebel, J.: Reliability of proxy response on mental health indices for aged, community-dwelling women. Psychology of Aging 5(1), 127–132 (1990)
Tamborini, H., Kim, C.: Are proxy interviews associated with biased earnings reports? Marital status and gender effects of proxy. Social Science Research 42(2), 499–512 (2013)
Reynolds, J., Wenger, J.: He said, she said: the gender wage gap according to self and proxy reports in the current population survey. Social Science Research 41(2), 392–411 (2012)
Kojetin, B., Mullin, P.: The Quality of Proxy Reports on the Current Population Survey (CPS). In: Proceedings of the Section on Survey Research Methods, American Statistical Association, pp. 1110–1115 (1995)
Looker, E.: Accuracy of proxy reports of parental status characteristics. Sociology of Education 62(4) (1989)
Ehmke, C., Wilson, S.: Identifying Web usability problems from eye-tracking data. In: Proceedings of HCI (2007)
Poole, A., Ball, L.: Eye tracking in human-computer interaction and usability research. In: Ghaoui, C. (ed.) Encyclopedia of human computer interaction, pp. 211–219. Idea Group, Pennsylvania (2005)
Horwitz, R., Tancreto, J., Zelenak, M.F., Davis, M.: Using Paradata to Identify Potential Issues and Trends in the American Community Survey Internet Instrument. U.S. Census Bureau: 2013 American Community Survey Research and Evaluation Report Memorandum Series #ACS13-RER-01 (2013), http://www.census.gov/acs/www/Downloads/library/2013/2013_Horwitz_02.pdf (accessed January 28, 2014)
Chin, J.P., Diehl, V.A., Norman, K.L.: Development of an instrument measuring user satisfaction of the human-computer interface. In: Proceedings of SIGCHI 1988, pp. 213–218 (1988)
Tobii Studio version 3.1.3.6615-RC (2012), http://www.tobii.com/en/eye-tracking-research/global/
Bradshaw, B., Akers, D.: Age heaping in the 1960 Census of Population. US Census Bureau memorandum (1962)
Nielsen, J.: Why You Only Need to Test With 5 Users. Alertbox, http://www.useit.com (March 19, 2000)
Faulkner, L.: Beyond the five-user assumption: Benefits of increased sample sizes in usability testing. Behavior Research Methods, Instruments, & Computers 35(3), 379–383 (2003)
Goldberg, J., Kotval, X.: Computer interface evaluation using eye movements: Methods and constructs. International Journal of Industrial Ergonomics 24(6), 631–645 (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Additional information
Disclaimer: This report is released to inform interested parties of research and to encourage discussion. Any views expressed on the methodological issues are those of the authors and not necessarily those of the U.S. Census Bureau.
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Olmsted-Hawala, E., Holland, T., Nichols, E. (2014). Answers for Self and Proxy – Using Eye Tracking to Uncover Respondent Burden and Usability Issues in Online Questionnaires. In: Stephanidis, C., Antona, M. (eds) Universal Access in Human-Computer Interaction. Design for All and Accessibility Practice. UAHCI 2014. Lecture Notes in Computer Science, vol 8516. Springer, Cham. https://doi.org/10.1007/978-3-319-07509-9_56
Download citation
DOI: https://doi.org/10.1007/978-3-319-07509-9_56
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-07508-2
Online ISBN: 978-3-319-07509-9
eBook Packages: Computer ScienceComputer Science (R0)