Abstract
Today’s personal computers enable complex forms of user interaction. Unlike older mainframe computers that required batch processing, personal computers enable real-time user control on a one-to-one basis. Such user interaction involves mixed initiative, logic, language and pointing gestures, features reminiscent of interaction with another human. Yet there are also major differences between computer interaction and human interaction, such as computers’ inability to stray from scripts or to adapt to the idiosyncrasies of particular recipients or situations. Given these similarities and differences, can we study computer interaction using methods similar to those for studying human interaction? If so, are the findings from the analysis of human interaction also useful in understanding computer interaction? In this paper, we explore these questions and outline a novel methodological approach for examining human-computer interaction, which we call “computer interaction analysis.” We build on earlier approaches to human interaction with a computer and adapt them to the latest technologies for computer screen capture and eye tracking. In doing so, we propose a new transcription notation scheme that is designed to represent the interweaving streams of input actions, display events and eye movements. Finally we demonstrate the approach with concrete examples involving the phenomena of placeholding, repair and referential practices.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
We are of the school that recognizes conversation analysis as a form of ethnomethodological inquiry. While some (Lynch 1993) attempt to drive a wedge between ethnomethodology and conversation analysis, we adopt the more inclusive view, following Maynard and Clayman (1991), that ethnomethodology and conversation analysis belong to the same family of approaches despite the diversity across them.
Suchman’s pairs of users are reminiscent of Friedrich Schrecker’s analysis of how interaction between a pair of people can make the work of a chemistry experiment visible to an analyst (see Lynch 1993, p. 200, note 97).
Actually verbal requests are the exception in shoe stores, which are spatially organized to enable customers to point at samples so they do not need to provide names or descriptions.
References
Card, S. K., Pirolli, P., Wege, M. V. D., Morrison, J. B., Reeder, R. W., Schraedley, P. K., and Boshart, J. (2001). Information Scent as a Driver of Web Behavior Graphs: Results of a Protocol Analysis Method of Web Usability. In Proceedings of SIGCHI’01, New York: ACM. Pp. 498–505.
Crabtree, A. (2003). Designing collaborative systems: A practical guide to ethnography. London: Springer.
Frohlich, D. M., & Luff, P. (1990). Applying the technology o f conversation to the technology for conversation. In P. Luff, G. N. Gilbert, & D. M. Frohlich (Eds.), Computers and conversation (pp. 187–220). London: Academic.
Frohlich, D. M., Drew, P., & Monk, A. (1993). The management of repair in human computer interaction. Bristol: Internal Report, Hewlett-Packard Company.
Garcia, A., & Jacobs, J. (1999). The eyes of the beholder: understanding the turn-taking system in quasi-synchronous CMC. Research on Language and Social Interaction, 32(4), 337–367.
Garfinkel, H. (1967). Studies in ethnomethodology. Englewood: Prentice-Hall.
Goldberg, H. J., & Kotval, X. P. (1999). Computer interface evaluation using eye movements: methods and constructs. International Journal of Industrial Ergonomics, 24, 631–645.
Goodwin, C. (1979). The interactive construction of a sentence in natural conversation. In G. Psathas (Ed.), Everyday language: Studies in ethnomethodology (pp. 97–121). New York: Irvington.
Guo, Q., & Agichtein, E. (2010). Towards Predicting Web Searcher Gaze Position from Mouse Movements. In Proc. of the 28th ACM Conference on Human Factors in Computing Systems (CHI) Works-In-Progress.
Heritage, J. (1984). Garfinkel and ethnomethodology. Cambridge: Polity.
Hindmarsh, J., & Heath, C. (2000). Embodied reference: a study of deixis in workplace interaction. Journal of Pragmatics, 32, 1855–78.
Hindmarsh, J., Fraser, M., Heath, C., & Benford, S. (2001). Virtually Missing the Point: Configuring CVEs for Object-Focused Interaction. In Collaborative Virtual Environments, Springer, September 2001, pp. 115–133.
Hornoff, A., Cavender, A., & Hoselton, R. (2004). Eyedraw: a system for drawing pictures with eye movements. Proceedings of the 6th international ACM SIGACCESS conference on Computers and accessibility, 86–93.
Jefferson, G. (2004). Glossary of transcript symbols with an introduction. In G. Lerner (Ed.), Conversation analysis: Studies from the first generation (pp. 13–31). Amsterdam: John Benjamins.
Just, M. A., & Carpenter, P. A. (1976). Eye fixations and cognitive processes. Cognitive Psychology, 8, 441–480.
Kern, D., Marshall, P., & Schmidt, A. (2010). Gazemarks - Gaze-Based Visual Placeholders to Ease Attention Switching. Proceedings of the 28th ACM Conference on Human Factors in Computing Systems (CHI ’10). Atlanta, Georgia, USA.
Kirkegaard, K. M., Jensen, J. M., & Larsen, B. (2007). A Qualitative Look at Eye-tracking for Implicit Relevance Feedback. Proceedings of the CIR’07 Workshop on Context-Based Information Retrieval in conjunction with CONTEXT-07, Roskilde, Denmark, 20 August.
Luff, P., & Heath, C. (1993a). The practicalities of menu use: improvisation in a screen-based activity. Journal of Intelligent Systems, 3, 251–296.
Luff, P., & Heath, C. (1993b). System Use and Social Organisation: Observations on human-computer interaction in an architectural practice. In G. Button (Ed.), Technology in working order: Studies of work, interaction and technology (pp. 184–210). London: Routledge.
Luff, P., & Heath, C. (2000). The collaborative production of computer commands in command and control. International Journal of Human-Computer Studies, 52(4), 669–699.
Lynch, M. (1993). Scientific practice and ordinary action: Ethnomethodology and social studies of science. New York: Cambridge University Press.
MacKenzie, I. S. (2010). An eye on input: research challenges in using the eye for computer input control. Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications, 11–12.
Maynard, D. W., & Clayman, S. E. (1991). The diversity of ethnomethodology. In W. R. Scott & J. Blake (Eds.), Annual review of sociology (Vol. 17, pp. 385–418). Palo Alto: Annual Reviews, Inc.
McIlvenny, P. (1990). Communicative action and computers: Re-embodying conversation analysis? In P. Luff, N. Gilbert, & D. Frohlich (Eds.), Computers and conversation (pp. 91–132). London: Academic.
Moore, R. J. (2008). When names fail: referential practice in face-to-face service encounters. Language in Society, 37(3), 385–413.
Moore, R. J. (2011). “A Name is Worth a Thousand Pictures: Referential Practice in Search-Engine Interactions.” Presented at the American Sociological Association Annual Meetings, Las Vegas, NV, August 20.
Moore, R. J., Ducheneaut, N., & Nickell, E. (2007). Doing virtually nothing: awareness and accountability in massively multiplayer online worlds. Computer Supported Cooperative Work, 16, 265–305.
Moore, R. J., Gathman, C., Ducheneaut, N., & Nickell, E. (2007b). Coordinating joint activity in avatar-mediated interaction. In Proceedings of CHI 2007 (pp. 21–30). New York: ACM.
Moore, R. J., Churchill, E. F., & Kantamneni, R. G. P. (2011). Three sequential positions of query repair in interactions with internet search engines, In Proceedings of CSCW 2011 (pp. 415–424). New York: ACM.
Poole, A., & Ball, L. J. (2005). Eye tracking in human-computer interaction and usability research. In C. Ghaoui (Ed.), Encyclopedia of human computer interaction (pp. 211–219). Pennsylvania: Idea Group.
Raudaskoski, P. (1990). Repair work in human-computer interaction - a conversation analytic perspective. In Luff, P., Gilbert, N. & Frohlich, D. (eds.), Computers and Conversation Analysis. Academic Press: 151–171.
Reeves, B., & Nass, C. (1996). The media equation: How people treat computers, television, and new media like real people and places. Cambridge: Cambridge University Press.
Rodden, K., Fu, X., Aula, A., & Spiro, I. (2008). Eye-mouse coordination patterns on web search results pages, Extended Abstracts of ACM CHI.
Sacks, H. (1984). Notes on methodology. In J. M. Atkinson & J. C. Heritage (Eds.), Structures in social action: Studies in conversation analysis (pp. 21–27). Cambridge: Cambridge University Press.
Schegloff, E. A. (1992a). Repair after next turn: the last structurally provided defense of intersubjectivity in conversation. American Journal of Sociology, 97(5), 1295–345.
Schegloff, E. A. (1992b). In another context. In A. Duranti & C. Goodwin (Eds.), Rethinking context: Language as an interactive phenomenon (pp. 193–227). Cambridge: Cambridge University Press.
Schegloff, E. A., Jefferson, G., & Sacks, H. (1977). The preference for self-correction in the organization of repair in conversation. Language, 53(2), 361–382.
Suchman, L. (1987). Plans and situated actions: The problem of human-machine communication. New York: Cambridge University Press.
Tatar, D., Foster, G., & Bobrow, D. (1991). Design for Conversation: lessons from Cognoter. International Journal of Man-Machine Studies , 34 pp. 185–209. Reprinted in Ronald M. Baecker (Ed.) Readings in Groupware and Computer-Supported Cooperative Work. 1993. Morgan Kaufman (San Mateo, CA).
Turing, A. M. (1950). Computing machinery and intelligence. Mind, 59(236), 433–61.
Turkle, S. (2005). The second self: Computers and the human spirit. Cambridge: MIT.
Whalen, J., & Vinkhuyzen, E. (2000). Expert systems in (inter)action: Diagnosing document machine problems over the telephone. In P. Luff, J. Hindmarsh, & C. Heath (Eds.), Workplace studies: Recovering work practice and informing systems design (pp. 92–140). Cambridge: Cambridge University Press.
Winograd, T., & Flores, F. (1986). Understanding computers and cognition: A new foundation for design. Menlo Park California: Addison-Wesley Publishing Company Inc.
Wittgenstein, L. (1953). Philosophical Investigations. (trans.) G.E.M. Anscombe. New York: MacMillan.
Wright, R. D., & Ward, L. M. (2008). Orienting of attention. New York: Oxford University Press.
Zimmerman, D. H. (1970). The practicalities of rule use. In J. Douglas (Ed.), Understanding everyday life (pp. 221–238). Chicago: Aldine.
Acknowledgements
We would like to thank Prasad Kantamneni of Yahoo!’s Human Perception Center Of Excellence for access to existing eye-tracking data, as well as his guidance in collecting new data.
Author information
Authors and Affiliations
Corresponding author
Appendices
Appendix A: Computer Interaction Analysis Transcription Notation (Description-Level)
Appendix B: Fixation-Level Transcription Notation
Rights and permissions
About this article
Cite this article
Moore, R.J., Churchill, E.F. Computer Interaction Analysis: Toward an Empirical Approach to Understanding User Practice and Eye Gaze in GUI-Based Interaction. Comput Supported Coop Work 20, 497–528 (2011). https://doi.org/10.1007/s10606-011-9142-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10606-011-9142-2