Skip to main content
SpringerLink
Log in

A Computational Approach for Prediction of Problem-Solving Behavior Using Support Vector Machines and Eye-Tracking Data

  • Chapter
Eye Gaze in Intelligent User Interfaces

Abstract

Inference about high-level cognitive states during interaction is a fundamental task in building proactive intelligent systems that would allow effective offloading of mental operations to a computational architecture. We introduce an improved machine-learning pipeline able to predict user interactive behavior and performance using real-time eye-tracking. The inference is carried out using a support-vector machine (SVM) on a large set of features computed from eye movement data that are linked to concurrent high-level behavioral codes based on think aloud protocols. The differences between cognitive states can be inferred from overt visual attention patterns with accuracy over chance levels, although the overall accuracy is still low. The system can also classify and predict performance of the problem-solving users with up to 79 % accuracy. We suggest this prediction model as a universal approach for understanding of gaze in complex strategic behavior. The findings confirm that eye movement data carry important information about problem solving processes and that proactive systems can benefit from real-time monitoring of visual attention.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Anderson JR, Bothell D, Douglass S (2004) Eye movements do not reflect retrieval: limits of the eye-mind hypothesis. Psychol Sci 15:225–231

    Article  Google Scholar 

  • Bailey BP, Iqbal ST (2008) Understanding changes in mental workload during execution of goal-directed tasks and its application for interruption management. ACM Trans Comput-Hum Interact 14(4):1–28

    Article  Google Scholar 

  • Bednarik R (2005) Potentials of eye-movement tracking in adaptive systems. In: Proceedings of the fourth workshop on the evaluation of adaptive systems, held in conjunction with the 10th international conference on user modeling (UM’05), pp 1–8

    Google Scholar 

  • Bednarik R, Tukiainen M (2008) Temporal eye-tracking data: evolution of debugging strategies with multiple representations. In: Proceedings of the 2008 symposium on eye tracking research & applications. ACM, New York, pp 99–102

    Chapter  Google Scholar 

  • Bednarik R, Myller N, Sutinen E, Tukiainen M (2006) Analyzing individual differences in program comprehension. Technol Instr Cogn Learn 3(3/4):205

    Google Scholar 

  • Bednarik R, Gowases T, Tukiainen M (2009) Gaze interaction enhances problem solving: effects of dwell-time based, gaze-augmented, and mouse interaction on problem-solving strategies and user experience. J Eye Movement Res 3(1):1–10

    Google Scholar 

  • Bednarik R, Vrzakova H, Hradis M (2012) What you want to do next: a novel approach for intent prediction in gaze-based interaction. In: Proceedings of the 2012 symposium on eye-tracking research & applications, ETRA’12. ACM, New York

    Google Scholar 

  • Chang CC, Lin CJ (2011) LibSVM: a library for support vector machines. Science 2(3):1–39

    Google Scholar 

  • Conati C, Merten C (2007) Eye-tracking for user modeling in exploratory learning environments: an empirical evaluation. Knowl-Based Syst 20:557–574

    Article  Google Scholar 

  • Davies SP (2003) Initial and concurrent planning in solutions to well-structured problems. Q J Exp Psychol, A Hum Exp Psychol 56(7):1147–1164

    Article  Google Scholar 

  • Eivazi S, Bednarik R (2010) Inferring problem solving strategies using eye-tracking: system description and evaluation. In: Proceedings of the 10th Koli Calling international conference on computing education research, Koli Calling’10. ACM, New York, pp 55–61

    Chapter  Google Scholar 

  • Eivazi S, Bednarik R (2011) Predicting problem-solving behavior and performance levels from visual attention data. In: Proceedings of 2nd workshop on eye gaze in intelligent human machine interaction at IUI, pp 9–16

    Google Scholar 

  • Ericcson KA (1975) Instruction to verbalize as a means to study problem solving process with the 8-puzzle: a preliminary study. Department of Psychology, University of Stockholm

    Google Scholar 

  • Ericsson KA, Simon HA (1993) Protocol analysis: verbal reports as data revised edition. MIT Press, Cambridge

    Google Scholar 

  • Glöckner A, Herbold AK (2010) An eye-tracking study on information processing in risky decisions: evidence for compensatory strategies based on automatic processes. J Behav Decis Mak 41(1):71–98

    Google Scholar 

  • Goldberg JH, Kotval XP (1999) Computer interface evaluation using eye movements: methods and constructs. Int J Ind Ergon 24:631–645

    Article  Google Scholar 

  • Graf ABA, Borer S (2001) Normalization in support vector machines. In: Proceedings of the 23rd DAGM-symposium on pattern recognition. Springer, London, pp 277–282

    Google Scholar 

  • Hsu CW, Chang CC, Lin CJ (2003) A practical guide to support vector classification. Technical report, National Taiwan University

    Google Scholar 

  • Ishii R, Nakano YI (2008) Estimating user’s conversational engagement based on gaze behaviors. In: Proceedings of the 8th international conference on intelligent virtual agents (IVA’08), pp 200–207

    Google Scholar 

  • Just MA, Carpenter PA (1976) Eye fixations and cognitive processes. J Cogn Psychol 8:441–480

    Article  Google Scholar 

  • Kaller CP, Rahm B, Bolkenius K, Unterrainer JM (2009) Eye movements and visuospatial problem solving: identifying separable phases of complex cognition. Psychophysiology 46:818–830

    Article  Google Scholar 

  • Liang Y, Reyes ML, Lee JD (2007) Real-time detection of driver cognitive distraction using support vector machines. IEEE Trans Intell Transp Syst 8:340–350

    Article  Google Scholar 

  • Lipps M, Pelz JB (2004) Yarbus revisited: task-dependent oculomotor behavior. J Vis 4(8):115

    Article  Google Scholar 

  • Liu Y, Hsueh PY, Lai J, Sangin M, Nüssli MA, Dillenbourg P (2009) Who is the expert? Analyzing gaze data to predict expertise level in collaborative applications. In: Proceedings of the 2009 IEEE international conference on multimedia and expo

    Google Scholar 

  • Loboda TD, Brusilovsky P (2010) User-adaptive explanatory program visualization: evaluation and insights from eye movements. User Model User-Adapt Interact 20:191–226

    Article  Google Scholar 

  • Meyer D, Leischa F, Hornikb K (2003) The support vector machine under test. Neurocomputing 55:169–186

    Article  Google Scholar 

  • Morgan PL, Waldron SM, King SL, Patrick J (2007) Harder to access, better performance? The effects of information access cost on strategy and performance. In: Proceedings of the 2007 conference on human interface: part I. Springer, Berlin, pp 115–125

    Google Scholar 

  • O’Hara KP, Payne SJ (1998) The effects of operator implementation cost on planfulness of problem solving and learning. Cogn Psychol 35:34–70

    Article  Google Scholar 

  • Rayner K (1998) Eye movements in reading and information processing: 20 years of research. Psychol Bull 124(3):372–422

    Article  Google Scholar 

  • Salvucci DD (2001) An integrated model of eye movements and visual encoding. J Cogn Syst 1(4):201–220

    Article  Google Scholar 

  • Salvucci DD, Goldberg JH (2000) Identifying fixations and saccades in eye-tracking protocols. In: Proceedings of the 2000 symposium on eye tracking research & applications, ETRA’00. ACM, New York, pp 71–78

    Chapter  Google Scholar 

  • Simola J, Salojärvi J, Kojo I (2008) Using hidden Markov model to uncover processing states from eye movements in information search tasks. Cogn Syst Res 9(4):237–251

    Article  Google Scholar 

  • Smith JD, Graham TCN (2006) Use of eye movements for video game control. In: ACM advancements in computer entertainment technology (ACE’06). ACM, New York, article no. 20

    Google Scholar 

  • Surakka V, Illi M, Isokoski P (2003) Voluntary eye movements in human-computer interaction. North-Holland, Amsterdam, p 471 (Chap 22)

    Google Scholar 

  • van Someren MW, Barnard YF, Sandberg JAC (1994) The think aloud method: a practical guide to modelling cognitive processes. Academic Press, San Diego

    Google Scholar 

  • Velichkovsky BM (1999) From levels of processing to stratification of cognition: converging evidence from three domains of research. Benjamins, Amsterdam

    Google Scholar 

  • Vidal M, Bulling A, Gellersen H (2011) Analysing EOG signal features for the discrimination of eye movements with wearable devices. In: Proceedings of the 1st international workshop on pervasive eye tracking and mobile eye-based interaction, PETMEI’11. ACM, New York, pp 15–20

    Google Scholar 

  • Vrochidis S, Patras I, Kompatsiaris I (2011) An eye-tracking-based approach to facilitate interactive video search. In: Proceedings of the 1st ACM international conference on multimedia retrieval, ICMR’11. ACM, New York, pp 43:1–43:8

    Google Scholar 

  • Xu S, Jiang H, Lau FC (2008) Personalized online document, image and video recommendation via commodity eye-tracking. In: Proceedings of the 2008 ACM conference on recommender systems, RecSys’08. ACM, New York, pp 83–90

    Chapter  Google Scholar 

  • Xu S, Jiang H, Lau FC (2009) User-oriented document summarization through vision-based eye-tracking. In: Proceedings of the 14th international conference on intelligent user interfaces, IUI’09. ACM, New York, pp 7–16

    Google Scholar 

  • Yarbus AL (1967) Eye movements during perception of complex objects. Plenum, New York, pp 171–196 (Chap VII)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Eastern Finland, Yliopistokatu 2, P.O. Box 111, 80101, Joensuu, Finland

    Roman Bednarik, Shahram Eivazi & Hana Vrzakova

Authors
  1. Roman Bednarik
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shahram Eivazi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hana Vrzakova
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Roman Bednarik .

Editor information

Editors and Affiliations

  1. Dept. of Computer & Information Science, Seikei University, Kichijoji-kitamachi 3-3-1, Musashino-shi, 180-8633, Tokyo, Japan

    Yukiko I. Nakano

  2. Department of Computer Science, University of British Columbia, Main Mall 2366, Vancouver, V6T1Z4, British Columbia, Canada

    Cristina Conati

  3. Research & Development, AGT International, Hilpertstr. 35, Darmstadt, 64295, Germany

    Thomas Bader

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag London

About this chapter

Cite this chapter

Bednarik, R., Eivazi, S., Vrzakova, H. (2013). A Computational Approach for Prediction of Problem-Solving Behavior Using Support Vector Machines and Eye-Tracking Data. In: Nakano, Y., Conati, C., Bader, T. (eds) Eye Gaze in Intelligent User Interfaces. Springer, London. https://doi.org/10.1007/978-1-4471-4784-8_7

Download citation

  • DOI: https://doi.org/10.1007/978-1-4471-4784-8_7

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-4471-4783-1

  • Online ISBN: 978-1-4471-4784-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation