Eye gaze patterns in emotional pictures

  • Antonio Lanatà
  • Gaetano Valenza
  • Enzo Pasquale Scilingo
Original Research


This paper reports on a preliminary study aiming at investigating the eye gaze pattern and pupil size variation to discriminate emotional states induced by looking at pictures having different arousal content. A wearable and wireless eye gaze tracking system, hereinafter called HATCAM, which was able to robustly detect eye tracking and pupil area was used. A group of ten volunteers was presented with a set of neutral and arousal pictures extracted from the International Affective Picture System according to an ad-hoc experimental protocol. A set of features was extracted from eye gaze patterns and pupil size variations and used to classify the two classes of pictures. Although preliminary, results are very promising for affective computing applications.


Eye gaze tracking Wearable systems Affective computing Emotions Pattern recognition 



This research is partially supported by the EU Commission under contract FP7-ICT-247777 Psyche, and partially supported by the EU Commission under contract FP7-ICT-258749 CEEDs.


  1. Andreassi JL (2006) Psychophysiology Human behavior and physiological response. Lawrence Erlbaum Assoc Inc., ISBN: 0805849513Google Scholar
  2. Armato A, Lanatà A, Scilingo EP (2011) Comparitive study on photometric normalization algorithms for an innovative, robust and real-time eye gaze tracker. J Real-Time Image Process 1–13. doi: 10.1007/s11554-011-0217-6
  3. ASL (2009) Applied science laboratories:mobile eye and eye tracking services.
  4. Beatty J, Lucero-Wagoner B (2000) The pupillary system. Cambridge University Press, CambridgeGoogle Scholar
  5. Bennett N, Burridge R, Saito N (2002) A method to detect and characterize ellipses using the Hough transform. IEEE Trans Pattern Anal Mach Intell 21(7):652–657CrossRefGoogle Scholar
  6. Bradley MM, Sabatinelli D, Lang PJ, Fitzsimmons JR, King W, Desai P (2003) Activation of the visual cortex in motivated attention. Behav Neurosci 117(2):369CrossRefGoogle Scholar
  7. Calvo MG, Lang PJ (2004) Gaze patterns when looking at emotional pictures: motivationally biased attention. Motiv Emot 28(3):221–243CrossRefGoogle Scholar
  8. Chen W, Er MJ, Wu S (2006) Illumination compensation and normalization for robust face recognition using discrete cosine transform in logarithm domain. IEEE Trans Syst Man Cybernet Part B Cybernet 36(2):458–466CrossRefGoogle Scholar
  9. Christianson SÅ, Loftus EF, Hoffman H, Loftus GR (1991) Eye fixations and memory for emotional events. J Exp Psychol Learn Memory Cognit 17(4):693CrossRefGoogle Scholar
  10. Damasio AR (2000) Descartes’ error: emotion, reason, and the human brain. Quill, New York, ISBN: 0380726475Google Scholar
  11. Darwin C, Ekman P, Prodger P (2002) The expression of the emotions in man and animals. Oxford University Press, Oxford, ISBN: 0195158067Google Scholar
  12. Duda R, Hart P, Stork D (2001) Pattern classification, 2nd edn. Wiley, New YorkGoogle Scholar
  13. Ekman P, Levenson RW, Friesen WV (1983) Autonomic nervous system activity distinguishes among emotions. Science, ISSN: 0036-8075Google Scholar
  14. Fitzgibbon A, Pilu M, Fisher RB (2002) Direct least square fitting of ellipses. IEEE Trans Pattern Anal Mach Intell 21(5):476–480CrossRefGoogle Scholar
  15. Forsyth DA, Ponce J (2002) Computer vision: a modern approach, Professional Technical Reference. Prentice Hall, Upper Saddle RiverGoogle Scholar
  16. Friedman J, Hastie T, Tibshirani R (2000) Additive logistic regression: A statistical view of boosting. Annals Stat 38(2):337–374MathSciNetCrossRefGoogle Scholar
  17. Gross JJ, Muñoz RF (1995) Emotion regulation and mental health. Clin Psychol Sci Pract 2(2):151–164CrossRefGoogle Scholar
  18. Heijden F, Duin R, Ridder D, Tax D (2004) Classification, parameter estimation and state estimation. Wiley, New YorkGoogle Scholar
  19. Hess EH (1972) Pupillometrics: a method of studying mental, emotional and sensory processes, handbook of psychophysiology. Holt, Rinehart & Winston, New York, pp 91–531Google Scholar
  20. Hess EH, Petrovich SB (1987) Pupillary behavior in communication: nonverbal behaviour and communication. Erlbaum, Hillsdale, pp 327–348Google Scholar
  21. ISCAN (2007) Inc. Eye and target tracking solutions.
  22. Isen AM, Means B (1983) The influence of positive affect on decision-making strategy. Soc Cognit 2(1):18–31CrossRefGoogle Scholar
  23. Jain AK, Duin RPW, Mao J (2000) Statistical pattern recognition: a review. IEEE Trans Pattern Anal Mach Intell 22(1):4–37CrossRefGoogle Scholar
  24. Janisse MP (1974) Pupil size, affect and exposure frequency. Soc Behav Pers 2(2):125–146CrossRefGoogle Scholar
  25. KinneBrock W (1992) Neural networks. Oldenburg Verlag, MunchenGoogle Scholar
  26. Kruskal WH, Wallis WA (1952) Use of ranks in one-criterion variance analysis. J Am Stat Assoc 47(260):583–621CrossRefzbMATHGoogle Scholar
  27. Lanatà A, Armato A, Valenza G, Scilingo EP (2011) Eye tracking and pupil size variation as response to affective stimuli: a preliminary study. In: 2011 5th international conference on pervasive computing technologies for healthcare (pervasive health), IEEE, pp 78–84Google Scholar
  28. Land EH, McCann JJ (1971) Lightness and retinex theory. J Opt Soc Am 61(1):1–11CrossRefGoogle Scholar
  29. Lang PJ, Bradley MM, Cuthbert BN (1997) International affective picture system (IAPS): Technical manual and affective ratings. NIMH Center for the Study of Emotion and AttentionGoogle Scholar
  30. Lazarus RS, Averill JR (1972) Emotion and cognition: With special reference to anxiety. Anxiety Curr Trends Theory Res 2:242–284Google Scholar
  31. Levenson RW (1992) Autonomic nervous system differences among emotions. Psychol Sci 3(1):23CrossRefGoogle Scholar
  32. Li D, Babcock J, Parkhurst DJ (2006) openeyes: a low-cost head-mounted eye-tracking solution. In: Proceedings of the 2006 symposium on Eye tracking research and applications, ACM, pp 95–100Google Scholar
  33. Lilliefors HW (1967) On the Kolmogorov-Smirnov test for normality with mean and variance unknown. J Am Stat Assoc 62(318):399–402CrossRefGoogle Scholar
  34. Loewenfeld IE (1966) Comment on hess’ findings. Surv Ophthalmol 11:293–294Google Scholar
  35. Lohse GL, Johnson EJ (2002) A comparison of two process tracing methods for choice tasks. In: Proceedings of the Twenty-Ninth Hawaii International Conference on System sciences, 1996, vol 4, pp 86–97. IEEE, ISBN 0818673249Google Scholar
  36. Maini E (2005) Robust ellipse-specific fitting for real-time machine vision. In: Brain, vision, and artificial intelligence. Springer, Berlin, pp 318–327Google Scholar
  37. Marwan N, Carmen Romano M, Thiel M, Kurths J (2007) Recurrence plots for the analysis of complex systems. Phys Reports 438(5–6):237–329CrossRefGoogle Scholar
  38. Partala T, Surakka V (2003) Pupil size variation as an indication of affective processing. Int J Human-Comput Stud 59(1):185–198CrossRefGoogle Scholar
  39. Picard RW (2000) Affective computing. The MIT Press, Cambridge, ISBN 0262661152Google Scholar
  40. Schlesinger M, Hlavac V (2002) Ten lectures on statistical and structural pattern recognition. Kluwer Academic Publishers, DordtrechtGoogle Scholar
  41. SMI (2011) Sensomotoric instruments: gaze and eye tracking systems.
  42. S R Research Ltd. (2010) S R research:complete eye tracking solutions.
  43. Stavig GR, Gibbons JD (1977) Comparing the mean and the median as measures of centrality. Int Stat Rev/Revue Internationale de Statistique 45(1):63–70CrossRefzbMATHGoogle Scholar
  44. TOBII Technology (2011) Tobii eye tracking innovator.
  45. Valenza G, Lanatà A, Scilingo E (2011) The role of nonlinear dynamics in affective valence and arousal recognition. EEE Trans Affect Comput. doi: 10.1109/T-AFFC.2011.30
  46. Vapnik V (1998) Statistical learning theory. Wiley, New YorkGoogle Scholar
  47. Webb A (2002) Statistical pattern recognition. Wiley, New YorkGoogle Scholar
  48. Zajonc RB (1984) On the primacy of affect. Am Psychol 39(2):117–123CrossRefGoogle Scholar
  49. Zbilut JP, Webber Jr CL (1994) Recurrence quantification analysis. Wiley Online LibraryGoogle Scholar
  50. Zhang GP (2000) Neural networks for classification: a survey. IEEE Trans Syst Man Cybernet Part C Appl Rev 30(4):451–462CrossRefGoogle Scholar
  51. Zhou J (1998) Feedback svalence, feedback style, task autonomy, and achievement orientation: interactive effects on creative performance. J Appl Psychol 83(2):261CrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2012

Authors and Affiliations

  • Antonio Lanatà
    • 1
  • Gaetano Valenza
    • 1
  • Enzo Pasquale Scilingo
    • 1
  1. 1.Department of Information Engineering, Faculty of Engineering, Interdepartimental Research Centre “E.Piaggio”University of PisaPisaItaly

Personalised recommendations