Eye gaze patterns in emotional pictures

Abstract

This paper reports on a preliminary study aiming at investigating the eye gaze pattern and pupil size variation to discriminate emotional states induced by looking at pictures having different arousal content. A wearable and wireless eye gaze tracking system, hereinafter called HATCAM, which was able to robustly detect eye tracking and pupil area was used. A group of ten volunteers was presented with a set of neutral and arousal pictures extracted from the International Affective Picture System according to an ad-hoc experimental protocol. A set of features was extracted from eye gaze patterns and pupil size variations and used to classify the two classes of pictures. Although preliminary, results are very promising for affective computing applications.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

References

  1. Andreassi JL (2006) Psychophysiology Human behavior and physiological response. Lawrence Erlbaum Assoc Inc., ISBN: 0805849513

  2. Armato A, Lanatà A, Scilingo EP (2011) Comparitive study on photometric normalization algorithms for an innovative, robust and real-time eye gaze tracker. J Real-Time Image Process 1–13. doi:10.1007/s11554-011-0217-6

  3. ASL (2009) Applied science laboratories:mobile eye and eye tracking services. http://www.asleyetracking.com

  4. Beatty J, Lucero-Wagoner B (2000) The pupillary system. Cambridge University Press, Cambridge

  5. Bennett N, Burridge R, Saito N (2002) A method to detect and characterize ellipses using the Hough transform. IEEE Trans Pattern Anal Mach Intell 21(7):652–657

    Article  Google Scholar 

  6. Bradley MM, Sabatinelli D, Lang PJ, Fitzsimmons JR, King W, Desai P (2003) Activation of the visual cortex in motivated attention. Behav Neurosci 117(2):369

    Article  Google Scholar 

  7. Calvo MG, Lang PJ (2004) Gaze patterns when looking at emotional pictures: motivationally biased attention. Motiv Emot 28(3):221–243

    Article  Google Scholar 

  8. Chen W, Er MJ, Wu S (2006) Illumination compensation and normalization for robust face recognition using discrete cosine transform in logarithm domain. IEEE Trans Syst Man Cybernet Part B Cybernet 36(2):458–466

    Article  Google Scholar 

  9. Christianson SÅ, Loftus EF, Hoffman H, Loftus GR (1991) Eye fixations and memory for emotional events. J Exp Psychol Learn Memory Cognit 17(4):693

    Article  Google Scholar 

  10. Damasio AR (2000) Descartes’ error: emotion, reason, and the human brain. Quill, New York, ISBN: 0380726475

  11. Darwin C, Ekman P, Prodger P (2002) The expression of the emotions in man and animals. Oxford University Press, Oxford, ISBN: 0195158067

  12. Duda R, Hart P, Stork D (2001) Pattern classification, 2nd edn. Wiley, New York

  13. Ekman P, Levenson RW, Friesen WV (1983) Autonomic nervous system activity distinguishes among emotions. Science, ISSN: 0036-8075

  14. Fitzgibbon A, Pilu M, Fisher RB (2002) Direct least square fitting of ellipses. IEEE Trans Pattern Anal Mach Intell 21(5):476–480

    Article  Google Scholar 

  15. Forsyth DA, Ponce J (2002) Computer vision: a modern approach, Professional Technical Reference. Prentice Hall, Upper Saddle River

  16. Friedman J, Hastie T, Tibshirani R (2000) Additive logistic regression: A statistical view of boosting. Annals Stat 38(2):337–374

    MathSciNet  Article  Google Scholar 

  17. Gross JJ, Muñoz RF (1995) Emotion regulation and mental health. Clin Psychol Sci Pract 2(2):151–164

    Article  Google Scholar 

  18. Heijden F, Duin R, Ridder D, Tax D (2004) Classification, parameter estimation and state estimation. Wiley, New York

  19. Hess EH (1972) Pupillometrics: a method of studying mental, emotional and sensory processes, handbook of psychophysiology. Holt, Rinehart & Winston, New York, pp 91–531

  20. Hess EH, Petrovich SB (1987) Pupillary behavior in communication: nonverbal behaviour and communication. Erlbaum, Hillsdale, pp 327–348

  21. ISCAN (2007) Inc. Eye and target tracking solutions. http://www.iscaninc.com/

  22. Isen AM, Means B (1983) The influence of positive affect on decision-making strategy. Soc Cognit 2(1):18–31

    Article  Google Scholar 

  23. Jain AK, Duin RPW, Mao J (2000) Statistical pattern recognition: a review. IEEE Trans Pattern Anal Mach Intell 22(1):4–37

    Article  Google Scholar 

  24. Janisse MP (1974) Pupil size, affect and exposure frequency. Soc Behav Pers 2(2):125–146

    Article  Google Scholar 

  25. KinneBrock W (1992) Neural networks. Oldenburg Verlag, Munchen

  26. Kruskal WH, Wallis WA (1952) Use of ranks in one-criterion variance analysis. J Am Stat Assoc 47(260):583–621

    Article  MATH  Google Scholar 

  27. Lanatà A, Armato A, Valenza G, Scilingo EP (2011) Eye tracking and pupil size variation as response to affective stimuli: a preliminary study. In: 2011 5th international conference on pervasive computing technologies for healthcare (pervasive health), IEEE, pp 78–84

  28. Land EH, McCann JJ (1971) Lightness and retinex theory. J Opt Soc Am 61(1):1–11

    Article  Google Scholar 

  29. Lang PJ, Bradley MM, Cuthbert BN (1997) International affective picture system (IAPS): Technical manual and affective ratings. NIMH Center for the Study of Emotion and Attention

  30. Lazarus RS, Averill JR (1972) Emotion and cognition: With special reference to anxiety. Anxiety Curr Trends Theory Res 2:242–284

    Google Scholar 

  31. Levenson RW (1992) Autonomic nervous system differences among emotions. Psychol Sci 3(1):23

    Article  Google Scholar 

  32. Li D, Babcock J, Parkhurst DJ (2006) openeyes: a low-cost head-mounted eye-tracking solution. In: Proceedings of the 2006 symposium on Eye tracking research and applications, ACM, pp 95–100

  33. Lilliefors HW (1967) On the Kolmogorov-Smirnov test for normality with mean and variance unknown. J Am Stat Assoc 62(318):399–402

    Article  Google Scholar 

  34. Loewenfeld IE (1966) Comment on hess’ findings. Surv Ophthalmol 11:293–294

    Google Scholar 

  35. Lohse GL, Johnson EJ (2002) A comparison of two process tracing methods for choice tasks. In: Proceedings of the Twenty-Ninth Hawaii International Conference on System sciences, 1996, vol 4, pp 86–97. IEEE, ISBN 0818673249

  36. Maini E (2005) Robust ellipse-specific fitting for real-time machine vision. In: Brain, vision, and artificial intelligence. Springer, Berlin, pp 318–327

  37. Marwan N, Carmen Romano M, Thiel M, Kurths J (2007) Recurrence plots for the analysis of complex systems. Phys Reports 438(5–6):237–329

    Article  Google Scholar 

  38. Partala T, Surakka V (2003) Pupil size variation as an indication of affective processing. Int J Human-Comput Stud 59(1):185–198

    Article  Google Scholar 

  39. Picard RW (2000) Affective computing. The MIT Press, Cambridge, ISBN 0262661152

  40. Schlesinger M, Hlavac V (2002) Ten lectures on statistical and structural pattern recognition. Kluwer Academic Publishers, Dordtrecht

  41. SMI (2011) Sensomotoric instruments: gaze and eye tracking systems. http://www.smivision.com/

  42. S R Research Ltd. (2010) S R research:complete eye tracking solutions. http://www.sr-research.com/

  43. Stavig GR, Gibbons JD (1977) Comparing the mean and the median as measures of centrality. Int Stat Rev/Revue Internationale de Statistique 45(1):63–70

    Article  MATH  Google Scholar 

  44. TOBII Technology (2011) Tobii eye tracking innovator. http://www.tobii.com/

  45. Valenza G, Lanatà A, Scilingo E (2011) The role of nonlinear dynamics in affective valence and arousal recognition. EEE Trans Affect Comput. doi:10.1109/T-AFFC.2011.30

  46. Vapnik V (1998) Statistical learning theory. Wiley, New York

  47. Webb A (2002) Statistical pattern recognition. Wiley, New York

  48. Zajonc RB (1984) On the primacy of affect. Am Psychol 39(2):117–123

    Article  Google Scholar 

  49. Zbilut JP, Webber Jr CL (1994) Recurrence quantification analysis. Wiley Online Library

  50. Zhang GP (2000) Neural networks for classification: a survey. IEEE Trans Syst Man Cybernet Part C Appl Rev 30(4):451–462

    Article  Google Scholar 

  51. Zhou J (1998) Feedback svalence, feedback style, task autonomy, and achievement orientation: interactive effects on creative performance. J Appl Psychol 83(2):261

    Article  Google Scholar 

Download references

Acknowledgments

This research is partially supported by the EU Commission under contract FP7-ICT-247777 Psyche, and partially supported by the EU Commission under contract FP7-ICT-258749 CEEDs.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Antonio Lanatà.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Lanatà, A., Valenza, G. & Scilingo, E.P. Eye gaze patterns in emotional pictures. J Ambient Intell Human Comput 4, 705–715 (2013). https://doi.org/10.1007/s12652-012-0147-6

Download citation

Keywords

  • Eye gaze tracking
  • Wearable systems
  • Affective computing
  • Emotions
  • Pattern recognition