Skip to main content
SpringerLink
Log in

Extended Eye Landmarks Detection for Emerging Applications

  • Chapter
  • First Online:
Advances in Face Detection and Facial Image Analysis

  • 2658 Accesses

Abstract

In this chapter we focus on the eye landmarking and eye components identification in the framework of emerging psychology-related eye tracking applications. Traditional eye landmarking separates the identification of eye centers and of eye corners and margins, while here we discuss their joint use for face expression analysis in unconstrained environments and precise estimation of non-visual gaze directions, as suggested by the Eye Accessing Cues (EAC) of the Neuro-Linguistic Programming (NLP). Such a system involves a combination of low-level feature extraction, heuristic pre-processing and trained classifiers. The approach is extensively tested across several classical image databases and compared with state of the art traditional methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 54.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://www.bioid.com/downloads/software/bioid-face-database.html.

  2. 2.

    http://www.pitt.edu/~emotion/ck-spread.htm.

  3. 3.

    vision.ucsd.edu/~leekc/ExtYaleDatabase/ExtYaleB.html.

  4. 4.

    The database is available at http://vis-www.cs.umass.edu/lfw/.

  5. 5.

    At http://blog.gimiatlicho.webfactional.com/?page_id=38.

  6. 6.

    http://imag.pub.ro/common/staff/cflorea/EyeChimeraReleaseAgreement.pdf.

  7. 7.

    http://emotion-research.net/toolbox/toolboxdatabase.2010-02-03.4835728381.

  8. 8.

    http://www.uni-ulm.de/in/neuroinformatik/mitarbeiter/g-layher/image-databases.html.

  9. 9.

    https://biometrics.cie.put.poznan.pl.

  10. 10.

    http://vis-www.cs.umass.edu/fddb/results.html.

  11. 11.

    http://www.cbsr.ia.ac.cn/faceevaluation.

  12. 12.

    http://thume.ca/projects/2012/11/04/simple-accurate-eye-center-tracking-in-opencv/.

References

  1. R. Bandler, J. Grinder, Frogs into Princes: Neuro Linguistic Programming (Real People Press, Moab, 1979)

    Google Scholar 

  2. P. Tsiamyrtzis, J. Dowdall, D. Shastri, I.T. Pavlidis, M.G. Frank, P. Ekman, Imaging facial physiology for the detection of deceit. Int. J. Comput. Vis. 71, 197–214 (2007)

    Article  Google Scholar 

  3. A.B. Ashraf, S. Lucey, J.F. Cohn, T. Chen, Z. Ambadar, K.M. Prkachin, P. Solomon, The painful face – pain expression recognition using active appearance models. Image Vis. Comput. 27, 1788–1796 (2009)

    Article  Google Scholar 

  4. C. Florea, L. Florea, C. Vertan, Learning pain from emotion: transferred hot data representation for pain intensity estimation, in Proceedings of European Conference on Computer Vision Workshop on ACVR (2014)

    Google Scholar 

  5. D.S. Messinger, M.H. Mahoor, S.M. Chow, J. Cohn, Automated measurement of facial expression in infant-mother interaction: a pilot study. Infancy 14(3), 285–305 (2009)

    Article  Google Scholar 

  6. D. McDuff, R.E. Kaliouby, R. Picard, Predicting online media effectiveness based on smile responses gathered over the internet, in IEEE Face and Gesture (2013), pp. 1–8

    Google Scholar 

  7. J. Rehg, G. Abowd, A. Rozga et al., Decoding children’s social behavior, in Proceedings of Computer Vision and Pattern Recognition (2013), pp. 3414–3421

    Google Scholar 

  8. J.F. Cohn, F. De la Torre, Automated face analysis for affective computing, in The Oxford Handbook of Affective Computing (Oxford University Press, Oxford, 2014)

    Google Scholar 

  9. A. Frischen, A.P. Bayliss, S.P. Tipper, Gaze cueing of attention. Psychol. Bull. 133, 694–724 (2007)

    Article  Google Scholar 

  10. R. Vranceanu, C. Florea, L. Florea, C. Vertan, Gaze direction estimation by component separation for recognition of eye accessing cues. Mach. Vis. Appl. 26(2–3), 267–278 (2015)

    Article  Google Scholar 

  11. W. James, The Principles of Psychology (Harvard University Press, Cambridge, 1890)

    Book  Google Scholar 

  12. L. Nummenmaa, A. Calder, Neural mechanisms of social attention. Trends Cogn. Sci. 13, 135–43 (2009)

    Article  Google Scholar 

  13. S. Liversedge, J. Findlay, Saccadic eye movements and cognition. Trends Cogn. Sci. 4(1), 6–14 (2000)

    Article  Google Scholar 

  14. R. Adams, R.E. Kleck, Effects of direct and averted gaze on the perception of facially communicated emotion. Emotion 5, 3–11 (2005)

    Article  Google Scholar 

  15. H. Joseph, K. Nation, S.P. Liversedge, Using eye movements to investigate word frequency effects in children’s sentence reading. Sch. Psychol. Rev. 42, 207–222 (2013)

    Google Scholar 

  16. A. Godfroid, F. Boers, A. Housen, An eye for words: gauging the role of attention in incidental l2 vocabulary acquisition by means of eye-tracking. Stud. Second Lang. Acquis. 35, 483–517 (2013)

    Article  Google Scholar 

  17. K. Rayner, T.J. Slattery, D. Drieghe, S.P. Liversedge, Eye movements and word skipping during reading: effects of word length and predictability. J. Exp. Psychol. Hum. Percept. Perform. 37, 514–528 (2011)

    Article  Google Scholar 

  18. K. Rayner, B.R. Foorman, C.A. Perfetti, D. Pesetsky, M.S. Seidenberg, How psychological science informs the teaching of reading. Psychol. Sci. Public Interest 2, 31–74 (2001)

    Article  Google Scholar 

  19. M.M. Chun, Contextual cueing of visual attention. Trends Cogn. Sci. 4, 170–178 (2000)

    Article  Google Scholar 

  20. A. Bulling, T. Zander, Cognition-aware computing. IEEE Trans. Pervasive Comput. 13, 80–83 (2014)

    Article  Google Scholar 

  21. B. Meijering, H. van Rijn, N.A. Taatgen, R. Verbrugge, What eye movements can tell about theory of mind in a strategic game. PLoS One 7(9) (2012) doi:10.1371/journal.pone.0045961

    Google Scholar 

  22. K. Krejtz, C. Biele, D. Chrzastowski, A. Kopacz, A. Niedzielska, P. Toczyski, A. Duchowski, Gaze-controlled gaming: immersive and difficult but not cognitively overloading, in Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication (2014), pp. 1123–1129

    Google Scholar 

  23. J. Sturt, S. Ali, W. Robertson, D. Metcalfe, A. Grove, C. Bourne, C. Bridle, Neurolinguistic programming: systematic review of the effects on health outcomes. Br. J. Gen. Pract. 62, 757–764 (2012)

    Article  Google Scholar 

  24. R. Vranceanu, C. Florea, L. Florea, C. Vertan, NLP EAC recognition by component separation in the eye region, in Proceedings of Computer Analysis and Image Processing (2013), pp. 225–232

    Google Scholar 

  25. B. Laeng, D.S. Teodorescu, Eye scanpaths during visual imagery reenact those of perception of the same visual scene. Cogn. Sci. 26, 207–231 (2002)

    Article  Google Scholar 

  26. T. Kanade, J.F. Cohn, Y. Tian, Comprehensive database for facial expression analysis, in IEEE Face and Gesture (2000), pp. 46–53

    Google Scholar 

  27. K. Lee, J. Ho, D. Kriegman, Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans. Pattern Anal. Mach. Intell. 27, 684–698 (2005)

    Article  Google Scholar 

  28. P. Belhumeur, D. Jacobs, D. Kriegman, N. Kumar, Localizing parts of faces using a consensus of exemplars, in Proceedings of Computer Vision and Pattern Recognition (2011), pp. 545–552

    Google Scholar 

  29. G. Huang, M. Ramesh, T. Berg, E. Learned-Miller, Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical report, University of Massachusetts, 2007

    Google Scholar 

  30. L. Florea, C. Florea, R. Vranceanu, C. Vertan, Can your eyes tell me how you think? A gaze directed estimation of the mental activity, in Proceedings of British Machine Vision Conference (2013)

    Google Scholar 

  31. S. Asteriadis, D. Soufleros, K. Karpouzis, S. Kollias, A natural head pose and eye gaze dataset, in ACM Workshop on Affective Interaction in Natural Environments (2009), pp. 1–4

    Google Scholar 

  32. U. Weidenbacher, G. Layher, P. Strauss, H. Neumann, A comprehensive head pose and gaze database, in IET International Conference on Intelligent Environments (2007), pp. 455–458

    Google Scholar 

  33. A. Kasinśki, A. Florek, A. Schmidt, The PUT face database. Image Process. Commun. 13, 59–64 (2008)

    Google Scholar 

  34. L. Wolf, Z. Freund, S. Avidan, An eye for an eye: a single camera gaze-replacement method, in Proceedings of Computer Vision and Pattern Recognition (2010), pp. 817–824

    Google Scholar 

  35. K. Radlak, M. Kawulok, B. Smolka, N. Radlak, Gaze direction estimation from static images, in Proceedings of IEEE Multimedia Signal Processing (2014), pp. 1–4

    Google Scholar 

  36. P. Viola, M. Jones, Robust real-time face detection. Int. J. Comput. Vis. 57, 137–154 (2004)

    Article  Google Scholar 

  37. M. Mathias, R. Benenson, M. Pedersoli, L.V. Gool, Face detection without bells and whistles, in Proceedings of the European Conference on Computer Vision, vol. 8692 (2014), pp. 720–735

    Google Scholar 

  38. P. Felzenszwalb, R. Girshick, D. McAllester, D. Ramanan, Object detection with discriminatively trained part-based models. Pattern Recogn. Lett. 19, 899–906 (2010)

    Google Scholar 

  39. F. Song, X. Tan, S. Chen, Z. Zhoub, A literature survey on robust and efficient eye localization in real-life scenarios. Br. J. Gen. Pract. 46, 3157–3173 (2013)

    Google Scholar 

  40. M. Hamouz, J. Kittlerand, J.K. Kamarainen, P. Paalanen, H. Kalviainen, J. Matas, Feature-based affine-invariant localization of faces. IEEE Trans. Pattern Anal. Mach. Intell. 27, 643–660 (2005)

    Article  Google Scholar 

  41. S. Asteriadis, N. Nikolaidis, I. Pitas, Facial feature detection using distance vector fields. Pattern Recogn. 42, 1388–1398 (2009)

    Article  MATH  Google Scholar 

  42. J. Wu, Z.H. Zhou, Efficient face candidates selector for face detection. Pattern Recogn. 36, 1175–1186 (2003)

    Article  Google Scholar 

  43. R. Valenti, T. Gevers, Accurate eye center location and tracking using isophote curvature, in Proceedings of Computer Vision and Pattern Recognition (2008), pp. 1–8

    Google Scholar 

  44. O. Jesorsky, K. Kirchberg, R. Frischholz, Robust face detection using the Hausdorff distance, in Proceedings of International Conference on Audio- and Video-Based Biometric Person Authentication (2001), pp. 90–95

    Google Scholar 

  45. T. Kanade, Picture processing by computer complex and recognition of human faces. Technical Report, Kyoto University, Department of Information Science, 1973

    Google Scholar 

  46. G.C. Feng, P.C. Yuen, Variance projection function and its application to eye detection for human face recognition. Pattern Recogn. Lett. 19, 899–906 (1998)

    Article  Google Scholar 

  47. Z. Zhou, Projection functions for eye detection. Pattern Recogn. 37, 1049–1056 (2004)

    Article  MATH  Google Scholar 

  48. M. Turkan, M. Pardas, A.E. Cetin, Edge projections for eye localization. Opt. Eng. 47, 047–054 (2008)

    Google Scholar 

  49. M. Verjak, M. Stephancic, An anthropological model for automatic recognition of the male human face. Ann. Hum. Biol. 21, 363–380 (1994)

    Article  Google Scholar 

  50. D. Cristinacce, T. Cootes, I. Scott, A multi-stage approach to facial feature detection, in Proceedings of British Machine Vision Conference (2004), pp. 277–286

    Google Scholar 

  51. P. Campadelli, R. Lanzarotti, G. Lipori, Precise eye localization through a general-to-specific model definition, in Proceedings of British Machine Vision Conference, I, 187–196 (2006)

    MATH  Google Scholar 

  52. Z. Niu, S. Shan, S. Yan, X. Chen, W. Gao, 2D cascaded adaboost for eye localization, in Proceedings of International Conference of Pattern Recognition (2006), pp. 1216–1219

    Google Scholar 

  53. S. Kim, S.T. Chung, S. Jung, D. Oh, J. Kim, S. Cho, World Academy of Science, Engineering and Technology, in WASET, vol. 21 (World Academy of Science, Engineering and Technology, 2007), pp. 483–487

    Google Scholar 

  54. M. Asadifard, J. Shanbezadeh, Automatic adaptive center pupil detection using face detection and CDF analysis, in Proceedings of International Multimedia Conference of Engineers and Computer Scientist (2010), pp. 130–133

    Google Scholar 

  55. L. Ding, A.M. Martinez, Features versus context: an approach for precise and detailed detection and delineation of faces and facial features. IEEE Trans. Pattern Anal. Mach. Intell. 32, 2022–2038 (2010)

    Article  Google Scholar 

  56. F. Timm, E. Barth, Accurate eye centre localisation by means of gradients, in Proceedings of International Conference on Computer Theory and Applications (2011), pp. 125–130

    Google Scholar 

  57. M. Kawulok, J. Szymanek, Precise multi-level face detector for advanced analysis of facial images. IET Image Process. 6, 95–103 (2012)

    Article  MathSciNet  Google Scholar 

  58. C. Florea, L. Florea, C. Vertan, Robust eye centers localization with zero-crossing encoded image projections. Pattern Anal. Applic. 1–17 (2015), DOI:10.1007/s10044-015-0479-x, http://dx.doi.org/10.1007/s10044-015-0479-x

    Google Scholar 

  59. R. Valenti, T. Gevers, Accurate eye center location through invariant isocentric patterns. IEEE Trans. Pattern Anal. Mach. Intell. 34, 1785–1798 (2012)

    Article  Google Scholar 

  60. H.C. Becker, W.J. Nettleton, P.H. Meyers, J.W. Sweeney, C.M. Nice, Digital computer determination of a medical diagnostic index directly from chest X-ray images. IEEE Trans. Biomed. Eng. 11, 62–72 (1964)

    Google Scholar 

  61. F. Crow, Summed-area tables for texture mapping. Proc. SIGGRAPH 18, 207–212 (1984)

    Article  Google Scholar 

  62. G.E. Blelloch, Prefix sums and their applications. synthesis of parallel algorithms. Technical report, University of Massachusetts, 1990

    Google Scholar 

  63. R.A. King, T.C. Phipps, Shannon, TESPAR and approximation strategies. Comput. Secur. 18, 445–453 (1999)

    Google Scholar 

  64. X. Chen, H. Wu, X. Jin, Q. Zhao, Face illumination manipulation using a single reference image by adaptive layer decomposition. IEEE Trans. Image Processing 22(11), 4249–4259 (2013)

    Article  MathSciNet  Google Scholar 

  65. B. Kroon, A. Hanjalic, S.M. Maas, Eye localization for face matching: is it always useful and under what conditions, in Proceedings of International Conference on Content-Based Image and Video Retrieval (2008), pp. 379–387

    Google Scholar 

  66. M. Ciesla, P. Koziol, Eye pupil location using webcam. CoRR, (2012) http://arxiv.org/abs/1202.6517

    Google Scholar 

  67. M. Dantone, J. Gall, G. Fanelli, L.V. Gool, Real-time facial feature detection using conditional regression forests, in Proceedings of Computer Vision and Pattern Recognition (2012), pp. 2578–2585

    Google Scholar 

  68. Y. Sun, X. Wang, X. Tang, Deep convolutional network cascade for facial point detection, in Proceedings of Computer Vision and Pattern Recognition (2013), pp. 3476–3483

    Google Scholar 

  69. T. Cootes, C. Taylor, D. Cooper, J. Graham, Active shape models - their training and application. Comput. Vis. Image Underst. 61, 38–59 (1995)

    Article  Google Scholar 

  70. T.F. Cootes, G.J. Edwards, C.J. Taylor, Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23, 681–685 (2001)

    Article  Google Scholar 

  71. T. Leung, M. Burl, P. Perona, Finding faces in cluttered scenes using random labeled graph matching, in Proceedings of International Conference on Computer Vision (1995), pp. 637–644

    Google Scholar 

  72. S. Milborrow, F. Nicolls, Locating facial features with an extended active shape model, in Proceedings of European Conference on Computer Vision (2008), pp. 504–513

    Google Scholar 

  73. V. Le, J. Brandt, Z. Lin, L. Bourdev, T.S. Huang, Interactive facial feature localization, in Proceedings of European Conference on Computer Vision (2012), pp. 679–692

    Google Scholar 

  74. D. Cristinacce, T. Cootes, Feature detection and tracking with constrained local models, in Proceedings of British Machine Vision Conference (2006), pp. 929–938

    Google Scholar 

  75. P. Tresadern, H. Bhaskar, S. Adeshina, C. Taylor, T. Cootes, Combining local and global shape models for deformable object matching, in Proceedings of British Machine Vision Conference (2009)

    Google Scholar 

  76. T. Cootes, M.C. Ionita, C. Lindner, P. Sauer, Robust and accurate shape model fitting using random forest regression voting, in Proceedings of European Conference on Computer Vision (2012)

    Google Scholar 

  77. J. Saragih, S. Lucey, J. Cohn, Deformable model fitting by regularized landmark mean-shift. Int. J. Comput. Vis. 91, 200–215 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  78. M. Valstar, T. Martinez, X. Binefa, M. Pantic, Facial point detection using boosted regression and graph models, in Proceedings of Computer Vision and Pattern Recognition (2010), pp. 2729–2736

    Google Scholar 

  79. X. Zhu, D. Ramanan, Face detection, pose estimation, and landmark localization in the wild, in Proceedings of Computer Vision and Pattern Recognition (2012), pp. 2879–2886

    Google Scholar 

  80. X. Yu, J. Huang, S. Zhang, W. Yan, D.N. Metaxas, Pose-free facial landmark fitting via optimized part mixtures and cascaded deformable shape model, in Proceedings of International Conference on Computer Vision (2013), pp. 1944–1951

    Google Scholar 

  81. B. Martinez, M.F. Valstar, X. Binefa, M. Pantic, Local evidence aggregation for regression based facial point detection. IEEE Trans. Pattern Anal. Mach. Intell. 35, 1149–1163 (2013)

    Article  Google Scholar 

  82. P. Wang, M.B. Green, Q. Ji, J. Wayman, Automatic eye detection and its validation, in IEEE Workshop on FRGC, Computer Vision and Pattern Recognition (2005), p. 164

    Google Scholar 

  83. A. Duchowski, Eye Tracking Methodology: Theory and Practice (Springer, Berlin, 2007)

    MATH  Google Scholar 

  84. D. Hansen, J. Qiang, In the eye of the beholder: a survey of models for eyes and gaze. IEEE Trans. Pattern Anal. Mach. Intell. 32, 478–500 (2010)

    Article  Google Scholar 

  85. D. Yoo, M. Chung, A novel non-intrusive eye gaze estimation using cross-ratio under large head motion. Comput. Vis. Image Underst. 98, 25–51 (2005)

    Article  Google Scholar 

  86. B. Pires, M. Hwangbo, M. Devyver, T. Kanade, Visible-spectrum gaze tracking for sports, in WACV (2013)

    Google Scholar 

  87. D. Hansen, A. Pece, Eye tracking in the wild. Comput. Vis. Image Underst. 98, 182–210 (2005)

    Article  Google Scholar 

  88. S. Cadavid, M. Mahoor, D. Messinger, J. Cohn, Automated classification of gaze direction using spectral regression and support vector machine, in Proceedings of Affective Computing and Intelligent Interaction (2009), pp. 1–6

    Google Scholar 

  89. T. Heyman, V. Spruyt, A. Ledda, 3d face tracking and gaze estimation using a monocular camera, in Proceedings of International Conference on Positioning and Context-Awareness (2011), pp. 23–28

    Google Scholar 

  90. M. Everingham, A. Zisserman, Regression and classification approaches to eye localization in face images, in IEEE Face and Gesture (2006), pp. 441–446

    Google Scholar 

  91. G. Diamantopoulos, Novel eye feature extraction and tracking for non-visual eye-movement applications. Ph.D. thesis, University of Birmingham, 2010

    Google Scholar 

  92. S. le Cessie, J. van Houwelingen, Ridge estimators in logistic regression. Appl. Stat. 41, 191–201 (1992)

    Article  MATH  Google Scholar 

Download references

Acknowledgements

This work was partially supported by the Romanian Sectoral Operational Programme Human Resources Development 2007–2013 through the European Social Fund Financial Agreements POSDRU/159/1.5/S/134398 (Knowledge).

Author information

Authors and Affiliations

  1. Image Processing and Analysis Laboratory (LAPI), University “Politehnica” of Bucharest, Bucharest, Romania

    Laura Florea, Corneliu Florea & Constantin Vertan

Authors
  1. Laura Florea
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Corneliu Florea
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Constantin Vertan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Laura Florea .

Editor information

Editors and Affiliations

  1. Automatic Control, Silesian University of Technology, Kawulok, Poland

    Michal Kawulok

  2. Computer Science, Louisiana State University in Shreveport, Shreveport, Louisiana, USA

    M. Emre Celebi

  3. Silesian University of Technology, Gliwice, Poland

    Bogdan Smolka

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Florea, L., Florea, C., Vertan, C. (2016). Extended Eye Landmarks Detection for Emerging Applications. In: Kawulok, M., Celebi, M., Smolka, B. (eds) Advances in Face Detection and Facial Image Analysis. Springer, Cham. https://doi.org/10.1007/978-3-319-25958-1_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-25958-1_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-25956-7

  • Online ISBN: 978-3-319-25958-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation