Skip to main content
SpringerLink
Log in

Expression Recognition

  • Chapter
  • First Online:
Orthogonal Image Moments for Human-Centric Visual Pattern Recognition

Part of the book series: Cognitive Intelligence and Robotics ((CIR))

  • 308 Accesses

Abstract

Facial expression is regarded as one of the most powerful means for humans to convey their feelings, attitudes, or opinions to each other. It has been revealed from the psychological studies that during conversations between humans, over 50% of information is conveyed through facial expressions [39]. Automatic facial expression recognition (FER), which uses machines to recognize human facial expressions, has been an active area of research due to its several notable applications. Examples include lie detection, intelligent interaction in social media, emotional therapy for autistic patient, e-commerce, and multimodal human–computer interface [52].

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. GENKI-4K Database. http://mplab.ucsd.edu. Accessed 01 Jan 2015

  2. N. Alugupally, A. Samal, D. Marx, S. Bhatia, Analysis of landmarks in recognition of face expressions. Pattern Recognit. Image Anal. 21(4), 681–693 (2011)

    Article  Google Scholar 

  3. M.S. Bartlett, G.C. Littlewort, M.G. Frank, C. Lainscsek, I.R. Fasel, J.R. Movellan, Automatic recognition of facial actions in spontaneous expressions. J. Multimed. 1(6), 22–35 (2006)

    Article  Google Scholar 

  4. J.N. Bassili, Facial motion in the perception of faces and of emotional expression. J. Exp. Psychol. - Hum. Percept. Perform. 4(3), 373–379 (1978)

    Google Scholar 

  5. V. Bettadapura, Facial expression recognition and analysis: the state of the art. Technical report 1203.6722, Cornell University, arXiv e-prints (2012)

    Google Scholar 

  6. A. Bordes, S. Ertekin, J. Weston, L. Bottou, Fast kernel classifiers with online and active learning. J. Mach. Learn. Res. 6, 1579–1619 (2005)

    MathSciNet  MATH  Google Scholar 

  7. I. Borg, P.J.F. Groenen, Modern Multidimensional Scaling, 2nd edn. (Springer, New York, 2005)

    MATH  Google Scholar 

  8. C.C. Chang, C.J. Lin, LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(3, Article 27), 1–27 (2011)

    Google Scholar 

  9. I. Cohen, N. Sebe, A. Garg, L.S. Chen, T.S. Huanga, Facial expression recognition from video sequences: temporal and static modeling. Comput. Vis. Image Underst. 91(1–2), 160–187 (2003)

    Article  Google Scholar 

  10. G. Donato, M.S. Bartlett, J.C. Hager, P. Ekman, T.J. Sejnowski, Classifying facial actions. IEEE Trans. Pattern Anal. Mach. Intell. 21(10), 974–989 (1999)

    Google Scholar 

  11. J.G. Dy, Unsupervised feature selection, in Computational Methods of Feature Selection, ed. by H. Liu, H. Motoda (Taylor & Francis, Florida, 2008)

    Google Scholar 

  12. P. Ekman, W.V. Friesen, Constants across cultures in the face and emotion. J. Personal. Soc. Psychol. 17(2), 124–129 (1971)

    Article  Google Scholar 

  13. D. Ghimire, J. Lee, Z.N. Li, S. Jeong, S.H. Park, H.S. Choi, Recognition of facial expressions based on tracking and selection of discriminative geometric features. Int. J. Multimed. Ubiquitous Eng. 10(3), 35–44 (2015)

    Article  Google Scholar 

  14. G. Guo, R. Guo, X. Li, Facial expression recognition influenced by human aging. IEEE Trans. Affect. Comput. 4(3), 291–298 (2013)

    Article  Google Scholar 

  15. S.L. Happy, A. Routray, Automatic facial expression recognition using features of salient facial patches. IEEE Trans. Affect. Comput. 6(1), 1–12 (2014)

    Article  Google Scholar 

  16. Y. Hu, Z. Zeng, L. Yin, X. Wei, X. Zhou, T.S. Huang, Multi-view facial expression recognition, in Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition (Amsterdam, Netherlands 2008), pp. 1–6

    Google Scholar 

  17. S.M. Imran, S.M.M. Rahman, D. Hatzinakos, Differential components of discriminative 2D Gaussian-Hermite moments for recognition of facial expressions. Pattern Recognit. 56, 100–115 (2016)

    Article  Google Scholar 

  18. A. Jain, D. Zongker, Feature selection: evaluation, application, and small sample performance. IEEE Trans. Pattern Anal. Mach. Intell. 19(2), 153–158 (1997)

    Google Scholar 

  19. V. Jain, J. Crowley, Smile detection using multi-scale gaussian derivatives, in WSEAS International Conference on Signal Processing, Robotics and Automation (Cambridge, United Kingdom, 2013), pp. 149–154

    Google Scholar 

  20. Y. Ji, K. Idrissi, Automatic facial expression recognition based on spatiotemporal descriptors. Pattern Recognit. Lett. 33(10), 1373–1380 (2012)

    Article  Google Scholar 

  21. M.H. Kabir, T. Jabid, O. Chae, A local directional pattern variance (LDPv) based face descriptor for human facial expression recognition, in Proceedings of the IEEE International Conference on Advanced Video and Signal Based Surveillance (Boston, MA, 2010), pp. 526–532

    Google Scholar 

  22. S.E. Kahou, P. Froumenty, C. Pal, Facial expression analysis based on high dimensional binary features, in Lecture Notes in Computer Science: European Conference on Computer Vision (Zurich, Switzerland, 2014), pp. 135–147

    Google Scholar 

  23. T. Kanade, J.F. Cohn, Y. Tian, Comprehensive database for facial expression analysis, in Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition (Grenoble, France, 2000), pp. 484–490

    Google Scholar 

  24. S.B. Kazmi, Q. Ain, M.A. Jaffar, Wavelets-based facial expression recognition using a bank of support vector machines. Soft Comput. 16(3), 369–379 (2012)

    Article  Google Scholar 

  25. K.J. Kelly, J. Metcalfe, Metacognition of emotional face recognition. Emotion 11(4), 896–906 (2011)

    Article  Google Scholar 

  26. D.H. Kim, W.J. Baddar, J. Jang, Y.M. Ro, Multi-objective based spatio-temporal feature representation learning robust to expression intensity variations for facial expression recognition. IEEE Trans. Affect. Comput. 1–15 (2017). https://doi.org/10.1109/TAFFC.2017.2695999

  27. S.R.V. Kittusamy, V. Chakrapani, Facial expressions recognition using eigenspaces. J. Comput. Sci. 8(10), 1674–1679 (2012)

    Article  Google Scholar 

  28. B.C. Ko, A brief review of facial emotion recognition based on visual information. Sensors 18(2)

    Google Scholar 

  29. S.M. Lajevardi, Z.M. Hussain, Higher order orthogonal moments for invariant facial expression recognition. Digit. Signal Process. 20(6), 1771–1779 (2010)

    Article  Google Scholar 

  30. S.M. Lajevardi, H.R. Wu, Facial expression recognition in perceptual color space. IEEE Trans. Image Process. 21(8), 3721–3733 (2012)

    Article  MathSciNet  Google Scholar 

  31. S.M. Lajeverdi, Z.M. Hussain, Automatic facial expression recognition: feature extraction and selection. Signal Image Video Process. 6(1), 159–169 (2012)

    Article  Google Scholar 

  32. A. Lanitis, C.J. Taylor, T.F. Cootes, Automatic interpretation and coding of face images using flexible models. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 743–756 (1997)

    Google Scholar 

  33. H. Li, J.M. Morvan, L. Chen, 3D facial expression recognition based on histograms of surface differential quantities, in Proceedings of the International Conference on Advanced Concepts for Intelligent Vision Systems, vol. 6915 (Ghent, Belgium, 2011), pp. 483–494

    Google Scholar 

  34. Y. Li, S. Wang, Y. Zhao, Q. Ji, Simultaneous facial feature tracking and facial expression recognition. IEEE Trans. Image Process. 22(7), 2559–2573 (2013)

    Article  Google Scholar 

  35. Z. Li, J. Imai, M. Kaneko, Face and expression recognition based on bag of words method considering holistic and local image features, in Proceedings of the International Symposium on Communications and Information Technologies (Tokyo, Japan, 2010), pp. 1–6

    Google Scholar 

  36. R. Londhe, V. Pawar, Facial expression recognition based on affine moment invariants. IJCSI Int. J. Comput. Sci. Issues 9(2), 388–392 (2012)

    Google Scholar 

  37. M.J. Lyons, J. Budynek, S. Akamatsu, Automatic classification of single facial images. IEEE Trans. Pattern Anal. Mach. Intell. 21(12), 1357–1362 (1999)

    Google Scholar 

  38. O. Maimon, L. Rokach, Data Mining and Knowledge Discovery Handbook, 2nd edn. (Springer, New York, 2010)

    Book  Google Scholar 

  39. A. Mehrabian, Communication without words. Psychol. Today 2(4), 53–56 (1968)

    Google Scholar 

  40. S. Mitra, N.A. Lazar, Y. Liu, Understanding the role of facial asymmetry in human face identification. Stat. Comput. 17(1), 57–70 (2007)

    Article  MathSciNet  Google Scholar 

  41. L. Oliveira, A.L. Koerich, M. Mansano, A.S. Britto, 2D principal component analysis for face and facial-expression recognition. Comput. Sci. Eng. 13(3), 9–13 (2011)

    Article  Google Scholar 

  42. M. Pantic, M. Valstar, R. Rademaker, L. Maat, Web-based database for facial expression analysis, in Proceedings of the IEEE International Conference on Multimedia and Expo (Amsterdam, The Netherlands 2005), pp. 1–5

    Google Scholar 

  43. P.J. Phillips, P.J. Flynn, T. Scruggs, K.W. Bowyer, J. Chang, K. Hoffman, J. Marques, J. Min, W. Worek, Overview of the face recognition grand challenge, in Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (San Diego, CA, USA, 2005), pp. 947–954

    Google Scholar 

  44. S.M.M. Rahman, T. Howlader, D. Hatzinakos, On the selection of 2D Krawtchouk moments for face recognition. Pattern Recognit. 54, 83–93 (2016)

    Article  Google Scholar 

  45. A.R. Rivera, J.R. Castillo, O. Chae, Local directional number pattern for face analysis: face and expression recognition. IEEE Trans. Image Process. 22(5), 1740–1752 (2013)

    Article  MathSciNet  Google Scholar 

  46. H. Rodger, L. Vizioli, X. Ouyang, R. Caldara, Mapping the development of facial expression recognition. Dev. Sci. 18(6), 926–939 (2015)

    Article  Google Scholar 

  47. E. Sariyanidi, H. Gunes, A. Cavallaro, Automatic analysis of facial affect: a Survey of registration, representation, and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 37(6), 1113–1133 (2015)

    Google Scholar 

  48. C. Shan, Smile detection by boosting pixel differences. IEEE Trans. Image Process. 21(1), 431–436 (2012)

    Article  MathSciNet  Google Scholar 

  49. C. Shan, S. Gong, P.W. McOwan, Facial expression recognition based on local binary patterns: a comprehensive study. Image Vis. Comput. 27(6), 803–816 (2009)

    Article  Google Scholar 

  50. Y. Sun, X. Wang, X. Tang, Deep convolutional network cascade for facial point detection, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (Portland, OR, 2013), pp. 3476–3483

    Google Scholar 

  51. C.E. Thomaz, D.F. Gillies, R.Q. Feitosa, Using mixture covariance matrices to improve face and facial expression recognitions. Pattern Recognit. Lett. 24(13), 2159–2165 (2003)

    Article  Google Scholar 

  52. M. Turk, Multimodal human-computer interaction, in Real-Time Vision for Human-Computer Interaction (Springer, New York, 2005), pp. 269–283

    Google Scholar 

  53. M.Z. Uddin, W. Khaksar, J. Torresen, Facial expression recognition using salient features and convolutional neural network. IEEE Access 5, 26146–26161 (2017)

    Article  Google Scholar 

  54. L. Wang, R.F. Li, K. Wang, J. Chen, Feature representation for facial expression recognition based on FACS and LBP. Int. J. Autom. Comput. 11(5), 459–468 (2014)

    Article  Google Scholar 

  55. S. Wang, Q. Ji, Video affective content analysis: a survey of state-of-the-art methods. IEEE Trans. Affect. Comput. 6(4), 410–430 (2015)

    Article  Google Scholar 

  56. Z. Wang, Y. Li, S. Wang, Q. Ji, Capturing global semantic relationships for facial action unit recognition, in Proceedings of the IEEE Conference on Computer Vision (Sydney, NSW, 2013), pp. 3304–3311

    Google Scholar 

  57. Z. Wang, S. Wang, Q. Ji, Capturing complex spatio-temporal relations among facial muscles for facial expression recognition, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (Portland, OR, 2013), pp. 3422–3429

    Google Scholar 

  58. J. Whitehill, G. Littlewort, I. Fasel, M. Bartlett, J. Movellan, Toward practical smile detection. IEEE Trans. Pattern Anal. Mach. Intell. 31(11), 2106–2111 (2009)

    Google Scholar 

  59. W.M. Wundt, Grundzüge de Physiologischen Psychologie (Engelman, Leipzig, 1905)

    Google Scholar 

  60. R. Xiao, Q. Zhao, D. Zhang, P. Shi, Facial expression recognition on multiple manifolds. Pattern Recognit. 44(1), 107–116 (2011)

    Article  Google Scholar 

  61. J. Xibin, B. Xiyuan, D.M.W. Powers, L. Yujian, Facial expression recognition based on block Gabor wavelet fusion feature. J. Converg. Inf. Technol. 8(5), 282–289 (2013)

    Google Scholar 

  62. B. Yang, M. Dai, Image analysis by Gaussian-Hermite moments. Signal Process. 91(10), 2290–2303 (2011)

    Article  Google Scholar 

  63. B. Yang, G. Li, H. Zhang, M. Dai, Rotation and translation invariants of Gaussian-Hermite moments. Pattern Recognit. Lett. 32(9), 1283–1298 (2011)

    Article  Google Scholar 

  64. S. Yang, B. Bhanu, Facial expression recognition using emotion avatar image, in Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition and Workshops (Santa Barbara, CA, 2011), pp. 866–871

    Google Scholar 

  65. L. Zhang, D. Tjondronegoro, Facial expression recognition using facial movement features. IEEE Trans. Affect. Comput. 2(4), 219–229 (2011)

    Article  Google Scholar 

  66. W. Zhang, Y. Zhang, L. Ma, J. Guan, S. Gong, Multimodal learning for facial expression recognition. Pattern Recognit. 48(10), 3191–3202 (2015)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Electronic Engineering, Bangladesh University of Engineering and Technology, Dhaka, Bangladesh

    S. M. Mahbubur Rahman

  2. Institute of Statistical Research and Training, University of Dhaka, Dhaka, Bangladesh

    Tamanna Howlader

  3. Department of Electrical and Computer Engineering, University of Toronto, Toronto, ON, Canada

    Dimitrios Hatzinakos

Authors
  1. S. M. Mahbubur Rahman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tamanna Howlader
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dimitrios Hatzinakos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to S. M. Mahbubur Rahman .

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Singapore Pte Ltd.

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Rahman, S.M.M., Howlader, T., Hatzinakos, D. (2019). Expression Recognition. In: Orthogonal Image Moments for Human-Centric Visual Pattern Recognition. Cognitive Intelligence and Robotics. Springer, Singapore. https://doi.org/10.1007/978-981-32-9945-0_4

Download citation

  • DOI: https://doi.org/10.1007/978-981-32-9945-0_4

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-32-9944-3

  • Online ISBN: 978-981-32-9945-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation