Skip to main content
SpringerLink
Log in
Book cover

International Work-Conference on the Interplay Between Natural and Artificial Computation

IWINAC 2022: Bio-inspired Systems and Applications: from Robotics to Ambient Intelligence pp 65–74Cite as

Evaluation of Selected APIs for Emotion Recognition from Facial Expressions

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13259))

Abstract

Facial expressions convey the vast majority of the emotional information contained in social utterances. From the point of view of affective intelligent systems, it is therefore important to develop appropriate emotion recognition models based on facial images. As a result of the high interest of the research and industrial community in this problem, many ready-to-use tools are being developed, which can be used via suitable web APIs. In this paper, two of the most popular APIs were tested: Microsoft Face API and Kairos Emotion Analysis API. The evaluation was performed on images representing 8 emotions—anger, contempt, disgust, fear, joy, sadness, surprise and neutral—distributed in 4 benchmark datasets: Cohn-Kanade (CK), Extended Cohn-Kanade (CK+), Amsterdam Dynamic Facial Expression Set (ADFES) and Radboud Faces Database (RaFD). The results indicated a significant advantage of the Microsoft API in the accuracy of emotion recognition both in photos taken en face and at a 45\(^\circ \) angle. Microsoft’s API also has an advantage in the larger number of recognised emotions: contempt and neutral are also included.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    See: https://azure.microsoft.com/en-us/services/cognitive-services/.

  2. 2.

    See: https://azure.microsoft.com/en-us/services/cognitive-services/face/.

  3. 3.

    See: https://www.kairos.com/docs/api/.

References

  1. Cohn, J.F., De La Torre, F.: Automated face analysis for affective computing. In: Calvo, R.A., D’Mello, S., Gratch, J., Kappas, A. (eds.) The Oxford Handbook of Affective Computing, pp. 131–150. Oxford University Press (2015). https://doi.org/10.1093/oxfordhb/9780199942237.013.020

  2. Deshmukh, R.S., Jagtap, V.: A survey: software API and database for emotion recognition. In: 2017 International Conference on Intelligent Computing and Control Systems (ICICCS), pp. 284–289 (2017). https://doi.org/10.1109/ICCONS.2017.8250727

  3. Ekman, P., Friesen, W.V.: Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press, Palo Alto (1978)

    Google Scholar 

  4. Ekman, P., Rosenberg, E.L.: What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System (FACS). Oxford University Press, Oxford (2005)

    Book  Google Scholar 

  5. Ekman, P., Friesen, W.V.: Constants across cultures in the face and emotion. J. Pers. Soc. Psychol. 17(2), 124–129 (1971)

    Article  Google Scholar 

  6. Ekman, P., Friesen, W.V., Hager, J.C.: Facial Action Coding System (FACS): Manual. A Human Face, Salt Lake City (2002)

    Google Scholar 

  7. Essa, I.A., Pentland, A.P.: Coding, analysis, interpretation, and recognition of facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 757–763 (1997)

    Article  Google Scholar 

  8. Kanade, T., Cohn, J.F., Tian, Y.: Comprehensive database for facial expression analysis. In: Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition (FG 2000), Grenoble, France, pp. 46–53 (2000)

    Google Scholar 

  9. Khanal, S.R., Barroso, J., Lopes, N., Sampaio, J., Filipe, V.: Performance analysis of Microsoft’s and Google’s emotion recognition API using pose-invariant faces. In: Proceedings of the 8th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion, DSAI 2019, Thessaloniki, Greece, 20–22 June 2018, pp. 172–178. ACM (2018). https://doi.org/10.1145/3218585.3224223

  10. Konar, A., Chakraborty, A. (eds.): Emotion Recognition: A Pattern Analysis Approach. Wiley, New Jersey (2015)

    Google Scholar 

  11. Kutt, K., Drążyk, D., Bobek, S., Nalepa, G.J.: Personality-based affective adaptation methods for intelligent systems. Sensors 21(1), 163 (2021). https://doi.org/10.3390/s21010163

  12. Kutt, K., et al.: BIRAFFE: bio-reactions and faces for emotion-based personalization. In: AFCAI 2019: Workshop on Affective Computing and Context Awareness in Ambient Intelligence. CEUR Workshop Proceedings, vol. 2609. CEUR-WS.org (2020)

  13. Kutt, K., Drążyk, D., Szelążek, M., Bobek, S., Nalepa, G.J.: The BIRAFFE2 experiment. Study in bio-reactions and faces for emotion-based personalization for AI systems. CoRR abs/2007.15048 (2020). https://arxiv.org/abs/2007.15048

  14. Langner, O., Dotsch, R., Bijlstra, G., Wigboldus, D., Hawk, S., van Knippenberg, A.: Presentation and validation of the radboud face database. Cogn. Emotion 24(8), 1377–1388 (2010)

    Article  Google Scholar 

  15. Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended Cohn-Kanade dataset (CK+): a complete dataset for action unit and emotion-specified expression. In: Proceedings of the Third International Workshop on CVPR for Human Communicative Behavior Analysis (CVPR4HB 2010), San Francisco, USA, pp. 94–101, (2010)

    Google Scholar 

  16. Mehrabian, A.: Communication without words. Psychol. Today 2(4), 53–56 (1968)

    Google Scholar 

  17. Nalepa, G.J., Kutt, K., Bobek, S.: Mobile platform for affective context-aware systems. Future Generar. Comput. Syst. 92, 490–503 (2019). https://doi.org/10.1016/j.future.2018.02.033

  18. Picard, R.W.: Affective Computing. MIT Press, Cambridge (1997)

    Google Scholar 

  19. Russell, J.A.: A circumplex model of affect. J. Pers. Soc. Psychol. 39, 1161–1178 (1980)

    Article  Google Scholar 

  20. van der Schalk, J., Hawk, S., Fischer, A., Doosje, B.: Moving faces, looking places: validation of the Amsterdam dynamic facial expression set (ADFES). Emotion 11(4), 907–920 (2011)

    Article  Google Scholar 

  21. Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004). https://doi.org/10.1023/B:VISI.0000013087.49260.fb

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Jagiellonian Human-Centered Artificial Intelligence Laboratory (JAHCAI) and Institute of Applied Computer Science, Jagiellonian University, Kraków, Poland

    Krzysztof Kutt, Piotr Sobczyk & Grzegorz J. Nalepa

Authors
  1. Krzysztof Kutt
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Piotr Sobczyk
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Grzegorz J. Nalepa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Krzysztof Kutt .

Editor information

Editors and Affiliations

  1. Universidad Politécnica de Cartagena, Cartagena, Spain

    José Manuel Ferrández Vicente

  2. Universidad Nacional de Educación a Distancia, Madrid, Spain

    José Ramón Álvarez-Sánchez

  3. Universidad Nacional de Educación a Distancia, Madrid, Spain

    Félix de la Paz López

  4. Ohio State University, Columbus, OH, USA

    Hojjat Adeli

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kutt, K., Sobczyk, P., Nalepa, G.J. (2022). Evaluation of Selected APIs for Emotion Recognition from Facial Expressions. In: Ferrández Vicente, J.M., Álvarez-Sánchez, J.R., de la Paz López, F., Adeli, H. (eds) Bio-inspired Systems and Applications: from Robotics to Ambient Intelligence. IWINAC 2022. Lecture Notes in Computer Science, vol 13259. Springer, Cham. https://doi.org/10.1007/978-3-031-06527-9_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-06527-9_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-06526-2

  • Online ISBN: 978-3-031-06527-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation