Skip to main content
SpringerLink
Log in

A Bimodal System for Emotion Recognition via Computer of Known or Unknown Persons in Normal or Fatigue Situations

  • Chapter
  • First Online:
Advances in Core Computer Science-Based Technologies

Part of the book series: Learning and Analytics in Intelligent Systems ((LAIS,volume 14))

Abstract

The recognition of emotion/mood in groups of people is particularly interesting as it constitutes a whole new research area that can provide solutions to various high level problems and applications, such as, for example, the development of educational technologies for groups of learners in which teaching is adjusted to the mood of the group. The purpose of this chapter is to summarize and present research on the recognition via computer of emotions and/or the mood of a single person or groups of people by using a visual-facial modality combined with a keyboard-based modality. Additionally, this chapter presents research about a program that recognizes the emotions/mood of groups of people in live pictures collected directly from webcams. New algorithmic approaches are proposed and conclusions are drawn for use in similar systems. The implemented system successfully recognizes emotions in the following situations:

  1. 1.

    The system is only trained to recognize emotions/the mood of a specific person.

  2. 2.

    The system is trained with one set of sample emotions/mood from which it aims at generalizing and recognizing emotions/mood of unknown people.

  3. 3.

    The system is trained with several sets of sample emotions/mood from which it aims at generalizing and recognizing emotions/the mood of unknown people.

The chapter is complemented with illustrations of use of the implemented system, as well as suggestions for future related work in this area.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. R.W. Picard, Affective Computing (MIT Press, 1997)

    Google Scholar 

  2. I.-O. Stathopoulou, G.A. Tsihrintzis, Visual Affect Recognition, Frontiers in Artificial Intelligence, vol. 214 (IOS Press, 2010)

    Google Scholar 

  3. P. Ekkekakis, The Measurement of Affect, Mood, and Emotion—A Guide for Health-Behavioral Research (Cambridge University Press, 2013)

    Google Scholar 

  4. E. Alepis, M. Virvou, Object-Oriented User Interfaces for Personalized Mobile Learning, Intelligent Systems Reference Library, vol. 64 (Springer, 2014)

    Google Scholar 

  5. E. Alepis, M. Virvou, S. Drakoulis, Human smartphone interaction: exploring smartphone senses, in Proceedings of the 5th International Conference on Information, Intelligence, Systems and Applications (IISA 2014), Chania, Greece, pp. 44–48 (2014)

    Google Scholar 

  6. S. Bhattacharya, A predictive linear regression model for affective state detection of mobile touch screen users. Int. J. Mob. Hum. Comput. Interact. (IJMHCI) 9(1), 30–44 (2017)

    Article  Google Scholar 

  7. M. Pantic, L.J.M. Rothkrantz, Automatic analysis of facial expressions: the state of the art. IEEE Trans. Pattern Anal. Mach. Intell. 22(12), 1424–1445 (2000)

    Article  Google Scholar 

  8. M. Pantic, L.J. Rothkrantz, Toward an affect-sensitive multimodal human-computer interaction. Proc. IEEE 91(9), 1370–1390 (2003)

    Article  Google Scholar 

  9. A. Jaimes, N. Sebe, Multimodal human-computer interaction: a survey. Comput. Vis. Image Underst. 108(1), 116–134 (2007)

    Article  Google Scholar 

  10. I.-O. Stathopoulou, E. Alepis, G.A. Tsihrintzis, M. Virvou, On assisting a visual-facial affect recognition system with keyboard-stroke pattern information. Knowl.-Based Syst. 23(4), 350–356 (2010)

    Google Scholar 

  11. M.A. Nicolaou, H. Gunes, M. Pantic, Continuous prediction of spontaneous affect from multiple cues and modalities in valence-arousal space. IEEE Trans. Affect. Comput. 2(2), 92–105 (2011)

    Article  Google Scholar 

  12. H. Gunes, B. Schuller, M. Pantic, R. Cowie, Emotion representation, analysis and synthesis in continuous space: a survey, in Proceedings of 2011 IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011), pp. 827–834 (2011)

    Google Scholar 

  13. M. El Ayadi, M.S. Kamel, F. Karray, Survey on speech emotion recognition: features, classification schemes and databases. Pattern Recogn. 44(3), 572–587 (2011)

    Article  Google Scholar 

  14. M. Virvou, G.A. Tsihrintzis, E. Alepis, I.-O. Stathopoulou, K. Kabassi, Emotion recognition: empirical studies towards the combination of audio-lingual and visual-facial modalities through multi-attribute decision making. Int. J. Artif. Intell. Tools 21(02) (2012)

    Google Scholar 

  15. C.-H. Wu, J.-C. Lin, W.-L. Wei, Survey on audiovisual emotion recognition: databases, features, and data fusion strategies. APSIPA Trans. Sig. Inf. Process. 3, e12 (2014)

    Article  Google Scholar 

  16. E. Sariyanidi, H. Gunes, A. Cavallaro, Automatic analysis of facial affect: a survey of registration, representation, and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 37(6), 1113–1133 (2015)

    Article  Google Scholar 

  17. S.S. Tomkins, Affect Imagery Consciousness: Volume I: The Positive Affects (Springer, 1962)

    Google Scholar 

  18. C. Darwin, The Expression of Emotions in Man and Animals, 3rd edn. (Oxford University Press, 1872/1998)

    Google Scholar 

  19. P. Ekman, An argument for basic emotions. Cogn. Emot. 6(3–4), 169–200 (1992)

    Article  Google Scholar 

  20. P. Ekman, Emotions Revealed: Recognizing Faces and Feelings to Improve Communication and Emotional Life (Macmillan, 2007)

    Google Scholar 

  21. D. Watson, L.A. Clark, Measurement and mismeasurement of mood: recurrent and emergent issues. J. Pers. Assess. 68(2), 267–296 (1997)

    Article  Google Scholar 

  22. J.A. Russell, L.F. Barrett, Core affect, prototypical emotional episodes, and other things called emotion: dissecting the elephant. J. Pers. Soc. Psychol. 76(5), 805 (1999)

    Article  Google Scholar 

  23. J.A. Russell, Core affect and the psychological construction of emotion. Psychol. Rev. 110(1), 145 (2003)

    Article  Google Scholar 

  24. C. Beedie, P. Terry, A. Lane, Distinctions between emotion and mood. Cogn. Emot. 19(6), 847–878 (2005)

    Article  Google Scholar 

  25. A.M. Triantafyllou, G.A. Tsihrintzis, Group affect recognition: visual-facial data collection, in Proceedings of 29th IEEE International Conference on Information, Intelligence, Systems and Applications (ICTAI2017), Boston, MA, USA, pp. 677–681, Nov 6–8, 2017

    Google Scholar 

  26. A.M. Triantafyllou, G.A. Tsihrintzis, Group affect recognition: optimization of automatic classification, in M. Virvou, F. Kumeno, K. Oikonomou (eds.), Knowledge-based Software Engineering: 2018, Proceedings of 12th Joint Conference on Knowledge-based Software Engineering (JCKBSE2018), vol. 108 in Smart Innovation, Systems and Technologies (Springer, 2018)

    Google Scholar 

  27. A.M. Triantafyllou, G.A. Tsihrintzis, Group affect recognition: evaluation of basic automated sorting, in Proceedings of 9th IEEE International Conference on Information, Intelligence, Systems and Applications (IISA2018), Zakynthos, Greece, July 18–20, 2018

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Software Engineering Lab, Department of Informatics, University of Piraeus, Piraeus, 185 34, Greece

    Andreas M. Triantafyllou, George A. Tsihrintzis, Maria Virvou & Efthimios Alepis

  2. Pattern Recognition and Machine Learning-Multimedia Systems Lab, Department of Informatics, University of Piraeus, Piraeus, 185 34, Greece

    Andreas M. Triantafyllou, George A. Tsihrintzis, Maria Virvou & Efthimios Alepis

Authors
  1. Andreas M. Triantafyllou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. George A. Tsihrintzis
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Maria Virvou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Efthimios Alepis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to George A. Tsihrintzis .

Editor information

Editors and Affiliations

  1. Department of Informatics, University of Piraeus, Piraeus, Greece

    George A. Tsihrintzis

  2. Department of Informatics, University of Piraeus, Piraeus, Greece

    Maria Virvou

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Triantafyllou, A.M., Tsihrintzis, G.A., Virvou, M., Alepis, E. (2021). A Bimodal System for Emotion Recognition via Computer of Known or Unknown Persons in Normal or Fatigue Situations. In: Tsihrintzis, G., Virvou, M. (eds) Advances in Core Computer Science-Based Technologies. Learning and Analytics in Intelligent Systems, vol 14. Springer, Cham. https://doi.org/10.1007/978-3-030-41196-1_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-41196-1_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-41195-4

  • Online ISBN: 978-3-030-41196-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation