Skip to main content
SpringerLink
Log in

A Novel EEG-Based Real-Time Emotion Recognition Approach Using Deep Neural Networks on Raspberry Pi

  • Conference paper
  • First Online:
Human-Computer Interaction (HCII 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14012))

Included in the following conference series:

  • 722 Accesses

Abstract

Automated emotion classification becomes more and more important, as intelligent software systems can better serve users, when they can reliably assess their emotional state and adapt interactive applications accordingly and in real-time. EEG-based brain-computer interfaces (BCI) provide the individual data that can be exploited for emotion classification. However, AI-based emotion classification on EEG-data typically requires computationally intensive training and powerful hardware when the results are needed in real-time. A survey of the related work has shown that not many real-time solutions exist for energy-efficient hardware.

In this paper we present an approach for finding a global best channel set universally suitable for all subjects with high classification accuracy. In our research we used Russel’s emotion model and the DEAP data set. By applying a total of six nature-based swarm channel selection algorithms and one classical selection algorithm, the different algorithms could be compared with each other. The resulting reduced channel set consists of only 7 channels.

With the set it is possible to classify emotions in real-time using low-level energy-efficient hardware. Emotion classification on the Raspberry Pi only takes between 82 and 93ms.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.emotiv.com/epoc-X.

  2. 2.

    https://github.com/Mozartuss/Real-Time-Emotion-Recognition-CNN-LSTM.

References

  1. Ekman, P.: Emotions revealed. BMJ 328, 0405184 (2004). https://doi.org/10.1136/SBMJ.0405184

    Article  Google Scholar 

  2. Gilbert, D.T.: Stumbling on happiness. Vintage Books, New York (2007)

    Google Scholar 

  3. Keltner, D., Lerner, J.S.: Emotion. Handbook of. Social Psychology. (2010). https://doi.org/10.1002/9780470561119.SOCPSY001009

    Article  Google Scholar 

  4. Oatley, K.: Best Laid Schemes: The Psychology of the Emotions (1992)

    Google Scholar 

  5. Kim, S.-H., Yang, H.-J., Nguyen, N.A.T., Prabhakar, S.K., Lee, S.-W.: WeDea: a new EEG-based framework for emotion recognition. IEEE J. Biomed. Health Inform. 26, 264–275 (2022). https://doi.org/10.1109/JBHI.2021.3091187

    Article  Google Scholar 

  6. Pothula, P.K., Marisetty, S., Rao, M.: A real-time seizure classification system using computer vision techniques. SysCon 2022 - 16th Annual IEEE International Systems Conference, Proceedings (2022). https://doi.org/10.1109/SYSCON53536.2022.9773923

  7. Li, W.-C., Yang, C.-J., Liu, B.-T., Fang, W.-C.: A real-time affective computing platform integrated with ai system-on-chip design and multimodal signal processing system. In: 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), pp. 522–526. IEEE (2021). https://doi.org/10.1109/EMBC46164.2021.9630979

  8. Leite, D., Frigeri, V., Medeiros, R.: Adaptive gaussian fuzzy classifier for real-time emotion recognition in computer games. In: 2021 IEEE Latin American Conference on Com-putational Intelligence, LA-CCI 2021 (2021). https://doi.org/10.48550/arxiv.2103.03488

  9. Huang, H., et al.: An EEG-based brain computer interface for emotion recognition and its application in patients with disorder of consciousness. IEEE Trans. Affect Comput. 12, 832–842 (2021). https://doi.org/10.1109/TAFFC.2019.2901456

    Article  Google Scholar 

  10. Kim, S.-H., Yang, H.-J., Nguyen, N.A.T., Lee, S.-W.: AsEmo: automatic approach for EEG-based multiple emotional state identification. IEEE J Biomed. Health Inform. 25, 1508–1518 (2021). https://doi.org/10.1109/JBHI.2020.3032678

    Article  Google Scholar 

  11. Bandara, S.K., Jayalath, B.P., Wijesinghe, U.C., Bandara, S.K., Haddela, P.S., Wick-ramasinghe, L.M.: EEG based real-time system for video advertisement recommendation. In: 21st International Conference on Advances in ICT for Emerging Regions, ICter 2021 - Proceedings, pp. 201–206 (2021). https://doi.org/10.1109/ICTER53630.2021.9774791

  12. Khateeb, M., Anwar, S.M., Alnowami, M.: Multi-domain feature fusion for emotion classification using DEAP dataset. IEEE Access. 9, 12134–12142 (2021). https://doi.org/10.1109/ACCESS.2021.3051281

    Article  Google Scholar 

  13. Wang, K.Y., Huang, Y. de, Ho, Y.L., Fang, W.C.: A customized convolutional neural network design using improved softmax layer for real-time human emotion recognition. In:  Proceedings 2019 IEEE International Conference on Artificial Intelligence Circuits and Systems, AICAS 2019, pp. 102–106 (2019). https://doi.org/10.1109/AICAS.2019.8771616

  14. Yang, C.J., Li, W.C., Wan, M.T., Fang, W.C.: Real-time EEG-based affective computing using on-chip learning long-term recurrent convolutional network. In: Proceedings - IEEE In-ternational Symposium on Circuits and Systems. 2021-May (2021). https://doi.org/10.1109/ISCAS51556.2021.9401588

  15. Li, W.-C., Yang, C.-J., Fang, W.-C.: A real-time emotion recognition system based on an AI system-on-chip design. In: 2020 International SoC Design Conference (ISOCC), pp. 29–30. IEEE (2020). https://doi.org/10.1109/ISOCC50952.2020.9333072

  16. Val-Calvo, M., Alvarez-Sanchez, J.R., Ferrandez-Vicente, J.M., Fernandez, E.: Affective robot story-telling human-robot interaction: exploratory real-time emotion estimation analysis using facial expressions and physiological signals. IEEE Access. 8, 134051–134066 (2020). https://doi.org/10.1109/ACCESS.2020.3007109

    Article  Google Scholar 

  17. Alakus, T.B., Gonen, M., Turkoglu, I.: Database for an emotion recognition system based on EEG signals and various computer games – GAMEEMO. Biomed. Signal Process Control. 60,(2020)

    Google Scholar 

  18. Gonzalez, H.A., Muzaffar, S., Yoo, J., Elfadel, I.M.: An inference hardware accelerator for EEG-based emotion detection. In: Proceedings - IEEE International Symposium on Circuits and Systems. 2020-October (2020). https://doi.org/10.1109/ISCAS45731.2020.9180728/VIDEO

  19. Aslam, A.R., Altaf, M.A.: bin: an on-chip processor for chronic neurological disorders assistance using negative affectivity classification. IEEE Trans. Biomed. Circuits Syst. 14, 838–851 (2020). https://doi.org/10.1109/TBCAS.2020.3008766

    Article  Google Scholar 

  20. Wundt, W.: Grundzuge der Physiologischen Psychologie. Am. J. Psychol. 6, 298 (1894). https://doi.org/10.2307/1410982

    Article  Google Scholar 

  21. Russell, J.A.: A circumplex model of affect. J. Pers. Soc. Psychol. 39, 1161–1178 (1980). https://doi.org/10.1037/h0077714

    Article  Google Scholar 

  22. Koelstra, S., et al.: DEAP: a database for emotion analysis; using physiological signals. IEEE Trans. Affect Comput. 3, 18–31 (2012). https://doi.org/10.1109/T-AFFC.2011.15

    Article  Google Scholar 

  23. Balic, S., Kleybolte, L., Märtin, C.: A Swarm Intelligence Approach: Combination of Different EEG-channel optimization techniques to enhance emotion recognition.  In: Kurosu, M. (eds.) Human-Computer Interaction. Technological Innovation. HCII 2022. Lecture Notes in Computer Science, vol. 13303, pp. 303–317. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-05409-9_23

  24. Developers, T.: TensorFlow (2022). https://doi.org/10.5281/ZENODO.7604251

  25. Harris, C.R., et al.: Array programming with NumPy. Nature 585, 357–362 (2020). https://doi.org/10.1038/s41586-020-2649-2

Download references

Author information

Authors and Affiliations

  1. Faculty of Computer Science, Augsburg University of Applied Sciences, An Der Hochschule 1, 86161, Augsburg, Germany

    Lukas A. Kleybolte & Christian Märtin

Authors
  1. Lukas A. Kleybolte
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christian Märtin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lukas A. Kleybolte .

Editor information

Editors and Affiliations

  1. The Open University of Japan, Chiba, Japan

    Masaaki Kurosu

  2. Hosei University, Tokyo, Japan

    Ayako Hashizume

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kleybolte, L.A., Märtin, C. (2023). A Novel EEG-Based Real-Time Emotion Recognition Approach Using Deep Neural Networks on Raspberry Pi. In: Kurosu, M., Hashizume, A. (eds) Human-Computer Interaction. HCII 2023. Lecture Notes in Computer Science, vol 14012. Springer, Cham. https://doi.org/10.1007/978-3-031-35599-8_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-35599-8_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-35598-1

  • Online ISBN: 978-3-031-35599-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation