Abstract
The human body as an entire structure of a person contains physiological and physical reactions that connect with emotions. Emotions can be thought of as psychological states brought on by neurophysiological changes, multifariously accompanied with thoughts, feelings, bodily responses, and a level of pleasure or displeasure. Emotions play a crucial role in our day-to-day activities, not only the way we interact with colleagues but also in our decision-making processes. Emotion recognition from multimodal signals allows a diverse and direct assessment of the innermost state of a person which is regarded an important component of human-computer interactions (HCI). HCI requires the sophisticated use of computer technology, which focuses directly on the interfaces between humans and computers with the purpose of designing technologies allowing humans and computers to interact in novel ways.
In this chapter, we discuss the proposal of a feature learning approach based on multimodal human body data for emotion recognition. We analyzed the broad learning system (BLS) algorithms and explored artificial neural network (ANN) as the deep learning method for classification. We discuss the application of a feature learning technique using a Hyper-Enhanced Learning System (HELS). Experiments are conducted on a database for emotion analysis using physiological signals (DEAP) and a multimodal database for affect and implicit tagging (MAHNOB-HCI) to evaluate the performance of these neuro-hybrid deep and enhanced structures.
We utilized five signals which include electroencephalogram (EEG), galvanic skin response (GSR), respiration (RES), electromyogram (EMG), and electrocardiograph (ECG). Signals were preprocessed to remove artifacts and noise and feature extraction is done to obtain relevant features. We then introduced the proposed feature enhancement system that generates random weights and enhances feature nodes. Training models on enhanced physiological features significantly reduce divergence. Emotions are assorted into three categories and classified based on the valence-arousal dimensional model. Obtained results show that combining physiological signals is relevant for accurate human emotion recognition task.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abdullah, S. M. S. A., Ameen, S. Y. A., Sadeeq, M. A. M., & Zeebaree, S. (2021). Multimodal emotion recognition using deep learning. Journal of Applied Science and Technology Trends, 2(02), 52–58. https://doi.org/10.38094/jastt20291
Ahmad, J., Farman, H., & Jan, Z. (2019). Deep learning methods and applications. In Springer briefs in computer science, pp. 31–42. Springer.
Alswaidan, N., & Menai, M. E. B. (2020). A survey of state-of-the-art approaches for emotion recognition in text. Knowledge and Information Systems, 62(8), 2937–2987. https://doi.org/10.1007/s10115-020-01449-0
Anderson, K., & McOwan, P. W. (2006). A real-time automated system for the recognition of human facial expressions. IEEE Transactions on Systems, Man, and Cybernetics, Part B, Cybernetics, 36(1), 96–105. https://doi.org/10.1109/TSMCB.2005.854502
Araño, K. A., Gloor, P., Orsenigo, C., & Vercellis, C. (2021). When old meets new: Emotion recognition from speech signals. Cognitive Computation, 13(3), 771–783. https://doi.org/10.1007/s12559-021-09865-2
Batbaatar, E., Li, M., & Ryu, K. H. (2019). Semantic-emotion neural network for emotion recognition from text. IEEE Access, 7, 111866–111878. https://doi.org/10.1109/ACCESS.2019.2934529
Baveye, Y., Chamaret, C., Dellandrea, E., & Chen, L. (2018). Affective video content analysis: A multidisciplinary insight. IEEE Transactions on Affective Computing, 9(4), 396–409. https://doi.org/10.1109/TAFFC.2017.2661284
Bota, P., Wang, C., Fred, A., & Silva, H. (2020). Emotion assessment using feature fusion and decision fusion classification based on physiological data: Are we there yet? Sensors (Switzerland), 20(17), 4723. https://doi.org/10.3390/s20174723
Buhrmester, V., Münch, D., & Arens, M. (2021). Analysis of explainers of black box deep neural networks for computer vision: A survey. Machine Learning and Knowledge Extraction, 3(4), 966–989. https://doi.org/10.3390/make3040048
Chen, C. L. P., & Liu, Z. (2018). Broad learning system: An effective and efficient incremental learning system without the need for deep architecture. IEEE Transactions on Neural Networks and Learning Systems, 29(1), 10–24. https://doi.org/10.1109/TNNLS.2017.2716952
Chen, M., Xu, Z., Weinberger, K. Q., & Sha, F. (2012). Marginalized denoising autoencoders for domain adaptation. In Proceedings of the 29th International Conference on Machine Learning, ICML 2012, vol. 1, pp. 767–774.
Chen, C. L. P., Zhang, C. Y., Chen, L., & Gan, M. (2015). Fuzzy restricted Boltzmann machine for the enhancement of deep learning. IEEE Transactions on Fuzzy Systems, 23(6), 2163–2173. https://doi.org/10.1109/TFUZZ.2015.2406889
Chen, D. W., et al. (2019a). A feature extraction method based on differential entropy and linear discriminant analysis for emotion recognition. Sensors (Switzerland), 19(7), 1631. https://doi.org/10.3390/s19071631
Chen, C. L. P., Liu, Z., & Feng, S. (2019b). Universal approximation capability of broad learning system and its structural variations. IEEE Transactions on Neural Networks and Learning Systems, 30(4), 1191–1204. https://doi.org/10.1109/TNNLS.2018.2866622
Chiang, J. T. J., Chen, X. P., Liu, H., Akutsu, S., & Wang, Z. (2021). We have emotions but can’t show them! Authoritarian leadership, emotion suppression climate, and team performance. Human Relations, 74(7), 1082–1111. https://doi.org/10.1177/0018726720908649
Choi, K. H., Kim, J., Kwon, O. S., Kim, M. J., Ryu, Y. H., & Park, J. E. (2017). Is heart rate variability (HRV) an adequate tool for evaluating human emotions? – A focus on the use of the International Affective Picture System (IAPS). Psychiatry Research, 251, 192–196. https://doi.org/10.1016/j.psychres.2017.02.025
Chu, W. L., Huang, M. W., Jian, B. L., & Cheng, K. S. (2017). Analysis of EEG entropy during visual evocation of emotion in schizophrenia. Annals of General Psychiatry, 16(1), 1–9. https://doi.org/10.1186/s12991-017-0157-z
Cimtay, Y., Ekmekcioglu, E., & Caglar-Ozhan, S. (2020). Cross-subject multimodal emotion recognition based on hybrid fusion. IEEE Access, 8, 168865–168878. https://doi.org/10.1109/ACCESS.2020.3023871
Corive, R., et al. (2001). Emotion recognition in human-computer interaction. IEEE Signal Processing Magazine, 18(1), 32–80. https://doi.org/10.1109/79.911197
Dara, S., & Tumma, P. (2018, September). Feature extraction by using deep learning: A survey. In Proceedings of the 2nd International Conference on Electronics, Communication and Aerospace Technology, ICECA 2018, pp. 1795–1801. https://doi.org/10.1109/ICECA.2018.8474912
Drigas, A. S., & Papoutsi, C. (2018). A new layered model on emotional intelligence. Behavioral Sciences (Basel), 8(5), 1–17. https://doi.org/10.3390/bs8050045
Ekman, P. (1992). An argument for basic emotions. Cognition & Emotion, 6(3–4), 169–200. https://doi.org/10.1080/02699939208411068
Erenel, Z., Adegboye, O. R., & Kusetogullari, H. (2020). A new feature selection scheme for emotion recognition from text. Applied Sciences, 10(15), 1–13. https://doi.org/10.3390/APP10155351
Feng, S., & Chen, C. L. P. (2018). A fuzzy restricted Boltzmann machine: Novel learning algorithms based on the crisp possibilistic mean value of fuzzy numbers. IEEE Transactions on Fuzzy Systems, 26(1), 117–130. https://doi.org/10.1109/TFUZZ.2016.2639064
Fesas, A., et al. (2021). Cardiac autonomic nervous system and ventricular arrhythmias: The role of radionuclide molecular imaging. Diagnostics, 11(7), MDPI, 1273. https://doi.org/10.3390/diagnostics11071273
Fordson, P., & Xu, X. (2018). Research on emotion recognition and feature learning method based on Multimodal human data. Dissertation, South China University of Technology. https://cdmd.cnki.com.cn/Article/CDMD-10561-10118875306.htm, pp. 1–53.
Fordson, H. P., Xing, X., Guo, K., & Xu, X. (2021). A feature learning approach based on multimodal human body data for emotion recognition. In 2021 IEEE Signal Processing in Medicine and Biology Symposium, SPMB 2021 - Proceedings, pp. 1–6. https://doi.org/10.1109/SPMB52430.2021.9672303
Fresco, D. M., Mennin, D. S., Moore, M. T., Heimberg, R. G., & Hambrick, J. (2014). Changes in explanatory flexibility among individuals with generalized anxiety disorder in an emotion evocation challenge. Cognitive Therapy and Research, 38(4), 416–427. https://doi.org/10.1007/s10608-014-9601-4
Friedman, N., Geiger, D., & Goldszmidt, M. (1997). Bayesian network classifiers. Machine Learning, 29(2–3), 131–163. https://doi.org/10.1023/a:1007465528199
Goldstein, T., O’Donoghue, B., Setzep, S., & Baraniuk, R. (2014). Fast alternating direction optimization methods. SIAM Journal on Imaging Sciences, 7(3), 1588–1623. https://doi.org/10.1137/120896219
Gong, M., Liu, J., Li, H., Cai, Q., & Su, L. (2015). A multiobjective sparse feature learning model for deep neural networks. IEEE Transactions on Neural Networks and Learning Systems, 26(12), 3263–3277. https://doi.org/10.1109/TNNLS.2015.2469673
Goshvarpour, A., Abbasi, A., & Goshvarpour, A. (2017). An accurate emotion recognition system using ECG and GSR signals and matching pursuit method. Biomedical Journal, 40(6), 355–368. https://doi.org/10.1016/j.bj.2017.11.001
Hagemann, D., Waldstein, S. R., & Thayer, J. F. (2003). Central and autonomic nervous system integration in emotion. Brain and Cognition, 52(1), 79–87. https://doi.org/10.1016/S0278-2626(03)00011-3
Hajarolasvadi, N., Ramirez, M. A., Beccaro, W., & Demirel, H. (2020). Generative adversarial networks in human emotion synthesis: A review. IEEE Access, 8, 218499–218529. https://doi.org/10.1109/ACCESS.2020.3042328
Hajncl, L., & Vučenović, D. (2020). Effects of measures of emotional intelligence on the relationship between emotional intelligence and transformational leadership. Psihološke teme, 29(1), 119–134. https://doi.org/10.31820/pt.29.1.7
Hasnul, M. A., Aziz, N. A. A., Alelyani, S., Mohana, M., & Aziz, A. A. (2021). Electrocardiogram-based emotion recognition systems and their applications in healthcare—A review. Sensors, 21(15), MDPI AG. https://doi.org/10.3390/s21155015
Hinton, G. E., & Salakhutdinov, R. R. (2006). Reducing the dimensionality of data with neural networks. Science (80-), 313(5786), 504–507. https://doi.org/10.1126/science.1127647
Hinton, G. E., Osindero, S., & Teh, Y. W. (2006). A fast learning algorithm for deep belief nets. Neural Computation, 18(7), 1527–1554. https://doi.org/10.1162/neco.2006.18.7.1527
Hsu, Y. L., Wang, J. S., Chiang, W. C., & Hung, C. H. (2020). Automatic ECG-based emotion recognition in music listening. IEEE Transactions on Affective Computing, 11(1), 85–99. https://doi.org/10.1109/TAFFC.2017.2781732
Huang, J., Liu, B., & Tao, J. (2021). Learning long-term temporal contexts using skip RNN for continuous emotion recognition. Virtual Reality & Intelligent Hardware, 3(1), 55–64. https://doi.org/10.1016/j.vrih.2020.11.005
Imani, M., & Montazer, G. A. (2019). A survey of emotion recognition methods with emphasis on E-Learning environments. Journal of Network and Computer Applications, 147. Academic Press, 102423. https://doi.org/10.1016/j.jnca.2019.102423
Issah, M. (2018). Change leadership: The role of emotional intelligence. SAGE Open, 8(3), 1–6. https://doi.org/10.1177/2158244018800910
Jermsittiparsert, K., et al. (2020). Pattern recognition and features selection for speech emotion recognition model using deep learning. International Journal of Speech Technology, 23(4), 799–806. https://doi.org/10.1007/s10772-020-09690-2
Jerritta, S., Murugappan, M., Wan, K., & Yaacob, S. (2014). Emotion recognition from facial EMG signals using higher order statistics and principal component analysis. Journal of the Chinese Institute of Engineers, 37(3), 385–394. https://doi.org/10.1080/02533839.2013.799946
Johnson, E. L., Kam, J. W. Y., Tzovara, A., & Knight, R. T. (2020). Insights into human cognition from intracranial EEG: A review of audition, memory, internal cognition, and causality. Journal of Neural Engineering, 17(5), 051001. https://doi.org/10.1088/1741-2552/abb7a5
Khalil, R. A., Jones, E., Babar, M. I., Jan, T., Zafar, M. H., & Alhussain, T. (2019). Speech emotion recognition using deep learning techniques: A review. IEEE Access, 7, 117327–117345. https://doi.org/10.1109/ACCESS.2019.2936124
Khenkar, S., & Jarraya, S. K. (2022). Engagement detection based on analyzing micro body gestures using 3D CNN. Computers, Materials & Continua, 70(2), 2655–2677. https://doi.org/10.32604/cmc.2022.019152
Koelstra, S., et al. (2012). DEAP: A database for emotion analysis using physiological signals. IEEE Transactions on Affective Computing, 3(1), 18–31. https://doi.org/10.1109/T-AFFC.2011.15
Kollias, D., & Zafeiriou, S. (2021). Exploiting multi-CNN features in CNN-RNN based dimensional emotion recognition on the OMG in-the-wild dataset. IEEE Transactions on Affective Computing, 12(3), 595–606. https://doi.org/10.1109/TAFFC.2020.3014171
Kong, T., Shao, J., Hu, J., Yang, X., Yang, S., & Malekian, R. (2021). Eeg-based emotion recognition using an improved weighted horizontal visibility graph. Sensors, 21(5), 1–22. https://doi.org/10.3390/s21051870
Leshno, M., Lin, V. Y., Pinkus, A., & Schocken, S. (1993). Multilayer feedforward networks with a nonpolynomial activation function can approximate any function. Neural Networks, 6(6), 861–867. https://doi.org/10.1016/S0893-6080(05)80131-5
Li, Y., Kumar, R., Lasecki, W. S., & Hilliges, O. (2020). “Artificial intelligence for HCI: A modern approach. In Conference on Human Factors in Computing Systems - Proceedings, pp. 1–8. https://doi.org/10.1145/3334480.3375147
Liu, Z., et al. (2017). A facial expression emotion recognition based human-robot interaction system. IEEE/CAA Journal of Automatica Sinica, 4(4), 668–676. https://doi.org/10.1109/JAS.2017.7510622
Liu, Y. J., Yu, M., Zhao, G., Song, J., Ge, Y., & Shi, Y. (2018). Real-time movie-induced discrete emotion recognition from EEG signals. IEEE Transactions on Affective Computing, 9(4), 550–562. https://doi.org/10.1109/TAFFC.2017.2660485
Luo, Y., et al. (2020). EEG-based emotion classification using spiking neural networks. IEEE Access, 8, 46007–46016. https://doi.org/10.1109/ACCESS.2020.2978163
Mahata, S., Herencsar, N., & Kubanek, D. (2021). Optimal approximation of fractional-order butterworth filter based on weighted sum of classical butterworth filters. IEEE Access, 9, 81097–81114. https://doi.org/10.1109/ACCESS.2021.3085515
Mano, L. Y., et al. (2019). Using emotion recognition to assess simulation-based learning. Nurse Education in Practice, 36, 13–19. https://doi.org/10.1016/j.nepr.2019.02.017
Mao, S., Tao, D., Zhang, G., Ching, P. C., & Lee, T. (2019, May). Revisiting hidden Markov models for speech emotion recognition. In ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, vol. 2019-May, pp. 6715–6719. https://doi.org/10.1109/ICASSP.2019.8683172
Martínez, A., Pujol, F. A., & Mora, H. (2020). Application of texture descriptors to facial emotion recognition in infants. Applied Sciences, 10(3), 1–15. https://doi.org/10.3390/app10031115
Masood, N., & Farooq, H. (2019). Investigating EEG patterns for dual-stimuli induced human fear emotional state. Sensors (Switzerland), 19(3), 522. https://doi.org/10.3390/s19030522
Mishra, P., & Salankar, N. (2020). Automation of emotion quadrant identification by using second order difference plots and support vector machines. In Proceedings of the IEEE Signal Processing in Medicine and Biology Symposium (SPMB), pp. 1–4. https://doi.org/10.1109/SPMB50085.2020.9353637
Mithbavkar, S. A., & Shah, M. S. (2021). Analysis of EMG based emotion recognition for multiple people and emotions. In 3rd IEEE Eurasia Conference on Biomedical Engineering, Healthcare and Sustainability, ECBIOS 2021, pp. 1–4. https://doi.org/10.1109/ECBIOS51820.2021.9510858
Nithya Roopa, S. (2019). Emotion recognition from facial expression using deep learning. International Journal of Engineering and Advanced Technology, 8. 6 Special Issue, 91–95. https://doi.org/10.35940/ijeat.F1019.0886S19
Nivetha, K., Ragavi Ram, G., & Ajitha, P. (2016, November). Opinion mining from social media using Fuzzy Inference System (FIS). In International Conference on Communication and Signal Processing, ICCSP 2016, pp. 2171–2175. https://doi.org/10.1109/ICCSP.2016.7754566
Özerdem, M. S., & Polat, H. (2017). Emotion recognition based on EEG features in movie clips with channel selection. Brain Informatics, 4(4), 241–252. https://doi.org/10.1007/s40708-017-0069-3
Pao, Y. H., & Takefuji, Y. (1992). Functional-link net computing: Theory, system architecture, and functionalities. Computer (Long. Beach. Calif), 25(5), 76–79. https://doi.org/10.1109/2.144401
Pao, Y. H., Park, G. H., & Sobajic, D. J. (1994). Learning and generalization characteristics of the random vector functional-link net. Neurocomputing, 6(2), 163–180. https://doi.org/10.1016/0925-2312(94)90053-1
Reed, C. L., Moody, E. J., Mgrublian, K., Assaad, S., Schey, A., & McIntosh, D. N. (2020). Body matters in emotion: Restricted body movement and posture affect expression and recognition of status-related emotions. Frontiers in Psychology, 11, 1961. https://doi.org/10.3389/fpsyg.2020.01961
Ren, F., & Bao, Y. (2020). A review on human-computer interaction and intelligent robots. International Journal of Information Technology and Decision Making, 19(1), 5–47. https://doi.org/10.1142/S0219622019300052
Richardson, B., & Li, H. Y. (2021). Designing wearable electronic textiles to detect early signs of neurological injury and disease: A review. In Textile Bioengineering and Informatics Symposium Proceedings 2021 - 14th Textile Bioengineering and Informatics Symposium, TBIS 2021, pp. 11–18.
Rovetta, S., Mnasri, Z., Masulli, F., & Cabri, A. (2021). Emotion recognition from speech: An unsupervised learning approach. International Journal of Computational Intelligence Systems, 14(1), 23–35. https://doi.org/10.2991/ijcis.d.201019.002
Salmam, F. Z., Madani, A., & Kissi, M. (2018). Emotion recognition from facial expression based on fiducial points detection and using neural network. International Journal of Electrical and Computer Engineering, 8(1), 52–59. https://doi.org/10.11591/ijece.v8i1.pp52-59
Schouten, A., Boiger, M., Kirchner-Häusler, A., Uchida, Y., & Mesquita, B. (2020). Cultural differences in emotion suppression in Belgian and Japanese couples: A social functional model. Frontiers in Psychology, 11, 1–12. https://doi.org/10.3389/fpsyg.2020.01048
Shaffer, F., & Ginsberg, J. P. (2017. September 28). An overview of heart rate variability metrics and norms. Frontiers in Public Health, 5. Frontiers Media S.A., 258. https://doi.org/10.3389/fpubh.2017.00258
Shahin, I., Nassif, A. B., & Hamsa, S. (2019). Emotion recognition using hybrid Gaussian mixture model and deep neural network. IEEE Access, 7, 26777–26787. https://doi.org/10.1109/ACCESS.2019.2901352
Shangguan, P., Liu, G., & Wen, W. (2014). The emotion recognition based on GSR signal by curve fitting. Journal of Information and Computing Science, 11(8), 2635–2646. https://doi.org/10.12733/jics20103685
Shu, L., et al. (2018). A review of emotion recognition using physiological signals. Sensors (Switzerland), 18(7), 2074. https://doi.org/10.3390/s18072074
Shukla, J., Barreda-Angeles, M., Oliver, J., Nandi, G. C., & Puig, D. (2019). Feature extraction and selection for emotion recognition from electrodermal activity. IEEE Transactions on Affective Computing, 3045, 1–1. https://doi.org/10.1109/TAFFC.2019.2901673
Sinaga, K. P., & Yang, M. S. (2020). Unsupervised K-means clustering algorithm. IEEE Access, 8, 80716–80727. https://doi.org/10.1109/ACCESS.2020.2988796
Singla, C., Singh, S., & Pathak, M. (2020). Automatic audio based emotion recognition system: Scope and challenges. SSRN Electronic Journal, 6. https://doi.org/10.2139/ssrn.3565861
Soleymani, M., Lichtenauer, J., Pun, T., & Pantic, M. (2012). A multimodal database for affect recognition and implicit tagging. IEEE Transactions on Affective Computing, 3(1), 42–55. https://doi.org/10.1109/T-AFFC.2011.25
Soleymani, M., Villaro-Dixon, F., Pun, T., & Chanel, G. (2017). Toolbox for emotional feature extraction from physiological signals (TEAP). Frontiers in ICT, 4, 1. https://doi.org/10.3389/fict.2017.00001
Song, T., Zheng, W., Lu, C., Zong, Y., Zhang, X., & Cui, Z. (2019). MPED: A multi-modal physiological emotion database for discrete emotion recognition. IEEE Access, 7, 12177–12191. https://doi.org/10.1109/ACCESS.2019.2891579
Song, T., Zheng, W., Song, P., & Cui, Z. (2020). EEG emotion recognition using dynamical graph convolutional neural networks. IEEE Transactions on Affective Computing, 11(3), 532–541. https://doi.org/10.1109/TAFFC.2018.2817622
Subasi, A., Tuncer, T., Dogan, S., Tanko, D., & Sakoglu, U. (2021). EEG-based emotion recognition using tunable Q wavelet transform and rotation forest ensemble classifier. Biomedical Signal Processing and Control, 68, 102648. https://doi.org/10.1016/j.bspc.2021.102648
Taherkhani, A., Cosma, G., & McGinnity, T. M. (2018). Deep-FS: A feature selection algorithm for Deep Boltzmann Machines. Neurocomputing, 322, 22–37. https://doi.org/10.1016/j.neucom.2018.09.040
Tang, J., Deng, C., & Bin Huang, G. (2016). Extreme learning machine for multilayer perceptron. IEEE Transactions on Neural Networks and Learning Systems, 27(4), 809–821. https://doi.org/10.1109/TNNLS.2015.2424995
Thomas, A. R., Pop, N. A., Iorga, A. M., & Ducu, C. (2016). Ethics and neuromarketing: Implications for market research and business practice. Springer International Publishing.
Topic, A., & Russo, M. (2021). Emotion recognition based on EEG feature maps through deep learning network. Engineering Science and Technology, an International Journal, 24(6), 1442–1454. https://doi.org/10.1016/j.jestch.2021.03.012
Wang, S., Li, J., Cao, T., Wang, H., Tu, P., & Li, Y. (2020a). Dance emotion recognition based on Laban motion analysis using convolutional neural network and long short-term memory. IEEE Access, 8, 124928–124938. https://doi.org/10.1109/ACCESS.2020.3007956
Wang, G., Qiao, J., Bi, J., Jia, Q. S., & Zhou, M. C. (2020b). An adaptive deep belief network with sparse restricted Boltzmann machines. IEEE Transactions on Neural Networks and Learning Systems, 31(10), 4217–4228. https://doi.org/10.1109/TNNLS.2019.2952864
Wang, J., Liu, H., Liu, F., & Wang, Q. (2020c). Human-computer interaction speech emotion recognition based on random forest and convolution feature learning. Xitong Fangzhen Xuebao / Journal of System Simulation, 32(12), 2388–2400. https://doi.org/10.16182/j.issn1004731x.joss.20-FZ0494E
Wang, W., Tran, D., & Feiszli, M. (2020d). What makes training multi-modal classification networks hard?. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 12692–12702, https://doi.org/10.1109/CVPR42600.2020.01271
Wei, W., & Jia, Q. (2016). Weighted feature Gaussian Kernel SVM for emotion recognition. Computational Intelligence and Neuroscience, 2016, 1–7. https://doi.org/10.1155/2016/7696035
Wu, J., Zhang, Y., Sun, S., Li, Q., & Zhao, X. (2021). Generalized zero-shot emotion recognition from body gestures. Applied Intelligence, 52, 1–12. https://doi.org/10.1007/s10489-021-02927-w
Xia, R., & Liu, Y. (2016, September). DBN-ivector framework for acoustic emotion recognition. In Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, 2016, vol. 08–12, pp. 480–484. https://doi.org/10.21437/Interspeech.2016-488
Xie, W., & Xue, W. (2021). WB-KNN for emotion recognition from physiological signals. Optoelectronics Letters, 17(7), 444–448. https://doi.org/10.1007/s11801-021-0118-2
Yan, J., Zheng, W., Xin, M., & Yan, J. (2014). Integrating facial expression and body gesture in videos for emotion recognition. IEICE Transactions on Information and Systems, E97-D(3), 610–613. https://doi.org/10.1587/transinf.E97.D.610
Yang, Z., Wang, J., & Chen, Y. (2014). Surface EMG based emotion recognition model for body language of head movements. Jisuanji Fuzhu Sheji Yu Tuxingxue Xuebao/Journal of Computer-Aided Design and Computer, 26(9), 1396–1402.
Yang, H., Han, J., & Min, K. (2019). A multi-column CNN model for emotion recognition from EEG signals. Sensors (Switzerland), 19(21), 1–12. https://doi.org/10.3390/s19214736
Yao, Z., Wang, Z., Liu, W., Liu, Y., & Pan, J. (2020). Speech emotion recognition using fusion of three multi-task learning-based classifiers: HSF-DNN, MS-CNN and LLD-RNN. Speech Communication, 120, 11–19. https://doi.org/10.1016/j.specom.2020.03.005
Yegnanarayana, B. (1994). Artificial neural networks for pattern recognition. Sadhana, 19(2), 189–238. https://doi.org/10.1007/BF02811896
Yu, D., & Sun, S. (2020). A systematic exploration of deep neural networks for EDA-based emotion recognition. Information, 11(4), 212–212. https://doi.org/10.3390/INFO11040212
Yu, Z., Li, L., Liu, J., & Han, G. (2015). Hybrid adaptive classifier ensemble. IEEE Transactions on Cybernetics, 45(2), 177–190. https://doi.org/10.1109/TCYB.2014.2322195
Yu, Z., et al. (2016a). Incremental semi-supervised clustering ensemble for high dimensional data clustering. IEEE Transactions on Knowledge and Data Engineering, 28(3), 701–714. https://doi.org/10.1109/TKDE.2015.2499200
Yu, Z., et al. (2016b). Hybrid k -nearest neighbor classifier. IEEE Transactions on Cybernetics, 46(6), 1263–1275.
Yu, M., et al. (2019). A review of EEG features for emotion recognition. Scientia Sinica Informationis, 49(9), 1097–1118. https://doi.org/10.1360/n112018-00337
Yun, Y., Ma, D., & Yang, M. (2021). Human–computer interaction-based decision support system with applications in data mining. Future Generation Computer Systems, 114, 285–289. https://doi.org/10.1016/j.future.2020.07.048
Zapf, D., Kern, M., Tschan, F., Holman, D., & Semmer, N. K. (2021). Emotion work: A work psychology perspective. Annual Review of Organizational Psychology and Organizational Behavior, 8. Annual Reviews Inc., 139–172. https://doi.org/10.1146/annurev-orgpsych-012420-062451
Zhang, Q., Chen, X., Zhan, Q., Yang, T., & Xia, S. (2017). Respiration-based emotion recognition with deep learning. Computers in Industry, 92–93, 84–90. https://doi.org/10.1016/j.compind.2017.04.005
Zhang, X., et al. (2020a). Emotion recognition from multimodal physiological signals using a regularized deep fusion of kernel machine. IEEE Transactions on Cybernetics, 59(9), 4386–4399. https://doi.org/10.1109/tcyb.2020.2987575
Zhang, J., Yin, Z., Chen, P., & Nichele, S. (2020b). Emotion recognition using multi-modal data and machine learning techniques: A tutorial and review. Information Fusion, 59, 103–126. https://doi.org/10.1016/j.inffus.2020.01.011
Zhang, Y., Zhao, C., Chen, M., & Yuan, M. (2021). Integrating stacked sparse auto-encoder into matrix factorization for rating prediction. IEEE Access, 9, 17641–17648. https://doi.org/10.1109/ACCESS.2021.3053291
Zhao, Y., & Chen, D. (2021). Expression EEG multimodal emotion recognition method based on the bidirectional LSTM and attention mechanism. Computational and Mathematical Methods in Medicine, 2021, 1–12. https://doi.org/10.1155/2021/9967592
Zhao, H., Zheng, J., Deng, W., & Song, Y. (2020). Semi-supervised broad learning system based on manifold regularization and broad network. IEEE Transactions on Circuits and Systems I: Regular Papers, 67(3), 983–994. https://doi.org/10.1109/TCSI.2019.2959886
Zheng, W. (2017). Multichannel EEG-based emotion recognition via group sparse canonical correlation analysis. IEEE Transactions on Cognitive and Developmental Systems, 9(3), 281–290. https://doi.org/10.1109/TCDS.2016.2587290
Zhong, P., Wang, D., & Miao, C. (2020). EEG-based emotion recognition using regularized graph neural networks. IEEE Transactions on Affective Computing, 13, 1–1. https://doi.org/10.1109/taffc.2020.2994159
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Perry Fordson, H., Xing, X., Guo, K., Xu, X., Anderson, A., DeRosa, E. (2023). Hyper-Enhanced Feature Learning System for Emotion Recognition. In: Obeid, I., Picone, J., Selesnick, I. (eds) Signal Processing in Medicine and Biology. Springer, Cham. https://doi.org/10.1007/978-3-031-21236-9_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-21236-9_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-21235-2
Online ISBN: 978-3-031-21236-9
eBook Packages: MedicineMedicine (R0)