Abstract
Recently, commercial physiological sensors and computing devices have become cheaper and more accessible, while computer systems have become increasingly aware of their contexts, including but not limited to users’ emotions. Consequently, many studies on emotion recognition have been conducted. However, boredom has received relatively little attention as a target emotion due to its diverse nature. Moreover, only a few researchers have tried classifying boredom using electroencephalogram (EEG). In this study, to perform this classification, we first reviewed studies that tried classifying emotions using EEG. Further, we designed and executed an experiment, which used a video stimulus to evoke boredom and non-boredom, and collected EEG data from 28 Korean adult participants. After collecting the data, we extracted its absolute band power, normalized absolute band power, differential entropy, differential asymmetry, and rational asymmetry using EEG, and trained these on three machine learning algorithms: support vector machine, random forest, and k-nearest neighbors (k-NN). We validated the performance of each training model with 10-fold cross validation. As a result, we achieved the highest accuracy of 86.73% using k-NN. The findings of this study can be of interest to researchers working on emotion recognition, physiological signal processing, machine learning, and emotion-aware system development.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Aha DW, Kibler D, Albert MK (1991) Instance-based learning algorithms. Mach Learn 6(1):37–66. https://doi.org/10.1007/BF00153759
van der Akker J, Keursten P, Plomp T (1992) The integration of computer use in education. Int J Educ Res 17(1):65–76
Aksoy S, Haralick RM (2001) Feature normalization and likelihood-based similarity measures for image retrieval. Pattern Recogn Lett 22(5):563–582
Alm CO, Roth D, Sproat R (2005) Emotions from text: machine learning for text-based emotion prediction. In: Proceedings of the conference on human language technology and Empirical methods in natural language processing, association for computational Linguistics, Stroudsburg, PA, USA, HLT ’05, pp 579–586, https://doi.org/10.3115/1220575.1220648
Altman NS (1992) An introduction to kernel and nearest-neighbor nonparametric regression. Am Stat 46(3):175–185. https://doi.org/10.1080/00031305.1992.10475879
Baker R, D’Mello S, Rodrigo M, Graesser A (2010) Better to be frustrated than bored: the incidence and persistence of affect during interactions with three different computer-based learning environments. Int J Hum Comput Stud 68(4):223–241
Barandiaran I (1998) The random subspace method for constructing decision forests. IEEE Trans Pattern Anal Mach Intell 20(8):56
Baumgartner T, Esslen M, Jäncke L (2006) From emotion perception to emotion experience: emotions evoked by pictures and classical music. Int J Psychophysiol 60(1):34–43. https://doi.org/10.1016/j.ijpsycho.2005.04.007
Bechara A, Damasio H, Damasio AR (2000) Emotion, decision making and the orbitofrontal cortex. Cerebral Cortex 10(3):295–307. https://doi.org/10.1093/cercor/10.3.295
Bednarik R, Vrzakova H, Hradis M (2012) What do you want to do next: a novel approach for intent prediction in gaze-based interaction. In: Proceedings of the symposium on eye tracking research and applications, ACM, pp 83–90
Bereiter C (2002) Education and mind in the knowledge age. Routledge, Abingdon
Bruce BC, Rubin A (1993) Electronic quills a situated evaluation of using computers for writing in classrooms. Routledge, Abingdon
Busso C, Deng Z, Yildirim S, Bulut M, Lee CM, Kazemzadeh A, Lee S, Neumann U, Narayanan S (2004) Analysis of emotion recognition using facial expressions, speech and multimodal information. In: Proceedings of the 6th international conference on Multimodal interfaces—ICMI ’04, https://doi.org/10.1145/1027933.1027968,arXiv:1011.1669v3
Castellano G, Kessous L, Caridakis G (2008) Emotion recognition through multiple modalities: face, body gesture, speech. Affect and emotion in human-computer interaction. Springer, Berlin, pp 92–103
Chanel G, Kronegg J, Grandjean D, Pun T (2006) Emotion assessment: arousal evaluation using EEG’s and peripheral physiological signals. In: Gunsel B, Jain AK, Tekalp AM, Sankur B (eds) Multimedia content representation, classification and security. MRCS 2006. Lecture notes in computer science, vol 4105. Springer, Berlin, Heidelberg
Chi TS, Yeh LY, Hsu CC (2012) Robust emotion recognition by spectro-temporal modulation statistic features. J Ambient Intell Hum Comput 3(1):47–60. https://doi.org/10.1007/s12652-011-0088-5
Chuang ZJ, Wu Ch (2004) Multi-modal emotion recognition from speech and text. J Comput Linguist Chin Lang Process 9(2):45–62
Cole M (1998) Cultural psychology: a once and future discipline. Harvard University Press, Cambridge
Davidson RJ (1992) Anterior cerebral asymmetry and the nature of emotion. Brain Cogn 20(1):125–151
Davidson RJ, Fox NA (1982) Asymmetrical brain activity discriminates between positive and negative affective stimuli in human infants. Science 218(4578):1235–1237
Eastwood JD, Frischen A, Fenske MJ, Smilek D (2012) The unengaged mind defining boredom in terms of attention. Perspect Psychol Sci 7(5):482–495. https://doi.org/10.1177/1745691612456044
Fagerberg P, Ståhl A, Höök K (2004) EMoto: emotionally engaging interaction. Pers Ubiquitous Comput 8(5):377–381
Fahlman SA, Mercer-Lynn KB, Flora DB, Eastwood JD (2013) Development and validation of the multidimensional state boredom scale. Assessment 20(1):68–85. https://doi.org/10.1177/1073191111421303
Feldman L (1995) Variations in the circumplex structure of mood. Personal Soc Psychol Bull 21(8):806–817
Giakoumis D, Vogiannou A, Kosunen I, Moustakas K, Tzovaras D, Hassapis G (2010) Identifying psychophysiological correlates of boredom and negative mood induced during HCI. In: Bio-inspired human-machine interfaces and healthcare applications, January 2010, pp 3–12, https://doi.org/10.5220/0002812600030012
Giakoumis D, Tzovaras D, Moustakas K, Hassapis G (2011) Automatic recognition of boredom in video games using novel biosignal moment-based features. IEEE Trans Affect Comput 2(3):119–133. https://doi.org/10.1109/T-AFFC.2011.4
Glowinski D, Camurri A, Volpe G, Dael N, Scherer K (2008) Technique for automatic emotion recognition by body gesture analysis. In: 2008 IEEE computer society conference on computer vision and pattern recognition workshops, CVPR Workshops pp 1–6, https://doi.org/10.1109/CVPRW.2008.4563173
Gruber S, Peyton JK, Bruce BC (1994) Collaborative writing in multiple discourse contexts. CSCW 3(3–4):247–269. https://doi.org/10.1007/BF00750742
Gunes H, Piccardi M (2007) Bi-modal emotion recognition from expressive face and body gestures. J Netw Comput Appl 30(4):1334–1345. https://doi.org/10.1016/j.jnca.2006.09.007
Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten IH (2009) The weka data mining software: an update. ACM SIGKDD Explor Newslett 11(1):10–18
Ho TK (1995) Random decision forests. In: Document analysis and recognition, 1995, proceedings of the third international conference on, IEEE, vol 1, pp 278–282
Horlings R, Datcu D, Rothkrantz LJM (2008) Emotion recognition using brain activity. In: Proceedings of the 9th international conference on computer systems and technologies and workshop for PhD students in computing, ACM, March, p 6
Jang EH, Park BJ, Park MS, Kim SH, Sohn JH (2015) Analysis of physiological signals for recognition of boredom, pain, and surprise emotions. J Physiol Anthropol 34(1):1–12. https://doi.org/10.1186/s40101-015-0063-5
Jaques N, Conati C, Harley JM, Azevedo R (2014) Predicting affect from gaze data during interaction with an intelligent tutoring system. Lect Notes Comput Sci 8474:29–38
Jasper H (1958) Report of the committee on methods of clinical examination in electroencephalography: 1957. Electroencephalogr Clin Neurophysiol 10(2):370–375. https://doi.org/10.1016/0013-4694(58)90053-1
Jiang R, Ho AT, Cheheb I, Al-Maadeed N, Al-Maadeed S, Bouridane A (2017) Emotion recognition from scrambled facial images via many graph embedding. Pattern Recogn 67:245–251. https://doi.org/10.1016/j.patcog.2017.02.003
Kanevsky LS (1994) A comparative study of children’s learning in the zone of proximal development. Eur J High Abil 5(2):163–175. https://doi.org/10.1080/0937445940050206
Katahira K, Yamazaki Y, Yamaoka C, Ozaki H, Nakagawa S, Nagata N (2018) Eeg correlates of the flow state: a combination of increased frontal theta and moderate frontocentral alpha rhythm in the mental arithmetic task. Front Psychol 9:300. https://doi.org/10.3389/fpsyg.2018.00300
Khalili Z, Moradi MH (2008) Emotion detection using brain and peripheral signals. In: 2008 Cairo international biomedical engineering conference, pp 1–4, https://doi.org/10.1109/CIBEC.2008.4786096
Kim D, Kim Y, Kim S, Park Y, Park J, Bae K, Lee S, Lee J, Lim C, Jeon Y, Jin S, Chae J, Hwang H (2017) Understanding and application of EEG. Hakjisa, Seogyo-dong
Kim DJ, Kim YS (2010) Pattern classification of four emotions using EEG. J Korea Inst Inf Electron Commun Technol 3:23–27
Kim HD, Sim KB (2007) Brain-wave analysis using fMRI, TRS and EEG for human emotion recognition. In: Proceedings of KFIS autumn conference 2007, Korean Institute of Intelligent Systems, vol 17, pp 7–10
Kim J, Seo J, Laine TH (2018) Detecting Boredom from Eye Gaze and EEG. Biomed Signal Process Control 46:302–313
Lang PJ, Bradley MM, Cuthbert BN (2008) International affective picture system (IAPS): affective ratings of pictures and instruction manual, Technical Report. University of Florida, Gainesville, Florida
Lee CC, Shih CY, Lai WP, Lin PC (2012) An improved boosting algorithm and its application to facial emotion recognition. J Ambient Intell Hum Comput 3(1):11–17. https://doi.org/10.1007/s12652-011-0085-8
Lee HJ, Shin DI, Shin DK (2013) A Study on an emotion-classification algorithm of users adapting Brainwave. In: Proceedings of symposium of the Korean Institute of communications and Information Sciences, Korea Institute Of Communication Sciences, pp 786–787
Lee HJ, Shin DI, Shin DK (2014) The classification algorithm of users emotion using brain-wave. J Kor Inst Commun Inf Sci 39(02):122–129
Lee S, Song K (2007) The effects of affective feedbacks according to the learner’s emotions in e-learning. J Kor Soc Comput Inf 10(4):56
Li M, Lu BL (2009) Emotion classification based on gamma-band EEG. In: 2009 Annual International Conference of the IEEE Engineering in medicine and biology society pp 1223–1226, https://doi.org/10.1109/IEMBS.2009.5334139
Lin HCK, Hsieh MC, Loh LC, Wang CH (2012) An emotion recognition mechanism based on the combination of mutual information and semantic clues. J Ambient Intell Hum Comput 3(1):19–29. https://doi.org/10.1007/s12652-011-0086-7
Lin YP, Wang CH, Wu TL, Jeng SK, Chen JH (2008) Support vector machine for EEG signal classification during listening to emotional music. In: 2008 IEEE 10th workshop on multimedia signal processing, pp 127–130, https://doi.org/10.1109/MMSP.2008.4665061
Lin YP, Wang CH, Wu TL, Jeng SK, Chen JH (2009) EEG-based emotion recognition in music listening: a comparison of schemes for multiclass support vector machine. In: 2009 IEEE international conference on Acoustics, speech and signal processing pp 489–492, https://doi.org/10.1109/ICASSP.2009.4959627
Lin YP, Wang CH, Jung TP, Wu TL, Jeng SK, Duann JR, Chen JH (2010) EEG-based emotion recognition in music listening. IEEE Trans Biomed Eng 57(7):1798–1806. https://doi.org/10.1109/TBME.2010.2048568
Logeswaran N, Bhattacharya J (2009) Crossmodal transfer of emotion by music. Neurosci Lett 455(2):129–133. https://doi.org/10.1016/j.neulet.2009.03.044
Mandryk RL, Atkins MS (2007) A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies. Int J Hum Comput Stud 65(4):329–347
Mao X, Li Z (2009) Implementing emotion-based user-aware e-learning. Int Conf Hum Factors Comput Syst 56:3787–3792
Mistry K, Zhang L, Neoh SC, Lim CP, Fielding B (2017) A micro-GA embedded PSO feature selection approach to intelligent facial emotion recognition. IEEE Trans Cybern 47(6):1496–1509. https://doi.org/10.1109/TCYB.2016.2549639
Mohammadpour M, Hashemi SMR, Houshmand N (2017) Classification of EEG-based emotion for BCI applications. In: 2017 artificial intelligence and robotics (IRANOPEN), pp 127–131, https://doi.org/10.1109/RIOS.2017.7956455
Murugappan M, Rizon M, Nagarajan R, Yaacob S, Zunaidi I, Hazry D (2008) Lifting scheme for human emotion recognition using EEG. In: 2008 International symposium on information technology 2:0–6, https://doi.org/10.1109/ITSIM.2008.4631646
MUSE (2018) MUSE for developer. http://developer.choosemuse.com/ Accessed: 2018-11-30
Nicolopoulou A, Cole M (1996) Generation and transmission of shared knowledge in the culture of collaborative learning: the Fifth Dimension, its play-world and its institutional contexts. Oxford University Press, Oxford
Nie D, Wang XW, Shi LC, Lu BL (2011) EEG-based emotion recognition during watching movies. In: 2011 5th International IEEE/EMBS conference on neural engineering, pp 667–670, https://doi.org/10.1109/NER.2011.5910636
Nwe TL, Foo SW, De Silva LC (2003) Speech emotion recognition using hidden Markov models. Speech Commun 41(4):603–623. https://doi.org/10.1016/S0167-6393(03)00099-2
Oostenveld R, Praamstra P (2001) The five percent electrode system for high-resolution eeg and erp measurements. Clin Neurophysiol 112(4):713–719. https://doi.org/10.1016/S1388-2457(00)00527-7
Oroujlou N, Vahedi M (2011) Motivation, attitude, and language learning. Procedia Soc Beh Sci 29:994–1000. https://doi.org/10.1016/j.sbspro.2011.11.333
Petrantonakis PC, Hadjileontiadis LJ (2010) Emotion recognition from brain signals using hybrid adaptive filtering and higher order crossings analysis. IEEE Trans Affect Comput 1(2):81–97. https://doi.org/10.1109/T-AFFC.2010.7
Petrantonakis PC, Hadjileontiadis LJ (2011) A novel emotion elicitation index using frontal brain asymmetry for enhanced EEG-Based emotion recognition. IEEE Trans Inf Technol Biomed 15(5):737–746. https://doi.org/10.1109/TITB.2011.2157933
Picard RW (1995) Affective computing. Tech Rep 5:6–12. https://doi.org/10.1007/BF01238028
Rosenblatt F (1961) Principles of neurodynamics perceptrons and the theory of brain mechanisms. Cornell aeronautical lab, Buffalo
Russell JA (1980) A circumplex model of affect. J Personal Soc Psychol 39(6):1161–1178. https://doi.org/10.1037/h0077714
Sanei S, Chambers JA (2013) EEG signal processing. Wiley, Hoboken
Scardamalia M, Bereiter C (1994) Computer support for knowledge-building communities. J Learn Sci 3(3):265–283. https://doi.org/10.1207/s15327809jls0303_3
Schmidt LA, Trainor LJ (2001) Frontal brain electrical activity (EEG) distinguishes valence and intensity of musical emotions. Cogn Emot 15(4):487–500. https://doi.org/10.1080/02699930126048
Schuller B, Rigoll G, Lang M (2003) Hidden markov model-based speech emotion recognition. In: 2003 IEEE international conference on Acoustics, speech, and signal processing, 2003. Proceedings. (ICASSP ’03)., vol 2, pp II–1, https://doi.org/10.1109/ICASSP.2003.1202279
Shen L, Leon E, Callaghan V, Shen R (2007) Exploratory research on an Affective e-Learning Model. In: Proceedings of workshop on blended learning, vol 267-278, 10.1007/978-3-540-78139-4, 9780201398298
Shen L, Wang M, Shen R (2009) Affective e-learning: using emotional data to improve learning in pervasive learning environment related work and the pervasive e-learning platform. Educ Technol Soc 12(2):176–189
Sidney KD, Craig SD, Gholson B, Franklin S, Picard R, Graesser AC (2005) Integrating affect sensors in an intelligent tutoring system. In: In affective interactions: the computer in the affective loop workshop at 2005 international conference on intelligent user interfaces, pp 7–13
Singh G, Jati A, Khasnobish A, Bhattacharyya S, Konar A, Tibarewala DN, Janarthanan R (2012) Negative emotion recognition from stimulated EEG signals. In: Computing communication networking technologies (ICCCNT), 2012 third international conference on, pp 1–8, https://doi.org/10.1109/ICCCNT.2012.6395891
Sottilare R, Goldberg B (2012) Designing adaptive computer-based tutoring systems to accelerate learning and facilitate retention. J Cogn Technol 17(1):19–33
Takahashi K (2005) Remarks on emotion recognition from bio-potential signals. In: in 2nd Int. Conf. on Autonomous Robots and Agents
Tan JW, Walter S, Scheck A, Hrabal D, Hoffmann H, Kessler H, Traue HC (2012) Repeatability of facial electromyography (EMG) activity over corrugator supercilii and zygomaticus major on differentiating various emotions. J Ambient Intell Hum Comput 3(1):3–10. https://doi.org/10.1007/s12652-011-0084-9
Timmermann M, Jeung H, Schmitt R, Boll S, Freitag CM, Bertsch K, Herpertz SC (2017) Oxytocin improves facial emotion recognition in young adults with antisocial personality disorder. Psychoneuroendocrinology 85(July):158–164. https://doi.org/10.1016/j.psyneuen.2017.07.483
Vijayan AE, Sen D, Sudheer A (2015) EEG-Based Emotion Recognition Using Statistical Measures and Auto-Regressive Modeling. In: 2015 IEEE international conference on computational intelligence and communication technology, pp 587–591, https://doi.org/10.1109/CICT.2015.24
Vogel-Walcutt JJ, Fiorella L, Carper T, Schatz S (2012) The definition, assessment, and mitigation of state boredom within educational settings: a comprehensive review. Educ Psychol Rev 24(1):89–111. https://doi.org/10.1007/s10648-011-9182-7
Wu CH, Chuang ZJ, Lin YC (2006) Emotion recognition from text using semantic labels and separable mixture models. ACM Trans Asian Lang Inf Process 5(2):165–183. https://doi.org/10.1145/1165255.1165259
Wundt W (1897) Outline of psychology (CH Judd, Trans.). Williams and Norgate, London, Great Britain (Wilhelm Engelmann, Leipzig, Germany)
Yeager DS, Henderson MD, Paunesku D, Walton GM, D'Mello S, Spitzer BJ, Duckworth AL (2014) Boring but important: a self-transcendent purpose for learning fosters academic self-regulation. J Pers Soc Psychol 107(4):559
Zhao Y, Wang X, Goubran M, Whalen T, Petriu EM (2013) Human emotion and cognition recognition from body language of the head using soft computing techniques. J Ambient Intell Hum Comput 4(1):121–140. https://doi.org/10.1007/s12652-012-0107-1
Zheng WL, Lu BL (2015) Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE Trans Auton Ment Dev 7(3):162–175. https://doi.org/10.1109/TAMD.2015.2431497
Zheng WL, Zhu JY, Lu BL (2017) Identifying stable patterns over time for emotion recognition from EEG. In: IEEE transactions on affective computing PP(99):1, https://doi.org/10.1109/TAFFC.2017.2712143
Acknowledgements
This research was supported by the MSIT (Ministry of Science and ICT), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2018-2018-0-01431) supervised by the IITP (Institute for Information & communications Technology Promotion). We would like to show our appreciation to the participants in our data collection. Special thanks to Ms. Wooryeon Go (Emily), Dr. Carolina Islas Sedano, Dr. Hana Vrzakova, Dr. Roman Bednarik, and Dr. Bednarik’s research group members at the University of Eastern Finland. Furthermore, we would like to thank Mr. Joochan Kim, at the Luleå University of Technology in Sweden.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Seo, J., Laine, T.H. & Sohn, KA. Machine learning approaches for boredom classification using EEG. J Ambient Intell Human Comput 10, 3831–3846 (2019). https://doi.org/10.1007/s12652-019-01196-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12652-019-01196-3