Abstract
Affect detection from physiological signals has received considerable attention. One challenge is that physiological measures exhibit considerable variations over time, making classification of future data difficult. The present study addresses this issue by providing insights on how diagnostic physiological features of affect change over time. Affective physiological data (electrocardiogram, electromyogram, skin conductivity, and respiration) was collected from four participants over five sessions each. Classification performance of a number of training strategies, under different conditions of features selection and engineering, were compared using an adaptive classifier ensemble algorithm. Analysis of the performance of individual physiological channels for affect detection is also provided. The key result is that using pooled features set for affect detection is more accurate than using day-specific features. A decision fusion strategy which combines decisions from classifiers trained on individual channels data outperformed a features fusion strategy. Results also show that the performance of the ensemble is affected by the choice of the base classifier and the alpha factor used to update the member classifiers of the ensemble. Finally, the corrugator and zygomatic facial EMGs were found to be more reliable measures for detecting the valence component of affect compared to other channels.
Similar content being viewed by others
References
Allanson J, Fairclough SH (2004) A research agenda for physiological computing. Interact Comput 16(5):857–878
Alzoubi O (2012) Automatic affect detection from physiological signals: practical issues. PhD thesis, University of Sydney, Camperdown
AlZoubi O, D’Mello SK, Calvo RA (2012) Detecting naturalistic expressions of nonbasic affect using physiological signals. IEEE Trans Affect Comput 3(3):298–310
AlZoubi O, Calvo RA, Stevens RH (2009) Classification of EEG for affect recognition: an adaptive approach. In: Nicholson A, Li X (eds) AI 2009: advances in artificial intelligence. Springer, Heidelberg, pp 52–61
AlZoubi O, Hussain MS, D’Mello S, Calvo RA (2011) Affective modeling from multichannel physiology: analysis of day differences. In: Proceedings of the 4th international conference on affective computing and intelligent interaction, vol I. Springer, Heidelberg, pp 4–13
Andreassi JL (2007) Psychophysiology: human behavior and physiological response, 5th edn. Lawrence Erlbaum Associates, New Jersey
Angelov P, Filev DP, Kasabov N (2010) Evolving intelligent systems: methodology and applications, vol 12. Wiley, New York
Bifet A, Holmes G, Pfahringer B, Kirkby R, Gavaldà R (2009) New ensemble methods for evolving data streams. In: Proceedings of the 15th ACM SIGKDD international conference on knowledge discovery and data mining, ACM, New York, NY, USA, KDD ’09, pp 139–148
Bradley M, Lang PJ (2007) The international affective picture system (IAPS) in the study of emotion and attention. Oxford University Press, New York, pp 29–46
van den Broek EL, Schut MH, Westerink JHDM, Tuinenbreijer K (2009) Unobtrusive sensing of emotions (use). J Ambient Intell Smart Environ 1(3):287–299
Chanel G, Kronegg J, Grandjean D, Pun T (2006) Emotion assessment: arousal evaluation using eegs and peripheral physiological signals. In: Gunsel B, Jain AK, Tekalp AM, Sankur B (eds) Multimedia content representation, classification and security. Springer, Heidelberg, pp 530–537
Cieslak D, Chawla N (2009) A framework for monitoring classifiers performance: when and why failure occurs? Knowl Inf Syst 18(1):83–109
Duda R, Hart P, Stork D (2001) Pattern classification. Wiley, New York
Ekman P (1992) An argument for basic emotions. Cognit Emot 6(3):169–200
Ekman P (1994) Moods, emotions and traits. Oxford University Press, New York, 56–58
Gomez P, Zimmermann PG, Schär SG, Danuser B (2009) Valence lasts longer than arousal. J Psychophysiol 23(1):7–17
Haag A, Goronzy S, Schaich P, Williams J (2004) Emotion recognition using bio-sensors: first steps towards an automatic system. In: André E, Dybkjaer L, Minker W, Heisterkamp P (eds) Affective dialogue systems. Springer, Heidelberg, pp 36–48
Hamm AO, Schupp HT, Weike AI (2003) Motivational organization of emotions: autonomic changes, cortical responses, and reflex modulation. In: Davidson RJ, Scherer KR, Goldsmith HH (eds) Handbook of affective sciences. Oxford university press, Oxford, UK, pp 187–211
Heijden F, Duin R, Ridder D, Tax D (2004) Classification, parameter estimation and state estimation—an engineering approach using Matlab. Wiley, Chichester, UK
Jain AK, Duin RPW, Jianchang M (2000) Statistical pattern recognition: a review. IEEE Trans Pattern Anal Mach Intell 22(1):4–37
Kim K, Bang S, Kim S (2004) Emotion recognition system using short-term monitoring of physiological signals. Medical Biol Eng Comput 42(3):419–427
Kim J, André E (2006) Emotion recognition using physiological and speech signal in short-term observation. Perception and interactive technologies. Springer, Berlin, pp 53–64
Kim J, André E (2008) Emotion recognition based on physiological changes in music listening. IEEE Trans Pattern Anal Mach Intell 30(12):2067–2083
Kolter JZ, Maloof M (2003) Dynamic weighted majority: a new ensemble method for tracking concept drift. In: Third IEEE international conference on data mining, ICDM 2003, 2003. IEEE, pp 123–130
Kreibig SD (2010) Autonomic nervous system activity in emotion: a review. Biol Psychol 84(3):394–421
Kuncheva L (2004a) Classifier ensembles for changing environments. In: Roli F, Kittler J, Windeatt T (eds) Multiple classifier systems, vol 3077., Lecture notes in computer scienceSpringer, Berlin, pp 1–15
Kuncheva L (2004b) Combining pattern classifiers: methods and algorithms. Wiley, Hoboken, NJ
Kuncheva L, Christy T, Pierce I, Mansoor S (2011) Multi-modal biometric emotion recognition using classifier ensembles. In: Mehrotra K, Mohan C, Oh J, Varshney P, Ali M (eds) Modern approaches in applied intelligence, vol 6703., Lecture notes in computer scienceSpringer, Berlin, pp 317–326
Lang PJ, Bradley MM, Cuthbert BN et al (2005) International affective picture system (IAPS): affective ratings of pictures and instruction manual. NIMH, Center for the Study of Emotion and Attention, University of Florida, Gainesville, FL
Lang PJ (1995) The emotion probe. studies of motivation and attention. Am Psychol 50(5):372–385
Lang PJ, Bradley MM, Cuthbert BN (1995) International affective picture system (IAPS): technical manual and affective ratings. The Center for Research in Psychophysiology, University of Florida, Gainesville, FL
Last M (2002) Online classification of nonstationary data streams. Intell Data Anal 6(2):129–147
Lazarus R (1991) Emotion and adaptation. Oxford University Press, New York
Lee H, Shackman A, Jackson D, Davidson R (2009) Test–retest reliability of voluntary emotion regulation. Psychophysiol 46(4):874–879
Levenson RW (1992) Autonomic nervous system differences among emotions. Psychol Sci 3(1):23–27
Lichtenstein A, Oehme A, Kupschick S, Jrgensohn T, (2008) Comparing two emotion models for deriving affective states from physiological data. In: Peter C, Beale R (eds) Affect and emotion in human–computer interaction, vol 4868. Lecture notes in computer science. Springer, Berlin, pp 35–50
Lowne DR, Roberts SJ, Garnett R (2010) Sequential non-stationary dynamic classification with sparse feedback. Pattern Recognit 43(3):897–905
Maier-Hein L, Metze F, Schultz T, Waibel A (2005) Session independent non-audible speech recognition using surface electromyography. In: IEEE workshop on automatic speech recognition and understanding, 2005, pp 331–336
Muhlbaier M, Polikar R (2007) An ensemble approach for incremental learning in nonstationary environments. In: Haindl M, Kittler J, Roli F (eds) Multiple classifier systems, vol 4472. Springer, Berlin, pp 490–500
Nishida K, Yamauchi K, Omori T (2005) ACE: adaptive classifiers–ensemble system for concept-drifting environments. In: Oza N, Polikar R, Kittler J, Roli F (eds) Multiple classifier systems. Springer, Berlin. Lecture notes in computer science, vol 3541, pp 176–185
Oza NC, Russell S (2001) Online bagging and boosting. In: Richardson T, Jaakkola T (eds) Artificial intelligence and statistics. Morgan Kaufmann, Los Angeles, CA, pp 105–112
Oza NC, Tumer K (2008) Classifier ensembles: select real-world applications. Inf Fus 9(1):4–20
Picard RW (1997) Affective computing, 2nd edn. The MIT Press, Cambridge, MA
Picard RW, Vyzas E, Healey J (2001) Toward machine emotional intelligence: analysis of affective physiological state. IEEE Trans Pattern Anal Mach Intell 23(10):1175–1191
Plarre K, Raij A, Hossain M, Ali A, Nakajima M, al Absi M, Ertin E, Kamarck T, Kumar S, Scott M, Siewiorek D, Smailagic A, Wittmers L (2011) Continuous inference of psychological stress from sensory measurements collected in the natural environment. In: 10th international conference on information processing in sensor networks (IPSN), 2011, pp 97–108
Polikar R, Upda L, Upda SS, Honavar V (2001) Learn++: an incremental learning algorithm for supervised neural networks. IEEE Trans Syst Man Cybernet C: Applicat Rev 31(4):497–508
Polikar R (2006) Ensemble based systems in decision making. IEEE Circuits Syst Mag 6(3):21–45
Russell JA (1980) A circumplex model of affect. J Personal Soc Psychol 39:1161–1178
Russell JA, Weiss A, Mendelsohn GA (1989) Affect grid: a single-item scale of pleasure and arousal. J Personal Soc Psychol 57(3):493–502
Sayed-Mouchaweh M, Lughofer E (2012) Learning in non-stationary environments. Springer, New York
Vyzas E, Picard RW (1998) Affective pattern classification. In: Canamero D (ed) Emotional and intelligent: the tangled knot of cognition. Proceedings of the AAAI fall Symposium series. AAAI, Menlo Park, CA, pp 176–182
Wagner J (2009) Augsburg biosignal toolbox (aubt). http://hcm-lab.de/files/project_content/33/219_AuBTGuide.pdf. Accessed 25 Apr 2014
Wagner J, Kim J, André E (2005) From physiological signals to emotions: implementing and comparing selected methods for feature extraction and classification. In: IEEE international conference on multimedia and expo, ICME 2005, pp 940–943
Webb AR (2002) Statistical pattern recognition. Wiley, New Jersey
Whang M, Lim J (2008) A physiological approach to affective computing. In: Affective computing: focus on emotion expression, synthesis, and recognition. I-Tech Education and Publishing, Vienna, Austria, pp 310–318
Widmer G, Kubat M (1996) Learning in the presence of concept drift and hidden contexts. Mach Learn 23(1):69–101
Witten I, Frank E (2005) Data mining: practical machine learning tools and techniques, 2nd edn., Series in data management systems. Morgan Kaufmann, Burlington, MA
Yue S, Guojun M, Xu L, Chunnian L (2007) Mining concept drifts from data streams based on multi-classifiers. In: 21st international conference on advanced information networking and applications workshops, AINAW ’07, 2007, vol 2, pp 257–263
Zeng ZH, Pantic M, Roisman GI, Huang TS (2009) A survey of affect recognition methods: audio, visual, and spontaneous expressions. IEEE Trans Pattern Anal Mach Intell 31(1):39–58
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
AlZoubi, O., Fossati, D., D’Mello, S. et al. Affect detection from non-stationary physiological data using ensemble classifiers. Evolving Systems 6, 79–92 (2015). https://doi.org/10.1007/s12530-014-9123-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12530-014-9123-z