Skip to main content
Log in

Emotion recognition system using short-term monitoring of physiological signals

  • Published:
Medical and Biological Engineering and Computing Aims and scope Submit manuscript

Abstract

A physiological signal-based emotion recognition system is reported. The system was developed to operate as a user-independent system, based on physiological signal databases obtained from multiple subjects. The input signals were electrocardiogram, skin temperature variation and electrodermal activity, all of which were acquired without much discomfort from the body surface, and can reflect the influence of emotion on the autonomic nervous system. The system consisted of preprocessing, feature extraction and pattern classification stages. Preprocessing and feature extraction methods were devised so that emotion-specific characteristics could be extracted from short-segment signals. Although the features were carefully extracted, their distribution formed a classification problem, with large overlap among clusters and large variance within clusters. A support vector machine was adopted as a pattern classifier to resolve this difficulty. Correct-classification ratios for 50 subjects were 78.4% and 61.8%, for the recognition of three and four categories, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Andreassi, J. L. (2000): ‘Psychophysiology: human behavior and physiological response’ (Lawrence Erlbaum Associates, New Jersey, 2000)

    Google Scholar 

  • Ark, W., Dryer, D. C., andLu, D. J. (1999): ‘The emotion mouse’ 8th Int. Conf. Human-computer Interaction, pp. 818–823

  • Arkin, R. C., Fujita, M., Takagi, T., andHasegawa, R. (2001): ‘Ethological modeling and architecture for an entertainment robot’. IEEE Int. Conf. Robotics & Automation, pp. 453–458

  • Berger, R. D., Akselrod, S., Gordon, D., andCohen, R. J. (1986): ‘An efficient algorithm for spectral analysis of heart rate variability’,IEEE Trans. Biomed. Eng.,33, pp. 900–904

    Google Scholar 

  • Bing-Hwang, J., andFurui, S. (2000): ‘Automatic recognition and understanding of spoken language—a first step toward natural human-machine communication’,Proc. IEEE,88, pp. 1142–1165

    Google Scholar 

  • Boucsein, W. (1992) ‘Electrodermal activity’ (Plenum Press, New York, 1992)

    Google Scholar 

  • Broersen, P. M. T. (2000a): ‘Facts and fiction in spectral analysis’,IEEE Trans. Instrum. Meas.,49, pp. 766–772

    Google Scholar 

  • Broersen, P. M. T. (2000b): ‘Finite sample criteria for autoregressive order selection’,IEEE Trans. Signal Process.,48, pp. 3550–3558

    MathSciNet  Google Scholar 

  • Burges, C. J. C. (1998): ‘A tutorial on support vector machines for pattern recognition’,Knowledge Discov. Data Mining,2, pp. 1–43

    Google Scholar 

  • Chapelle, O., Haffner, P., andVapnik, V. N. (1999): ‘Support vector machines for histogram-based image classification’,IEEE Trans. Neural Netw.,10, pp. 1055–1064

    Article  Google Scholar 

  • Cowie, R., Douglas-Cowie, E., Tsapatsoulis, N., Votsis, G., Kollias, S., Fellenz, W., andTaylor, J. G. (2001): ‘Emotion recognition in human-computer interaction’,IEEE Signal Process. Mag.,18, pp. 32–80

    Article  Google Scholar 

  • Drummond, P. D., andQuah, S. H. (2001): ‘The effect of expressing anger on cardiovascular reactivity and facial blood flow in Chinese and Caucasians’,Psychophysiology,38, pp. 190–196

    Article  Google Scholar 

  • Duda, R. O., Hart, P. E., andStork, D. G. (2001): ‘Pattern classification’, 2nd edn (Wiley, New York, 2001)

    Google Scholar 

  • Fernandez, R., andPicard, R. W. (1998): ‘Signal processing for recognition of human frustration’. IEEE Int. Conf. Acoustic Speech. Signal Processing, pp. 3773–3776

  • Friedman, B. H., andThayer, J. F. (1998): Autonomic balance revisited: panic anxiety and heart rate variability’,J. Psychosom. Res.,44, pp. 133–151

    Article  Google Scholar 

  • Garcia-Gonzalez, M. A., andPallas-Areny, R. A. (2001): ‘Novel robust index to assess beat-to-beat variability in heart rate timeseries analysis’,IEEE Trans. Biomed. Eng.,48, pp. 617–621

    Article  Google Scholar 

  • Haykin, S. (1999): ‘Neural networks’, 2nd edn (Prentice Hall, New Jersey, 1999)

    Google Scholar 

  • Hirsh, H., Coen, M. H., Mozer, M. C., Hasha, R., andFlanagan, J. L. (1999): ‘Room service, AI-style’,IEEE Intell. Syst.,14, pp. 8–19

    Article  Google Scholar 

  • Jiang, L., Qing, Z., andWenyuan, W. (2000): ‘A novel approach to analyze the result of polygraph’. IEEE Int. Conf. Systens Man Cybernetics, pp. 2884–2886

  • Kataoka, H., Kano, H., Yoshida, H., Saijo, A., Yasuda, M., andOsumi, M. (1998): ‘Development of a skin temperature measuring system for non-contact stress evaluation’. IEEE Ann. Conf. Engineering Medicine Biology Society, pp. 940–943

  • Knerr, S., Personnaz, L., andDreyfus, G. (1990): ‘Single-layer learning revisited: a stepwise procedure for building and training a neural network’ inFogelman, J. (Ed.): ‘Neurocomputing: algorithms, archtectures and applications’ (Springer-Verlag, Heidelberg, 1990)

    Google Scholar 

  • Lang, P. J., Ohman, A., andVaitl, D. (1988): ‘The international affective picture system (photographic slides)’. Center for Research in Psychophysiology, University of Florida, Gainesville, USA

    Google Scholar 

  • Maragos, P., Kaiser, J. F., andQuatieri, T. F. (1993): ‘On amplitude and frequency demodulation using energy operators’,IEEE Trans. Signal Process,41, pp. 1532–1550

    Google Scholar 

  • McCraty, R., Atkinson, M., Tiller, W. A., Rein, G., andWatkins, A. D. (1995): ‘The effects of emotions on short-term power spectrum analysis of heart rate variability’,Am. J. Cardiol.,76, pp. 1089–1093

    Article  Google Scholar 

  • Phan, K. L., Wagner, T., Taylor, S. F., andLiberzon, I. (2002): ‘Functional neuroanatomy of emotion: a meta-analysis of emotion activation studies in PET and fMRI’,Neuroimage,16, pp. 331–348

    Article  Google Scholar 

  • Picard, R. W. (1995): ‘Affective computing’ (MIT Press, Cambridge, 1995)

    Google Scholar 

  • Picard, R. W., andHealey, J. (1998): ‘Digital processing of affective signals’,IEEE Int. Conf. Acoustic Speech Signal Processing, pp. 3749–3752

  • Picard, R. W., Vyzas, E., andHealey, J. (2001): ‘Toward machine emotional intelligence: analysis of affective physiological state’,IEEE Trans. Pattern Anal. Mach. Intell.,23, pp. 1175–1191

    Article  Google Scholar 

  • Pinna, G. D., Maestri, R., andSanarico, M. (1996): ‘Effects of record length selection on the accuracy of spectral estimates of heart rate variability’,IEEE Trans. Biomed. Eng.,43, pp. 754–757

    Article  Google Scholar 

  • Shusterman, V., andBarnea, O. (1995): ‘Analysis of skin-temperature variability compared to variability of blood pressure and heart rate’,IEEE Ann. Conf. Engineering Medicine Biology Society, pp. 1027–1028

  • Tefas, A., Kotropoulos, C., andPitas, I. (2001): ‘Using support vector machines to enhance the performance of elastic graph matching for frontal face authentication”,IEEE Trans. Pattern Anal. Mach. Intell.,23, pp. 735–746

    Article  Google Scholar 

  • Vapnik, V. N. (1999): ‘An overview of statistical learning theory’,IEEE Trans. Neural Netw.,10, pp. 988–999

    Article  Google Scholar 

  • Wang, F., Sagawa, K., andInooka, H. (1998): ‘Time domain heart rate variability index for assessment of dynamic stress’,Comput. Cardiol., pp. 97–100

  • Yang, G., Lee, K., Lee, J., Choi, J., Bang, S., Kim, J., Lee, H., andSohn, J. (2000): ‘Development of emotion induction protocol for children’,Ann. Conf. Society for Emotion and Sensibility of Korea, pp. 20–25

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to K. H. Kim.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kim, K.H., Bang, S.W. & Kim, S.R. Emotion recognition system using short-term monitoring of physiological signals. Med. Biol. Eng. Comput. 42, 419–427 (2004). https://doi.org/10.1007/BF02344719

Download citation

  • Received:

  • Accepted:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02344719

Keywords

Navigation