Emotion recognition system using short-term monitoring of physiological signals

Article

Abstract

A physiological signal-based emotion recognition system is reported. The system was developed to operate as a user-independent system, based on physiological signal databases obtained from multiple subjects. The input signals were electrocardiogram, skin temperature variation and electrodermal activity, all of which were acquired without much discomfort from the body surface, and can reflect the influence of emotion on the autonomic nervous system. The system consisted of preprocessing, feature extraction and pattern classification stages. Preprocessing and feature extraction methods were devised so that emotion-specific characteristics could be extracted from short-segment signals. Although the features were carefully extracted, their distribution formed a classification problem, with large overlap among clusters and large variance within clusters. A support vector machine was adopted as a pattern classifier to resolve this difficulty. Correct-classification ratios for 50 subjects were 78.4% and 61.8%, for the recognition of three and four categories, respectively.

Keywords

Emotion recognition Autonomic nervous system Physiological signal processing Support vector machine 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Andreassi, J. L. (2000): ‘Psychophysiology: human behavior and physiological response’ (Lawrence Erlbaum Associates, New Jersey, 2000)Google Scholar
  2. Ark, W., Dryer, D. C., andLu, D. J. (1999): ‘The emotion mouse’ 8th Int. Conf. Human-computer Interaction, pp. 818–823Google Scholar
  3. Arkin, R. C., Fujita, M., Takagi, T., andHasegawa, R. (2001): ‘Ethological modeling and architecture for an entertainment robot’. IEEE Int. Conf. Robotics & Automation, pp. 453–458Google Scholar
  4. Berger, R. D., Akselrod, S., Gordon, D., andCohen, R. J. (1986): ‘An efficient algorithm for spectral analysis of heart rate variability’,IEEE Trans. Biomed. Eng.,33, pp. 900–904Google Scholar
  5. Bing-Hwang, J., andFurui, S. (2000): ‘Automatic recognition and understanding of spoken language—a first step toward natural human-machine communication’,Proc. IEEE,88, pp. 1142–1165Google Scholar
  6. Boucsein, W. (1992) ‘Electrodermal activity’ (Plenum Press, New York, 1992)Google Scholar
  7. Broersen, P. M. T. (2000a): ‘Facts and fiction in spectral analysis’,IEEE Trans. Instrum. Meas.,49, pp. 766–772Google Scholar
  8. Broersen, P. M. T. (2000b): ‘Finite sample criteria for autoregressive order selection’,IEEE Trans. Signal Process.,48, pp. 3550–3558MathSciNetGoogle Scholar
  9. Burges, C. J. C. (1998): ‘A tutorial on support vector machines for pattern recognition’,Knowledge Discov. Data Mining,2, pp. 1–43Google Scholar
  10. Chapelle, O., Haffner, P., andVapnik, V. N. (1999): ‘Support vector machines for histogram-based image classification’,IEEE Trans. Neural Netw.,10, pp. 1055–1064CrossRefGoogle Scholar
  11. Cowie, R., Douglas-Cowie, E., Tsapatsoulis, N., Votsis, G., Kollias, S., Fellenz, W., andTaylor, J. G. (2001): ‘Emotion recognition in human-computer interaction’,IEEE Signal Process. Mag.,18, pp. 32–80CrossRefGoogle Scholar
  12. Drummond, P. D., andQuah, S. H. (2001): ‘The effect of expressing anger on cardiovascular reactivity and facial blood flow in Chinese and Caucasians’,Psychophysiology,38, pp. 190–196CrossRefGoogle Scholar
  13. Duda, R. O., Hart, P. E., andStork, D. G. (2001): ‘Pattern classification’, 2nd edn (Wiley, New York, 2001)Google Scholar
  14. Fernandez, R., andPicard, R. W. (1998): ‘Signal processing for recognition of human frustration’. IEEE Int. Conf. Acoustic Speech. Signal Processing, pp. 3773–3776Google Scholar
  15. Friedman, B. H., andThayer, J. F. (1998): Autonomic balance revisited: panic anxiety and heart rate variability’,J. Psychosom. Res.,44, pp. 133–151CrossRefGoogle Scholar
  16. Garcia-Gonzalez, M. A., andPallas-Areny, R. A. (2001): ‘Novel robust index to assess beat-to-beat variability in heart rate timeseries analysis’,IEEE Trans. Biomed. Eng.,48, pp. 617–621CrossRefGoogle Scholar
  17. Haykin, S. (1999): ‘Neural networks’, 2nd edn (Prentice Hall, New Jersey, 1999)Google Scholar
  18. Hirsh, H., Coen, M. H., Mozer, M. C., Hasha, R., andFlanagan, J. L. (1999): ‘Room service, AI-style’,IEEE Intell. Syst.,14, pp. 8–19CrossRefGoogle Scholar
  19. Jiang, L., Qing, Z., andWenyuan, W. (2000): ‘A novel approach to analyze the result of polygraph’. IEEE Int. Conf. Systens Man Cybernetics, pp. 2884–2886Google Scholar
  20. Kataoka, H., Kano, H., Yoshida, H., Saijo, A., Yasuda, M., andOsumi, M. (1998): ‘Development of a skin temperature measuring system for non-contact stress evaluation’. IEEE Ann. Conf. Engineering Medicine Biology Society, pp. 940–943Google Scholar
  21. Knerr, S., Personnaz, L., andDreyfus, G. (1990): ‘Single-layer learning revisited: a stepwise procedure for building and training a neural network’ inFogelman, J. (Ed.): ‘Neurocomputing: algorithms, archtectures and applications’ (Springer-Verlag, Heidelberg, 1990)Google Scholar
  22. Lang, P. J., Ohman, A., andVaitl, D. (1988): ‘The international affective picture system (photographic slides)’. Center for Research in Psychophysiology, University of Florida, Gainesville, USAGoogle Scholar
  23. Maragos, P., Kaiser, J. F., andQuatieri, T. F. (1993): ‘On amplitude and frequency demodulation using energy operators’,IEEE Trans. Signal Process,41, pp. 1532–1550Google Scholar
  24. McCraty, R., Atkinson, M., Tiller, W. A., Rein, G., andWatkins, A. D. (1995): ‘The effects of emotions on short-term power spectrum analysis of heart rate variability’,Am. J. Cardiol.,76, pp. 1089–1093CrossRefGoogle Scholar
  25. Phan, K. L., Wagner, T., Taylor, S. F., andLiberzon, I. (2002): ‘Functional neuroanatomy of emotion: a meta-analysis of emotion activation studies in PET and fMRI’,Neuroimage,16, pp. 331–348CrossRefGoogle Scholar
  26. Picard, R. W. (1995): ‘Affective computing’ (MIT Press, Cambridge, 1995)Google Scholar
  27. Picard, R. W., andHealey, J. (1998): ‘Digital processing of affective signals’,IEEE Int. Conf. Acoustic Speech Signal Processing, pp. 3749–3752Google Scholar
  28. Picard, R. W., Vyzas, E., andHealey, J. (2001): ‘Toward machine emotional intelligence: analysis of affective physiological state’,IEEE Trans. Pattern Anal. Mach. Intell.,23, pp. 1175–1191CrossRefGoogle Scholar
  29. Pinna, G. D., Maestri, R., andSanarico, M. (1996): ‘Effects of record length selection on the accuracy of spectral estimates of heart rate variability’,IEEE Trans. Biomed. Eng.,43, pp. 754–757CrossRefGoogle Scholar
  30. Shusterman, V., andBarnea, O. (1995): ‘Analysis of skin-temperature variability compared to variability of blood pressure and heart rate’,IEEE Ann. Conf. Engineering Medicine Biology Society, pp. 1027–1028Google Scholar
  31. Tefas, A., Kotropoulos, C., andPitas, I. (2001): ‘Using support vector machines to enhance the performance of elastic graph matching for frontal face authentication”,IEEE Trans. Pattern Anal. Mach. Intell.,23, pp. 735–746CrossRefGoogle Scholar
  32. Vapnik, V. N. (1999): ‘An overview of statistical learning theory’,IEEE Trans. Neural Netw.,10, pp. 988–999CrossRefGoogle Scholar
  33. Wang, F., Sagawa, K., andInooka, H. (1998): ‘Time domain heart rate variability index for assessment of dynamic stress’,Comput. Cardiol., pp. 97–100Google Scholar
  34. Yang, G., Lee, K., Lee, J., Choi, J., Bang, S., Kim, J., Lee, H., andSohn, J. (2000): ‘Development of emotion induction protocol for children’,Ann. Conf. Society for Emotion and Sensibility of Korea, pp. 20–25Google Scholar

Copyright information

© IFMBE 2004

Authors and Affiliations

  1. 1.Department of Biomedical Engineering, College of Health ScienceYonsei UniversitySouth Korea
  2. 2.Human-computer Interaction LaboratorySamsung Advanced Institute of TechnologySouth Korea

Personalised recommendations