Abstract
This paper addresses emotion recognition based on physiological signal sequences (e.g., blood pressure, galvanic skin response and respiration) that can be obtained using state-of-the-art wearable sensors. We formulate this as a machine learning problem to distinguish sequences labelled with a certain emotion from the other sequences. In particular, we explore how to extract a feature that effectively characterises a sequence and yields accurate emotion recognition. With respect to this, existing methods rely on hand-crafted features that are manually defined based on prior knowledge about physiological signals. However, in addition to intensive labour, it is difficult to manually design features which can represent the details of a sequence. To overcome this, we propose a codebook approach where a sequence is represented with a feature describing the distribution of characteristic subsequences, called codewords. These are statistically justified because they are obtained by clustering a large number of subsequences. In addition, the details of the sequence can be maintained by considering the distribution of hundreds of codewords. Experimental results validate the effectiveness of our codebook-based emotion recognition method.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013)
Chang, C.C., Lin, C.J.: Libsvm: A library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(3), 27:1–27:27 (2011)
Chatfield, K., Lempitsky, V., Vedaldi, A., Zisserman, A.: The devil is in the details: an evaluation of recent feature encoding methods. In: Proceedings of the 22th British Machine Vision Conference (BMVC 2011), pp. 76.1–76.12 (2011)
Chen, Y., Keogh, E., Hu, B., Begum, N., Bagnall, A., Mueen, A., Batista, G.: The UCR time series classification archive (2015). www.cs.ucr.edu/~eamonn/time_series_data/. Accessed 24 Feb 2016
Garbarino, M. et al.: Empatica E3 - A wearable wireless multi-sensor device for real-time computerized biofeedback and data acquisition. In: Proceedings of the 2014 EAI 4th International Conference on Wireless Mobile Communication and Healthcare (Mobihealth 2014), pp. 39–42 (2014)
Han, J., Kamber, M., Pei, J.: Data Mining: Concepts and Techniques, 3rd edn. Morgan Kaufmann (2011)
Healey, J.A.: Wearable and Automotive Systems for Affect Recognition from Physiology. Ph.D. thesis, Massachusetts Institute of Technology (2000)
Hong, J.H., Ramos, J., Dey, A.K.: Understanding physiological responses to stressors during physical activity. In: Proceedings of the 14th ACM International Conference on Ubiquitous Computing (UbiComp 2012), pp. 270–279 (2012)
Jiang, Y.G., Yang, J., Ngo, C.W., Hauptmann, A.G.: Representations of keypoint-based semantic concept detection: a comprehensive study. IEEE Trans. Multimed. 12(1), 42–53 (2010)
Kim, J., Andre, E.: Emotion recognition based on physiological changes in music listening. IEEE Tran. Pattern Anal. Mach. Intell. 30(12), 2067–2083 (2008)
Koelstra, S., Mühl, C., Soleymani, M., Lee, J.S., Yazdani, A., Ebrahimi, T., Pun, T., Nijholt, A., Patras, I.: DEAP: a database for emotion analysis using physiological signals. IEEE Trans. Affect. Comput. 3(1), 18–31 (2012)
Metz, R.: This fitness wristband wants to play doctor. MIT Technol. Rev. (2014). https://www.technologyreview.com/s/524376/this-fitness-wristband-wants-to-play-doctor/. Accessed 24 Feb 2016
Microsoft Corporation: Microsoft Band Official Site. https://www.microsoft.com/microsoft-band/en-us. Accessed 24 Feb 2016
Pentland, A., Lazer, D., Brewer, D., Heibeck, T.: Using reality mining to improve public health and medicine. In: Bushko, R.G. (ed.) Strategy for the Future of Health, pp. 93–102. IOS press (2009)
Picard, R.W., Vyzas, E., Healey, J.: Toward machine emotional intelligence: analysis of affective physiological state. IEEE Trans. Pattern Anal. Mach. Intell. 23(10), 1175–1191 (2001)
Plarre, K. et al.: Continuous inference of psychological stress from sensory measurements collected in the natural environment. In: Proceedings of the 10th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN 2011), pp. 97–108 (2011)
Rashidi, P., Mihailidis, A.: A survey on ambient-assisted living tools for older adults. IEEE J. Biomed. Health Inf. 17(3), 579–590 (2013)
Shirahama, K., Grzegorzek, M.: Towards large-scale multimedia retrieval enriched by knowledge about human interpretation: retrospective survey. Multimed. Tools Appl. 75(1), 297–331 (2016)
Shirahama, K., Uehara, K.: Kobe university and Muroran institute of technology at TRECVID 2012 semantic indexing task. In: Proceedings of the TREC Video Retrieval Evaluation Workshop (TRECVID 2012), pp. 239–247 (2012)
Snoek, C.G.M., Worring, M., Smeulders, A.W.M.: Early versus late fusion in semantic video analysis. In: Proceedings of the 13th Annual ACM International Conference on Multimedia (MM 2005), pp. 399–402 (2005)
Soleymani, M., Lichtenauer, J., Pun, T., Pantic, M.: A multimodal database for affect recognition and implicit tagging. IEEE Trans. Affect. Comput. 3(1), 42–55 (2012)
van de Sande, K.E., Gevers, T., Snoek, C.G.: Evaluating color descriptors for object and scene recognition. IEEE Trans. Pattern Anal. Mach. Intell. 32(9), 1582–1596 (2010)
Van Gemert, J.C., Veenman, C.J., Smeulders, A.W.M., Geusebroek, J.M.: Visual word ambiguity. IEEE Trans. Pattern Anal. Mach. Intell. 32(7), 1271–1283 (2010)
Vapnik, V.N.: Statistical Learning Theory. Wiley-Interscience (1998)
Acknowledgments
Research and development activities leading to this article have been supported by the German Federal Ministry of Education and Research within the project Cognitive Village: Adaptively Learning Technical Support System for Elderly (Grant Number: 16SV7223K).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Shirahama, K., Grzegorzek, M. (2016). Emotion Recognition Based on Physiological Sensor Data Using Codebook Approach. In: Piętka, E., Badura, P., Kawa, J., Wieclawek, W. (eds) Information Technologies in Medicine. ITiB 2016. Advances in Intelligent Systems and Computing, vol 472. Springer, Cham. https://doi.org/10.1007/978-3-319-39904-1_3
Download citation
DOI: https://doi.org/10.1007/978-3-319-39904-1_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-39903-4
Online ISBN: 978-3-319-39904-1
eBook Packages: EngineeringEngineering (R0)