Abstract
Multi-modal context-aware systems can provide user-adaptive services, but it requires complicated recognition models with larger resources. The limitations to build optimal models and infer the context efficiently make it difficult to develop practical context-aware systems. We developed a multi-modal context-aware system with various wearable sensors including accelerometers, gyroscopes, physiological sensors, and data gloves. The system used probabilistic models to handle the uncertain and noisy time-series sensor data. In order to construct the efficient probabilistic models, this paper uses an evolutionary algorithm to model structure and EM algorithm to determine parameters. The trained models are selectively inferred based on a semantic network which describes the semantic relations of the contexts and sensors. Experiments with the real data collected show the usefulness of the proposed method.
Similar content being viewed by others
References
Lester J, Choudhury T, Kern N, Borriello G, Hannaford B (2005) A hybrid discriminative/generative approach for modeling human activities. Paper presented at the proceedings of the 19th international joint conference on artificial intelligence, Edinburgh, Scotland
Choudhury T, Borriello G, Consolvo S, Haehnel D, Harrison B, Hemingway B, Hightower J, Klasnja PP, Koscher K, LaMarca A, Landay JA, LeGrand L, Lester J, Rahimi A, Rea A, Wyatt D (2008) The mobile sensing platform: an embedded activity recognition system. Pervasive Comput IEEE 7(2):32–41. doi:10.1109/MPRV.2008.39
Ogris G, Stiefmeier T, Lukowicz P, Troster G (2008) Using a complex multi-modal on-body sensor system for activity spotting. Paper presented at the proceedings of the 2008 12th IEEE international symposium on wearable computers
Roy N, Gu T, Das SK (2010) Supporting pervasive computing applications with active context fusion and semantic context delivery. Pervasive Mobile Comput 6(1):21–42. doi:10.1016/j.pmcj.2009.10.002
Van Laerhoven K, Aidoo KA, Lowette S (2001) Real-time analysis of data from many sensors with neural networks. In: Proceedings of the fifth international symposium on wearable computers, 2001, pp 115–122. doi:10.1109/iswc.2001.962112
Wang Y, Lin J, Annavaram M, Jacobson QA, Hong J, Krishnamachari B, Sadeh N (2009) A framework of energy efficient mobile sensing for automatic user state recognition. Paper presented at the proceedings of the 7th international conference on mobile systems, applications, and services, Kraków, Poland
Krause A, Ihmig M, Rankin E, Leong D, Gupta S, Siewiorek D, Smailagic A, Deisher M, Sengupta U (2005) Trading off prediction accuracy and power consumption for context-aware wearable computing. Paper presented at the proceedings of the ninth IEEE international symposium on wearable computers
Murao K, Terada T, Takegawa Y, Nishio S (2008) A context-aware system that changes sensor combinations considering energy consumption. Paper presented at the proceedings of the 6th international conference on pervasive computing, Sydney, Australia
Hwang K-S, Cho S-B (2009) Landmark detection from mobile life log using a modular Bayesian network model. Expert Syst Appl 36(10):12065–12076. doi:10.1016/j.eswa.2009.03.002
Hong J-H, Yang S-I, Cho S-B (2010) ConaMSN: a context-aware messenger using dynamic Bayesian networks with wearable sensors. Expert Syst Appl 37(6):4680–4686. doi:10.1016/j.eswa.2009.12.040
Bao L, Intille S (2004) Activity recognition from user-annotated acceleration data. doi:http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.68.5133
Brezmes T, Gorricho J-L, Cotrina J (2009) Activity recognition from accelerometer data on a mobile phone. Paper presented at the proceedings of the 10th international work-conference on artificial neural networks: part II: distributed computing, artificial intelligence, bioinformatics, soft computing, and ambient assisted living, Salamanca, Spain
Huỳnh T, Blanke U, Schiele B (2007) Scalable recognition of daily activities with wearable sensors. Paper presented at the Proceedings of the 3rd international conference on Location-and context-awareness, Oberpfaffenhofen, Germany
Jatoba LC, Grossmann U, Kunze C, Ottenbacher J, Stork W (2008) Context-aware mobile health monitoring: evaluation of different pattern recognition methods for classification of physical activity. In: 30th annual international conference of the IEEE on Engineering in Medicine and Biology Society. EMBS 2008, 20–25 August 2008, pp 5250–5253. doi:10.1109/iembs.2008.4650398
Godfrey A, Conway R, Meagher D, ÓLaighin G (2008) Direct measurement of human movement by accelerometry. Med Eng Phys 30(10):1364–1386. doi:10.1016/j.medengphy.2008.09.005
Blum M, Pentland A, Troster G (2006) InSense: interest-based life logging. Multimedia IEEE 13(4):40–48. doi:10.1109/mmul.2006.87
Maurer U, Rowe A, Smailagic A, Siewiorek D (2006) Location and activity recognition using ewatch: a wearable sensor platform. In: Yang C, Julio A (eds) Ambient intelligence in everyday Life. Springer, Berlin, pp 86–102
Oliver N, Garg A, Horvitz E (2004) Layered representations for learning and inferring office activity from multiple sensory channels. Comput Vis Image Underst 96(2):163–180. doi:10.1016/j.cviu.2004.02.004
Harrison BL, Consolvo S, Choudhury T (2010) Using multi-modal sensing for human activity modeling in the real world handbook of ambient intelligence and smart environments. In: Nakashima H, Aghajan H, Augusto JC (eds), Springer, Berlin, pp 463–478. doi:10.1007/978-0-387-93808-0_17
Ermes M, Parkka J, Mantyjarvi J, Korhonen I (2008) Detection of daily activities and sports with wearable sensors in controlled and uncontrolled conditions. IEEE Trans Inf Technol Biomed 12(1):20–26. doi:10.1109/titb.2007.899496
Oliver N, Horvitz E (2005) S-SEER: selective perception in a multimodal office activity recognition system. Paper presented at the proceedings of the first international conference on machine learning for multimodal interaction, Martigny, Switzerland
Schmidt A, Aidoo KA, Takaluoma A, Tuomela U, Laerhoven KV, Velde WVd (1999) Advanced Interaction in Context. Paper presented at the proceedings of the 1st international symposium on handheld and ubiquitous computing, Karlsruhe, Germany
Ming L, Rozgic V, Thatte G, Sangwon L, Emken A, Annavaram M, Mitra U, Spruijt-Metz D, Narayanan S (2010) Multimodal physical activity recognition by fusing temporal and cepstral information. IEEE Trans Neural Syst Rehabil Eng 18(4):369–380. doi:10.1109/tnsre.2010.2053217
Marinov M, Zheliazkova I (2005) An interactive tool based on priority semantic networks. Know Based Syst 18(2–3):71–77. doi:10.1016/j.knosys.2004.08.001
Kofod-Petersen A, Cassens J (2006) Using activity theory to model context awareness. In: Second international workshop on modeling and retrieval of context. MRC 2005, Edinburgh, July 31–August 1, pp 1–17. doi:10.1007/11740674_1
Goldberg DE (1989) Genetic algorithms in search, optimization and machine learning, 1st edn. Addison-Wesley Longman Publishing Co., Inc., Boston
Ling CX, Sheng VS (2008) Cost-sensitive learning and the class imbalance problem. Encyclopedia of machine learning. Springer, Berlin
Thai-Nghe N, Gantner Z, Schmidt-Thieme L (2010) Cost-sensitive learning methods for imbalanced data. In: The 2010 international joint conference on neural networks (IJCNN), 18–23 July 2010, pp 1–8. doi:10.1109/ijcnn.2010.5596486
Tapia EM (2008) Using machine learning for real-time activity recognition and estimation of energy expenditure. Massachusetts Institute of Technology, Cambridge
Acknowledgments
The authors would like to thank Dr. J.-K. Min, Mr. S.-I. Yang and Dr. J.-H. Hong for their help to implement the system used in this paper. This research is supported by Ministry of Culture, Sports and Tourism (MCST) and Korea Creative Content Agency (KOCCA) in the Culture Technology (CT) Research & Development Program.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Lee, YS., Cho, SB. Recognizing multi-modal sensor signals using evolutionary learning of dynamic Bayesian networks. Pattern Anal Applic 17, 695–707 (2014). https://doi.org/10.1007/s10044-012-0300-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10044-012-0300-z