Abstract
Activity recognition algorithms have matured and become more ubiquitous in recent years. However, these algorithms are typically customized for a particular sensor platform. In this paper, we introduce PECO, a Personalized activity ECOsystem, that transfers learned activity information seamlessly between sensor platforms in real time so that any available sensor can continue to track activities without requiring its own extensive labeled training data. We introduce a multi-view transfer learning algorithm that facilitates this information handoff between sensor platforms and provide theoretical performance bounds for the algorithm. In addition, we empirically evaluate PECO using datasets that utilize heterogeneous sensor platforms to perform activity recognition. These results indicate that not only can activity recognition algorithms transfer important information to new sensor platforms, but any number of platforms can work together as colleagues to boost performance.
Similar content being viewed by others
References
Solanas A, Patsakis C, Conti M, Vlachos I, Ramos V, Falcone F, Postolache O, Perez-Martinez PA, Di Pietro R, Perrea DN, Martinez-Balleste A (2014) Smart health: a context-aware health paradigm within smart cities. IEEE Commun Mag 52(8):74–81
Aggarwal JK, Ryoo MS (2011) Human activity analysis: a review. ACM Comput Surv 43(3):1–47
Chen L, Hoey J, Nugent CD, Cook DJ, Yu Z (2012) Sensor-based activity recognition. IEEE Trans Syst Man Cybern Part C Appl Rev 42(6):790–808
Ke S-R, Thuc HLU, Lee Y-J, Hwang J-N, Yoo J-H, Choi K-H (2013) A review on video-based human activity recognition. Computers 2(2):88–131
Bulling A, Blanke U, Schiele B (2014) A tutorial on human activity recognition using body-worn inertial sensors. ACM Comput Surv 46(3):107–140
Reiss A, Stricker D, Hendeby G (2013) Towards robust activity recognition for everyday life: methods and evaluation. In: Pervasive Computing Technologies for Healthcare, pp. 25–32
Vishwakarma S, Agrawal A (2013) A survey on activity recognition and behavior understanding in video surveillance. Vis Comput 29(10):983–1009
Cook D (2012) Learning setting-generalized activity models for smart spaces. IEEE Intell Syst 27(1):32–38
Hagras H, Doctor F, Lopez A, Callaghan V (2007) An incremental adaptive life long learning approach for type-2 fuzzy embedded agents in ambient intelligent environments. IEEE Trans Fuzzy Syst 15(1):41–55
Munguia-Tapia E, Intille SS, Larson K (2004) Activity recognition in the home using simple and ubiquitous sensors. In: Pervasive, pp. 158–175
van Kasteren T, Noulas A, Englebienne G, Krose B (2008) Accurate activity recognition in a home setting. In: ACM conference on ubiquitous computing, pp. 1–9
He W, Guo Y, Gao C, Li X (2012) Recognition of human activities with wearable sensors. J Adv Signal Process 108:1–13
Junker H, Amft O, Lukowicz P, Groster G (2008) Gesture spotting with body-worn inertial sensors to detect user activities. Pattern Recognit 41:2010–2024
Maurer U, Smailagic A, Siewiorek D, Deisher M (2006) Activity recognition and monitoring using multiple sensors on different body positions. In: Proceedings of the international workshop on wearable and implantable body sensor networks, pp. 113–116
Bulling A, Ward JA, Gellersen H (2012) Multimodal recognition of reading activity in transit using body-worn sensors. ACM Trans Appl Percept 9(1):2:1–2:21
Lara O, Labrador MA (2013) A survey on human activity recognition using wearable sensors. IEEE Commun Surv Tutor. 15(3):1192–1209
Gu T, Chen S, Tao X, Lu J (2010) An unsupervised approach to activity recognition and segmentation based on object-use fingerprints. Data Knowl Eng 69(6):533–544
Philipose M, Fishkin KP, Perkowitz M, Patterson DJ, Hahnel D, Fox D, Kautz H (2004) Inferring activities from interactions with objects. IEEE Pervasive Comput 3(4):50–57
Buettner M, Prasad R, Philipose M, Wetherall D (2009) Recognizing daily activities with RFID-based sensors. In: International conference on ubiquitous computing, pp. 51–60
Kwapisz J, Weiss G, Moore S (2010) Activity recognition using cell phone accelerometers. In: International workshop on knowledge discovery from sensor data, pp. 10–18
Zhixian Y, Vigneshwaran S, Chakraborty D, Misra A, Aberer K (2012) Energy-efficient continuous activity recognition on mobile phones: an activity-adaptive approach. In: International symposium on wearable computers, pp. 17–24
Moncrieff S, Venkatesh S, West G, Greenhill S (2007) Multi-modal emotive computing in a smart house environment. Pervasive Mob Comput 3(2):79–94
Candamo J, Shreve M, Goldgof D, Sapper D, Kasturi R (2010) Understanding transit scenes: a survey on human behavior recognition algorithms. IEEE Trans Intell Transp Syst 11(1):206–224
Gill T, Keller JM, Anderson DT, Luke RH (2011) A system for change detection and human recognition in voxel space using the Microsoft Kinect sensor. In: IEEE applied imagery pattern recognition workshop, pp. 1–8
Malgireddy MR, Nwogu I, Govindaraju V (2013) Language-motivated approaches to action recognition. J Mach Learn Res 14:2189–2212
Wang J, Liu Z, Wu Y (2013) Learning actionlet ensemble for 3D human action recognition. IEEE Trans Pattern Anal Mach Intell 36(5):914–927
Bao L, Intille S (2004) Activity recognition from user annotated acceleration data. In: Pervasive, pp. 1–17
Ravi N, Dandekar N, Mysore P, Littman ML (2005) Activity recognition from accelerometer data. In: Innovative applications of artificial intelligence, pp. 1541–1546
Ward JA, Lukowicz P, Troster G, Starner TE (2006) Activity recognition of assembly tasks using body-worn microphones and accelerometers. IEEE Trans Pattern Anal Mach Intell 28(10):1553–1567
Singla G, Cook DJ, Schmitter-Edgecombe M (2010) Recognizing independent and joint activities among multiple residents in smart environments. Ambient Intell Humaniz Comput J 1(1):57–63
Lester J, Choudhury T, Kern N, Borriello G, Hannaford B (2005) A hybrid discriminative/generative approach for modeling human activities. In: International joint conference on artificial intelligence, pp. 766–772
Amft O, Troster G (2009) On-body sensing solutions for automatic dietary monitoring. IEEE Pervasive Comput 8:62–70
Blanke U, Schiele B, Kreil M, Lukowicz P, Sick B, Gruber T (2010) All for one or one for all? Combining heterogeneous features for activity spotting. In: IEEE international conference on pervasive computing and communications workshops, pp. 18–24
Wang S, Pentney W, Popescu AM, Choudhury T, Philipose M (2007) Common sense based joint training of human activity recognizers. In: International joint conference on artificial intelligence, pp. 2237–2242
Lester J, Choudhury T, Borriello G (2006) A practical approach to recognizing physical activities. In: International conference on pervasive computing, pp. 1–16
Krishnan N, Cook DJ (2014) Activity recognition on streaming sensor data. Pervasive Mob Comput 10:138–154
Arnold A, Nallapati R, Cohen WW (2007) A comparative study of methods for transductive transfer learning. In: International conference on data mining
Elkan C (2011) The foundations of cost-sensitive learning. In: International joint conference on artificial intelligence, pp. 973–978
Thrun S (1996) Explanation-based neural network learning: a lifelong learning approach. Kluwer Academic Publishers, Bostom, MA
Thrun S, Pratt L (1998) Learning to learn. Kluwer Academic Publishers, Bostom, MA
Vilalta R, Drissi Y (2002) A prospective view and survey of meta-learning. Artif Intell Rev 18:77–95
Raina R, Battle A, Lee H, Packer B, Ng AY (2007) Self-taught learning: transfer learning from unlabeled data. In: International conference on machine learning, pp. 759–766
Hachiya H, Sugiyama M, Ueda N (2012) Importance-weighted least-squares probabilistic classifier for covariate shift adaptation with application to human activity recognition. Neurocomputing 80:93–101
Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359
Duan L, Xu D, Tsang I, Luo J (2012) Visual event recognition in videos by learning from web data. IEEE Trans Pattern Anal Mach Intell 34(9):1667–1680
Wei B, Pal C (2011) Heterogeneous transfer learning with RBMs. In: AAAI conference on artificial intelligence, pp. 531–536
Yang W, Wang Y, Mori G (2011) Learning transferable distance functions for human action recognition. In: Machine learning for vision-based motion analysis, pp. 349–370
Feuz K, Cook DJ (2015) Transfer learning across feature-rich heterogeneous feature spaces via feature-space remapping. ACM Trans Intell Syst Technol 6(1):3
Zhao Z, Chen Y, Liu J, Shen Z, Liu M (2011) Cross-people mobile-phone based activity recognition. In: International joint conference on artificial intelligence, pp. 2545–2550
Cook DJ, Feuz K, Krishnan N (2013) Transfer learning for activity recognition: a survey. Knowl Inf Syst 36(3):537–556
Ren Y, Wu Y, Ge Y (2014) A co-training algorithm for EEG classification with biomimetic pattern recognition and sparse representation. Neurocomputing 2(137):212–222
Kurz M, Holzl G, Ferscha A, Calatroni A, Roggen D, Troster G (2011) Real-time transfer and evaluation of activity recognition capabilities in an opportunistic system. In: International conference on adaptive and self-adaptive systems and applications, pp. 73–78
Hu DH, Yang Q (2011) Transfer learning for activity recognition via sensor mapping. In: International joint conference on artificial intelligence, pp. 1962–1967
Daume H, Marcu D (2006) Domain adaptation for statistical classifiers. J Artif Intell Res 26(1):101–126
Daume H, Kumar A, Saha A (2010) Co-regularization based semi-supervised domain adaptation. In: Advances in neural information processing systems, pp. 478–486
Blitzer J, Dredze M, Pereira F (2007) Biographies, bollywood, boom-boxes and blenders: domain adaptation for sentiment classification. In: Annual meeting of the association for computational linguistics
Zhong E, Fan W, Peng J, Zhang K, Ren J, Turaga D, Verscheure O (2009) Cross domain distribution adaptation via kernel mapping. In: ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1027–1036
Prettenhofer P, Stein B (2011) Cross-lingual adaptation using structural correspondence learning. ACM Trans Intell Syst Technol 3(1):13
Zhou JT, Pan SJ, Tsang IW, Yan Y (2014) Hybrid heterogeneous transfer learning through deep learning. In: AAAI conference on artificial intelligence, pp. 2213–2219
Shi X, Yu P (2012) Dimensionality reduction on heterogeneous feature space. In: International conference on data mining, pp. 635–644
Yang Q, Chen Y, Xue G-R, Dai W, Yu Y (2009) Heterogeneous transfer learning for image clustering via the social web. In: Joint conference of the annual meeting of the ACL, pp. 1–9
Kira Z (2010) Inter-robot transfer learning for perceptual classification. In: International conference on autonomous agents and multiagent systems, pp. 13–20
Blum A, Mitchell T (1998) Combining labeled and unlabeled data with co-training. In: Annual conference on computational learning theory, pp. 92–100
Sun S (2013) A survey of multi-view machine learning. Neural Comput Appl 23(7–8):2031–2038
Wang C, Mahadevan S (2008) Manifold alignment using procrustes analysis. In: International conference on machine learning, pp. 1120–1127
Pansiot J, Stoyanov D, McIlwraith D, Lo BPL, Yang GZ (2007) Ambient and wearable sensor fusion for activity recognition in healthcare monitoring systems. In: International workshop on wearable and implantable body sensor networks, pp. 208–212
Ugolotti R, Sassi F, Mordonini M, Cagnoni S (2013) Multi-sensor system for detection and classification of human activities. J Ambient Intell Humaniz Comput 1(4):27–41
Banos O, Damas M, Pomares H, Rojas I (2013) Activity recognition based on a multi-sensor meta-classifier. In: Advances in computational intelligence: international work conference on artificial neural networks, pp. 208–215
Nigam K, Ghani R (2000) Analyzing the effectiveness and applicability of co-training. In: International conference on information and knowledge management, pp. 86–93
Kakade SM, Foster DP (2007) Multi-view regression via canonical correlation analysis. In: Learning theory, Springer, pp. 82–96
Wall ME, Rechtsteiner A, Rocha LM (2003) Singular value decomposition and principal component analysis. In: A practical approach to microarray data analysis, pp. 91–109
Valiant LG (1984) A theory of the learnable. CACM 27(11):1134–1142
Cook DJ, Crandall A, Thomas B, Krishnan N (2012) CASAS: a smart home in a box. IEEE Comput 46(7):62–69
Pedregosa F, Varoquaux G, Framfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825–2830
Roggen D, Frster K, Calatroni A, Trster G (2013) The adARC pattern analysis architecture for adaptive human activity recognition systems. J Ambient Intell Humaniz Comput 4:169–186
Sagha H, Digumarti ST, del R. Millan J, Chavarriaga R, Calatroni A, Roggen D, Troster G (2011) Benchmarking classification techniques using the opportunity human activity dataset. In: IEEE international conference on systems, man, and cybernetics, pp. 36–40
Sahaf Y, Krishnan N, Cook DJ (2011) Defining the complexity of an activity. In: Workshop AAAI on techniques and languages, activity context representation
Acknowledgements
The authors would like to thank Aaron Crandall and Biswaranjan Das for their help in collecting and processing Kinect data. Funding for research was provided by the National Science Foundation (DGE-0900781, IIS-1064628) and by the National Institutes of Health (R01EB015853).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Feuz, K.D., Cook, D.J. Collegial activity learning between heterogeneous sensors. Knowl Inf Syst 53, 337–364 (2017). https://doi.org/10.1007/s10115-017-1043-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10115-017-1043-3