Multimedia Tools and Applications

, Volume 76, Issue 8, pp 10779–10799 | Cite as

Towards online and personalized daily activity recognition, habit modeling, and anomaly detection for the solitary elderly through unobtrusive sensing

  • Lei Meng
  • Chunyan Miao
  • Cyril Leung


Rapid population aging and advances in sensing technologies motivate the development of unobtrusive healthcare systems, designed to unobtrusively collect an elderly’s personalized information of daily living and help him actively enjoy a healthy lifestyle. Existing studies towards this goal typically focus on recognition of activities of daily living (ADLs) and abnormal behavior detection. However, the applicability of these approaches is often limited by an offline analysis strategy, complex parameter tuning, obtrusive data collection, and a need for training data. To overcome these shortcomings, this paper presents a novel framework, named the online daily habit modeling and anomaly detection (ODHMAD) model, for the real-time personalized ADL recognition, habit modeling, and anomaly detection for the solitary elderly. In contrast to most existing studies which consider activity recognition and abnormal behavior detection separately, ODHMAD links both in a system. Specifically, ODHMAD performs online recognition of the elderly’s daily activities and dynamically models the elderly’s daily habit. In this way, ODHMAD recognizes the personalized abnormal behavior of an elderly by detecting anomalies in his learnt daily habit. The developed online activity recognition (OAR) algorithm determines the occurrence of activities by modeling the activation status of sensors. It has advantages of online learning, light parameter tuning, and no training data required. Moreover, OAR is able to obtain details of the detected activities. Experimental results demonstrate the effectiveness of the proposed OAR model for online activity recognition in terms of precision, false alarm rate, and miss detection rate.


Healthcare system for the elderly Online activity recognition Personalized daily habit modeling Personalized anomaly detection 



This research is supported by the National Research Foundation Singapore under its Interactive Digital Media (IDM) Strategic Research Programme.


  1. 1.
    Avgerinakis K, Briassouli A, Kompatsiaris I (2013) Activity detection and recognition of daily living events. In: Proceedings of the ACM international workshop on Multimedia indexing and information retrieval for healthcare. ACM, pp 3–10Google Scholar
  2. 2.
    Bogomolov A, Lepri B, Ferron M, Pianesi F, Pentland AS (2014) Daily stress recognition from mobile phone data, weather conditions and individual traits. In: Proceedings of the ACM international conference on multimedia. ACM, pp 477–486Google Scholar
  3. 3.
    Caroux L, Consel C, Dupuy L, Sauzéon H (2014) Verification of daily activities of older adults: a simple, non-intrusive, low-cost approach. In: Proceedings of the international ACM SIGACCESS conference on computers & accessibility. ACM, pp 43–50Google Scholar
  4. 4.
    Chang X, Yang Y, Xing E, Yu Y (2015) Complex event detection using semantic saliency and nearly-isotonic svm. In: Proceedings of the international conference on machine learning (ICML), pp 1348–1357Google Scholar
  5. 5.
    Chavarriaga R, Sagha H, Millán JDR (2011) Ensemble creation and reconfiguration for activity recognition: An information theoretic approach. In: Proceedings of the IEEE international conference on systems, man, and cybernetics (SMC). IEEE, pp 2761–2766Google Scholar
  6. 6.
    Chen J, Kwong K, Chang D, Luk J, Bajcsy R (2006) Wearable sensors for reliable fall detection. In: Proceedings of the international conference of the engineering in medicine and biology society. IEEE, pp 3551–3554Google Scholar
  7. 7.
    Cheng J, Chen X, Shen M (2013) A framework for daily activity monitoring and fall detection based on surface electromyography and accelerometer signals. IEEE J Biomed Health Inform 17(1):38–45CrossRefGoogle Scholar
  8. 8.
    Cheng H, Liu Z, Zhao Y, Ye G, Sun X (2014) Real world activity summary for senior home monitoring. Multimed Tools Appl 70(1):177–197CrossRefGoogle Scholar
  9. 9.
    Gayathri K, Elias S, Ravindran B (2015) Hierarchical activity recognition for dementia care using markov logic network. Pers Ubiquit Comput 19(2):271–285CrossRefGoogle Scholar
  10. 10.
    Hajihashemi Z, Popescu M (2014) A new illness recognition framework using frequent temporal pattern mining. In: Proceedings of the ACM international joint conference on pervasive and ubiquitous computing: adjunct publication. ACM, pp 1241–1247Google Scholar
  11. 11.
    Hevesi P, Wille S, Pirkl G, Wehn N, Lukowicz P (2014) Monitoring household activities and user location with a cheap, unobtrusive thermal sensor array. In: Proceedings of the ACM international joint conference on pervasive and ubiquitous computing. ACM, pp 141–145Google Scholar
  12. 12.
    Ignatov AD, Strijov VV (2015) Human activity recognition using quasiperiodic time series collected from a single tri-axial accelerometer. Multimed Tools App, pp 1–14Google Scholar
  13. 13.
    Ince NF, Min CH, Tewfik AH (2007) A feature combination approach for the detection of early morning bathroom activities with wireless sensors. In: Proceedings of the ACM SIGMOBILE international workshop on Systems and networking support for healthcare and assisted living environments. ACM, pp 61–63Google Scholar
  14. 14.
    Khan SS, Karg ME, Hoey J, Kulic D (2012) Towards the detection of unusual temporal events during activities using hmms. In: Proceedings of the ACM conference on ubiquitous computing. ACM, pp 1075–1084Google Scholar
  15. 15.
    Lee MS, Lim JG, Park KR, Kwon DS (2009) Unsupervised clustering for abnormality detection based on the tri-axial accelerometer. In: ICCAS-SICE, 2009. IEEE, pp 134–137Google Scholar
  16. 16.
    Lepri B, Mana N, Cappelletti A, Pianesi F, Zancanaro M (2010) What is happening now? detection of activities of daily living from simple visual features. Pers Ubiquit Comput 14(8):749–766CrossRefGoogle Scholar
  17. 17.
    Li H, Ye C, Sample AP (2015) IDSense: a human object interaction detection system based on passive UHF RFID. In: Proceedings of the ACM conference on human factors in computing systems (CHI), pp 43–50Google Scholar
  18. 18.
    Li Q, Stankovic J, Hanson M, Barth AT, Lach J, Zhou G et al (2009) Accurate, fast fall detection using gyroscopes and accelerometer-derived posture information. In: Proceedings of the international workshop on wearable and implantable body sensor networks. IEEE, pp 138–143Google Scholar
  19. 19.
    Lotfi A, Langensiepen C, Mahmoud SM, Akhlaghinia MJ (2012) Smart homes for the elderly dementia sufferers: identification and prediction of abnormal behaviour. J Ambient Intell Humaniz Comput 3(3):205–218CrossRefGoogle Scholar
  20. 20.
    Mirmahboub B, Samavi S, Karimi N, Shirani S (2013) Automatic monocular system for human fall detection based on variations in silhouette area. IEEE Trans Biomed Eng 60(2):427–436CrossRefGoogle Scholar
  21. 21.
    Nait Aicha A, Englebienne G, Kröse B (2013) How lonely is your grandma?: detecting the visits to assisted living elderly from wireless sensor network data. In: Proceedings of the ACM conference on pervasive and ubiquitous computing: adjunct publication. ACM, pp 1285–1294Google Scholar
  22. 22.
    Nie L, Li T, Akbari M, Shen J, Chua TS (2014) Wenzher: comprehensive vertical search for healthcare domain. In: Proceedings of the international ACM SIGIR conference on research & development in information retrieval. ACM, pp 1245–1246Google Scholar
  23. 23.
    Nie L, Wang M, Zhang L, Yan S, Bo Z, Chua TS (2014) Disease inference from health-related questions via sparse deep learning. IEEE Trans Knowl Data Eng 27(8):2107–2119CrossRefGoogle Scholar
  24. 24.
    Nie L, Zhang L, Yang Y, Wang M, Hong R, Chua TS (2015) Beyond doctors: future health prediction from multimedia and multimodal observations. In: Proceedings of the ACM conference on multimedia. ACM, pp 591–600Google Scholar
  25. 25.
    Nie L, Zhao YL, Akbari M, Shen J, Chua TS (2015) Bridging the vocabulary gap between health seekers and healthcare knowledge. IEEE Trans Knowl Data Eng 27(2):396–409CrossRefGoogle Scholar
  26. 26.
    Ojetola O (2014) Detection of human falls using wearable sensors. In: PhD thesis, Coventry University, Coventry UKGoogle Scholar
  27. 27.
    Ojetola O, Gaura E, Brusey J (2015) Data set for fall events and daily activities from inertial sensors. In: Proceedings of the ACM multimedia systems conference. ACM, pp 243–248Google Scholar
  28. 28.
    Ordóñez FJ, Toledo P, Sanchis A (2015) Sensor-based bayesian detection of anomalous living patterns in a home setting. Pers Ubiquit Comput 19(2):259–270CrossRefGoogle Scholar
  29. 29.
    Park K, Lin Y, Metsis V, Le Z, Makedon F (2010) Abnormal human behavioral pattern detection in assisted living environments. In: Proceedings of the international conference on pervasive technologies related to assistive environments. ACM, No. 9Google Scholar
  30. 30.
    Piyathilaka L, Kodagoda S (2013) Gaussian mixture based HMM for human daily activity recognition using 3D skeleton features. In: Proceedings of the IEEE conference on industrial electronics and applications (ICIEA). IEEE, pp 567–572Google Scholar
  31. 31.
    Rakhecha S, Hsu K (2013) Reliable and secure body fall detection algorithm in a wireless mesh network. In: Proceedings of the international conference on body area networks, pp 420–426Google Scholar
  32. 32.
    Rashidi P, Cook DJ (2013) Com: a method for mining and monitoring human activity patterns in home-based health monitoring systems. ACM Trans Intell Syst Technol (TIST) 4(4). No. 64Google Scholar
  33. 33.
    Rashidi P, Mihailidis A (2013) A survey on ambient-assisted living tools for older adults. IEEE J Biomed Health Inform 17(3):579–590CrossRefGoogle Scholar
  34. 34.
    Sagha H, Digumarti ST, Millán JDR, Chavarriaga R, Calatroni A, Roggen D, Tröster G (2011) Benchmarking classification techniques using the opportunity human activity dataset. In: Proceedings of the IEEE international conference on systems, man, and cybernetics (SMC). IEEE, pp 36–40Google Scholar
  35. 35.
    Seeger C, Buchmann A, Van Laerhoven K (2011) Myhealthassistant: a phone-based body sensor network that captures the wearer’s exercises throughout the day. In: Proceedings of the international conference on body area networks, pp 1–7Google Scholar
  36. 36.
    Sodemann A, Ross MP, Borghetti BJ et al (2012) A review of anomaly detection in automated surveillance. IEEE Trans Syst Man Cybern Part C Appl Rev 42(6):1257–1272Google Scholar
  37. 37.
    Song X, Shao X, Zhang Q, Shibasaki R, Zhao H, Cui J, Zha H (2013) A fully online and unsupervised system for large and high-density area surveillance: tracking, semantic scene learning and abnormality detection. ACM Trans Intell Syst Technol (TIST) 4(2). No. 35Google Scholar
  38. 38.
    Sung J, Ponce C, Selman B, Saxena A (2012) Unstructured human activity detection from rgbd images. In: Proceedings of the IEEE international conference on robotics and automation (ICRA). IEEE, pp 842–849Google Scholar
  39. 39.
    Tang Y, Jin S, Yang Z, You S (2006) Detection elder abnormal activities by using omni-directional vision sensor: activity data collection and modeling. In: SICE-ICASE international joint conference. IEEE, pp 3850–3853Google Scholar
  40. 40.
    Tapia EM, Intille SS, Larson K (2004) Activity recognition in the home using simple and ubiquitous sensors. Pervasive Computing 3001:158–175CrossRefGoogle Scholar
  41. 41.
    Tong L, Song Q, Ge Y, Liu M (2013) Hmm-based human fall detection and prediction method using tri-axial accelerometer. IEEE Sensors J 13(5):1849–1856CrossRefGoogle Scholar
  42. 42.
    Uddin M, Salem A, Nam I, Nadeem T (2015) Wearable sensing framework for human activity monitoring. In: Proceedings of the workshop on wearable systems and applications. ACM, pp 21–26Google Scholar
  43. 43.
    Yan Y, Liu G, Ricci E, Sebe N (2013) Multi-task linear discriminant analysis for multi-view action recognition. In: Proceedings of the IEEE international conference on image processing (ICIP). IEEE, pp 2842–2846Google Scholar
  44. 44.
    Yan Y, Ricci E, Liu G, Sebe N (2015) Egocentric daily activity recognition via multitask clustering. IEEE Trans Image Process 24(10):2984–2995MathSciNetCrossRefGoogle Scholar
  45. 45.
    Yang Y, Ma Z, Nie F, Chang X, Hauptmann AG (2014) Multi-class active learning by uncertainty sampling with diversity maximization. Int J Comput Vis 113(2):113–127MathSciNetCrossRefGoogle Scholar
  46. 46.
    Yin J, Yang Q, Pan JJ (2008) Sensor-based abnormal human-activity detection. IEEE Trans Knowl Data Eng 20(8):1082–1090CrossRefGoogle Scholar
  47. 47.
    Yuan B, Herbert J (2014) Context-aware hybrid reasoning framework for pervasive healthcare. Pers Ubiquit Comput 18(4):865–881CrossRefGoogle Scholar
  48. 48.
    Zhang S, Ang M, Xiao W, Tham C (2008) Detection of activities for daily life surveillance: eating and drinking. In: Proceedings of the international conference on e-health networking, applications and services (HealthCom). IEEE, pp 171–176Google Scholar
  49. 49.
    Zheng YL, Ding XR, Poon CCY, Lo BPL, Zhang H, Zhou XL, Yang GZ, Zhao N, Zhang YT (2014) Unobtrusive sensing and wearable devices for health informatics. IEEE Trans Biomed Eng 61(5):1538–1554CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.Joint NTU-UBC Research Centre of Excellence in Active Living for the Elderly (LILY)Nanyang Technological UniversitySingaporeSingapore
  2. 2.School of Computer EngineeringNanyang Technological UniversitySingaporeSingapore
  3. 3.Department of Electrical and Computer EngineeringUniversity of British ColumbiaVancouverCanada

Personalised recommendations