Multimedia Tools and Applications

, Volume 70, Issue 1, pp 177–197 | Cite as

Real world activity summary for senior home monitoring

  • Hong Cheng
  • Zicheng Liu
  • Yang Zhao
  • Guo Ye
  • Xinghai Sun
Article
First Online:

Abstract

It is a common knowledge that the daily activities of senior people tell a lot about their health condition. Thus, we believe that analysing their activities at home will improve the health care. Toward this goal, we propose a senior home activity summary system. There are two challenging problems in such a real world application. First, the amount of data for different activity categories is extremely unbalanced, which severely degrades the classifying performance. Second, senior’s activities are usually accompanied by nurse’s walking. It is impractical to predefine and label all the possible activities of all the potential visitors. Consequently, we propose a technique called subspace Naive-Bayesian Mutual Information Maximization (sNBMIM). It divides the feature space into a number of subspaces and allows the kernel and normalization parameters to vary between different subspaces. Moreover, we propose a novel feature filtering technique to reduce or eliminate the effects of the interest points that belong to other people. To evaluate the proposed activity summary system, we have collected a Senior home Activity Recognition dataset (UESTC-SAR), and performed activity recognition for eight categories. The experimental results show that the proposed system provides quite accurate activity summaries for a real world application scenario.

Keywords

Activity recognition Activity summary Senior home monitoring Health care Feature filtering Temporal smoothing 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgements

This research was partially supported by the grant from NSFC (No. 61075045), the Program for New Century Excellent Talents in University (NECT-10-0292), the National Basic Research Program of China (No. 2011CB707000), and the Fundamental Research Funds for the Central Universities. We also thank the Jinrui Honghe Garden for the seniors and anonymous reviewers for their valuable suggestions.

References

  1. 1.
    Amini S, Narasimhan P (2009) TwitterJacket: an automated activity and health monitoring solution for the elderly. CMU-CyLab-10-003Google Scholar
  2. 2.
    Atrey PK, Kankanhalli MS (2006) Information assimilation framework for event detection multimedia surveillance systems. Multimedia Syst J 12(3):239–253CrossRefGoogle Scholar
  3. 3.
    Bao L, Intille SS (2004) Activity recognition from user-annotated acceleration data. In: Pervasive computing, pp 1–17Google Scholar
  4. 4.
    Behmo R, Marcombes P, Dalalyan A, Prinet V (2010) Towards optimal naive Bayes nearest neighbor. In: ECCV, pp 171–184Google Scholar
  5. 5.
    Bishop CM (2006) Pattern recognition and machine learning. Springer, New YorkMATHGoogle Scholar
  6. 6.
    Blank M, Gorelick L, Shechtman E, Irani M, Basri R (2005) Actions as space-times shapes. In: IEEE ICCVGoogle Scholar
  7. 7.
    Boiman O, Shechtman E, Irani M (2008) In defense of nearest-neighbor based image classification. In: IEEE CVPRGoogle Scholar
  8. 8.
    Bulling A, Ward JA, Gellersen H, Tröster G (2010) Eye movement analysis for activity recognition using electrooculography. IEEE TPAMI 33(4):741–753CrossRefGoogle Scholar
  9. 9.
    Bussmann JBJ, Martens WLJ, Tulen JHM, Schasfoort FC, van den Berg-Emons HJG, Stam HJ (2001) Measuring daily behavior using ambulatory accelerometry: the activity monitor. Behav Res Methods (Springer) 33(3):349–356CrossRefGoogle Scholar
  10. 10.
    Cao L, Liu Z, Huang TS (2010) Cross-dataset action detection. In: IEEE CVPRGoogle Scholar
  11. 11.
    Cheng H, Liu Z, Zhao Y, Ye G (2011) Real world activity summary for senior home monitroing. In: IEEE ICMEGoogle Scholar
  12. 12.
    Choudhury T, Consolvo S, Harrison B, Hightower J, LaMarca A, LeGrand L, Rahimi A, Rea A, Borriello G, Hemingway B, Klasnja PP, Lester J, Wyatt D (2008) The mobile sensing platform: an embedded activity recognition system. IEEE Pervasive Computing 7(2):32–41CrossRefGoogle Scholar
  13. 13.
    Consolvo S, McDonald DW, Toscos T, Chen MY, Froehlich J, Harrison B, Klasnja P, LaMarca A, LeGrand L, Libby R, Smith I, Landay JA (2008) Activity sensing in the wild: a field trial of ubifit garden. In: Proceeding of the twenty-sixth annual SIGCHI conference on human factors in computing systems, pp 1797–1806Google Scholar
  14. 14.
    Duan L, Xu D, Tsang IW, Luo J (2010) Visual event recognition in videos by learning from web data. In: IEEE CVPRGoogle Scholar
  15. 15.
    Duong TV, Bui HH, Phung DQ, Venkatesh S (2005) Activity recognition and abnormality detection with the switching hidden semi-markov model. In: IEEE CVPRGoogle Scholar
  16. 16.
    Gu T, Chen S, Tao X, Lu J (2010) An unsupervised approach to activity recognition and segmentation based on object-use fingerprints. Data Knowl Eng 69(6):533–544CrossRefGoogle Scholar
  17. 17.
    Hu D, Pan S, Zheng V, Liu N (2008) Real world activity recognition with multiple goals. In: ACM UbiCompGoogle Scholar
  18. 18.
    Hu Y, Cao L, Lv F, Yan S, Gong Y, Huang TS (2009) Action detection in complex scenes with spatial and temporal ambiguities. In: IEEE ICCVGoogle Scholar
  19. 19.
    Ke Y, Sukthankar R, Hebert M (2007) Event detection in crowded videos. In: IEEE ICCVGoogle Scholar
  20. 20.
    Kim E, Helal S, Cook D (2010) Human activity recognition and pattern discovery. Pervasive Computing 9(1):48–53CrossRefGoogle Scholar
  21. 21.
    Kovashka A, Grauman K (2010) Learning a hierarchy of discriminative space-time neighborhood features for human action recognition. In: IEEE CVPRGoogle Scholar
  22. 22.
    Krafft M, Coskun K (2009) Design aspects for elderly using a health smart home. Bachelor of Software Engineering Management Thesis (No. 2009:028), University of GothenburgGoogle Scholar
  23. 23.
    Lai C, Huang Y, Chao H (2010) Adaptive body posture analysis using collaborative mulit-sensors for elderly falling detection. IEEE Intell Syst 99:1172–1177Google Scholar
  24. 24.
    Laptev I (2005) On space-time interest points. IJCV 64:107–123CrossRefGoogle Scholar
  25. 25.
    Laptev I, Marszalek M, Schmid C, Rozenfeld B (2008) Learning realistic human actions from movies. In: IEEE CVPRGoogle Scholar
  26. 26.
    Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: IEEE CVPRGoogle Scholar
  27. 27.
    Lee SW, Mase K (2002) Activity and location recognition using wearable sensors. Pervasive Computing 1(3):24–32CrossRefGoogle Scholar
  28. 28.
    Liao L (20036) Location-based activity recognition. Ph.D. thesis, University of WashingtonGoogle Scholar
  29. 29.
    Litvak D, Zigel Y, Gannot I (2008) Fall detection of elderly through floor vibrations and sound. In: International conference of Engineering in Medicine and Biology SocietyGoogle Scholar
  30. 30.
    Liu J, Luo J, Shah M (2009) Recognizing realistic actions from videos “in the wild”. In: IEEE CVPRGoogle Scholar
  31. 31.
    Matsumoto N, Ueda H, Yamazaki T, Murai H (2009) Life with a robot companion: video analysis of 16-days of interaction with a home robot in a ubiquitous home environment. In: Human–computer interaction. Novel interaction methods and techniques. Springer, Berlin, pp 341–350CrossRefGoogle Scholar
  32. 32.
    Messing R, Pal C, Kautz H (2009) Activity recognition using the velocity histories of tracked keypoints. In: IEEE ICCVGoogle Scholar
  33. 33.
    Miluzzo E, Lane ND, Fodor K, Peterson R, Lu H, Musolesi M, Eisenman SB, Zheng X, Campbell AT (2008) Sensing meets mobile social networks: the design, implementation and evaluation of the CenceMe application. In: Proceedings of the 6th ACM conference on embedded network sensor systems, pp 337–350Google Scholar
  34. 34.
    Nambu M, Nakajima K, Noshiro M, Tamura T (2005) An algorithm for the automatic detection of health conditions. IEEE Eng Med Biol Mag 24:38–42CrossRefGoogle Scholar
  35. 35.
    Ni B, Wang G, Moulin P (2011) RGBD-HuDaAct: a color-depth video database for human daily activity recognition. In: IEEE ICCV workshopGoogle Scholar
  36. 36.
    Niebles JC, Wang H, Li F-F (2008) Unsupervised learning of human action categories using spatial-temporal words. IJCV 79(3):299–318CrossRefGoogle Scholar
  37. 37.
    Palmes P, Pung HK, Gu T, Xue W, Chen S (2010) Object relevance weight pattern mining for activity recognition and segmentation. Pervasive and Mobile Computing 6(1):43–57CrossRefGoogle Scholar
  38. 38.
    Pirsiavash H, Ramanan D (2012) Detecting activities of daily living in first-person camera views. In: IEEE CVPRGoogle Scholar
  39. 39.
    Prabhakar S, Pankanti S, Jan AK (2003) Biometric recognition security and privacy concerns. In: IEEE safety & privacyGoogle Scholar
  40. 40.
    Ramasso E, Rombaut M, Pellerin D (2006) A temporal belief filter improving human action recognition in videos. In: IEEE ICASSPGoogle Scholar
  41. 41.
    Ravi N, Dandekar N, Mysore P, Littman ML (2005) Activity recognition from accelerometer data. In: Proceedings of the national conference on artificial intelligenceGoogle Scholar
  42. 42.
    Rialle V, Lamy JB, Noury N, Bajolle L (2003) Telemonitoring of patients at home: a software agent approach. Comput Methods Programs Biomed 72:257–268CrossRefGoogle Scholar
  43. 43.
    Rohrbach M, Amin S, Andriluka M, Schiele B (2012) A database for fine grained activity detection of cooking activities. In: IEEE CVPRGoogle Scholar
  44. 44.
    Schuldt C, Laptev I, Caputo B (2004) Recognizing human actions: a local SVM approach. In: IEEE ICPRGoogle Scholar
  45. 45.
    Tapia EM, Intille SS, Larson K (2004) Activity recognition in the home using simple and ubiquitous sensors. In: Pervasive computing, pp 158–175Google Scholar
  46. 46.
    Turaga P, Chellappa R, Subrahmanian VS, Udrea O (2008) Machine recognition of human activities: a survey. IEEE TCSVT 18(11):1473–1488Google Scholar
  47. 47.
    United Nations Population Division (2002) Department of Economic and Social Affairs, Magnitude and speed of population ageing. In: World population ageing: 1950–2050Google Scholar
  48. 48.
    Wang P, Abowd GD, Rehg JM (2009) Quasi-periodic event analysis for social game retrieval. In: IEEE ICCVGoogle Scholar
  49. 49.
    Ward JA, Lukowicz P, Troster G, Starner TE (2006) Activity recognition of assembly tasks using body-worn microphones and accelerometers. IEEE TPAMI 28(10):1553–1567CrossRefGoogle Scholar
  50. 50.
    Yuan J, Liu Z (2010) TechWare: video-based human action detection resources [best of the Web]. Signal Process Mag 27(5):136–139CrossRefGoogle Scholar
  51. 51.
    Yuan J, Liu Z, Wu Y (2009) Discriminative subvolume search for efficient action detection. In: IEEE CVPRGoogle Scholar
  52. 52.
    Yuan J, Liu Z, Wu Y, Zhang Z (2009) Speeding up spatio-temporal sliding-window search for efficient event detection in crowded videos. In: Proceedings of the 1st ACM international workshop on events in multimediaGoogle Scholar
  53. 53.
    Zigel Y, Litvak D, Gannot I (2009) A method for automatic fall detection of ederly people using floor vibrations and sound-proof of concept on human mimicking doll falls. IEEE Trans Biomed Eng 56(12):2858–2867CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  • Hong Cheng
    • 1
  • Zicheng Liu
    • 2
  • Yang Zhao
    • 1
  • Guo Ye
    • 1
  • Xinghai Sun
    • 1
  1. 1.University of Electronic Science and TechnologyChengduChina
  2. 2.Microsoft Research Redmond, One Microsoft WayRedmondUSA

Personalised recommendations