Simultaneous Tracking and Activity Recognition (STAR) Using Many Anonymous, Binary Sensors

  • D. H. Wilson
  • C. Atkeson
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3468)


In this paper we introduce the simultaneous tracking and activity recognition (STAR) problem, which exploits the synergy between location and activity to provide the information necessary for automatic health monitoring. Automatic health monitoring can potentially help the elderly population live safely and independently in their own homes by providing key information to caregivers. Our goal is to perform accurate tracking and activity recognition for multiple people in a home environment. We use a “bottom-up” approach that primarily uses information gathered by many minimally invasive sensors commonly found in home security systems. We describe a Rao-Blackwellised particle filter for room-level tracking, rudimentary activity recognition (i.e., whether or not an occupant is moving), and data association. We evaluate our approach with experiments in a simulated environment and in a real instrumented home.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Living Independently - QuietCare system TM (2005),
  2. 2.
    Abowd, G., Atkeson, C., Bobick, A., Essa, I., Mynatt, B., Starner, T.: Living laboratories: The Future Computing Environments group at the Georgia Institute of Technology. In: Proceedings of the 2000 Conference on Human Factors in Computing Systems, CHI (2000)Google Scholar
  3. 3.
    Addlesee, M., Curwen, R., Hodges, S., Newman, J., Steggles, P., Ward, A., Hopper, A.: Implementing a sentient computing system. IEEE Computer 34(8), 50–56 (2001)Google Scholar
  4. 4.
    Allin, S., Barucha, A., Zimmerman, J., Wilson, D., Roberson, M., Stevens, S., Watclar, H., Atkeson, C.: Toward the automatic assessment of behavioral disturbances of dementia. In: UbiHealth 2003 (2003)Google Scholar
  5. 5.
    Anderson, R.N.: Method for constructing complete annual US life tables, national center for health statistics. Vital and Health Stat. 2, 129 (1999)Google Scholar
  6. 6.
    Barger, T., Brown, D., Alwan, M.: Health status monitoring through analysis of behavioral patterns. In: AI*IA 2003: Workshop on Ambient Intelligence (2003)Google Scholar
  7. 7.
    Beckman, C., Consolvo, S.: Sensor configuration tool for end-users: Low-fidelity prototype evaluation # 1. Technical Report IRS-TR-03-009, Intel Research Seattle (July 2003)Google Scholar
  8. 8.
    Bennewitz, M., Burgard, W., Thrun, S.: Learning motion patterns of persons for mobile service robots. In: Proceedings of ICRA (2002)Google Scholar
  9. 9.
    Bilmes, J.: A gentle tutorial on the EM algorithm and its application to parameter estimation for gaussian mixture and hidden Markov models. Technical Report ICSI-TR-97-021. University of Berkeley (1997)Google Scholar
  10. 10.
    Burgio, L., Scilley, K., Hardin, J.M., Hsu, C.: Temporal patterns of disruptive vocalization in elderly nursing home residents. Int. Journal of Geriatric Psychiatry 16(1), 378–386 (2001)CrossRefGoogle Scholar
  11. 11.
    Burgio, L., Scilley, K., Hardin, J.M., Janosky, J., Bonino, P., Slater, S., Engberg, R.: Studying disruptive vocalization and contextual factors in the nursing home using computer-assisted real-time observation. Journal of Gerontology 49(5), 230–239 (1994)Google Scholar
  12. 12.
    Clarkson, B., Sawhney, N., Pentland, A.: Auditory context awareness via wearable computing. In: Proceedings of the Perceptual User Interfaces Workshop (1998)Google Scholar
  13. 13.
    Davis, L., Buckwalter, K., Burgio, L.: Measuring problem behaviors in dementia: Developing a methodological agenda. Advances in Nursing Science 20(1), 40–55 (1997)Google Scholar
  14. 14.
    Doucet, A., de Freitas, N., Gordon, N.: Sequential Monte Carlo Methods in Practice. Springer, New York (2001)MATHGoogle Scholar
  15. 15.
    Fox, D., Hightower, J., Liao, L., Schulz, D., Borriello, G.: Bayesian filtering for location estimation. IEEE Pervasive Computing 2(3), 24–33 (2003)CrossRefGoogle Scholar
  16. 16.
    Helal, S., Winkler, B., Lee, C., Kaddoura, Y., Ran, L., Giraldo, C., Kuchibhotla, S., Mann, W.: Enabling location-aware pervasive computing applications for the elderly. In: First IEEE International Conference on Pervasive Computing and Communications, p. 531 (2003)Google Scholar
  17. 17.
    Intille, S., Larson, K.: Designing and evaluating technology for independent aging in the home. In: Proceedings of the Int. Conference on Aging, Disability and Independence (2003)Google Scholar
  18. 18.
    Kanade, T., Collins, R., Lipton, A., Burt, P., Wixson, L.: Advances in cooperative multisensor video surveillance. In: Proceedings of DARPA Image Understanding Workshop, vol. 1, pp. 3–24 (1998)Google Scholar
  19. 19.
    Khan, Z., Balch, T., Dellaert, F.: Efficient particle filter-based tracking of multiple interacting targets using an mrf-based motion model. In: Proceedings of ICRA (2003)Google Scholar
  20. 20.
    Levine, R.A., Casella, G.: Implementations of the Monte Carlo EM algorithm. Journal of Computational and Graphical Statistics (2000)Google Scholar
  21. 21.
    Ogawa, M., Togawa, T.: The concept of the home health monitoring. In: Proceedings of Enterprise Networking and Computing in Healthcare Industry. Healthcom 2003, pp. 71–73 (2003)Google Scholar
  22. 22.
    Oliver, N., Horvitz, E., Garg, A.: Layered representations for human activity recognition. In: Fourth IEEE International Conference on Multimodal Interfaces, pp. 3–8 (2002)Google Scholar
  23. 23.
    Patterson, D., Liao, L., Fox, D., Kautz, H.: Inferring high-level behavior from low-level sensors. In: Dey, A.K., Schmidt, A., McCarthy, J.F. (eds.) UbiComp 2003. LNCS, vol. 2864, pp. 73–89. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  24. 24.
    Philipose, M., Fishkin, K., Perkowitz, M., Patterson, D., Kautz, H., Hahnel, D.: Inferring activities from interactions with objects. IEEE Pervasive Computing Magazine: Mobile & Ubiquitous Systems 3(4), 50–57 (2004)Google Scholar
  25. 25.
    Rabiner, L.: A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE 77(2), 257–286 (1989)CrossRefGoogle Scholar
  26. 26.
    Sidenbladh, H., Black, M.: Learning image statistics for bayesian tracking. In: IEEE International Conference on Computer Vision (ICCV), vol. 2, pp. 709–716 (2001)Google Scholar
  27. 27.
    Tapia, E.M.: Activity recognition in the home setting using simple and ubiquitous sensors. Master’s thesis, Massachusetts Institute of Technology (September 2003)Google Scholar
  28. 28.
    Tapia, E.M., Intille, S.S., Larson, K.: Activity recognition in the home setting using simple and ubiquitous sensors. In: Ferscha, A., Mattern, F. (eds.) PERVASIVE 2004. LNCS, vol. 3001, pp. 158–175. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  29. 29.
    Thrun, S., Langford, J.: Monte Carlo hidden markov models. Technical Report TR CMUCS- 98-179, Carnegie Mellon University (1997)Google Scholar
  30. 30.
    Van Haitsma, K., Lawton, P., Kleban, M., Klapper, J., Corn, J.: Methodological aspects of the study of streams of behavior in elders with dementing illness. Alzheimer Disease and Associated Disorders 11(4), 228–238 (1997)Google Scholar
  31. 31.
    Wei, G., Tanner, M.: A Monte Carlo implementation of the EM algorithm and the poor man’s data augmentation algorithms. Journal of the American Statistical Association 85(411), 699–704 (1990)CrossRefGoogle Scholar
  32. 32.
    Wilson, D.H., Long, A.C.: A context aware recognition survey for data collection using ubiquitous sensors in the home. In: Proceedings of CHI 2005 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • D. H. Wilson
    • 1
  • C. Atkeson
    • 1
  1. 1.Robotics InstituteCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations