Hierarchical Activity Recognition Using Automatically Clustered Actions

  • Tim L. M. van Kasteren
  • Gwenn Englebienne
  • Ben J. A. Kröse
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7040)


The automatic recognition of human activities such as cooking, showering and sleeping allows many potential applications in the area of ambient intelligence. In this paper we show that using a hierarchical structure to model the activities from sensor data can be very beneficial for the recognition performance of the model. We present a two-layer hierarchical model in which activities consist of a sequence of actions. During training, sensor data is automatically clustered into clusters of actions that best fit to the data, so that sensor data only has to be labeled with activities, not actions. Our proposed model is evaluated on three real world datasets and compared to two non-hierarchical temporal probabilistic models. The hierarchical model outperforms the non-hierarchical models in all datasets and does so significantly in two of the three datasets.


Hierarchical Models Activity Recognition Sensor Networks 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bilmes, J.: A gentle tutorial of the EM algorithm and its application to parameter estimation for Gaussian mixture and hidden Markov models. Technical Report TR-97-021, International Computer Science Institute (1997)Google Scholar
  2. 2.
    Duong, T.V., Bui, H.H., Phung, D.Q., Venkatesh, S.: Activity recognition and abnormality detection with the switching hidden semi-markov model. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, CVPR, pp. 838–845. IEEE Computer Society, Washington, DC, USA (2005)Google Scholar
  3. 3.
    van Kasteren, T.L.M., Englebienne, G., Kröse, B.J.A.: Activity Recognition in Pervasive Intelligent Environments. In: Human Activity Recognition from Wireless Sensor Network Data: Benchmark and Software, ch. 8, pp. 165–186. Atlantis Press (2011)Google Scholar
  4. 4.
    van Kasteren, T.L.M., Noulas, A., Englebienne, G., Kröse, B.J.A.: Accurate activity recognition in a home setting. In: Proceedings of the 10th International Conference on Ubiquitous Computing, Ubicomp, pp. 1–9. ACM, New York (2008)CrossRefGoogle Scholar
  5. 5.
    Lühr, S., Bui, H.H., Venkatesh, S., West, G.A.W.: Recognition of human activity through hierarchical stochastic learning. In: Proceedings of the 1st International Conference on Pervasive Computing and Communications, PerCom, pp. 416–422. IEEE Computer Society, Washington, DC, USA (2003)Google Scholar
  6. 6.
    Muncaster, J., Ma, Y.: Hierarchical model-based activity recognition with automatic low-level state discovery. Journal of Multimedia 2(5), 66 (2007)CrossRefGoogle Scholar
  7. 7.
    Murphy, K., Paskin, M.: Linear time inference in hierarchical hmms. In: Advances in Neural Information Processing Systems 14, NIPS (2001)Google Scholar
  8. 8.
    Nguyen, N.T., Phung, D.Q., Venkatesh, S., Bui, H.: Learning and detecting activities from movement trajectories using the hierarchical hidden markov models. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, CVPR, pp. 955–960. IEEE Computer Society, Washington, DC, USA (2005)Google Scholar
  9. 9.
    Patterson, D.J., Fox, D., Kautz, H.A., Philipose, M.: Fine-grained activity recognition by aggregating abstract object usage. In: Proceedings of the 9th International Symposium on Wearable Computers, ISWC, pp. 44–51. IEEE Computer Society (2005)Google Scholar
  10. 10.
    Turaga, P., Chellappa, R., Subrahmanian, V., Udrea, O.: Machine recognition of human activities: A survey. IEEE Trans. on Circuits and Systems for Video Technology 18(11), 1473–1488 (2008)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Tim L. M. van Kasteren
    • 1
  • Gwenn Englebienne
    • 2
  • Ben J. A. Kröse
    • 2
  1. 1.Department of Computer EngineeringBoğaziçi UniversityIstanbulTurkey
  2. 2.Intelligent Systems Lab AmsterdamUniversity of AmsterdamThe Netherlands

Personalised recommendations