Human Activity Recognition for Domestic Robots

  • Lasitha PiyathilakaEmail author
  • Sarath Kodagoda
Part of the Springer Tracts in Advanced Robotics book series (STAR, volume 105)


Capabilities of domestic service robots could be further improved, if the robot is equipped with an ability to recognize activities performed by humans in its sensory range. For example in a simple scenario a floor cleaning robot can vacuum the kitchen floor after recognizing human activity ”cooking in the kitchen”. Most of the complex human activities can be sub divided into simple activities which can later used for recognize complex activities. Activities like ”take meditation” can be sub divided into simple activities like ”opening pill container” and ”drinking water”. However, even recognizing simple activities are highly challenging due to the similarities between some inter activities and dissimilarities of intra activities which are performed by different people, body poses and orientations. Even a simple human activity like ”drinking water” can be performed while the subject is in different body poses like sitting, standing or walking. Therefore building machine learning techniques to recognize human activities with such complexities is non trivial. To address this issue, we propose a human activity recognition technique that uses 3D skeleton features produced by a depth camera. The algorithm incorporates importance weights for skeleton 3D joints according to the activity being performed. This allows the algorithm to ignore the confusing or irrelevant features while relying on informative features. Later these joints were ensembled together to train Dynamic Bayesian Networks (DBN), which is then used to infer human activities based on likelihoods. The proposed activity recognition technique is tested on a publicly available dataset and UTS experiments with overall accuracies of 85% and 90%.


Gaussian Mixture Model Dynamic Bayesian Network Observation Sequence Human Activity Recognition Depth Camera 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Brand, M., Oliver, N., Pentland, A.: Coupled hidden Markov models for complex action recognition. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 994–999 (1997), doi:10.1109/CVPR.1997.609450Google Scholar
  2. 2.
    Demiris, Y., Meltzoff, A.: The robot in the crib: A developmental analysis of imitation skills in infants and robots. Infant Child Dev. 17(1), 43–53 (2008)CrossRefGoogle Scholar
  3. 3.
    Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society, Series B 39(1), 1–38 (1977)zbMATHMathSciNetGoogle Scholar
  4. 4.
    Stork, J.A., Spinello, L., Silva, J., Arras, K.O.: Audio-based human activity recognition using non-markovian ensemble voting. In: Proc. of IEEE International Symposium on Robot and Human Interactive Communication, RoMan (2012)Google Scholar
  5. 5.
    Lopes, M., Melo, F.S., Montesano, L.: Affordance-based imitation learning in robots. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, USA, pp. 1015–1021 (2007)Google Scholar
  6. 6.
    Martinez-Contreras, F., Orrite-Urunuela, C., Herrero-Jaraba, E., Ragheb, H., Velastin, S.A.: Recognizing Human Actions Using Silhouette-based HMM. In: 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance, pp. 43–48 (2009)Google Scholar
  7. 7.
    Murphy, K.: Dynamic bayesian networks: Representation, inference and learning. Ph.D. thesis, UC Berkeley, Computer Science Division (2002)Google Scholar
  8. 8.
    Murphy, K.P.: The bayes net toolbox for matlab. Computing Science and Statistics 33 (2001)Google Scholar
  9. 9.
    Piyathilaka, L., Kodagoda, S.: Gaussian mixture based hmm for human activity recognition uisng 3d skeleton features. In: 8th IEEE Conference on Industrial Electronics and Applications (2013)Google Scholar
  10. 10.
    Rabiner, L.R.: A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE 77(2), 257–286 (1989), doi:10.1109/5.18626CrossRefGoogle Scholar
  11. 11.
    Sung, J., Ponce, C., Selman, B., Saxena, A.: Human activity detection from rgbd images. In: Plan, Activity, and Intent Recognition, vol. WS-11-16. AAAI (2011)Google Scholar
  12. 12.
    Theodoridis, T., Agapitos, A., Hu, H., Lucas, S.M.: Ubiquitous robotics in physical human action recognition: A comparison between dynamic anns and gp. In: ICRA, pp. 3064–3069. IEEE (2008)Google Scholar
  13. 13.
    Wu, Y., Yuan, J., Liu, Z., Wang, J.: Mining actionlet ensemble for action recognition with depth cameras. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1290–1297 (2012), doi:

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Centre for Autonomous SystemsUniversity of TechnologySydenyAustralia

Personalised recommendations