Knowledge Based Activity Recognition with Dynamic Bayesian Network

  • Zhi Zeng
  • Qiang Ji
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6316)


In this paper, we propose solutions on learning dynamic Bayesian network (DBN) with domain knowledge for human activity recognition. Different types of domain knowledge, in terms of first order probabilistic logics (FOPLs), are exploited to guide the DBN learning process. The FOPLs are transformed into two types of model priors: structure prior and parameter constraints. We present a structure learning algorithm, constrained structural EM (CSEM), on learning the model structures combining the training data with these priors. Our method successfully alleviates the common problem of lack of sufficient training data in activity recognition. The experimental results demonstrate simple logic knowledge can compensate effectively for the shortage of the training data and therefore reduce our dependencies on training data.


Domain Knowledge Training Sequence Dynamic Bayesian Network Candidate Structure Defense Advance Research Project Agency 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Yamato, J., Ohaya, J., Ishii, K.: Recognizing human action in time-sequential images using hidden markov model. In: CVPR (1992)Google Scholar
  2. 2.
    Zhang, D., Perez, D., McCowan, I.: Semi-supervised adapted hmms for unusual event detection. In: CVPR (2005)Google Scholar
  3. 3.
    Vogler, C., Metaxas, D.: A framework for recognizing the simultaneous sspects of american sign language. In: CVIU (2001)Google Scholar
  4. 4.
    Oliver, N.M., Rosario, B., Pentland, A.P.: A bayesian computer vision system for modeling human interations. PAMI (2000)Google Scholar
  5. 5.
    Xiang, T., Gong, S.: Beyond tracking: Modelling activity and understanding behavior. In: IJCV (2006)Google Scholar
  6. 6.
    Oliver, N., Horvitz, E., Garg, A.: Layered representation for human activity recognition. CVIU (2004)Google Scholar
  7. 7.
    Duong, T., Bui, H., Phung, D.: Activity recognition and abnormality detection with the switching hidden semi-markov model. In: CVPR (2005)Google Scholar
  8. 8.
    Lv, F., Nevatia, R.: Single view human action recognition using key pose matching and viterbi path searching. In: CVPR (2007)Google Scholar
  9. 9.
    Xiang, T., Gong, S.: Video behavior profiling for anomly detection. PAMI (2008)Google Scholar
  10. 10.
    Wu, J., Osuntogun, A., Choudhury, T., Philipose, M., Regh, J.: A scalable approach to activity recognition based on object use. In: ICCV (2007)Google Scholar
  11. 11.
    Laxton, B., Lim, J., Kriegman, D.: Leveraging temporal, contextual and ordering constraints for recognizing complex activities in video. In: CVPR (2007)Google Scholar
  12. 12.
    Tran, S., Davis, L.: Event modeling and recognition using markov logic networks. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 610–623. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  13. 13.
    Biswas, R., Thrun, S., Fujimura, K.: Recognizing activities with multiple cues. In: IEEE Works. on Human Motion (2007)Google Scholar
  14. 14.
    Richardson, M., Domingos, P.: Markov logic networks. In: Machine Learning (2006)Google Scholar
  15. 15.
    Tong, Y., Ji, Q.: Learning bayesian network with qualitative constraints. In: CVPR (2008)Google Scholar
  16. 16.
    Friedman, N.: The bayesian structural em algorithm. In: UAI (1998)Google Scholar
  17. 17.
    Murphy, K.: Dynamic bayesian networks: representation, inference and learning. Ph.D. dissertation, University of California (2002)Google Scholar
  18. 18.
    Milch, B., Russell, S.: First-order probabilistic languages: Into the unknown. In: ILP (2006)Google Scholar
  19. 19.
    Halpern, J.: An analysis of first-order logics of probability. Artificial Intelligence (1990)Google Scholar
  20. 20.
    Heckerman, D.: A tutorial on learning with bayesian networks. Learning in Graphical Models (1999)Google Scholar
  21. 21.
    Fathi, A., Mori, G.: Action recognition by learning mid-level motion features. In: CVPR (2008)Google Scholar
  22. 22.
    Jhuang, H., Serre, T., Wolf, L., Poggio, T.: A biologicallyinspired system for action recognition. In: ICCV (2007)Google Scholar
  23. 23.
    Thurau, C., Hlavac, V.: Pose primitive based human action recognition in videos or still image. In: CVPR (2008)Google Scholar
  24. 24.
    Niebles, J., Fei-Fei, L.: A hierarchical model of shape and appearance for human action classification. In: CVPR (2008)Google Scholar
  25. 25.
    Yuan, J., Liu, Z., Wu, Y.: Discriminative subvolume search for efficient action detection. In: CVPR (2009)Google Scholar
  26. 26.
    Laptev, I., Marszalek, M., Schmid, C., Rozenfeld, B.: Learning realistic human acions from movies. In: CVPR (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Zhi Zeng
    • 1
  • Qiang Ji
    • 1
  1. 1.Rensselaer Polytechnic InstituteTroyUSA

Personalised recommendations