Kernelized Temporal Cut for Online Temporal Segmentation and Recognition

  • Dian Gong
  • Gérard Medioni
  • Sikai Zhu
  • Xuemei Zhao
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7574)


We address the problem of unsupervised online segmenting human motion sequences into different actions. Kernelized Temporal Cut (KTC), is proposed to sequentially cut the structured sequential data into different regimes. KTC extends previous works on online change-point detection by incorporating Hilbert space embedding of distributions to handle the nonparametric and high dimensionality issues. Based on KTC, a realtime online algorithm and a hierarchical extension are proposed for detecting both action transitions and cyclic motions at the same time. We evaluate and compare the approach to state-of-the-art methods on motion capture data, depth sensor data and videos. Experimental results demonstrate the effectiveness of our approach, which yields realtime segmentation, and produces higher action segmentation accuracy. Furthermore, by combining with sequence matching algorithms, we can online recognize actions of an arbitrary person from an arbitrary viewpoint, given realtime depth sensor input.


Action Recognition Spectral Cluster Reproduce Kernel Hilbert Space Rand Index Depth Sensor 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Barbic, J., Safonova, A., Pan, J.Y., Faloutsos, C., Hodgins, J.K., Pollard, N.S.: Segmenting motion capture data into distinct behaviors. In: Proc. Graphics Interface, pp. 185–194 (2004)Google Scholar
  2. 2.
    Lv, F., Nevatia, R.: Recognition and Segmentation of 3-D Human Action Using HMM and Multi-class AdaBoost. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006, Part IV. LNCS, vol. 3954, pp. 359–372. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  3. 3.
    Laptev, I., Marszalek, M., Schmid, C., Rozenfeld, B.: Learning realistic human actions from movies. In: Proc. CVPR (2008)Google Scholar
  4. 4.
    Weinland, D., Özuysal, M., Fua, P.: Making Action Recognition Robust to Occlusions and Viewpoint Changes. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part III. LNCS, vol. 6313, pp. 635–648. Springer, Heidelberg (2010)Google Scholar
  5. 5.
    Zhong, H., Shi, J., Visontai, M.: Detecting unusual activity in video. In: Proc. CVPR, pp. 816–8231 (2004)Google Scholar
  6. 6.
    Zhou, F., De la Torre, F., Hodgins, J.K.: Hierarchical aligned cluster analysis for temporal clustering of human motion. Under review at IEEE PAMI (2011)Google Scholar
  7. 7.
    Niebles, J.C., Chen, C.-W., Fei-Fei, L.: Modeling Temporal Structure of Decomposable Motion Segments for Activity Classification. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part II. LNCS, vol. 6312, pp. 392–405. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  8. 8.
    Chen, J., Gupta, A.: Parametric Statistical Change-point Analysis. Birkhäuser (2000)Google Scholar
  9. 9.
    Adams, R.P., MacKay, D.J.: Bayesian online changepoint detection. University of Cambridge Technical Report (2007)Google Scholar
  10. 10.
    De la Torre, F., Campoy, J., Ambadar, Z., Conn, J.F.: Temporal segmentation of facial behavior. In: Proc. ICCV (2007)Google Scholar
  11. 11.
    Hofmann, T., Scholkopf, B., Smola, A.: Kernel methods in machine learning. Annals of Statistics 36, 1171–1220 (2008)MathSciNetzbMATHCrossRefGoogle Scholar
  12. 12.
    Smola, A.J., Gretton, A., Song, L., Schölkopf, B.: A Hilbert Space Embedding for Distributions. In: Hutter, M., Servedio, R.A., Takimoto, E. (eds.) ALT 2007. LNCS (LNAI), vol. 4754, pp. 13–31. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  13. 13.
    Gretton, A., Fukumizu, K., Harchaoui, Z., Sriperumbudur, B.: A fast, consistent kernel two-sample test. In: NIPS, vol. 19, pp. 673–681 (2009)Google Scholar
  14. 14.
    Gretton, A., Borgwardt, K., Rasch, M., Scholkopf, B., Smola, A.: A kernel method for the two-sample-problem. In: NIPS, vol. 19, pp. 513–520 (2007)Google Scholar
  15. 15.
    Zhou, F., De la Torre, F.: Canonical time warping for alignment of human behavior. In: NIPS, vol. 22, pp. 2286–2294 (2009)Google Scholar
  16. 16.
    Xuan, X., Murphy, K.: Modeling changing dependency structure in multivariate time series. In: Proc. ICML (2007)Google Scholar
  17. 17.
    Saatci, Y., Turner, R., Rasmussen, C.: Gaussian process change point models. In: Proc. ICML (2010)Google Scholar
  18. 18.
    Desobry, F., Davy, M., Doncarli, C.: An online kernel change detection algorithm. IEEE Trans. on Signal Processing 53, 2961–2974 (2005)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Harchaoui, Z., Bach, F., Moulines, E.: Kernel change-point analysis. In: NIPS 21, pp. 609–616 (2009)Google Scholar
  20. 20.
    Ng, A.Y., Jordan, M.I., Weiss, Y.: On spectral clustering: Analysis and an algorithm. In: NIPS, vol. 14, pp. 849–856 (2002)Google Scholar
  21. 21.
    von Luxburg, U.: A tutorial on spectral clustering. Statistics and Computing 17, 395–416 (2007)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Fox, E., Sudderth, E., Jordan, M., Willsky, A.: Nonparametric bayesian learning of switching linear dynamical systems. In: NIPS 21, pp. 457–464 (2009)Google Scholar
  23. 23.
    Zelnik-Manor, L., Irani, M.: Statistical analysis of dynamic actions. IEEE PAMI 28, 1530–1535 (2006)CrossRefGoogle Scholar
  24. 24.
    Satkin, S., Hebert, M.: Modeling the Temporal Extent of Actions. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 536–548. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  25. 25.
    Gong, D., Medioni, G.: Dynamic manifold warping for view invariant action recognition. In: Proc. ICCV, pp. 571–578 (2011)Google Scholar
  26. 26.
    Hoai, M., Lan, Z., De la Torre, F.: Joint segmentation and classification of human actions in video. In: Proc. CVPR (2011)Google Scholar
  27. 27.
    Faivishevsky, L., Goldberger, J.: A nonparametric information theoretic clustering algorithm. In: Proc. ICML, pp. 351–358 (2010)Google Scholar
  28. 28.
    Bach, F.R., Jordan, M.I.: Kernel independent component analysis. JMLR 3, 1–48 (2003)MathSciNetzbMATHGoogle Scholar
  29. 29.
    Laptev, I., Belongie, S., Perez, P., Wills, J.: Periodic motion detection and segmentation via approximate sequence alignment. In: Proc. ICCV, pp. 816–8231 (2005)Google Scholar
  30. 30.
    Sigal, L., Balan, A.O., Black, M.J.: Humaneva: Synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion. IJCV 87, 4–27 (2010)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Dian Gong
    • 1
  • Gérard Medioni
    • 1
  • Sikai Zhu
    • 1
  • Xuemei Zhao
    • 1
  1. 1.Institute for Robotics and Intelligent SystemsUniversity of Southern CaliforniaLos AngelesUSA

Personalised recommendations