Multi-scale Clustering of Frame-to-Frame Correspondences for Motion Segmentation

  • Ralf Dragon
  • Bodo Rosenhahn
  • Jörn Ostermann
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7573)


We present an approach for motion segmentation using independently detected keypoints instead of commonly used tracklets or trajectories. This allows us to establish correspondences over non- consecutive frames, thus we are able to handle multiple object occlusions consistently. On a frame-to-frame level, we extend the classical split-and-merge algorithm for fast and precise motion segmentation. Globally, we cluster multiple of these segmentations of different time scales with an accurate estimation of the number of motions. On the standard benchmarks, our approach performs best in comparison to all algorithms which are able to handle unconstrained missing data. We further show that it works on benchmark data with more than 98% of the input data missing. Finally, the performance is evaluated on a mobile-phone-recorded sequence with multiple objects occluded at the same time.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Lezama, J., Alahari, K., Sivic, J., Laptev, I.: Track to the future: Spatio-temporal video segmentation with long-range motion cues. In: CVPR, pp. 3369–3376 (2011)Google Scholar
  2. 2.
    Ochs, P., Brox, T.: Object segmentation in video: A hierarchical variational approach for turning point trajectories into dense regions. In: Proc. ICCV (2011)Google Scholar
  3. 3.
    Sheikh, Y., Javed, O., Kanade, T.: Background subtraction for freely moving cameras. In: Proc. ICCV, pp. 1219–1225 (2009)Google Scholar
  4. 4.
    Brox, T., Malik, J.: Object Segmentation by Long Term Analysis of Point Trajectories. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 282–295. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  5. 5.
    Cifuentes, C.G., Sturzel, M., Jurie, F., Brostow, G.J.: Motion models that only work sometimes. In: Proc. BMVC (2012)Google Scholar
  6. 6.
    Sand, P., Teller, S.: Particle video: Long-range motion estimation using point trajectories. IJCV 80, 72–91 (2008)CrossRefGoogle Scholar
  7. 7.
    Cheriyadat, A., Radke, R.: Non-negative matrix factorization of partial track data for motion segmentation. In: Proc. ICCV, pp. 865–872 (October 2009)Google Scholar
  8. 8.
    Sivic, J., Schaffalitzky, F., Zisserman, A.: Object level grouping for video shots. IJCV 67(2), 189–210 (2006)CrossRefGoogle Scholar
  9. 9.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. IJCV 60, 91–110 (2004)CrossRefGoogle Scholar
  10. 10.
    Favaro, P., Vidal, R., Ravichandran, A.: A closed form solution to robust subspace estimation and clustering. In: Proc. CVPR, pp. 1801–1807 (2011)Google Scholar
  11. 11.
    Chen, G., Lerman, G.: Motion segmentation by scc on the hopkins 155 database. In: Proc. ICCV Workshop on Dynamical Vision (2009)Google Scholar
  12. 12.
    Yu, J., Chin, T.J., Suter, D.: A global optimization approach to robust multi-model fitting. In: Proc. CVPR (2011)Google Scholar
  13. 13.
    Vidal, R., Hartley, R.: Motion segmentation with missing data using powerfactorization and gpca. In: Proc. CVPR, pp. 310–316 (2004)Google Scholar
  14. 14.
    Rao, S., Tron, R., Vidal, R., Ma, Y.: Motion segmentation in the presence of outlying, incomplete, or corrupted trajectories. TPAMI 32, 1832–1845 (2010)CrossRefGoogle Scholar
  15. 15.
    Ng, A.Y., Jordan, M.I., Weiss, Y.: On spectral clustering: Analysis and an algorithm. In: Proc. NIPS, pp. 849–856 (2001)Google Scholar
  16. 16.
    Fradet, M., Robert, P., Perez, P.: Clustering point trajectories with various life-spans. In: Proc. CVMP, pp. 7–14 (2009)Google Scholar
  17. 17.
    Toldo, R., Fusiello, A.: Robust Multiple Structures Estimation with J-Linkage. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 537–547. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  18. 18.
    Elhamifar, E., Vidal, R.: Sparse subspace clustering. In: Proc. CVPR, pp. 2790–2797 (2009)Google Scholar
  19. 19.
    Fouhey, D.F., Scharstein, D., Briggs, A.J.: Multiple plane detection in image pairs using j-linkage. In: Proc. ICPR (2010)Google Scholar
  20. 20.
    Jain, R., Kasturi, R., Schunck, B.G.: Machine Vision. McGraw-Hill (1995)Google Scholar
  21. 21.
    Fischler, M.A., Bolles, R.C.: Random sample consensus. CACM 24, 381–395 (1981)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Tron, R., Vidal, R.: A benchmark for the comparison of 3-d motion segmentation algorithms. In: Proc. CVPR (2007)Google Scholar
  23. 23.
    Chin, T.J., Wang, H., Suter, D.: The ordered residual kernel for robust motion subspace clustering. In: Proc. NIPS, pp. 333–341 (2009)Google Scholar
  24. 24.
    Chin, T.J., Suter, D., Wang, H.: Multi-structure model selection via kernel optimisation. In: Proc. CVPR, pp. 3586–3593 (2010)Google Scholar
  25. 25.
    Wu, C.: SiftGPU: A GPU implementation of scale invariant feature transform (SIFT) (2007),

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Ralf Dragon
    • 1
  • Bodo Rosenhahn
    • 1
  • Jörn Ostermann
    • 1
  1. 1.Institut für InformationsverarbeitungLeibniz Universität HannoverGermany

Personalised recommendations