3D Motion Consistency Analysis for Segmentation in 2D Video Projection

  • Wei ZhaoEmail author
  • Nico Roos
  • Ralf Peeters
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10425)


Motion segmentation for 2D videos is usually based on tracked 2D point motions, obtained for a sequence of frames. However, the 3D real world motion consistency is easily lost in the process, due to projection from 3D space to the 2D image plane. Several approaches have been proposed in the literature to recover 3D motion consistency from 2D point motions. To further improve on this, we here propose a new criterion and associated technique, which can be used to determine whether a group of points show 2D motions consistent with joint 3D motion. It is also applicable for estimating the 3D motion information content. We demonstrate that the proposed criterion can be applied to improve segmentation results in two ways: finding the misclassified points in a group, and assigning unclassified points to the correct group. Experiments with synthetic data and different noise levels, and with real data taken from a benchmark, give insight in the performance of the algorithm under various conditions.


  1. 1.
    Altunbasak, Y., Eren, P.E., Tekalp, A.M.: Region-based parametric motion segmentation using color information. Graph. Model. Image Process. 60(1), 13–23 (1998)CrossRefGoogle Scholar
  2. 2.
    Borshukov, G.D., Bozdagi, G., Altunbasak, Y., Tekalp, A.M.: Motion segmentation by multistage affine classification. IEEE Trans. Image Process. 6(11), 1591–1594 (1997)CrossRefGoogle Scholar
  3. 3.
    Boult, T.E., Brown, L.G.: Factorization-based segmentation of motions. In: 1991 Proceedings of the IEEE Workshop on Visual Motion, pp. 179–186. IEEE (1991)Google Scholar
  4. 4.
    Bovik, A.C.: Handbook of Image and Video Processing. Academic Press, London (2010)zbMATHGoogle Scholar
  5. 5.
    Costeira, J., Kanade, T.: A multi-body factorization method for motion analysis. In: 1995 Proceedings of the Fifth International Conference on Computer Vision, pp. 1071–1076. IEEE (1995)Google Scholar
  6. 6.
    Elhamifar, E., Vidal, R.: Sparse subspace clustering. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 2790–2797. IEEE (2009)Google Scholar
  7. 7.
    Gruber, A., Weiss, Y.: Multibody factorization with uncertainty and missing data using the EM algorithm. In: 2004 Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2004, vol. 1, p. I. IEEE (2004)Google Scholar
  8. 8.
    Gruber, A., Weiss, Y.: Incorporating non-motion cues into 3D motion segmentation. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3953, pp. 84–97. Springer, Heidelberg (2006). doi: 10.1007/11744078_7 CrossRefGoogle Scholar
  9. 9.
    Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, New York (2004). ISBN: 0521540518CrossRefzbMATHGoogle Scholar
  10. 10.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2003)zbMATHGoogle Scholar
  11. 11.
    Horn, B.K., Schunck, B.G.: Determining optical flow. In: 1981 Technical Symposium East, pp. 319–331. International Society for Optics and Photonics (1981)Google Scholar
  12. 12.
    Jian, Y.D., Chen, C.S.: Two-view motion segmentation with model selection and outlier removal by Ransac-enhanced Dirichlet process mixture models. Int. J. Comput. Vision 88(3), 489–501 (2010)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Jodoin, P.M., Pierard, S., Wang, Y., Van Droogenbroeck, M.: Overview and benchmarking of motion detection methods (2014)Google Scholar
  14. 14.
    Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: a factorization method. Int. J. Comput. Vision 9(2), 137–154 (1992)CrossRefGoogle Scholar
  15. 15.
    Torr, P.H., Szeliski, R., Anandan, P.: An integrated Bayesian approach to layer extraction from image sequences. IEEE Trans. Pattern Anal. Mach. Intell. 23(3), 297–303 (2001)CrossRefGoogle Scholar
  16. 16.
    Torr, P.H.S., Zisserman, A.: Concerning Bayesian motion segmentation, model averaging, matching and the trifocal tensor. In: Burkhardt, H., Neumann, B. (eds.) ECCV 1998. LNCS, vol. 1406, pp. 511–527. Springer, Heidelberg (1998). doi: 10.1007/BFb0055687 Google Scholar
  17. 17.
    Tron, R., Vidal, R.: A benchmark for the comparison of 3-D motion segmentation algorithms. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2007, pp. 1–8. IEEE (2007)Google Scholar
  18. 18.
    Ullman, S.: The interpretation of structure from motion. Proc. R. Soc. Lond. B Biol. Sci. 203(1153), 405–426 (1979)CrossRefGoogle Scholar
  19. 19.
    Vidal, R., Soatto, S., Ma, Y., Sastry, S.: Segmentation of dynamic scenes from the multibody fundamental matrix. Urbana 51(61801), 1–2Google Scholar
  20. 20.
    Wang, H., Kläser, A., Schmid, C., Liu, C.L.: Dense trajectories and motion boundary descriptors for action recognition. Int. J. Comput. Vision 103(1), 60–79 (2013)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Wang, J.Y., Adelson, E.H.: Layered representation for motion analysis. In: 1993 IEEE Proceedings of the Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 1993, pp. 361–366. IEEE (1993)Google Scholar
  22. 22.
    Weiss, Y.: Smoothness in layers: motion segmentation using nonparametric mixture estimation. In: 1997 IEEE Proceedings of the Computer Society Conference on Computer Vision and Pattern Recognition, pp. 520–526. IEEE (1997)Google Scholar
  23. 23.
    Yuan, C.: Motion segmentation and dense reconstruction of scenes containing moving objects observed by a moving camera. ProQuest (2007)Google Scholar
  24. 24.
    Zelnik-Manor, L., Machline, M., Irani, M.: Multi-body factorization with uncertainty: revisiting motion consistency. Int. J. Comput. Vision 68(1), 27–41 (2006)CrossRefGoogle Scholar
  25. 25.
    Zhang, J., Shi, F., Liu, Y.: Motion segmentation by multibody trifocal tensor using line correspondence. In: 2006 Proceedings of the 18th International Conference on Pattern Recognition, ICPR 2006, vol. 1, pp. 599–602. IEEE (2006)Google Scholar
  26. 26.
    Zhao, W., Roos, N.: Motion based segmentation for robot vision using adapted em algorithm. In: Proceedings of the 11th International Conference on Computer Vision Theory and Applications, VISIGRAp 2016, pp. 649–656 (2016)Google Scholar
  27. 27.
    Zhao, W., Roos, N.: An EM based approach for motion segmentation of video sequence. In: Proceedings of the 24th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision, WSCG 2016, pp. 61–69 (2016)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Department of Data Science and Knowledge EngineeringMaastricht UniversityMaastrichtThe Netherlands

Personalised recommendations