Advertisement

It’s Moving! A Probabilistic Model for Causal Motion Segmentation in Moving Camera Videos

  • Pia Bideau
  • Erik Learned-Miller
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9912)

Abstract

The human ability to detect and segment moving objects works in the presence of multiple objects, complex background geometry, motion of the observer, and even camouflage. In addition to all of this, the ability to detect motion is nearly instantaneous. While there has been much recent progress in motion segmentation, it still appears we are far from human capabilities. In this work, we derive from first principles a likelihood function for assessing the probability of an optical flow vector given the 2D motion direction of an object. This likelihood uses a novel combination of the angle and magnitude of the optical flow to maximize the information about how objects are moving differently. Using this new likelihood and several innovations in initialization, we develop a motion segmentation algorithm that beats current state-of-the-art methods by a large margin. We compare to five state-of-the-art methods on two established benchmarks, and a third new data set of camouflaged animals, which we introduce to push motion segmentation to the next level.

Keywords

Motion segmentation Video segmentation Optical flow Moving camera Background subtraction 

Supplementary material

419983_1_En_26_MOESM1_ESM.pdf (1.2 mb)
Supplementary material 1 (pdf 1180 KB)

References

  1. 1.
    Torr, P.H.: Geometric motion segmentation and model selection. Philos. Trans. Royal Soc. Lond. Math. Phys. Eng. Sci. 356(1740), 1321–1340 (1998)CrossRefzbMATHMathSciNetGoogle Scholar
  2. 2.
    Brox, T., Malik, J.: Object segmentation by long term analysis of point trajectories. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 282–295. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  3. 3.
    Tron, R., Vidal, R.: A benchmark for the comparison of 3-D motion segmentation algorithms. In: CVPR (2007)Google Scholar
  4. 4.
    Narayana, M., Hanson, A., Learned-Miller, E.: Coherent motion segmentation in moving camera videos using optical flow orientations. In: 2013 IEEE International Conference on Computer Vision (ICCV), pp. 1577–1584. IEEE (2013)Google Scholar
  5. 5.
    Grundmann, M., Kwatra, V., Han, M., Essa, I.: Efficient hierarchical graph-based video segmentation. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2141–2148. IEEE (2010)Google Scholar
  6. 6.
    Lezama, J., Alahari, K., Sivic, J., Laptev, I.: Track to the future: spatio-temporal video segmentation with long-range motion cues. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2011)Google Scholar
  7. 7.
    Kumar, M.P., Torr, P.H., Zisserman, A.: Learning layered motion segmentations of video. Int. J. Comput. Vis. 76(3), 301–319 (2008)CrossRefGoogle Scholar
  8. 8.
    Irani, M., Rousso, B., Peleg, S.: Computing occluding and transparent motions. Int. J. Comput. Vis. 12, 5–16 (1994)CrossRefGoogle Scholar
  9. 9.
    Ren, Y., Chua, C.S., Ho, Y.K.: Statistical background modeling for non-stationary camera. Pattern Recogn. Lett. 24, 183–196 (2003)CrossRefzbMATHGoogle Scholar
  10. 10.
    Sheikh, Y., Javed, O., Kanade, T.: Background subtraction for freely moving cameras. In: IEEE 12th International Conference on Computer Vision, ICCV 2009, Kyoto, Japan, September 27–October 4 2009, pp. 1219–1225. IEEE (2009)Google Scholar
  11. 11.
    Elqursh, A., Elgammal, A.: Online moving camera background subtraction. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part VI. LNCS, vol. 7577, pp. 228–241. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  12. 12.
    Ochs, P., Brox, T.: Higher order motion models and spectral clustering. In: CVPR (2012)Google Scholar
  13. 13.
    Kwak, S., Lim, T., Nam, W., Han, B., Han, J.H.: Generalized background subtraction based on hybrid inference by belief propagation and Bayesian filtering. In: ICCV (2011)Google Scholar
  14. 14.
    Rahmati, H., Dragon, R., Aamo, O.M., Gool, L., Adde, L.: Motion segmentation with weak labeling priors. In: Jiang, X., Hornegger, J., Koch, R. (eds.) GCPR 2014. LNCS, vol. 8753, pp. 159–171. Springer, Heidelberg (2014). doi: 10.1007/978-3-319-11752-2_13 Google Scholar
  15. 15.
    Jain, S.D., Grauman, K.: Supervoxel-consistent foreground propagation in video. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part IV. LNCS, vol. 8692, pp. 656–671. Springer, Heidelberg (2014)Google Scholar
  16. 16.
    Zamalieva, D., Yilmaz, A., Davis, J.W.: A multi-transformational model for background subtraction with moving cameras. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part I. LNCS, vol. 8689, pp. 803–817. Springer, Heidelberg (2014)Google Scholar
  17. 17.
    Papazoglou, A., Ferrari, V.: Fast object segmentation in unconstrained video. In: 2013 IEEE International Conference on Computer Vision (ICCV), pp. 1777–1784. IEEE (2013)Google Scholar
  18. 18.
    Keuper, M., Andres, B., Brox, T.: Motion trajectory segmentation via minimum cost multicuts. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3271–3279 (2015)Google Scholar
  19. 19.
    Taylor, B., Karasev, V., Soatto, S.: Causal video object segmentation from persistence of occlusions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4268–4276 (2015)Google Scholar
  20. 20.
    Fragkiadaki, K., Zhang, G., Shi, J.: Video segmentation by tracing discontinuities in a trajectory embedding. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1846–1853. IEEE (2012)Google Scholar
  21. 21.
    Sawhney, H.S., Guo, Y., Asmuth, J., Kumar, R.: Independent motion detection in 3D scenes. In: The Proceedings of the Seventh IEEE International Conference on Computer Vision, vol. 1, pp. 612–619. IEEE (1999)Google Scholar
  22. 22.
    Dey, S., Reilly, V., Saleemi, I., Shah, M.: Detection of independently moving objects in non-planar scenes via multi-frame monocular epipolar constraint. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part V. LNCS, vol. 7576, pp. 860–873. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  23. 23.
    Namdev, R.K., Kundu, A., Krishna, K.M., Jawahar, C.V.: Motion segmentation of multiple objects from a freely moving monocular camera. In: International Conference on Robotics and Automation (2012)Google Scholar
  24. 24.
    Csurka, G., Bouthemy, P.: Direct identification of moving objects and background from 2D motion models. In: The Proceedings of the Seventh IEEE International Conference on Computer Vision, vol. 1, pp. 566–571. IEEE (1999)Google Scholar
  25. 25.
    Sharma, R., Aloimonos, Y.: Early detection of independent motion from active control of normal image flow patterns. IEEE Trans. Syst. Man Cybernet. B (Cybernetics) 26(1), 42–52 (1996)CrossRefGoogle Scholar
  26. 26.
    Elhamifar, E., Vidal, R.: Sparse subspace clustering. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 2790–2797. IEEE (2009)Google Scholar
  27. 27.
    Horn, B.K.: Projective geometry considered harmful (1999)Google Scholar
  28. 28.
    Ogale, A.S., Fermüller, C., Aloimonos, Y.: Motion segmentation using occlusions. IEEE Trans. Pattern Anal. Mach. Intell. 27(6), 988–992 (2005)CrossRefGoogle Scholar
  29. 29.
    Horn, B.: Robot Vision. MIT Press, Cambridge (1986)Google Scholar
  30. 30.
    Bruss, A.R., Horn, B.K.: Passive navigation. Comput. Vis. Graph. Image Process. 21(1), 3–20 (1983)CrossRefGoogle Scholar
  31. 31.
    Sun, D., Roth, S., Black, M.J.: Secrets of optical flow estimation and their principles. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2432–2439. IEEE (2010)Google Scholar
  32. 32.
    Narayana, M., Hanson, A., Learned-Miller, E.G.: Background subtraction: separating the modeling and the inference. Mach. Vis. Appl. 25(5), 1163–1174 (2014)CrossRefGoogle Scholar
  33. 33.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)CrossRefMathSciNetGoogle Scholar
  34. 34.
    Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Susstrunk, S.: SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2274–2282 (2012)CrossRefGoogle Scholar
  35. 35.
    Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man, Cybernet. 9, 62–66 (1979)CrossRefGoogle Scholar
  36. 36.
    Wang, J.Y., Adelson, E.H.: Representing moving images with layers. IEEE Trans. Image Process 3(5), 625–638 (1994)CrossRefGoogle Scholar
  37. 37.
    Powers, D.M.: Evaluation: from precision, recall and f-measure to ROC, informedness, markedness and correlation. Technical report SIE-07-001. Flinders University, Adelaide (2007)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.College of Information and Computer SciencesUniversity of MassachusettsAmherstUSA

Personalised recommendations