Dense Scene Flow Based on Depth and Multi-channel Bilateral Filter

  • Xiaowei Zhang
  • Dapeng Chen
  • Zejian Yuan
  • Nanning Zheng
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7726)


There is close relationship between depth information and scene flow. However, it’s not fully utilized in most of scene flow estimators. In this paper, we propose a method to estimate scene flow with monocular appearance images and corresponding depth images. We combine a global energy optimization and a bilateral filter into a two-step framework. Occluded pixels are detected by the consistency of appearance and depth, and the corresponding data errors are excluded from the energy function. The appearance and depth information are also utilized in anisotropic regularization to suppress over-smoothing. The multi-channel bilateral filter is introduced to correct scene flow with various information in non-local areas. The proposed approach is tested on Middlebury dataset and the sequences captured by KINECT. Experiment results show that it can estimate dense and accurate scene flow in challenging environments and keep the discontinuity around motion boundaries.


Energy Function Depth Information Motion Boundary Smoothness Constraint Weak Texture 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Huguet, F., Devernay, F.: A variational method for scene flow estimation from stereo sequences. In: ICCV, pp. 1–7. IEEE (2007)Google Scholar
  2. 2.
    Wedel, A., Brox, T., Vaudrey, T., Rabe, C., Franke, U., Cremers, D.: Stereoscopic Scene Flow Computation for 3D Motion Understanding. IJCV, 1–23 (2010)Google Scholar
  3. 3.
    Basha, T., Moses, Y., Kiryati, N.: Multi-view scene flow estimation: A view centered variational approach. In: CVPR, pp. 1506–1513. IEEE (2010)Google Scholar
  4. 4.
    Brox, T., Malik, J.: Large displacement optical flow: Descriptor matching in variational motion estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence 33, 500–513 (2011)CrossRefGoogle Scholar
  5. 5.
    Zhu, J., Wang, L., Yang, R., Davis, J.: Fusion of time-of-flight depth and stereo for high accuracy depth maps. In: CVPR, pp. 1–8. IEEE (2008)Google Scholar
  6. 6.
    Jan, Sanchez-Riera, J., Horaud, R.: Scene flow estimation by growing correspondence seeds. In: CVPR, pp. 3129–3136. IEEE (2011)Google Scholar
  7. 7.
    Hadfield, S., Bowden, R.: Kinecting the dots: Particle based scene flow from depth sensors. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 2290–2295. IEEE (2011)Google Scholar
  8. 8.
    Nagel, H., Enkelmann, W.: An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE Transactions on Pattern Analysis and Machine Intelligence, 565–593 (1986)Google Scholar
  9. 9.
    Zimmer, H., Bruhn, A., Weickert, J., Valgaerts, L., Salgado, A., Rosenhahn, B., Seidel, H.-P.: Complementary Optic Flow. In: Cremers, D., Boykov, Y., Blake, A., Schmidt, F.R. (eds.) EMMCVPR 2009. LNCS, vol. 5681, pp. 207–220. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  10. 10.
    Li, R., Sclaroff, S.: Multi-scale 3d scene flow from binocular stereo sequences. Computer Vision and Image Understanding 110, 75–90 (2008)CrossRefGoogle Scholar
  11. 11.
    Xiao, J., Cheng, H., Sawhney, H.S., Rao, C., Isnardi, M.: Bilateral Filtering-Based Optical Flow Estimation with Occlusion Detection. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 211–224. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  12. 12.
    Sun, D., Roth, S., Black, M.J.: Secrets of optical flow estimation and their principles. In: CVPR, pp. 2432–2439. IEEE (2010)Google Scholar
  13. 13.
    Courchay, J., Pons, J.-P., Monasse, P., Keriven, R.: Dense and Accurate Spatio-temporal Multi-view Stereovision. In: Zha, H., Taniguchi, R.-i., Maybank, S. (eds.) ACCV 2009, Part II. LNCS, vol. 5995, pp. 11–22. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  14. 14.
    Furukawa, Y., Ponce, J.: Dense 3d motion capture from synchronized video streams. Image and Geometry Processing for 3-D Cinematography, 193–211 (2010)Google Scholar
  15. 15.
    Vedula, S., Baker, S., Kanade, T.: Image-based spatio-temporal modeling and view interpolation of dynamic events. ACM Transactions on Graphics (TOG) 24, 240–261 (2005)CrossRefGoogle Scholar
  16. 16.
    Spies, H., Jähne, B., Barron, J.: Range flow estimation. Computer Vision and Image Understanding 85, 209–231 (2002)zbMATHCrossRefGoogle Scholar
  17. 17.
    Lukins, T.C., Fisher, R.B.: Colour constrained 4d flow. In: Proceedings of the British Machine Vision Conference, pp. 340–348 (2005)Google Scholar
  18. 18.
    Schuchert, T., Aach, T., Scharr, H.: Range Flow for Varying Illumination. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 509–522. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  19. 19.
    Lepetit, V., Moreno-Noguer, F., Fua, P.: Epnp: An accurate o (n) solution to the pnp problem. International Journal of Computer Vision 81, 155–166 (2009)CrossRefGoogle Scholar
  20. 20.
    Wedel, A., Pock, T., Zach, C., Bischof, H., Cremers, D.: An improved algorithm for tv-l 1 optical flow. Statistical and Geometrical Approaches to Visual Motion Analysis, 23–45 (2009)Google Scholar
  21. 21.
    Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High Accuracy Optical Flow Estimation Based on a Theory for Warping. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  22. 22.
    Bleyer, M., Gelautz, M.: Graph-cut-based stereo matching using image segmentation with symmetrical treatment of occlusions. Signal Processing: Image Communication 22, 127–143 (2007)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Xiaowei Zhang
    • 1
  • Dapeng Chen
    • 1
  • Zejian Yuan
    • 1
  • Nanning Zheng
    • 1
  1. 1.Institute of AI & RoboticsXi’an Jiaotong UniversityChina

Personalised recommendations