Sparse Stereo Disparity Map Densification Using Hierarchical Image Segmentation

  • Sébastien DrouyerEmail author
  • Serge Beucher
  • Michel Bilodeau
  • Maxime Moreaud
  • Loïc Sorbier
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10225)


We describe a novel method for propagating disparity values using hierarchical segmentation by waterfall and robust regression models. High confidence disparity values obtained by state of the art stereo matching algorithms are interpolated using a coarse to fine approach. We start from a coarse segmentation of the image and try to fit each region’s disparities using robust regression models. If the fit is not satisfying, the process is repeated on a finer region’s segmentation. Erroneous values in the initial sparse disparity maps are generally excluded, as we use robust regressions algorithms and left-right consistency checks. Final disparity maps are therefore not only denser but can also be more accurate. The proposed method is general and independent from the sparse disparity map generation: it can therefore be used as a post-processing step for any stereo-matching algorithm.


Stereo Hierarchical segmentation Robust regression model Waterfall Disparity map Densification 


  1. 1.
    Alvarez, L., Deriche, R., Sánchez, J., Weickert, J.: Dense disparity map estimation respecting image discontinuities: a PDE and scale-space based approach. JVCIR 13(1–2), 3–21 (2002)Google Scholar
  2. 2.
    Ayache, N., Hansen, C.: Rectification of images for binocular and trinocular stereovision. In: ICPR 1988 (1988)Google Scholar
  3. 3.
    Barron, J.T., Poole, B.: The fast bilateral solver. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 617–632. Springer, Cham (2016). doi: 10.1007/978-3-319-46487-9_38 CrossRefGoogle Scholar
  4. 4.
    Beucher, S.: Image segmentation and mathematical morphology. Theses, École Nationale Supérieure des Mines de Paris, June 1990Google Scholar
  5. 5.
    Beucher, S.: Towards a unification of waterfalls, standard and P algorithms, working paper or preprint, January 2013Google Scholar
  6. 6.
    Bricola, J.-C., Bilodeau, M., Beucher, S.: A top-down methodology to depth map estimation controlled by morphological segmentation. Technical report (2014)Google Scholar
  7. 7.
    Bricola, J.-C., Bilodeau, M., Beucher, S.: A multi-scale and morphological gradient preserving contrast. In: 14th International Congress for Stereology and Image Analysis, Liège, Belgium, Eric Pirard, July 2015Google Scholar
  8. 8.
    Bricola, J.-C., Bilodeau, M., Beucher, S.: Morphological processing of stereoscopic image superimpositions for disparity map estimation, working paper or preprint, March 2016Google Scholar
  9. 9.
    Facciolo, G., de Franchis, C., Meinhardt, E.: MGM: a significantly more global matching for stereovision. In: BMVC (2015)Google Scholar
  10. 10.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Fua, P.: A parallel stereo algorithm that produces dense depth maps and preserves image features. Mach. Vis. Appl. 6(1), 35–49 (1993)CrossRefGoogle Scholar
  12. 12.
    Hirschmüller, H.: Stereo processing by semiglobal matching and mutual information. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 328–341 (2008)CrossRefGoogle Scholar
  13. 13.
    Itseez. Open source computer vision library. (2015)
  14. 14.
    Kanade, T., Okutomi, M.: A stereo matching algorithm with an adaptive window: theory and experiment. In: Proceedings of IEEE ICRA (1991)Google Scholar
  15. 15.
    Kolmogorov, V.: Graph based algorithms for scene reconstruction from two or more views. Ph.D. thesis, Ithaca, NY, USA (2004). AAI3114475Google Scholar
  16. 16.
    Konolige, K.: Small vision systems: hardware and implementation. In: Shirai, Y., Hirose, S. (eds.) Robotics Research, pp. 203–212. Springer Nature, London (1998)CrossRefGoogle Scholar
  17. 17.
    Min, D., Choi, S., Lu, J., Ham, B., Sohn, K., Do, M.N.: Fast global image smoothing based on weighted least squares. IEEE Trans. Image Process. 23(12), 5638–5653 (2014)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Moravec, K., Harvey, R., Bangham, J.A.: Improving stereo performance in regions of low texture. In: BMVC (1998)Google Scholar
  19. 19.
    Ralli, J., Díaz, J., Ros, E.: A method for sparse disparity densification using voting mask propagation. J. Vis. Commun. Image Represent. 21(1), 67–74 (2010)CrossRefGoogle Scholar
  20. 20.
    Ralli, J., Pelayo, F., Diaz, J.: Increasing efficiency in disparity calculation. In: Mele, F., Ramella, G., Santillo, S., Ventriglia, F. (eds.) BVAI 2007. LNCS, vol. 4729, pp. 298–307. Springer Nature, Heidelberg (2007). doi: 10.1007/978-3-540-75555-5_28 CrossRefGoogle Scholar
  21. 21.
    Salembier, P., Garrido, L.: Binary partition tree as an efficient representation for filtering, segmentation and information retrieval. In: ICIP 1998 (1998)Google Scholar
  22. 22.
    Scharstein, D., Hirschmüller, H., Kitajima, Y., Krathwohl, G., Nešić, N., Wang, X., Westling, P.: High-resolution stereo datasets with subpixel-accurate ground truth. In: Jiang, X., Hornegger, J., Koch, R. (eds.) GCPR 2014. LNCS, vol. 8753, pp. 31–42. Springer, Cham (2014). doi: 10.1007/978-3-319-11752-2_3 Google Scholar
  23. 23.
    Scharstein, D., Szeliski, D.: Stereo matching with non-linear diffusion. In: CVPR (1996)Google Scholar
  24. 24.
    Vachier, C., Meyer, F.: The viscous watershed transform. J. Math. Imaging Vis. 22(2–3), 251–267 (2005)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Weickert, J.: Anisotropic diffusion in image processing. Ph.D. thesis (1998)Google Scholar
  26. 26.
    Yang, Q., Wang, L., Yang, R., Stewenius, H., Nister, D.: Stereo matching with color-weighted correlation, hierarchical belief propagation and occlusion handling. In: CVPR (2006)Google Scholar
  27. 27.
    Zbontar, J., LeCun, Y.: Stereo matching by training a convolutional neural network to compare image patches. CoRR, abs/1510.05970 (2015)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Sébastien Drouyer
    • 1
    • 2
    Email author
  • Serge Beucher
    • 1
  • Michel Bilodeau
    • 1
  • Maxime Moreaud
    • 1
    • 2
  • Loïc Sorbier
    • 2
  1. 1.CMM - Centre de Morphologie MathématiqueMines ParisTech, PSL Research UniversityFontainebleauFrance
  2. 2.IFP Energies nouvellesSolaizeFrance

Personalised recommendations