A Convex Solution to Disparity Estimation from Light Fields via the Primal-Dual Method

  • Mahdad Hosseini Kamal
  • Paolo Favaro
  • Pierre Vandergheynst
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8932)


We present a novel approach to the reconstruction of depth from light field data. Our method uses dictionary representations and group sparsity constraints to derive a convex formulation. Although our solution results in an increase of the problem dimensionality, we keep numerical complexity at bay by restricting the space of solutions and by exploiting an efficient Primal-Dual formulation. Comparisons with state of the art techniques, on both synthetic and real data, show promising performances.


Light fields multi-view stereo primal-dual formulation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
  2. 2.
    Ayvaci, A., Raptis, M., Soatto, S.: Sparse occlusion detection with optical flow. IJCV (2012)Google Scholar
  3. 3.
    Basha, T., Avidan, S., Hornung, A., Matusik, W.: Structure and motion from scene registration. In: CVPR. IEEE (2012)Google Scholar
  4. 4.
    Bishop, T., Favaro, P.: The light field camera: extended depth of field, aliasing and superresolution. PAMI (2012)Google Scholar
  5. 5.
    Bolles, R.C., Baker, H.H., Marimont, D.H.: Epipolar-plane image analysis: An approach to determining structure from motion. IJCV (1987)Google Scholar
  6. 6.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. PAMI (2001)Google Scholar
  7. 7.
    Combettes, P.L., Pesquet, J.C.: Proximal Splitting Methods in Signal Processing. In: Fixed-Point Alg. for Inv. Prob. in Science and Eng. (2011)Google Scholar
  8. 8.
    Donatsch, D., Bigdeli, S.A., Robert, P., Zwicker, M.: Hand-held 3d light field photography and applications. The Visual Computer (2014)Google Scholar
  9. 9.
    Elhamifar, E., Vidal, R.: Sparse subspace clustering. In: CVPR. IEEE (2009)Google Scholar
  10. 10.
    Faugeras, O., Keriven, R.: Variational principles, surface evolution, PDE’s, level set methods and the stereo problem. IEEE (2002)Google Scholar
  11. 11.
    Fitzgibbon, A.W., Wexler, Y., Zisserman, A., et al.: Image-based rendering using image-based priors. In: ICCV, vol. 3, pp. 1176–1183 (2003)Google Scholar
  12. 12.
    Goldluecke, B., Cremers, D.: An approach to vectorial total variation based on geometric measure theory. In: CVPR (2010)Google Scholar
  13. 13.
    Goldluecke, B., Magnor, M.A.: Joint 3d-reconstruction and background separation in multiple views using graph cuts. In: CVPR. IEEE (2003)Google Scholar
  14. 14.
    Heber, S., Ranftl, R., Pock, T.: Variational shape from light field. In: Heyden, A., Kahl, F., Olsson, C., Oskarsson, M., Tai, X.-C. (eds.) EMMCVPR 2013. LNCS, vol. 8081, pp. 66–79. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  15. 15.
    Hosseini Kamal, M., Favaro, P., Vandergheynst, P.: A Convex Solution to Disparity Estimation from Light Fields via the Primal-Dual Method. oai:infoscience.epfl.ch:202076 (2014)Google Scholar
  16. 16.
    Humayun, A., Mac Aodha, O., Brostow, G.J.: Learning to find occlusion regions. In: CVPR. IEEE (2011)Google Scholar
  17. 17.
    Kang, S.B., Szeliski, R.: Extracting view-dependent depth maps from a collection of images. IJCV 58(2), 139–163 (2004)CrossRefGoogle Scholar
  18. 18.
    Kang, S.B., Szeliski, R., Chai, J.: Handling occlusions in dense multi-view stereo. In: CVPR. IEEE (2001)Google Scholar
  19. 19.
    Kim, C., Zimmer, H., Pritch, Y., Sorkine-Hornung, A., Gross, M.: Scene reconstruction from high spatio-angular resolution light fields. In: SIGGRAPH (2013)Google Scholar
  20. 20.
    Kutulakos, K.N., Seitz, S.M.: A theory of shape by space carving. IJCV (2000)Google Scholar
  21. 21.
    Liu, C., Yuen, J., Torralba, A., Sivic, J., Freeman, W.T.: SIFT flow: Dense correspondence across different scenes. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 28–42. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  22. 22.
    Liu, G., Lin, Z., Yu, Y.: Robust subspace segmentation by low-rank representation. In: ICML (2010)Google Scholar
  23. 23.
    Ng, R., Levoy, M., Brédif, M., Duval, G., Horowitz, M., Hanrahan, P.: Light field photography with a hand-held plenoptic camera. CSTR (2005)Google Scholar
  24. 24.
    Pock, T., Chambolle, A.: Diagonal preconditioning for first order primal-dual algorithms in convex optimization. In: ICCV, pp. 1762–1769 (2011)Google Scholar
  25. 25.
    Pock, T., Cremers, D., Bischof, H., Chambolle, A.: Global solutions of variational models with convex regularization. SIAM J. on Imag. Sciences (2010)Google Scholar
  26. 26.
    Pock, T., Schoenemann, T., Graber, G., Bischof, H., Cremers, D.: A convex formulation of continuous multi-label problems. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 792–805. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  27. 27.
    Sun, X., Mei, X., Zhou, M., Wang, H., et al.: Stereo matching with reliable disparity propagation. In: 3DIMPVT. IEEE (2011)Google Scholar
  28. 28.
    Szeliski, R., Scharstein, D.: Symmetric sub-pixel stereo matching. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002, Part II. LNCS, vol. 2351, pp. 525–540. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  29. 29.
    Tseng, P.: Convergence of a block coordinate descent method for nondifferentiable minimization. J. Optim. Theory Appl. (2001)Google Scholar
  30. 30.
    Vaish, V., Wilburn, B., Joshi, N., Levoy, M.: Using plane+ parallax for calibrating dense camera arrays. In: CVPR. IEEE (2004)Google Scholar
  31. 31.
    Wanner, S., Meister, S., Goldluecke, B.: Datasets and benchmarks for densely sampled 4d light fields. In: Vision, Modelling and Visualization, (VMV) (2013)Google Scholar
  32. 32.
    Wanner, S., Goldluecke, B.: Globally consistent depth labeling of 4d light fields. In: CVPR. IEEE (2012)Google Scholar
  33. 33.
    Wright, J., Yang, A.Y., Ganesh, A., Sastry, S.S., Ma, Y.: Robust face recognition via sparse representation. PAMI 31(2), 210–227 (2009)CrossRefGoogle Scholar
  34. 34.
    Yuan, M., Lin, Y.: Model selection and estimation in regression with grouped variables. J. of the Royal StatistSociety: Series B (Stat. Meth.) (2006)Google Scholar
  35. 35.
    Zach, C., Pock, T., Bischof, H.: A globally optimal algorithm for robust TV − ℓ1 range image integration. In: ICCV, pp. 1–8. IEEE (2007)Google Scholar
  36. 36.
    Ziegler, R., Bucheli, S., Ahrenberg, L., Magnor, M., Gross, M.: A bidirectional light field-hologram transform. In: Computer Graphics Forum., vol. 26, pp. 435–446. Wiley Online Library (2007)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Mahdad Hosseini Kamal
    • 1
  • Paolo Favaro
    • 2
  • Pierre Vandergheynst
    • 1
  1. 1.Ecole Polytechnique Fédérale de LausanneLausanneSwitzerland
  2. 2.University of BernBernSwitzerland

Personalised recommendations