Beyond Multi-view Stereo: Shading-Reflectance Decomposition

  • Jean MélouEmail author
  • Yvain Quéau
  • Jean-Denis Durou
  • Fabien Castan
  • Daniel Cremers
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10302)


We introduce a variational framework for separating shading and reflectance from a series of images acquired under different angles, when the geometry has already been estimated by multi-view stereo. Our formulation uses an \(l^1\)-TV variational framework, where a robust photometric-based data term enforces adequation to the images, total variation ensures piecewise-smoothness of the reflectance, and an additional multi-view consistency term is introduced for resolving the arising ambiguities. Optimisation is carried out using an alternating optimisation strategy building upon iteratively reweighted least-squares. Preliminary results on both a synthetic dataset, using various lighting and reflectance scenarios, and a real dataset, confirm the potential of the proposed approach.


Reflectance Multi-view Shading Variational methods 


  1. 1.
    Agarwal, S., Snavely, N., Simon, I., Seitz, S.M., Szeliski, R.: Building Rome in a day. In: Proceedings of ICCV (2009)Google Scholar
  2. 2.
    Aujol, J.F., Gilboa, G., Chan, T., Osher, S.: Structure-texture image decomposition - modeling, algorithms, and parameter selection. Int. J. Comput. Vis. 67(1), 111–136 (2006)CrossRefzbMATHGoogle Scholar
  3. 3.
    Basri, R., Jacobs, D., Kemelmacher, I.: Photometric stereo with general, unknown lighting. Int. J. Comput. Vis. 72(3), 239–257 (2007)CrossRefGoogle Scholar
  4. 4.
    Horn, B.K.P.: Shape from shading: a method for obtaining the shape of a smooth opaque object from one view. Ph.D. thesis, Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology (1970)Google Scholar
  5. 5.
    Jin, H., Cremers, D., Wang, D., Yezzi, A., Prados, E., Soatto, S.: 3-D reconstruction of shaded objects from multiple images under unknown illumination. Int. J. Comput. Vis. 76(3), 245–256 (2008)CrossRefGoogle Scholar
  6. 6.
    Kim, K., Torii, A., Okutomi, M.: Multi-view inverse rendering under arbitrary illumination and albedo. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 750–767. Springer, Cham (2016). doi: 10.1007/978-3-319-46487-9_46 CrossRefGoogle Scholar
  7. 7.
    Langguth, F., Sunkavalli, K., Hadap, S., Goesele, M.: Shading-aware multi-view stereo. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 469–485. Springer, Cham (2016). doi: 10.1007/978-3-319-46487-9_29 CrossRefGoogle Scholar
  8. 8.
    Le Guen, V.: Cartoon + Texture image decomposition by the TV-L1 model. Image Process. On Line 4, 204–219 (2014).
  9. 9.
    Maurer, D., Ju, Y.C., Breuß, M., Bruhn, A.: Combining shape from shading and stereo: a variational approach for the joint estimation of depth, illumination and albedo. In: Proceedings of BMVC (2016)Google Scholar
  10. 10.
    Moulon, P., Monasse, P., Marlet, R.: openMVG: an open multiple view geometry library.
  11. 11.
    Ramamoorthi, R., Hanrahan, P.: An efficient representation for irradiance environment maps. In: Proceedings of SIGGRAPH (2001)Google Scholar
  12. 12.
    Seitz, S.M., Curless, B., Diebel, J., Scharstein, D., Szeliski, R.: A comparison and evaluation of multi-view stereo reconstruction algorithms. In: Proceedings of CVPR (2006)Google Scholar
  13. 13.
    Strecha, C., Von Hansen, W., Van Gool, L.J., Fua, P., Thoennessen, U.: On benchmarking camera calibration and multi-view stereo for high resolution imagery. In: Proceedings of CVPR (2008)Google Scholar
  14. 14.
    Woodham, R.J.: Photometric method for determining surface orientation from multiple images. Opt. Eng. 19(1), 139–144 (1980)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Jean Mélou
    • 1
    • 3
    Email author
  • Yvain Quéau
    • 2
  • Jean-Denis Durou
    • 1
  • Fabien Castan
    • 3
  • Daniel Cremers
    • 2
  1. 1.IRIT, UMR CNRS 5505, Université de ToulouseToulouseFrance
  2. 2.Department of InformaticsTechnical University MunichMunichGermany
  3. 3.Mikros ImageLevallois-PerretFrance

Personalised recommendations