Journal of Mathematical Imaging and Vision

, Volume 60, Issue 9, pp 1527–1546 | Cite as

Variational Reflectance Estimation from Multi-view Images

  • Jean Mélou
  • Yvain QuéauEmail author
  • Jean-Denis Durou
  • Fabien Castan
  • Daniel Cremers


We tackle the problem of reflectance estimation from a set of multi-view images, assuming known geometry. The approach we put forward turns the input images into reflectance maps, through a robust variational method. The variational model comprises an image-driven fidelity term and a term which enforces consistency of the reflectance estimates with respect to each view. If illumination is fixed across the views, then reflectance estimation remains under-constrained: A regularization term, which ensures piecewise-smoothness of the reflectance, is thus used. Reflectance is parameterized in the image domain, rather than on the surface, which makes the numerical solution much easier, by resorting to an alternating majorization–minimization approach. Experiments on both synthetic and real-world datasets are carried out to validate the proposed strategy.


Reflectance Multi-view Shading Variational methods 



Yvain Quéau and Daniel Cremers were supported by the ERC Consolidator Grant “3D Reloaded.”


  1. 1.
    Adelson, E.H., Pentland, A.P.: Perception as Bayesian inference, chap. The perception of shading and reflectance, pp. 409–423. Cambridge University Press (1996)Google Scholar
  2. 2.
    Agarwal, S., Snavely, N., Simon, I., Seitz, S.M., Szeliski, R.: Building Rome in a Day. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 72–79 (2009)Google Scholar
  3. 3.
    Aujol, J.F., Gilboa, G., Chan, T., Osher, S.: Structure-texture image decomposition–modeling, algorithms, and parameter selection. Int. J. Comput. Vis. 67(1), 111–136 (2006)CrossRefGoogle Scholar
  4. 4.
    Barron, J., Malik, J.: Shape, illumination, and reflectance from shading. IEEE Trans. Pattern Anal. Mach. Intell. 37(8), 1670–1687 (2015)CrossRefGoogle Scholar
  5. 5.
    Basri, R., Jacobs, D., Kemelmacher, I.: Photometric stereo with general, unknown lighting. Int. J. Comput. Vis. 72(3), 239–257 (2007)CrossRefGoogle Scholar
  6. 6.
    Basri, R., Jacobs, D.P.: Lambertian reflectances and linear subspaces. IEEE Trans. Pattern Anal. Mach. Intell. 25(2), 218–233 (2003)CrossRefGoogle Scholar
  7. 7.
    Bell, S., Bala, K., Snavely, N.: Intrinsic images in the wild. ACM Trans. Gr. 33(4), 159:1–159:12 (2014)CrossRefGoogle Scholar
  8. 8.
    Bousseau, A., Paris, S., Durand, F.: User assisted intrinsic images. ACM Trans. Gr. 28(5), 130:1–130:10 (2009)CrossRefGoogle Scholar
  9. 9.
    Chen, Q., Koltun, V.: A simple model for intrinsic image decomposition with depth cues. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 241–248 (2013)Google Scholar
  10. 10.
    Cho, D., Matsushita, Y., Tai, Y.W., Kweon, I.S.: Photometric stereo under non-uniform light intensities and exposures. In: Proceedings of the European Conference on Computer Vision. pp. 170–186 (2016)CrossRefGoogle Scholar
  11. 11.
    Frolova, D., Simakov, D., Basri, R.: Accuracy of spherical harmonic approximations for images of Lambertian objects under far and near lighting. In: Proceedings of the European Conference on Computer Vision. pp. 574–587 (2004)Google Scholar
  12. 12.
    Furukawa, Y., Hernández, C., et al.: Multi-view stereo: a tutorial. Found. Trends Comput. Gr. Vis. 9(1–2), 1–148 (2015)Google Scholar
  13. 13.
    Garces, E., Munoz, A., Lopez-Moreno, J., Gutierrez, D.: Intrinsic images by clustering. Comput. Gr. Forum 31(4), 1415–1424 (2012)CrossRefGoogle Scholar
  14. 14.
    Gehler, P., Rother, C., Kiefel, M., Zhang, L., Schölkopf, B.: Recovering Intrinsic Images with a Global Sparsity Prior on Reflectance. In: Advances in Neural Information Processing Systems. pp. 765–773 (2011)Google Scholar
  15. 15.
    Golub, G.H., Van Loan, C.F.: Matrix Computations, 4th edn. Johns Hopkins University Press, Baltimore (2013)zbMATHGoogle Scholar
  16. 16.
    Horn, B.K.P.: Shape From Shading: A Method for Obtaining the Shape of a Smooth Opaque Object From One View. Ph.D. thesis, Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology (1970)Google Scholar
  17. 17.
    Jin, H., Cremers, D., Wang, D., Yezzi, A., Prados, E., Soatto, S.: 3-D reconstruction of shaded objects from multiple images under unknown illumination. Int. J. Comput. Vis. 76(3), 245–256 (2008)CrossRefGoogle Scholar
  18. 18.
    Kim, K., Torii, A., Okutomi, M.: Multi-view Inverse Rendering Under Arbitrary Illumination and Albedo. In: Proceedings of the European Conference on Computer Vision. pp. 750–767 (2016)CrossRefGoogle Scholar
  19. 19.
    Laffont, P.Y., Bousseau, A., Drettakis, G.: Rich intrinsic image decomposition of outdoor scenes from multiple views. IEEE Trans. Vis. Comput. Gr. 19(2), 210–224 (2013)CrossRefGoogle Scholar
  20. 20.
    Laffont, P.Y., Bousseau, A., Paris, S., Durand, F., Drettakis, G.: Coherent intrinsic images from photo collections. ACM Trans. Gr. 31, 202:1–202:11 (2012)CrossRefGoogle Scholar
  21. 21.
    Land, E.H., McCann, J.J.: Lightness and retinex theory. J. Opt. Soc. Am. 61, 1–11 (1971)CrossRefGoogle Scholar
  22. 22.
    Langguth, F., Sunkavalli, K., Hadap, S., Goesele, M.: Shading-aware Multi-view Stereo. In: Proceedings of the European Conference on Computer Vision. pp. 469–485 (2016)CrossRefGoogle Scholar
  23. 23.
    Le Guen, V.: Cartoon + Texture image decomposition by the TV-L1 model. Image Process. Line 4, 204–219 (2014)CrossRefGoogle Scholar
  24. 24.
    Maier, R., Kim, K., Cremers, D., Kautz, J., Nießner, M.: Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting. In: Proceedings of the IEEE International Conference on Computer Vision (2017)Google Scholar
  25. 25.
    Maurer, D., Ju, Y.C., Breuß, M., Bruhn, A.: Combining shape from shading and stereo: a variational approach for the joint estimation of depth, illumination and Albedo. In: Proceedings of the British Machine Vision Conference (2016)Google Scholar
  26. 26.
    Mélou, J., Quéau, Y., Durou, J.D., Castan, F., Cremers, D.: Beyond Multi-view stereo: shading-reflectance decomposition. In: Proceedings of the International Conference on Scale Space and Variational Methods in Computer Vision. pp. 694–705 (2017)Google Scholar
  27. 27.
    Moulon, P., Monasse, P., Marlet, R.: openMVG: An open multiple view geometry library. (2014). Accessed 16 Mar 2018
  28. 28.
    Mumford, D.: Bayesian rationale for the variational formulation. In: Geometry-Driven Diffusion in Computer Vision, pp. 135–146. Springer (1994)Google Scholar
  29. 29.
    Nadian-Ghomsheh, A., Hassanian, Y., Navi, K.: Intrinsic image decomposition via structure-preserving image smoothing and material recognition. PLoS ONE 11(12), 1–22 (2016)CrossRefGoogle Scholar
  30. 30.
    Park, J., Sinha, S.N., Matsushita, Y., Tai, Y.W., Kweon, I.S.: Robust multiview photometric stereo using planar mesh parameterization. IEEE Trans. Pattern Anal. Mach. Intell. 39(8), 1591–1604 (2017)CrossRefGoogle Scholar
  31. 31.
    Quéau, Y., Durix, B., Wu, T., Cremers, D., Lauze, F., Durou, J.D.: LED-based Photometric Stereo: Modeling, Calibration and Numerical Solution. J. Math. Imaging Vis. (2018), (to appear)Google Scholar
  32. 32.
    Quéau, Y., Pizenberg, M., Durou, J.D., Cremers, D.: Microgeometry capture and RGB albedo estimation by photometric stereo without demosaicing. In: Proceedings of the International Conference on Quality Control by Artificial Vision (2017)Google Scholar
  33. 33.
    Ramamoorthi, R., Hanrahan, P.: An Efficient Representation for Irradiance Environment Maps. In: Proceedings of the Annual Conference on Computer Graphics and Interactive Techniques. pp. 497–500 (2001)Google Scholar
  34. 34.
    Reinhard, E., Ward, G., Pattanaik, S., Debevec, P.: High Dynamic Range Imaging. Morgan Kaufmann, Los Altos (2005)Google Scholar
  35. 35.
    Seitz, S.M., Curless, B., Diebel, J., Scharstein, D., Szeliski, R.: A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 519–528 (2006)Google Scholar
  36. 36.
    Shen, L., Yeo, C.: Intrinsic images decomposition using a local and global sparse representation of reflectance. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 697–704 (2011)Google Scholar
  37. 37.
    Song, J., Cho, H., Yoon, J., Yoon, S.M.: Structure adaptive total variation minimization-based image decomposition. IEEE Trans. Circuits Syst. Video Technol. (2017), (to appear)Google Scholar
  38. 38.
    Storath, M., Weinmann, A.: Fast partitioning of vector-valued images. SIAM J. Imaging Sci. 7(3), 1826–1852 (2014)MathSciNetCrossRefGoogle Scholar
  39. 39.
    Unity-Technologies: Delightingtool, Accessed 16 Mar 2018
  40. 40.
    Woodham, R.J.: Photometric method for determining surface orientation from multiple images. Opt. Eng. 19(1), 139–144 (1980)CrossRefGoogle Scholar
  41. 41.
    Wu, C., Wilburn, B., Matsushita, Y., Theobalt, C.: High-quality shape from multiview stereo and shading under general illumination. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 969–976 (2011)Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.IRIT, UMR CNRS 5505, Université de ToulouseToulouseFrance
  2. 2.Mikros ImageLevallois-PerretFrance
  3. 3.Department of Computer ScienceTechnical University of MunichGarchingGermany

Personalised recommendations