Skip to main content

Practical BRDF reconstruction using reliable geometric regions from multi-view stereo

Abstract

In this paper, we present a practical method for reconstructing the bidirectional reflectance distribution function (BRDF) from multiple images of a real object composed of a homogeneous material. The key idea is that the BRDF can be sampled after geometry estimation using multi-view stereo (MVS) techniques. Our contribution is selection of reliable samples of lighting, surface normal, and viewing directions for robustness against estimation errors of MVS. Our method is quantitatively evaluated using synthesized images and its effectiveness is shown via real-world experiments.

References

  1. Ullman, S. The interpretation of structure from motion. Proceedings of the Royal Society of London Series b Biological Sciences Vol. 203, No. 1153, 405–426, 1979.

    Article  Google Scholar 

  2. Lu, F.; Matsushita, Y.; Sato, I.; Okabe, T.; Sato, Y. From intensity profile to surface normal: Photometric stereo for unknown light sources and isotropic reflectances. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 37, No. 10, 1999–2012, 2015.

    Article  Google Scholar 

  3. Nielsen, J. B.; Jensen, H. W.; Ramamoorthi, R. On optimal, minimal BRDF sampling for reflectance acquisition. ACM Transactions on Graphics Vol. 34, No. 6, Article No. 186, 2015.

    Google Scholar 

  4. Seitz, S. M.; Dyer, C. R. Photorealistic scene reconstruction by voxel coloring. International Journal of Computer Vision, Vol. 35, No. 2, 151–173, 1999.

    Article  Google Scholar 

  5. Kutulakos, K. N.; Seitz, S. M. A theory of shape by space carving. In: Proceedings of the 7th IEEE International Conference on Computer Vision, Vol. 1, 307–314, 1999.

    MATH  Google Scholar 

  6. Vogiatzis, G.; Torr, P. H. S.; Cipolla, R. Multi-view stereo via volumetric graph-cuts. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, 391–398, 2005.

    Google Scholar 

  7. Koenderink, J. J.; van Doorn, A. J. Photometric invariants related to solid shape. In: Shape from Shading. MIT Press, 301–321, 1989.

    Google Scholar 

  8. Zisserman, A.; Giblin, P.; Blake, A. The information available to a moving observer from specularities. Image and Vision Computing Vol. 7, No. 1, 38–42, 1989.

    Article  Google Scholar 

  9. Furukawa, Y.; Ponce, J. Accurate, dense, and robust multiview stereopsis. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 32, No. 8, 1362–1376, 2010.

    Article  Google Scholar 

  10. Treuille, A.; Hertzmann, A.; Seitz, S. M. Example-based stereo with general BRDFs. In: Computer Vision-ECCV 2004. Lecture Notes in Computer Science, Vol. 3022. Pajdla, T.; Matas, J. Eds. Springer Berlin Heidelberg, 457–469, 2004.

    Google Scholar 

  11. Chandraker, M.; Reddy, D.; Wang, Y. Z.; Ramamoorthi, R. What object motion reveals about shape with unknown BRDF and lighting. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2523–2530, 2013.

    Google Scholar 

  12. Chandraker, M. What camera motion reveals about shape with unknown BRDF. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2179–2186, 2014.

    Google Scholar 

  13. Li, Z. Q.; Xu, Z. X.; Ramamoorthi, R.; Chandraker, M. Robust energy minimization for BRDF-invariant shape from light fields. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 578–586, 2017.

    Google Scholar 

  14. Schwartz C.; Klein, R. Acquisition and presentation of virtual surrogates for cultural heritage artefacts. In: Proceedings of the EVA, 50–57, 2012.

    Google Scholar 

  15. Xia, R.; Dong, Y.; Peers, P.; Tong, X. Recovering shape and spatially-varying surface reflectance under unknown illumination. ACM Transactions on Graphics Vol. 35, No. 6, Article No. 187, 2016.

    Google Scholar 

  16. Oxholm, G.; Nishino, K. Shape and reflectance estimation in the wild. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 38, No. 2, 376–389, 2016.

    Article  Google Scholar 

  17. Erb, W. Computer-controlled gonioreflectometer for the measurement of spectral reflection characteristics. Applied Optics Vol. 19, No. 22, 3789–3794, 1980.

    Article  Google Scholar 

  18. Miyashita, L.; Watanabe, Y.; Ishikawa, M. Rapid SVBRDF measurement by algebraic solution based on adaptive illumination. In: Proceedings of the 2nd International Conference on 3D Vision, 232–239, 2014.

    Google Scholar 

  19. Marschner, S. R.; Westin, S. H.; Lafortune, E. P. F.; Torrance, K. E.; Greenberg, D. P. Image-based BRDF measurement including human skin. In: Proceedings of the Eurographics Workshop on Rendering, 131–144, 1999.

    Google Scholar 

  20. Matusik, W.; Pfister, H.; Brand, M.; McMillan, L. Efficient isotropic BRDF measurement. In: Proceedings of the 14th Eurographics Workshop on Rendering, 241–247, 2003.

    Google Scholar 

  21. Rusinkiewicz, S. M. A new change of variables for efficient BRDF representation. In: Proceedings of the Eurographics Workshop on Rendering, 11–22, 1998.

    Google Scholar 

  22. Higo, T.; Matsushita, Y.; Ikeuchi, K. Consensus photometric stereo. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1157–1164, 2010.

    Google Scholar 

  23. Vergne, R.; Pacanowski, R.; Barla, P.; Granier, X.; Schlick, C. Light warping for enhanced surface depiction. ACM Transactions on Graphics Vol. 28, No. 3, Article No. 25, 2009.

    Google Scholar 

  24. Pharr M.; Humphreys, G. Physically based Rendering: From Theory to Implementation, 2nd edn. Morgan Kaufmann Publishers Inc., 2010.

    Google Scholar 

Download references

Acknowledgements

This work was partly supported by JSPS KAKENHI JP15K16027, JP26700013, JP15H05918, JP19H04138, JST CREST JP179423, and the Foundation for Nara Institute of Science and Technology.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Hiroyuki Kubo.

Additional information

Taishi Ono received his M.E. degree in Nara Institute of Science and Technology (NAIST), Japan, in 2017. His research interests include computer graphics. He is an ACM member.

Hiroyuki Kubo has been an assistant professor at NAIST since 2014. His research interests include computer graphics and computer animation. He received his M.S. and Ph.D degrees from Waseda University, in 2008 and 2012, respectively. He is an ACM member.

Kenichiro Tanaka received his M.S. and Ph.D. degrees in computer science from Osaka University in 2014 and 2017, respectively. In April 2017, he joined NAIST as an assistant professor. His research interests include computer vision and computational photography, imaging, and illumination. He is a member of the IEEE and CVF.

Takuya Funatomi has been an associate professor at NAIST since 2015. He was an assistant professor at Kyoto University, Japan, from 2007 to 2015, and a visiting assistant professor at Stanford University, USA, in 2014. He received his Ph.D. degree in informatics from Kyoto University, Japan, in 2007. His research interests include computer vision, computer graphics, and pattern recognition. He is a member of the IEEE.

Yasuhiro Mukaigawa received his M.E. and Ph.D. degrees from the University of Tsukuba in 1994 and 1997, respectively. He became a research associate at Okayama University in 1997, an assistant professor at University of Tsukuba in 2003, an associate professor at Osaka University in 2004, and a professor at NAIST in 2014. His current research interests include photometric analysis and computational photography. He is a member of the IEEE.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.

The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ono, T., Kubo, H., Tanaka, K. et al. Practical BRDF reconstruction using reliable geometric regions from multi-view stereo. Comp. Visual Media 5, 325–336 (2019). https://doi.org/10.1007/s41095-019-0150-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41095-019-0150-3

Keywords

  • BRDF reconstruction
  • multi-view stereo (MVS)
  • photogrammetry
  • rendering