Abstract
In this study, we present a new and innovative framework for acquiring high-quality SVBRDF maps. Our approach addresses the limitations of the current methods and proposes a new solution. The core of our method is a simple hardware setup consisting of a consumer-level camera, LED lights, and a carefully designed network that can accurately obtain the high-quality SVBRDF properties of a nearly planar object. By capturing a flexible number of images of an object, our network uses different subnetworks to train different property maps and employs appropriate loss functions for each of them. To further enhance the quality of the maps, we improved the network structure by adding a novel skip connection that connects the encoder and decoder with global features. Through extensive experimentation using both synthetic and real-world materials, our results demonstrate that our method outperforms previous methods and produces superior results. Furthermore, our proposed setup can also be used to acquire physically based rendering maps of special materials.
Article PDF
Similar content being viewed by others
References
Holroyd, M.; Lawrence, J.; Zickler, T. A coaxial optical scanner for synchronous acquisition of 3D geometry and surface reflectance. ACM Transactions on Graphics Vol. 29, No. 4, Article No. 99, 2010.
Dana, K. J.; van Ginneken, B.; Nayar, S. K.; Koenderink, J. J. Reflectance and texture of real-world surfaces. ACM Transactions on Graphics Vol. 18, No. 1, 1–34, 1999.
Lawrence, J.; Ben-Artzi, A.; DeCoro, C.; Matusik, W.; Pfister, H.; Ramamoorthi, R.; Rusinkiewicz, S. Inverse shade trees for non-parametric material representation and editing. ACM Transactions on Graphics Vol. 25, No. 3, 735–745, 2006.
Li, Z. Q.; Sunkavalli, K.; Chandraker, M. Materials for masses: SVBRDF acquisition with a single mobile phone image. In: Computer Vision–ECCV 2018. Lecture Notes in Computer Science, Vol. 11207. Ferrari, V.; Hebert, M.; Sminchisescu, C.; Weiss, Y. Eds. Springer Cham, 74–90, 2018.
Deschaintre, V.; Aittala, M.; Durand, F.; Drettakis, G.; Bousseau, A. Single-image SVBRDF capture with a rendering-aware deep network. ACM Transactions on Graphics Vol. 37, No. 4, Article No. 128, 2018.
Deschaintre, V.; Aittala, M.; Durand, F.; Drettakis, G.; Bousseau, A. Flexible SVBRDF capture with a multi-image deep network. Computer Graphics Forum Vol. 38, No. 4, 1–13, 2019.
Asselin, L. P.; Laurendeau, D.; Lalonde, J. F. Deep SVBRDF estimation on real materials. In: Proceedings of the International Conference on 3D Vision, 1157–1166, 2020.
Guo, J.; Lai, S. C.; Tao, C. Z.; Cai, Y. L.; Wang, L.; Guo, Y. W.; Yan, L. Q. Highlight-aware two-stream network for single-image SVBRDF acquisition. ACM Transactions on Graphics Vol. 40, No. 4, Article No. 123, 2021.
Zhao, Y. Z.; Wang, B. B.; Xu, Y. N.; Zeng, Z.; Wang, L.; Holzschuch, N. Joint SVBRDF recovery and synthesis from a single image using an unsupervised generative adversarial network. In: Proceedings of the Eurographics Symposium on Rendering, 53–66, 2020.
Wen, T.; Wang, B. B.; Zhang, L.; Guo, J.; Holzschuch, N. SVBRDF recovery from a single image with highlights using a pre-trained generative adversarial network. Computer Graphics Forum Vol. 41, No. 6, 110–123, 2022.
Kang, K. Z.; Xie, C. H.; He, C. G.; Yi, M. Q.; Gu, M. Y.; Chen, Z. M.; Zhou, K.; Wu, H. Z. Learning efficient illumination multiplexing for joint capture of reflectance and shape. ACM Transactions on Graphics Vol. 38, No. 6, Article No. 165, 2019.
Zhou, X. L.; Kalantari, N. K. Adversarial single-image SVBRDF estimation with hybrid training. Computer Graphics Forum Vol. 40, No. 2, 315–325, 2021.
Guo, Y.; Smith, C.; Hasan, M.; Sunkavalli, K.; Zhao, S. MaterialGAN: Reflectance capture using a generative SVBRDF model. ACM Transactions on Graphics Vol. 39, No. 6, Article No. 254, 2020.
Gao, D.; Li, X.; Dong, Y.; Peers, P.; Xu, K.; Tong, X. Deep inverse rendering for high-resolution SVBRDF estimation from an arbitrary number of images. ACM Transactions on Graphics Vol. 38, No. 4, Article No. 134, 2019.
Ye, W. J.; Dong, Y.; Peers, P.; Guo, B. N. Deep reflectance scanning: Recovering spatially-varying material appearance from a flash-lit video sequence. Computer Graphics Forum Vol. 40, No. 6, 409–427, 2021.
Albert, R. A.; Chan, D. Y.; Goldman, D. B.; O’Brien, J. F. Approximate svBRDF estimation from mobile phone video. In: Proceedings of the Eurographics Symposium on Rendering: Experimental Ideas & Implementations, 11–22, 2018.
Baek, S. H.; Jeon, D. S.; Tong, X.; Kim, M. H. Simultaneous acquisition of polarimetric SVBRDF and normals. ACM Transactions on Graphics Vol. 37, No. 6, Article No. 268, 2018.
Deschaintre, V.; Lin, Y. M.; Ghosh, A. Deep polarization imaging for 3D shape and SVBRDF acquisition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 15562–15571, 2021.
Ma, X. H.; Kang, K. Z.; Zhu, R. S.; Wu, H. Z.; Zhou, K. Free-form scanning of non-planar appearance with neural trace photography. ACM Transactions on Graphics Vol. 40, No. 4, Article No. 124, 2021.
Tunwattanapong, B.; Fyffe, G.; Graham, P.; Busch, J.; Yu, X. M.; Ghosh, A.; Debevec, P. Acquiring reflectance and shape from continuous spherical harmonic illumination. ACM Transactions on Graphics Vol. 32, No. 4, Article No. 109, 2013.
Nam, G.; Lee, J. H.; Gutierrez, D.; Kim, M. H. Practical SVBRDF acquisition of 3D objects with unstructured flash photography. ACM Transactions on Graphics Vol. 37, No. 6, Article No. 267, 2018.
Xia, R.; Dong, Y.; Peers, P.; Tong, X. Recovering shape and spatially-varying surface reflectance under unknown illumination. ACM Transactions on Graphics Vol. 35, No. 6, Article No. 187, 2016.
Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015. Lecture Notes in Computer Science, Vol. 9351. Navab, N.; Hornegger, J.; Wells, W.; Frangi, A. Eds. Springer Cham, 234–241, 2015.
Zhou, Z. W.; Rahman Siddiquee, M. M.; Tajbakhsh, N.; Liang, J. M. UNet++: A nested U-net architecture for medical image segmentation. In: Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. DLMIA ML-CDS 2018. Lecture Notes in Computer Science, Vol. 11045. Stoyanov, D., et al. Eds. Springer Cham, 3–11, 2018.
Hu, J.; Shen, L.; Sun, G. Squeeze-and-excitation networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 7132–7141, 2018.
X-Rite: TAC7 appearance scanner. 2017. Available at https://www.xrite.com/categories/appearance/total-appearance-capture-ecosystem/tac7
Cycles: Open source production rendering. 2022. Available at https://www.cycles-renderer.org/
Kingma, D. P.; Ba, J. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
Smith, A. R.; Blinn, J. F. Blue screen matting. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, 259–268, 1996.
Acknowledgements
This study was supported by the Nature Science Fund of Guangdong Province (No. 2021A1515011849) and the Key Area Research and Development of Guangdong Province (No. 2022A0505050014).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
The authors have no competing interests to declare that are relevant to the content of this article.
Additional information
Electronic Supplementary Material
We have provided a video displaying our results. This video is available at https://github.com/hgjljx/Delving-High-quality-SVBRDF-Acquisition-a-New-Setup-and-Method.git.
Chuhua Xian is currently an associate professor with the School of Computer Science and Engineering at South China University of Technology. He received his Ph.D. degree in computer science from the State Key Lab of CAD&CG at Zhejiang University in 2012. He was a postdoctoral researcher at CUHK from 11/2013 to 05/2014 and 09/2015 to 04/2016. His research interests include computer graphics, computer vision, image processing, and geometric processing.
Jiaxin Li is currently a graduate student at the School of Computer Science and Engineering at South China University of Technology. Her research interests include computer graphics, vision, and inverse rendering.
Hao Wu is currently the leader of the AI Lab of Guangdong Shidi Intelligence Technology, Ltd. She got her Ph.D. degree in computer science from Hong Kong University in 2019. Her research interests include computer graphics, rendering, computer vision, and generative models.
Zisen Lin is currently the CEO of Guangdong Shidi Intelligence Technology, Ltd. His research interests include computer graphics and computer vision.
Guiqing Li is currently a full-time professor at the School of Computer Science and Engineering, South China University of Technology in Guangzhou, China. His research interests include computer graphics, and geometric and image processing.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.
The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.
About this article
Cite this article
Xian, C., Li, J., Wu, H. et al. Delving into high-quality SVBRDF acquisition: A new setup and method. Comp. Visual Media (2024). https://doi.org/10.1007/s41095-023-0352-6
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s41095-023-0352-6