Skip to main content
SpringerLink
Log in

Multi-view photometric stereo using surface deformation

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

This paper presents a hybrid approach for 3D reconstruction by fusing photometric stereo and multi-view stereo. The 3D surface is obtained by capturing a set of images taken from different viewpoints under time-varying illuminations. Key factors in the reconstruction process are surface normals that are obtained from photometric stereo. The surface is initialized by integrating the normals and then refined by performing iterative deformations on the initial surface and thereby optimizing image and normal consistency in multiple views. Benefiting from the employment of the deformation approach, we are able to perform image and normal consistency optimization without using matching windows. Instead, always the complete surface is back-projected. This makes the proposed approach much simpler and more robust compared to window-based approaches, which typically require global optimization with constraints on neighboring windows. Experiments on real-world data and ground-truth data show that for diffuse midsized objects without large depth discontinuities our approach improves the accuracy of the reconstructions compared to exiting approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Woodham, R.J.: Photometric method for determining surface orientation from multiple images. Opt. Eng. 19(1), 139–144 (1980)

    Article  Google Scholar 

  2. Alldrin, N., Zickler, T., Kriegman, D.: Photometric stereo with non-parametric and spatially-varying reflectance. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2008)

  3. Goldman, D. B., Curless, B., Hertzmann, A., Seitz, S. M.: Shape and spatially-varying BRDFs from photometric stereo. In: IEEE International Conference on Computer Vision (2005)

  4. Shi, B., Tan, P., Matsushita, Y., Ikeuchi, K.: Elevation angle from reflectance monotonicity: photometric stereo for general isotropic reflectances. In: Proceedings of European Conference on Computer Vision (ECCV) (2012)

    Chapter  Google Scholar 

  5. Xu, S., Wallace, A.M.: Recovering surface reflectance and multiple light locations and intensities from image data. Pattern Recognit. Lett. 29(11), 1639–1647 (2008)

    Article  Google Scholar 

  6. Chandraker, M., Kahl, F., Kriegman, D.J.: Reflections on the generalized bas-relief ambiguity. In: Proceedings of Computer Vision and Pattern Recognition, pp. 788–795 (2005)

  7. Chandraker, M., Agarwal, S., Kriegman, D.: ShadowCuts: photometric stereo with shadows. In: Proceedings of Computer Vision and Pattern Recognition (2007)

  8. Sunkavalli, K., Zickler, T., Pfister, H.: Visibility subspaces: uncalibrated photometric stereo with shadows. In: European Conference on Computer Vision, Part II, 251–264 (2010)

    Chapter  Google Scholar 

  9. Okabe, T., Sato, I., Sato, Y.: Attached shadow coding: estimating surface normals from shadows under unknown reflectance and lighting conditions. In: Proceedings of the International Conference on Computer Vision, pp. 1693–1700 (2009)

  10. Basri, R., Jacobs, D.: Photometric stereo with general, unknown lighting. In: Proceednigs of the Computer Vision and Pattern Recognition Conference, pp. II-374-II-381 (2001)

  11. Favaro, P., Papadhimitri, T.: A closed-form solution to uncalibrated photometric stereo via diffuse maxima. In: Proceedings of the CVPR, pp. 821–828. IEEE (2012)

  12. Nayar, S. K., Ikeuchi, K., Kanade, T.: Determining shape and reflectance of Lambertian, specular, and hybrid surfaces using extended sources. In: International Workshop on Industrial Applications of Machine Intelligence and Vision, pp. 169–175 (1989)

  13. Aittala, M., Weyrich, T., Lehtinen, J.: Practical SVBRDF capture in the frequency domain. ACM Trans. Graph. 32(4), 110:1–110:12 (2013)

    Article  Google Scholar 

  14. Shi, B., Tan, P., Matsushita, Y., Ikeuchi, K.: A biquadratic reflectance model for radiometric image analysis. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2012)

  15. Birchfield, S., Tomasi, C.: A pixel dissimilarity measure that is insensitive to image sampling. IEEE Trans. Pattern Anal. Mach. Intell. 20(4), 401–406 (1998)

    Article  Google Scholar 

  16. Zabih, R., Woodfill, J.: Non-parametric local transforms for computing visual correspondence. In: Third European Conference on Computer Vision (ECCV’94), pp. 151–158, Springer, Stockholm (1994)

  17. Hirschmüller, H., Scharstein, D.: Evaluation of stereo matching costs on images with radiometric differences. TPAMI 31(9), 1582–1599 (2009)

    Article  Google Scholar 

  18. Mei, X., Sun, X., Zhou, M., Jiao, S., Wang, H., Zhang, X.: On building an accurate stereo matching system on graphics hardware. In: GPUCV (2011)

  19. Furukawa, Y., Ponce, J.: Accurate, dense, and robust multi-view stereopsis. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2007)

  20. Zheng, E., Jojic, V., Dunn, E., Frahm, J.-M.: Patchmatch based joint view selection and depthmap estimation. In: Proceedings of CVPR (2014)

  21. Anderson, R., Stenger, B., Cipolla, R.: Color photometric stereo for multicolored surfaces. In: IEEE International Conference on Computer Vision (2011)

  22. Park, J., Sinha, S.N., Matsushita, Y., Tai, Y.W., Kweon, I.S.: Multiview photometric stereo using planar mesh parameterization. In: Proceedings of International Conference on Computer Vision (ICCV) (2013)

  23. Zhang, Q., Ye, M., Yang, R., Matsushita, Y., Wilburn, B., Yu, H.: Edge-preserving photometric stereo via depth fusion. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2472–2479 (2012)

  24. Wu, C., Liu, Y., Dai, Q., Wilburn, B.: Fusing multiview and photometric stereo for 3d reconstruction under uncalibrated illumination. TVCG 17, 1082–1095 (2011)

    Google Scholar 

  25. Shi, B., Inose, K., Matsushita, Y., Tan, P., Yeung, S.-K., Ikeuchi, K.: Photometric stereo using internet images. In: International Conference on 3D Vision (3DV) (2014)

  26. Horovitz, I., Kiryati, N.: Bias correction in photometric stereo using control points. Technical Report, Department of Electrical Engineering–Systems, Tel Aviv University, Israel (2000)

  27. Nehab, D., Rusinkiewicz, S., Davis, J., Ramamoorthi, R.: Efficiently combining positions and normals for precise 3d geometry. In: Proceedings of ACM SIGGRAPH, pp. 536–543 (2005)

  28. Hernandez, C., Vogiatzis, G., Brostow, G.J., Stenger, B., Cipolla, R.: Multiview photometric stereo. IEEE Trans. Pattern Anal. Mach. Intell. 30(3), 548–554 (2008)

    Article  Google Scholar 

  29. Grochulla, M., Thormählen, T.: Combining photometric normals and multi-view stereo for 3D reconstruction. In: 12th European Conference on Visual Media Production (CVMP) (2015)

  30. Vlasic, D., Peers, P., Baran, I., Debevec, P., Popović, J., Rusinkiewicz, S., Matusik, W.: Dynamic shape capture using multiview photometric stereo. ACM Trans. Graph. 28(5), 174 (2009)

    Article  Google Scholar 

  31. Grochulla, M., Thormählen, T., Seidel, H. P.: Using spatially distributed patterns for multiple view camera calibration. In: Proceedings of MIRAGE, pp. 110–121 (2001)

    Chapter  Google Scholar 

  32. Ma, W.-C., Hawkins, T., Peers, P., Chabert, C., Weiss, M., Debevec, P.: Rapid acquisition of specular and diffuse normal maps from polarized spherical gradient illumination. In: Rendering, Techniques, pp. 183–194 (2007)

  33. Wilson, C.A., Ghosh, A., Peers, P., Chiang, J.Y., Busch, J., Debevec, P.: Temporal upsampling of performance geometry using photometric alignment. ACM Trans. Graph. 29(2), 17 (2010)

    Article  Google Scholar 

  34. Wu, C.: VisualSFM: A Visual Structure from Motion System. http://ccwu.me/vsfm/ (2011)

  35. Wu, C., Agarwal, S., Curless, B., Seitz, S. M.: Multicore Bundle Adjustment. In: CVPR (2011)

  36. Nehab, D.: Mesh opt: Combining positions and normals. http://w3.impa.br/~diego/software/NehEtAl05/ (2006)

Download references

Author information

Authors and Affiliations

  1. University of Marburg, Marburg, Germany

    Jiangbin Gan, Alwin Wilbert & Thorsten Thormählen

  2. Beiersdorf AG, Hamburg, Germany

    Philip Drescher & Ralf Hagens

Authors
  1. Jiangbin Gan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alwin Wilbert
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thorsten Thormählen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Philip Drescher
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ralf Hagens
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiangbin Gan.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gan, J., Wilbert, A., Thormählen, T. et al. Multi-view photometric stereo using surface deformation. Vis Comput 34, 1551–1561 (2018). https://doi.org/10.1007/s00371-017-1430-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-017-1430-5

Keywords

Search

Navigation