A 3D Imaging Framework Based on High-Resolution Photometric-Stereo and Low-Resolution Depth

  • Zheng LuEmail author
  • Yu-Wing Tai
  • Fanbo Deng
  • Moshe Ben-Ezra
  • Michael S. Brown


This paper introduces a 3D imaging framework that combines high-resolution photometric stereo and low-resolution depth. Our approach targets imaging scenarios based on either macro-lens photography combined with focal stacking or a large-format camera that are able to image objects with more than 600 samples per mm\(^2\). These imaging techniques allow photometric stereo algorithms to obtain surface normals at resolutions that far surpass corresponding depth values obtained with traditional approaches such as structured-light, passive stereo, or depth-from-focus. Our work offers two contributions for 3D imaging based on these scenarios. The first is a multi-resolution, patched-based surface reconstruction scheme that can robustly handle the significant resolution difference between our surface normals and depth samples. The second is a method to improve the initial normal estimation by using all the available focal information for images obtained using a focal stacking technique.


3D Reconstruction High resolution Photometric stereo Focal stack 


  1. Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., et al. (2004). Interactive digital photomontage. ACM Transactions on Graphics (SIGGRAPH), 23(3), 294–302.Google Scholar
  2. Agrawal, A., Raskar, R.,& Chellappa, R. (2006). What is the range of surface reconstructions from a gradient field? In European conference on computer vision (ECCV). Graz, Austria: Springer.Google Scholar
  3. Agrawal, A., Xu, Y.,& Raskar, R. (2009). Invertible motion blur in video. ACM Transactions on Graphics (SIGGRAPH), 28(3), 1–8.CrossRefGoogle Scholar
  4. Anagramm and Digital Reproduction (1998). Accesed 1 August 2011.
  5. Banerjee, S., Sastry, P.,& Venkatesh, Y. (1992). Surface reconstruction from disparate shading: An integration of shape-from-shading and stereopsis. In 11th IAPR International conference on pattern recognition. The Hague, The Netherlands: IEEE Computer Society.Google Scholar
  6. Bernardini, F., Rushmeier, H., Martin, I. M., Mittleman, J.,& Taubin, G. (2002). Building a digital model of Michelangelo’s Florentine Pieta. IEEE Computer Graphics and Applications, 22(1), 59–67.Google Scholar
  7. Chen, C. Y., Klette, R.,& Chen, C. F. (2003). Shape from photometric stereo and contours. In Proceedings of computer analysis of images patterns (CAIP). Groningen, The Netherlands: Springer.Google Scholar
  8. Cignoni, P., Rocchini, C.,& Scopigno, R. (1998). Metro: Measuring error on simplified surfaces. Computer Graphics Forum, 17(2), 167–174.Google Scholar
  9. Darrell, T.,& Wohn, K. (1988). Pyramid based depth from focus. In Computer vision and pattern recognition (CVPR). Ann Arbor, MI: IEEE Computer Society.Google Scholar
  10. Fua, P.,& Leclerc, Y. G. (1994). Using 3-dimensional meshes to combine image-based and geometry-based constraints. In European conference on computer vision (ECCV). Stockholm, Sweden: Springer.Google Scholar
  11. Hausler, G. (1972). A method to increase the depth of focus by two step image processing. Optics Communications, 6(1), 38–42.CrossRefGoogle Scholar
  12. Hernández, C., Vogiatzis, G.,& Cipolla, R. (2008). Multi-view photometric stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(3), 548–554.Google Scholar
  13. Higo, T., Matsushita, Y., Joshi, N.,& Ikeuchi, K. (2009). A hand-held photometric stereo camera for 3D modeling. In Computer vision and pattern recognition (CVPR). Miami, FL: IEEE Computer Society.Google Scholar
  14. Horn, B.,& Brooks, M. (1989). Shape from shading. Cambridge: MIT Press.Google Scholar
  15. Ikeuchi, K. (1987). Determining a depth map using a dual photometric stereo. International Journal of Robotics Research, 6(1), 15–31.CrossRefGoogle Scholar
  16. Lange, H. (1999). Advances in the cooperation of shape from shading and stereo vision. In Proceedings 3DIM. Ottawa, Canada: IEEE Computer Society.Google Scholar
  17. Lu, Z., Tai, Y. W., Ben-Ezra, M.,& Brown, M. S. (2010). A framework for ultra high resolution 3D imaging. In Computer vision and pattern recognition (CVPR). San Francisco, CA: IEEE Computer Society.Google Scholar
  18. Malik, A. S.,& Choi, T. S. (2008). A novel algorithm for estimation of depth map using image focus for 3d shape recovery in the presence of noise. Pattern Recognition, 41(7), 2200–2225.zbMATHCrossRefGoogle Scholar
  19. Nayar, S. K.,& Nakagawa, Y. (1994). Shape from focus. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(8), 824–831.CrossRefGoogle Scholar
  20. Nayar, S. K., Fang, X. S.,& Boult, T. (1997). Separation of reflection components using color and polarization. International Journal of Computer Vision, 21(3), 163–186.Google Scholar
  21. Nehab, D., Rusinkiewicz, S., Davis, J.,& Ramamoorthi, R. (2005). Efficiently combining positions and normals for precise 3d geometry. ACM Transactions on Graphics (SIGGRAPH), 24(3), 536–543.Google Scholar
  22. Reid, J. K.,& Scott, J. A. (2009). An out-of-core sparse cholesky solver. ACM Transactions on Mathematical Software, 36(2), 1–33.MathSciNetCrossRefGoogle Scholar
  23. Scharstein, D.,& Szeliski, R. (2002). A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(1–3), 7–42.Google Scholar
  24. Scharstein, D.,& Szeliski, R. (2003). High-accuracy stereo depth maps using structured light. In Computer vision and pattern recognition (CVPR). Madison, WI: IEEE Computer Society.Google Scholar
  25. Seitz, S. M., Curless, B., Diebel, J., Scharstein, D.,& Szeliski, R. (2006). A comparison and evaluation of multi-view stereo reconstruction algorithms. In Computer vision and pattern recognition (CVPR). New York, NY: IEEE Computer Society. Google Scholar
  26. Terzopoulos, D. (1988). The computation of visible-surface representations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 10(4), 417–438.Google Scholar
  27. Vlasic, D., Peers, P., Baran, I., Debevec, P., Popovi’c, J., Rusinkiewicz, S., et al. (2009). Dynamic shape capture using multi-view photometric stereo. ACM Transactions on Graphics (SIGGRAPH-ASIA), 28(5), 1–11.Google Scholar
  28. Wholer, C. (2009). 3D computer vision: efficient methods and applications. New York: Springer.CrossRefGoogle Scholar
  29. Woodham, R. J. (1980). Photometric method for determining surface orientation from multiple images. Optical Engineering, 19(1), 139–144.CrossRefGoogle Scholar
  30. Wu, T. P.,& Tang, C. K. (2006). Visible surface reconstruction from normals with discontinuity consideration. In: Computer Vision and Pattern Recognition (CVPR).Google Scholar
  31. Wu, T. P., Sun, J., Tang, C. K.,& Shum, H. Y. (2008). Interactive normal reconstruction from a single image. ACM Transactions on Graphics (SIGGRAPH-ASIA), 27(5), 1–9.Google Scholar
  32. Xiong, Y.,& Shafer, S. (1993). Depth from focusing and defocusing. In Computer vision and pattern recognition (CVPR). New York, NY: IEEE Computer Society.Google Scholar

Copyright information

© Springer Science+Business Media New York 2012

Authors and Affiliations

  • Zheng Lu
    • 1
    • 3
    Email author
  • Yu-Wing Tai
    • 2
  • Fanbo Deng
    • 1
  • Moshe Ben-Ezra
    • 3
  • Michael S. Brown
    • 1
  1. 1.National University of SingaporeSingaporeSingapore
  2. 2.Department of Computer ScienceKorea Advanced Institute of Science and TechnologyDaejeonSouth Korea
  3. 3.Microsoft Research AsiaHaidian District, BeijingPeople’s Republic of China

Personalised recommendations