A Streamlined Photometric Stereo Framework for Cultural Heritage

  • Chia-Kai YehEmail author
  • Nathan Matsuda
  • Xiang Huang
  • Fengqiang Li
  • Marc Walton
  • Oliver Cossairt
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9913)


In this paper, we propose a streamlined framework of robust 3D acquisition for cultural heritage using both photometric stereo and photogrammetric information. An uncalibrated photometric stereo setup is augmented by a synchronized secondary witness camera co-located with a point light source. By recovering the witness camera’s position for each exposure with photogrammetry techniques, we estimate the precise 3D location of the light source relative to the photometric stereo camera. We have shown a significant improvement in both light source position estimation and normal map recovery compared to previous uncalibrated photometric stereo techniques. In addition, with the new configuration we propose, we benefit from improved surface shape recovery by jointly incorporating corrected photometric stereo surface normals and a sparse 3D point cloud from photogrammetry.


Photometric stereo Reflectance transformation imaging Near light position calibration Photogrammetry 3D surface shape reconstruction 



This project was undertaken at the Northwestern University/Art Institute of Chicago Center for Scientific Studies in the Arts (NU-ACCESS). NU-ACCESS is funded through a generous grant from the Andrew W. Mellon Foundation. Supplemental support is provided by the Materials Research Center, the Office of the Vice President for Research, the McCormick School of Engineering and Applied Science and the Department of Materials Science and Engineering at Northwestern University. Additionally, this work was supported in part by NSF CAREER grant IIS-1453192.


  1. 1.
    Cultural heritage imaging: reflectance transformation imaging (rti) (2013).
  2. 2.
    Ackermann, J., Goesele, M.: A survey of photometric stereo techniques. Found. Trends Comput. Graph. Vis. 9(3–4), 149–254 (2015). CrossRefzbMATHGoogle Scholar
  3. 3.
    Agisoft LLC: Agisoft PhotoScan.
  4. 4.
    Ahmed, N., Theobalt, C., Dobrev, P., Peter Seidel, H., Thrun, S.: Robust fusion of dynamic shape and normal capture for high-quality reconstruction of time-varying geometry. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008)Google Scholar
  5. 5.
    Alldrin, N., Zickler, T., Kriegman, D.: Photometric stereo with non-parametric and spatially-varying reflectance. In: 26th IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2008)Google Scholar
  6. 6.
    Belhumeur, P.N., Kriegman, D.J., Yuille, A.L.: The bas-relief ambiguity. IJCV 35(1), 33–44 (1999)CrossRefGoogle Scholar
  7. 7.
    Birkbeck, N., Cobzas, D., Sturm, P., Jagersand, M.: Variational shape and reflectance estimation under changing light and viewpoints. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006, Part I. LNCS, vol. 3951, pp. 536–549. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  8. 8.
    Elhabian, S.Y., Rara, H., Farag, A.A.: Towards accurate and efficient representation of image irradiance of convex-Lambertian objects under unknown near lighting. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1732–1737 (2011)Google Scholar
  9. 9.
    Esteban, C.H., Vogiatzis, G., Cipolla, R.: Multiview photometric stereo. IEEE Trans. Pattern Anal. Mach. Intell. 30(3), 548–554 (2008)CrossRefGoogle Scholar
  10. 10.
    Goldman, D.B., Curless, B., Hertzmann, A., Seitz, S.M.: Shape and spatially-varying BRDFs from photometric stereo. IEEE Trans. Pattern Anal. Mach. Intell. 32(6), 1060–1071 (2010)CrossRefGoogle Scholar
  11. 11.
    Hertzmann, A., Seitz, S.: Shape and materials by example: a photometric stereo approach. In: 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 1–8 (2003)Google Scholar
  12. 12.
    Horn, B.K.P.: Obtaining shape from shading information. In: The Psychology of Computer Vision, pp. 115–155 (1975)Google Scholar
  13. 13.
    Huang, X., Walton, M., Bearman, G., Cossairt, O.: Near light correction for image relighting and 3D shape recovery. In: Guidi, G., Scopigno, R., Brunet, P. (eds.) International Congress on Digital Heritage - Theme 2 - Computer Graphics and Interaction. IEEE (2015)Google Scholar
  14. 14.
    Ikehata, S., Wipf, D., Matsushita, Y., Aizawa, K.: Robust photometric stereo using sparse regression. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 318–325 (2012)Google Scholar
  15. 15.
    Malzbender, T., Gelb, D., Wolters, H.: Polynomial texture maps. In: Proceedings of SIGGRAPH 2001. Annual Conference Series, pp. 519–528. ACM Press, New York (2001)Google Scholar
  16. 16.
    Nehab, D., Rusinkiewicz, S., Davis, J., Ramamoorthi, R.: Efficiently combining positions and normals for precise 3d geometry. ACM Trans. Graph. 24(3), 536–543 (2005). CrossRefGoogle Scholar
  17. 17.
    Palma, G., Corsini, M., Cignoni, P., Scopigno, R., Mudge, M.: Dynamic shading enhancement for reflectance transformation imaging. J. Comput. Cult. Heritage 3(2), 1–20 (2010)CrossRefGoogle Scholar
  18. 18.
    Papadhimitri, T., Favaro, P.: A new perspective on uncalibrated photometric stereo. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1474–1481 (2013)Google Scholar
  19. 19.
    Papadhimitri, T., Favaro, P., Bern, U.: Uncalibrated near-light photometric stereo. In: Proceedings of the British Machine Vision Conference, pp. 1–12 (2014)Google Scholar
  20. 20.
    Quéau, Y., Durou, J.D.: Some illumination models for industrial applications of photometric stereo. In: QCAV (2015)Google Scholar
  21. 21.
    Sabzevari, R., Del Bue, A., Murino, V.: Structure from motion and photometric stereo for dense 3D shape recovery. In: Maino, G., Foresti, G.L. (eds.) ICIAP 2011, Part I. LNCS, vol. 6978, pp. 660–669. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  22. 22.
    Snavely, K.N.: Scene reconstruction and visualization from internet photo collections. Ph.D. thesis, Seattle, WA, USA (2009)Google Scholar
  23. 23.
    Snavely, N., Seitz, S.M., Szeliski, R.: Skeletal graphs for efficient structure from motion. In: Proceedings of Computer Vision and Pattern Recognition (2008)Google Scholar
  24. 24.
    Wetzler, A., Kimmel, R., Bruckstein, A.M., Mecca, R.: Close-range photometric stereo with point light sources. In: 2014 2nd International Conference on 3D Vision, pp. 115–122 (2014)Google Scholar
  25. 25.
    Wu, C., Liu, Y., Dai, Q., Member, S., Wilburn, B.: Fusing multiview and photometric stereo for 3d reconstruction under uncalibrated illumination. Trans. Vis. Comput. Graph. 17(8), 1082–1095 (2011)CrossRefGoogle Scholar
  26. 26.
    Wu, L., Ganesh, A., Shi, B., Matsushita, Y., Wang, Y., Ma, Y.: Robust photometric stereo via low-rank matrix completion and recovery. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010. LNCS, vol. 6494, pp. 703–717. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-19318-7_55 CrossRefGoogle Scholar
  27. 27.
    Zhang, M.: Robust surface normal estimation via greedy sparse regression. Ph.D. thesis (2014)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Chia-Kai Yeh
    • 1
    Email author
  • Nathan Matsuda
    • 1
  • Xiang Huang
    • 3
  • Fengqiang Li
    • 1
  • Marc Walton
    • 2
  • Oliver Cossairt
    • 1
  1. 1.Department of EECSNorthwestern UniversityEvanstonUSA
  2. 2.Northwestern University/Art Institute of Chicago Center for Scientific Studies in the Art (NU-ACCESS)EvanstonUSA
  3. 3.Argonne National LaboratoryLemontUSA

Personalised recommendations