Resolution-Enhanced Photometric Stereo

  • Ping Tan
  • Stephen Lin
  • Long Quan
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3953)


Conventional photometric stereo has a fundamental limitation that the scale of recovered geometry is limited to the resolution of the input images. However, surfaces that contain sub-pixel geometric structures are not well modelled by a single normal direction per pixel. In this work, we propose a technique for resolution-enhanced photometric stereo, in which surface geometry is computed at a resolution higher than that of the input images. To achieve this goal, our method first utilizes a generalized reflectance model to recover the distribution of surface normals inside each pixel. This normal distribution is then used to infer sub-pixel structures on a surface of uniform material by spatially arranging the normals among pixels at a higher resolution according to a minimum description length criterion on 3D textons over the surface. With the presented method, high resolution geometry that is lost in conventional photometric stereo can be recovered from low resolution input images.


Gaussian Mixture Model Minimum Description Length Resolution Enhancement Integrability Constraint Photometric Stereo 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Woodham, R.: Photometric stereo: A reflectance map technique for determining surface orientation from image intensities, pp. 136–143 (1978)Google Scholar
  2. 2.
    Silver, W.: Determining Shape and Reflectance Using Multiple Images, PhD thesis, MIT (1980)Google Scholar
  3. 3.
    Shashua, A.: Geometry and Photometry in 3D Visual Recognition. PhD thesis, MIT (1992) Google Scholar
  4. 4.
    Coleman, E., Jain, R.: Obtaining 3-dimensional shape of textured and specular surfaces using four-source photometry. Computer Graphics and Image Processing 18, 309–328 (1982)CrossRefGoogle Scholar
  5. 5.
    Solomon, F., Ikeuchi, K.: Extracting the shape and roughness of specular lobe objects using four light photometric stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence 18, 449–454 (1996)CrossRefGoogle Scholar
  6. 6.
    Drbohlav, O., Sara, R.: Specularities reduce ambiguity of uncalibrated photometric stereo. In: Proc. European Conference on Computer Vision, vol. 2, pp. 46–62 (2002)Google Scholar
  7. 7.
    Tagare, H., de Figueiredo, R.: A theory of photometric stereo for a class of diffuse non-lambertian surfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence 13 (1991) 133 – 152 CrossRefGoogle Scholar
  8. 8.
    Georghiades, A.: Incorporating the torrance and sparrow model of reflectance in uncalibrated photometric stereo. In: Proceeding of IEEE International Conference on Computer Vision 2003, vol. 2, pp. 816–823. IEEE Computer Society Press, Los Alamitos (2003)CrossRefGoogle Scholar
  9. 9.
    Rissanen, J.: A universal prior for integers and estimation by minimum description length. The Annals of Statistics 11, 416–431 (1983)zbMATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Leung, T., Malik, J.: Representing and recognizing the visual appearance of materials using three-dimensional textons. International Journal of Computer Vision 43, 29–44 (2001)zbMATHCrossRefGoogle Scholar
  11. 11.
    Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Francisco (1988)Google Scholar
  12. 12.
    Kolmogorov, V.: Convergent tree-reweighted message passing for energy minimization. Microsoft Technical Report MSR-TR-2005-38 (2005) Google Scholar
  13. 13.
    Geman, S., Geman, D.: Stochastic relaxation, gibbs distribution, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence 6, 721–741 (1984)zbMATHCrossRefGoogle Scholar
  14. 14.
    Irani, M., Peleg, S.: Improving resolution by image registration. CVGIP: Graphical Models and Image Processing 53, 231–239 (1991)CrossRefGoogle Scholar
  15. 15.
    Ben-Ezra, M., Zomet, A., Nayar, S.K.: Video super-resolution using controlled subpixel detector shifts. IEEE Transactions on Pattern Analysis and Machine Intelligence 27, 977–987 (2005)CrossRefGoogle Scholar
  16. 16.
    Freeman, W., Pasztor, E., Carmichael, O.: Learning low-level vision. International Journal of Computer Vision 40, 24–57 (2000)CrossRefGoogle Scholar
  17. 17.
    Sun, J., Zheng, N.N., Tao, H., Shum, H.Y.: Image hallucination with primal sketch priors. In: Proceedings of IEEE Computer Vision and Pattern Recognition (2003)Google Scholar
  18. 18.
    Cook, R.L., Torrance, K.E.: A reflectance model for computer graphics. In: Proceedings of SIGGRAPH 1981, pp. 307–316 (1981)Google Scholar
  19. 19.
    Westin, S.H., Arvo, J.R., Torrance, K.E.: Predicting reflectance functions from complex surfaces. In: Proceedings of SIGGRAPH 1992, pp. 255–264 (1992)Google Scholar
  20. 20.
    Ashikhmin, M., Premonze, S., Shirley, P.: A microfacet-based brdf generator. In: Proceedings of SIGGRAPH 2000, pp. 65–74 (2000)Google Scholar
  21. 21.
    Ngan, A., Durand, F., Matusik, W.: Experimental analysis of brdf models. In: Proceedings of 16th Eurographics Symposium on Rendering (2005)Google Scholar
  22. 22.
    Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum-likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society 39(1), 1–38 (1977)zbMATHMathSciNetGoogle Scholar
  23. 23.
    Bilmes, J.: A gentle tutorial on the em algorithm and its application to parameter estimation for gaussian mixture and hidden markov models. Technical Report ICSI-TR-97-021, Univ. of California-Berkeley (1997) Google Scholar
  24. 24.
    Frankot, R., Chellappa, R.: A method for enforcing integrability in shape from shading algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence 10, 439–451 (1988)zbMATHCrossRefGoogle Scholar
  25. 25.
    Kovesi, P.: Shapelets correlated with surface normals produce surfaces. In: Proceedings of IEEE International Conference on Computer Vision (2005)Google Scholar
  26. 26.
    Agrawal, A., Chellappa, R., Raskar, R.: An algebraic approach to surface reconstruction from gradient fields. In: Proceedings of IEEE International Conference on Computer Vision (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Ping Tan
    • 1
  • Stephen Lin
    • 2
  • Long Quan
    • 1
  1. 1.Computer Science DepartmentHong Kong University of Science and Technology 
  2. 2.Microsoft Research Asia 

Personalised recommendations