Advertisement

Computational Visual Media

, Volume 4, Issue 1, pp 83–102 | Cite as

Photometric stereo for strong specular highlights

  • Maryam Khanian
  • Ali Sharifi Boroujerdi
  • Michael Breuß
Open Access
Research Article

Abstract

Photometric stereo is a fundamental technique in computer vision known to produce 3D shape with high accuracy. It uses several input images of a static scene taken from one and the same camera position but under varying illumination. The vast majority of studies in this 3D reconstruction method assume orthographic projection for the camera model. In addition, they mainly use the Lambertian reflectance model as the way that light scatters at surfaces. Thus, providing reliable photometric stereo results from real world objects still remains a challenging task. We address 3D reconstruction by use of a more realistic set of assumptions, combining for the first time the complete Blinn–Phong reflectance model and perspective projection. Furthermore, we compare two different methods of incorporating the perspective projection into our model. Experiments are performed on both synthetic and real world images; the latter do not benefit from laboratory conditions. The results show the high potential of our method even for complex real world applications such as medical endoscopy images which may include many specular highlights.

Keywords

photometric stereo (PS) complete Blinn–Phong model perspective projection diffuse reflection specular reflection 

Notes

Acknowledgements

This work was supported by the Deutsche Forschungsgemeinschaft under grant number BR2245/4–1. The authors would like to thank the anonymous reviewers for helpful comments to improve the quality of the paper.

References

  1. [1]
    Horn, B. K. P. Robot Vision. The MIT Press, 1986.Google Scholar
  2. [2]
    Trucco, E.; Verri, A. Introductory Techniques for 3-D Computer Vision. Prentice Hall PTR, 1998.Google Scholar
  3. [3]
    Wöhler, C. 3D Computer Vision. Springer-Verlag, 2013.CrossRefGoogle Scholar
  4. [4]
    Ihrke, I.; Kutulakos, K. N.; Lensch, H. P. A.; Magnor, M.; Heidrich, W. Transparent and specular object reconstruction. Computer Graphics Forum Vol. 29, No. 8, 2400–2426, 2010.CrossRefGoogle Scholar
  5. [5]
    Xiong, Y.; Shafer, S. A. Depth from focusing and defocusing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 68–73, 1993.CrossRefGoogle Scholar
  6. [6]
    Faugeras, O. Three-Dimensional Computer Vision. The MIT Press, 1993.Google Scholar
  7. [7]
    Tomasi, C.; Kanade, T. Shape and motion from image streams under orthography: A factorization method. International Journal of Computer Vision Vol. 9, No. 2, 137–154, 1992.CrossRefGoogle Scholar
  8. [8]
    Adato, Y.; Vasilyev, Y.; Zickler, T.; Ben-Shahar, O. Shape from specular flow. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 32, No. 11, 2054–2070, 2010.CrossRefGoogle Scholar
  9. [9]
    Godard, C.; Hedman, P.; Li, W.; Brostow, G. J. Multi-view reconstruction of highly specular surfaces in uncontrolled environments. In: Proceedings of the International Conference on 3D Vision, 19–27, 2015.Google Scholar
  10. [10]
    Sankaranarayanan, A. C.; Veeraraghavan, A.; Tuzel, O.; Agrawal, A. Specular surface reconstruction from sparse reflection correspondences. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1245–1252, 2010.Google Scholar
  11. [11]
    Woodham, R. J. Photometric stereo: A reflectance map technique for determining surface orientation from image intensity. In: Proceedings of the SPIE 0155, Image Understanding Systems and Industrial Applications I, 136–143, 1978.Google Scholar
  12. [12]
    Horn, B. K. P.; Woodham, R. J.; Silver, W. M. Determining shape and reflectance using multiple images. MIT Artificial Intelligence Laboratory, Memo 490, 1978.Google Scholar
  13. [13]
    Woodham, R. J. Photometric method for determining surface orientation from multiple images. Optical Engineering Vol. 19, No. 1, 134–144, 1980.CrossRefGoogle Scholar
  14. [14]
    Lambert, J. H.; DiLaura, D. L. Photometry, or, on the measure and gradations of light, colors, and shade: Translation from the Latin of photometria, sive, de mensura et gradibus luminis, colorum et umbrae. Illuminating Engineering Society of North America, 2001.Google Scholar
  15. [15]
    Beckmann, P.; Spizzichino, A. The Scattering of Electromagnetic Waves from Rough Surfaces. Norwood, MA, USA: Artech House, Inc., 1987.zbMATHGoogle Scholar
  16. [16]
    Brandenberg, W. M.; Neu, J. T. Undirectional reflectance of imperfectly diffuse surfaces. Journal of the Optical Society of America Vol. 56, No. 1, 97–103, 1966.CrossRefGoogle Scholar
  17. [17]
    Tagare, H. D.; Defigueiredo, R. J. P. A framework for the construction of general reflectance maps for machine vision. CVGIP: Image Understanding Vol. 57, No. 3, 265–282, 1993.CrossRefGoogle Scholar
  18. [18]
    Tankus, A.; Sochen, N.; Yeshurun, Y. Shape-fromshading under perspective projection. International Journal of Computer Vision Vol. 63, No. 1, 21–43, 2005.CrossRefGoogle Scholar
  19. [19]
    Mukaigawa, Y.; Ishii, Y.; Shakunaga, T. Analysis of photometric factors based on photometric linearization. Journal of the Optical Society of America A Vol. 24, No. 10, 3326–3334, 2007.CrossRefGoogle Scholar
  20. [20]
    Mallick, S. P.; Zickler, T. E.; Kriegman, D. J.; Belhumeur, P. N. Beyond Lambert: Reconstructing specular surfaces using color. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, 619–626, 2005.Google Scholar
  21. [21]
    Yu, C.; Seo, Y.; Lee, S. W. Photometric stereo from maximum feasible Lambertian reflections. In: Computer Vision–ECCV 2010. Lecture Notes in Computer Science, Vol. 6314. Daniilidis, K.; Maragos, P.; Paragios, N. Eds. Springer, Berlin, Heidelberg, 115–126, 2010.Google Scholar
  22. [22]
    Miyazaki, D.; Hara, K.; Ikeuchi, K. Median photometric stereo as applied to the segonko tumulus and museum objects. International Journal of Computer Vision Vol. 86, Nos. 2–3, 229–242, 2010.CrossRefGoogle Scholar
  23. [23]
    Tang, K.-L.; Tang, C.-K.; Wong, T.-T. Dense photometric stereo using tensorial belife propagation. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1, 132–139, 2005.Google Scholar
  24. [24]
    Wu, L.; Ganesh, A.; Shi, B.; Matsushita, Y.; Wang, Y.; Ma, Y. Robust photometris stereo via low-rank matrix completion and recovery. In: Computer Vision–ACCV 2010. Lecture Notes in Computer Science, Vol. 6494. Kimmel, R.; Klette, R.; Sugimoto, A. Eds. Springer, Berlin, Heidelberg, 703–717, 2010.Google Scholar
  25. [25]
    Smith, W.; Fang, F. Height from photometric ratio with model-based light source selection. Computer Vision and Image Understanding Vol. 145, 128–138, 2016.CrossRefGoogle Scholar
  26. [26]
    Hertzmann, A.; Seitz, S. M. Shape and materials by example: A photometric stereo approach. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1, I-533–I-540, 2003.Google Scholar
  27. [27]
    Goldman, D. B.; Curless, B.; Hertzmann, A.; Seitz, S. M. Shape and spatially-varying BRDFs from photometric stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 32, No. 6, 1060–1071, 2010.CrossRefGoogle Scholar
  28. [28]
    Oxholm, G.; Nishino, K. Multiview shape and reflectance from natural illumination. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2163–2170, 2014.Google Scholar
  29. [29]
    Galo, M.; Tozzi, C. L. Surface reconstruction using multiple light sources and perspective projection. In: Proceedings of the 3rd IEEE International Conference on Image Processing, Vol. 2, 309–312, 1996.CrossRefGoogle Scholar
  30. [30]
    Tankus, A.; Kiryati, N. Photometric stereo under perspective projection. In: Proceedings of the 10th IEEE International Conference on Computer Vision, Vol. 1, 611–616, 2005.Google Scholar
  31. [31]
    Mecca, R.; Tankus, A; Bruckstein, A. M. Twoimage perspective photometric stereo using shapefrom-shading. In: Computer Vision–ACCV 2012. Lecture Notes in Computer Science, Vol. 7727. Lee, K. M.; Matsushita, Y.; Rehg, J. M.; Hu, Z. Eds. Springer, Berlin, Heidelberg, 110–121, 2013.Google Scholar
  32. [32]
    Vogel, O.; Valgaerts, L.; Breuß, M.; Weickert, J. Making shape from shading work for real-world images. In: Pattern Recognition. Lecture Notes in Computer Science, Vol. 5748. Denzler, J.; Notni, G.; Süße, H. Eds. Springer, Berlin, Heidelberg, 191–200, 2009.CrossRefGoogle Scholar
  33. [33]
    Cho, S.-Y.; Chow, T. W. S. Shape recovery from shading by a new neural-based reflectance model. IEEE Transactions on Neural Networks Vol. 10, No. 6, 1536–1541, 1999.CrossRefGoogle Scholar
  34. [34]
    Blinn, J. F. Models of light reflection for computer synthesized pictures. In: Proceedings of the 4th Annual Conference on Computer Graphics and Interactive Techniques, 192–198, 1977.Google Scholar
  35. [35]
    Phong, B. T. Illumination for computer generated pictures. Communications of ACM Vol. 18, No. 6, 311–317, 1975.CrossRefGoogle Scholar
  36. [36]
    Hartley, R.; Zisserman, A. Multiple View Geometry in Computer Vision. Cambridge University Press, 2003.zbMATHGoogle Scholar
  37. [37]
    Mecca, R.; Rodolà, E.; Cremers, D. Realistic photometric stereo using partial differential irradiance equation ratios. Computers & Graphics Vol. 51, 8–16, 2015.CrossRefGoogle Scholar
  38. [38]
    Mecca, R.; Quéau, Y. Unifying diffuse and specular reflections for the photometric stereo problem. In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision, 1–9, 2016.Google Scholar
  39. [39]
    Tozza, S.; Mecca, R.; Duocastella, M.; Del Bue, A. Direct differential photometric stereo shape recovery of diffuse and specular surfaces. Journal of Mathematical Imaging and Vision Vol. 56, No. 1, 57–76, 2016.MathSciNetCrossRefzbMATHGoogle Scholar
  40. [40]
    Kim, H.; Jin, H.; Hadap, S.; Kweon, K. Specular reflection separation using dark channel prior. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1460–1467, 2013.Google Scholar
  41. [41]
    Mallick, S. P.; Zickler, T. E.; Kriegman, D. J.; Belhumeur, P. N. Beyond Lambert: Reconstructing specular surfaces using color. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, 619–626, 2005.Google Scholar
  42. [42]
    Tan, R. T.; Ikeuchi, K. Separating reflection components of textured surfaces using a single image. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol.27, No. 2, 178–193, 2005.CrossRefGoogle Scholar
  43. [43]
    Khanian, M.; Sharifi Boroujerdi, A.; Breuß, M. Perspective photometric stereo beyond Lambert. In: Proceedings of Vol. 9534, the 12th International Conference on Quality Control by Artificial Vision, 95341F, 2015.Google Scholar
  44. [44]
    Papadhimitri, T.; Favaro, P. A new perspective on uncalibrated photometric stereo. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1474–1481, 2013.Google Scholar
  45. [45]
    Quéau, Y.; Durou, J.-D. Edge-preserving integration of a normal field: Weighted least-squares, TV and L1 approaches. In: Scale Space and Variational Methods in Computer Vision. Lecture Notes in Computer Science, Vol. 9087. Aujol, J. F.; Nikolova, M.; Papadakis, N. Eds. Springer, Cham, 576–588, 2015.Google Scholar
  46. [46]
    Camilli, F.; Tozza, S. A unified approach to the wellposedness of some non-Lambertian models in shapefrom-shading. SIAM Journal on Imaging Sciences Vol. 10, No. 1, 26–46, 2017.MathSciNetCrossRefzbMATHGoogle Scholar
  47. [47]
    Levenberg, K. A method for the solution of certain non-linear problems in least squares. Quarterly of Applied Mathematics Vol. 2, No. 2, 164–168, 1944.MathSciNetCrossRefzbMATHGoogle Scholar
  48. [48]
    Marquardt, D. An algorithm for least squares estimation on nonlinear parameters. Journal of the Society of Industrial and Applied Mathematics Vol. 11, No. 2, 431–441, 1963.MathSciNetCrossRefzbMATHGoogle Scholar
  49. [49]
    Bähr, M.; Breuß, M.; Quéau, Y.; Boroujerdi, A. S.; Durou, J.-D. Fast and accurate surface normal integration on non-rectangular domains. Computational Visual Media Vol. 3, No. 2, 107–129, 2017.CrossRefGoogle Scholar
  50. [50]
    The Stanford 3D scanning repository. Available at http://graphics.stanford.edu/data/3Dscanrep/.Google Scholar
  51. [51]
    Sumner, R. W.; Popović, J. Deformation transfer for triangle meshes. ACM Transactions on Graphics Vol. 23, No. 3, 399–405, 2004.CrossRefGoogle Scholar
  52. [52]
    Norman, J. F.; Todd, J. T.; Norman, H. F.; Clayton, A. M.; McBride, T. R. Visual discrimination of local surface structure: Slant, tilt, and curvedness. Vision Research Vol. 46, Nos. 6–7, 1057–1069, 2006.CrossRefGoogle Scholar
  53. [53]
    Rosenberg, A.; Cowan, N. J.; Angelaki, D. E. The visual representation of 3D object orientation in parietal cortex. Journal of Neuroscience Vol. 33, No. 49, 19352–19361, 2013.CrossRefGoogle Scholar
  54. [54]
    Sugihara, H.; Murakami, I.; Shenoy, K. V.; Andersen, R. A.; Komatsu, H. Response of MSTD neurons to simulated 3D orientation of rotating planes. Journal of Neurophysiology Vol. 87, No. 1, 273–285, 2002.CrossRefGoogle Scholar
  55. [55]
    Saunders, J. A.; Knill, D. C. Perception of 3D surface orientation from skew symmetry. Vision Research Vol. 41, No. 24, 3163–3183, 2001.CrossRefGoogle Scholar
  56. [56]
    Stevens, K. A. Surface tilt (the direction of slant): A neglected psychophysical variable. Perception & Psychophysics Vol. 33, No. 3, 241–250, 1983.CrossRefGoogle Scholar
  57. [57]
    Braunstein, M. L.; Payne, J. W. Perspective and form ratio as determinants of relative slant judgments. Journal of Experimental Psychology Vol. 81, No. 3, 584–590, 1969.CrossRefGoogle Scholar
  58. [58]
    Tibau, S.; Willems, B.; Van Den Bergh, E.; Wagemans, J. The role of the centre of projection in the estimation of slant from texture of planar surfaces. Perception Vol. 30, No. 2, 185–193, 2001.CrossRefGoogle Scholar
  59. [59]
    Tankus, A.; Sochen, N.; Yeshurun, Y. Reconstruction of medical images by perspective shape-from-shading. In: Proceedings of the 17th International Conference on Pattern Recognition, Vol. 3, 778–781, 2004.CrossRefGoogle Scholar
  60. [60]
    Tatemasu, K.; Iwahori, Y.; Nakamura, T.; Fukui, S.; Woodham, R. J.; Kasugai, K. Shape from endoscope image based on photometric and geometric constraints. Procedia Computer Science Vol. 22, 1285–1293, 2013.CrossRefGoogle Scholar
  61. [61]
    Pharr, M.; Jakob, W.; Humphreys, G. Physically Based Rendering: From Theory to Implementation. Morgan Kaufmann Publishers Inc., 2010.Google Scholar

Copyright information

© The Author(s) 2017

Open Access The articles published in this journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.

Authors and Affiliations

  • Maryam Khanian
    • 1
  • Ali Sharifi Boroujerdi
    • 1
  • Michael Breuß
    • 1
  1. 1.Chair of Applied MathematicsBrandenburg University of TechnologyCottbusGermany

Personalised recommendations