Abstract
Photometric stereo is a fundamental technique in computer vision known to produce 3D shape with high accuracy. It uses several input images of a static scene taken from one and the same camera position but under varying illumination. The vast majority of studies in this 3D reconstruction method assume orthographic projection for the camera model. In addition, they mainly use the Lambertian reflectance model as the way that light scatters at surfaces. Thus, providing reliable photometric stereo results from real world objects still remains a challenging task. We address 3D reconstruction by use of a more realistic set of assumptions, combining for the first time the complete Blinn–Phong reflectance model and perspective projection. Furthermore, we compare two different methods of incorporating the perspective projection into our model. Experiments are performed on both synthetic and real world images; the latter do not benefit from laboratory conditions. The results show the high potential of our method even for complex real world applications such as medical endoscopy images which may include many specular highlights.
Article PDF
Similar content being viewed by others
Explore related subjects
Find the latest articles, discoveries, and news in related topics.Avoid common mistakes on your manuscript.
References
Horn, B. K. P. Robot Vision. The MIT Press, 1986.
Trucco, E.; Verri, A. Introductory Techniques for 3-D Computer Vision. Prentice Hall PTR, 1998.
Wöhler, C. 3D Computer Vision. Springer-Verlag, 2013.
Ihrke, I.; Kutulakos, K. N.; Lensch, H. P. A.; Magnor, M.; Heidrich, W. Transparent and specular object reconstruction. Computer Graphics Forum Vol. 29, No. 8, 2400–2426, 2010.
Xiong, Y.; Shafer, S. A. Depth from focusing and defocusing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 68–73, 1993.
Faugeras, O. Three-Dimensional Computer Vision. The MIT Press, 1993.
Tomasi, C.; Kanade, T. Shape and motion from image streams under orthography: A factorization method. International Journal of Computer Vision Vol. 9, No. 2, 137–154, 1992.
Adato, Y.; Vasilyev, Y.; Zickler, T.; Ben-Shahar, O. Shape from specular flow. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 32, No. 11, 2054–2070, 2010.
Godard, C.; Hedman, P.; Li, W.; Brostow, G. J. Multi-view reconstruction of highly specular surfaces in uncontrolled environments. In: Proceedings of the International Conference on 3D Vision, 19–27, 2015.
Sankaranarayanan, A. C.; Veeraraghavan, A.; Tuzel, O.; Agrawal, A. Specular surface reconstruction from sparse reflection correspondences. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1245–1252, 2010.
Woodham, R. J. Photometric stereo: A reflectance map technique for determining surface orientation from image intensity. In: Proceedings of the SPIE 0155, Image Understanding Systems and Industrial Applications I, 136–143, 1978.
Horn, B. K. P.; Woodham, R. J.; Silver, W. M. Determining shape and reflectance using multiple images. MIT Artificial Intelligence Laboratory, Memo 490, 1978.
Woodham, R. J. Photometric method for determining surface orientation from multiple images. Optical Engineering Vol. 19, No. 1, 134–144, 1980.
Lambert, J. H.; DiLaura, D. L. Photometry, or, on the measure and gradations of light, colors, and shade: Translation from the Latin of photometria, sive, de mensura et gradibus luminis, colorum et umbrae. Illuminating Engineering Society of North America, 2001.
Beckmann, P.; Spizzichino, A. The Scattering of Electromagnetic Waves from Rough Surfaces. Norwood, MA, USA: Artech House, Inc., 1987.
Brandenberg, W. M.; Neu, J. T. Undirectional reflectance of imperfectly diffuse surfaces. Journal of the Optical Society of America Vol. 56, No. 1, 97–103, 1966.
Tagare, H. D.; Defigueiredo, R. J. P. A framework for the construction of general reflectance maps for machine vision. CVGIP: Image Understanding Vol. 57, No. 3, 265–282, 1993.
Tankus, A.; Sochen, N.; Yeshurun, Y. Shape-fromshading under perspective projection. International Journal of Computer Vision Vol. 63, No. 1, 21–43, 2005.
Mukaigawa, Y.; Ishii, Y.; Shakunaga, T. Analysis of photometric factors based on photometric linearization. Journal of the Optical Society of America A Vol. 24, No. 10, 3326–3334, 2007.
Mallick, S. P.; Zickler, T. E.; Kriegman, D. J.; Belhumeur, P. N. Beyond Lambert: Reconstructing specular surfaces using color. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, 619–626, 2005.
Yu, C.; Seo, Y.; Lee, S. W. Photometric stereo from maximum feasible Lambertian reflections. In: Computer Vision–ECCV 2010. Lecture Notes in Computer Science, Vol. 6314. Daniilidis, K.; Maragos, P.; Paragios, N. Eds. Springer, Berlin, Heidelberg, 115–126, 2010.
Miyazaki, D.; Hara, K.; Ikeuchi, K. Median photometric stereo as applied to the segonko tumulus and museum objects. International Journal of Computer Vision Vol. 86, Nos. 2–3, 229–242, 2010.
Tang, K.-L.; Tang, C.-K.; Wong, T.-T. Dense photometric stereo using tensorial belife propagation. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1, 132–139, 2005.
Wu, L.; Ganesh, A.; Shi, B.; Matsushita, Y.; Wang, Y.; Ma, Y. Robust photometris stereo via low-rank matrix completion and recovery. In: Computer Vision–ACCV 2010. Lecture Notes in Computer Science, Vol. 6494. Kimmel, R.; Klette, R.; Sugimoto, A. Eds. Springer, Berlin, Heidelberg, 703–717, 2010.
Smith, W.; Fang, F. Height from photometric ratio with model-based light source selection. Computer Vision and Image Understanding Vol. 145, 128–138, 2016.
Hertzmann, A.; Seitz, S. M. Shape and materials by example: A photometric stereo approach. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1, I-533–I-540, 2003.
Goldman, D. B.; Curless, B.; Hertzmann, A.; Seitz, S. M. Shape and spatially-varying BRDFs from photometric stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 32, No. 6, 1060–1071, 2010.
Oxholm, G.; Nishino, K. Multiview shape and reflectance from natural illumination. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2163–2170, 2014.
Galo, M.; Tozzi, C. L. Surface reconstruction using multiple light sources and perspective projection. In: Proceedings of the 3rd IEEE International Conference on Image Processing, Vol. 2, 309–312, 1996.
Tankus, A.; Kiryati, N. Photometric stereo under perspective projection. In: Proceedings of the 10th IEEE International Conference on Computer Vision, Vol. 1, 611–616, 2005.
Mecca, R.; Tankus, A; Bruckstein, A. M. Twoimage perspective photometric stereo using shapefrom-shading. In: Computer Vision–ACCV 2012. Lecture Notes in Computer Science, Vol. 7727. Lee, K. M.; Matsushita, Y.; Rehg, J. M.; Hu, Z. Eds. Springer, Berlin, Heidelberg, 110–121, 2013.
Vogel, O.; Valgaerts, L.; Breuß, M.; Weickert, J. Making shape from shading work for real-world images. In: Pattern Recognition. Lecture Notes in Computer Science, Vol. 5748. Denzler, J.; Notni, G.; Süße, H. Eds. Springer, Berlin, Heidelberg, 191–200, 2009.
Cho, S.-Y.; Chow, T. W. S. Shape recovery from shading by a new neural-based reflectance model. IEEE Transactions on Neural Networks Vol. 10, No. 6, 1536–1541, 1999.
Blinn, J. F. Models of light reflection for computer synthesized pictures. In: Proceedings of the 4th Annual Conference on Computer Graphics and Interactive Techniques, 192–198, 1977.
Phong, B. T. Illumination for computer generated pictures. Communications of ACM Vol. 18, No. 6, 311–317, 1975.
Hartley, R.; Zisserman, A. Multiple View Geometry in Computer Vision. Cambridge University Press, 2003.
Mecca, R.; Rodolà, E.; Cremers, D. Realistic photometric stereo using partial differential irradiance equation ratios. Computers & Graphics Vol. 51, 8–16, 2015.
Mecca, R.; Quéau, Y. Unifying diffuse and specular reflections for the photometric stereo problem. In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision, 1–9, 2016.
Tozza, S.; Mecca, R.; Duocastella, M.; Del Bue, A. Direct differential photometric stereo shape recovery of diffuse and specular surfaces. Journal of Mathematical Imaging and Vision Vol. 56, No. 1, 57–76, 2016.
Kim, H.; Jin, H.; Hadap, S.; Kweon, K. Specular reflection separation using dark channel prior. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1460–1467, 2013.
Mallick, S. P.; Zickler, T. E.; Kriegman, D. J.; Belhumeur, P. N. Beyond Lambert: Reconstructing specular surfaces using color. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, 619–626, 2005.
Tan, R. T.; Ikeuchi, K. Separating reflection components of textured surfaces using a single image. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol.27, No. 2, 178–193, 2005.
Khanian, M.; Sharifi Boroujerdi, A.; Breuß, M. Perspective photometric stereo beyond Lambert. In: Proceedings of Vol. 9534, the 12th International Conference on Quality Control by Artificial Vision, 95341F, 2015.
Papadhimitri, T.; Favaro, P. A new perspective on uncalibrated photometric stereo. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1474–1481, 2013.
Quéau, Y.; Durou, J.-D. Edge-preserving integration of a normal field: Weighted least-squares, TV and L1 approaches. In: Scale Space and Variational Methods in Computer Vision. Lecture Notes in Computer Science, Vol. 9087. Aujol, J. F.; Nikolova, M.; Papadakis, N. Eds. Springer, Cham, 576–588, 2015.
Camilli, F.; Tozza, S. A unified approach to the wellposedness of some non-Lambertian models in shapefrom-shading. SIAM Journal on Imaging Sciences Vol. 10, No. 1, 26–46, 2017.
Levenberg, K. A method for the solution of certain non-linear problems in least squares. Quarterly of Applied Mathematics Vol. 2, No. 2, 164–168, 1944.
Marquardt, D. An algorithm for least squares estimation on nonlinear parameters. Journal of the Society of Industrial and Applied Mathematics Vol. 11, No. 2, 431–441, 1963.
Bähr, M.; Breuß, M.; Quéau, Y.; Boroujerdi, A. S.; Durou, J.-D. Fast and accurate surface normal integration on non-rectangular domains. Computational Visual Media Vol. 3, No. 2, 107–129, 2017.
The Stanford 3D scanning repository. Available at http://graphics.stanford.edu/data/3Dscanrep/.
Sumner, R. W.; Popović, J. Deformation transfer for triangle meshes. ACM Transactions on Graphics Vol. 23, No. 3, 399–405, 2004.
Norman, J. F.; Todd, J. T.; Norman, H. F.; Clayton, A. M.; McBride, T. R. Visual discrimination of local surface structure: Slant, tilt, and curvedness. Vision Research Vol. 46, Nos. 6–7, 1057–1069, 2006.
Rosenberg, A.; Cowan, N. J.; Angelaki, D. E. The visual representation of 3D object orientation in parietal cortex. Journal of Neuroscience Vol. 33, No. 49, 19352–19361, 2013.
Sugihara, H.; Murakami, I.; Shenoy, K. V.; Andersen, R. A.; Komatsu, H. Response of MSTD neurons to simulated 3D orientation of rotating planes. Journal of Neurophysiology Vol. 87, No. 1, 273–285, 2002.
Saunders, J. A.; Knill, D. C. Perception of 3D surface orientation from skew symmetry. Vision Research Vol. 41, No. 24, 3163–3183, 2001.
Stevens, K. A. Surface tilt (the direction of slant): A neglected psychophysical variable. Perception & Psychophysics Vol. 33, No. 3, 241–250, 1983.
Braunstein, M. L.; Payne, J. W. Perspective and form ratio as determinants of relative slant judgments. Journal of Experimental Psychology Vol. 81, No. 3, 584–590, 1969.
Tibau, S.; Willems, B.; Van Den Bergh, E.; Wagemans, J. The role of the centre of projection in the estimation of slant from texture of planar surfaces. Perception Vol. 30, No. 2, 185–193, 2001.
Tankus, A.; Sochen, N.; Yeshurun, Y. Reconstruction of medical images by perspective shape-from-shading. In: Proceedings of the 17th International Conference on Pattern Recognition, Vol. 3, 778–781, 2004.
Tatemasu, K.; Iwahori, Y.; Nakamura, T.; Fukui, S.; Woodham, R. J.; Kasugai, K. Shape from endoscope image based on photometric and geometric constraints. Procedia Computer Science Vol. 22, 1285–1293, 2013.
Pharr, M.; Jakob, W.; Humphreys, G. Physically Based Rendering: From Theory to Implementation. Morgan Kaufmann Publishers Inc., 2010.
Acknowledgements
This work was supported by the Deutsche Forschungsgemeinschaft under grant number BR2245/4–1. The authors would like to thank the anonymous reviewers for helpful comments to improve the quality of the paper.
Author information
Authors and Affiliations
Corresponding author
Additional information
This article is published with open access at Springerlink.com
Maryam Khanian is a Ph.D. student at Brandenburg University of Technology in Germany. She has worked in the area of image processing for several years and presented different researches. She is mainly interested in computer vision, 3D reconstruction, deep learning, machine learning, neural networks, and data analysis.
Ali Sharifi Boroujerdi is a Ph.D. student at Brandenburg University of Technology in Germany. After being a bachelor of software engineering, he received his master degree in software engineering in 2013, where he introduced a new class of Neuro–Fuzzy classifiers to detect DDoS cyber attacks. He is mainly interested in deep learning, especially applied to computer vision. His research interests also include machine learning, reinforcement learning, data analysis, and 3D reconstruction. He is still playing with the intrusion detection systems in his spare time.
Michael Breuß received his doctorate degree in mathematics from the University of Hamburg in 2001, and the habilitation in mathematics from the Technical University in Brunswick in 2006. For several years he had been a member of the mathematical image analysis group in Saarbrcken, Germany. Since 2016 he is a professor for applied mathematics at Brandenburg University of Technology in Germany. His research interests are mainly in mathematical image processing and 3D vision, and include in particular numerical methods.
Rights and permissions
Open Access The articles published in this journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.
About this article
Cite this article
Khanian, M., Boroujerdi, A.S. & Breuß, M. Photometric stereo for strong specular highlights. Comp. Visual Media 4, 83–102 (2018). https://doi.org/10.1007/s41095-017-0101-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41095-017-0101-9