Computational Visual Media

, Volume 3, Issue 2, pp 107–129 | Cite as

Fast and accurate surface normal integration on non-rectangular domains

  • Martin Bähr
  • Michael BreußEmail author
  • Yvain Quéau
  • Ali Sharifi Boroujerdi
  • Jean-Denis Durou
Open Access
Research Article


The integration of surface normals for the purpose of computing the shape of a surface in 3D space is a classic problem in computer vision. However, even nowadays it is still a challenging task to devise a method that is flexible enough to work on non-trivial computational domains with high accuracy, robustness, and computational efficiency. By uniting a classic approach for surface normal integration with modern computational techniques, we construct a solver that fulfils these requirements. Building upon the Poisson integration model, we use an iterative Krylov subspace solver as a core step in tackling the task. While such a method can be very efficient, it may only show its full potential when combined with suitable numerical preconditioning and problem-specific initialisation. We perform a thorough numerical study in order to identify an appropriate preconditioner for this purpose. To provide suitable initialisation, we compute this initial state using a recently developed fast marching integrator. Detailed numerical experiments illustrate the benefits of this novel combination. In addition, we show on real-world photometric stereo datasets that the developed numerical framework is flexible enough to tackle modern computer vision applications.


surface normal integration Poisson integration conjugate gradient method preconditioning fast marching method Krylov subspace methods photometric stereo 3D reconstruction 


  1. [1]
    Pérez, P.; Gangnet, M.; Blake, A. Poisson image editing. ACM Transactions on Graphics Vol. 22, No. 3, 313–318, 2003.CrossRefGoogle Scholar
  2. [2]
    Horn, B. K. P.; Brooks, M. J. The variational approach to shape from shading. Computer Vision, Graphics and Image Processing Vol. 33, No. 2, 174–208, 1986.CrossRefzbMATHGoogle Scholar
  3. [3]
    Woodham, R. J. Photometric method for determining surface orientation from multiple images. Optical Engineering Vol. 19, No. 1, 191139, 1980.CrossRefGoogle Scholar
  4. [4]
    Zafeiriou, S.; Atkinson, G. A.; Hansen, M. F.; Smith, W. A. P.; Argyriou, V.; Petrou, M.; Smith, M. L.; Smith, L. N. Face recognition and verification using photometric stereo: The photoface database and a comprehensive evaluation. IEEE Transactions on Information Forensics and Security Vol. 8, No. 1, 121–135, 2013.CrossRefGoogle Scholar
  5. [5]
    Smith, M. L.; Stamp, R. J. Automated inspection of textured ceramic tiles. Computers in Industry Vol. 43, No. 1, 73–82, 2000.CrossRefGoogle Scholar
  6. [6]
    Esteban, C. H.; Vogiatzis, G.; Cipolla, R. Multiview photometric stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 30, No. 3, 548–554, 2008.CrossRefGoogle Scholar
  7. [7]
    Haque, S. M.; Chatterjee, A.; Govindu, V. M. High quality photometric reconstruction using a depth camera. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2275–2282, 2014.Google Scholar
  8. [8]
    Harker, M.; O’Leary, P. Least squares surface reconstruction from measured gradient fields. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1–7, 2008.Google Scholar
  9. [9]
    Durou, J.-D.; Aujol, J.-F.; Courteille, F. Integrating the normal field of a surface in the presence of discontinuities. In: Energy Minimization Methods in Computer Vision and Pattern Recognition. Cremers, D.; Boykov, Y.; Blake, A.; Schmidt, F. R. Eds. Springer Berlin Heidelberg, 261–273, 2009.CrossRefGoogle Scholar
  10. [10]
    Klette, R.; Schlüns, K. Height data from gradient fields. In: Proceedings of SPIE 2908, Machine Vision Applications, Architectures, and Systems Integration V, 204–215, 1996.CrossRefGoogle Scholar
  11. [11]
    Coleman Jr., E. N.; Jain, R. Obtaining 3-dimensional shape of textured and specular surfaces using foursource photometry. Computer Graphics and Image Processing Vol. 18, No. 4, 309–328, 1982.CrossRefGoogle Scholar
  12. [12]
    Wu, Z.; Li, L. A line-integration based method for depth recovery from surface normals. Computer Vision, Graphics and Image Processing Vol. 43, No. 1, 53–66, 1988.CrossRefGoogle Scholar
  13. [13]
    Robles-Kelly, A.; Hancock, E. R. A graphspectral method for surface height recovery. Pattern Recognition Vol. 38, No. 8, 1167–1186, 2005.CrossRefGoogle Scholar
  14. [14]
    Ho, J.; Lim, J.; Yang, M. H.; Kriegmann, D. Integrating surface normal vectors using fast marching method. In: Computer Vision–ECCV 2006. Leonardis, A.; Bischof, H.; Pinz, A. Eds. Springer Berlin Heidelberg, 239–250, 2006.CrossRefGoogle Scholar
  15. [15]
    Galliani, S.; Breuß, M.; Ju, Y. C. Fast and robust surface normal integration by a discrete eikonal equation. In: Proceedings of the 23rd British Machine Vision Conference, 2012.Google Scholar
  16. [16]
    Bähr, M.; Breuß, M. An improved eikonal method for surface normal integration. In: Pattern Recognition. Gall, J.; Gehler, P.; Leibe, B. Eds. Springer International Publishing, 274–284, 2015.CrossRefGoogle Scholar
  17. [17]
    Frankot, R. T.; Chellappa, R. A method for enforcing integrability in shape from shading algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 10, No. 4, 439–451, 1988.CrossRefzbMATHGoogle Scholar
  18. [18]
    Simchony, T.; Chellappa, R.; Shao, M. Direct analytical methods for solving Poisson equations in computer vision problems. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 12, No. 5, 435–446, 1990.CrossRefzbMATHGoogle Scholar
  19. [19]
    Wei, T.; Klette, R. A wavelet-based algorithm for height from gradients. In: Robot Vision. Klette, R.; Peleg, S.; Sommer, G. Eds. Springer Berlin Heidelberg, 84–90, 2001.CrossRefGoogle Scholar
  20. [20]
    Kovesi, P. Shapelets correlated with surface normals produce surfaces. In: Proceedings of the 10th IEEE International Conference on Computer Vision, Vol. 2, 994–1001, 2005.Google Scholar
  21. [21]
    Wei, T.; Klette, R. Depth recovery from noisy gradient vector fields using regularization. In: Computer Analysis of Images and Patterns. Petkov, N.; Westenberg, M. A. Eds. Springer Berlin Heidelberg, 116–123, 2003.CrossRefGoogle Scholar
  22. [22]
    Karaçali, B.; Snyder, W. Noise reduction in surface reconstruction from a given gradient field. International Journal on Computer Vision Vol. 60, No. 1, 25–44, 2004.CrossRefGoogle Scholar
  23. [23]
    Agrawal, A.; Raskar, R.; Chellappa, R. What is the range of surface reconstructions from a gradient field? In: Computer Vision–ECCV 2006. Leonardis, A.; Bischof, H.; Pinz, A. Eds. Springer Berlin Heidelberg, 578–591, 2006.CrossRefGoogle Scholar
  24. [24]
    Badri, H.; Yahia, H. M.; Aboutajdine, D. Robust surface reconstruction via triple sparsity. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2283–2290, 2014.Google Scholar
  25. [25]
    Du, Z.; Robles-Kelly, A.; Lu, F. Robust surface reconstruction from gradient field using the L1 norm. In: Proceedings of the 9th Biennial Conference of the Australian Pattern Recognition Society on Digital Image Computing Techniques and Applications, 203–209, 2007.Google Scholar
  26. [26]
    Reddy, D.; Agrawal, A. K.; Chellappa, R. Enforcing integrability by error correction using l1-minimization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2350–2357, 2009.Google Scholar
  27. [27]
    Quéau, Y.; Durou, J.-D. Edge-preserving integration of a normal field: Weighted least squares, TV and L1 approaches. In: Scale Space and Variational Methods in Computer Vision. Aujol, J.-F.; Nikolova, M.; Papadakis, N. Eds. Springer International Publishing 576–588, 2015.Google Scholar
  28. [28]
    Harker, M.; O’Leary, P. Regularized reconstruction of a surface from its measured gradient field. Journal of Mathematical Imaging and Vision Vol. 51, No. 1, 46–70, 2015.MathSciNetCrossRefzbMATHGoogle Scholar
  29. [29]
    Breuß, M.; Quéau, Y.; Bähr, M.; Durou, J.-D. Highly efficient surface normal integration. In: Proceedings of the 20th Conference on Scientific Computing, 204–213, 2016.Google Scholar
  30. [30]
    Meister, A. Comparison of different Krylov subspace methods embedded in an implicit finite volume scheme for the computation of viscous and inviscid flow fields on unstructured grids. Journal of Computational Physics Vol. 140, No. 2, 311–345, 1998.MathSciNetCrossRefzbMATHGoogle Scholar
  31. [31]
    Saad, Y. Iterative Methods for Sparse Linear Systems. Society for Industrial and Applied Mathematics, 2003.CrossRefzbMATHGoogle Scholar
  32. [32]
    Durou, J.-D.; Courteille, F. Integration of a normal field without boundary condition. In: Proceedings of the 1st International Workshop on Photometric Analysis for Computer Vision, 2007.Google Scholar
  33. [33]
    Kimmel, R.; Sethian, J. A. Optimal algorithm for shape from shading and path planning. Journal of Mathematical Imaging and Vision Vol. 14, No. 3, 237–244, 2001.MathSciNetCrossRefzbMATHGoogle Scholar
  34. [34]
    Tsitsiklis, J. N. Efficient algorithms for globally optimal trajectories. IEEE Transactions on Automatic Control Vol. 40, No. 9, 1528–1538, 1995.MathSciNetCrossRefzbMATHGoogle Scholar
  35. [35]
    Sethian, J. A. A fast marching level set method for monotonically advancing fronts. Proceedings of the National Academy of Sciences of the United States of America Vol. 93, No. 4, 1591–1595, 1996.MathSciNetCrossRefzbMATHGoogle Scholar
  36. [36]
    Helmsen, J. J.; Puckett, E. G.; Colella, P.; Dorr, M. Two new methods for simulating photolithography development in 3D. In: Proceedings of SPIE 2726, Optical Microlithography IX, 253–261, 1996.Google Scholar
  37. [37]
    Sethian, J. A. Level Set Methods and Fast Marching Methods: Evolving Interfaces in Computational Geometry, Fluid Mechanics, Computer Vision, and Materials Science. Cambridge University Press, 1999.zbMATHGoogle Scholar
  38. [38]
    Yatziv, L.; Bartesaghi, A.; Sapiro, G. O(N) implementation of the fast marching algorithm. Journal of Computational Physics Vol. 212, No. 2, 393–399, 2006.CrossRefzbMATHGoogle Scholar
  39. [39]
    Cacace, S.; Cristiani, E.; Falcone, M. Can local single-pass methods solve any stationary Hamilton–Jacobi–Bellman equation? SIAM Journal on Scientific Computing Vol. 36, No. 2, A570–A587, 2014.MathSciNetCrossRefzbMATHGoogle Scholar
  40. [40]
    Zimmer, H.; Bruhn, A.; Valgaerts, L.; Breuß, M.; Weickert, J.; Rosenhahn, B.; Seidel, H.-P. PDEbased anisotropic disparity-driven stereo vision. In: Proceddings of the 13th International Fall Workshop Vision, Modeling, and Visualization, 263–272, 2008.Google Scholar
  41. [41]
    Meister, A. Numerik Linearer Gleichungssysteme. Eine Einführung in Moderne Verfahren. Springer Spektrum, 2014.Google Scholar
  42. [42]
    Hestenes, M. R.; Stiefel, E. Methods of conjugate gradients for solving linear systems. Journal of Research of the National Bureau of Standards Vol. 6, No. 49, 46–70, 1952.MathSciNetzbMATHGoogle Scholar
  43. [43]
    Meurant, G. Computer Solution of Large Linear Systems. Elsevier Science, 1999.zbMATHGoogle Scholar
  44. [44]
    Meurant, G. The Lanczos and Conjugate Gradient Algorithms: From Theory to Finite Precision Computations. Society for Industrial and Applied Mathematics, 2006.CrossRefzbMATHGoogle Scholar
  45. [45]
    Golub, G. H.; van Loan, C. F. Matrix Computation, 3rd edn. Johns Hopkins, 1996.zbMATHGoogle Scholar
  46. [46]
    Meijerink, J. A.; van der Vorst, H. A. An iterative solution method for linear systems of which the coefficient matrix is a symmetric M-matrix. Mathematics of Computation Vol. 31, No. 137, 148–162, 1977.MathSciNetzbMATHGoogle Scholar
  47. [47]
    Kershaw, D. S. The incomplete Cholesky-conjugate gradient method for the iterative solution of systems of linear equations. Journal of Computational Physics Vol. 26, No. 1, 43–65, 1978.MathSciNetCrossRefzbMATHGoogle Scholar
  48. [48]
    Benzi, M. Preconditioning techniques for large linear systems: A survey. Journal of Computational Physics Vol. 182, No. 2, 418–477, 2002.MathSciNetCrossRefzbMATHGoogle Scholar
  49. [49]
    Kaasschieter, E. F. Preconditioned conjugate gradients for solving singular systems. Journal of Computational and Applied Mathematics Vol. 24, Nos. 1–2, 265–275, 1988.MathSciNetCrossRefzbMATHGoogle Scholar
  50. [50]
    Tang, J. M.; Vuik, C. Acceleration of preconditioned Krylov solvers for bubbly flow problems. In: Parallel Processing and Applied Mathematics. Wyrzykowski, R.; Dongarra, J.; Karczewski, K.; Wasniewski, J. Eds. Springer Berlin Heidelberg, 1323–1332, 2008.CrossRefGoogle Scholar
  51. [51]
    Manteuffel, T. A. An incomplete factorization technique for positive definite linear systems. Mathematics of Computation Vol. 34, No. 150, 473–497, 1980.MathSciNetCrossRefzbMATHGoogle Scholar
  52. [52]
    Wang, Z.; Bovik, A. C.; Sheikh, H. R.; Simoncelli, E. P. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing Vol. 13, No. 4, 600–612, 2004.CrossRefGoogle Scholar
  53. [53]
    Noakes, L.; Kozera, R. Nonlinearities and noise reduction in 3-source photometric stereo. Journal of Mathematical Imaging and Vision Vol. 18, No. 2, 119–127, 2003.MathSciNetCrossRefzbMATHGoogle Scholar
  54. [54]
    Chang, J. Y.; Lee, K. M.; Lee, S. U. Multiview normal field integration using level set methods. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1–8, 2007.Google Scholar

Copyright information

© The Author(s) 2016

Open Access The articles published in this journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  • Martin Bähr
    • 1
  • Michael Breuß
    • 1
    Email author
  • Yvain Quéau
    • 2
  • Ali Sharifi Boroujerdi
    • 1
  • Jean-Denis Durou
    • 3
  1. 1.Brandenburg Technical University, Institute for Mathematics, Chair for Applied MathematicsCottbusGermany
  2. 2.Technical University MunichGarchingGermany
  3. 3.Université de Toulouse, IRIT, UMR CNRS 5505ToulouseFrance

Personalised recommendations