Skip to main content
SpringerLink
Log in

LED-Based Photometric Stereo: Modeling, Calibration and Numerical Solution

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

We conduct a thorough study of photometric stereo under nearby point light source illumination, from modeling to numerical solution, through calibration. In the classical formulation of photometric stereo, the luminous fluxes are assumed to be directional, which is very difficult to achieve in practice. Rather, we use light-emitting diodes to illuminate the scene to be reconstructed. Such point light sources are very convenient to use, yet they yield a more complex photometric stereo model which is arduous to solve. We first derive in a physically sound manner this model, and show how to calibrate its parameters. Then, we discuss two state-of-the-art numerical solutions. The first one alternatingly estimates the albedo and the normals, and then integrates the normals into a depth map. It is shown empirically to be independent from the initialization, but convergence of this sequential approach is not established. The second one directly recovers the depth, by formulating photometric stereo as a system of nonlinear partial differential equations (PDEs), which are linearized using image ratios. Although the sequential approach is avoided, initialization matters a lot and convergence is not established either. Therefore, we introduce a provably convergent alternating reweighted least-squares scheme for solving the original system of nonlinear PDEs. Finally, we extend this study to the case of RGB images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Source: http://www.lumileds.com/uploads/28/DS64-pdf)

Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Source: http://www.lumileds.com/uploads/28/DS64-pdf

Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

Notes

  1. The equalities (1.3) are in fact proportionality relationships: see the expression (2.12) of \(I(\mathbf {p})\).

  2. We use white LUXEON Rebel LEDs: http://www.luxeonstar.com/luxeon-rebel-leds.

  3. The intensity is expressed in lumen per steradian (\(\hbox {lm} \, \hbox {sr}^{-1}\)), i.e., in candela (cd).

  4. It is also necessary to calibrate the camera, since the 3D-frame is attached to it. We assume that this has been made beforehand.

  5. A luminance is expressed in \(\hbox {lm}\, \hbox {m}^{-2} \,\hbox {sr}^{-1}\) (or \(\hbox {cd} \,\hbox {m}^{-2}\)), an illuminance in \(\hbox {lm} \, \hbox {m}^{-2}\), or lux (lx).

  6. The reflectance is generally referred to as the bidirectional reflectance distribution function, or BRDF.

  7. Negative values in the right hand side of Eq. (2.9) are clamped to zero in order to account for self-shadows.

  8. Provided that the RAW image format is used.

  9. To perform these operations, we use the Computer Vision toolbox from MATLAB.

  10. Without this change of variable, one would obtain a system of homogeneous PDEs in lieu of (3.23), which would need regularization to be solved, see [56].

  11. In fact, any noise assumption should be formulated on the images, and not on Model (3.23), which was obtained by considering ratios of gray levels: if the noise on gray levels is Gaussian, then that on ratios is Cauchy-distributed [25]. Hence, the least-squares solution (3.24) is the best linear unbiased estimator, but it is not the optimal solution.

  12. In our experiments, the gradient operator \(\nabla \) is discretized by forward, first-order finite differences with a Neumann boundary condition.

  13. In our experiments, we use the same discretization as in Sect. 3.2, for fair comparison.

  14. See [58] for some discussion and comparison with state-of-the-art robust methods [31, 41, 65].

  15. We use the notation \(\frac{\partial }{\partial }\) to avoid the confusion with the spatial derivatives denoted by \(\nabla \), and neglect the fraction when the derivation variable is obvious.

  16. The right hand side function in Eq. (4.14) is a majorant of \(\mathcal {E}(\cdot ,\tilde{\varvec{z}}^{(k)})\), and it is easily verified that its value and gradient are equal to those of \(\mathcal {E}(\cdot ,\tilde{\varvec{z}}^{(k)})\) in \(\tilde{\varvec{\rho }}^{(k)}\). It is therefore suitable as approximation.

  17. Since \(\phi \) is supposed even and monotonically increasing over \(\mathbb {R}^+\), this variable can be used as weight because, \(\forall x \in \mathbb {R} \backslash \{0\}\), \(\phi '(x) / x \ge 0\) and thus \(w^i_j(\tilde{\varvec{\rho }},\tilde{\varvec{z}}) \ge 0\).

  18. Lemma 1 shows that it is a positive semi-definite approximation of the Hessian \(\frac{\partial ^2 \mathcal {E}}{\partial \tilde{\varvec{\rho }}^2}(\tilde{\varvec{\rho }}^{(k)},\tilde{\varvec{z}}^{(k)})\), hence the notation.

  19. Similar to the \(\tilde{\varvec{\rho }}\)-subproblem, \(\tilde{\varvec{z}}^{(k+1)}\) is taken to be of minimal distance to \(\tilde{\varvec{z}}^{(k)}\) whenever non-uniqueness of the solution in (4.23) is encountered. The pseudo-inverse operator in (4.24) takes care of such cases [19, Theorem 5.5.1].

  20. And thus a quasi-Newton step with respect to the \(\tilde{\varvec{z}}\)-subproblem in (4.5), since \(\partial \tilde{\mathcal {E}}_{\tilde{\varvec{z}}}(\tilde{\varvec{z}}^{(k)};\tilde{\varvec{\rho }}^{(k+1)},\tilde{\varvec{z}}^{(k)}) = \frac{\partial \mathcal {E}}{\partial \tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^{(k+1)},\tilde{\varvec{z}}^{(k)})\).

  21. Since each colored intensity \(\varPhi _\star \) depends on the transmission spectrum \(c_\star (\lambda )\) by its definition (5.9), (5.13) implies that \(\varPhi _\star \) also depends on the color of the paper upon which the checkerboard is printed. Hence, the color of the paper will somehow influence the estimated color of the observed scene.

References

  1. Ackermann, J., Fuhrmann, S., Goesele, M.: Geometric point light source calibration. In: Proceedings of the 18th International Workshop on Vision, Modeling and Visualization, pp. 161–168. Lugano, Switzerland (2013)

  2. Ahmad, J., Sun, J., Smith, L., Smith, M.: An improved photometric stereo through distance estimation and light vector optimization from diffused maxima region. Pattern Recogn. Lett. 50, 15–22 (2014)

    Article  Google Scholar 

  3. Angelopoulou, M.E., Petrou, M.: Uncalibrated flatfielding and illumination vector estimation for photometric stereo face reconstruction. Mach. Vis. Appl. 25(5), 1317–1332 (2013)

    Article  Google Scholar 

  4. Aoto, T., Taketomi, T., Sato, T., Mukaigawa, Y., Yokoya, N.: Position estimation of near point light sources using a clear hollow sphere. In: Proceedings of the 21st International Conference on Pattern Recognition, pp. 3721–3724. Tsukuba, Japan (2012)

  5. Barsky, S., Petrou, M.: The 4-source photometric stereo technique for three-dimensional surfaces in the presence of highlights and shadows. IEEE Trans. Pattern Anal. Mach. Intell. 25(10), 1239–1252 (2003)

    Article  Google Scholar 

  6. Basri, R., Jacobs, D.W.: Lambertian reflectance and linear subspaces. IEEE Trans. Pattern Anal. Mach. Intell. 25(2), 218–233 (2003)

    Article  Google Scholar 

  7. Bennahmias, M., Arik, E., Yu, K., Voloshenko, D., Chua, K., Pradhan, R., Forrester, T., Jannson, T.: Modeling of non-Lambertian sources in lighting applications. In: Proceedings of SPIE Optical Engineering and Applications, vol. 6669. San Diego, USA (2007)

  8. Bony, A., Bringier, B., Khoudeir, M.: Tridimensional reconstruction by photometric stereo with near spot light sources. In: Proceedings of the 21st European Signal Processing Conference. Marrakech, Morocco (2013)

  9. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)

    Article  MATH  Google Scholar 

  10. Bringier, B., Bony, A., Khoudeir, M.: Specularity and shadow detection for the multisource photometric reconstruction of a textured surface. J. Opt. Soc. Am. A 29(1), 11–21 (2012)

    Article  Google Scholar 

  11. Ciortan, I., Pintus, R., Marchioro, G., Daffara, C., Giachetti, A., Gobbetti, E.: A practical reflectance transformation imaging pipeline for surface characterization in cultural heritage. In: Proceedings of the 14th Eurographics Workshop on Graphics and Cultural Heritage. Genova, Italy (2016)

  12. Clark, J.J.: Active photometric stereo. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 29–34 (1992)

  13. Collins, T., Bartoli, A.: 3D Reconstruction in laparoscopy with close-range photometric stereo. In: Proceedings of the 15th International Conference on Medical Imaging and Computer Assisted Intervention, pp. 634–642. Nice, France (2012)

  14. Durou, J.D., Aujol, J.F., Courteille, F.: Integrating the normal field of a surface in the presence of discontinuities. In: Proceedings of the 7th International Conference on Energy Minimization Methods in Computer Vision and Pattern Recognition, Lecture Notes in Computer Science, vol. 5681, pp. 261–273. Bonn, Germany (2009)

  15. Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput. Math. Appl. 2(1), 17–40 (1976)

    Article  MATH  Google Scholar 

  16. Gardner, I.C.: Validity of the cosine-fourth-power law of illumination. J. Res. Nat. Bur. Stand. 39, 213–219 (1947)

    Article  Google Scholar 

  17. Giachetti, A., Daffara, C., Reghelin C., Gobbetti, E., Pintus, R.: Light calibration and quality assessment methods for reflectance transformation imaging applied to artworks’ analysis. In: Proceedings of SPIE Optics for Arts, Architecture, and Archaeology V, vol. 9527. Munich, Germany (2015)

  18. Glowinski, R., Marroco, A.: Sur l’approximation, par éléments finis d’ordre un, et la résolution, par pénalisation-dualité d’une classe de problèmes de Dirichlet non linéaires. ESAIM Math. Model. Numer. Anal. 9(2), 41–76 (1975)

    MATH  Google Scholar 

  19. Golub, G.H., Van Loan, C.F.: Matrix Computations, 4th edn. John Hopkings University Press, Baltimore (2013)

    MATH  Google Scholar 

  20. Gotardo, P.F.U., Simon, T., Sheikh, Y., Matthews, I.: Photogeometric scene flow for high-detail dynamic 3D reconstruction. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 846–854. Santiago, Chile (2015)

  21. Gratton, S., Lawless, S., Nichols, N.K.: Approximate Gauss–Newton methods for nonlinear least squares problems. SIAM J. Optim. 18, 106–132 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  22. Hara, K., Nishino, K., Ikeuchi, K.: Light source position and reflectance estimation from a single view without the distant illumination assumption. IEEE Trans. Pattern Anal. Mach. Intell. 27(4), 493–505 (2005)

    Article  Google Scholar 

  23. Hernández, C., Vogiatzis, G., Brostow, G.J., Stenger, B., Cipolla, R.: Non-rigid photometric stereo with colored lights. In: Proceedings of the 11th IEEE International Conference on Computer Vision. Rio de Janeiro, Brazil (2007)

  24. Hernández, C., Vogiatzis, G., Cipolla, R.: Multiview photometric stereo. IEEE Trans. Pattern Anal. Mach. Intell. 30(3), 548–554 (2008)

    Article  Google Scholar 

  25. Hinkley, D.V.: On the ratio of two correlated normal random variables. Biometrika 56(3), 635–639 (1969)

    Article  MathSciNet  MATH  Google Scholar 

  26. Hoeltgen, L., Quéau, Y., Breuss, M., Radow, G.: Optimised photometric stereo via non-convex variational minimisation. In: Proceedings of the 27th British Machine Vision Conference. York, UK (2016)

  27. Horn, B.K.P.: Robot Vision. MIT Press, Cambridge (1986)

    Google Scholar 

  28. Horn, B.K.P., Brooks, M.J. (eds.): Shape from Shading. MIT Press, Cambridge (1989)

    Google Scholar 

  29. Huang, X., Walton, M., Bearman, G., Cossairt, O.: Near light correction for image relighting and 3D shape recovery. In: Proceedings of the International Congress on Digital Heritage, vol. 1, pp. 215–222. Granada, Spain (2015)

  30. Ikeda, O., Duan, Y.: Color photometric stereo for albedo and shape reconstruction. In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision. Lake Placid, USA (2008)

  31. Ikehata, S., Wipf, D., Matsushita, Y., Aizawa, K.: Photometric stereo using sparse Bayesian regression for general diffuse surfaces. IEEE Trans. Pattern Anal. Mach. Intell. 36(9), 1816–1831 (2014)

    Article  Google Scholar 

  32. Iwahori, Y., Sugie, H., Ishii, N.: Reconstructing shape from shading images under point light source illumination. In: Proceedings of the 19th International Conference on Pattern Recognition, vol. 1, pp. 83–87. Atlantic City, USA (1990)

  33. Jiang, J., Liu, D., Gu, J., Süsstrunk, S.: What is the space of spectral sensitivity functions for digital color cameras? In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision, pp. 168–179. Clearwater, USA (2013)

  34. Kolagani, N., Fox, J.S., Blidberg, D.R.: Photometric stereo using point light sources. In: Proceedings of the 9th IEEE International Conference on Robotics and Automation, vol. 2, pp. 1759–1764. Nice, France (1992)

  35. Kontsevich, L.L., Petrov, A.P., Vergelskaya, I.S.: Reconstruction of shape from shading in color images. J. Opt. Soc. Am. A 11(3), 1047–1052 (1994)

    Article  Google Scholar 

  36. Koppal, S.J., Narasimhan, S.G.: Novel depth cues from uncalibrated near-field lighting. In: Proceedings of the IEEE International Conference on Computer Vision (2007)

  37. Liao, J., Buchholz, B., Thiery, J.M., Bauszat, P., Eisemann, E.: Indoor scene reconstruction using near-light photometric stereo. IEEE Trans. Image Process. 26(3), 1089–1101 (2016)

    Article  MathSciNet  Google Scholar 

  38. Logothetis, F., Mecca, R., Cipolla, R.: Semi-calibrated near field photometric stereo. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA (2017)

  39. Logothetis, F., Mecca, R., Quéau, Y., Cipolla, R.: Near-field photometric stereo in ambient light. In: Proceedings of the 27th British Machine Vision Conference. York, UK (2016)

  40. McGunnigle, G., Chantler, M.J.: Resolving handwriting from background printing using photometric stereo. Pattern Recogn. 36(8), 1869–1879 (2003)

    Article  Google Scholar 

  41. Mecca, R., Quéau, Y., Logothetis, F., Cipolla, R.: A single lobe photometric stereo approach for heterogeneous material. SIAM J. Imaging Sci. 9(4), 1858–1888 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  42. Mecca, R., Rodolà, E., Cremers, D.: Realistic photometric stereo using partial differential irradiance equation ratios. Comput. Graph. 51, 8–16 (2015)

    Article  Google Scholar 

  43. Mecca, R., Wetzler, A., Bruckstein, A.M., Kimmel, R.: Near Field Photometric Stereo with Point Light Sources. SIAM J. Imaging Sci. 7(4), 2732–2770 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  44. Migita, T., Ogino, S., Shakunaga, T.: Direct bundle estimation for recovery of shape, reflectance property and light position. In: Proceedings of the 10th European Conference on Computer Vision, Lecture Notes in Computer Science, vol. 5304, pp. 412–425. Marseille, France (2008)

  45. Moreno, I., Avendaño Alejo, M., Tzonchev, R.I.: Designing light-emitting diode arrays for uniform near-field irradiance. Appl. Opt. 45(10), 2265–2272 (2006)

    Article  Google Scholar 

  46. Moreno, I., Sun, C.C.: Modeling the radiation pattern of LEDs. Opt. Express 16(3), 1808–1819 (2008)

    Article  Google Scholar 

  47. Nie, Y., Song, Z.: A novel photometric stereo method with nonisotropic point light sources. In: Proceedings of the 23rd International Conference on Pattern Recognition, pp. 1737–1742. Cancun, Mexico (2016)

  48. Nie, Y., Song, Z., Ji, M., Zhu, L.: A novel calibration method for the photometric stereo system with non-isotropic LED lamps. In: Proceedings of the IEEE Conference on Real-time Computing and Robotics, pp. 289–294. Angkor Wat, Cambodia (2016)

  49. Oren, M., Nayar, S.K.: Generalization of the Lambertian model and implications for machine vision. Int. J. Comput. Vis. 14(3), 227–251 (1995)

    Article  Google Scholar 

  50. Ortega, J.M., Rheinboldt, W.C.: Iterative Solution of Nonlinear Equations in Several Variables. Academic Press, New York (1970)

    MATH  Google Scholar 

  51. Papadhimitri, T., Favaro, P.: Uncalibrated near-light photometric stereo. In: Proceedings of the 25th British Machine Vision Conference. Nottingham, UK (2014)

  52. Pătrăucean, V., Gurdjos, P., Grompone von Gioi, R.: A parameterless line segment and elliptical arc detector with enhanced ellipse fitting. In: Proceedings of the 12th European Conference on Computer Vision, pp. 572–585. Florence, Italy (2012)

  53. Pintus, R., Ciortan, I., Giachetti, A., Gobbetti, E.: Practical free-form RTI acquisition with local spot lights. In: Smart Tools and Applications for Graphics. Genova, Italy (2016)

  54. Powell, M.W., Sarkar, S., Goldgof, D.: A simple strategy for calibrating the geometry of light sources. IEEE Trans. Pattern Anal. Mach. Intell. 23(9), 1022–1027 (2001)

    Article  Google Scholar 

  55. Quéau, Y., Durou, J.D., Aujol, J.F.: Normal Integration—Part I: A Survey (2016). https://hal.archives-ouvertes.fr/hal-01334349

  56. Quéau, Y., Mecca, R., Durou, J.D.: Unbiased photometric stereo for colored surfaces: a variational approach. In: Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition, pp. 4350–4358. Las Vegas, USA (2016)

  57. Quéau, Y., Wu, T., Cremers, D.: Semi-calibrated near-light photometric stereo. In: Proceedings of the 6th International Conference on Scale Space and Variational Methods in Computer Vision, Lecture Notes in Computer Science, vol. 10302, pp. 656–668. Kolding, Denmark (2017)

  58. Quéau, Y., Wu, T., Lauze, F., Durou, J.D., Cremers, D.: A non-convex variational approach to photometric stereo under inaccurate lighting. In: Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA (2017)

  59. Shen, H.L., Cheng, Y.: Calibrating light sources by using a planar mirror. J. Electron. Imaging 20(1), 013002 (2011)

  60. Smith, W., Fang, F.: Height from photometric ratio with model-based light source selection. Comput. Vis. Image Underst. 145, 128–138 (2016)

    Article  Google Scholar 

  61. Sun, J., Smith, M., Smith, L., Farooq, A.: Sampling Light Field for Photometric Stereo. Int. J. Comput. Theory Eng. 5(1), 14–18 (2013)

    Article  Google Scholar 

  62. Takai, T., Maki, A., Niinuma, K., Matsuyama, T.: Difference sphere: an approach to near light source estimation. Comput. Vis. Image Underst. 113(9), 966–978 (2009)

    Article  Google Scholar 

  63. Wolke, R., Schwetlick, H.: Iteratively reweighted least squares: algorithms, convergence analysis, and numerical comparisons. SIAM J. Sci. Stat. Comput. 9(5), 907–921 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  64. Woodham, R.J.: Photometric method for determining surface orientation from multiple images. Opt. Eng. 19(1), 139–144 (1980)

    Article  Google Scholar 

  65. Wu, L., Ganesh, A., Shi, B., Matsushita, Y., Wang, Y., Ma, Y.: Robust photometric stereo via low-rank matrix completion and recovery. In: Proceedings of the Asian Conference on Computer Vision, Lecture Notes in Computer Science, vol. 6494, pp. 703–717. Queenstown, New-Zealand (2010)

  66. Wu, Z., Li, L.: A line-integration based method for depth recovery from surface normals. Comput. Vis. Graph. Image Process. 43(1), 53–66 (1988)

    Article  Google Scholar 

  67. Xie, L., Song, Z., Jiao, G., Huang, X., Jia, K.: A practical means for calibrating an LED-based photometric stereo system. Opt. Lasers Eng. 64, 42–50 (2015)

    Article  Google Scholar 

  68. Xie, W., Dai, C., Wang, C.C.L.: Photometric stereo with near point lighting: a solution by mesh deformation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA (2015)

  69. Yeh, C.K., Matsuda, N., Huang, X., Li, F., Walton, M., Cossairt, O.: A streamlined photometric stereo framework for cultural heritage. In: Proceedings of the 14th European Conference on Computer Vision, pp. 738–752. Amsterdam, The Netherlands (2016)

Download references

Acknowledgements

Yvain Quéau, Tao Wu and Daniel Cremers were supported by the ERC Consolidator Grant “3D Reloaded”. Funding was provided by European Research Council (Grant No. 649323).

Author information

Authors and Affiliations

  1. Department of Computer Science, Technical University of Munich, Garching, Germany

    Yvain Quéau, Tao Wu & Daniel Cremers

  2. IRIT, UMR CNRS 5505, Université de Toulouse, Toulouse, France

    Bastien Durix & Jean-Denis Durou

  3. Department of Computer Science, University of Copenhagen, Copenhagen, Denmark

    François Lauze

Authors
  1. Yvain Quéau
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bastien Durix
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tao Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Daniel Cremers
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. François Lauze
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jean-Denis Durou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yvain Quéau.

Appendices

Appendix A: Proof of Lemma 1

Proof

First note that, under the condition (4.29), the function \(\mathcal {E}(\cdot ,\tilde{\varvec{z}})\) (resp. \(\tilde{\mathcal {E}}_{\tilde{\varvec{z}}}(\cdot ;\tilde{\varvec{\rho }},\tilde{\varvec{z}})\)) is twice continuously differentiable at \(\tilde{\varvec{\rho }}\) (resp. \(\tilde{\varvec{z}}\)), whenever \((\tilde{\varvec{\rho }},\tilde{\varvec{z}})\) is sufficiently close to \((\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)\). The corresponding second-order derivatives are calculated as follows:

$$\begin{aligned}&\delta \tilde{\varvec{\rho }}^\top \frac{\partial ^2 \mathcal {E}}{\partial \tilde{\varvec{\rho }}^2}(\tilde{\varvec{\rho }},\tilde{\varvec{z}})\delta \tilde{\varvec{\rho }}= \sum _{j=1}^n\sum _{i=1}^m \phi ''(r^i_j(\tilde{\varvec{\rho }},\tilde{\varvec{z}})) \left( \delta \tilde{\rho }_j\{\zeta ^i_j(\tilde{\varvec{z}})\}_+\right) ^2, \end{aligned}$$
(A.1)
$$\begin{aligned}&\delta \tilde{\varvec{z}}^\top \partial ^2 \tilde{\mathcal {E}}_{\tilde{\varvec{z}}}(\tilde{\varvec{z}};\tilde{\varvec{\rho }},\tilde{\varvec{z}})\delta \tilde{\varvec{z}}\nonumber \\&\quad = \sum _{j=1}^n\sum _{i=1}^m \phi ''(r^i_j(\tilde{\varvec{\rho }},\tilde{\varvec{z}}))\left( \tilde{\rho }_j \, \chi (\zeta ^i_j(\tilde{\varvec{z}})) \, \delta \tilde{\varvec{z}}^\top \partial \zeta ^i_j(\tilde{\varvec{z}})\right) ^2. \end{aligned}$$
(A.2)

Comparing the above two formulas with (4.21) and (4.25), the conclusion follows from condition (4.8). \(\square \)

Appendix B: Proof of Theorem 1

Proof

First note that condition (4.32) implies that

$$\begin{aligned}&\dfrac{\partial ^2 \mathcal {E}}{\partial \tilde{\varvec{\rho }}^2}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*) \succ {\varvec{O}}, \end{aligned}$$
(B.1)
$$\begin{aligned}&\frac{\partial ^2 \mathcal {E}}{\partial \tilde{\varvec{z}}^2}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*) -\dfrac{\partial ^2 \mathcal {E}}{\partial \tilde{\varvec{\rho }}\partial \tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*) \dfrac{\partial ^2 \mathcal {E}}{\partial \tilde{\varvec{\rho }}^2}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)^{-1} \dfrac{\partial ^2 \mathcal {E}}{\partial \tilde{\varvec{\rho }}\partial \tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*) \succ {\varvec{O}}. \end{aligned}$$
(B.2)

Utilizing Lemma 1 in conjunction with (B.2) and (4.33), we obtain

$$\begin{aligned}&H_{\tilde{\varvec{\rho }}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*) \succ {\varvec{O}}, \quad H_{\tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*) \succ {\varvec{O}}, \end{aligned}$$
(B.3)
$$\begin{aligned}&\frac{\partial ^2 \mathcal {E}}{\partial \tilde{\varvec{z}}^2}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*) -\dfrac{\partial ^2 \mathcal {E}}{\partial \tilde{\varvec{\rho }}\partial \tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*) H_{\tilde{\varvec{\rho }}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)^{-1} \dfrac{\partial ^2 \mathcal {E}}{\partial \tilde{\varvec{\rho }}\partial \tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*\!,\!\tilde{\varvec{z}}^*) \succ {\varvec{O}}. \end{aligned}$$
(B.4)

Now consider the iteration

$$\begin{aligned}&\tilde{\varvec{z}}^{(k+1)} = \tilde{\varvec{z}}^{(k)}-H_{\tilde{\varvec{z}}}\left( \tilde{\varvec{\rho }}^{(k+1)},\tilde{\varvec{z}}^{(k)}\right) ^{-1}\frac{\partial \mathcal {E}}{\partial \tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^{(k+1)},\tilde{\varvec{z}}^{(k)}) \nonumber \\&\quad = \tilde{\varvec{z}}^{(k)} - H_{\tilde{\varvec{z}}} \left( \tilde{\varvec{\rho }}^{(k)}-H_{\tilde{\varvec{\rho }}}(\tilde{\varvec{\rho }}^{(k)}, \tilde{\varvec{z}}^{(k)})^{-1}\frac{\partial \mathcal {E}}{\partial \tilde{\varvec{\rho }}}(\tilde{\varvec{\rho }}^{(k)}, \tilde{\varvec{z}}^{(k)}),\tilde{\varvec{z}}^{(k)}\right) ^{-1} \nonumber \\&\quad \frac{\partial \mathcal {E}}{\partial \tilde{\varvec{z}}} \left( \tilde{\varvec{\rho }}^{(k)}-H_{\tilde{\varvec{\rho }}}(\tilde{\varvec{\rho }}^{(k)},\tilde{\varvec{z}}^{(k)})^{-1}\frac{\partial \mathcal {E}}{\partial \tilde{\varvec{\rho }}}(\tilde{\varvec{\rho }}^{(k)},\tilde{\varvec{z}}^{(k)}),\tilde{\varvec{z}}^{(k)} \right) \end{aligned}$$
(B.5)

as a map \(\tilde{\varvec{z}}^{(k)}\mapsto \tilde{\varvec{z}}^{(k+1)}\). By the Ostrowski theorem [50, Proposition 10.1.3], the local convergence of \(\{\tilde{\varvec{z}}^{(k)}\}\) to \(\tilde{\varvec{z}}^*\) follows if the spectral radius of the Jacobian

$$\begin{aligned} \frac{\partial \tilde{\varvec{z}}^{(k+1)}}{\partial \tilde{\varvec{z}}^{(k)}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)&=\text {id}-H_{\tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)^{-1}\frac{\partial ^2 \mathcal {E}}{\partial \tilde{\varvec{z}}^2}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*) \nonumber \\&\quad +H_{\tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)^{-1}\frac{\partial ^2 \mathcal {E}}{\partial \tilde{\varvec{\rho }}\partial \tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*) H_{\tilde{\varvec{\rho }}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)^{-1}\nonumber \\&\quad \frac{\partial ^2 \mathcal {E}}{\partial \tilde{\varvec{\rho }}\partial \tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*) \end{aligned}$$
(B.6)

is strictly less than 1. Using the similarity transform with \(H_{\tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)^{\frac{1}{2}}\), we derive:

$$\begin{aligned}&\mathrm {sr}\left( \frac{\partial \tilde{\varvec{z}}^{(k+1)}}{\partial \tilde{\varvec{z}}^{k}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)\right) \nonumber \\&\quad = \mathrm {sr}\left( H_{\tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)^{\frac{1}{2}}\frac{\partial \tilde{\varvec{z}}^{(k+1)}}{\partial \tilde{\varvec{z}}^{k}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)H_{\tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)^{-\frac{1}{2}}\right) \end{aligned}$$
(B.7)
$$\begin{aligned}&= \mathrm {sr}\bigg ( \text {id}-H_{\tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)^{-\frac{1}{2}}\frac{\partial ^2 \mathcal {E}}{\partial \tilde{\varvec{z}}^2}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)H_{\tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)^{-\frac{1}{2}} \nonumber \\&\quad + H_{\tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)^{-\frac{1}{2}}\frac{\partial ^2 \mathcal {E}}{\partial \tilde{\varvec{\rho }}\partial \tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*) H_{\tilde{\varvec{\rho }}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)^{-1} \nonumber \\&\quad \frac{\partial ^2 \mathcal {E}}{\partial \tilde{\varvec{\rho }}\partial \tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)H_{\tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)^{-\frac{1}{2}} \bigg ) \end{aligned}$$
(B.8)
$$\begin{aligned}&= \sup _{\Vert \mathbf {v}\Vert =1} \bigg | \Vert \mathbf {v}\Vert ^2 \nonumber \\&\quad -\mathbf {v}^\top H_{\tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)^{-\frac{1}{2}}\frac{\partial ^2 \mathcal {E}}{\partial \tilde{\varvec{z}}^2}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)H_{\tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)^{-\frac{1}{2}}\mathbf {v}\nonumber \\&\quad +\mathbf {v}^\top H_{\tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)^{-\frac{1}{2}}\frac{\partial ^2 \mathcal {E}}{\partial \tilde{\varvec{\rho }}\partial \tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*) H_{\tilde{\varvec{\rho }}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)^{-1} \nonumber \\&\quad \frac{\partial ^2 \mathcal {E}}{\partial \tilde{\varvec{\rho }}\partial \tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)H_{\tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)^{-\frac{1}{2}}\mathbf {v}\bigg |. \end{aligned}$$
(B.9)

It follows from condition (4.34) that

$$\begin{aligned} \frac{\partial ^2 \mathcal {E}}{\partial \tilde{\varvec{z}}^2}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)\prec 2\partial ^2 \tilde{\mathcal {E}}_{\tilde{\varvec{z}}}(\tilde{\varvec{z}}^*;\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)\preceq 2 H_{\tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*), \end{aligned}$$
(B.10)

and hence

$$\begin{aligned} \text {id}-H_{\tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)^{-\frac{1}{2}}\frac{\partial ^2 \mathcal {E}}{\partial \tilde{\varvec{z}}^2}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)H_{\tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)^{-\frac{1}{2}}\succ -\text {id}. \end{aligned}$$
(B.11)

Consequently, there exists \(\epsilon _1\in (0,1)\) such that the following inequality holds for an arbitrary \(\mathbf {v}\):

$$\begin{aligned}&\Vert \mathbf {v}\Vert ^2-\mathbf {v}^\top H_{\tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)^{-\frac{1}{2}}\frac{\partial ^2 \mathcal {E}}{\partial \tilde{\varvec{z}}^2}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)H_{\tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)^{-\frac{1}{2}}\mathbf {v}\nonumber \\&\quad \ge -(1-\epsilon _1)\Vert \mathbf {v}\Vert ^2. \end{aligned}$$
(B.12)

Meanwhile, condition (B.4) implies that, for some \(\epsilon _2\in (0,1)\):

$$\begin{aligned}&\mathbf {v}^\top H_{\tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)^{-\frac{1}{2}}\frac{\partial ^2 \mathcal {E}}{\partial \tilde{\varvec{z}}^2}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)H_{\tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)^{-\frac{1}{2}}\mathbf {v}\nonumber \\&\quad -\mathbf {v}^\top H_{\tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)^{-\frac{1}{2}}\frac{\partial ^2 \mathcal {E}}{\partial \tilde{\varvec{\rho }}\partial \tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*) H_{\tilde{\varvec{\rho }}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)^{-1} \nonumber \\&\quad \frac{\partial ^2 \mathcal {E}}{\partial \tilde{\varvec{\rho }}\partial \tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)H_{\tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)^{-\frac{1}{2}}\mathbf {v}\nonumber \\&\quad = (H_{\tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)^{-\frac{1}{2}}\mathbf {v})^\top \nonumber \\&\quad \qquad \Big ( \frac{\partial ^2 \mathcal {E}}{\partial \tilde{\varvec{z}}^2}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)-\frac{\partial ^2 \mathcal {E}}{\partial \tilde{\varvec{\rho }}\partial \tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*) H_{\tilde{\varvec{\rho }}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)^{-1}\frac{\partial ^2 \mathcal {E}}{\partial \tilde{\varvec{\rho }}\partial \tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*) \Big ) \nonumber \\&\qquad \quad \left( H_{\tilde{\varvec{z}}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)^{-\frac{1}{2}}\mathbf {v}\right) \end{aligned}$$
(B.13)
$$\begin{aligned}&\ge \epsilon _2\Vert \mathbf {v}\Vert ^2. \end{aligned}$$
(B.14)

Altogether, we conclude

$$\begin{aligned} \mathrm {sr}\left( \frac{\partial \tilde{\varvec{z}}^{(k+1)}}{\partial \tilde{\varvec{z}}^{k}}(\tilde{\varvec{\rho }}^*,\tilde{\varvec{z}}^*)\right) \le 1-\min (\epsilon _1,\epsilon _2), \end{aligned}$$
(B.15)

and hence the convergence of \(\{\tilde{\varvec{z}}^{(k)}\}\). The convergence of \(\{\tilde{\varvec{\rho }}^{(k)}\}\) to \(\tilde{\varvec{\rho }}^*\) follows from a similar argument. \(\square \)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Quéau, Y., Durix, B., Wu, T. et al. LED-Based Photometric Stereo: Modeling, Calibration and Numerical Solution. J Math Imaging Vis 60, 313–340 (2018). https://doi.org/10.1007/s10851-017-0761-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-017-0761-1

Keywords

Search

Navigation