Keywords

1 Problem Formulation

The task of surface reconstruction from its image (or images) poses one of the major challenges in computer vision. There are two common approaches adopted to tackle this problem. The first one (termed a stereo-like method) involves projective geometry tools applied to multiple camera images [1–3]. Solving the so–called matching problem constitutes here the gist of the entire reconstruction process based on incorporating the triangulation-like technique [1, 2, 4]. The second approach, coined shape from shading [3–5], relies on a priori knowledge of the physical properties of light reflectance inherent to the specific material coating the unknown surface S. In this model the surface S is assumed to be formed as a graph \(S=graph(u)\) of the unknown \(C^k\) (\(k=0,1\)) function \(u:\varOmega \rightarrow {\mathbb {R}}\) defined over an image domain \(\varOmega \subset {\mathbb {R}}^2\). Recall that the normal \(n(s)=(n_1(s),n_2(s),n_3(s))\) to S at the point \(s=(x,y,u(x,y))\in S\) (with \(n_3(s)<0\)) reads as \(n(s)=(u_x(x,y),u_y(x,y),-1)\). Assuming a distant light–source (yielding a parallel light-beam along the direction \(p=(p_1,p_2,p_3)\)) the so-called image irradiance equation

$$\begin{aligned} R(u_x(x,y),u_y(x,y))=E(x,y) \end{aligned}$$
(1)

encapsulates the relation between reflected light from surface S and absorbed light in the image \(\varOmega \) - see [4]. Here E(x, y) denotes the intensity of the absorbed light at the image point \((x,y)\in \varOmega \). On the other hand, R called reflectance map refers to the intensity of the reflected light from \(s\in S\) with normal n(s). In the special case of a Lambertian surface \(S_L\), \(R_L\) is proportional to \(\cos (\alpha )\), where \(\alpha \) represents the angle between the normal n(s) and a given light-source direction \(p=(p_1,p_2,p_3)\) - see [4]. Consequently, for \(S_L\) the corresponding image irradiance Eq. (1) (over \(\varOmega =\{(x,y)\in {\mathbb {R}}^2: 0\le E(x,y)\le 1\}\)) reads as:

$$\begin{aligned} {\langle p|n(s)\rangle \over \Vert p\Vert \Vert n(s)\Vert }={p_1u_x(x,y)+p_2u_y(x,y)-p_3\over \sqrt{p_1^2+p_2^2+p_2^2}\sqrt{u_x^2(x,y)+u_y^2(x,y)+1}}=E(x,y). \end{aligned}$$
(2)

By Cauchy-Schwartz inequality \(|E(x,y)|\le 1\) holds. The extra condition required \(E(x,y)\ge 0\) excludes shadowed (x, y)-points. The Eq. (2) yields the first order non-linear PDE (in two variables) modelling a single image shape from shading for \(S_L\). Commonly, in solving (2) one usually searches for \(u\in C^k\) (\(k=1,2\)), modulo a vertical shift \(v=u+c\) (here c is an arbitrary constant). A single image irradiance Eq. (2) renders generically an ill-posed problem (see [6–10]). Though various mathematical extra assumptions can disambiguate (2) or limit its number of solutions [4, 11–15], they often turn unrealistic or difficult to be met from real camera images.

A feasible remedy is to employ a photometric stereo technique for which the unknown surface \(S_L\) (or S) is consecutively illuminated by distant multiple light–sources positioned along linearly independent directions [4, 16]. In this set-up, contrary to the classical stereo method relying on images taken from multiple cameras, only a single camera is deployed.

As demonstrated in [4, 16, 17], three-light source photometric stereo suffices to guarantee a unique surface \(S_L=graph(u)\) (up to a vertical shift) with \(u\in C^1\) satisfying the corresponding system of three image irradiance Eq. (2) (formulated for p, q and r) over an image \(\varOmega =\varOmega _1\,\cap \,\varOmega _2\,\cap \,\varOmega _3\). The respective right-hand sides of (2) (i.e. the intensities \(E_i\ge 0\) over \(\varOmega _i\), for \(i=1,2,3\)) are obtained by consecutive illuminations of \(S_L\) along three linearly independent directions p, q \(r\in {\mathbb {R}}^3\). The entire reconstruction process is decomposed here into two following steps:

(a) a gradient computation (an algebraic step):

$$\begin{aligned} u_x(x,y)=f_1(x,y,E_1,E_2,E_3),\quad \quad u_y(x,y)=f_2(x,y,E_1,E_2,E_3). \end{aligned}$$
(3)

In case of three light-sources the resulting vector field \(\nabla u=(u_x,u_y)\) satisfying three equations of type (2) is uniquely determined in (3) in terms of image intensities \(E_i\) (\(i=1,2,3\)) and light-source directions p, q and r - see [4, 17].

(b) a gradient integration (an analytic step):

$$\begin{aligned} u(x,y)=u(x_0,y_0)+\int _{\gamma }u_x(x,y)dx+u_y(x,y)dy, \end{aligned}$$
(4)

where \((x_0,y_0),(x,y)\in \varOmega \) and \(\gamma :[t_0,t_1]\rightarrow \varOmega \) is an arbitrary piecewise-\(C^1\) curve such that \(\gamma (t_0)=(x_0,y_0)\) and \(\gamma (t_1)=(x,y)\). Note that \((x_0,y_0)\) is fixed here and \(u(x_0,y_0)\) represents any constant c. The choice of \(\gamma \) joining \((x_0,y_0)\) with varying (x, y) is arbitrary (at least in continuous model of (1)), provided \(\varOmega \) is simply connected and the vector field (3) fulfills the so-called integrability condition. The latter reads for \(u\in C^2\) as

$$\begin{aligned} u_{xy}(x,y)=u_{yx}(x,y), \end{aligned}$$
(5)

and for \(u\in C^1\) as

$$\begin{aligned} \int _{\gamma _c}u_x(x,y)dx+u_y(x,y)dy=0, \end{aligned}$$
(6)

holding for any piecewise-\(C^1\) loop \(\gamma _c\subset \varOmega \) (with \(\gamma _c(t_0)=\gamma _c(t_1)\)). Assuming \(E_i\) (\(i=1,2,3\)) are formed by a genuine Lambertian surface \(S_L=graph(u)\), a unique vector field \(\nabla u\) obtained from (3) is automatically integrable and hence the formula (4) determines an unambiguous \(u\in C^1\) (up to a constant).

The case of two-light source photometric stereo requires more intricate analysis (see [17–19]). The respective system of two image irradiance equations:

$$\begin{aligned} {p_1u_x(x,y)+p_2u_y(x,y)-p_3\over \sqrt{p_1^2+p_2^2+p_3^2}\sqrt{u_x^2(x,y)+u_y(x,y)^2+1}}= & {} E_1(x,y),\\\\\\{q_1u_x(x,y)+q_2u_y(x,y)-q_3\over \sqrt{q_1^2+q_2^2+q_3^2}\sqrt{u_x^2(x,y)+u_y(x,y)^2+1}}= & {} E_2(x,y) \end{aligned}$$
(7)

is solved by the following vector field (see [17, 18]):

$$\begin{aligned} u_x= & {} {\Vert p\Vert (q_1\langle p|q \rangle -p_1\Vert q\Vert ^2)E_1+\Vert q\Vert (p_1\langle p| q\rangle - q_1\Vert p\Vert ^2)E_2 +(p_3q_2-p_2q_3)\varepsilon \sqrt{\varLambda } \over \Vert p\Vert (p_3\Vert q\Vert ^2-q_3\langle p|q\rangle )E_1+\Vert q\Vert (q_3\Vert p\Vert ^2-p_3\langle p|q\rangle )E_2+(p_1q_2-p_2q_1)\varepsilon \sqrt{\varLambda }},\\\\\\u_y= & {} {\Vert p\Vert (q_2\langle p|q \rangle -p_2\Vert q\Vert ^2)E_1+\Vert q\Vert (p_2\langle p| q\rangle - q_2\Vert p\Vert ^2)E_2 +(p_1q_3-p_3q_1)\varepsilon \sqrt{\varLambda } \over \Vert p\Vert (p_3\Vert q\Vert ^2-q_3\langle p|q\rangle )E_1+\Vert q\Vert (q_3\Vert p\Vert ^2-p_3\langle p|q\rangle )E_2+(p_1q_2-p_2q_1)\varepsilon \sqrt{\varLambda }},\\&\&\end{aligned}$$
(8)

where

$$\begin{aligned} \varLambda= & {} \Vert p\Vert ^2\Vert q\Vert ^2\left( 1-E_1^2(x,y)-E_2^2(x,y)\right) -\langle p|q\rangle \left( \langle p| q\rangle -2\Vert p\Vert \Vert q\Vert E_1(x,y)E_2(x,y)\right) \quad \end{aligned}$$
(9)

with the function \(\varepsilon (x,y)\) taking values \(\pm 1\) so that \(f(x,y)=\varepsilon (x,y)\sqrt{\varLambda (x,y)}\) is continuous (for \(u\in C^1\)) or smooth (for \(u\in C^2\)).

As discussed in [17, 18] an image domain \(\varOmega \) is often decomposed into \(\varOmega =\varOmega ^{(1)}\cup \varOmega ^{(2)}\cup \varGamma \) with \(\varLambda >0\) over disjoint sub-domains \(\varOmega ^{(j)}\subset \varOmega \) (here \(j=1,2\)) and with \(\varLambda =0\) satisfied along some smooth curve \(\varGamma \). Consequently, over each component \(\varOmega ^{(j)}\) either \(\varepsilon (x,y)\equiv 1\) or \(\varepsilon (x,y)\equiv -1\) hold, yielding the respective two vector fields \((u_x^{\pm },u_y^{\pm })\) in (8) satisfying (7). Assuming \(E_1\) and \(E_2\) are generated by a genuine Lambertian surface \(S_L\), at least one of these two pairs of vector fields (over each \(\varOmega ^{(j)}\)) is integrable.

The ambiguous case of more than one integrable vector field from (8) is well-understood merely for the special configuration of light-source directions, namely when \(p=(0,0,-1)\) and \(q=(q_1,q_2,q_3)\) (here \(q_1^2+q_2^2>0\)) - see [17, 18]. In fact, generically there is only one integrable vector field over each \(\varOmega ^{(j)}\) (and thus over \(\varOmega \)). The latter is governed by the fulfillment of the tight condition (10). Indeed, given two simply connected sub-images \(\varOmega ^{(j)}\), a necessary and sufficient condition enforcing both vector fields \((u_x^{+},u_y^{+})\) and \((u_x^{-},u_y^{-})\) to be integrable (and thus yielding the existence of \(u^{+},u^{-}\in C^2(\varOmega ^{(i)})\) solving (7)) reads as:

$$\begin{aligned} q_1q_2\left( u_{yy}(x,y)-u_{xx}(x,y)\right) +(q_1^2-q_2^2)u_{xy}(x,y)=0, \end{aligned}$$
(10)

which is to be satisfied by either \(u^{+}\) or \(u^{-}\). In addition (see [17, 18]), if one of \(u^{+}\) (or of \(u^{-}\)) satisfies (10) then so does the complementary one i.e. \(u^{-}\) (or \(u^{+}\)). In a rare situation of (10) holding, both pairs \((u^{+},u^{-})\), determined by (4) over \(\varOmega ^{(j)}\) (\(j=1,2\)), can bifurcate (i.e. can be glued together) along the curve \(\varGamma \) to yield either zero or two or four global solutions \(u\in C^2\) over the whole image \(\varOmega \) (see [17, 18]). As it turns out, there are extra geometrical relations between both graphs of \(u^{+}\) and \(u^{-}\). Indeed, let \(K_u(x,y)\) denote the Gaussian curvature of the surface \(S_L=graph(u)\) taken at the point \((x,y,u(x,y))\in S_L\) and determined by the formula (see [20]):

$$\begin{aligned} K_u(x,y)={u_{xx}(x,y)u_{yy}(x,y)-u_{xy}^2(x,y)\over (1+u_x^2(x,y)+u_y^2(x,y))^2}. \end{aligned}$$
(11)

Interestingly, it is proved in [17, 18] that if both (7) and (10) hold then:

$$\begin{aligned} K_{u^{+}}(x,y)=-K_{u^{-}}(x,y). \end{aligned}$$
(12)

Consequently, if \(s_1=(x,y,u^{\pm }(x,y))\in S_L^{\pm }\) yields a hyperbolic point (having negative Gaussian curvature) then \(s_2=(x,y,u^{\mp }(x,y))\in S_L^{\mp }\) renders an elliptic point (having positive Gaussian curvature). Thus for \(p=(0,0,-1)\) and \(q=(q_1,q_2,q_3)\) the convexity/concavity ambiguity is automatically excluded. Noticeably, such ambiguity eventuates for single image shape from shading with \(p=(0,0,-1)\) (see e.g. [4, 8]). Finally, if \(s_1\) is a parabolic point (with vanishing Gaussian curvature) then so is \(s_2\).

This paper extends the above results with the following (see Sect. 2):

  1. 1.

    The necessary and sufficient condition (10) for testing the ambiguity in (7) is extended to arbitrary pairs of linearly independent light-source directions \(p=(p_1,p_2,p_3)\) and \(q=(q_1,q_2,q_3).\) Ultimately, such general case leads to the non-linear second-order PDE (see (18)), which inherently constitutes a challenging problem.

  2. 2.

    To alleviate the above difficulty, special configurations of two light-source directions (different from \(p=(0,0,-1)\) and \(q=(q_1,q_2,q_3)\)) are here admitted. Namely, the family of orthogonal unit vectors (parallel to OXZ-plane):

    $$\begin{aligned} p_{\alpha }=(-\sqrt{1-\alpha ^2},0,-\alpha ), \quad q_{\alpha }=(\alpha ,0,-\sqrt{1-\alpha ^2}) \end{aligned}$$
    (13)

    (with \(0<\alpha <1\)) is considered. The corresponding analysis for solving (7) (with \(p=p_{\alpha }\) and \(q=q_{\alpha }\)) addresses the uniqueness issue and establishes, in rare ambiguity cases, intrinsic geometrical inter-relations between multiple solutions in two light-source photometric stereo (see also Theorem 1).

The theoretical Sect. 2 is also supplemented (see Sect. 3) by illustrative examples supported by Mathematica numerical and symbolic computation applied to the continuous Lambertian model with pixels represented as ideal points \((x,y)\in \varOmega \) and image intensities \((E_1,E_2)\) simulated here from u as left-hand sides in (7). Finally, the closing Sect. 4 summarizes the main thrust of this paper together with indicating its extensions and hints their possible solutions.

2 Orthogonal Illumination Directions in Vertical Plane

In the first part of this section (i.e. in (i)) we consider an arbitrary configuration of linearly independent light-source directions p and q with \(\Vert p\Vert =\Vert q\Vert =1\). Under such general assumption, condition (10) (testing whether the second vector field from (8) over each \(\varOmega ^{(j)}\) is also integrable) is subsequently extended to the non-linear PDE expressed by (18). Noticeably, the latter constitutes a difficult task for further theoretical analysis. In order to deal with the latter somehow, a special case of orthogonal vectors p and q contained in the OXZ-plane is here admitted (see (ii)). The analysis to follow complements already established results in [17], covering a different special choice of \(p=(0,0,-1)\) and \(q=(q_1,q_2,q_3)\) with \(q_1^2+q_2^2>0\).

(i) Assume now that \(E_1\) and \(E_2\) introduced in (7) are generated by a genuine Lambertian surface \(S_L=graph(u)\), where \(u\in C^2\). Evidently, still by (7) both image intensities are also expressible in terms of \(u_x\) and \(u_y\). Using symbolic computation in Mathematica [21–23] (or alternatively see complicated proof in [17]) one arrives at:

$$\begin{aligned} \varLambda (x,y)={\left( u_x(p_2q_3-p_3q_2)-u_y(p_1q_3-p_3q_1)+p_2q_1-p_1q_2\right) ^2\over u_x^2+u_y^2+1} ={(\langle p \times q|n\rangle )^2\over u_x^2+u_y^2+1}. \end{aligned}$$
(14)

Combining the latter with (8) (over each \(\varOmega ^{(j)}\) determined by \(\varLambda >0\)) leads to:

$$\begin{aligned} u_x={\left\{ \begin{array}{ll} u_x,&{}\text{ if } \ \ sgn(\varepsilon )sgn(\theta )>0;\\ {(a^2-b^2-c^2)u_x+2acu_y+2ab\over 2abu_x+2bcu_y+b^2-a^2-c^2}, &{}\text{ if } \ \ sgn(\varepsilon )sgn(\theta )<0 \end{array}\right. } \end{aligned}$$
(15)

and

$$\begin{aligned} u_y={\left\{ \begin{array}{ll} u_y,&{}\text{ if } \ \ sgn(\varepsilon )sgn(\theta )>0;\\ {2acu_x+(c^2-a^2-b^2)u_y+2bc\over 2abu_x+2bcu_y+b^2-a^2-c^2}, &{}\text{ if } \ \ sgn(\varepsilon )sgn(\theta )<0, \end{array}\right. } \end{aligned}$$
(16)

where

$$\begin{aligned} a=p_3q_2-p_2q_3, \quad b=p_1q_2-p_2q_1, \quad \mathrm{and}\quad c=p_1q_3-p_3q_1, \end{aligned}$$
(17)

and \(\theta =u_x(p_2q_3-p_3q_2)-u_y(p_1q_3-p_3q_1)+p_2q_1-p_1q_2\). Here the function \(\varepsilon =\varepsilon (x,y)\) is everywhere constant taking values \(\pm 1\). The first pair \((u_x,u_y)\) from (15) and (16) satisfies integrability condition (5) over each \(\varOmega ^{(j)}\) as \(u\in C^2\). On the other hand, the integrability of the second vector field from (15) and (16) eventuates, if and only if:

$$\begin{aligned} \left( {(a^2-b^2-c^2)u_x+2acu_y+2ab\over 2abu_x+2bcu_y+b^2-a^2-c^2}\right) _y= \left( {2acu_x+(c^2-a^2-b^2)u_y+2bc\over 2abu_x+2bcu_y+b^2-a^2-c^2}\right) _x. \end{aligned}$$

Upon resorting to the symbolic computation in Mathematica, the last equation is transformable into the following non-linear PDE:

$$\begin{aligned} c\left( a-bu_x(x,y)\right) u_{yy}(x,y)+ & {} \left( a^2-c^2+bcu_y(x,y)-abu_x(x,y)\right) u_{xy}(x,y)\nonumber \\&\&+a\left( bu_y(x,y)-c\right) u_{xx}(x,y)=0, \end{aligned}$$
(18)

which is to be satisfied by u. Evidently, the latter does not hold generically and therefore the resulting integrability condition (18) disambiguates almost always two-source photometric stereo modelled by (7) - see also Example 1. Still, in a pursue of solving a rare ambiguity in (7), one ought to deal with (18) which inevitably leads to a non-trivial task. Thus in the next step (ii) of this section, a tighter constraint imposed on illumination directions p and q is considered.

(ii) Suppose now that two light-source directions \(p_{\alpha }\) and \(q_{\alpha }\) are introduced according to (13). The resulting two image irradiance equations coincide with:

$$\begin{aligned} {\alpha -\sqrt{1-\alpha ^2}u_x(x,y)\over \sqrt{u_x^2(x,y)+u_y(x,y)^2+1}}=E_1(x,y),\quad {\alpha u_x(x,y)+\sqrt{1-\alpha ^2}\over \sqrt{u_x^2(x,y)+u_y(x,y)^2+1}}=E_2(x,y).\quad \end{aligned}$$
(19)

Since \(p_{\alpha }\perp q_{\alpha }\) (i.e. are orthogonal), the function \(\varLambda \) defined in (9) simplifies into:

$$\begin{aligned} \varLambda (x,y)=1-E_1^2(x,y)-E_2^2(x,y)={u_y^2(x,y)\over 1+u_x^2(x,y)+u_y^2(x,y)}. \end{aligned}$$
(20)

Thus \(\varOmega _{\varLambda >0}=\{(x,y)\in \varOmega : u_y(x,y)\ne 0\}\) and \(\varOmega _{\varLambda \equiv 0}=\{(x,y)\in \varOmega : u_y(x,y)=0\}\). Similarly to the special case of \(p=(0,0,-1)\) and \(q=(q_1,q_2,q_3)\) discussed in [17, 18], often the triples u, \(p_{\alpha }\) and \(q_{\alpha }\) (see Example 1) yield \(\varOmega _{\varLambda >0}=\varOmega ^{(1)}\cup \varOmega ^{(2)}\), with \(\varOmega ^{(j)}\) (\(j=1,2\)) standing for two disjoint sub-domains of \(\varOmega \) and \(\varOmega _{\varLambda \equiv 0}=\varGamma \) representing a smooth curve in \(\varOmega \). Here a bifurcation curve (along which solutions over \(\varOmega ^{(1)}\) and \(\varOmega ^{(2)}\) are glued) coincides with the curve (overlapping with \(\varOmega _{\varLambda \equiv 0}\)):

$$\begin{aligned} \varGamma =\{(x,y)\in \varOmega : u_y(x,y)=0\}. \end{aligned}$$
(21)

Furthermore in (17) , since \(p_2=q_2=0\) then \(a=b=0\) and since \(q_1=-p_3=\alpha \) and \(p_1=q_3=-\sqrt{1-\alpha ^2}\) then \(c=1\). Consequently, both Formulae (15) and (16) are reducible into:

$$\begin{aligned} u_x={\left\{ \begin{array}{ll} u_x,&{}\text{ if } \ \ sgn(\varepsilon )sgn(\theta )>0;\\ &{} \\ u_x, &{}\text{ if } \ \ sgn(\varepsilon )sgn(\theta )<0; \end{array}\right. } \quad \quad u_y={\left\{ \begin{array}{ll} u_y,&{}\text{ if } \ \ sgn(\varepsilon )sgn(\theta )>0;\\ &{} \\ -u_y, &{}\text{ if } \ \ sgn(\varepsilon )sgn(\theta )<0. \end{array}\right. } \end{aligned}$$
(22)

Thus for the non-generic case of the second vector field \((u_x,-u_y)\) also integrable (over each \(\varOmega ^{(i)}\)), the function u should satisfy the following linear PDE:

$$\begin{aligned} u_{xy}(x,y)=0. \end{aligned}$$
(23)

Note that the last equation can be independently reached by substituting \(a=b=0\) and \(c=1\) into (18). The generic case of (23) not fulfilled is illustrated also in Example 1. Upon double integration (first over x and then over y) of (23), the following representation for u (if two vector fields from (22) are to be integrable over \(\varOmega ^{(i)}\)) holds:

$$\begin{aligned} u(x,y)=\phi _1(x)+\psi _1(y), \end{aligned}$$
(24)

for some twice continuously differentiable functions \(\phi _1\) and \(\psi _1\) in a single variable. It is not difficult to show that \(\phi _1(x)=u(x,0)-c\), where \(\psi _1(0)=c\). Similarly, \(\psi (y)=u(0,y)-\phi _1(0)=u(0,y)-u(0,0)+c\). Naturally, an analogous argument applies to the second solution \(v\in C^2\) to (19) resulting in v satisfying (23) and hence \(v(x,y)=\phi _2(x)+\psi _2(y)\), where \(\phi _2\) and \(\psi _2\) are defined similarly to the introduction of \(\phi _1\) and \(\psi _1\). Furthermore, by (22), the second function \(v\in C^2\) (over each \(\varOmega ^{(j)}\)) fulfills \((v_x,v_y)=(u_x,-u_y)\). Combining the latter with \(u_x(x,y)=\phi _1'(x)\), \(u_y(x,y)=\psi _1'(y)\), \(v_x(x,y)=\phi _2'(x)\) and \(v_y(x,y)=\psi _2'(y)\) yields “a conjugate-like” relation between u and v:

$$\begin{aligned} u(x,y)=\phi _1(x)+\psi _1(y),\quad \quad v(x,y)=\phi _1(x)-\psi _1(y)+c_1, \end{aligned}$$
(25)

with \(c_1\) being a constant. Formula (25) determines specific analytic representations of the solutions to (19) over each \(\varOmega ^{(j)}\) in a rare situation of the ambiguous two light-source photometric stereo. A straightforward verification shows that the geometrical constraint from (12) is also preserved for \(p_{\alpha }\) and \(q_{\alpha }\). Indeed, combining (11) with (25) leads to \(K_{u}(x,y)=(\phi _1''(x)\psi _1''(y))(1+(\phi '_1(x))^2+(\psi '_1(y))^2)^{-2}=-K_{v}(x,y).\) Note that the condition (21) coupled with (25) implies that \(\varGamma \) (or more general \(\varOmega _{\varLambda \equiv 0}\)) represents a line \(L=\{(x,y*)\in \varOmega : \psi _1'(y*)=0\}\) (or a collection of lines) parallel to the OX-axis. In addition, as \(\nabla u=(u_x,u_y)\) and \(\nabla v=(u_x,-u_y)\) visibly any critical point of u is also a critical point of v. Moreover, if such point represents a local minimum (maximum, saddle) for u then it is also a local maximum (minimum, saddle) for v. Note also that by (21) any critical point of u (and thus of v) belongs to the set \(\varOmega _{\varLambda \equiv 0}\) and thus to the potential bifurcation curve \(\varGamma \). The non-generic ambiguity case discussed above for \(p_{\alpha }\) and \(q_{\alpha }\) is illustrated in Example 2.

Evidently, as (18) or (23) are generically not fulfilled, there exists only one solution \(u\in C^2\) to (7) or (19) over each \(\varOmega ^{(j)}\). Upon gluing u together, only one global solution \(u\in C^2\) prevails over entire image \(\varOmega \). On the other hand, the rare scenario of the existence of two solutions \(u,v\in C^2\) over each \(\varOmega ^{(j)}\) (i.e. satisfying (23)) leads to the possible bifurcations along \(\varGamma \) rendering 0, 2 or 4 global solutions of class \(C^2\) over entire image \(\varOmega \). Recall that we assume here \(\varOmega =\varOmega ^{(1)}\cup \varOmega ^{(2)}\cup \varGamma \). The other decomposition topologies of \(\varOmega =\varOmega _{\varLambda >0}\cup \varOmega _{\varLambda =0}\) are discussed in [17]. The detailed analysis justifying necessary and sufficient conditions to guarantee successful \(C^k\) bifurcations (\(k=0,1,2\)) (in case of \(\varOmega =\varOmega ^{(1)}\cup \varOmega ^{(2)}\cup \varGamma \)) holding along \(\varGamma \) exceeds the scope of this paper and therefore is here omitted.

Taking into account the above argument, the main theoretical contribution of this paper can be summarized into the following:

Theorem 1

Assume that \(u\in C^2\) (which graph \(S_L=graph(u)\) represents an illuminated genuine Lambertian surface) satisfies (19) or (7) with \(p_{\alpha }\) and \(q_{\alpha }\) determined by (13) (or with arbitrary linearly independent p and q). Suppose, moreover that \(\varLambda \) in (9) satisfies \(\varLambda >0\) over simply-connected \(\varOmega ^{(j)}\) (\(j=1,2\)). In order that there exists only one more solution \(v\in C^2\) to (19) over \(\varOmega ^{(j)}\) it is necessary and sufficient for u to satisfy (23) or (18). In addition, the ambiguous case for \(p_{\alpha }\) and \(q_{\alpha }\) yields \(u(x,y)=\phi (x)+\psi (y)\) and \(v(x,y)=\phi (x)-\psi (y)\), with \(\phi \) and \(\psi \) determined as in (24). Finally, the Gaussian curvatures of graphs of u and v at respective points (x, y, u(x, y)) and (x, y, v(x, y)) (see (11)) satisfy \(K_u(x,y)=-K_v(x,y)\).

As already pointed out, a local ambiguity to (19) or (7) (over \(\varOmega _{\varLambda >0}\)) can even be more proliferated to a global one (over entire image \(\varOmega \)) due to possible bifurcations of u and v along \(\varOmega _{\varLambda \equiv 0}\). The matter gets more complicated if \(\varOmega _{\varLambda \equiv 0}\) forms an open subset of \(\varOmega \) (see [17, 18]). The latter occurs once \(\langle p\times q|n \rangle =0\) as implied by (14). Due to these intricacies, the respective discussion on bifurcation issue in two light-source photometric stereo is here left out.

3 Experiments

This section includes two examples illustrating the main results established in Sect. 2 (see also Theorem 1). The experiments presented here are carried out with the aid of Mathematica symbolic computation. The corresponding pictures of images (with the respective intensities \(E_1\) and \(E_1\)) are simulated synthetically upon admitting arbitrary or specific illumination directions and assuming a genuine u as temporarily initially given.

Example 1

(a) Consider a Lambertian hemi-sphere \(S_L^{1}=graph(u_1)\) with \(u_1\in C^2(\hat{\varOmega })\) defined as \(u_1(x,y)=\sqrt{R^2-x^2-y^2}\), where \(\hat{\varOmega }=\{(x,y)\in {\mathbb {R}}^2: x^2+y^2\le R^2\}\). For two linearly independent normalized light-source directions p and q the respective image irradiance equations read as:

$$\begin{aligned} {p_1u_x+p_2u_y-p_3\over \sqrt{u_x^2+u_y^2+1}}= & {} E_{11}(x,y)={-p_1x-p_2y-p_3\sqrt{R^2-x^2-y^2}\over R},\nonumber \\ {q_1u_x+q_2u_y-q_3\over \sqrt{u_x^2+u_y^2+1}}= & {} E_{21}(x,y)={-q_1x-q_2y-q_3\sqrt{R^2-x^2-y^2}\over R}, \end{aligned}$$
(26)

over \(\varOmega =\varOmega _1\cap \varOmega _2\cap \hat{\varOmega }\), where \(\varOmega _i=\{(x,y)\in {\mathbb {R}}^2: E_{i1}^1(x,y)\ge 0\}\) (for \(i=1,2\)). The negative values of \(E_{i1}\) represent shadowed subareas of \(\hat{\varOmega }\). The bifurcation curve \(\varGamma \) (see (14)) reduces into a planar quadratic determined by:

$$\begin{aligned} (p_3q_2-p_2q_3)x-(p_3q_1-p_1q_3)y+(p_2q_1-p_1q_2)\sqrt{R^2-x^2-y^2}=0. \end{aligned}$$
(27)

Furthermore, the condition (18) ascertaining the existence of exactly one solution \(u_1\in C^2(\varOmega )\) (modulo its vertical shift) to (26) enforces \(u_1\) to satisfy:

$$\begin{aligned} (cx-ay)\left( ax+cy-b\sqrt{R^2-x^2-y^2}\right) =0, \end{aligned}$$
(28)

over \(\varOmega ^{(j)}\), with a, b and c defined as in (17). Clearly, the Eq. (28) is not satisfied by \(u_1\) for all \((x,y)\in \varOmega ^{(j)}\). This yields uniqueness in solving (26) (i.e. \(u=u_1\)) within \(u\in C^2(\varOmega )\). In particular, for \(p_{\alpha }\) and \(q_{\alpha }\) (see (13)), (17) combined with \(a=b=0\) and \(c=1\) reduce (27) into \(y=0\) rendering \(\varGamma =\{(x,y)\in \hat{\varOmega }: y=0\}\), \(\varOmega ^{(1)}=\{(x,y)\in \varOmega : y<0\}\) and \(\varOmega ^{(2)}=\{(x,y)\in \varOmega : y>0\}\). The ambiguity condition (28) is transformed into \(xy=0\) merely fulfilled along both X- and Y-axes. Hence, again for arbitrary \(p_{\alpha }\) and \(q_{\alpha }\), uniqueness of \(u_1\) prevails. This is expected since \(p_{\alpha }\) and \(q_{\alpha }\) represents a special case of general positions of p and q analyzed above.

(b) Let a Lambertian hill-like surface \(S_L^{2}=graph(u_2)\) with \(u_2\in C^2(\hat{\varOmega })\) be defined according to \(u_2(x,y)=(2(1+x^2+y^2))^{-1}\), over e.g. \(\hat{\varOmega }=\{(x,y)\in {\mathbb {R}}^2: |x|\le 1 \ \mathrm{and} \ |y|\le 1\}\). The respective two image irradiance equations read as:

$$\begin{aligned} {p_1u_x+p_2u_y-p_3\over \sqrt{u_x^2+u_y^2+1}}= & {} E_{12}(x,y)={-p_1x-p_2y-p_3(x^2+y^2+1)^2\over \sqrt{x^2+y^2+(x^2+y^2+1)^4}},\\\\{q_1u_x+q_2u_y-q_3\over \sqrt{u_x^2+u_y^2+1}}= & {} E_{22}(x,y)={-q_1x-q_2y-q_3(x^2+y^2+1)^2\over \sqrt{x^2+y^2+(x^2+y^2+1)^4}}, \end{aligned}$$
(29)

over unshadowed \(\varOmega =\varOmega _1\cap \varOmega _2\cap \hat{\varOmega }\), where \(\varOmega _i=\{(x,y)\in {\mathbb {R}}^2: E_{i2}^1(x,y)\ge 0\}\) (for \(i=1,2\)). The bifurcation curve \(\varGamma \) from (14) is defined by the following equation:

$$\begin{aligned} (p_2q_1-p_1q_2)+{(p_3q_2-p_2q_3)x\over 2(1+x^2+y^2)^{3/2}} +{(p_3q_1-p_1q_3)y\over 2(1+x^2+y^2)^{3/2}}=0. \end{aligned}$$
(30)

On the other hand, the integrability condition (18) stipulates \(u_2\) to satisfy

$$\begin{aligned} (cx-ay)\left( b+6(ax+cy)\sqrt{1+x^2+y^2}\right) =0, \end{aligned}$$
(31)

over \(\varOmega ^{(j)}\), with a, b and c introduced as in (17). Again, (31) is not fulfilled by \(u_2\) for all \((x,y)\in \varOmega ^{(j)}\). Hence there exists a unique solution of class \(C^2\) to (29) (i.e. \(u=u_2\)) over \(\varOmega \). In the special case of \(p_{\alpha }\) and \(q_{\alpha }\) (see (13)), (17) coupled with \(a=b=0\) and \(c=1\) transform (30) into \(y=0\). Hence again \(\varGamma =\{(x,y)\in \hat{\varOmega }: y=0\}\), \(\varOmega ^{(1)}=\{(x,y)\in \varOmega : y<0\}\) and \(\varOmega ^{(2)}=\{(x,y)\in \varOmega : y>0\}\). Furthermore the ambiguity constraint (31) is reduced into \(xy=0\) which holds again only along both X- and Y-axes. Thus for \(p_{\alpha }\) and \(q_{\alpha }\) the uniqueness of \(u_2\) to (29) eventuates (as also follows from the above general (p, q)-position analysis).

Fig. 1.
figure 1

Two images of \(S_L^3=graph(u_3)\) illuminated along \(p_{1/\sqrt{2}}\) and \(q_{1/\sqrt{2}}\) directions.

Full size image
Fig. 2.
figure 2

Decomposition of \(\varOmega _{\alpha }=\varOmega ^{(1)}_{\alpha }\cup \varOmega ^{(2)}_{\alpha }\cup \varGamma _{\alpha }\) for (32) with (a) \(\alpha =1/\sqrt{2}\), (b) \(\alpha =1/5\).

Full size image
Fig. 3.
figure 3

Two \(C^2\) class global solutions \(u_3\) and \(u_4\) to (32), over \(\varOmega _{1/\sqrt{2}}=[-1,1]\times [-1,1]\).

Full size image
Fig. 4.
figure 4

Two \(C^1\) class global solutions \(u_{34}\) and \(u_{43}\) to (32), over \(\varOmega _{1/\sqrt{2}}=[-1,1]\times [-1,1]\).

Full size image

Example 2

Consider now a Lambertian paraboloid \(S_L^{3}=graph(u_3)\) with \(u_3\in C^2(\hat{\varOmega })\) defined as \(u_3(x,y)=(x^2+y^2)/2\) over \(\hat{\varOmega }=\{(x,y)\in {\mathbb {R}}^2: |x|\le 1 \ \mathrm{and} \ |y|\le 1\}\). For a general position of p and q, the integrability condition (18) reads here as \(b(ay-cx)=0\). Provided \(b\ne 0\), such constraint never holds over any open subset of \(\varOmega \) and thus uniqueness of \(u_3\) follows. Noticeably the bifurcation curve \(\varGamma \) (a line) is determined here by \(p_2q_1-p_1q_2+(p_2q_3-p_3q_2)x+(p_3q_1-p_1q_3)y=0\).

For \(p_{\alpha }\) and \(q_{\alpha }\) from (13) the resulting two image irradiance equations are:

$$\begin{aligned} {\alpha -\sqrt{1-\alpha ^2}u_x(x,y)\over \sqrt{u_x^2(x,y)+u_y(x,y)^2+1}}= & {} E_{13}(x,y)={\alpha -\sqrt{1-\alpha ^2}x\over \sqrt{x^2+y^2+1}},\\\\{\alpha u_x(x,y)+\sqrt{1-\alpha ^2}\over \sqrt{u_x^2(x,y)+u_y(x,y)^2+1}}= & {} E_{23}(x,y)={\alpha x+\sqrt{1-\alpha ^2}\over \sqrt{x^2+y^2+1}}, \end{aligned}$$
(32)

over \(\varOmega _{\alpha }=\hat{\varOmega }\) (for \(\alpha =1/\sqrt{2}\)) and over \(\varOmega _{\alpha }=\{(x,y)\in \hat{\varOmega }_{\alpha }: -1 \le x\le 0.204\}\) (for \(\alpha =1/5\)). Figure 1 shows images of \(S_L^3\) with \(\alpha =1/\sqrt{2}\) over \(\varOmega _{1/\sqrt{2}}=\hat{\varOmega }=[-1,1]\times [-1,1]\). The bifurcation curve \(\varGamma _{\alpha }\) in (21) coincides here with the X–axis (i.e. here \(y=0\)) trimmed either to \(-1\le x\le 1\) or to \(-1\le x\le 0.204\), for \(\alpha =1/\sqrt{2}\) or \(\alpha =1/5\), respectively. The remaining decomposition components of \(\varOmega _{\alpha }\) (along which \(\varLambda >0\)) read as \(\varOmega ^{(1)}_{\alpha }=\{(x,y)\in \varOmega _{\alpha }: y>0\}\) and \(\varOmega ^{(2)}_{\alpha }=\{(x,y)\in \varOmega _{\alpha }: y<0\}\) - see Fig. 2. Furthermore, since by (17) the constant \(b=0\), the ambiguity condition \(b(ay-cx)=0\) is now satisfied. Hence, upon combining (25) with \(\phi (x)=x^2/2\) and \(\psi (y)=y^2/2\), the only one another \(C^2\) solution to (32) equals to \(u_4(x,y)=(x^2-y^2)/2\) (modulo a vertical shift), over each \(\varOmega _{\alpha }^{(j)}\) (here \(j=1,2\)). Once \(u_k\) are glued with itself (\(k=3,4\)) along X-axis, two \(C^2\) class global solutions to (32) over entire image \(\varOmega _{\alpha }\) are defined (i.e. \(u_3\) and \(u_4\)) - see Fig. 3 for \(\alpha =1/\sqrt{2}\) (with gluing curve). On the other hand the local solutions \(u_3\) (or \(u_4\)) over \(\varOmega ^{(1)}_{\alpha }\) cross-bifurcate along \(\varGamma _{1/\sqrt{2}}\) with \(u_4\) (or with \(u_3\)) over \(\varOmega ^{(2)}_{\alpha }\) to yield next two only \(C^1\) class solutions \(u_{34}\) and \(u_{43}\) to (32) over \(\varOmega _{\alpha }\). Indeed, a \(C^2\) differentiability is excluded as \(\lim _{(x,y)\rightarrow (x,0)}u_{3yy}(x,y)=2\ne -2=\lim _{(x,y)\rightarrow (x,0)}u_{4yy}(x,y)\). The remaining two \(C^1\) global solutions to (32) over \(\varOmega _{1/\sqrt{2}}\) (i.e. \(u_{34}\) and \(u_{43}\)) are plotted in Fig. 4. The case for \(\alpha =1/5\) differs merely by different \(\varOmega _{1/5}\). Note that by (11) the respective Gaussian curvatures yield \(K_{u_3}(x,y)=4(1+x^2+y^2)^{-2}=-K_{u_4}(x,y)\) which is consistent with Theorem 1. A unique critical point (0, 0) of \(u_3,u_4,u_{34}\) and \(u_{43}\) belongs, as expected to \(\varGamma \).

4 Conclusion

This paper extends the claims of [17, 18], where a special configuration of two light-source directions \(p=(0,0,-1)\) and \(q=(q_1,q_2,q_3)\) (for \(q_1^2+q_2^2>0\)) in photometric stereo is studied. In particular, a respective integrability condition expressed by the non-linear PDE (18) is derived here for a general configuration of linearly independent p and q. Not unexpectedly, such PDE forms a difficult theoretical problem and consequently a new family of orthogonal illumination directions \(p_{\alpha }\) and \(q_{\alpha }\) is introduced in (13). The corresponding system of two image irradiance Eq. (19) is subsequently analyzed and the resulting ambiguity versus uniqueness question is addressed in Theorem 1. As proved in this paper, similar ambiguity results are obtainable for \(p_{\alpha }\) and \(q_{\alpha }\) defined in (13) (see Theorem 1 and Sect. 1) as compared to those already established in [17, 18] for \(p=(0,0,-1)\) and \(q=(q_1,q_2,q_3)\). The experiments reported in Sect. 3 illustrate the main results from Sect. 2. A possible extension of this paper includes a relevant ambiguity analysis performed for the general positions of two linearly independent light-source directions. Inevitably, any pending argument would rely on characterizing the multiple solutions to the non-linear PDE defined in (18). Another worth investigation venue is to complete a global uniqueness-ambiguity analysis for two image photometric stereo over entire image \(\varOmega \) (see also [17, 18]). The latter should first cover the case of \(p_{\alpha }\) and \(q_{\alpha }\) defined in (13) and then should refer to an arbitrary selection of two linearly independent illumination directions. In particular, an extra attention should be paid here in derivation of analytical and numerical methods designed to localize possible bifurcation curve(s) \(\varGamma \). Some relevant clues concerning this task can be found in [24–28, 30]. Finally, a similar analysis based on real Lambertian images, where pixelization and noise occur, forms another vital extension topic of this paper. Usually, handling noisy and digitized image data requires a more robust integration techniques dealing computationally with u recovery phase determined by theoretical formula (4). Such analytic step is often substituted by the pertinent optimization task designed to compute numerically the closest discrete integrable vector field - for more see e.g. [4, 5, 29].