Perspective Shape from Shading

Abstract

Shape from Shading (SFS) is a fundamental task in computer vision. By given information about the reflectance of an object’s surface and the position of the light source, the SFS problem is to reconstruct the 3D depth of the object from a single grayscale 2D input image. A modern class of SFS models relies on the property that the camera performs a perspective projection. The corresponding perspective SFS methods have been the subject of many investigations within the last years. The goal of this chapter is to give an overview of these developments. In our discussion, we focus on important model aspects, and we investigate some prominent algorithms appearing in the literature in more detail than it was done in previous works.

Appendices

Appendices

Appendix 1

The irradiance equation

$$\begin{aligned} I_{0} =\frac{ \left| z_{0}A_3+B_3\right| }{ \sqrt{ \left( z_{0}A_1+B_1\right) ^2+ \left( z_{0}A_2+B_2\right) ^2+ \left( z_{0}A_3+B_3\right) ^2 } } \end{aligned}$$

is reformulated as

$$\begin{aligned} I^2_{0}\left( \left( z_{0}A_1+B_1\right) ^2+ \left( z_{0}A_2+B_2\right) ^2+ \left( z_{0}A_3+B_3\right) ^2 \right) = \left( z_{0}A_3+B_3\right) ^2 \end{aligned}$$

and further simplified as

$$\begin{aligned} I^2_{0}(&\cdots \nonumber \\&\left( z_{0}^2A_1^2+B_1^2+2z_{0}A_1B_1\right) +\nonumber \\&\left( z_{0}^2A_2^2+B_2^2+2z_{0}A_2B_2\right) +\nonumber \\&\left( z_{0}^2A_3^2+B_3^2+2z_{0}A_3B_3\right) \nonumber \\&\cdots )= z_{0}^2A_3^2+B_3^2+2z_{0}A_3B_3. \end{aligned}$$

(2.45)

By distributing the $I^2_{0}$, on rewrites (2.45) as

$$\begin{aligned}&z_{0}^2A_1^2I^2_{0}+B_1^2I^2_{0}+2z_{0}A_1B_1I^2_{0}+\nonumber \\&z_{0}^2A_2^2I^2_{0}+B_2^2I^2_{0}+2z_{0}A_2B_2I^2_{0}+\nonumber \\&z_{0}^2A_3^2I^2_{0}+B_3^2I^2_{0}+2z_{0}A_3B_3I^2_{0}-\nonumber \\&z_{0}^2A_3^2+B_3^2+2z_{0}A_3B_3=0. \end{aligned}$$

(2.46)

Rearranging (2.46) while factoring out $z^2_{0}$ and $z_0$ results it to be written as

$$\begin{aligned}&\overbrace{ \left( I^2_{0} \left( A_1^2+ A_2^2+ A_3^2 \right) -A_3^2 \right) }^{C_1} z_{0}^2+\nonumber \\&\overbrace{\left( 2I^2_{0}\left( A_1B_1+A_2B_2+A_3B_3\right) -2A_3B_3\right) }^{C_2}z_{0}+\nonumber \\&\overbrace{\left( I^2_{0}\left( B_1^2+B_"^2+B_3^2\right) -B_3^2\right) }^{C_3}=0, \end{aligned}$$

(2.47)

and finally the image irradiance equation is reformulated in the form of the quadratic equation

$$\begin{aligned} C_1z_{0}^2+C_2z_{0}+C_3=0. \end{aligned}$$

(2.48)

Appendix 2

In this case, the term $\left( P_a-P_0\right) ^\top$ is redefined as

$$\begin{aligned} P_a-P_0= \begin{pmatrix} v_bz_b-v_0z_0\\ -\left( u_bz_b-u_0z_0\right) \\ 0 \end{pmatrix} \end{aligned}$$

(2.49)

restricting the wave-front to only propagate from the direction of $P_b$, namely $\left( P_a-P_0\right) \cdot \left( P_b-P_0\right) =0$, leading the derivation of the normal N to proceed as

$$\begin{aligned} \mathbf {N}=\left( P_a-P_0\right) ^\top \times \left( P_b-P_0\right) ^\top =&\begin{pmatrix} v_bz_b-v_0z_0\\ -\left( u_bz_b-u_0z_0\right) \\ 0 \end{pmatrix} \times \begin{pmatrix}\frac{u_bz_b-u_0z_0}{f}\\ \frac{v_bz_b-v_0z_0}{f}\\ z_b-z_0 \end{pmatrix}. \end{aligned}$$

(2.50)

Now by letting

$$\begin{aligned} x_b:=\left( u_bz_b-u_0z_0\right) \text {and} y_b:=v_bz_b-v_0z_0 \end{aligned}$$

one writes (2.50) as

(2.51)

as the normal vector to the surface point $P_0$ in case of the degenerated case $\eta _1=+\infty$.

Appendix 3

Because $\left( u_b,v_b\right)$ is a neighbor of $\left( u_0,v_0\right)$, one can write $x_b$ and $y_b$ as

$$\begin{aligned} x_b = u_0\left( z_b-z_0\right) + \varDelta _1 z_b \end{aligned}$$

(2.52)

and

$$\begin{aligned} y_b = v_0\left( z_b-z_0\right) + \varDelta _2 z_b \end{aligned}$$

(2.53)

with $\left( \varDelta _1,\varDelta _2\right) \in \left\{ \left( 0,\pm 1\right) ,\left( \pm 1,0\right) \right\}$, and substitute them into the numerator of the irradiance image

$$\begin{aligned} I_{0} =\frac{ x_b^2+y_b^2 }{ \sqrt{ f^2x_b^2\left( z_b-z_0\right) ^2+ f^2y_b^2\left( z_b-z_0\right) ^2+ \left( x_b^2+y_b^2\right) ^2 } } \end{aligned}$$

to get

$$\begin{aligned} I_{0}=\frac{ \overbrace{ \left( u_0\left( z_b-z_0\right) + \varDelta _1 z_b\right) ^2 }^{x_b^2}+ \overbrace{ \left( v_0\left( z_b-z_0\right) + \varDelta _2 z_b\right) ^2 }^{y_b^2} }{ \sqrt{ f^2x_b^2\left( z_b-z_0\right) ^2+ f^2y_b^2\left( z_b-z_0\right) ^2+ \left( x_b^2+y_b^2\right) ^2 } } \end{aligned}$$

that is expanded as

$$\begin{aligned} I_{0}= \frac{ \left. {\left\{ \begin{array}{ll} u_0^2\left( z_b-z_0\right) ^2 + \varDelta _1^2 z_b^2 + 2 u_0\left( z_b-z_0\right) \varDelta _1 z_b +\cdots \\ v_0^2\left( z_b-z_0\right) ^2 + \varDelta _2^2 z_b^2 + 2 v_0\left( z_b-z_0\right) \varDelta _2 z_b \end{array}\right. } \right\} }{ \sqrt{ f^2x_b^2\left( z_b-z_0\right) ^2+ f^2y_b^2\left( z_b-z_0\right) ^2+ \left( x_b^2+y_b^2\right) ^2 } }. \end{aligned}$$

(2.54)

In addition, by factoring $f^2\left( z_b-z_0\right)$ from the first two terms of the (2.54) denominator, we have

$$\begin{aligned} I_{0}= \frac{ \left. {\left\{ \begin{array}{ll} u_0^2\left( z_b-z_0\right) ^2 + \varDelta _1^2 z_b^2 + 2 u_0\left( z_b-z_0\right) \varDelta _1 z_b +\cdots \\ v_0^2\left( z_b-z_0\right) ^2 + \varDelta _2^2 z_b^2 + 2 v_0\left( z_b-z_0\right) \varDelta _2 z_b \end{array}\right. } \right\} }{ \sqrt{ f^2\left( z_b-z_0\right) ^2\left( x_b^2+y_b^2\right) + \left( x_b^2+y_b^2\right) ^2 } } \end{aligned}$$

that is more simplified in its denominator as

$$\begin{aligned} I_{0}= \frac{ \left. {\left\{ \begin{array}{ll} u_0^2\left( z_b-z_0\right) ^2 + \varDelta _1^2 z_b^2 + 2 u_0\left( z_b-z_0\right) \varDelta _1 z_b +\cdots \\ v_0^2\left( z_b-z_0\right) ^2 + \varDelta _2^2 z_b^2 + 2 v_0\left( z_b-z_0\right) \varDelta _2 z_by_b \end{array}\right. } \right\} }{ \sqrt{ \left( x_b^2+y_b^2\right) \left( f^2\left( z_b-z_0\right) ^2+\left( x_b^2+y_b^2\right) \right) } }. \end{aligned}$$

(2.55)

Taking both sides of (2.55) to the power of 2, we are lead to

$$\begin{aligned} I_{0}^2= \frac{ \left. {\left\{ \begin{array}{ll} u_0^2\left( z_b-z_0\right) ^2 + \varDelta _1^2 z_b^2 + 2 u_0\left( z_b-z_0\right) \varDelta _1 z_b +\cdots \\ v_0^2\left( z_b-z_0\right) ^2 + \varDelta _2^2 z_b^2 + 2 v_0\left( z_b-z_0\right) \varDelta _2 z_b \end{array}\right. } \right\} ^2\equiv \left( x_b^2+y_b^2\right) ^2 }{ \left( x_b^2+y_b^2\right) \left( f^2\left( z_b-z_0\right) ^2+\left( x_b^2+y_b^2\right) \right) } \end{aligned}$$

letting us to have the image irradiance equation as

$$\begin{aligned} I_{0}^2= \frac{ \left. {\left\{ \begin{array}{ll} u_0^2\left( z_b-z_0\right) ^2 + \varDelta _1^2 z_b^2 + 2 u_0\left( z_b-z_0\right) \varDelta _1 z_b +\cdots \\ v_0^2\left( z_b-z_0\right) ^2 + \varDelta _2^2 z_b^2 + 2 v_0\left( z_b-z_0\right) \varDelta _2 z_b \end{array}\right. } \right\} }{ f^2\left( z_b-z_0\right) ^2+\left( x_b^2+y_b^2\right) }. \end{aligned}$$

Now, all terms are taken to the same side

$$\begin{aligned} \left. {\left\{ \begin{array}{ll} u_0^2\left( z_b-z_0\right) ^2 + \varDelta _1^2 z_b^2 + 2 u_0\left( z_b-z_0\right) \varDelta _1 z_b +\cdots \\ v_0^2\left( z_b-z_0\right) ^2 + \varDelta _2^2 z_b^2 + 2 v_0\left( z_b-z_0\right) \varDelta _2 z_b -\cdots \\ I_{0}^2f^2\left( z_b-z_0\right) ^2-I_{0}^2\left( x_b^2+y_b^2\right) \end{array}\right. } \right\} =0 \end{aligned}$$

and further simplified based on the common factor $\left( z_b-z_0\right)$ as

$$\begin{aligned} \left. {\left\{ \begin{array}{ll} \left( u_0^2+v_0^2-I_{0}^2f^2\right) \left( z_b-z_0\right) ^2 +\cdots \\ \left( 2 u_0\varDelta _1 z_b+2 v_0\varDelta _2 z_b\right) \left( z_b-z_0\right) +\cdots \\ \varDelta _1^2 z_b^2+ \varDelta _2^2 z_b^2 - I_{0}^2\left( x_b^2+y_b^2\right) \end{array}\right. } \right\} =0. \end{aligned}$$

(2.56)

To this end, once again the terms $x_b$ and $y_b$ in (2.56) need to be replaced by (2.52) and (2.53) as

$$\begin{aligned} \left. {\left\{ \begin{array}{ll} \left( u_0^2+v_0^2-I_{0}^2f^2\right) \left( z_b-z_0\right) ^2 +\cdots \\ \left( 2 u_0\varDelta _1 z_b+2 v_0\varDelta _2 z_b\right) \left( z_b-z_0\right) +\cdots \\ \varDelta _1^2 z_b^2+ \varDelta _2^2 z_b^2 -I_{0}^2 \left. {\left\{ \begin{array}{ll} \overbrace{u_0^2\left( z_b-z_0\right) ^2 + \varDelta _1^2 z_b^2 + 2 u_0\left( z_b-z_0\right) \varDelta _1 z_b}^{x_b^2} +\cdots \\ \underbrace{v_0^2\left( z_b-z_0\right) ^2 + \varDelta _2^2 z_b^2 + 2 v_0\left( z_b-z_0\right) \varDelta _2 z_b}_{y_b^2} \end{array}\right. } \right\} \end{array}\right. } \right\} =0 \end{aligned}$$

and once again rearranged based on $\left( z_b-z_0\right)$ and $\left( z_b-z_0\right) ^2$ as

$$\begin{aligned} \left. {\left\{ \begin{array}{ll} \left( \left( u_0^2+v_0^2-I_{0}^2f^2\right) -I_{0}^2\left( u_0^2+v_0^2\right) \right) \left( z_b-z_0\right) ^2 +\cdots \\ \left( 2 u_0\varDelta _1 z_b+2 v_0\varDelta _2 z_b -I_{0}^2 2 u_0\varDelta _1 z_b -I_{0}^2 2 v_0\varDelta _2 z_b \right) \left( z_b-z_0\right) +\cdots \\ \varDelta _1^2 z_b^2+ \varDelta _2^2 z_b^2 -I_{0}^2\varDelta _1^2 z_b^2 -I_{0}^2\varDelta _2^2 z_b^2 \end{array}\right. } \right\} =0 \end{aligned}$$

or

$$\begin{aligned} \left. {\left\{ \begin{array}{ll} \left( \left( u_0^2+v_0^2-I_{0}^2f^2\right) -I_{0}^2\left( u_0^2+v_0^2\right) \right) \left( z_b-z_0\right) ^2 +\cdots \\ \left( 2 u_0\varDelta _1 z_b\left( 1-I_{0}^2\right) +2 v_0\varDelta _2 z_b\left( 1-I_{0}^2\right) \right) \left( z_b-z_0\right) +\cdots \\ \varDelta _1^2 z_b^2\left( 1-I_{0}^2\right) + \varDelta _2^2 z_b^2\left( 1-I_{0}^2\right) \end{array}\right. } \right\} =0 \end{aligned}$$

that leads to

$$\begin{aligned} \left. {\left\{ \begin{array}{ll} \overbrace{ \left( \left( u_0^2+v_0^2-I_{0}^2f^2\right) -I_{0}^2\left( u_0^2+v_0^2\right) \right) }^{D_1}\left( z_b-z_0\right) ^2 +\cdots \\ \overbrace{ \left( \left( 2 u_0\varDelta _1 z_b+2 v_0\varDelta _2 z_b\right) \left( 1-I_{0}^2\right) \right) }^{D_2}\left( z_b-z_0\right) +\cdots \\ \overbrace{ \left( \varDelta _1^2 z_b^2+\varDelta _2^2 z_b^2\right) \left( 1-I_{0}^2\right) }^{D_3} \end{array}\right. } \right\} =0 \end{aligned}$$

and finally written as a quadratic equation

$$\begin{aligned} D_1 \left( z_b-z_0\right) ^2 + D_2 \left( z_b-z_0\right) + D_3 = 0 \end{aligned}$$

(2.57)

with below coefficients$:$

$$\begin{aligned} D_1 :=\left( u_0^2+v_0^2\right) -I_{0}^2\left( f^2+u_0^2+v_0^2\right) , \quad D_2 :=2z_b\left( u_0\varDelta _1 + v_0\varDelta _2 \right) \left( 1-I_{0}^2\right) , \end{aligned}$$

$$\begin{aligned} D_3 :=\left( \varDelta _1^2 z_b^2+\varDelta _2^2 z_b^2\right) \left( 1-I_{0}^2\right) . \end{aligned}$$

Appendix 4

Steps in the direction of normal vector derivation by Tankus et al. [54]$:$

$$\begin{aligned} \mathbf{n}&{\mathop {=}\limits ^{}} \left( \frac{d}{du}C_1\left( u\right) \right) \times \left( \frac{d}{dv}C_2\left( v\right) \right) \nonumber \\&{\mathop {=}\limits ^{}} \frac{1}{f} \begin{pmatrix} -z-uz_u\\ -v_0z_u\\ fz_u \end{pmatrix} \times \frac{1}{f} \begin{pmatrix} -u_0z_v\\ -z-vz_v\\ fz_v \end{pmatrix}\nonumber \\&{\mathop {=}\limits ^{}} \frac{1}{f^2} \begin{pmatrix} -z-uz_u\\ -v_0z_u\\ fz_u \end{pmatrix} \times \begin{pmatrix} -u_0z_v\\ -z-vz_v\\ fz_v \end{pmatrix}\nonumber \\&{\mathop {=}\limits ^{}} \frac{1}{f^2} \begin{pmatrix} -vz_u\cdot fz_v+fz_u\left( z+vz_v\right) \\ -fz_u\cdot uz_v+\left( z+uz_u\right) fz_v\\ \left( z+uz_u\right) \left( z+vz_v\right) -vz_u\cdot uz_v \end{pmatrix}\nonumber \\&{\mathop {=}\limits ^{}} \frac{1}{f^2} \begin{pmatrix} -fvz_uz_v+fzz_u+fvz_uz_v\\ -fuz_uz_v+fzz_v+fuz_uz_v\\ z^2+vzz_v+uzz_u+uvz_uz_v-uvz_uz_v \end{pmatrix}\nonumber \\&{\mathop {=}\limits ^{}} \frac{1}{f^2} \begin{pmatrix} fzz_u\\ fzz_v\\ z^2+vzz_v+uzz_u \end{pmatrix}\nonumber \\&{\mathop {=}\limits ^{}} \frac{1}{f^2} \begin{pmatrix} fzz_u\\ fzz_v\\ z^2+z\left( vz_v+uz_u\right) \end{pmatrix}\nonumber \\&{\mathop {=}\limits ^{}} \frac{z}{f^2} \begin{pmatrix} fz_u\\ fz_v\\ z+vz_v+uz_u \end{pmatrix}. \end{aligned}$$

(2.58)

Based on (2.58), the unit normal vector is found as

$$\begin{aligned} \hat{\mathbf{n }}&{\mathop {=}\limits ^{}} \frac{\mathbf{n }}{\Vert \mathbf{n }\Vert }\nonumber \\&{\mathop {=}\limits ^{}} \frac{\frac{z}{f^2}\left( fz_u,fz_v,z+vz_v+uz_u\right) }{\sqrt{f^2z_u^2\frac{z^2}{f^4}+f^2z_v^2\frac{z^2}{f^4}+\left( z+vz_v+uz_u\right) ^2}\frac{z^2}{f^4}}\nonumber \\&{\mathop {=}\limits ^{}} \frac{\frac{z}{f^2}\left( fz_u,fz_v,z+vz_v+uz_u\right) }{\frac{z}{f^2} \sqrt{f^2z_u^2+f^2z_v^2+\left( z+vz_v+uz_u\right) ^2}} \nonumber \\&{\mathop {=}\limits ^{}} \frac{\left( fz_u,fz_v,z+vz_v+uz_u\right) }{\sqrt{f^2\left( z_u^2+z_v^2\right) +\left( z+vz_v+uz_u\right) ^2}}. \end{aligned}$$

(2.59)

Appendix 5

Steps to derive the image irradiance equation (2.37) proposed by Tankus et al. [54] and based on (2.34), (2.35) and (2.36).

$$\begin{aligned} I&{\mathop {=}\limits ^{}} \frac{\left( u-fp_s\right) z_u+\left( v-fq_s\right) z_v+z}{\Vert L\Vert \sqrt{f^2\left( z_u^2+z_v^2\right) +\left( z+vz_v+uz_u\right) ^2}}\\&{\mathop {=}\limits ^{}} \frac{\left( u-fp_s\right) pz+\left( v-fq_s\right) qz+z}{\Vert L\Vert \sqrt{f^2 \left( \underbrace{ \left( pz\right) ^2 }_{z_u^2} +\underbrace{ \left( qz\right) ^2 }_{z_v^2} \right) +\left( z+vqz+upz\right) ^2 } }\\&{\mathop {=}\limits ^{}} \frac{\left( u-fp_s\right) pz+\left( v-fq_s\right) qz+z}{\Vert L\Vert \sqrt{f^2 \underbrace{ \left( p^2z^2+q^2z^2\right) }_{\beta } +\left( z+vqz+upz\right) ^2 } }\\&{\mathop {=}\limits ^{}} \frac{\left( u-fp_s\right) pz+\left( v-fq_s\right) qz+z}{\Vert L\Vert \sqrt{f^2z^2 \underbrace{ \left( p^2+q^2\right) }_{\beta } +\left( z+vqz+upz\right) ^2 } }\\&{\mathop {=}\limits ^{}} \frac{\left( u-fp_s\right) pz+\left( v-fq_s\right) qz+z}{\Vert L\Vert \sqrt{f^2z^2\beta +\left( z+vqz+upz\right) ^2 } }\\&{\mathop {=}\limits ^{}} \frac{\left( u-fp_s\right) pz+\left( v-fq_s\right) qz+z}{\Vert L\Vert \sqrt{f^2z^2\beta +\left( z^2+v^2q^2z^2+u^2p^2z^2+2vqz^2+2upz^2+2uvpqz^2\right) } }\\&{\mathop {=}\limits ^{}} \frac{\left( u-fp_s\right) pz+\left( v-fq_s\right) qz+z}{\Vert L\Vert \sqrt{f^2z^2\beta +z^2\left( 1+v^2q^2+u^2p^2+2vq+2up+2uvpq\right) } }\\&{\mathop {=}\limits ^{}} \frac{\left( u-fp_s\right) pz+\left( v-fq_s\right) qz+z}{\Vert L\Vert \sqrt{f^2z^2\beta +z^2\left( up+vq+1\right) ^2 } }\\&{\mathop {=}\limits ^{}} \frac{z\left( \left( u-fp_s\right) p+\left( v-fq_s\right) q+1\right) }{z\Vert L\Vert \sqrt{f^2\beta +\left( up+vq+1\right) ^2 } }\\&{\mathop {=}\limits ^{}} \frac{\left( u-fp_s\right) p+\left( v-fq_s\right) q+1}{\Vert L\Vert \sqrt{f^2\beta +\left( up+vq+1\right) ^2 } }\\&{\mathop {=}\limits ^{}} \frac{\left( u-fp_s\right) p+\left( v-fq_s\right) q+1}{\Vert L\Vert \sqrt{f^2\left( p^2+q^2\right) +\left( up+vq+1\right) ^2 } }\\&{\mathop {=}\limits ^{}} \frac{\left( u-fp_s\right) p+\left( v-fq_s\right) q+1}{\Vert L\Vert \sqrt{f^2\left( p^2+q^2\right) +\left( up+vq+1\right) ^2 } }. \end{aligned}$$

Appendix 6

Steps to further simplify the image irradiance equation (2.37) of Tankus et al. [54] to the form shown in (2.38) proceeds by letting both sides of (2.37) to the power of 2 as

$$\begin{aligned} I^2&{\mathop {=}\limits ^{}} \frac{\left( \left( u-fp_s\right) p+\left( v-fq_s\right) q+1 \right) ^2}{\left( \Vert L\Vert \sqrt{f^2\left( p^2+q^2\right) +\left( up+vq+1\right) ^2} \right) ^2} \end{aligned}$$

and rearranging it as

$$\begin{aligned} I^2\left( \Vert L\Vert \sqrt{f^2\left( p^2+q^2\right) +\left( up+vq+1\right) ^2} \right) ^2 = \left( \left( u-fp_s\right) p+\left( v-fq_s\right) q+1 \right) ^2. \end{aligned}$$

Taking all the terms to the left side

$$\begin{aligned} I^2\left( \Vert L\Vert \sqrt{f^2\left( p^2+q^2\right) +\left( up+vq+1\right) ^2} \right) ^2 -\left( \left( u-fp_s\right) p+\left( v-fq_s\right) q+1 \right) ^2=0, \end{aligned}$$

that simplifies to

$$\begin{aligned} \left( I^2 \Vert L\Vert ^2 \left( f^2\left( p^2+q^2\right) +\left( up+vq+1\right) ^2 \right) \right) -\left( \left( u-fp_s\right) p+\left( v-fq_s\right) q+1 \right) ^2=0, \end{aligned}$$

by canceling the square root and finally appears as

$$\begin{aligned} \left( \underbrace{ I^2\Vert L\Vert ^2f^2\left( p^2+q^2\right) }_{\alpha _1} + \underbrace{ I^2\Vert L\Vert ^2\left( up+vq+1\right) ^2 }_{\alpha _2} \right) - \underbrace{ \left( \left( u-fp_s\right) p + \left( v-fq_s\right) q + 1 \right) ^2 }_{\alpha _3} =0. \end{aligned}$$

Appendix 7

The expanded forms of $\alpha _1$, $\alpha _2$ and $\alpha _3$ are provided as$:$

• $\alpha _1:$

  $$\begin{aligned} \begin{aligned} \alpha _1&{\mathop {=}\limits ^{}} I^2\Vert L\Vert ^2f^2\left( p^2+q^2\right) \\&{\mathop {=}\limits ^{}} I^2\Vert L\Vert ^2f^2p^2 + I^2\Vert L\Vert ^2f^2q^2 \end{aligned} \end{aligned}$$

• $\alpha _2:$

  $$\begin{aligned} \begin{aligned} \alpha _2 {\mathop {=}\limits ^{}}&I^2\Vert L\Vert ^2\left( up+vq+1\right) ^2 \\ {\mathop {=}\limits ^{}}&I^2\Vert L\Vert ^2\left( u^2p^2+v^2q^2+1+2pquv+2up+2vq\right) \\ {\mathop {=}\limits ^{}}&I^2\Vert L\Vert ^2u^2p^2+I^2\Vert L\Vert ^2v^2q^2+ I^2\Vert L\Vert ^2+ \cdots \\ {}&2I^2\Vert L\Vert ^2pquv+2I^2\Vert L\Vert ^2up+2I^2\Vert L\Vert ^2vq \\ \end{aligned} \end{aligned}$$

• $\alpha _3:$

  $$\begin{aligned} \begin{aligned} \alpha _3 {\mathop {=}\limits ^{}}&-\left( \left( u-fp_s\right) p + \left( v-fq_s\right) q + 1 \right) ^2 \\ {\mathop {=}\limits ^{}}&-\left( u-fp_s\right) ^2p^2 - \left( v-fq_s\right) ^2q^2 - 1 - \cdots \\ {}&-2pq\left( u-fp_s\right) \left( v-fq_s\right) -2p\left( u-fp_s\right) -2q\left( v-fq_s\right) \end{aligned}. \end{aligned}$$

Appendix 8

To derive (2.39), let us start from (2.38) and proceed as

$$\begin{aligned}&\left. {\left\{ \begin{array}{ll} \underbrace{I^2\Vert L\Vert ^2f^2\left( p^2+q^2\right) }_{\alpha _1} +\cdots \\ \underbrace{I^2\Vert L\Vert ^2\left( up+vq+1\right) ^2 }_{\alpha _2} -\cdots \\ \underbrace{ \left( \left( u-fp_s\right) p + \left( v-fq_s\right) q + 1 \right) ^2 }_{\alpha _3} \end{array}\right. } \right\} =0. \end{aligned}$$

Now, those components have the terms of interest $p^2$, $q^2$, 2pq, 2p and 2q in common which are marked as

that leads to

$$\begin{aligned}&\left. {\left\{ \begin{array}{ll} p^2 \overbrace{ \left( I^2\Vert L\Vert ^2 \left( f^2+u^2\right) -\left( u-fp_s\right) ^2 \right) }^{:=A}+ \cdots \\ q^2 \overbrace{ \left( I^2\Vert L\Vert ^2\left( f^2+v^2\right) -\left( v-fq_s\right) ^2\right) }^{:=B}+\cdots \\ 2pq \overbrace{ \left( I^2\Vert L\Vert ^2uv - \left( u-fp_s\right) \left( v-fq_s\right) \right) }^{:=C}+\cdots \\ 2p \overbrace{ \left( I^2\Vert L\Vert ^2u-\left( u-fp_s\right) \right) }^{:=D}+\cdots \\ 2q \overbrace{ \left( I^2\Vert L\Vert ^2v-\left( v-fq_s\right) \right) }^{:=E}+\cdots \\ \overbrace{I^2\Vert L\Vert ^2-1}^{:=F} \end{array}\right. } \right\} =0, \end{aligned}$$

and finally to

$$\begin{aligned} p^2A+q^2B+2pqC+2pD+2qE+F=0. \end{aligned}$$