Guaranteed Ellipse Fitting with a Confidence Region and an Uncertainty Measure for Centre, Axes, and Orientation

Abstract

A simple and fast ellipse estimation method is presented based on optimisation of the Sampson distance serving as a measure of the quality of fit between a candidate ellipse and data points. Generation of ellipses, not just conics, as estimates is ensured through the use of a parametrisation of the set of all ellipses. Optimisation of the Sampson distance is performed with the aid of a custom variant of the Levenberg–Marquardt algorithm. The method is supplemented with a measure of uncertainty of an ellipse fit in two closely related forms. One of these concerns the uncertainty in the algebraic parameters of the fit and the other pertains to the uncertainty in the geometrically meaningful parameters of the fit such as the centre, axes, and major axis orientation. In addition, a means is provided for visualising the uncertainty of an ellipse fit in the form of planar confidence regions. For moderate noise levels, the proposed estimator produces results that are fully comparable in accuracy to those produced by the much slower maximum likelihood estimator. Due to its speed and simplicity, the method may prove useful in numerous industrial applications where a measure of reliability for geometric ellipse parameters is required.

Appendices

Appendices

1.1 Appendix 1: Proof of Equation (4.14)

In this appendix, we establish Eq. (4.14). The proof will not rely on general identities for pseudo-inverse matrices but rather will involve the specifics of $\mathbf {r}(\varvec{\theta })$ and $\mathbf {\varvec{\pi }}(\varvec{\eta })$.

Differentiating $\mathbf {r}(t \varvec{\theta }) = \mathbf {r}(\varvec{\theta })$ with respect to $t$ and evaluating at $t = 1$, we find that $\partial _{\varvec{\theta }}{\mathbf {r}}(\varvec{\theta }) \varvec{\theta }= \mathbf {0}.$ Hence, in particular, $\partial _{\varvec{\theta }}{\mathbf {r}}(\varvec{\pi }(\varvec{\eta }))\varvec{\pi }(\varvec{\eta }) = \mathbf {0}.$ Consequently, recalling (4.4), we have

$$\begin{aligned} \partial _{\varvec{\theta }}{\mathbf {r}}(\varvec{\pi }(\varvec{\eta }))\mathbf {P}_{\varvec{\pi }(\varvec{\eta })}^\perp&= \partial _{\varvec{\theta }}{\mathbf {r}}(\varvec{\pi }(\varvec{\eta }))\\&\quad - \Vert \varvec{\pi }(\varvec{\eta }) \Vert ^{-2} \partial _{\varvec{\theta }}{\mathbf {r}}(\varvec{\pi }(\varvec{\eta }))\varvec{\pi }(\varvec{\eta })\varvec{\pi }(\varvec{\eta })^\mathsf {T}\\&= \partial _{\varvec{\theta }}{\mathbf {r}}(\varvec{\pi }(\varvec{\eta })). \end{aligned}$$

We summarise this simply as

$$\begin{aligned} \partial _{\varvec{\theta }}{\mathbf {r}} \, \mathbf {P}_{\varvec{\pi }(\varvec{\eta })}^\perp = \partial _{\varvec{\theta }}{\mathbf {r}} \end{aligned}$$

in line with our earlier convention that $\partial _{\varvec{\theta }}{\mathbf {r}}$ be evaluated at $\varvec{\pi }(\varvec{\eta })$. As an immediate consequence, we obtain

$$\begin{aligned} ((\partial _{\varvec{\theta }}{\mathbf {r}})^\mathsf {T}\partial _{\varvec{\theta }}{\mathbf {r}} + \lambda \mathbf {I}_6) \mathbf {P}_{\varvec{\pi }(\varvec{\eta })}^\perp = (\partial _{\varvec{\theta }}{\mathbf {r}})^\mathsf {T}\partial _{\varvec{\theta }}{\mathbf {r}} + \lambda \mathbf {P}_{\varvec{\pi }(\varvec{\eta })}^\perp . \end{aligned}$$

This together with the observation that $(\partial _{\varvec{\theta }}{\mathbf {r}})^\mathsf {T}\partial _{\varvec{\theta }}{\mathbf {r}} + \lambda \mathbf {P}_{\varvec{\pi }(\varvec{\eta })}^\perp$ is symmetric (being the sum of the symmetric matrices $(\partial _{\varvec{\theta }}{\mathbf {r}})^\mathsf {T}\partial _{\varvec{\theta }}{\mathbf {r}}$ and $\lambda \mathbf {P}_{\varvec{\pi }(\varvec{\eta })}^\perp$) yields

$$\begin{aligned} ((\partial _{\varvec{\theta }}{\mathbf {r}})^\mathsf {T}\partial _{\varvec{\theta }}{\mathbf {r}} + \lambda \mathbf {I}_6) \mathbf {P}_{\varvec{\pi }(\varvec{\eta })}^\perp&= (((\partial _{\varvec{\theta }}{\mathbf {r}})^\mathsf {T}\partial _{\varvec{\theta }}{\mathbf {r}} + \lambda \mathbf {I}_6) \mathbf {P}_{\varvec{\pi }(\varvec{\eta })}^\perp )^\mathsf {T}\nonumber \\&= (\mathbf {P}_{\varvec{\pi }(\varvec{\eta })}^\perp )^\mathsf {T}((\partial _{\varvec{\theta }}{\mathbf {r}})^\mathsf {T}\partial _{\varvec{\theta }}{\mathbf {r}} + \lambda \mathbf {I}_6)^\mathsf {T}\nonumber \\&= \mathbf {P}_{\varvec{\pi }(\varvec{\eta })}^\perp ((\partial _{\varvec{\theta }}{\mathbf {r}})^\mathsf {T}\partial _{\varvec{\theta }}{\mathbf {r}} + \lambda \mathbf {I}_6). \end{aligned}$$

(13.1)

Now note that if $\mathbf {A}$ and $\mathbf {B}$ are square matrices of the same sizes, $\mathbf {A}$ is invertible, and $\mathbf {A}\mathbf {B} = \mathbf {B}\mathbf {A}$, then $\mathbf {A}^{-1}\mathbf {B} = \mathbf {B}\mathbf {A}^{-1}$, as is easily seen by pre- and post-multiplying the both sides of $\mathbf {A}\mathbf {B} = \mathbf {B}\mathbf {A}$ by $\mathbf {A}^{-1}$. This in conjunction with (13.1) implies

$$\begin{aligned} ((\partial _{\varvec{\theta }}{\mathbf {r}})^\mathsf {T}\partial _{\varvec{\theta }}{\mathbf {r}} + \lambda \mathbf {I}_6)^{-1} \mathbf {P}_{\varvec{\pi }(\varvec{\eta })}^\perp = \mathbf {P}_{\varvec{\pi }(\varvec{\eta })}^\perp ((\partial _{\varvec{\theta }}{\mathbf {r}})^\mathsf {T}\partial _{\varvec{\theta }}{\mathbf {r}} + \lambda \mathbf {I}_6)^{-1}. \end{aligned}$$

(13.2)

Differentiating the identity $\Vert \varvec{\pi }(\varvec{\eta }) \Vert ^2 = \varvec{\pi }(\varvec{\eta })^\mathsf {T}\varvec{\pi }(\varvec{\eta }) = 1$ with respect to $\varvec{\eta }$, we get

$$\begin{aligned} (\partial _{\varvec{\eta }}{\varvec{\pi }})^\mathsf {T}\varvec{\pi }(\varvec{\eta }) = 0. \end{aligned}$$

(13.3)

Hence

$$\begin{aligned} (\partial _{\varvec{\eta }}{\varvec{\pi }})^\mathsf {T}\mathbf {P}_{\varvec{\pi }(\varvec{\eta })}^\perp&= (\partial _{\varvec{\eta }}{\varvec{\pi }})^\mathsf {T}- \Vert \varvec{\pi }(\varvec{\eta }) \Vert ^{-2} (\partial _{\varvec{\eta }}{\varvec{\pi }})^\mathsf {T}\varvec{\pi }(\varvec{\eta })\varvec{\pi }(\varvec{\eta })^\mathsf {T}\\&= (\partial _{\varvec{\eta }}{\varvec{\pi }})^\mathsf {T}. \end{aligned}$$

Pre-multiplying both sides of this equality by $((\partial _{\varvec{\eta }}{\varvec{\pi }})^\mathsf {T}\partial _{\varvec{\eta }}{\varvec{\pi }})^{-1}$ and invoking (4.12a), we obtain

$$\begin{aligned} (\partial _{\varvec{\eta }}{\varvec{\pi }})^+ \mathbf {P}_{\varvec{\pi }(\varvec{\eta })}^\perp = (\partial _{\varvec{\eta }}{\varvec{\pi }})^+. \end{aligned}$$

(13.4)

Now recall that for any matrix $\mathbf {A}$, the matrix $\mathbf {A}^+ \mathbf {A}$ represents the orthogonal projection onto the range (column space) of $\mathbf {A}^\mathsf {T}$, or equivalently, the orthogonal projection onto the orthogonal complement of the null space of $\mathbf {A}$. Since, on account of (13.3), the null space of $(\partial _{\varvec{\eta }}{\varvec{\pi }})^\mathsf {T}$ is spanned by $\varvec{\pi }(\varvec{\eta })$, it follows that

$$\begin{aligned} ((\partial _{\varvec{\eta }}{\varvec{\pi }})^\mathsf {T})^+ (\partial _{\varvec{\eta }}{\varvec{\pi }})^\mathsf {T}= \mathbf {P}_{\varvec{\pi }(\varvec{\eta })}^\perp . \end{aligned}$$

(13.5)

We now have all ingredients needed to establish (4.14). We calculate as follows:

$$\begin{aligned}&\!\!\!(\partial _{\varvec{\eta }}{\varvec{\pi }})^+ ((\partial _{\varvec{\theta }}{\mathbf {r}})^\mathsf {T}\partial _{\varvec{\theta }}{\mathbf {r}} +\lambda \mathbf {I}_6)^{-1} ((\partial _{\varvec{\eta }}{\varvec{\pi }})^\mathsf {T})^+ (\partial _{\varvec{\eta }}{\varvec{\pi }})^\mathsf {T}((\partial _{\varvec{\theta }}{\mathbf {r}})^\mathsf {T}\partial _{\varvec{\theta }}{\mathbf {r}} +\lambda \mathbf {I}_6) \partial _{\varvec{\eta }}{\varvec{\pi }}\\&\qquad = (\partial _{\varvec{\eta }}{\varvec{\pi }})^+ ((\partial _{\varvec{\theta }}{\mathbf {r}})^\mathsf {T}\partial _{\varvec{\theta }}{\mathbf {r}} +\lambda \mathbf {I}_6)^{-1} \mathbf {P}_{\varvec{\pi }(\varvec{\eta })}^\perp ((\partial _{\varvec{\theta }}{\mathbf {r}})^\mathsf {T}\partial _{\varvec{\theta }}{\mathbf {r}} +\lambda \mathbf {I}_6) \partial _{\varvec{\eta }}{\varvec{\pi }}\\&\qquad =(\partial _{\varvec{\eta }}{\varvec{\pi }})^+ \mathbf {P}_{\varvec{\pi }(\varvec{\eta })}^\perp ((\partial _{\varvec{\theta }}{\mathbf {r}})^\mathsf {T}\partial _{\varvec{\theta }}{\mathbf {r}} +\lambda \mathbf {I}_6)^{-1} ((\partial _{\varvec{\theta }}{\mathbf {r}})^\mathsf {T}\partial _{\varvec{\theta }}{\mathbf {r}} +\lambda \mathbf {I}_6) \partial _{\varvec{\eta }}{\varvec{\pi }}\\&\qquad =(\partial _{\varvec{\eta }}{\varvec{\pi }})^+ \mathbf {P}_{\varvec{\pi }(\varvec{\eta })}^\perp \partial _{\varvec{\eta }}{\varvec{\pi }}\\&\qquad =(\partial _{\varvec{\eta }}{\varvec{\pi }})^+ \partial _{\varvec{\eta }}{\varvec{\pi }}\\&\qquad =\mathbf {I}_5. \end{aligned}$$

In the above the second line comes from the first by (13.5); the third line comes from the second by (13.2); the fourth line comes from the third by the tautological identity $((\partial _{\varvec{\theta }}{\mathbf {r}})^\mathsf {T}\partial _{\varvec{\theta }}{\mathbf {r}} +\lambda \mathbf {I}_6)^{-1} ((\partial _{\varvec{\theta }}{\mathbf {r}})^\mathsf {T}\partial _{\varvec{\theta }}{\mathbf {r}} +\lambda \mathbf {I}_6) =\mathbf {I}_6;$ the fifth line comes from the fourth by (13.4); and the sixth line comes from the fifth by (4.13a). The end result of our calculation is what is exactly needed to establish (4.14).

1.2 Appendix 2: Proof of Equation (5.2)

In this appendix, we establish the covariance formula (5.2). The derivation is based on two ingredients: an equation characterising $\widehat{\varvec{\theta }}_{\mathrm {AML}}$ and a covariance propagation formula. The first ingredient, embodied in Eq. (13.7) below, comes from the optimality condition that $\widehat{\varvec{\theta }}_{\mathrm {AML}}$ satisfies as the minimiser of $J_{\mathrm {AML}}$:

$$\begin{aligned}{}[\partial _{\varvec{\theta }}{J_{\mathrm {AML}}(\varvec{\theta }; \mathbf {x}_1, \dots , \mathbf {x}_N)}]_{\varvec{\theta }= \widehat{\varvec{\theta }}_{\mathrm {AML}}} = \mathbf {0}^\mathsf {T}. \end{aligned}$$

(13.6)

Direct computation shows that

$$\begin{aligned}{}[\partial _{\varvec{\theta }}{J_{\mathrm {AML}}(\varvec{\theta }; \mathbf {x}_1, \dots , \mathbf {x}_N)}]^\mathsf {T}= 2 \mathbf {X}_{\varvec{\theta }} \varvec{\theta }, \end{aligned}$$

where

$$\begin{aligned} \mathbf {X}_{\varvec{\theta }} = \sum _{n=1}^N \frac{\mathbf {A}_n}{\varvec{\theta }^{\mathsf {T}} \mathbf {B}_n\varvec{\theta }} - \sum _{n=1}^N \frac{\varvec{\theta }^{\mathsf {T}} \mathbf {A}_n \varvec{\theta }}{(\varvec{\theta }^{\mathsf {T}} \mathbf {B}_n \varvec{\theta })^2} \mathbf {B}_n. \end{aligned}$$

Accordingly, (13.6) can be reformulated as

$$\begin{aligned} \mathbf {X}_{\hat{\varvec{\theta }}} \hat{\varvec{\theta }}= \mathbf {0}, \end{aligned}$$

(13.7)

where $\widehat{\varvec{\theta }}_{\mathrm {AML}}$ is abbreviated to $\hat{\varvec{\theta }}$ for clarity.

The second ingredient, which will be used in combination with the first, is the covariance propagation formula

$$\begin{aligned} \varvec{\varLambda }_{\hat{\varvec{\theta }}}^{} = \sum _{n=1}^N \partial _{\mathbf {x}_n}{\hat{\varvec{\theta }}} \varvec{\varLambda }_{\mathbf {x}_n}^{} (\partial _{\mathbf {x}_n}{\hat{\varvec{\theta }}})^{\mathsf {T}} \end{aligned}$$

(13.8)

(cf. [15, 22]). Here it is tacitly assumed that, corresponding to varying sets of data points $\mathbf {x}_1, \dots , \mathbf {x}_N$, the normalised vectors $\hat{\varvec{\theta }}= \hat{\varvec{\theta }}(\mathbf {x}_1, \dots , \mathbf {x}_N)$ have been chosen in a coordinated way, so that $\hat{\varvec{\theta }}$ varies smoothly, without sign flipping, as a function of $\mathbf {x}_1, \dots , \mathbf {x}_N$ and in particular may be differentiated.

As a first step towards the derivation of formula (5.2), we differentiate $\Vert \hat{\varvec{\theta }}\Vert ^2 =1$ with respect to $\mathbf {x}_n$ to get $(\partial _{\mathbf {x}_n}{\hat{\varvec{\theta }}})^{\mathsf {T}}\hat{\varvec{\theta }}= \mathbf {0}.$ This jointly with (4.4) and (13.8) then implies

$$\begin{aligned} \mathbf {P}_{\hat{\varvec{\theta }}}^\perp \varvec{\varLambda }_{\hat{\varvec{\theta }}}^{} = \varvec{\varLambda }_{\hat{\varvec{\theta }}}^{} \mathbf {P}_{\hat{\varvec{\theta }}}^\perp = \varvec{\varLambda }_{\hat{\varvec{\theta }}}^{}. \end{aligned}$$

(13.9)

Next, letting $\mathbf {x}_n = [m_{n,1}, m_{n,2}]^\mathsf {T}$ and $\hat{\varvec{\theta }}= [\hat{\theta }_1, \dots , \hat{\theta }_6]^\mathsf {T}$, and differentiating (13.7) with respect to $m_{n,i}$, we obtain

$$\begin{aligned} \left[ [\partial _{m_{n,i}}{\mathbf {X}_{\varvec{\theta }}}]_{\varvec{\theta }= \hat{\varvec{\theta }}} + \sum _{j=1}^6 [\partial _{\theta _j}{\mathbf {X}_{\varvec{\theta }}}]_{\varvec{\theta }= \hat{\varvec{\theta }}} \partial _{m_{n,i}}{{\hat{\theta }}_j} \right] {\hat{\varvec{\theta }}} + \mathbf {X}_{\hat{\varvec{\theta }}} \partial _{m_{n,i}}{\hat{\varvec{\theta }}} = \mathbf {0}. \end{aligned}$$

Introducing the Gauss-Newton approximation, i.e., neglecting the terms that contain $\hat{\varvec{\theta }}^\mathsf {T}\mathbf {u}(\mathbf {x}_n)$, we arrive (after some calculations) at

$$\begin{aligned} \frac{\mathbf {u}(\mathbf {x}_n)[\partial _{m_{n,i}}{\mathbf {u}(\mathbf {x}_n)}]^{\mathsf {T}} \hat{\varvec{\theta }}}{\hat{\varvec{\theta }}^{\mathsf {T}} \mathbf {B}_n \hat{\varvec{\theta }}} + \left[ \sum _{n=1}^N \frac{\mathbf {A}_n}{\hat{\varvec{\theta }}^{\mathsf {T}} \mathbf {B}_n \hat{\varvec{\theta }}} \right] \partial _{m_{n,i}}{\hat{\varvec{\theta }}} = \mathbf {0}. \end{aligned}$$

This together with the observation that the scalar $[\partial _{m_{n,i}}{\mathbf {u}(\mathbf {x}_n)}]^{\mathsf {T}} \hat{\varvec{\theta }}$ can also be written as $\hat{\varvec{\theta }}^{\mathsf {T}} \partial _{m_{n,i}}{\mathbf {u}(\mathbf {x}_n)}$ leads to

$$\begin{aligned} \left[ \sum _{n=1}^N \frac{\mathbf {A}_n}{\hat{\varvec{\theta }}^{\mathsf {T}} \mathbf {B}_n \hat{\varvec{\theta }}} \right] \partial _{m_{n,i}}{\hat{\varvec{\theta }}}&= - \frac{\mathbf {u}(\mathbf {x}_n) [\partial _{m_{n,i}}{\mathbf {u}(\mathbf {x}_n)}]^{\mathsf {T}} \hat{\varvec{\theta }}}{\hat{\varvec{\theta }}^{\mathsf {T}} \mathbf {B}_n \hat{\varvec{\theta }}}\\&= - \frac{\mathbf {u}(\mathbf {x}_n) \hat{\varvec{\theta }}^{\mathsf {T}} \partial _{m_{n,i}}{\mathbf {u}(\mathbf {x}_n)}}{\hat{\varvec{\theta }}^{\mathsf {T}} \mathbf {B}_n \hat{\varvec{\theta }}}. \end{aligned}$$

Consequently,

$$\begin{aligned} \left[ \sum _{n=1}^N \frac{\mathbf {A}_n}{\hat{\varvec{\theta }}^{\mathsf {T}} \mathbf {B}_n \hat{\varvec{\theta }}} \right] \partial _{\mathbf {x}_n}{\hat{\varvec{\theta }}} = - \frac{\mathbf {u}(\mathbf {x}_n) \hat{\varvec{\theta }}^{\mathsf {T}} \partial _{\mathbf {x}_n}{\mathbf {u}(\mathbf {x}_n)}}{\hat{\varvec{\theta }}^{\mathsf {T}} \mathbf {B}_n \hat{\varvec{\theta }}} \end{aligned}$$

and further, recalling the definitions of $\mathbf {A}_n$ and $\mathbf {B}_n$ given in (2.5),

$$\begin{aligned}&\left[ \sum _{n=1}^N \frac{\mathbf {A}_n}{\hat{\varvec{\theta }}^{\mathsf {T}} \mathbf {B}_n \hat{\varvec{\theta }}} \right] \partial _{\mathbf {x}_n}{\hat{\varvec{\theta }}} \varvec{\varLambda }_{\mathbf {x}_n}^{} (\partial _{\mathbf {x}_n}{\hat{\varvec{\theta }}})^{\mathsf {T}} \left[ \sum _{n=1}^N \frac{\mathbf {A}_n}{\hat{\varvec{\theta }}^{\mathsf {T}} \mathbf {B}_n \hat{\varvec{\theta }}} \right] \\&\qquad = \frac{ \mathbf {u}(\mathbf {x}_n) \hat{\varvec{\theta }}^{\mathsf {T}} \partial _{\mathbf {x}_n}{\mathbf {u}(\mathbf {x}_n)} \varvec{\varLambda }_{\mathbf {x}_n}^{} [\partial _{\mathbf {x}_n}{\mathbf {u}(\mathbf {x}_n)}]^{\mathsf {T}} \hat{\varvec{\theta }}\mathbf {u}(\mathbf {x}_n)^{\mathsf {T}}}{(\hat{\varvec{\theta }}^{\mathsf {T}} \mathbf {B}_n \hat{\varvec{\theta }})^2}\\&\qquad = \frac{\mathbf {u}(\mathbf {x}_n) \mathbf {u}(\mathbf {x}_n)^{\mathsf {T}}}{\hat{\varvec{\theta }}^{\mathsf {T}} \mathbf {B}_n \hat{\varvec{\theta }}} = \frac{\mathbf {A}_n}{\hat{\varvec{\theta }}^{\mathsf {T}} \mathbf {B}_n \hat{\varvec{\theta }}}. \end{aligned}$$

Now

$$\begin{aligned}&\left[ \sum _{n=1}^N \frac{\mathbf {A}_n}{\hat{\varvec{\theta }}^{\mathsf {T}} \mathbf {B}_n \hat{\varvec{\theta }}} \right] \left[ \sum _{n=1}^N \partial _{\mathbf {x}_n}{\hat{\varvec{\theta }}} \varvec{\varLambda }_{\mathbf {x}_n}^{} (\partial _{\mathbf {x}_n}{\hat{\varvec{\theta }}})^{\mathsf {T}} \right] \left[ \sum _{n=1}^N \frac{\mathbf {A}_n}{\hat{\varvec{\theta }}^{\mathsf {T}} \mathbf {B}_n \hat{\varvec{\theta }}} \right] \\&\qquad = \sum _{n=1}^N \frac{\mathbf {A}_n}{\hat{\varvec{\theta }}^{\mathsf {T}} \mathbf {B}_n \hat{\varvec{\theta }}}. \end{aligned}$$

By (5.1) and (13.8), the last equality becomes

$$\begin{aligned} \mathbf {M}_{\hat{\varvec{\theta }}} \varvec{\varLambda }_{\hat{\varvec{\theta }}}^{} \mathbf {M}_{\hat{\varvec{\theta }}} = \mathbf {M}_{\hat{\varvec{\theta }}}. \end{aligned}$$

(13.10)

At this stage, one might be tempted to conclude that $\varvec{\varLambda }_{\hat{\varvec{\theta }}}^{} = \mathbf {M}_{\hat{\varvec{\theta }}}^{-1}$, but this would contravene the fact that $\varvec{\varLambda }_{\hat{\varvec{\theta }}}^{}$ is singular. In order to exploit (13.10) properly as an approximate equality, we first note that, in view of (13.9) and the fact that $\mathbf {P}_{\hat{\varvec{\theta }}}^\perp$ is idempotent, $\mathbf {P}_{\hat{\varvec{\theta }}}^\perp = (\mathbf {P}_{\hat{\varvec{\theta }}}^\perp )^2,$ we have

$$\begin{aligned} \mathbf {P}_{\hat{\varvec{\theta }}}^\perp \varvec{\varLambda }_{\hat{\varvec{\theta }}}^{} \mathbf {P}_{\hat{\varvec{\theta }}}^\perp = \varvec{\varLambda }_{\hat{\varvec{\theta }}}^{}, \end{aligned}$$

(13.11)

so (13.10) can be rewritten as

$$\begin{aligned} \mathbf {M}_{\hat{\varvec{\theta }}} \mathbf {P}_{\hat{\varvec{\theta }}}^\perp \varvec{\varLambda }_{\hat{\varvec{\theta }}}^{} \mathbf {P}_{\hat{\varvec{\theta }}}^\perp \mathbf {M}_{\hat{\varvec{\theta }}} = \mathbf {M}_{\hat{\varvec{\theta }}}. \end{aligned}$$

Pre- and post-multiplying the last equation by $\mathbf {P}_{\hat{\varvec{\theta }}}^\perp$ and letting

$$\begin{aligned} \mathbf {M}_{\hat{\varvec{\theta }}}^\perp = \mathbf {P}_{\hat{\varvec{\theta }}}^\perp \mathbf {M}_{\hat{\varvec{\theta }}} \mathbf {P}_{\hat{\varvec{\theta }}}^\perp \end{aligned}$$

now leads to

$$\begin{aligned} \mathbf {M}_{\hat{\varvec{\theta }}}^\perp \varvec{\varLambda }_{\hat{\varvec{\theta }}}^{} \mathbf {M}_{\hat{\varvec{\theta }}}^\perp = \mathbf {M}_{\hat{\varvec{\theta }}}^\perp . \end{aligned}$$

(13.12)

In turn, pre- and post-multiplying (13.12) by $(\mathbf {M}_{\hat{\varvec{\theta }}}^\perp )^+$ yields

$$\begin{aligned} (\mathbf {M}_{\hat{\varvec{\theta }}}^\perp )^+ \mathbf {M}_{\hat{\varvec{\theta }}}^\perp \varvec{\varLambda }_{\hat{\varvec{\theta }}}^{} \mathbf {M}_{\hat{\varvec{\theta }}}^\perp (\mathbf {M}_{\hat{\varvec{\theta }}}^\perp )^+ = (\mathbf {M}_{\hat{\varvec{\theta }}}^\perp )^+ \mathbf {M}_{\hat{\varvec{\theta }}}^\perp (\mathbf {M}_{\hat{\varvec{\theta }}}^\perp )^+. \end{aligned}$$

(13.13)

The matrix $\mathbf {M}_{\hat{\varvec{\theta }}}^\perp$ is symmetric and its null space is spanned by $\hat{\varvec{\theta }}$, so

$$\begin{aligned} \mathbf {M}_{\hat{\varvec{\theta }}}^\perp (\mathbf {M}_{\hat{\varvec{\theta }}}^\perp )^+ = (\mathbf {M}_{\hat{\varvec{\theta }}}^\perp )^+ \mathbf {M}_{\hat{\varvec{\theta }}}^\perp = \mathbf {P}_{\hat{\varvec{\theta }}}^\perp \end{aligned}$$

(cf. [4, Cor. 3.5]). We also have $(\mathbf {M}_{\hat{\varvec{\theta }}}^\perp )^+ \mathbf {M}_{\hat{\varvec{\theta }}}^\perp (\mathbf {M}_{\hat{\varvec{\theta }}}^\perp )^+ = (\mathbf {M}_{\hat{\varvec{\theta }}}^\perp )^+$ by virtue of one of the four defining properties of the pseudo-inverse [4, Thm. 3.9]. Therefore (13.13) can be restated as

$$\begin{aligned} \mathbf {P}_{\hat{\varvec{\theta }}}^\perp \varvec{\varLambda }_{\hat{\varvec{\theta }}}^{} \mathbf {P}_{\hat{\varvec{\theta }}}^\perp = (\mathbf {M}_{\hat{\varvec{\theta }}}^\perp )^+, \end{aligned}$$

which, on account of (13.11), implies

$$\begin{aligned} \varvec{\varLambda }_{\hat{\varvec{\theta }}}^{} = (\mathbf {M}_{\hat{\varvec{\theta }}}^\perp )^+. \end{aligned}$$

(13.14)

We now deduce our final formula for $\varvec{\varLambda }_{\hat{\varvec{\theta }}}^{}$, namely

$$\begin{aligned} \varvec{\varLambda }_{\hat{\varvec{\theta }}}^{} = \mathbf {P}_{\hat{\varvec{\theta }}}^\perp (\mathbf {M}_{\hat{\varvec{\theta }}})^+_5 \mathbf {P}_{\hat{\varvec{\theta }}}^\perp , \end{aligned}$$

(13.15)

which is nothing else but Eq. (5.2) transcribed to the present notation. First we note that as, by (13.7), $\hat{\varvec{\theta }}$ spans the null space of $\mathbf {X}_{\hat{\varvec{\theta }}}$, $\mathbf {X}_{\hat{\varvec{\theta }}}$ has rank $5$. Next we observe that in the Gauss–Newton approximation $\mathbf {X}_{\hat{\varvec{\theta }}}$ is equal to $\mathbf {M}_{\hat{\varvec{\theta }}}$, so, having rank $5$, $\mathbf {X}_{\hat{\varvec{\theta }}}$ is also approximately equal to $(\mathbf {M}_{\hat{\varvec{\theta }}})_5$. This in turn implies that, approximately,

$$\begin{aligned} (\mathbf {M}_{\hat{\varvec{\theta }}})^+_5 = \mathbf {X}_{\hat{\varvec{\theta }}}^+, \end{aligned}$$

given that the function $\mathbf {A} \mapsto \mathbf {A}^+$ is continuous when considered on sets of matrices of equal rank [20, 36, 45, 52], The last equality together with $\mathbf {P}_{\hat{\varvec{\theta }}}^\perp \mathbf {X}_{\hat{\varvec{\theta }}}^+ \mathbf {P}_{\hat{\varvec{\theta }}}^\perp = \mathbf {X}_{\hat{\varvec{\theta }}}^+,$ which immediately follows from the facts that $\mathbf {X}_{\hat{\varvec{\theta }}}$ is symmetric and that $\hat{\varvec{\theta }}$ spans the null space of $\mathbf {X}_{\hat{\varvec{\theta }}}$, implies

$$\begin{aligned} \mathbf {P}_{\hat{\varvec{\theta }}}^\perp (\mathbf {M}_{\hat{\varvec{\theta }}})^+_5 \mathbf {P}_{\hat{\varvec{\theta }}}^\perp = \mathbf {X}_{\hat{\varvec{\theta }}}^+. \end{aligned}$$

(13.16)

As $\mathbf {M}_{\hat{\varvec{\theta }}}$ is approximately equal to $\mathbf {X}_{\hat{\varvec{\theta }}}$, it is clear that $\mathbf {M}_{\hat{\varvec{\theta }}}^\perp$ ($= \mathbf {P}_{\hat{\varvec{\theta }}}^\perp \mathbf {M}_{\hat{\varvec{\theta }}} \mathbf {P}_{\hat{\varvec{\theta }}}^\perp$) is approximately equal to $\mathbf {P}_{\hat{\varvec{\theta }}}^\perp \mathbf {X}_{\hat{\varvec{\theta }}} \mathbf {P}_{\hat{\varvec{\theta }}}^\perp = \mathbf {X}_{\hat{\varvec{\theta }}}$. Both $\mathbf {M}_{\hat{\varvec{\theta }}}^\perp$ and $\mathbf {X}_{\hat{\varvec{\theta }}}$ have rank $5$, so, as they are approximately equal, their pseudo-inverses are also approximately equal,

$$\begin{aligned} (\mathbf {M}_{\hat{\varvec{\theta }}}^\perp )^+ = \mathbf {X}_{\hat{\varvec{\theta }}}^+, \end{aligned}$$

by the aforementioned continuity property of the pseudo-inverse. Combining this last equation with (13.16) yields

$$\begin{aligned} (\mathbf {M}_{\hat{\varvec{\theta }}}^\perp )^+ = \mathbf {P}_{\hat{\ varvec{\theta }}}^\perp (\mathbf {M}_{\hat{\varvec{\theta }}})^+_5 \mathbf {P}_{\hat{\varvec{\theta }}}^\perp , \end{aligned}$$

and this together (13.14) establishes (13.15).