Efficient computation of the Gauss-Newton direction when fitting NURBS using ODR

Abstract

We consider a subproblem in parameter estimation using the Gauss-Newton algorithm with regularization for NURBS curve fitting. The NURBS curve is fitted to a set of data points in least-squares sense, where the sum of squared orthogonal distances is minimized. Control-points and weights are estimated. The knot-vector and the degree of the NURBS curve are kept constant. In the Gauss-Newton algorithm, a search direction is obtained from a linear overdetermined system with a Jacobian and a residual vector. Because of the properties of our problem, the Jacobian has a particular sparse structure which is suitable for performing a splitting of variables. We are handling the computational problems and report the obtained accuracy using different methods, and the elapsed real computational time. The splitting of variables is a two times faster method than using plain normal equations.

Appendix

In this appendix we are giving some details about the origin of G, g and (3.9). The rows in $\tilde{\mathbf{J}}$ consisting of $\lambda_{u_{k}}$ and c′ _k is here denoted $[\bar{\boldsymbol{\mathcal{L}}}_{k} \ \bar{\boldsymbol{\mathcal{S}}}_{k}]$, and the corresponding rows in $\tilde{\mathbf{r}}$ is denoted $\bar{\mathbf{r}}_{k}$. The parameter $\lambda_{u_{k}}$ is in the first row of $\bar{\boldsymbol{\mathcal{L}}}_{k}$. Explicitly written, we have

and from these quantities we form the matrices

$$\bar{\mathbf{J}}=\left[\begin{array}{c@{\quad }c}\bar{\boldsymbol{\mathcal{L}}}_0 & \bar{\boldsymbol{\mathcal{S}}}_0 \\\bar{\boldsymbol{\mathcal{L}}}_1 & \bar{\boldsymbol{\mathcal{S}}}_1 \\\vdots & \vdots\\\bar{\boldsymbol{\mathcal{L}}}_m & \bar{\boldsymbol{\mathcal{S}}}_m \\\mathbf{0} & \boldsymbol{\Lambda}_{\mathrm{s}}\\\end{array}\right],\qquad \bar{\mathbf{r}}=\left[\begin{array}{c}\bar{\mathbf{r}}_0\\\bar{\mathbf{r}}_1\\\vdots\\\bar{\mathbf{r}}_m\\\mathbf{0}\\\end{array}\right],$$

which are same as $\tilde{\mathbf{J}}$ and $\tilde{\mathbf{r}}$, but with interchanged rows.

We use an orthogonal matrix similar to a Householder transformation [11],

$$\boldsymbol{\Phi}_{k}=\frac{2}{\| \mathbf{v}_{k}\| _2^2}\mathbf{v}_{k}\mathbf{v}_{k}^{\mathrm{T}}-\mathbf{I}_{d+1},$$

where

$$\mathbf{v}_{k}=\left[\begin{array}{c}\lambda_{u_k}+\sqrt{\sigma_k}\\{\mathbf{c}'}_k\\\end{array}\right].$$

It gives that $\| \mathbf{v}_{k} \| _{2}^{2}=2(\sigma_{k}+\lambda _{u_{k}}\sqrt{\sigma_{k}})$. Usually the opposite sign of Φ _k is used, making it to a reflection matrix. We could have done that but our way makes subsequent expressions simpler without minus signs. We multiply $\bar{\boldsymbol{\mathcal{L}}}_{k}$, $\bar{\boldsymbol{\mathcal{S}}}_{k}$ and $\bar{\mathbf{r}}_{k}$ with Φ _k . That results in

$$ \everymath{\displaystyle}\begin{array}{rcl}\boldsymbol{\Phi}_{k}\bar{\boldsymbol{\mathcal{L}}}_k&= &\left[\begin{array}{c@{\quad }c@{\quad }c@{\quad }c@{\quad }c@{\quad }c@{\quad }c}0 & \ldots& 0 & \sqrt{\sigma_k} & 0 & \ldots& 0 \\\mathbf{0} & \ldots& \mathbf{0} & \mathbf{0} & \mathbf{0} & \ldots&\mathbf{0} \\\end{array}\right],\\[14pt]\boldsymbol{\Phi}_{k}\bar{\boldsymbol{\mathcal{S}}}_k&=&\bigg( \frac{2}{\| \mathbf{v}_{k} \| _2^2}\mathbf {v}_{k}\mathbf{v}_{k}^{\mathrm{T}}-\mathbf{I}_{d+1} \bigg)\left[\begin{array}{c@{\quad }c@{\quad }c@{\quad }c}\mathbf{0} & \mathbf{0} & \ldots& \mathbf{0}\\\boldsymbol{\mathcal{S}}_{k,1} & \boldsymbol{\mathcal{S}}_{k,2}& \ldots& \boldsymbol{\mathcal{S}}_{k,n-1}\\\end{array}\right]\\[14pt]&=&\left[\begin{array}{c@{\quad }c@{\quad }c@{\quad }c}\frac{1}{\sqrt{\sigma_k}} \boldsymbol{\mathcal{H}}_{k,1}&\frac{1}{\sqrt{\sigma_k}}\boldsymbol{\mathcal{H}}_{k,2}&\ldots &\frac{1}{\sqrt{\sigma_k}}\boldsymbol{\mathcal{H}}_{k,n-1}\\[9pt]\boldsymbol{\mathcal{G}}_{k,1}&\boldsymbol{\mathcal{G}}_{k,2}& \ldots &\boldsymbol{\mathcal{G}}_{k,n-1}\end{array} \right],\\[21pt]\boldsymbol{\Phi}_{k}\bar{\mathbf{r}}_k&=&2\frac{\mathbf{r}_k \cdot {\mathbf{c}'}_k}{\|\mathbf{v}_{k} \| _2^2}\left[\begin{array}{c}\lambda_{u_k}+\sqrt{\sigma_k}\\{\mathbf{c}'}_k\end{array}\right]-\left[\begin{array}{c}0\\{\mathbf{r}}_k\end{array}\right]=\left[\begin{array}{c}\frac{1}{\sqrt{\sigma_k}} \mathbf{r}_k ^{\mathrm{T}} {\mathbf{c}'}_k\\\mathbf{g}_{k}\\\end{array}\right],\end{array}$$

(8.1)

where

Let

$$\mathbf{Z} =\operatorname{diag}(\sqrt{\sigma_0},\sqrt{\sigma_1},\ldots,\sqrt{\sigma_m}),$$

be a diagonal matrix, and let

$$\mathbf{B}=\operatorname{diag}(\boldsymbol{\Phi}_{0},\ldots,\boldsymbol{\Phi}_{m},\mathbf{I}_{(n-1)(d+1)}), $$

be a block diagonal matrix. Note that Z ²=D. The matrix B is an orthogonal matrix since all of its diagonal blocks are orthogonal. The vector $\bar{\mathbf{J}}\boldsymbol{\delta}+ \bar{\mathbf{r}}$ is a row interchanged version of the vector $\tilde{\mathbf{J}}\boldsymbol{\delta} + \tilde {\mathbf{r}}$, and B is an orthogonal matrix, so

$$\| \tilde{\mathbf{J}}\boldsymbol{\delta} + \tilde{\mathbf {r}} \| _2^2=\| \bar{\mathbf{J}}\boldsymbol{\delta} + \bar{\mathbf{r}}\| _2^2=\| \mathbf{B}\bar{\mathbf{J}}\boldsymbol{\delta} + \mathbf {B}\bar{\mathbf{r}} \| _2^2.$$

Using (8.1) and doing relevant row interchanges in the last expression, which does not affect the norm, we get

$$\| \mathbf{B}\bar{\mathbf{J}}\boldsymbol{\delta} + \mathbf{B}\bar{\mathbf{r}} \| _2^2=\left\| \left[\begin{array}{c@{\quad}c}\mathbf{Z} & \mathbf{Z}^{-1} \mathbf{H}\\\mathbf{0} & \mathbf{G}\\\end{array}\right]\left[\begin{array}{c}\boldsymbol{\delta}_{\mathrm{l}}\\\boldsymbol{\delta}_{\mathrm{s}}\\\end{array}\right]+\left[\begin{array}{c}\mathbf{Z}^{-1} \mathbf{h}\\\mathbf{g}\\\end{array}\right]\right\| _2^2,$$

where

$$\mathbf{G}=\left[\begin{array}{c@{\quad }c@{\quad }c@{\quad }c}\boldsymbol{\mathcal{G}}_{0,1} & \boldsymbol{\mathcal{G}}_{0,2}& \ldots& \boldsymbol{\mathcal{G}}_{0,n-1}\\\boldsymbol{\mathcal{G}}_{1,1} & \boldsymbol{\mathcal{G}}_{1,2}& \ldots& \boldsymbol{\mathcal{G}}_{1,n-1}\\\vdots & \vdots &\ddots & \vdots \\\boldsymbol{\mathcal{G}}_{m,1} & \boldsymbol{\mathcal{G}}_{m,2}& \ldots& \boldsymbol{\mathcal{G}}_{m,n-1}\\\boldsymbol{\Lambda}_{\mathrm{s},1} &\mathbf{0} & \ldots& \mathbf{0}\\\mathbf{0}& \boldsymbol{\Lambda}_{\mathrm{s},2}& \ldots& \mathbf{0}\\\vdots & \vdots & \ddots& \vdots \\\mathbf{0} & \mathbf{0}& \ldots& \boldsymbol{\Lambda}_{\mathrm{s},n-1}\\\end{array}\right],\qquad \mathbf{g}=\left[\begin{array}{c}\mathbf{g}_0 \\\mathbf{g}_1 \\\vdots \\\mathbf{g}_m \\\mathbf{0}\\\mathbf{0}\\\vdots \\\mathbf{0}\end{array}\right],$$

and where Λ _s,i is the regularization parameters for (p _i ,w _i ). Finding the least-squares solution is done by solving (3.9). By forming the normal equations, we get the same expression as in (3.5) and (3.6), since A=G ^T G and b=−G ^T g.