A triangular radial cubic spline deformation model for efficient 3D beating heart tracking

Abstract

A novel deformable model is proposed for efficient 3D visual tracking of beating heart. The model is parameterized by the 3D coordinates of four control points: the three vertices and the circumcenter of a triangular target region. Nonlinear deformation on heart surfaces is handled by cubic spline interpolation based on radial pixel distances from the circumcenter. With a pre-computable design matrix, the model can be represented efficiently by a simple matrix equation. An iterative algorithm is developed based on the efficient second-order minimization to compute model parameters at each frame. The proposed tracking method is validated on the stereo-endoscopic videos of phantom heart and in vivo heart that are recorded by the da Vinci$^{\tiny \textregistered }$ surgical system.

Appendices

Appendix 1: Interpolation polynomial of radial cubic spline

As shown in Fig. 1, the nonlinear interpolation for the distorted triangle is a 1D cubic spline interpolation, which is composed of two symmetric cubic polynomial segments, passing through the three knots $(-R, 0)$, $(0, d_o)$ and (R, 0) in the r-z plane.

Without loss of generality, we write the two cubic polynomials as

$$\begin{aligned} D_+(r)= & {} a_3r^3+a_2r^2+a_1r+a_0 \quad \mathrm{for}\quad 0\le r\le R \end{aligned}$$

(20)

$$\begin{aligned} D_-(r)= & {} D_+(-r)\quad \mathrm{for}\quad -R\le r\le 0 \end{aligned}$$

(21)

As the polynomials pass through the three knots, we can obtain

$$\begin{aligned} D_+(0)= & {} D_-(0)=d_o \end{aligned}$$

(22)

$$\begin{aligned} D_+(R)= & {} D_-(-R)=0 \end{aligned}$$

(23)

To make the spline as smooth as possible, the first and second derivatives should be continuous at the knots:

$$\begin{aligned} D_+^{'}(0)=D_-^{'}(0) \end{aligned}$$

(24)

$$\begin{aligned} D_+^{''}(0)=D_-^{''}(0) \end{aligned}$$

(25)

To achieve a “natural” spline, the second derivatives should be zero at the ends:

$$\begin{aligned} D_+^{''}(R)=D_-^{''}(-R)=0 \end{aligned}$$

(26)

From (22) to (26), we can derive that

$$\begin{aligned} a_3=\frac{d_o}{2R^3},~a_2=-\frac{3d_o}{2R^2},~a_1=0, ~\mathrm{and}~ a_0=d_o \end{aligned}$$

(27)

Accordingly, there is

$$\begin{aligned} D_+(r)=\left( \frac{r^3}{2R^3}-\frac{3r^2}{2R^2}+1\right) d_o \end{aligned}$$

(28)

Let $r=\Vert \mathbf{m}-\mathbf{m}_o\Vert$ be the radial distance and $d_o=D(\mathbf{m}_o)$. We obtain the interpolation function $D(\mathbf{m})$.

Appendix 2: Jacobians computation

1.1 Computation of $\mathbf{J}(\mathbf{I}_\mathrm{X} |{\varvec{\theta }})$

The nth row of the Jacobian $\mathbf{J}(\mathbf{I}_\mathrm{X}|{{\varvec{\theta }}})$ is the derivative of $I_\mathrm{X}(\mathbf{m}_n^{(\mathrm{X})})$ at the current parameters ${\varvec{\theta }}$, which can be computed by the product of four sub-Jacobians

$$\begin{aligned} \nabla _{{\varvec{\theta }}}I_\mathrm{X}(\mathbf{m}_n^{(\mathrm{X})})\big |_{{\varvec{\theta }}} =\mathbf{J}_\mathrm{I}{} \mathbf{J}_{\varvec{{\mathrm{\Phi }}}}{} \mathbf{J}_\mathrm{C}{} \mathbf{J}_\mathrm{W} \end{aligned}$$

(29)

• (i) $\mathbf{J}_\mathrm{I}=\nabla _\mathbf{m}I_\mathrm{X}(\mathbf{m})\big |_{\mathbf{m}_n^{(\mathrm{X})}}$ is a $1\times 2$ matrix corresponding to the spatial derivative of the image $I_\mathrm{X}$ at the warped pixel $\mathbf{m}_n^{(\mathrm{X})}$.

• (ii) $\mathbf{J}_{\varvec{{\mathrm{\Phi }}}}=\nabla _\mathbf{h}{{\varvec{\varPhi }}}(\mathbf{h})\big |_{\mathbf{h}_n^{(\mathrm{X})}}$ is a $2\times 3$ matrix corresponding to the derivative of the function ${\varvec{\varPhi }}$ at the warped homogeneous coordinates $\mathbf{h}_n^{(\mathrm{X})}$. Given $\mathbf{h}_n^{(\mathrm{X})}=\left[ su, sv, s\right] ^\mathrm{T}$, the derivative is

  $$\begin{aligned} \mathbf{J}_{\varvec{{\mathrm{\Phi }}}}=\left[ \begin{array}{ccc} 1/s &{}\quad 0 &{}\quad -u/s\\ 0 &{}\quad 1/s &{}\quad -v/s \end{array}\right] \end{aligned}$$

  (30)

• (iii) $\mathbf{J}_\mathrm{C}=\nabla _\mathbf{p}{} \mathbf{h}^{(\mathrm{X})}=\left[ \mathbf{C}_\mathrm{X}\right] _{3\times 3}$ is a constant matrix, where $[\mathbf{X}]_{a\times b}$ denotes the left $a\times b$ sub-matrix of $\mathbf X$ .

• (iv) $\mathbf{J}_\mathrm{W}=\nabla _{{\varvec{\theta }}}{} \mathbf{W}(\mathbf{m}_n|{{\varvec{\varTheta }}})$ is a $3\times 12$ constant matrix corresponding to the derivative of the wrapping mapping $\mathbf W$ with respect to ${\varvec{\theta }}$ at $\mathbf{m}_n$. Given $\mathbf{q}(\mathbf{m}_n)$ in (11), it is written as

  $$\begin{aligned} \mathbf{J}_\mathrm{W}=\left[ \begin{array}{ccc} \mathbf{q}(\mathbf{m}_n) &{} \mathbf{0} &{} \mathbf{0}\\ \mathbf{0} &{} \mathbf{q}(\mathbf{m}_n) &{} \mathbf{0}\\ \mathbf{0} &{} \mathbf{0} &{} \mathbf{q}(\mathbf{m}_n) \end{array}\right] ^\mathrm{T} \end{aligned}$$

  (31)

1.2 Computation of $\mathbf{J}(\mathbf{I}_X |\varvec{\theta }^*)$

The Jacobians for the ideal parameter ${\varvec{\theta }}^*$ cannot be computed directly because ${\varvec{\theta }}^*$ is unknown. However, an effective approximation can be derived using the template image T and the current parameters ${\varvec{\theta }}$.

Let $\mathbf{F}^{-1}$ be an inverse mapping from $\mathbf{m}^{(\mathrm{X})}$ to $\mathbf m$. By using $I_\mathrm{X}(\mathbf{m}^{(\mathrm{X})})\big |_{{{\varvec{\theta }}}^*}=T(\mathbf{m})$ and $\mathbf{m}=\mathbf{F}^{-1}(\mathbf{m}^{(\mathrm{X})}|{\varvec{\theta }})$, one derives the nth row of the Jacobian

$$\begin{aligned}&\nabla _{{\varvec{\theta }}}I_\mathrm{X}(\mathbf{m}_n^{(\mathrm{X})})\big |_{\theta ^*} =\nabla _{{\varvec{\theta }}}T\left[ \mathbf{F}^{-1}(\mathbf{m}_n^{(\mathrm{X})}|{{\varvec{\theta }}})\right] \Big |_{\theta ^*} \nonumber \\&\quad =\mathbf{J}_\mathrm{T}{} \mathbf{J}_\mathrm{F}^{-1}{} \mathbf{J}_{\varvec{{\mathrm{\Phi }}}}^*\mathbf{J}_\mathrm{C}{} \mathbf{J}_\mathrm{W} \end{aligned}$$

(32)

• (i) $\mathbf{J}_\mathrm{T}=\nabla _\mathbf{m}T(\mathbf{m})\big |_{\mathbf{m}_n}$ is the spatial derivative of T at $\mathbf{m}_n$.

• (ii) $\mathbf{J}_\mathrm{F}^{-1}=\nabla _\mathbf{x}{} \mathbf{F}^{-1}(\mathbf{x}|{{\varvec{\theta }}}^*)\big |_{\mathbf{m}_n^{(\mathrm{X})}}=\left[ \nabla _\mathbf{x}{} \mathbf{F}(\mathbf{x}|{{\varvec{\theta }}}^*)\big |_{\mathbf{m}_n}\right] ^{-1}$ $\quad \quad =\left[ \mathbf{J}_{\varvec{{\mathrm{\Phi }}}}^*\mathbf{J}_\mathrm{C}\mathbf{J}_\mathrm{W}^*\right] ^{-1}$ can be approximated by

  $$\begin{aligned} \mathbf{J}_{\varvec{{\mathrm{\Phi }}}}^*= & {} \nabla _\mathbf{h}{{\varvec{\varPhi }}}(\mathbf{h})\big |_{(\mathbf{h}_n^{(\mathrm{X})}|{{\varvec{\theta }}}^*)} \approx \mathbf{J}_{\varvec{{\mathrm{\Phi }}}} \end{aligned}$$

  (33)
  $$\begin{aligned} \mathbf{J}_\mathrm{W}^*= & {} \nabla _\mathbf{m}{} \mathbf{W}(\mathbf{m}|{{\varvec{\varTheta }}}^*)\big |_{\mathbf{m}_n} ={{\varvec{\varTheta }}}^*\mathbf{J}_\mathrm{q} \approx {{\varvec{\varTheta }}}{} \mathbf{J}_\mathrm{q} \end{aligned}$$

  (34)

  where $\mathbf{J}_\mathrm{q}$ is a $4\times 2$ constant matrix

  $$\begin{aligned} \mathbf{J}_\mathrm{q} =\nabla _\mathbf{m}{} \mathbf{q}(\mathbf{m})\big |_{\mathbf{m}_n} =\left[ \begin{array}{c} \nabla _\mathbf{m}S(r)\big |_{\mathbf{m}_n}\\ \left[ \mathbf{M}^{-1}\right] _{3\times 2}-{{\varvec{\lambda }}}_o\nabla _\mathbf{m}S(r)\big |_{\mathbf{m}_n} \end{array}\right] \end{aligned}$$

  (35)

  with

  $$\begin{aligned} \nabla _\mathbf{m}S(r)\big |_{\mathbf{m}_n} =\frac{3\Vert \mathbf{m}_n-\mathbf{m}_o\Vert -6R}{2R^3}(\mathbf{m}_n-\mathbf{m}_o)^\mathrm{T} \end{aligned}$$

  (36)