## Abstract

This article presents a method to incorporate a deformation prior in image reconstruction via the formalism of deformation modules. The framework of deformation modules allows to build diffeomorphic deformations that satisfy a given structure. The idea is to register a template image against the indirectly observed data via a modular deformation, incorporating this way the deformation prior in the reconstruction method. We show that this is a well-defined regularisation method (proving existence, stability and convergence) and present numerical examples of reconstruction from 2-D tomographic simulations and partially observed images.

### Similar content being viewed by others

## References

Abraham, I., Abraham, R., Bergounioux, M., Carlier, G.: Tomographic reconstruction from a few views: a multi-marginal optimal transport approach. Appl. Math. Optim.

**75**(1), 55–73 (2017)Amit, Y., Grenander, U., Piccioni, M.: Structural image restoration through deformable templates. J. Am. Stat. Assoc.

**86**(414), 376–387 (1991)Arguillere, S.: Géométrie sous-riemannienne en dimension infinie et applications à l’analyse mathématique des formes. Ph.D. thesis, Paris 6 (2014)

Arguillere, S., Trélat, E., Trouvé, A., Younes, L.: Shape deformation analysis from the optimal control viewpoint. Journal de mathématiques pures et appliquées

**104**(1), 139–178 (2015)Aronszajn, N.: Theory of reproducing kernels. Trans. Am. Math. Soc.

**68**(3), 337–404 (1950)Arsigny, V., Commowick, O., Ayache, N., Pennec, X.: A fast and log-euclidean polyaffine framework for locally linear registration. J. Math. Imaging Vis.

**33**(2), 222–238 (2009)Arsigny, V., Pennec, X., Ayache, N.: Polyrigid and polyaffine transformations: a novel geometrical tool to deal with non-rigid deformations-application to the registration of histological slices. Med. Image Anal.

**9**(6), 507–523 (2005)Beg, M.F., Miller, M.I., Trouvé, A., Younes, L.: Computing large deformation metric mappings via geodesic flows of diffeomorphisms. Int. J. Comput. Vis.

**61**(2), 139–157 (2005)Benamou, J.-D., Carlier, G., Cuturi, M., Nenna, L., Peyré, G.: Iterative Bregman projections for regularized transportation problems. SIAM J. Sci. Comput.

**37**(2), A1111–A1138 (2015)Berkels, B., Effland, A., Rumpf, M.: Time discrete geodesic paths in the space of images. SIAM J. Imaging Sci.

**8**(3), 1457–1488 (2015)Blume, M., Martinez-Moller, A., Keil, A., Navab, N., Rafecas, M.: Joint reconstruction of image and motion in gated positron emission tomography. IEEE Trans. Med. Imaging

**29**(11), 1892–1906 (2010)Bruveris, M., Holm, D.D.: Geometry of image registration: the diffeomorphism group and momentum maps. In: Geometry, Mechanics, and Dynamics, pp. 19–56. Springer (2015)

Burger, M., Dirks, H., Frerking, L., Hauptmann, A., Helin, T., Siltanen, S.: A variational reconstruction method for undersampled dynamic X-ray tomography based on physical motion models. Inverse Probl.

**33**(12), 124008 (2017)Chen, C., Öktem, O.: Indirect image registration with large diffeomorphic deformations. SIAM J. Imaging Sci.

**11**(1), 575–617 (2018)Ehrhardt, J., Werner, R., Säring, D., Frenzel, T., Lu, W., Low, D., Handels, H.: An optical flow based method for improved reconstruction of 4D CT data sets acquired during free breathing. Med. Phys.

**34**(2), 711–721 (2007)Grenander, U., Srivastava, A., Saini, S.: A pattern-theoretic characterization of biological growth. IEEE Trans. Med. Imaging

**26**(5), 648–659 (2007)Gris, B.: Modular approach on shape spaces, sub-Riemannian geometry and computational anatomy. Ph.D. thesis, Université Paris-Saclay (2016)

Gris, B., Durrleman, S., Trouvé, A.: A sub-riemannian modular framework for diffeomorphism based analysis of shape ensembles. SIAM J. Imaging Sci.

**11**(1), 802–833 (2016)Gris, B., Öktem, O.: Image reconstruction through metamorphosis. arXiv preprint arXiv:1806.01225v2 (2018)

Haber, E., Modersitzki, J.: A multilevel method for image registration. SIAM J. Sci. Comput.

**27**(5), 1594–1607 (2006)Hahn, B.N.: Motion estimation and compensation strategies in dynamic computerized tomography. Sens. Imaging

**18**(1), 10 (2017)Hinkle, J., Szegedi, M., Wang, B., Salter, B., Joshi, S.: 4D CT image reconstruction with diffeomorphic motion model. Med. Image Anal.

**16**(6), 1307–1316 (2012)Isola, A., Ziegler, A., Koehler, T., Niessen, W., Grass, M.: Motion-compensated iterative cone-beam CT image reconstruction with adapted blobs as basis functions. Phys. Med. Biol.

**53**(23), 6777 (2008)Karlsson, J., Ringh, A.: Generalized Sinkhorn iterations for regularizing inverse problems using optimal mass transport. SIAM J. Imaging Sci.

**10**(4), 1935–1962 (2017)Katsevich, A.: An accurate approximate algorithm for motion compensation in two-dimensional tomography. Inverse Probl.

**26**(6), 065007 (2010)Lu, W., Mackie, T.R.: Tomographic motion detection and correction directly in sinogram space. Phys. Med. Biol.

**47**(8), 1267 (2002)Mair, B.A., Gilland, D.R., Sun, J.: Estimation of images and nonrigid deformations in gated emission CT. IEEE Trans. Med. Imaging

**25**(9), 1130–1144 (2006)McLeod, K., Sermesant, M., Beerbaum, P., Pennec, X.: Spatio-temporal tensor decomposition of a polyaffine motion model for a better analysis of pathological left ventricular dynamics. IEEE Trans. Med. Imaging

**34**(7), 1562–1575 (2015)Modersitzki, J.: Numerical Methods for Image Registration. Oxford University Press, Oxford (2004)

Neumayer, S., Persch, J., Steidl, G.: Regularization of inverse problems via time discrete geodesics in image spaces. arXiv preprint arXiv:1805.06362 (2018)

Oektem, O., Chen, C., Domanic, N.O., Ravikumar, P., Bajaj, C.: Shape-based image reconstruction using linearized deformations. Inverse Probl.

**33**(3), 035004 (2017)Portman, N.: The modelling of biological growth: a pattern theoretic approach. Ph.D. thesis, University of Waterloo (2009)

Reyes, M., Malandain, G., Koulibaly, P.M., González-Ballester, M.A., Darcourt, J.: Model-based respiratory motion compensation for emission tomography image reconstruction. Phys. Med. Biol.

**52**(12), 3579 (2007)Rit, S., Wolthaus, J., van Herk, M., Sonke, J.-J.: On-the-fly motion-compensated cone-beam ct using an a priori motion model. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 729–736. Springer (2008)

Ritchie, C.J., Hsieh, J., Gard, M.F., Godwin, J.D., Kim, Y., Crawford, C.R.: Predictive respiratory gating: a new method to reduce motion artifacts on CT scans. Radiology

**190**(3), 847–852 (1994)Rohé, M.-M., Duchateau, N., Sermesant, M., Pennec, X.: Combination of polyaffine transformations and supervised learning for the automatic diagnosis of LV infarct. In: International Workshop on Statistical Atlases and Computational Models of the Heart, pp. 190–198. Springer (2015)

Seiler, C., Pennec, X., Reyes, M.: Capturing the multiscale anatomical shape variability with polyaffine transformation trees. Med. Image Anal.

**16**(7), 1371–1384 (2012)Srivastava, A., Saini, S., Ding, Z., Grenander, U.: Maximum-likelihood estimation of biological growth variables. In: Energy Minimization Methods in Computer Vision and Pattern Recognition, pp. 107–118. Springer (2005)

Trouvé, A., Younes, L.: Metamorphoses through lie group action. Found. Comput. Math.

**5**(2), 173–198 (2005)Van Eyndhoven, G., Sijbers, J., Batenburg, J.: Combined motion estimation and reconstruction in tomography. In European Conference on Computer Vision, pp. 12–21. Springer (2012)

Younes, L.: Shapes and Diffeomorphisms, 171st edn. Springer, New York (2010)

Younes, L.: Constrained diffeomorphic shape evolution. Found. Comput. Math.

**12**(3), 295–325 (2012)

## Acknowledgements

The work by Barbara Gris was supported by the Swedish Foundation for Strategic Research grant AM13-0049.

## Author information

### Authors and Affiliations

### Corresponding author

## Additional information

### Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Appendices

### Appendix A. Algorithm

The algorithm that we computed to obtain the numerical results of Sect. 5 is available at.^{Footnote 2} It is implemented in a more generic context than the one presented here, in particular, deformation modules that are not CTG can be used. It involves concepts and notation that are not introduced in this article so in the following we will only give an overview of the implementation and not a full presentation.

The algorithm is implemented in Python and relies on the class of objects DeformationModules. It uses the Operator Discretization Library (ODL)^{Footnote 3} in order to define discretized images and vector fields.

### 1.1 A.1 Deformation Modules

An abstract class DeformationModule is defined and contains the functions specifying the field generator \(\zeta \) and the cost. CTG modules form a particular sub-class named TranslationBased. In the implementation we simplify slightly the definition of CTG module by defining the cost by \(c :(o,h) \mapsto h^2\). Doing this, we simplify the operator \(C_o :H \mapsto H\) which becomes the Identity operator. Theoretically the UEC condition is no longer satisfied, this could lead to pathological trajectories such as non integrable time-varying vector fields. However, as long as the points of the geometrical descriptors do not converge to each others during the minimisation, the UEC condition is still satisfied because the norm of the generated vector field can be lower bounded with the norm of the control. As we do not observe such a convergence in practice, we can keep this simplified cost function.

### 1.2 A.2 Functional Computation

The constrained indirect registration consists in the minimisation of the functional (5) with respect to the initial geometrical descriptor \(a \in \mathcal {O}\) and the initial momentum \(\eta \in (\mathbb {R}^n)^m\). A first step is to compute this functional. As the computation of the regularisation terms \((a, \eta _0 ) \in \mathcal {O}\times (\mathbb {R}^n)^m \mapsto \gamma R_1 (a) + \tau R_2 (\eta )\) is straightforward, we concentrate here on the computation of the attachment term \((a, \eta _0 ) \in \mathcal {O}\times (\mathbb {R}^n)^m \mapsto D(T(\varphi ^{\zeta _o (h)}_{t=1} \cdot I_0), d)^2\) where *D* is the \(L^2\) distance on the range of the operator *T*. The only difficult step here is to compute the transported image \(\varphi ^{\zeta _o (h)}_{t=1} \cdot I_0\). This is done by integrating (4) via an Euler scheme and by transporting simultaneously the template image, see the sketch in Algorithm 1. In the particular case of a deformation module obtained by combination of CTG modules, the complexity of this forward integration is \(O (N m p^2)\) with *m* the total numbers of points of the compound geometrical descriptor and \(p^2\) the number of pixels of the image.

### 1.3 A.3 Gradient Computation

As previously, the gradient of the regularisation terms \((a, \eta _0 ) \in \mathcal {O}\times (\mathbb {R}^n)^m \mapsto \gamma R_1 (a) + \tau R_2 (\eta )\) is straightforward but the gradient of the attachment term \((a, \eta _0 ) \in \mathcal {O}\times (\mathbb {R}^n)^m \mapsto D(T(\varphi ^{\zeta _o (h)}_{t=1} \cdot I_0), d)^2\) requires explanations. We use a *forward–backward* scheme based on the following result (which is a simplified version of a more general principle, see for instance [3]):

### Proposition 8

Let \(p \in \mathbb {N}\), \(f :\mathbb {R}^p \mapsto \mathbb {R}^p \) a \(C^j\) vector field with \(j \ge 1\) and \(G :q_0 \in \mathbb {R}^p \mapsto S(q(t=1))\) with \(S :\mathbb {R}^p \mapsto \mathbb {R}\)\(C^1\) and \(q :[0, 1] \mapsto \mathbb {R}^p\) defined by \(q(t=0) = q_0\) and

Then for all \(q_0 \in \mathbb {R}^p\), \(\nabla G (q_0) = Z(0)\) where \(Z :[0, 1] \mapsto \mathbb {R}^p\) is defined by \(Z(1) = \nabla S(q(t=1))\) and

This is called a *forward–backward scheme* because it consists in a forward step where Eq. 6 is integrated, then the variable *Z* is initialised at \(Z(1) = \nabla S(q(t=1))\) and integrated backward following Eq. 7. The computation of \( \mathrm {d}f (q(t)) ^T\) can be quite hard in practice but it can be simply approximated using the following result [3]:

### Proposition 9

Suppose that \(p = 2 p_1\) and that there exists \(\mathcal {H} :q \mapsto \mathbb {R}\) such that we can write, for \(q = (o, \eta ) \in \mathbb {R}^{p_1} \times \mathbb {R}^{p_1}\), \(f (q) = (\nabla _{\eta } \mathcal {H} (o, \eta ) , - \nabla _o \mathcal {H} ( o, \eta ) )\). Then for \(Z = (Z_1 , Z_2) \in \mathbb {R}^{p_1} \times \mathbb {R}^{p_1}\), \( \mathrm {d}f (q(t)) ^T Z = \mathrm {d}\Big ( \nabla \mathcal {H} \Big ) (o, \eta ) \cdot (- Z_2, Z_1)\).

The latter quantity is a directional derivative and can be approximated by a finit difference.

We apply these results on a discretization of \(\mathcal {O}\times (\mathbb {R}^n)^m \times L^2(\Omega , \mathbb {R})\), with the function *f* defined in the Algorithm 1. We are currently working on a new implementation of the gradient evaluation using automatic differentiation.

### B Summary of Notation

Notation | Signification |
---|---|

| Dimension of the ambient space and the images (2 or 3) |

\(Diff^\ell _0 (\Omega )\) | Space of diffeomorphisms (see Sect. 2.3) |

\(C^\ell _0 (\Omega , \mathbb {R}^n)\) | Space of vector fields (see Sect. 2.3) |

| Forward operator |

| Space of images (reconstruction space) |

| Data space |

| data |

\(\phi \) | Diffeomorphism |

| vector field |

\(\varphi ^v\) | Flow of |

\(K_\sigma \) | Scalar Gaussian kernel of scale \(\sigma \) (see Sect. 3.1) |

| Deformation module |

\(\mathcal {O}\) | Space of geometrical descriptors |

| Geometrical descriptor |

| Space of controls |

| Control |

\(h^*\) | Geodesic control (see Equation (4)) |

\(\zeta \) | Field generator |

| Cost |

\(\eta \) | Momentum (see Equation (4)) |

## Rights and permissions

## About this article

### Cite this article

Gris, B. Incorporation of a Deformation Prior in Image Reconstruction.
*J Math Imaging Vis* **61**, 691–709 (2019). https://doi.org/10.1007/s10851-018-0868-z

Received:

Accepted:

Published:

Issue Date:

DOI: https://doi.org/10.1007/s10851-018-0868-z