Skip to main content

Incorporation of a Deformation Prior in Image Reconstruction

Abstract

This article presents a method to incorporate a deformation prior in image reconstruction via the formalism of deformation modules. The framework of deformation modules allows to build diffeomorphic deformations that satisfy a given structure. The idea is to register a template image against the indirectly observed data via a modular deformation, incorporating this way the deformation prior in the reconstruction method. We show that this is a well-defined regularisation method (proving existence, stability and convergence) and present numerical examples of reconstruction from 2-D tomographic simulations and partially observed images.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Notes

  1. 1.

    https://github.com/odlgroup/odl.

  2. 2.

    https://github.com/bgris/ConstrainedIndirectRegistration.

  3. 3.

    https://github.com/odlgroup/odl.

References

  1. 1.

    Abraham, I., Abraham, R., Bergounioux, M., Carlier, G.: Tomographic reconstruction from a few views: a multi-marginal optimal transport approach. Appl. Math. Optim. 75(1), 55–73 (2017)

    MathSciNet  Article  MATH  Google Scholar 

  2. 2.

    Amit, Y., Grenander, U., Piccioni, M.: Structural image restoration through deformable templates. J. Am. Stat. Assoc. 86(414), 376–387 (1991)

    Article  Google Scholar 

  3. 3.

    Arguillere, S.: Géométrie sous-riemannienne en dimension infinie et applications à l’analyse mathématique des formes. Ph.D. thesis, Paris 6 (2014)

  4. 4.

    Arguillere, S., Trélat, E., Trouvé, A., Younes, L.: Shape deformation analysis from the optimal control viewpoint. Journal de mathématiques pures et appliquées 104(1), 139–178 (2015)

    MathSciNet  Article  MATH  Google Scholar 

  5. 5.

    Aronszajn, N.: Theory of reproducing kernels. Trans. Am. Math. Soc. 68(3), 337–404 (1950)

    MathSciNet  Article  MATH  Google Scholar 

  6. 6.

    Arsigny, V., Commowick, O., Ayache, N., Pennec, X.: A fast and log-euclidean polyaffine framework for locally linear registration. J. Math. Imaging Vis. 33(2), 222–238 (2009)

    MathSciNet  Article  Google Scholar 

  7. 7.

    Arsigny, V., Pennec, X., Ayache, N.: Polyrigid and polyaffine transformations: a novel geometrical tool to deal with non-rigid deformations-application to the registration of histological slices. Med. Image Anal. 9(6), 507–523 (2005)

    Article  Google Scholar 

  8. 8.

    Beg, M.F., Miller, M.I., Trouvé, A., Younes, L.: Computing large deformation metric mappings via geodesic flows of diffeomorphisms. Int. J. Comput. Vis. 61(2), 139–157 (2005)

    Article  Google Scholar 

  9. 9.

    Benamou, J.-D., Carlier, G., Cuturi, M., Nenna, L., Peyré, G.: Iterative Bregman projections for regularized transportation problems. SIAM J. Sci. Comput. 37(2), A1111–A1138 (2015)

    MathSciNet  Article  MATH  Google Scholar 

  10. 10.

    Berkels, B., Effland, A., Rumpf, M.: Time discrete geodesic paths in the space of images. SIAM J. Imaging Sci. 8(3), 1457–1488 (2015)

    MathSciNet  Article  MATH  Google Scholar 

  11. 11.

    Blume, M., Martinez-Moller, A., Keil, A., Navab, N., Rafecas, M.: Joint reconstruction of image and motion in gated positron emission tomography. IEEE Trans. Med. Imaging 29(11), 1892–1906 (2010)

    Article  Google Scholar 

  12. 12.

    Bruveris, M., Holm, D.D.: Geometry of image registration: the diffeomorphism group and momentum maps. In: Geometry, Mechanics, and Dynamics, pp. 19–56. Springer (2015)

  13. 13.

    Burger, M., Dirks, H., Frerking, L., Hauptmann, A., Helin, T., Siltanen, S.: A variational reconstruction method for undersampled dynamic X-ray tomography based on physical motion models. Inverse Probl. 33(12), 124008 (2017)

    MathSciNet  Article  MATH  Google Scholar 

  14. 14.

    Chen, C., Öktem, O.: Indirect image registration with large diffeomorphic deformations. SIAM J. Imaging Sci. 11(1), 575–617 (2018)

    MathSciNet  Article  MATH  Google Scholar 

  15. 15.

    Ehrhardt, J., Werner, R., Säring, D., Frenzel, T., Lu, W., Low, D., Handels, H.: An optical flow based method for improved reconstruction of 4D CT data sets acquired during free breathing. Med. Phys. 34(2), 711–721 (2007)

    Article  Google Scholar 

  16. 16.

    Grenander, U., Srivastava, A., Saini, S.: A pattern-theoretic characterization of biological growth. IEEE Trans. Med. Imaging 26(5), 648–659 (2007)

    Article  Google Scholar 

  17. 17.

    Gris, B.: Modular approach on shape spaces, sub-Riemannian geometry and computational anatomy. Ph.D. thesis, Université Paris-Saclay (2016)

  18. 18.

    Gris, B., Durrleman, S., Trouvé, A.: A sub-riemannian modular framework for diffeomorphism based analysis of shape ensembles. SIAM J. Imaging Sci. 11(1), 802–833 (2016)

    MathSciNet  Article  MATH  Google Scholar 

  19. 19.

    Gris, B., Öktem, O.: Image reconstruction through metamorphosis. arXiv preprint arXiv:1806.01225v2 (2018)

  20. 20.

    Haber, E., Modersitzki, J.: A multilevel method for image registration. SIAM J. Sci. Comput. 27(5), 1594–1607 (2006)

    MathSciNet  Article  MATH  Google Scholar 

  21. 21.

    Hahn, B.N.: Motion estimation and compensation strategies in dynamic computerized tomography. Sens. Imaging 18(1), 10 (2017)

    Article  Google Scholar 

  22. 22.

    Hinkle, J., Szegedi, M., Wang, B., Salter, B., Joshi, S.: 4D CT image reconstruction with diffeomorphic motion model. Med. Image Anal. 16(6), 1307–1316 (2012)

    Article  Google Scholar 

  23. 23.

    Isola, A., Ziegler, A., Koehler, T., Niessen, W., Grass, M.: Motion-compensated iterative cone-beam CT image reconstruction with adapted blobs as basis functions. Phys. Med. Biol. 53(23), 6777 (2008)

    Article  Google Scholar 

  24. 24.

    Karlsson, J., Ringh, A.: Generalized Sinkhorn iterations for regularizing inverse problems using optimal mass transport. SIAM J. Imaging Sci. 10(4), 1935–1962 (2017)

    MathSciNet  Article  MATH  Google Scholar 

  25. 25.

    Katsevich, A.: An accurate approximate algorithm for motion compensation in two-dimensional tomography. Inverse Probl. 26(6), 065007 (2010)

    MathSciNet  Article  MATH  Google Scholar 

  26. 26.

    Lu, W., Mackie, T.R.: Tomographic motion detection and correction directly in sinogram space. Phys. Med. Biol. 47(8), 1267 (2002)

    Article  Google Scholar 

  27. 27.

    Mair, B.A., Gilland, D.R., Sun, J.: Estimation of images and nonrigid deformations in gated emission CT. IEEE Trans. Med. Imaging 25(9), 1130–1144 (2006)

    Article  Google Scholar 

  28. 28.

    McLeod, K., Sermesant, M., Beerbaum, P., Pennec, X.: Spatio-temporal tensor decomposition of a polyaffine motion model for a better analysis of pathological left ventricular dynamics. IEEE Trans. Med. Imaging 34(7), 1562–1575 (2015)

    Article  Google Scholar 

  29. 29.

    Modersitzki, J.: Numerical Methods for Image Registration. Oxford University Press, Oxford (2004)

    MATH  Google Scholar 

  30. 30.

    Neumayer, S., Persch, J., Steidl, G.: Regularization of inverse problems via time discrete geodesics in image spaces. arXiv preprint arXiv:1805.06362 (2018)

  31. 31.

    Oektem, O., Chen, C., Domanic, N.O., Ravikumar, P., Bajaj, C.: Shape-based image reconstruction using linearized deformations. Inverse Probl. 33(3), 035004 (2017)

    MathSciNet  Article  MATH  Google Scholar 

  32. 32.

    Portman, N.: The modelling of biological growth: a pattern theoretic approach. Ph.D. thesis, University of Waterloo (2009)

  33. 33.

    Reyes, M., Malandain, G., Koulibaly, P.M., González-Ballester, M.A., Darcourt, J.: Model-based respiratory motion compensation for emission tomography image reconstruction. Phys. Med. Biol. 52(12), 3579 (2007)

    Article  Google Scholar 

  34. 34.

    Rit, S., Wolthaus, J., van Herk, M., Sonke, J.-J.: On-the-fly motion-compensated cone-beam ct using an a priori motion model. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 729–736. Springer (2008)

  35. 35.

    Ritchie, C.J., Hsieh, J., Gard, M.F., Godwin, J.D., Kim, Y., Crawford, C.R.: Predictive respiratory gating: a new method to reduce motion artifacts on CT scans. Radiology 190(3), 847–852 (1994)

    Article  Google Scholar 

  36. 36.

    Rohé, M.-M., Duchateau, N., Sermesant, M., Pennec, X.: Combination of polyaffine transformations and supervised learning for the automatic diagnosis of LV infarct. In: International Workshop on Statistical Atlases and Computational Models of the Heart, pp. 190–198. Springer (2015)

  37. 37.

    Seiler, C., Pennec, X., Reyes, M.: Capturing the multiscale anatomical shape variability with polyaffine transformation trees. Med. Image Anal. 16(7), 1371–1384 (2012)

    Article  Google Scholar 

  38. 38.

    Srivastava, A., Saini, S., Ding, Z., Grenander, U.: Maximum-likelihood estimation of biological growth variables. In: Energy Minimization Methods in Computer Vision and Pattern Recognition, pp. 107–118. Springer (2005)

  39. 39.

    Trouvé, A., Younes, L.: Metamorphoses through lie group action. Found. Comput. Math. 5(2), 173–198 (2005)

    MathSciNet  Article  MATH  Google Scholar 

  40. 40.

    Van Eyndhoven, G., Sijbers, J., Batenburg, J.: Combined motion estimation and reconstruction in tomography. In European Conference on Computer Vision, pp. 12–21. Springer (2012)

  41. 41.

    Younes, L.: Shapes and Diffeomorphisms, 171st edn. Springer, New York (2010)

    Book  MATH  Google Scholar 

  42. 42.

    Younes, L.: Constrained diffeomorphic shape evolution. Found. Comput. Math. 12(3), 295–325 (2012)

    MathSciNet  Article  MATH  Google Scholar 

Download references

Acknowledgements

The work by Barbara Gris was supported by the Swedish Foundation for Strategic Research grant AM13-0049.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Barbara Gris.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix A. Algorithm

The algorithm that we computed to obtain the numerical results of Sect. 5 is available at.Footnote 2 It is implemented in a more generic context than the one presented here, in particular, deformation modules that are not CTG can be used. It involves concepts and notation that are not introduced in this article so in the following we will only give an overview of the implementation and not a full presentation.

The algorithm is implemented in Python and relies on the class of objects DeformationModules. It uses the Operator Discretization Library (ODL)Footnote 3 in order to define discretized images and vector fields.

A.1 Deformation Modules

An abstract class DeformationModule is defined and contains the functions specifying the field generator \(\zeta \) and the cost. CTG modules form a particular sub-class named TranslationBased. In the implementation we simplify slightly the definition of CTG module by defining the cost by \(c :(o,h) \mapsto h^2\). Doing this, we simplify the operator \(C_o :H \mapsto H\) which becomes the Identity operator. Theoretically the UEC condition is no longer satisfied, this could lead to pathological trajectories such as non integrable time-varying vector fields. However, as long as the points of the geometrical descriptors do not converge to each others during the minimisation, the UEC condition is still satisfied because the norm of the generated vector field can be lower bounded with the norm of the control. As we do not observe such a convergence in practice, we can keep this simplified cost function.

A.2 Functional Computation

The constrained indirect registration consists in the minimisation of the functional (5) with respect to the initial geometrical descriptor \(a \in \mathcal {O}\) and the initial momentum \(\eta \in (\mathbb {R}^n)^m\). A first step is to compute this functional. As the computation of the regularisation terms \((a, \eta _0 ) \in \mathcal {O}\times (\mathbb {R}^n)^m \mapsto \gamma R_1 (a) + \tau R_2 (\eta )\) is straightforward, we concentrate here on the computation of the attachment term \((a, \eta _0 ) \in \mathcal {O}\times (\mathbb {R}^n)^m \mapsto D(T(\varphi ^{\zeta _o (h)}_{t=1} \cdot I_0), d)^2\) where D is the \(L^2\) distance on the range of the operator T. The only difficult step here is to compute the transported image \(\varphi ^{\zeta _o (h)}_{t=1} \cdot I_0\). This is done by integrating (4) via an Euler scheme and by transporting simultaneously the template image, see the sketch in Algorithm 1. In the particular case of a deformation module obtained by combination of CTG modules, the complexity of this forward integration is \(O (N m p^2)\) with m the total numbers of points of the compound geometrical descriptor and \(p^2\) the number of pixels of the image.

figureb

A.3 Gradient Computation

As previously, the gradient of the regularisation terms \((a, \eta _0 ) \in \mathcal {O}\times (\mathbb {R}^n)^m \mapsto \gamma R_1 (a) + \tau R_2 (\eta )\) is straightforward but the gradient of the attachment term \((a, \eta _0 ) \in \mathcal {O}\times (\mathbb {R}^n)^m \mapsto D(T(\varphi ^{\zeta _o (h)}_{t=1} \cdot I_0), d)^2\) requires explanations. We use a forward–backward scheme based on the following result (which is a simplified version of a more general principle, see for instance [3]):

Proposition 8

Let \(p \in \mathbb {N}\), \(f :\mathbb {R}^p \mapsto \mathbb {R}^p \) a \(C^j\) vector field with \(j \ge 1\) and \(G :q_0 \in \mathbb {R}^p \mapsto S(q(t=1))\) with \(S :\mathbb {R}^p \mapsto \mathbb {R}\)\(C^1\) and \(q :[0, 1] \mapsto \mathbb {R}^p\) defined by \(q(t=0) = q_0\) and

$$\begin{aligned} \frac{\mathrm {d}}{\mathrm {d}t} {q} (t) = f(q (t) ) \,. \end{aligned}$$
(6)

Then for all \(q_0 \in \mathbb {R}^p\), \(\nabla G (q_0) = Z(0)\) where \(Z :[0, 1] \mapsto \mathbb {R}^p\) is defined by \(Z(1) = \nabla S(q(t=1))\) and

$$\begin{aligned} \frac{\mathrm {d}}{\mathrm {d}t} {Z} (t) = \mathrm {d}f (q_t) ^T Z (t) \end{aligned}$$
(7)

This is called a forward–backward scheme because it consists in a forward step where Eq. 6 is integrated, then the variable Z is initialised at \(Z(1) = \nabla S(q(t=1))\) and integrated backward following Eq. 7. The computation of \( \mathrm {d}f (q(t)) ^T\) can be quite hard in practice but it can be simply approximated using the following result [3]:

Proposition 9

Suppose that \(p = 2 p_1\) and that there exists \(\mathcal {H} :q \mapsto \mathbb {R}\) such that we can write, for \(q = (o, \eta ) \in \mathbb {R}^{p_1} \times \mathbb {R}^{p_1}\), \(f (q) = (\nabla _{\eta } \mathcal {H} (o, \eta ) , - \nabla _o \mathcal {H} ( o, \eta ) )\). Then for \(Z = (Z_1 , Z_2) \in \mathbb {R}^{p_1} \times \mathbb {R}^{p_1}\), \( \mathrm {d}f (q(t)) ^T Z = \mathrm {d}\Big ( \nabla \mathcal {H} \Big ) (o, \eta ) \cdot (- Z_2, Z_1)\).

The latter quantity is a directional derivative and can be approximated by a finit difference.

We apply these results on a discretization of \(\mathcal {O}\times (\mathbb {R}^n)^m \times L^2(\Omega , \mathbb {R})\), with the function f defined in the Algorithm 1. We are currently working on a new implementation of the gradient evaluation using automatic differentiation.

B Summary of Notation

Notation Signification
n Dimension of the ambient space and the images (2 or 3)
\(Diff^\ell _0 (\Omega )\) Space of diffeomorphisms (see Sect. 2.3)
\(C^\ell _0 (\Omega , \mathbb {R}^n)\) Space of vector fields (see Sect. 2.3)
T Forward operator
X Space of images (reconstruction space)
Y Data space
d data
\(\phi \) Diffeomorphism
v vector field
\(\varphi ^v\) Flow of v (see Equation (1))
\(K_\sigma \) Scalar Gaussian kernel of scale \(\sigma \) (see Sect. 3.1)
M Deformation module
\(\mathcal {O}\) Space of geometrical descriptors
o Geometrical descriptor
H Space of controls
h Control
\(h^*\) Geodesic control (see Equation (4))
\(\zeta \) Field generator
c Cost
\(\eta \) Momentum (see Equation (4))

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Gris, B. Incorporation of a Deformation Prior in Image Reconstruction. J Math Imaging Vis 61, 691–709 (2019). https://doi.org/10.1007/s10851-018-0868-z

Download citation

Keywords

  • Image reconstruction
  • Inverse problem
  • Diffeomorphic deformation
  • Deformation prior
  • Image matching