Spatiotemporal Modeling for Image Time Series with Appearance Change: Application to Early Brain Development

  • James FishbaughEmail author
  • Martin Styner
  • Karen Grewen
  • John Gilmore
  • Guido Gerig
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11846)


There has been considerable research effort into image registration and regression, which address the problem of determining correspondence primarily through estimating models of structural change. There has been far less focus into methods which model both structural and intensity change. However, medical images often exhibit intensity changes over time. Of particular interest is MRI of the early developing brain, where such intensity change encodes rich information about development, such as rapidly increasing white matter intensity during the first years of life. In this paper, we develop a new spatiotemporal model which takes into account both structural and appearance changes jointly. This will not only lead to improved regression accuracy and data-matching in the presence of longitudinal intensity changes, but also facilitate the study of development by direct analysis of appearance change models. We propose to combine a diffeomorphic model of structural change with a Gompertz intensity model, which captures intensity trajectories with 3 intuitive parameters of asymptote, delay, and speed. We propose an optimization scheme which allows to control the balance between structural and intensity change via two data-matching terms. We show that Gompertz parameter maps show great promise to characterize regional patterns of development.



This work was supported by NIH grants NIBIB R01EB021391 (SlicerSALT), 1R01HD088125-01A1 (Down’s Syndrome), 2R01HD055741-11 (ACE-IBIS), 1R01DA038215-01A1 (Cocaine Effects) and the New York Center for Advanced Technology in Telecommunications (CATT). HPC resources used for this research provided by grant NSF MRI-1229185.


  1. 1.
    Rutherford, M.A., Bydder, G.M.: MRI of the Neonatal Brain. WB Saunders, London (2002)Google Scholar
  2. 2.
    Vardhan, A., Fishbaugh, J., Vachet, C., Gerig, G.: Longitudinal modeling of multi-modal image contrast reveals patterns of early brain growth. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10433, pp. 75–83. Springer, Cham (2017). Scholar
  3. 3.
    Niethammer, M., et al.: Geometric metamorphosis. In: Fichtinger, G., Martel, A., Peters, T. (eds.) MICCAI 2011. LNCS, vol. 6892, pp. 639–646. Springer, Heidelberg (2011). Scholar
  4. 4.
    Miller, M.I., Younes, L.: Group actions, homeomorphisms, and matching: a general framework. Int. J. Comput. Vis. 41(1–2), 61–84 (2001)CrossRefGoogle Scholar
  5. 5.
    Hong, Y., Joshi, S., Sanchez, M., Styner, M., Niethammer, M.: Metamorphic geodesic regression. In: Ayache, N., Delingette, H., Golland, P., Mori, K. (eds.) MICCAI 2012. LNCS, vol. 7512, pp. 197–205. Springer, Heidelberg (2012). Scholar
  6. 6.
    Periaswamy, S., Farid, H.: Elastic registration in the presence of intensity variations. IEEE Trans. Med. Imaging 22(7), 865–874 (2003)CrossRefGoogle Scholar
  7. 7.
    Gao, Y., Zhang, M., Grewen, K., Fletcher, P.T., Gerig, G.: Image registration and segmentation in longitudinal MRI using temporal appearance modeling. In: IEEE ISBI, pp. 629–632 (2016)Google Scholar
  8. 8.
    Csapo, I., Davis, B., Shi, Y., Sanchez, M., Styner, M., Niethammer, M.: Longitudinal image registration with temporally-dependent image similarity measure. IEEE Trans. Med. Imaging 32(10), 1939–1951 (2013)CrossRefGoogle Scholar
  9. 9.
    Bhushan, M., Schnabel, J.A., Risser, L., Heinrich, M.P., Brady, J.M., Jenkinson, M.: Motion correction and parameter estimation in dceMRI sequences: application to colorectal cancer. In: Fichtinger, G., Martel, A., Peters, T. (eds.) MICCAI 2011. LNCS, vol. 6891, pp. 476–483. Springer, Heidelberg (2011). Scholar
  10. 10.
    Fishbaugh, J., Gerig, G.: Acceleration controlled diffeomorphisms for nonparametric image regression. In: IEEE ISBI, pp. 1488–1491 (2019)Google Scholar
  11. 11.
    Niethammer, M., Huang, Y., Vialard, F.-X.: Geodesic regression for image time-series. In: Fichtinger, G., Martel, A., Peters, T. (eds.) MICCAI 2011. LNCS, vol. 6892, pp. 655–662. Springer, Heidelberg (2011). Scholar
  12. 12.
    Singh, N., Vialard, F.X., Niethammer, M.: Splines for diffeomorphisms. Med. Image Anal. 25(1), 56–71 (2015)CrossRefGoogle Scholar
  13. 13.
    Hinkle, J., Fletcher, P.T., Joshi, S.: Intrinsic polynomials for regression on riemannian manifolds. J. Math. Imaging Vis. 50(1–2), 32–52 (2014)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Dobbing, J., Sands, J.: Quantitative growth and development of human brain. Arch. Dis. Child. 48(10), 757–767 (1973)CrossRefGoogle Scholar
  15. 15.
    Fishbaugh, J., Durrleman, S., Gerig, G.: Estimation of smooth growth trajectories with controlled acceleration from time series shape data. In: Fichtinger, G., Martel, A., Peters, T. (eds.) MICCAI 2011. LNCS, vol. 6892, pp. 401–408. Springer, Heidelberg (2011). Scholar
  16. 16.
    Sadeghi, N., Prastawa, M., Fletcher, P.T., Wolff, J., Gilmore, J.H., Gerig, G.: Regional characterization of longitudinal dt-mri to study white matter maturation of the early developing brain. Neuroimage 68, 236–247 (2013)CrossRefGoogle Scholar
  17. 17.
    Paszke, A., et al.: Automatic differentiation in pytorch (2017)Google Scholar
  18. 18.
    Bône, A., Louis, M., Martin, B., Durrleman, S.: Deformetrica 4: an open-source software for statistical shape analysis. In: Shape in Medical Imaging, pp. 3–13 (2018)Google Scholar
  19. 19.
    Sweeney, E., Shinohara, R., Shea, C., Reich, D., Crainiceanu, C.M.: Automatic lesion incidence estimation and detection in multiple sclerosis using multisequence longitudinal mri. Am. J. Neuroradiol. 34(1), 68–73 (2013)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • James Fishbaugh
    • 1
    Email author
  • Martin Styner
    • 2
    • 3
  • Karen Grewen
    • 3
  • John Gilmore
    • 3
  • Guido Gerig
    • 1
  1. 1.Department of Computer Science and Engineering, Tandon School of EngineeringNYUNew YorkUK
  2. 2.Department of Computer ScienceUNC Chapel HillChapel HillUSA
  3. 3.Department of PsychiatryUNC School of MedicineChapel HillUSA

Personalised recommendations