Advertisement

LSTM Spatial Co-transformer Networks for Registration of 3D Fetal US and MR Brain Images

  • Robert Wright
  • Bishesh Khanal
  • Alberto Gomez
  • Emily Skelton
  • Jacqueline Matthew
  • Jo V. Hajnal
  • Daniel Rueckert
  • Julia A. Schnabel
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11076)

Abstract

In this work, we propose a deep learning-based method for iterative registration of fetal brain images acquired by ultrasound and magnetic resonance, inspired by “Spatial Transformer Networks”. Images are co-aligned to a dual modality spatio-temporal atlas, where computational image analysis may be performed in the future. Our results show better alignment accuracy compared to “Self-Similarity Context descriptors”, a state-of-the-art method developed for multi-modal image registration. Furthermore, our method is robust and able to register highly misaligned images, with any initial orientation, where similarity-based methods typically fail.

Notes

Acknowledgements

This work was supported by the Wellcome/EPSRC Centre for Medical Engineering [WT 203148/Z/16/Z], Wellcome Trust IEH Award [102431] and NVIDIA with the donation of a Titan Xp GPU.

References

  1. 1.
    Balakrishnan, G., Zhao, A., Sabuncu, M.R., Guttag, J., Dalca, A.V.: An unsupervised learning model for deformable medical image registration. In: CVPR (2018)Google Scholar
  2. 2.
    Cheng, X., Zhang, L., Zheng, Y.: Deep similarity learning for multimodal medical images. Comput. Meth. Biomech. Biomed. Eng. Imaging Vis. 6(3), 248–252 (2018)CrossRefGoogle Scholar
  3. 3.
    Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: EMNLP, pp. 1724–1734 (2014)Google Scholar
  4. 4.
    Haber, E., Modersitzki, J.: Intensity gradient based registration and fusion of multi-modal images. In: Larsen, R., Nielsen, M., Sporring, J. (eds.) MICCAI 2006. LNCS, vol. 4191, pp. 726–733. Springer, Heidelberg (2006).  https://doi.org/10.1007/11866763_89CrossRefGoogle Scholar
  5. 5.
    Heinrich, M.P., Jenkinson, M., Papież, B.W., Brady, S.M., Schnabel, J.A.: Towards realtime multimodal fusion for image-guided interventions using self-similarities. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013. LNCS, vol. 8149, pp. 187–194. Springer, Heidelberg (2013).  https://doi.org/10.1007/978-3-642-40811-3_24CrossRefGoogle Scholar
  6. 6.
    Jaderberg, M., Simonyan, K., Zisserman, A., Kavukcuoglu, K.: Spatial transformer networks. In: NIPS, pp. 2017–2025 (2015)Google Scholar
  7. 7.
    Kuklisova-Murgasova, M., et al.: Registration of 3D fetal brain US and MRI. In: MICCAI, pp. 667–674 (2012)CrossRefGoogle Scholar
  8. 8.
    Lin, C.H., Lucey, S.: Inverse compositional spatial transformer networks. In: CVPR, pp. 2252–2260 (2017)Google Scholar
  9. 9.
    Mellor, M., Brady, M.: Phase mutual information as a similarity measure for registration. Med. Image Anal. 9(4), 330–343 (2005). Functional Imaging and Modeling of the Heart - FIMH 2003Google Scholar
  10. 10.
    Ourselin, S., Roche, A., Subsol, G., Pennec, X., Ayache, N.: Reconstructing a 3D structure from serial histological sections. Image Vis. Comput. 19(1), 25–31 (2001)CrossRefGoogle Scholar
  11. 11.
    Pech-Pacheco, J.L., Cristobal, G., Chamorro-Martinez, J., Fernandez-Valdivia, J.: Diatom autofocusing in brightfield microscopy: a comparative study. ICPR 3, 314–317 (2000)Google Scholar
  12. 12.
    Rivaz, H., Karimaghaloo, Z., Collins, D.L.: Self-similarity weighted mutual information: a new nonrigid image registration metric. Med. Image Anal. 18(2), 343–358 (2014)CrossRefGoogle Scholar
  13. 13.
    Rohé, M.-M., Datar, M., Heimann, T., Sermesant, M., Pennec, X.: SVF-Net: learning deformable image registration using shape matching. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10433, pp. 266–274. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-66182-7_31CrossRefGoogle Scholar
  14. 14.
    Serag, A., Aljabar, P., Ball, G., Counsell, S.J., Boardman, J.P., Rutherford, M.A., Edwards, A.D., Hajnal, J.V., Rueckert, D.: Construction of a consistent high-definition spatio-temporal atlas of the developing brain using adaptive Kernel regression. NeuroImage 59(3), 2255–2265 (2012)CrossRefGoogle Scholar
  15. 15.
    Simonovsky, M., Gutiérrez-Becker, B., Mateus, D., Navab, N., Komodakis, N.: A deep metric for multimodal registration. In: MICCAI, pp. 10–18 (2016)Google Scholar
  16. 16.
    Wachinger, C., Navab, N.: Entropy and Laplacian images: structural representations for multi-modal registration. Med. Image Anal. 16(1), 1–17 (2012)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Robert Wright
    • 1
  • Bishesh Khanal
    • 1
  • Alberto Gomez
    • 1
  • Emily Skelton
    • 1
  • Jacqueline Matthew
    • 1
  • Jo V. Hajnal
    • 1
  • Daniel Rueckert
    • 2
  • Julia A. Schnabel
    • 1
  1. 1.School of Biomedical Engineering and Imaging SciencesKing’s College LondonLondonUK
  2. 2.Department of ComputingImperial College LondonLondonUK

Personalised recommendations