Advertisement

Predicting Slice-to-Volume Transformation in Presence of Arbitrary Subject Motion

  • Benjamin HouEmail author
  • Amir Alansary
  • Steven McDonagh
  • Alice Davidson
  • Mary Rutherford
  • Jo V. Hajnal
  • Daniel Rueckert
  • Ben Glocker
  • Bernhard Kainz
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10434)

Abstract

This paper aims to solve a fundamental problem in intensity-based 2D/3D registration, which concerns the limited capture range and need for very good initialization of state-of-the-art image registration methods. We propose a regression approach that learns to predict rotations and translations of arbitrary 2D image slices from 3D volumes, with respect to a learned canonical atlas co-ordinate system. To this end, we utilize Convolutional Neural Networks (CNNs) to learn the highly complex regression function that maps 2D image slices into their correct position and orientation in 3D space. Our approach is attractive in challenging imaging scenarios, where significant subject motion complicates reconstruction performance of 3D volumes from 2D slice data. We extensively evaluate the effectiveness of our approach quantitatively on simulated MRI brain data with extreme random motion. We further demonstrate qualitative results on fetal MRI where our method is integrated into a full reconstruction and motion compensation pipeline. With our CNN regression approach we obtain an average prediction error of 7 mm on simulated data, and convincing reconstruction quality of images of very young fetuses where previous methods fail. We further discuss applications to Computed Tomography (CT) and X-Ray projections. Our approach is a general solution to the 2D/3D initialization problem. It is computationally efficient, with prediction times per slice of a few milliseconds, making it suitable for real-time scenarios.

Notes

Acknowledgements

NVIDIA, Wellcome Trust/EPSRC iFIND [102431], EPSRC EP/N024494/1.

Supplementary material

451304_1_En_34_MOESM1_ESM.pdf (1.9 mb)
Supplementary material 1 (pdf 1899 KB)

References

  1. 1.
    Alzheimer’s Disease Neuroimaging Initiative (2017). http://adni.loni.usc.edu
  2. 2.
    Aljabar, P., et al.: Multi-atlas based segmentation of brain images: atlas selection and its effect on accuracy. NeuroImage 46(3), 726–738 (2009)CrossRefGoogle Scholar
  3. 3.
    Ghesu, F.C., Georgescu, B., Mansi, T., Neumann, D., Hornegger, J., Comaniciu, D.: An artificial agent for anatomical landmark detection in medical images. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016, Part III. LNCS, vol. 9902, pp. 229–237. Springer, Cham (2016). doi: 10.1007/978-3-319-46726-9_27CrossRefGoogle Scholar
  4. 4.
    Gholipour, A., et al.: Robust super-resolution volume reconstruction from slice acquisitions: application to fetal brain MRI. IEEE TMI 29(10), 1739–1758 (2010)Google Scholar
  5. 5.
    González, Á.: Measurement of areas on a sphere using fibonacci and latitude-longitude lattices. Math. Geosci. 42(1), 49 (2009)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Huynh, D.Q.: Metrics for 3D rotations: comparison and analysis. J. Math. Imaging Vis. 35(2), 155–164 (2009)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Jia, Y., et al.: Caffe: convolutional architecture for fast feature embedding. arXiv:1408.5093 (2014)
  8. 8.
    Kainz, B., et al.: Fast volume reconstruction from motion corrupted stacks of 2D slices. IEEE Trans. Med. Imaging 34(9), 1901–1913 (2015)CrossRefGoogle Scholar
  9. 9.
    Kainz, B., Grabner, M., Rüther, M.: Fast marker based C-Arm pose estimation. In: Metaxas, D., Axel, L., Fichtinger, G., Székely, G. (eds.) MICCAI 2008, Part II. LNCS, vol. 5242, pp. 652–659. Springer, Heidelberg (2008). doi: 10.1007/978-3-540-85990-1_78CrossRefGoogle Scholar
  10. 10.
    Keraudren, K., Kainz, B., Oktay, O., Kyriakopoulou, V., Rutherford, M., Hajnal, J.V., Rueckert, D.: Automated localization of fetal organs in MRI using random forests with steerable features. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015, Part III. LNCS, vol. 9351, pp. 620–627. Springer, Cham (2015). doi: 10.1007/978-3-319-24574-4_74CrossRefGoogle Scholar
  11. 11.
    Kim, K., et al.: Intersection based motion correction of multislice MRI for 3D in utero fetal brain image formation. Trans. Med. Imaging 29(1), 146–158 (2010)CrossRefGoogle Scholar
  12. 12.
    LeCun, Y., et al.: Deep learning. Nature 521(7553), 436–444 (2015)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Miao, S., et al.: A CNN regression approach for real-time 2D/3D registration. IEEE Trans. Med. Imaging 35(5), 1352–1363 (2016)CrossRefGoogle Scholar
  14. 14.
    Rousseau, F., et al.: Registration-based approach for reconstruction of high-resolution in utero fetal MR brain images. Acad Radiol. 13(9), 1072–1081 (2006)CrossRefGoogle Scholar
  15. 15.
    Wu, J.: ITK-based implementation of two-projection 2D/3D registration method with an application in patient setup for external beam radiotherapy. Insight J. 784 (2010). http://www.insight-journal.org/home/index
  16. 16.
    Xu, C., et al.: Multi-loss regularized deep neural network. IEEE Trans. Circ. Sys. Video Technol. 26(12), 2273–2283 (2016)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Benjamin Hou
    • 1
    Email author
  • Amir Alansary
    • 1
  • Steven McDonagh
    • 1
  • Alice Davidson
    • 2
  • Mary Rutherford
    • 2
  • Jo V. Hajnal
    • 2
  • Daniel Rueckert
    • 1
  • Ben Glocker
    • 1
  • Bernhard Kainz
    • 1
    • 2
  1. 1.Biomedical Image Analysis GroupImperial College LondonLondonUK
  2. 2.Division of Imaging Sciences and Biomedical EngineeringKings College LondonLondonUK

Personalised recommendations