Advertisement

Towards MR-Only Radiotherapy Treatment Planning: Synthetic CT Generation Using Multi-view Deep Convolutional Neural Networks

  • Yu Zhao
  • Shu Liao
  • Yimo Guo
  • Liang Zhao
  • Zhennan Yan
  • Sungmin Hong
  • Gerardo Hermosillo
  • Tianming Liu
  • Xiang Sean Zhou
  • Yiqiang Zhan
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11070)

Abstract

Recently, Magnetic Resonance imaging-only (MR-only) radiotherapy treatment planning (RTP) receives growing interests since it is radiation-free and time/cost efficient. A key step in MR-only RTP is the generation of a synthetic CT from MR for dose calculation. Although deep learning approaches have achieved promising results on this topic, they still face two major challenges. First, it is very difficult to get perfectly registered CT-MR pairs to learn the intensity mapping, especially for abdomen and pelvic scans. Slight registration errors may mislead the deep network to converge at a sub-optimal CT-MR intensity matching. Second, training of a standard 3D deep network is very memory-consuming. In practice, one has to either shrink the size of the training network (sacrificing the accuracy) or use a patch-based sliding-window scheme (sacrificing the speed). In this paper, we proposed a novel method to address these two challenges. First, we designed a max-pooled cost function to accommodate imperfect registered CT-MR training pairs. Second, we proposed a network that consists of multiple 2D sub-networks (from different 3D views) followed by a combination sub-network. It reduces the memory consumption without losing the 3D context for high quality CT synthesis. We demonstrated our method can generate high quality synthetic CTs with much higher runtime efficiency compared to the state-of-the-art as well as our own benchmark methods. The proposed solution can potentially enable more effective and efficient MR-only RTPs in clinical settings.

Keywords

Cross modality synthesis Deep learning Synthetic CT Radiotherapy 

References

  1. 1.
    Edmund, J.M., Nyholm, T.: A review of substitute CT generation for MRI-only radiation therapy. Radiat. Oncol. 12, 28 (2017)CrossRefGoogle Scholar
  2. 2.
    Edmund, J.M., et al.: SP-0510: dose planning based on MRI as the sole modality: why, how and when? Radiother. Oncol. 115, S248–S249 (2015)CrossRefGoogle Scholar
  3. 3.
    Paulson, E.S., Erickson, B., Schultz, C., Allen Li, X.: Comprehensive MRI simulation methodology using a dedicated MRI scanner in radiation oncology for external beam radiation treatment planning. Med. Phys. 42, 28–39 (2014)CrossRefGoogle Scholar
  4. 4.
    Sjölund, J., Forsberg, D., Andersson, M., Knutsson, H.: Generating patient specific pseudo-CT of the head from MR using atlas-based regression. Phys. Med. Biol. 60, 825–839 (2015)CrossRefGoogle Scholar
  5. 5.
    Delpon, G., et al.: Comparison of automated atlas-based segmentation software for postoperative prostate cancer radiotherapy. Front. Oncol. 6, 178 (2016)CrossRefGoogle Scholar
  6. 6.
    Dowling, J.A., et al.: An Atlas-based electron density mapping method for magnetic resonance imaging (MRI)-alone treatment planning and adaptive MRI-based prostate radiation therapy. Int. J. Radiat. Oncol. 83, e5–e11 (2012)CrossRefGoogle Scholar
  7. 7.
    Nie, D., et al.: Medical image synthesis with context-aware generative adversarial networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017, Part III. LNCS, vol. 10435, pp. 417–425. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-66179-7_48CrossRefGoogle Scholar
  8. 8.
    Wolterink, J.M., Dinkla, A.M., Savenije, M.H.F., Seevinck, P.R., van den Berg, C.A.T., Išgum, I.: Deep MR to CT synthesis using unpaired data. In: Tsaftaris, S.A., Gooya, A., Frangi, A.F., Prince, J.L. (eds.) SASHIMI 2017. LNCS, vol. 10557, pp. 14–23. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-68127-6_2CrossRefGoogle Scholar
  9. 9.
    Nie, D., Cao, X., Gao, Y., Wang, L., Shen, D.: Estimating CT image from MRI data using 3D fully convolutional networks, 1 January 2016Google Scholar
  10. 10.
    Andreasen, D., et al.: Computed tomography synthesis from magnetic resonance images in the pelvis using multiple random forests and auto-context features. In: Styner, M.A., Angelini, E.D. (eds.) SPIE Medical Imaging, p. 978417. International Society for Optics and Photonics (2016)Google Scholar
  11. 11.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015, Part III. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-24574-4_28CrossRefGoogle Scholar
  12. 12.
    Scherer, D., Müller, A., Behnke, S.: Evaluation of pooling operations in convolutional architectures for object recognition. In: Diamantaras, K., Duch, W., Iliadis, L.S. (eds.) ICANN 2010, Part III. LNCS, vol. 6354, pp. 92–101. Springer, Heidelberg (2010).  https://doi.org/10.1007/978-3-642-15825-4_10CrossRefGoogle Scholar
  13. 13.
    Liao, S., et al.: Automatic lumbar spondylolisthesis measurement in CT images. IEEE Trans. Med. Imaging 35, 1658–1669 (2016)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Yu Zhao
    • 1
  • Shu Liao
    • 2
  • Yimo Guo
    • 2
  • Liang Zhao
    • 2
  • Zhennan Yan
    • 2
  • Sungmin Hong
    • 3
  • Gerardo Hermosillo
    • 2
  • Tianming Liu
    • 1
  • Xiang Sean Zhou
    • 2
  • Yiqiang Zhan
    • 2
  1. 1.The University of GeorgiaAthensUSA
  2. 2.Siemens Medical SolutionsMalvernUSA
  3. 3.New York UniversityNew YorkUSA

Personalised recommendations