Advertisement

Multi-input Cardiac Image Super-Resolution Using Convolutional Neural Networks

  • Ozan OktayEmail author
  • Wenjia Bai
  • Matthew Lee
  • Ricardo Guerrero
  • Konstantinos Kamnitsas
  • Jose Caballero
  • Antonio de Marvao
  • Stuart Cook
  • Declan O’Regan
  • Daniel Rueckert
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9902)

Abstract

3D cardiac MR imaging enables accurate analysis of cardiac morphology and physiology. However, due to the requirements for long acquisition and breath-hold, the clinical routine is still dominated by multi-slice 2D imaging, which hamper the visualization of anatomy and quantitative measurements as relatively thick slices are acquired. As a solution, we propose a novel image super-resolution (SR) approach that is based on a residual convolutional neural network (CNN) model. It reconstructs high resolution 3D volumes from 2D image stacks for more accurate image analysis. The proposed model allows the use of multiple input data acquired from different viewing planes for improved performance. Experimental results on 1233 cardiac short and long-axis MR image stacks show that the CNN model outperforms state-of-the-art SR methods in terms of image quality while being computationally efficient. Also, we show that image segmentation and motion tracking benefits more from SR-CNN when it is used as an initial upscaling method than conventional interpolation methods for the subsequent analysis.

Keywords

Convolutional Neural Network Image Quality Assessment Super Resolution Structural Similarity Index Measure Convolutional Neural Network Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Alexander, D.C., Zikic, D., Zhang, J., Zhang, H., Criminisi, A.: Image quality transfer via random forest regression: applications in diffusion MRI. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014, Part III. LNCS, vol. 8675, pp. 225–232. Springer, Heidelberg (2014)Google Scholar
  2. 2.
    Bai, W., Shi, W., O’Regan, D.P., Tong, T., Wang, H., Jamil-Copley, S., Peters, N.S., Rueckert, D.: A probabilistic patch-based label fusion model for multi-atlas segmentation with registration refinement: application to cardiac MR images. IEEE TMI 32(7), 1302–1315 (2013)Google Scholar
  3. 3.
    Bhatia, K.K., Price, A.N., Shi, W., Rueckert, D.: Super-resolution reconstruction of cardiac MRI using coupled dictionary learning. In: IEEE ISBI, pp. 947–950 (2014)Google Scholar
  4. 4.
    Bromley, J., Guyon, I., Lecun, Y., Sckinger, E., Shah, R.: Signature verification using a “Siamese” time delay neural network. In: NIPS, pp. 737–744 (1994)Google Scholar
  5. 5.
    Dong, C., Deng, Y., Change Loy, C., Tang, X.: Compression artifacts reduction by a deep convolutional network. In: IEEE CVPR, pp. 576–584 (2015)Google Scholar
  6. 6.
    Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE PAMI 38(2), 295–307 (2016)CrossRefGoogle Scholar
  7. 7.
    Greenspan, H.: Super-resolution in medical imaging. Comput. J. 52(1), 43–63 (2009)MathSciNetCrossRefGoogle Scholar
  8. 8.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  9. 9.
    Lötjönen, J., Pollari, M., Kivistö, S., Lauerma, K.: Correction of movement artifacts from 4-D cardiac short- and long-axis MR data. In: Barillot, C., Haynor, D.R., Hellier, P. (eds.) MICCAI 2004. LNCS, vol. 3217, pp. 405–412. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  10. 10.
    Manjón, J.V., Coupé, P., Buades, A., Fonov, V., Collins, D.L., Robles, M.: Non-local MRI upsampling. MedIA 14(6), 784–792 (2010)Google Scholar
  11. 11.
    Odille, F., Bustin, A., Chen, B., Vuissoz, P.-A., Felblinger, J.: Motion-corrected, super-resolution reconstruction for high-resolution 3D cardiac cine MRI. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 435–442. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-24574-4_52 CrossRefGoogle Scholar
  12. 12.
    Plenge, E., Poot, D.H.J., Niessen, W.J., Meijering, E.: Super-resolution reconstruction using cross-scale self-similarity in multi-slice MRI. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013, Part III. LNCS, vol. 8151, pp. 123–130. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  13. 13.
    Rueckert, D., Sonoda, L.I., Hayes, C., Hill, D.L., Leach, M.O., Hawkes, D.J.: Nonrigid registration using free-form deformations: application to breast MR images. IEEE TMI 18(8), 712–721 (1999)Google Scholar
  14. 14.
    Rueda, A., Malpica, N., Romero, E.: Single-image super-resolution of brain MR images using overcomplete dictionaries. MedIA 17(1), 113–132 (2013)Google Scholar
  15. 15.
    Shi, W., Caballero, J., Ledig, C., Zhuang, X., Bai, W., Bhatia, K., de Marvao, A.M.S.M., Dawes, T., O’Regan, D., Rueckert, D.: Cardiac image super-resolution with global correspondence using multi-atlas patchmatch. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013, Part III. LNCS, vol. 8151, pp. 9–16. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  16. 16.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  17. 17.
    Su, H., Maji, S., Kalogerakis, E., Learned-Miller, E.: Multi-view convolutional neural networks for 3D shape recognition. In: IEEE CVPR, pp. 945–953 (2015)Google Scholar
  18. 18.
    Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE TIP 13(4), 600–612 (2004)Google Scholar
  19. 19.
    Zhao, H., Gallo, O., Frosio, I., Kautz, J.: Is L2 a good loss function for neural networks for image processing? arXiv preprint arXiv:1511.08861 (2015)

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Ozan Oktay
    • 1
    Email author
  • Wenjia Bai
    • 1
  • Matthew Lee
    • 1
  • Ricardo Guerrero
    • 1
  • Konstantinos Kamnitsas
    • 1
  • Jose Caballero
    • 3
  • Antonio de Marvao
    • 2
  • Stuart Cook
    • 2
  • Declan O’Regan
    • 2
  • Daniel Rueckert
    • 1
  1. 1.Biomedical Image Analysis GroupImperial College LondonLondonUK
  2. 2.Institute of Clinical ScienceImperial College LondonLondonUK
  3. 3.Magic Pony TechnologyLondonUK

Personalised recommendations