Full Quantification of Left Ventricle via Deep Multitask Learning Network Respecting Intra- and Inter-Task Relatedness

  • Wufeng Xue
  • Andrea Lum
  • Ashley Mercado
  • Mark Landis
  • James Warrington
  • Shuo LiEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10435)


Cardiac left ventricle (LV) quantification is among the most clinically important tasks for identification and diagnosis of cardiac diseases, yet still a challenge due to the high variability of cardiac structure and the complexity of temporal dynamics. Full quantification, i.e., to simultaneously quantify all LV indices including two areas (cavity and myocardium), six regional wall thicknesses (RWT), three LV dimensions, and one cardiac phase, is even more challenging since the uncertain relatedness intra and inter each type of indices may hinder the learning procedure from better convergence and generalization. In this paper, we propose a newly-designed multitask learning network (FullLVNet), which is constituted by a deep convolution neural network (CNN) for expressive feature embedding of cardiac structure; two followed parallel recurrent neural network (RNN) modules for temporal dynamic modeling; and four linear models for the final estimation. During the final estimation, both intra- and inter-task relatedness are modeled to enforce improvement of generalization: (1) respecting intra-task relatedness, group lasso is applied to each of the regression tasks for sparse and common feature selection and consistent prediction; (2) respecting inter-task relatedness, three phase-guided constraints are proposed to penalize violation of the temporal behavior of the obtained LV indices. Experiments on MR sequences of 145 subjects show that FullLVNet achieves high accurate prediction with our intra- and inter-task relatedness, leading to MAE of 190 mm\(^2\), 1.41 mm, 2.68 mm for average areas, RWT, dimensions and error rate of 10.4% for the phase classification. This endows our method a great potential in comprehensive clinical assessment of global, regional and dynamic cardiac function.


Left ventricle quantification Recurrent neural network Multi-task learning Task relatedness 


  1. 1.
    Afshin, M., Ayed, I.B., Islam, A., Goela, A., Peters, T.M., Li, S.: Global assessment of cardiac function using image statistics in MRI. In: Ayache, N., Delingette, H., Golland, P., Mori, K. (eds.) MICCAI 2012. LNCS, vol. 7511, pp. 535–543. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-33418-4_66 CrossRefGoogle Scholar
  2. 2.
    Afshin, M., Ben Ayed, I., Punithakumar, K., Law, M., Islam, A., Goela, A., Peters, T.M., Li, S.: Regional assessment of cardiac left ventricular myocardial function via MRI statistical features. IEEE TMI 33(2), 481–494 (2014)Google Scholar
  3. 3.
    Attili, A.K., Schuster, A., Nagel, E., Reiber, J.H., van der Geest, R.J.: Quantification in cardiac MRI: advances in image acquisition and processing. Int. J. Cardiovasc. Imaging 26(1), 27–40 (2010)CrossRefGoogle Scholar
  4. 4.
    Ayed, I.B., Chen, H.M., Punithakumar, K., Ross, I., Li, S.: Max-flow segmentation of the left ventricle by recovering subject-specific distributions via a bound of the Bhattacharyya measure. Med. Image Anal. 16(1), 87–100 (2012)CrossRefGoogle Scholar
  5. 5.
    Graves, A.: Supervised sequence labelling. Supervised Sequence Labelling with Recurrent Neural Networks. SCI, vol. 385, pp. 5–13. Springer, Heidelberg (2012)CrossRefzbMATHGoogle Scholar
  6. 6.
    Karamitsos, T.D., Francis, J.M., Myerson, S., Selvanayagam, J.B., Neubauer, S.: The role of cardiovascular magnetic resonance imaging in heart failure. J. Am. Coll. Cardiol. 54(15), 1407–1424 (2009)CrossRefGoogle Scholar
  7. 7.
    Kawel-Boehm, N., Maceira, A., Valsangiacomo-Buechel, E.R., Vogel-Claussen, J., Turkbey, E.B., Williams, R., Plein, S., Tee, M., Eng, J., Bluemke, D.A.: Normal values for cardiovascular magnetic resonance in adults and children. J. Cardiovasc. Magn. Reson. 17(1), 29 (2015)CrossRefGoogle Scholar
  8. 8.
    Kong, B., Zhan, Y., Shin, M., Denny, T., Zhang, S.: Recognizing end-diastole and end-systole frames via deep temporal regression network. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9902, pp. 264–272. Springer, Cham (2016). doi: 10.1007/978-3-319-46726-9_31 CrossRefGoogle Scholar
  9. 9.
    Peng, P., Lekadir, K., Gooya, A., Shao, L., Petersen, S.E., Frangi, A.F.: A review of heart chamber segmentation for structural and functional analysis using cardiac magnetic resonance imaging. Magn. Reson. Mater. Phys., Biol. Med. 29(2), 155–195 (2016)CrossRefGoogle Scholar
  10. 10.
    Petitjean, C., Dacher, J.N.: A review of segmentation methods in short axis cardiac MR images. Med. Image Anal. 15(2), 169–184 (2011)CrossRefGoogle Scholar
  11. 11.
    Poudel, R.P., Lamata, P., Montana, G.: Recurrent fully convolutional neural networks for multi-slice MRI cardiac segmentation arXiv:1608.03974 (2016)
  12. 12.
    Shin, H.C., Roth, H.R., Gao, M., Lu, L., Xu, Z., Nogues, I., Yao, J., Mollura, D., Summers, R.M.: Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE TMI 35(5), 1285–1298 (2016)Google Scholar
  13. 13.
    Suinesiaputra, A., Bluemke, D.A., Cowan, B.R., Friedrich, M.G., Kramer, C.M., Kwong, R., Plein, S., Schulz-Menger, J., Westenberg, J.J., Young, A.A., et al.: Quantification of LV function and mass by cardiovascular magnetic resonance: multi-center variability and consensus contours. J. Cardiovasc. Magn. Reson. 17(1), 63 (2015)CrossRefGoogle Scholar
  14. 14.
    Wang, Z., Ben Salah, M., Gu, B., Islam, A., Goela, A., Li, S.: Direct estimation of cardiac biventricular volumes with an adapted bayesian formulation. IEEE TBE 61(4), 1251–1260 (2014)Google Scholar
  15. 15.
    Zhang, Y., Yeung, D.Y.: A convex formulation for learning task relationships in multi-task learning. In: UAI, pp. 733–742 (2010)Google Scholar
  16. 16.
    Zhang, Z., Luo, P., Loy, C.C., Tang, X.: Facial landmark detection by deep multi-task learning. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8694, pp. 94–108. Springer, Cham (2014). doi: 10.1007/978-3-319-10599-4_7 CrossRefGoogle Scholar
  17. 17.
    Zhen, X., Islam, A., Bhaduri, M., Chan, I., Li, S.: Direct and simultaneous four-chamber volume estimation by multi-output regression. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9349, pp. 669–676. Springer, Cham (2015). doi: 10.1007/978-3-319-24553-9_82 CrossRefGoogle Scholar
  18. 18.
    Zhen, X., Wang, Z., Islam, A., Bhaduri, M., Chan, I., Li, S.: Direct estimation of cardiac bi-ventricular volumes with regression forests. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014. LNCS, vol. 8674, pp. 586–593. Springer, Cham (2014). doi: 10.1007/978-3-319-10470-6_73 CrossRefGoogle Scholar
  19. 19.
    Zhen, X., Wang, Z., Islam, A., Bhaduri, M., Chan, I., Li, S.: Multi-scale deep networks and regression forests for direct bi-ventricular volume estimation. Med. Image Anal. 30, 120–129 (2016)CrossRefGoogle Scholar
  20. 20.
    Zhen, X., Zhang, H., Islam, A., Bhaduri, M., Chan, I., Li, S.: Direct and simultaneous estimation of cardiac four chamber volumes by multioutput sparse regression. Med. Image Anal. 36, 184–196 (2017)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Wufeng Xue
    • 1
    • 2
  • Andrea Lum
    • 1
    • 2
  • Ashley Mercado
    • 1
    • 2
  • Mark Landis
    • 1
    • 2
  • James Warrington
    • 1
    • 2
  • Shuo Li
    • 1
    • 2
    Email author
  1. 1.Department of Medical ImagingWestern UniversityLondonCanada
  2. 2.Digital Imaging GroupLondonCanada

Personalised recommendations