Recognizing End-Diastole and End-Systole Frames via Deep Temporal Regression Network

  • Bin Kong
  • Yiqiang Zhan
  • Min Shin
  • Thomas Denny
  • Shaoting ZhangEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9902)


Accurate measurement of left ventricular volumes and Ejection Fraction from cine MRI is of paramount importance to the evaluation of cardiovascular functions, yet it usually requires laborious and tedious work of trained experts to interpret them. To facilitate this procedure, numerous computer aided diagnosis (CAD) methods and tools have been proposed, most of which focus on the left or right ventricle segmentation. However, the identification of ES and ED frames from cardiac sequences is largely ignored, which is a key procedure in the automated workflow. This seemingly easy task is quite challenging, due to the requirement of high accuracy (i.e., precisely identifying specific frames from a sequence) and subtle differences among consecutive frames. Recently, with the rapid growth of annotated data and the increasing computational power, deep learning methods have been widely exploited in medical image analysis. In this paper, we propose a novel deep learning architecture, named as temporal regression network (TempReg-Net), to accurately identify specific frames from MRI sequences, by integrating the Convolutional Neural Network (CNN) with the Recurrent Neural Network (RNN). Specifically, a CNN encodes the spatial information of a cardiac sequence, and a RNN decodes the temporal information. In addition, we design a new loss function in our network to constrain the structure of predicted labels, which further improves the performance. Our approach is extensively validated on thousands of cardiac sequences and the average difference is merely 0.4 frames, comparing favorably with previous systems.


Recurrent Neural Network Convolutional Neural Network Left Ventricle Volume Ground Truth Label Convolutional Neural Network Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Abboud, A.A., et al.: Automatic detection of the end-diastolic and end-systolic from 4d echocardiographic images. JCS 11(1), 230 (2015)Google Scholar
  2. 2.
    Chen, H., Dou, Q., Ni, D., Cheng, J.-Z., Qin, J., Li, S., Heng, P.-A.: Automatic fetal ultrasound standard plane detection using knowledge transferred recurrent neural networks. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9349, pp. 507–514. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-24553-9_62 CrossRefGoogle Scholar
  3. 3.
    Darvishi, S., et al.: Measuring left ventricular volumes in two-dimensional echocardiography image sequence using level-set method for automatic detection of end-diastole and end-systole frames. Res. Cardiocvasc. Med. 2(1), 39 (2013)Google Scholar
  4. 4.
    Dominguez, C.R., Kachenoura, N., Mulé, S., Tenenhaus, A., Delouche, A., Nardi, O., Gérard, O., Diebold, B., Herment, A., Frouin, F.: Classification of segmental wall motion in echocardiography using quantified parametric images. In: Frangi, A.F., Radeva, P., Santos, A., Hernandez, M. (eds.) FIMH 2005. LNCS, vol. 3504, pp. 477–486. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  5. 5.
    Donahue, J., et al.: Long-term recurrent convolutional networks for visual recognition and description. In: CVPR, pp. 2625–2634 (2015)Google Scholar
  6. 6.
    Gifani, P., et al.: Automatic detection of end-diastole and end-systole from echocardiography images using manifold learning. PMEA 31(9), 1091 (2010)Google Scholar
  7. 7.
    Greenspan, H., et al.: Guest editorial deep learning in medical imaging: overview and future promise of an exciting new technique. IEEE TMI 35(5), 1153–1159 (2016)Google Scholar
  8. 8.
    Jia, Y., et al.: Caffe: convolutional architecture for fast feature embedding. In: ACMMM, pp. 675–678. ACM (2014)Google Scholar
  9. 9.
    Marino, M., et al.: Fully automated assessment of left ventricular volumes, function and mass from cardiac MRI. In: CinC, pp. 109–112. IEEE (2014)Google Scholar
  10. 10.
    Russakovsky, O., et al.: Imagenet large scale visual recognition challenge. IJCV 115(3), 211–252 (2015)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Shalbaf, A., et al.: Automatic detection of end systole and end diastole within a sequence of 2-d echocardiographic images using modified isomap algorithm. In: MECBME, pp. 217–220. IEEE (2011)Google Scholar
  12. 12.
    Shin, H.C., et al.: Interleaved text/image deep mining on a very large-scale radiology database. In: CVPR, pp. 1090–1099 (2015)Google Scholar
  13. 13.
    Uzunbaş, M.G., et al.: Segmentation of myocardium using deformable regions and graph cuts. In: ISBI, pp. 254–257. IEEE (2012)Google Scholar
  14. 14.
    Yosinski, J., et al.: How transferable are features in deep neural networks? In: NIPS, pp. 3320–3328 (2014)Google Scholar
  15. 15.
    Zaremba, W., et al.: Learning to execute. CoRR abs/1410.4615 (2014)Google Scholar
  16. 16.
    Zhang, W., et al.: Deep convolutional neural networks for multi-modality isointense infant brain image segmentation. NeuroImage 108, 214–224 (2015)CrossRefGoogle Scholar
  17. 17.
    Zhen, X., et al.: Multi-scale deep networks and regression forests for direct bi-ventricular volume estimation. MedIA 30, 120–129 (2015)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Bin Kong
    • 1
  • Yiqiang Zhan
    • 2
  • Min Shin
    • 1
  • Thomas Denny
    • 3
  • Shaoting Zhang
    • 1
    Email author
  1. 1.Department of Computer ScienceUNC CharlotteCharlotteUSA
  2. 2.Siemens HealthcareMalvernUSA
  3. 3.MRI Research CenterAuburn UniversityAuburnUSA

Personalised recommendations