Multimodal Image Fusion for Cardiac Resynchronization Therapy Planning

  • Sophie Bruge
  • Antoine SimonEmail author
  • Nicolas Courtial
  • Julian Betancur
  • Alfredo Hernandez
  • François Tavard
  • Erwan Donal
  • Mathieu Lederlin
  • Christophe Leclercq
  • Mireille Garreau


Cardiac resynchronization therapy (CRT) has shown its efficiency to treat patients with left-sided heart failure, however with 30% of them not responding to the therapy. One way to optimize CRT is to pre-operatively plan the implantation of the CRT device and especially the positioning of the stimulation lead pacing the left ventricle (LV), which is implanted through the coronary veins. Indeed, it has been shown that this lead should target LV sites with a late mechanical activation and without fibrosis. Additional imaging modalities should therefore be part of CRT’s planning, in order to describe the anatomy, mechanical activation, and tissue characteristics of the LV. We developed a full workflow to process, register, and fuse CT images, ultrasound (US) images, and MRI, including cine-MRI and late gadolinium enhancement (LGE) MRI. It results in a 3D patient-specific model, describing the anatomy of the LV and of the coronary veins, the electro-mechanical delays, and the presence of fibrosis. The process includes a semi-automatic segmentation of CT images to extract the LV cavity and the veins. 2D US images are processed using speckle tracking echography (STE) to estimate the mechanical strains. LGE-MRI is segmented to extract macroscopic fibrosis. All these images are registered using CT as the anatomical reference. Registration methods have thus been developed to register STE to CT, LGE to cine-MRI, and cine-MRI to CT. This whole process furnishes to the physician, before the CRT implantation, a patient-specific 3D model representing all the information needed to select the most appropriate LV pacing sites. Results obtained on patients undergoing CRT are presented.


Cardiac imaging Multimodal imaging Registration Fusion Cardiac Resynchronization Therapy Planning 



This work was supported by the French National Research Agency (ANR) in the framework of the Investissement d’Avenir Program through Labex CAMI (ANR-11-LABX-0004). It was conducted in part in the experimental platform TherA-Image (Rennes, France), supported by Europe FEDER.


  1. 1.
    A.P. Ambrosy, G.C. Fonarow, J. Butler, O. Chioncel, S.J. Greene, M. Vaduganathan, S. Nodari, C.S. Lam, N. Sato, A.N. Shah, M. Gheorghiade, The global health and economic burden of hospitalizations for heart failure: lessons learned from hospitalized heart failure registries. J. Am. Coll. Cardiol. 63(12), 1123–1133 (2014)CrossRefGoogle Scholar
  2. 2.
    Z. Bakos, H. Markstad, E. Ostenfeld, M. Carlsson, A. Roijer, R. Borgquist, Combined preoperative information using a bullseye plot from speckle tracking echocardiography, cardiac CT scan, and MRI scan: targeted left ventricular lead implantation in patients receiving cardiac resynchronization therapy. Eur. Heart J. Cardiovasc. Imaging 15(5), 523–531 (2014)CrossRefGoogle Scholar
  3. 3.
    N. Baron, N. Kachenoura, P. Cluzel, F. Frouin, A. Herment, P. Grenier, G. Montalescot, F. Beygui, Comparison of various methods for quantitative evaluation of myocardial infarct volume from magnetic resonance delayed enhancement data. Int. J. Cardiol. 167(3), 739–744 (2013)CrossRefGoogle Scholar
  4. 4.
    J.M. Behar, S. Claridge, T. Jackson, B. Sieniewicz, B. Porter, J. Webb, R. Rajani, S. Kapetanakis, G. Carr-White, C.A. Rinaldi, The role of multi modality imaging in selecting patients and guiding lead placement for the delivery of cardiac resynchronization therapy. Expert. Rev. Cardiovasc. Ther. 15(2), 93–107 (2017)CrossRefGoogle Scholar
  5. 5.
    M. Bertini, D. Mele, M. Malagù, A. Fiorencis, T. Toselli, F. Casadei, T. Cannizzaro, C. Fragale, A. Fucili, E. Campagnolo et al., Cardiac resynchronization therapy guided by multimodality cardiac imaging. Eur. J. Heart Fail. 18(11), 1375–1382 (2016)CrossRefGoogle Scholar
  6. 6.
    J. Betancur, A. Simon, F. Tavard, B. Langella, C. Leclercq, M. Garreau, Segmentation-free MRI to CT 3d registration for cardiac resynchronization therapy optimization, in Computing in Cardiology Conference (IEEE, Piscataway, 2012), pp. 701–704Google Scholar
  7. 7.
    J. Betancur, A. Simon, E. Halbert, F. Tavard, F. Carré, A. Hernández, E. Donal, F. Schnell, M. Garreau, Registration of dynamic multiview 2D ultrasound and late gadolinium enhanced images of the heart: application to hypertrophic cardiomyopathy characterization. Med. Image Anal. 28, 13–21 (2016)CrossRefGoogle Scholar
  8. 8.
    J. Betancur, A. Simon, B. Langella, C. Leclercq, A. Hernández, M. Garreau, Synchronization and registration of cine magnetic resonance and dynamic computed tomography images of the heart. IEEE J. Biomed. Health Informatics 20(5), 1369–1376 (2016)CrossRefGoogle Scholar
  9. 9.
    S. Bruge, A. Simon, M. Lederlin, J. Betancur, A. Hernandez, E. Donal, C. Leclercq, M. Garreau, Multi-modal data fusion for cardiac resynchronization therapy planning and assistance, in 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (IEEE, Piscataway, 2015), pp. 2391–2394Google Scholar
  10. 10.
    P. Carità, E. Corrado, G. Pontone, A. Curnis, L. Bontempi, G. Novo, M. Guglielmo, G. Ciaramitaro, P. Assennato, S. Novo et al., Non-responders to cardiac resynchronization therapy: insights from multimodality imaging and electrocardiography. A brief review. Int. J. Cardiol. 225, 402–407 (2016)CrossRefGoogle Scholar
  11. 11.
    M.D. Cerqueira, N.J. Weissman, V. Dilsizian, A.K. Jacobs, S. Kaul, W.K. Laskey, D.J. Pennell, J.A. Rumberger, T. Ryan, M.S. Verani et al., Standardized myocardial segmentation and nomenclature for tomographic imaging of the heart. Circulation 105(4), 539–542 (2002)CrossRefGoogle Scholar
  12. 12.
    A.F. Frangi, W.J. Niessen, K.L. Vincken, M.A. Viergever, Multiscale vessel enhancement filtering, in Medical Image Computing and Computer-Assisted Intervention (MICCAI) (Springer, Berlin, 1998), pp. 130–137CrossRefGoogle Scholar
  13. 13.
    O. Goitein, J.M. Lacomis, J. Gorcsan, D. Schwartzman, Left ventricular pacing lead implantation: potential utility of multimodal image integration. Heart Rhythm 3(1), 91–94 (2006)CrossRefGoogle Scholar
  14. 14.
    B. Heydari, M. Jerosch-Herold, R.Y. Kwong, Imaging for planning of cardiac resynchronization therapy. JACC: Cardiovasc. Imaging 5(1), 93–110 (2012)Google Scholar
  15. 15.
    J. Hong, K. Konishi, H. Nakashima, S. Ieiri, K. Tanoue, M. Nakamuta, M. Hashizume, Integration of MRI and ultrasound in surgical navigation for robotic surgery, in World Congress on Medical Physics and Biomedical Engineering, pp. 3052–3055 (2007)Google Scholar
  16. 16.
    X. Huang, N. Hill, J. Ren, G. Guiraudon, T. Peters, Intra-cardiac 2D US to 3D CT image registration, in SPIE Medical Imaging, vol. 6509, pp. 65092E–1 (2007)Google Scholar
  17. 17.
    X. Huang, J. Moore, G. Guiraudon, D. Jones, D. Bainbridge, J. Ren, T. Peters, Dynamic 2D ultrasound and 3D CT image registration of the beating heart. IEEE Trans. Med. Imaging 28(8), 1179–1189 (2009)CrossRefGoogle Scholar
  18. 18.
    N. Kachenoura, A. Redheuil, A. Herment, E. Mousseaux, F. Frouin, Robust assessment of the transmural extent of myocardial infarction in late gadolinium-enhanced MRI studies using appropriate angular and circumferential subdivision of the myocardium. Eur. Radiol. 18(10), 2140–2147 (2008)CrossRefGoogle Scholar
  19. 19.
    C. Leclercq, S. Cazeau, P. Ritter, C. Alonso, D. Gras, P. Mabo, A. Lazarus, J. Daubert, A pilot experience with permanent biventricular pacing to treat advanced heart failure. Am. Heart J. 140(6), 862–870 (2000)CrossRefGoogle Scholar
  20. 20.
    M. Ledesma-Carbayo, J. Kybic, M. Desco, A. Santos, M. Suhling, P. Hunziker, M. Unser, Spatio-temporal nonrigid registration for ultrasound cardiac motion estimation. IEEE Trans. Med. Imaging 24(9), 1113–1126 (2005)CrossRefGoogle Scholar
  21. 21.
    D. Lesage, E.D. Angelini, I. Bloch, G. Funka-Lea, A review of 3D vessel lumen segmentation techniques: models, features and extraction schemes. Med. Image Anal. 13(6), 819–845 (2009)CrossRefGoogle Scholar
  22. 22.
    F. Li, P. Lang, M. Rajchl, E. Chen, G. Guiraudon, T. Peters, Towards real-time 3D US-CT registration on the beating heart for guidance of minimally invasive cardiac interventions, in SPIE Medical Imaging, pp. 831615–831615 (2012)Google Scholar
  23. 23.
    C. Linde, W.T. Abraham, M.R. Gold, M.S.J. Sutton, S. Ghio, C. Daubert, Randomized trial of cardiac resynchronization in mildly symptomatic heart failure patients and in asymptomatic patients with left ventricular dysfunction and previous heart failure symptoms. J. Am. Coll. Cardiol. 52(23), 1834–1843 (2008)CrossRefGoogle Scholar
  24. 24.
    T. Mäkelä, P. Clarysse, O. Sipila, N. Pauna, Q. Pham, T. Katila, I. Magnin, A review of cardiac image registration methods. IEEE Trans. Med. Imaging 21(9), 1011–1021 (2002)CrossRefGoogle Scholar
  25. 25.
    N. Mewton, C.Y. Liu, P. Croisille, D. Bluemke, J.A. Lima, Assessment of myocardial fibrosis with cardiovascular magnetic resonance. J. Am. Coll. Cardiol. 57(8), 891–903 (2011)CrossRefGoogle Scholar
  26. 26.
    D. Perperidis, R.H. Mohiaddin, D. Rueckert, Spatio-temporal free-form registration of cardiac MR image sequences. Med. Image Anal. 9(5), 441–456 (2005)CrossRefGoogle Scholar
  27. 27.
    E. Puyol-Antón, M. Sinclair, B. Gerber, M.S. Amzulescu, H. Langet, M. De Craene, P. Aljabar, P. Piro, A.P. King, A multimodal spatiotemporal cardiac motion atlas from MR and ultrasound data. Med. Image Anal. 40, 96–110 (2017)CrossRefGoogle Scholar
  28. 28.
    S. Saba, J. Marek, D. Schwartzman, S. Jain, E. Adelstein, P. White, O.A. Oyenuga, T. Onishi, P. Soman, J. Gorcsan, Echocardiography-guided left ventricular lead placement for cardiac resynchronization therapy results of the speckle tracking assisted resynchronization therapy for electrode region trial. Circ. Heart Fail. 6(3), 427–434 (2013)CrossRefGoogle Scholar
  29. 29.
    H. Sakoe, S. Chiba, Dynamic programming algorithm optimization for spoken word recognition. IEEE Trans. Acoust. Speech Signal Process. 26(1), 43–49 (1978)CrossRefGoogle Scholar
  30. 30.
    A. Savi, M. Gilardi, G. Rizzo, M. Pepi, C. Landoni, C. Rossetti, G. Lucignani, A. Bartorelli, F. Fazio, Spatial registration of echocardiographic and positron emission tomographic heart studies. Eur. J. Nucl. Med. Mol. Imaging 22(3), 243–247 (1995)CrossRefGoogle Scholar
  31. 31.
    A. Sommer, M.B. Kronborg, B.L. Nørgaard, S.H. Poulsen, K. Bouchelouche, M. Böttcher, H.K. Jensen, J.M. Jensen, J. Kristensen, C. Gerdes et al., Multimodality imaging-guided left ventricular lead placement in cardiac resynchronization therapy: a randomized controlled trial. Eur. J. Heart Fail. 18(11), 1365–1374 (2016)CrossRefGoogle Scholar
  32. 32.
    F. Tavard, A. Simon, C. Leclercq, E. Donal, A.I. Hernández, M. Garreau, Multimodal registration and data fusion for cardiac resynchronization therapy optimization. IEEE Trans. Med. Imaging 33(6), 1363–1372 (2014)CrossRefGoogle Scholar
  33. 33.
    F. Tournoux, R.C. Chan, R. Manzke, M.D. Hanschumacher, A.A. Chen-Tournoux, O. Gérard, J. Solis-Martin, E.K. Heist, P. Allain, V. Reddy et al., Integrating functional and anatomical information to guide cardiac resynchronization therapy. Eur. J. Heart Fail. 12(1), 52–57 (2010)CrossRefGoogle Scholar
  34. 34.
    W. Wein, A. Khamene, D. Clevert, O. Kutter, N. Navab, Simulation and fully automatic multimodal registration of medical ultrasound, in Medical Image Computing and Computer-Assisted Intervention (MICCAI) (Springer, Berlin, 2007), pp. 136–143Google Scholar
  35. 35.
    J.A. Wong, R. Yee, J. Stirrat, D. Scholl, A.D. Krahn, L.J. Gula, A.C. Skanes, P. Leong-Sit, G.J. Klein, D. McCarty et al., Influence of pacing site characteristics on response to cardiac resynchronization therapy. Circ. Cardiovasc. Imaging 6(4), 542–550 (2013)CrossRefGoogle Scholar
  36. 36.
    Q. Zhang, R. Eagleson, T. Peters, Real-time visualization of 4D cardiac MR images using graphics processing units, in IEEE International Symposium on Biomedical Imaging: Nano to Macro, pp. 343–346 (2006)Google Scholar
  37. 37.
    Q. Zhang, Y. Zhou, C.-M. Yu, Incidence, definition, diagnosis, and management of the cardiac resynchronization therapy nonresponder. Curr. Opin. Cardiol. 30(1), 40–49 (2015)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Sophie Bruge
    • 1
    • 2
  • Antoine Simon
    • 1
    • 2
    Email author
  • Nicolas Courtial
    • 1
    • 2
  • Julian Betancur
    • 1
    • 2
  • Alfredo Hernandez
    • 1
    • 2
  • François Tavard
    • 1
    • 2
  • Erwan Donal
    • 1
    • 2
  • Mathieu Lederlin
    • 1
    • 2
  • Christophe Leclercq
    • 1
    • 2
  • Mireille Garreau
    • 1
    • 2
  1. 1.Univ Rennes, CHU Rennes, Inserm, LTSI – UMR 1099RennesFrance
  2. 2.Université de Rennes 1, LTSIRennesFrance

Personalised recommendations