Skip to main content

In-Silico Deep Reinforcement Learning for Effective Cardiac Ablation Strategy

Abstract

Purpose

We propose an in-silico deep reinforcement learning scheme that combines a computer simulation model of cardiac tissue and deep reinforcement learning to unveil an effective ablation strategy for atrial fibrillation (AF).

Methods

The deep neural network-based ablation model (DAM) was designed to input a membrane potential movie and output an ablation pattern. As a virtual environment for in-silico learning, a numerical 2D cardiac tissue model was used. After training, the trained DAM was compared with two different strategies: the random ablation strategy (RND) and the rotor ablation strategy (ROT).

Results

We demonstrated the ablation area percentage of RND, ROT, and the best learned DAM was 7.0 ± 2.8%, 12.5 ± 5.7%, 6.5 ± 2.4%, and the AF termination rate was 12.6%, 8.5%, 74.1%, respectively. Results suggest that the DAM learned to effectively terminate spiral excitations by ablating the area from near the spiral center towards the tissue boundary without any prior knowledge. The learned DAM achieved a better AF termination rate compared to the other ablation strategies.

Conclusion

This study showed feasibility of in-silico learning for an effective ablation strategy. To the best of our knowledge, this is the first approach to optimize spatial ablation patterns by deep learning. The proposed learning method has possibility to contribute to establishing an effective ablation strategy for more complex excitations for which conventional heuristic ablation is ineffective.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

References

  1. 1.

    Andrade, J., Khairy, P., Dobrev, D., & Nattel, S. (2014). The clinical profile and pathophysiology of atrial fibrillation: relationships among clinical features, epidemiology, and mechanisms. Circulation research, 114(9), 1453–1468.

    CAS  Article  Google Scholar 

  2. 2.

    Hart, R. G., & Halperin, J. L. (2001). Atrial fibrillation and stroke: concepts and controversies. Stroke, 32(3), 803–808.

    CAS  Article  Google Scholar 

  3. 3.

    Thrall, G., Lane, D., Carroll, D., & Lip, G. Y. (2006). Quality of life in patients with atrial fibrillation: a systematic review. The American journal of medicine, 119(5), 448-e1.

    Article  Google Scholar 

  4. 4.

    Kalantarian, S., Stern, TA., Mansour, M., & Ruskin, JN. (2013) Cognitive impairment associated with atrial fibrillation: a meta-analysis. Annals of internal medicine 158: 338–346

  5. 5.

    Wolowacz, S. E., Samuel, M., Brennan, V. K., Jasso-Mosqueda, J. G., & Van Gelder, I. C. (2011). The cost of illness of atrial fibrillation: a systematic review of the recent literature. Europace, 13(10), 1375–1385.

    CAS  Article  Google Scholar 

  6. 6.

    Haissaguerre, M., Jaïs, P., Shah, D. C., Takahashi, A., Hocini, M., Quiniou, G., & Clémenty, J. (1998). Spontaneous initiation of atrial fibrillation by ectopic beats originating in the pulmonary veins. New England Journal of Medicine, 339(10), 659–666.

    CAS  Article  Google Scholar 

  7. 7.

    Calkins, H., Hindricks, G., Cappato, R., Kim, Y. H., Saad, E. B., Aguinaga, L., & Yamane, T. (2017). 2017 HRS/EHRA/ECAS/APHRS/SOLAECE expert consensus statement on catheter and surgical ablation of atrial fibrillation: executive summary. Heart Rhythm, 14(10), e445–e494.

    Article  Google Scholar 

  8. 8.

    Nademanee, K., McKenzie, J., Kosar, E., Schwab, M., Sunsaneewitayakul, B., Vasavakul, T., & Ngarmukos, T. (2004). A new approach for catheter ablation of atrial fibrillation: mapping of the electrophysiologic substrate. Journal of the American College of Cardiology, 43(11), 2044–2053.

    Article  Google Scholar 

  9. 9.

    Rolf, S., Kircher, S., Arya, A., Eitel, C., Sommer, P., Richter, S., Piorkowski, C. (2014) Tailored atrial substrate modification based on low-voltage areas in catheter ablation of atrial fibrillation. Circulation: Arrhythmia and Electrophysiology 7(5): 825–833

  10. 10.

    Sakata, K., Okuyama, Y., Ozawa, T., Haraguchi, R., Nakazawa, K., Tsuchiya, T., & Ashihara, T. (2018). Not all rotors, effective ablation targets for nonparoxysmal atrial fibrillation, are included in areas suggested by conventional indirect indicators of atrial fibrillation drivers: Ex TR a Mapping project. Journal of arrhythmia, 34(2), 176–184.

    Article  Google Scholar 

  11. 11.

    Elayi, C. S., Verma, A., Di Biase, L., Ching, C. K., Patel, D., Barrett, C., & Natale, A. (2008). Ablation for longstanding permanent atrial fibrillation: results from a randomized study comparing three different strategies. Heart Rhythm, 5(12), 1658–1664.

    Article  Google Scholar 

  12. 12.

    Oral, H., Chugh, A., Yoshida, K., Sarrazin, J. F., Kuhne, M., Crawford, T., & Morady, F. (2009). A randomized assessment of the incremental role of ablation of complex fractionated atrial electrograms after antral pulmonary vein isolation for long-lasting persistent atrial fibrillation. Journal of the American College of Cardiology, 53(9), 782–789.

    Article  Google Scholar 

  13. 13.

    Brooks, A. G., Stiles, M. K., Laborderie, J., Lau, D. H., Kuklik, P., Shipp, N. J., & Sanders, P. (2010). Outcomes of long-standing persistent atrial fibrillation ablation: a systematic review. Heart Rhythm, 7(6), 835–846.

    Article  Google Scholar 

  14. 14.

    Baykaner, T., Rogers, A. J., Meckler, G. L., Zaman, J., Navara, R., Rodrigo, M., ... & Heidenreich, P. A. (2018) Clinical implications of ablation of drivers for atrial fibrillation: a systematic review and meta-analysis. Circulation: Arrhythmia and Electrophysiology 11(5): e006119

  15. 15.

    Ruchat, P., Virag, N., Dang, L., Schlaepfer, J., Pruvot, E., & Kappenberger, L. (2007) A biophysical model of atrial fibrillation ablation: what can a surgeon learn from a computer model?. Europace 9(suppl_6): vi71-vi76

  16. 16.

    Hwang, M., Kwon, S. S., Wi, J., Park, M., Lee, H. S., Park, J. S., & Pak, H. N. (2014). Virtual ablation for atrial fibrillation in personalized in-silico three-dimensional left atrial modeling: comparison with clinical catheter ablation. Progress in biophysics and molecular biology, 116(1), 40–47.

    Article  Google Scholar 

  17. 17.

    Bayer, J. D., Roney, C. H., Pashaei, A., Jaïs, P., & Vigmond, E. J. (2016). Novel radiofrequency ablation strategies for terminating atrial fibrillation in the left atrium: a simulation study. Frontiers in physiology, 7, 108.

    Article  Google Scholar 

  18. 18.

    Zahid, S., Whyte, K. N., Schwarz, E. L., Blake, R. C., III., Boyle, P. M., Chrispin, J., & Trayanova, N. A. (2016). Feasibility of using patient-specific models and the “minimum cut” algorithm to predict optimal ablation targets for left atrial flutter. Heart Rhythm, 13(8), 1687–1698.

    Article  Google Scholar 

  19. 19.

    Hakim, J. B., Murphy, M. J., Trayanova, N. A., & Boyle, P. M. (2018). Arrhythmia dynamics in computational models of the atria following virtual ablation of re-entrant drivers. EP Europace20(suppl_3), iii45-iii54.

  20. 20.

    Boyle, P. M., Zghaib, T., Zahid, S., Ali, R. L., Deng, D., Franceschi, W. H., & Trayanova, N. A. (2019). Computationally guided personalized targeted ablation of persistent atrial fibrillation. Nature biomedical engineering, 3(11), 870–879.

    Article  Google Scholar 

  21. 21.

    Muffoletto, M., Fu, X., Roy, A., Varela, M., Bates, P. A., & Aslanidi, O. V. (2019, July). Development of a deep learning method to predict optimal ablation patterns for atrial fibrillation. In 2019 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB) (pp. 1–4). IEEE.

  22. 22.

    Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., ... & Hassabis, D. (2015). Human-level control through deep reinforcement learning. nature518(7540), 529–533.

  23. 23.

    Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., Van Den Driessche, G., ... & Hassabis, D. (2016). Mastering the game of Go with deep neural networks and tree search. nature529(7587), 484–489.

  24. 24.

    James, S., Davison, A. J., & Johns, E. (2017, October). Transferring end-to-end visuomotor control from simulation to real world for a multi-stage task. In Conference on Robot Learning (pp. 334–343). PMLR.

  25. 25.

    Quillen, D., Jang, E., Nachum, O., Finn, C., Ibarz, J., & Levine, S. (2018, May). Deep reinforcement learning for vision-based robotic grasping: A simulated comparative evaluation of off-policy methods. In 2018 IEEE International Conference on Robotics and Automation (ICRA) (pp. 6284–6291). IEEE.

  26. 26.

    Courtemanche, M., Ramirez, R. J., & Nattel, S. (1998). Ionic mechanisms underlying human atrial action potential properties: insights from a mathematical model. American Journal of Physiology-Heart and Circulatory Physiology, 275(1), H301–H321.

    CAS  Article  Google Scholar 

  27. 27.

    Ronneberger, O., Fischer, P., & Brox, T. (2015, October). U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention (pp. 234–241). Springer, Cham.

  28. 28.

    Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., ... & Lerer, A. (2017). Automatic differentiation in pytorch.

  29. 29.

    Zaman, J. A., Peters, N. S., & Narayan, S. M. (2015). Rotor mapping and ablation to treat atrial fibrillation. Current opinion in cardiology, 30(1), 24.

    Article  Google Scholar 

  30. 30.

    Tomii, N., Yamazaki, M., Arafune, T., Honjo, H., Shibata, N., & Sakuma, I. (2015). Detection algorithm of phase singularity using phase variance analysis for epicardial optical mapping data. IEEE Transactions on Biomedical Engineering, 63(9), 1795–1803.

    Article  Google Scholar 

  31. 31.

    Ashihara, T., Namba, T., Ito, M., Ikeda, T., Nakazawa, K., & Trayanova, N. (2004). Spiral wave control by a localized stimulus: a bidomain model study. Journal of cardiovascular electrophysiology, 15(2), 226–233.

    Article  Google Scholar 

  32. 32.

    Tomii, N., Yamazaki, M., Arafune, T., Kamiya, K., Nakazawa, K., Honjo, H., & Sakuma, I. (2018). Interaction of phase singularities on the spiral wave tail: reconsideration of capturing the excitable gap. American Journal of Physiology-Heart and Circulatory Physiology, 315(2), H318–H326.

    Article  Google Scholar 

  33. 33.

    Davidenko, J. M., Salomonsz, R., Pertsov, A. M., Baxter, W. T., & Jalife, J. (1995). Effects of pacing on stationary reentrant activity: theoretical and experimental study. Circulation research, 77(6), 1166–1179.

    CAS  Article  Google Scholar 

  34. 34.

    Carrick, R. T., Benson, B. E., Bates, J. H., & Spector, P. S. (2016). Prospective, tissue-specific optimization of ablation for multiwavelet reentry: predicting the required amount, location, and configuration of lesions. Circulation: Arrhythmia and Electrophysiology9(3), e003555.

  35. 35.

    Alessandrini, M., Valinoti, M., Unger, L., Oesterlein, T., Dössel, O., Corsi, C., & Severi, S. (2018). A computational framework to benchmark basket catheter guided ablation in atrial fibrillation. Frontiers in physiology, 9, 1251.

    Article  Google Scholar 

  36. 36.

    Seno, H., Tomii, N., Yamazaki, M., Honjo, H., Shibata, N., & Sakuma, I. (2020). Cardiac Spiral Wave Termination by Linear Regional Cooling Toward the Anatomical Boundary of the Heart. Journal of Medical and Biological Engineering, 1–9.

  37. 37.

    Song, J. S., Kim, J., Lim, B., Lee, Y. S., Hwang, M., Joung, B., & Pak, H. N. (2018). Pro-Arrhythmogenic effects of heterogeneous tissue curvature-a suggestion for role of left atrial appendage in atrial fibrillation-. Circulation Journal, 83(1), 32–40.

    Article  Google Scholar 

  38. 38.

    Augustin, C. M., Fastl, T. E., Neic, A., Bellini, C., Whitaker, J., Rajani, R., & Niederer, S. A. (2020). The impact of wall thickness and curvature on wall stress in patient-specific electromechanical models of the left atrium. Biomechanics and modeling in mechanobiology, 19(3), 1015–1034.

    Article  Google Scholar 

  39. 39.

    Tanaka, K., Zlochiver, S., Vikstrom, K. L., Yamazaki, M., Moreno, J., Klos, M., & Kalifa, J. (2007). Spatial distribution of fibrosis governs fibrillation wave dynamics in the posterior left atrium during heart failure. Circulation research, 101(8), 839–847.

    CAS  Article  Google Scholar 

  40. 40.

    Nattel, S., Burstein, B., & Dobrev, D. (2008). Atrial remodeling and atrial fibrillation: mechanisms and implications. Circulation: Arrhythmia and Electrophysiology1(1), 62–73

  41. 41.

    Aronis, K. N., Ali, R., & Trayanova, N. A. (2019). The role of personalized atrial modeling in understanding atrial fibrillation mechanisms and improving treatment. International journal of cardiology, 287, 139–147.

    Article  Google Scholar 

Download references

Funding

This study was supported by Grants-in Aid for Scientific Research (21J13347) to H.S., (18H02802) M.Y., (18H04161, 21H04953) I.S., and (18K18357, 21K18036) N.T., from the Japanese Society for Promotion of Science in Japan.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Hiroshi Seno.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Seno, H., Yamazaki, M., Shibata, N. et al. In-Silico Deep Reinforcement Learning for Effective Cardiac Ablation Strategy. J. Med. Biol. Eng. (2021). https://doi.org/10.1007/s40846-021-00664-6

Download citation

Keywords

  • Arrhythmia
  • Atrial fibrillation
  • Cardiac ablation
  • Deep learning