Abstract
Purpose
We propose an in-silico deep reinforcement learning scheme that combines a computer simulation model of cardiac tissue and deep reinforcement learning to unveil an effective ablation strategy for atrial fibrillation (AF).
Methods
The deep neural network-based ablation model (DAM) was designed to input a membrane potential movie and output an ablation pattern. As a virtual environment for in-silico learning, a numerical 2D cardiac tissue model was used. After training, the trained DAM was compared with two different strategies: the random ablation strategy (RND) and the rotor ablation strategy (ROT).
Results
We demonstrated the ablation area percentage of RND, ROT, and the best learned DAM was 7.0 ± 2.8%, 12.5 ± 5.7%, 6.5 ± 2.4%, and the AF termination rate was 12.6%, 8.5%, 74.1%, respectively. Results suggest that the DAM learned to effectively terminate spiral excitations by ablating the area from near the spiral center towards the tissue boundary without any prior knowledge. The learned DAM achieved a better AF termination rate compared to the other ablation strategies.
Conclusion
This study showed feasibility of in-silico learning for an effective ablation strategy. To the best of our knowledge, this is the first approach to optimize spatial ablation patterns by deep learning. The proposed learning method has possibility to contribute to establishing an effective ablation strategy for more complex excitations for which conventional heuristic ablation is ineffective.
Similar content being viewed by others
References
Andrade, J., Khairy, P., Dobrev, D., & Nattel, S. (2014). The clinical profile and pathophysiology of atrial fibrillation: relationships among clinical features, epidemiology, and mechanisms. Circulation research, 114(9), 1453–1468.
Hart, R. G., & Halperin, J. L. (2001). Atrial fibrillation and stroke: concepts and controversies. Stroke, 32(3), 803–808.
Thrall, G., Lane, D., Carroll, D., & Lip, G. Y. (2006). Quality of life in patients with atrial fibrillation: a systematic review. The American journal of medicine, 119(5), 448-e1.
Kalantarian, S., Stern, TA., Mansour, M., & Ruskin, JN. (2013) Cognitive impairment associated with atrial fibrillation: a meta-analysis. Annals of internal medicine 158: 338–346
Wolowacz, S. E., Samuel, M., Brennan, V. K., Jasso-Mosqueda, J. G., & Van Gelder, I. C. (2011). The cost of illness of atrial fibrillation: a systematic review of the recent literature. Europace, 13(10), 1375–1385.
Haissaguerre, M., Jaïs, P., Shah, D. C., Takahashi, A., Hocini, M., Quiniou, G., & Clémenty, J. (1998). Spontaneous initiation of atrial fibrillation by ectopic beats originating in the pulmonary veins. New England Journal of Medicine, 339(10), 659–666.
Calkins, H., Hindricks, G., Cappato, R., Kim, Y. H., Saad, E. B., Aguinaga, L., & Yamane, T. (2017). 2017 HRS/EHRA/ECAS/APHRS/SOLAECE expert consensus statement on catheter and surgical ablation of atrial fibrillation: executive summary. Heart Rhythm, 14(10), e445–e494.
Nademanee, K., McKenzie, J., Kosar, E., Schwab, M., Sunsaneewitayakul, B., Vasavakul, T., & Ngarmukos, T. (2004). A new approach for catheter ablation of atrial fibrillation: mapping of the electrophysiologic substrate. Journal of the American College of Cardiology, 43(11), 2044–2053.
Rolf, S., Kircher, S., Arya, A., Eitel, C., Sommer, P., Richter, S., Piorkowski, C. (2014) Tailored atrial substrate modification based on low-voltage areas in catheter ablation of atrial fibrillation. Circulation: Arrhythmia and Electrophysiology 7(5): 825–833
Sakata, K., Okuyama, Y., Ozawa, T., Haraguchi, R., Nakazawa, K., Tsuchiya, T., & Ashihara, T. (2018). Not all rotors, effective ablation targets for nonparoxysmal atrial fibrillation, are included in areas suggested by conventional indirect indicators of atrial fibrillation drivers: Ex TR a Mapping project. Journal of arrhythmia, 34(2), 176–184.
Elayi, C. S., Verma, A., Di Biase, L., Ching, C. K., Patel, D., Barrett, C., & Natale, A. (2008). Ablation for longstanding permanent atrial fibrillation: results from a randomized study comparing three different strategies. Heart Rhythm, 5(12), 1658–1664.
Oral, H., Chugh, A., Yoshida, K., Sarrazin, J. F., Kuhne, M., Crawford, T., & Morady, F. (2009). A randomized assessment of the incremental role of ablation of complex fractionated atrial electrograms after antral pulmonary vein isolation for long-lasting persistent atrial fibrillation. Journal of the American College of Cardiology, 53(9), 782–789.
Brooks, A. G., Stiles, M. K., Laborderie, J., Lau, D. H., Kuklik, P., Shipp, N. J., & Sanders, P. (2010). Outcomes of long-standing persistent atrial fibrillation ablation: a systematic review. Heart Rhythm, 7(6), 835–846.
Baykaner, T., Rogers, A. J., Meckler, G. L., Zaman, J., Navara, R., Rodrigo, M., ... & Heidenreich, P. A. (2018) Clinical implications of ablation of drivers for atrial fibrillation: a systematic review and meta-analysis. Circulation: Arrhythmia and Electrophysiology 11(5): e006119
Ruchat, P., Virag, N., Dang, L., Schlaepfer, J., Pruvot, E., & Kappenberger, L. (2007) A biophysical model of atrial fibrillation ablation: what can a surgeon learn from a computer model?. Europace 9(suppl_6): vi71-vi76
Hwang, M., Kwon, S. S., Wi, J., Park, M., Lee, H. S., Park, J. S., & Pak, H. N. (2014). Virtual ablation for atrial fibrillation in personalized in-silico three-dimensional left atrial modeling: comparison with clinical catheter ablation. Progress in biophysics and molecular biology, 116(1), 40–47.
Bayer, J. D., Roney, C. H., Pashaei, A., Jaïs, P., & Vigmond, E. J. (2016). Novel radiofrequency ablation strategies for terminating atrial fibrillation in the left atrium: a simulation study. Frontiers in physiology, 7, 108.
Zahid, S., Whyte, K. N., Schwarz, E. L., Blake, R. C., III., Boyle, P. M., Chrispin, J., & Trayanova, N. A. (2016). Feasibility of using patient-specific models and the “minimum cut” algorithm to predict optimal ablation targets for left atrial flutter. Heart Rhythm, 13(8), 1687–1698.
Hakim, J. B., Murphy, M. J., Trayanova, N. A., & Boyle, P. M. (2018). Arrhythmia dynamics in computational models of the atria following virtual ablation of re-entrant drivers. EP Europace, 20(suppl_3), iii45-iii54.
Boyle, P. M., Zghaib, T., Zahid, S., Ali, R. L., Deng, D., Franceschi, W. H., & Trayanova, N. A. (2019). Computationally guided personalized targeted ablation of persistent atrial fibrillation. Nature biomedical engineering, 3(11), 870–879.
Muffoletto, M., Fu, X., Roy, A., Varela, M., Bates, P. A., & Aslanidi, O. V. (2019, July). Development of a deep learning method to predict optimal ablation patterns for atrial fibrillation. In 2019 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB) (pp. 1–4). IEEE.
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., ... & Hassabis, D. (2015). Human-level control through deep reinforcement learning. nature, 518(7540), 529–533.
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., Van Den Driessche, G., ... & Hassabis, D. (2016). Mastering the game of Go with deep neural networks and tree search. nature, 529(7587), 484–489.
James, S., Davison, A. J., & Johns, E. (2017, October). Transferring end-to-end visuomotor control from simulation to real world for a multi-stage task. In Conference on Robot Learning (pp. 334–343). PMLR.
Quillen, D., Jang, E., Nachum, O., Finn, C., Ibarz, J., & Levine, S. (2018, May). Deep reinforcement learning for vision-based robotic grasping: A simulated comparative evaluation of off-policy methods. In 2018 IEEE International Conference on Robotics and Automation (ICRA) (pp. 6284–6291). IEEE.
Courtemanche, M., Ramirez, R. J., & Nattel, S. (1998). Ionic mechanisms underlying human atrial action potential properties: insights from a mathematical model. American Journal of Physiology-Heart and Circulatory Physiology, 275(1), H301–H321.
Ronneberger, O., Fischer, P., & Brox, T. (2015, October). U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention (pp. 234–241). Springer, Cham.
Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., ... & Lerer, A. (2017). Automatic differentiation in pytorch.
Zaman, J. A., Peters, N. S., & Narayan, S. M. (2015). Rotor mapping and ablation to treat atrial fibrillation. Current opinion in cardiology, 30(1), 24.
Tomii, N., Yamazaki, M., Arafune, T., Honjo, H., Shibata, N., & Sakuma, I. (2015). Detection algorithm of phase singularity using phase variance analysis for epicardial optical mapping data. IEEE Transactions on Biomedical Engineering, 63(9), 1795–1803.
Ashihara, T., Namba, T., Ito, M., Ikeda, T., Nakazawa, K., & Trayanova, N. (2004). Spiral wave control by a localized stimulus: a bidomain model study. Journal of cardiovascular electrophysiology, 15(2), 226–233.
Tomii, N., Yamazaki, M., Arafune, T., Kamiya, K., Nakazawa, K., Honjo, H., & Sakuma, I. (2018). Interaction of phase singularities on the spiral wave tail: reconsideration of capturing the excitable gap. American Journal of Physiology-Heart and Circulatory Physiology, 315(2), H318–H326.
Davidenko, J. M., Salomonsz, R., Pertsov, A. M., Baxter, W. T., & Jalife, J. (1995). Effects of pacing on stationary reentrant activity: theoretical and experimental study. Circulation research, 77(6), 1166–1179.
Carrick, R. T., Benson, B. E., Bates, J. H., & Spector, P. S. (2016). Prospective, tissue-specific optimization of ablation for multiwavelet reentry: predicting the required amount, location, and configuration of lesions. Circulation: Arrhythmia and Electrophysiology, 9(3), e003555.
Alessandrini, M., Valinoti, M., Unger, L., Oesterlein, T., Dössel, O., Corsi, C., & Severi, S. (2018). A computational framework to benchmark basket catheter guided ablation in atrial fibrillation. Frontiers in physiology, 9, 1251.
Seno, H., Tomii, N., Yamazaki, M., Honjo, H., Shibata, N., & Sakuma, I. (2020). Cardiac Spiral Wave Termination by Linear Regional Cooling Toward the Anatomical Boundary of the Heart. Journal of Medical and Biological Engineering, 1–9.
Song, J. S., Kim, J., Lim, B., Lee, Y. S., Hwang, M., Joung, B., & Pak, H. N. (2018). Pro-Arrhythmogenic effects of heterogeneous tissue curvature-a suggestion for role of left atrial appendage in atrial fibrillation-. Circulation Journal, 83(1), 32–40.
Augustin, C. M., Fastl, T. E., Neic, A., Bellini, C., Whitaker, J., Rajani, R., & Niederer, S. A. (2020). The impact of wall thickness and curvature on wall stress in patient-specific electromechanical models of the left atrium. Biomechanics and modeling in mechanobiology, 19(3), 1015–1034.
Tanaka, K., Zlochiver, S., Vikstrom, K. L., Yamazaki, M., Moreno, J., Klos, M., & Kalifa, J. (2007). Spatial distribution of fibrosis governs fibrillation wave dynamics in the posterior left atrium during heart failure. Circulation research, 101(8), 839–847.
Nattel, S., Burstein, B., & Dobrev, D. (2008). Atrial remodeling and atrial fibrillation: mechanisms and implications. Circulation: Arrhythmia and Electrophysiology, 1(1), 62–73
Aronis, K. N., Ali, R., & Trayanova, N. A. (2019). The role of personalized atrial modeling in understanding atrial fibrillation mechanisms and improving treatment. International journal of cardiology, 287, 139–147.
Funding
This study was supported by Grants-in Aid for Scientific Research (21J13347) to H.S., (18H02802) M.Y., (18H04161, 21H04953) I.S., and (18K18357, 21K18036) N.T., from the Japanese Society for Promotion of Science in Japan.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Seno, H., Yamazaki, M., Shibata, N. et al. In-Silico Deep Reinforcement Learning for Effective Cardiac Ablation Strategy. J. Med. Biol. Eng. 41, 953–965 (2021). https://doi.org/10.1007/s40846-021-00664-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40846-021-00664-6