Advertisement

Multiple Device Segmentation for Fluoroscopic Imaging Using Multi-task Learning

  • Katharina BreiningerEmail author
  • Tobias Würfl
  • Tanja Kurzendorfer
  • Shadi Albarqouni
  • Marcus Pfister
  • Markus Kowarschik
  • Nassir Navab
  • Andreas Maier
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11043)

Abstract

For endovascular aortic repair (EVAR), integrating preoperative information of the aortic anatomy with intraoperative fluoroscopy can aid in reducing radiation exposure, contrast agent and procedure time. However, the quality of this fusion may deteriorate over the course of the intervention due to patient movement or deformation of the vasculature caused by interventional tools. Automatically detecting the instruments present in the X-ray image can help to assess the degree of deterioration, trigger automatic re-registration or aid in automatic workflow phase detection and process modeling. In this work, we investigate a flexible approach to segment different devices based on fully convolutional neural networks using multi-task learning. We evaluate the proposed approach on a set of 38 X-ray images acquired during EVAR interventions by targeting the segmentation of aortic stents, stiff guidewires and pigtail catheters. We compare the results to the performance of single-task networks. We manage to keep similar performance compared to single-task networks with Dice coefficients between 0.95 and 0.80 depending on the device, while speeding up computation by a factor of two.

Keywords

Endovascular Aortic Repair Multi-task learning Convolutional Neural Networks Stents Catheters Guidewires X-ray 

References

  1. 1.
    Ambrosini, P., Ruijters, D., Niessen, W.J., Moelker, A., van Walsum, T.: Fully automatic and real-time catheter segmentation in X-ray fluoroscopy. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10434, pp. 577–585. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-66185-8_65CrossRefGoogle Scholar
  2. 2.
    Baur, C., Albarqouni, S., Demirci, S., Navab, N., Fallavollita, P.: CathNets: detection and single-view depth prediction of catheter electrodes. In: Zheng, G., Liao, H., Jannin, P., Cattin, P., Lee, S.-L. (eds.) MIAR 2016. LNCS, vol. 9805, pp. 38–49. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-43775-0_4CrossRefGoogle Scholar
  3. 3.
    Breininger, K., Albarqouni, S., Kurzendorfer, T., Pfister, M., Kowarschik, M., Maier, A.: Intraoperative stent segmentation in X-ray fluoroscopy for endovascular aortic repair. IJCARS 13, 1221–1231 (2018).  https://doi.org/10.1007/s11548-018-1779-6CrossRefGoogle Scholar
  4. 4.
    Demirci, S., et al.: 3D stent recovery from one X-ray projection. In: Fichtinger, G., Martel, A., Peters, T. (eds.) MICCAI 2011. LNCS, vol. 6891, pp. 178–185. Springer, Heidelberg (2011).  https://doi.org/10.1007/978-3-642-23623-5_23CrossRefGoogle Scholar
  5. 5.
    Hoffmann, M.: Electrophysiology catheter detection and reconstruction from two views in fluoroscopic images. IEEE Trans. Med. Imaging 35(2), 567–579 (2015).  https://doi.org/10.1109/TMI.2015.2482539CrossRefGoogle Scholar
  6. 6.
    Kauffmann, C.: Source of errors and accuracy of a two-dimensional/three-dimensional fusion road map for endovascular aneurysm repair of abdominal aortic aneurysm. JVIR 26(4), 544–551 (2015).  https://doi.org/10.1016/j.jvir.2014.12.019CrossRefGoogle Scholar
  7. 7.
    Kingma, D., Ba, J.: Adam: A method for stochastic optimization. In: International Conference on Learning Representations (ICLR) (2015). https://dare.uva.nl/search?identifier=a20791d3-1aff-464a-8544-268383c33a75
  8. 8.
    Lessard, S., et al.: Automatic detection of selective arterial devices for advanced visualization during abdominal aortic aneurysm endovascular repair. Med. Eng. Phys. 37(10), 979–986 (2015).  https://doi.org/10.1016/j.medengphy.2015.07.007CrossRefGoogle Scholar
  9. 9.
    McNally, M.M., Scali, S.T., Feezor, R.J., Neal, D., Huber, T.S., Beck, A.W.: Three-dimensional fusion computed tomography decreases radiation exposure, procedure time, and contrast use during fenestrated endovascular aortic repair. J. Vasc. Surg. 61(2), 309–316 (2015).  https://doi.org/10.1016/j.jvs.2014.07.097CrossRefGoogle Scholar
  10. 10.
    Milletari, F., Navab, N., Ahmadi, S.A.: V-Net: fully convolutional neural networks for volumetric medical image segmentation. In: IEEE International Conference on 3DVision (2016)Google Scholar
  11. 11.
    Moeskops, P., et al.: Deep learning for multi-task medical image segmentation in multiple modalities. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 478–486. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46723-8_55CrossRefGoogle Scholar
  12. 12.
    Panuccio, G., et al.: Computer-aided endovascular aortic repair using fully automated two-and three-dimensional fusion imaging. J. Vasc. Surg. 64, 1587–1594 (2016).  https://doi.org/10.1016/j.jvs.2016.05.100CrossRefGoogle Scholar
  13. 13.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-24574-4_28CrossRefGoogle Scholar
  14. 14.
    Tacher, V., et al.: Image guidance for endovascular repair of complex aortic aneurysms: comparison of two-dimensional and three-dimensional angiography and image fusion. JVIR 24(11), 1698–1706 (2013).  https://doi.org/10.1016/j.jvir.2013.07.016CrossRefGoogle Scholar
  15. 15.
    Toth, D., Pfister, M., Maier, A., Kowarschik, M., Hornegger, J.: Adaption of 3D models to 2D X-ray images during endovascular abdominal aneurysm repair. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9349, pp. 339–346. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-24553-9_42CrossRefGoogle Scholar
  16. 16.
    Volpi, D., Sarhan, M.H., Ghotbi, R., Navab, N., Mateus, D., Demirci, S.: Online tracking of interventional devices for endovascular aortic repair. IJCARS 10(6), 773–781 (2015).  https://doi.org/10.1007/s11548-015-1217-yCrossRefGoogle Scholar
  17. 17.
    Wang, C.: Segmentation of multiple structures in chest radiographs using multi-task fully convolutional networks. In: Sharma, P., Bianchi, F.M. (eds.) SCIA 2017. LNCS, vol. 10270, pp. 282–289. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-59129-2_24CrossRefGoogle Scholar
  18. 18.
    Zhang, Z., Luo, P., Loy, C.C., Tang, X.: Facial landmark detection by deep multi-task learning. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8694, pp. 94–108. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10599-4_7CrossRefGoogle Scholar
  19. 19.
    Zhong, X., Hoffmann, M., Strobel, N., Maier, A.: Improved semi-automatic basket catheter reconstruction from two X-ray views. In: Tolxdorff, T., Deserno, T.M., Handels, H., Meinzer, H.P. (eds.) Bildverarbeitung für die Medizin 2016. I, pp. 26–31. Springer, Heidelberg (2016).  https://doi.org/10.1007/978-3-662-49465-3_7CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Katharina Breininger
    • 1
    Email author
  • Tobias Würfl
    • 1
  • Tanja Kurzendorfer
    • 1
    • 2
  • Shadi Albarqouni
    • 3
  • Marcus Pfister
    • 2
  • Markus Kowarschik
    • 2
  • Nassir Navab
    • 3
    • 4
  • Andreas Maier
    • 1
    • 5
  1. 1.Pattern Recognition LabFriedrich-Alexander-Universität Erlangen-NürnbergErlangenGermany
  2. 2.Siemens Healthcare GmbHForchheimGermany
  3. 3.Computer Aided Medical Procedures (CAMP)Technische Universität MünchenMunichGermany
  4. 4.Whiting School of EngineeringJohns Hopkins UniversityBaltimoreUSA
  5. 5.School of Advanced Optical Techniques (SAOT)ErlangenGermany

Personalised recommendations