Interactive Neural Network Robot User Investigation for Medical Image Segmentation

  • Mario AmrehnEmail author
  • Maddalena Strumia
  • Markus Kowarschik
  • Andreas Maier
Conference paper
Part of the Informatik aktuell book series (INFORMAT)


Interactive image segmentation bears the advantage of correctional updates to the current segmentation mask when compared to fully automated systems. Especially in the field of inter-operative medical image processing of a single patient, where a high accuracy is an uncompromisable necessity, a human operator guiding a system towards an optimal segmentation result is a time-efficient constellation benefiting the patient. There are recent categories of neural networks which can incorporate human-computer interaction (HCI) data as additional input for segmentation. In this work, we simulate this HCI data during training with state-of-the-art user models, also called robot users, which aim to act similar to real users given interactive image segmentation tasks. We analyze the influence of chosen robot users, which mimic different types of users and scribble patterns, on the segmentation quality. We conclude that networks trained with robot users with the most spread out seeding patterns generalize well during inference with other robot users.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Lewandowski RJ, Geschwind JF, Liapi E, et al. Transcatheter intraarterial therapies: rationale and overview. Radiol. 2011;259(3):641–657.CrossRefGoogle Scholar
  2. 2.
    Strobel N, Meissner O, Boese J, et al. 3D imaging with flat-detector C-arm systems. Multislice CT. 2009; p. 33–51.Google Scholar
  3. 3.
    Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation. proc MICCAI. 2015; p. 234–241.Google Scholar
  4. 4.
    Xu N, Price B, Cohen S, et al. Deep interactive object selection. Proc CVPR. 2016; p. 373–381.Google Scholar
  5. 5.
    Liew JH,Wei Y, Xiong W, et al. Regional interactive image segmentation networks. Proc ICCV. 2017; p. 2746–2754.Google Scholar
  6. 6.
    Amrehn MP, Gaube S, Unberath M, et al. UI-net: interactive artificial neural networks for iterative image segmentation based on a user model. Proc VCBM. 2017; p. 143–147.Google Scholar
  7. 7.
    Amrehn MP, Steidl S, Kowarschik M, et al. Robust seed mask generation for interactive image segmentation. Proc NSS/MIC. 2017; p. 1–3.Google Scholar
  8. 8.
    Wang G, Zuluaga MA, Li W, et al. DeepIGeoS: a deep interactive geodesic framework for medical image segmentation. Trans Pattern Anal Mach Intell. 2018;.Google Scholar
  9. 9.
    Kohli P, Nickisch H, Rother C, et al. User-centric learning and evaluation of interactive segmentation systems. Proc IJCV. 2012;100(3):261–274.MathSciNetCrossRefGoogle Scholar
  10. 10.
    Vezhnevets V, Konouchine V. GrowCut: interactive multi-label ND image segmentation by cellular automata. Comput Graph Appl (Graphicon). 2005; p. 150–156.Google Scholar

Copyright information

© Springer Fachmedien Wiesbaden GmbH, ein Teil von Springer Nature 2019

Authors and Affiliations

  • Mario Amrehn
    • 1
    Email author
  • Maddalena Strumia
    • 2
  • Markus Kowarschik
    • 2
  • Andreas Maier
    • 1
    • 3
  1. 1.Pattern Recognition LabFriedrich-Alexander University Erlangen-Nürnberg (FAU)ErlangenDeutschland
  2. 2.Siemens Healthcare GmbHForchheimDeutschland
  3. 3.Erlangen Graduate School in Advanced Optical Technologies (SAOT)ErlangenDeutschland

Personalised recommendations