Evaluation of Interactive Segmentation Algorithms Using Densely Sampled Correct Interactions

  • S. M. Rafizul Haque
  • Mark G. Eramian
  • Kevin A. Schneider
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8156)


The accuracy and reproducibility of semiautomatic interactive segmentation algorithms are typically evaluated using only a small number of human observers which only considers a very small number of the possible correct interactions that an observer might provide. A correct interaction is one that provides contextual information that would be expected to result in a correct segmentation. In this paper, we demonstrate new evaluation methods for semiautomatic interactive segmentation algorithms that employ simulated observer models constructed from a large number of segmentations computed by uniformly sampling the entire set of possible correct interactions. The advantages of this method are that it is free of observer biases and the large number of segmentations produced for each object of interest to be segmented allow a range of statistical methods to be brought to bear on the analysis of segmentation algorithm performance. The methods are demonstrated using a semi-automated segmentation algorithm for ovarian follicles in ultrasonographic images.


interactive segmentation semiautomatic correct interaction reproducibility performance evaluation 


  1. 1.
    Baerwald, A.R., Adams, G.P., Pierson, R.A.: Characterization of ovarian follicular wave dynamics in women. Biology of Reproduction 69(3), 1023–1031 (2003)CrossRefGoogle Scholar
  2. 2.
    Bowyer, K.W.: Validation of medical image analysis techniques. In: Sonka, M., Fitzpatrick, J.M. (eds.) Handbook of Medical Imaging: Medical Image Processing and Analysis, vol. 2, SPIE press (2000)Google Scholar
  3. 3.
    Boykov, Y., Jolly, M.P.: Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images. In: Proceedings of the International Conference on Computer Vision (ICCV 2001), pp. 105–112 (2001)Google Scholar
  4. 4.
    Boykov, Y., Lea, G.F.: Graph cuts and efficient n-d image segmentation. International Journal of Computer Vision 70(2), 109–131 (2006)CrossRefGoogle Scholar
  5. 5.
    Byrum, C.E., MacFall, J.R., Charles, H.C., Chitilla, V.R., Boyko, O.B., Upchurch, L., Smith, J.S., Rajagopalan, P., Passe, T., Kim, D., Xanthakos, S., Ranga, K., Krishnan, R.: Accuracy and reproducability of brain and tissue volumes using a magnetic resonance segmentation method. Psychiatry Research: Neuroimaging 67, 215–234 (1996)CrossRefGoogle Scholar
  6. 6.
    Cates, J.E., Lefohn, A.E., Whitaker, R.T.: Gist: an interactive gpu-based level set segmentation tool for 3d medical images. Medical Image Analysis 8, 217–231 (2004)CrossRefGoogle Scholar
  7. 7.
    Coehn, B.A., Barash, I., Kim, D.C., Sanger, M.D., Babb, J.S., Chandarana, H.: Intraobserve and interobserver variability in renal volume measurements in polycystic kidney disease using a semiautomated mr segmentation algorithm. American Journal of Roentgenology 199, 387–393 (2012)CrossRefGoogle Scholar
  8. 8.
    Dach, C., Held, C., Wenzel, J., Gerlach, S., Lang, R., Palmisano, R., Wittenberg, T.: Evaluation of an interactive cell segmentation for flourescence microscopy based on the graph cut algorithm. In: Microscopic Image Analysis with Applications in Biology, Heidelberg, Germany (September 2, 2011) Google Scholar
  9. 9.
    Dice, L.R.: Measures of the amount of ecologic association between species. Ecology 26(3), 297–302 (1945)CrossRefGoogle Scholar
  10. 10.
    McGuinness, K., O’Connor, N.E.: A comparative evaluation of interactive segmentation algorithms. Pattern Recognition 43(2), 434–444 (2010)CrossRefzbMATHGoogle Scholar
  11. 11.
    Moschidis, E., Graham, J.: A systematic performance evaluation of interactive image segmentation methods based on simulated user interaction. In: IEEE International Symposium on Biomedical Imaging: From Nano to Macro, pp. 928–931 (2010)Google Scholar
  12. 12.
    Nickisch, H., Rother, C., Kohli, P., Rhemann, C.: Learning an interactive segmentation system. In: Proceedings of the Seventh Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP 2010), pp. 274–281 (2010)Google Scholar
  13. 13.
    Schenk, A., Prause, G.P.M., Peitgen, H.-O.: Efficient semiautomatic segmentation of 3D objects in medical images. In: Delp, S.L., DiGoia, A.M., Jaramaz, B. (eds.) MICCAI 2000. LNCS, vol. 1935, pp. 186–195. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  14. 14.
    Stammberger, T., Eckstein, F., Michaelis, M., Englmeier, K.-H., Reiser, M.: Interobserver reproducibility of quantitative cartilage measurements: Comparison of b-spline snakes and manual segmentation. Magnetic Resonance Imaging 17(7), 1033–1042 (1999)CrossRefGoogle Scholar
  15. 15.
    Veksler, O.: Star shape prior for graph-cut image segmentation. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 454–467. Springer, Heidelberg (2008)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • S. M. Rafizul Haque
    • 1
  • Mark G. Eramian
    • 1
  • Kevin A. Schneider
    • 1
  1. 1.University of SaskatchewanSaskatoonCanada

Personalised recommendations