Skip to main content
SpringerLink
Log in

Extended surface distance for local evaluation of 3D medical image segmentations

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

The evaluation of 3D medical image segmentation quality requires a reliable detailed comparison of a reference segmentation with an automatic segmentation. It should be able to measure the quality accurately and, thus, to reveal problematic regions. While several (global) measures, providing a single quality value, are available, the only widely used local measure is the Surface Distance (i.e., point-to-surface distance). This measure, however, has significant drawbacks such as asymmetry and underestimation in distant and differently formed regions. Other available measures have limited suitability for 3D medical segmentation evaluation. We present a more reliable distance measure for assessing and analyzing local differences between automatic and reference (i.e., ground truth) 3D segmentations. We identify and overcome Surface Distance drawbacks, esp. in regions with larger dissimilarities. We evaluated our approach on four real medical image datasets. The results indicate that our measure provides more accurate local distance values.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Aspert, N., Santa-Cruz, D., Ebrahimi, T.: Mesh: measuring errors between surfaces using the hausdorff distance. In: Proceedings of Multimedia and Expo, ICME, vol. 1, pp. 705–708. IEEE, New York (2002)

  2. Becker, M., Kirschner, M., Sakas, G.: Segmentation of risk structures for otologic surgery using the probabilistic active shape model (pasm). In: Proceedings of SPIE Medical Imaging, pp. 90,360O–90,360O. International Society for Optics and Photonics (2014)

  3. Belongie, S., Malik, J., Puzicha, J.: Matching shapes. In: Proceedings of Computer Vision, ICCV, vol. 1, pp. 454–461. IEEE, New York (2001)

  4. Brown, B.J., Rusinkiewicz, S.: Global non-rigid alignment of 3-d scans. In: Proceedings of ACM T. Graphics (TOG), vol. 26, p. 21. ACM, New York (2007)

  5. Cates, J., Meyer, M., Fletcher, T., Whitaker, R., et al.: Entropy-based particle systems for shape correspondence. In: Proceedings of 1st MICCAI Workshop on Mathematical Foundations of Computational Anatomy: Geometrical, Statistical and Registration Methods for Modeling Biological Shape Variability, pp. 90–99 (2006)

  6. Chalana, V., Kim, Y.: A methodology for evaluation of boundary detection algorithms on medical images. IEEE T. Med. Imaging 16(5), 642–652 (1997)

    Article  Google Scholar 

  7. Cohen-Steiner, D., Alliez, P., Desbrun, M.: Variational shape approximation. ACM T. Gr. 23(3), 905–914 (2004)

    Article  Google Scholar 

  8. Gerig, G., Jomier, M., Chakos, M.: Valmet: a new validation tool for assessing and improving 3D object segmentation. In: Proceedings of Medical Image Computing and Computer-Assisted Intervention, MICCAI, pp. 516–523. Springer, Berlin, Germany (2001)

  9. Gueziec, A.: Meshsweeper: dynamic point-to-polygonal mesh distance and applications. IEEE T. Vis. Comput. Gr. 7(1), 47–61 (2001)

    Article  Google Scholar 

  10. Haehnel, D., Thrun, S., Burgard, W.: An extension of the icp algorithm for modeling nonrigid objects with mobile robots. In: Proceedings of IJCAI, pp. 915–920 (2003)

  11. Heimann, T., van Ginneken, B., Styner, M.A., Arzhaeva, Y., Aurich, V., Bauer, C., Beck, A., Becker, C., Beichel, R., Bekes, G., et al.: Comparison and evaluation of methods for liver segmentation from ct datasets. IEEE T. Med. Imaging 28(8), 1251–1265 (2009)

    Article  Google Scholar 

  12. Heimann, T., Meinzer, H.P.: Statistical shape models for 3d medical image segmentation: a review. Med. Image Anal. 13(4), 543–563 (2009)

    Article  Google Scholar 

  13. Hoppe, H.: Progressive meshes. In: Proceedings of Computer Graphics and Interactive Techniques, pp. 99–108. ACM, New York (1996)

  14. Johnson, A.E.: Spin-images: a representation for 3-D surface matching. Ph.D. thesis, Carnegie Mellon University (1997)

  15. Kirschner, M., Becker, M., Wesarg, S.: 3D active shape model segmentation with nonlinear shape priors. In: Proceedings of Medical Image Computing and Computer-Assisted Intervention, MICCAI, LNCS, vol. 6892, pp. 492–499 (2011)

  16. Kraevoy, V., Sheffer, A.: Cross-parameterization and compatible remeshing of 3d models. In: Proceedings of ACM T. Graphics (TOG), vol. 23, pp. 861–869. ACM, New York (2004)

  17. Landesberger, T.V., Andrienko, G., Andrienko, N., Bremm, S., Kirschner, M., Wesarg, S., Kuijper, A.: Opening up the “black box” of medical image segmentation with statistical shape models. Vis. Comput. 29(9), 893–905 (2013)

    Article  Google Scholar 

  18. Li, H., Hartley, R.: The 3D–3D registration problem revisited. In: Proceedings of Computer Vision, ICCV, pp. 1–8. IEEE, New York (2007)

  19. Li, H., Sumner, R.W., Pauly, M.: Global correspondence optimization for non-rigid registration of depth scans. In: Proceedings of Computer Graphics Forum, vol. 27, pp. 1421–1430. Wiley, New Jersey (2008)

  20. Lipman, Y., Rustamov, R.M., Funkhouser, T.A.: Biharmonic distance. ACM T. Gr. 29(3), 27 (2010)

    Google Scholar 

  21. Lorenz, C., Krahnstover, N.: 3D statistical shape models for medical image segmentation. In: Proceedings of 3-D Digital Imaging and Modeling, pp. 414–423. IEEE, New York (1999)

  22. Martinek, M., Grosso, R., Greiner, G.: Interactive partial 3d shape matching with geometric distance optimization. Vis. Comput. 31(2), 223–233 (2015)

    Article  Google Scholar 

  23. Pizer, S.M., Fletcher, P.T., Joshi, S., Thall, A., Chen, J.Z., Fridman, Y., Fritsch, D.S., Gash, A.G., Glotzer, J.M., Jiroutek, M.R., et al.: Deformable m-reps for 3d medical image segmentation. Int. J. Comput. Vis. 55(2–3), 85–106 (2003)

    Article  Google Scholar 

  24. Sahillioglu, Y., Yemez, Y.: Coarse-to-fine combinatorial matching for dense isometric shape correspondence. In: Proceedings of Computer Graphics Forum, vol. 30, pp. 1461–1470. Wiley, New Jersey (2011)

  25. Strecha, C., von Hansen, W., Van Gool, L., Fua, P., Thoennessen, U.: On benchmarking camera calibration and multi-view stereo for high resolution imagery. In: Proceedings of Computer Vision and Pattern Recognition, CVPR, pp. 1–8. IEEE, New York (2008)

  26. Udupa, J.K., Leblanc, V.R., Zhuge, Y., Imielinska, C., Schmidt, H., Currie, L.M., Hirsch, B.E., Woodburn, J.: A framework for evaluating image segmentation algorithms. Comput. Med. Imaging Gr. 30(2), 75–87 (2006)

    Article  Google Scholar 

  27. Van Ginneken, B., Heimann, T., Styner, M.: 3D segmentation in the clinic: a grand challenge. 3D segmentation in the clinic: a grand challenge, pp. 7–15 (2007)

  28. Van Kaick, O., Zhang, H., Hamarneh, G., Cohen-Or, D.: A survey on shape correspondence. In: Proceedings of Computer Graphics Forum, vol. 30, pp. 1681–1707. Wiley, New Jersey (2011)

  29. Veltkamp, R.C., Hagedoorn, M.: State-of-the-art in shape matching. In: Proceedings of Technical Report, Principles of Visual Information Retrieval (1999)

  30. Wu, H., Miao, Z., Wang, Y., Lin, M.: Optimized recognition with few instances based on semantic distance. Vis. Comput. 31(4), 367–375 (2015)

    Article  Google Scholar 

  31. Zhang, H., Sheffer, A., Cohen-Or, D., Zhou, Q., Van Kaick, O., Tagliasacchi, A.: Deformation-driven shape correspondence. In: Proceedings of Computer Graphics Forum, vol. 27, pp. 1431–1439. Wiley, New York (2008)

  32. Zou, K.H., Warfield, S.K., Bharatha, A., Tempany, C.M., Kaus, M.R., Haker, S.J., Wells, W.M., Jolesz, F.A., Kikinis, R.: Statistical validation of image segmentation quality based on a spatial overlap index 1: scientific reports. Acad. Radiol. 11(2), 178–189 (2004)

    Article  Google Scholar 

Download references

Acknowledgments

The work has been partially supported by DFG within an SPP 1335 project. The authors are grateful to Prof. Georg Sakas and Dr. Meike Becker for the data provision, support with the project and helpful comments on the paper draft.

Author information

Authors and Affiliations

  1. TU Darmstadt, Darmstadt, Germany

    Roman Getto & Tatiana von Landesberger

  2. TU Darmstadt and Fraunhofer IGD, Darmstadt, Germany

    Arjan Kuijper

Authors
  1. Roman Getto
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arjan Kuijper
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tatiana von Landesberger
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Roman Getto.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Getto, R., Kuijper, A. & von Landesberger, T. Extended surface distance for local evaluation of 3D medical image segmentations. Vis Comput 31, 989–999 (2015). https://doi.org/10.1007/s00371-015-1113-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-015-1113-z

Keywords

Search

Navigation