The Visual Computer

, Volume 29, Issue 9, pp 893–905 | Cite as

Opening up the “black box” of medical image segmentation with statistical shape models

  • Tatiana von LandesbergerEmail author
  • Gennady Andrienko
  • Natalia Andrienko
  • Sebastian Bremm
  • Matthias Kirschner
  • Stefan Wesarg
  • Arjan Kuijper
Original Article


The importance of medical image segmentation increases in fields like treatment planning or computer aided diagnosis. For high quality automatic segmentations, algorithms based on statistical shape models (SSMs) are often used. They segment the image in an iterative way. However, segmentation experts and other users can only asses the final segmentation results, as the segmentation is performed in a “black box manner”. Users cannot get deeper knowledge on how the (possibly bad) output was produced. Moreover, they do not see whether the final output is the result of a stabilized process.

We present a novel Visual Analytics method, which offers this desired deeper insight into the image segmentation. Our approach combines interactive visualization and automatic data analysis. It allows the expert to assess the quality development (convergence) of the model both on global (full organ) and local (organ areas, landmarks) level. Thereby, local patterns in time and space, e.g., non-converging parts of the organ during the segmentation, can be identified. The localization and specifications of such problems helps the experts creating segmentation algorithms to identify algorithm drawbacks and thus it may point out possible ways how to improve the algorithms systematically.

We apply our approach on real-world data showing its usefulness for the analysis of the segmentation process with statistical shape models.


Medical imaging Medical modeling Visual analytics Image segmentation Statistical shape models Spatio-temporal data 



The work has been partially supported by the DFG SPP 1335 project “Visual Analytics Methods for Modeling in Medical Imaging”. The authors would like to thank J. Beutel for his support with the project.


  1. 1.
    Andrienko, G., Andrienko, N., Bremm, S., Schreck, T., Von Landesberger, T., Bak, P., Keim, D.: Space-in-time and time-in-space self-organizing maps for exploring spatiotemporal patterns. Comput. Graph. Forum 29(3), 913–922 (2010) CrossRefGoogle Scholar
  2. 2.
    Andrienko, N., Andrienko, G.: Spatial generalization and aggregation of massive movement data. IEEE Trans. Vis. Comput. Graph. 17(2), 205–219 (2011) CrossRefGoogle Scholar
  3. 3.
    Andrienko, N., Andrienko, G.: Visual analytics of movement: an overview of methods, tools and procedures. Inf. Vis. 12(1), 3–24 (2013) CrossRefGoogle Scholar
  4. 4.
    Angelelli, P., Viola, I., Nylund, K., Gilja, O.H., Hauser, H.: Guided visualization of ultrasound image sequences. In: Eurographics Workshop on Visual Computing for Biology and Medicine (VCBM), pp. 125–132 (2010) Google Scholar
  5. 5.
    Bruckner, S., Möller, T.: Isosurface similarity maps. Comput. Graph. Forum 29(3), 773–782 (2010) CrossRefGoogle Scholar
  6. 6.
    Busking, S., Botha, C.P., Ferrarini, L., Milles, J., Post, F.H.: Image-based rendering of intersecting surfaces for dynamic comparative visualization. Vis. Comput. 27(5), 347–363 (2011) CrossRefGoogle Scholar
  7. 7.
    Busking, S., Botha, C.P., Post, F.H.: Dynamic multi-view exploration of shape spaces. Comput. Graph. Forum 29(3), 973–982 (2010) CrossRefGoogle Scholar
  8. 8.
    Cootes, T.F., Taylor, C.J., Cooper, D.H., Graham, J.: Active shape models—their training and application. Comput. Vis. Image Underst. 61(1), 38–59 (1995) CrossRefGoogle Scholar
  9. 9.
    Demšar, U., Fotheringham, A.S., Charlton, M.: Exploring the spatio-temporal dynamics of geographical processes with geographically weighted regression and geovisual analytics. Inf. Vis. 7(3), 181–197 (2008) Google Scholar
  10. 10.
    Dick, C., Burgkart, R., Westermann, R.: Distance visualization for interactive 3d implant planning. IEEE Trans. Vis. Comput. Graph. 17(12), 2173–2182 (2011) CrossRefGoogle Scholar
  11. 11.
    Heimann, T., van Ginneken, B., Styner, M., et al.: Comparison and evaluation of methods for liver segmentation from CT datasets. IEEE Trans. Med. Imaging 28, 1251–1265 (2009) CrossRefGoogle Scholar
  12. 12.
    Heimann, T., Meinzer, H.P.: Statistical shape models for 3D medical image segmentation: a review. Med. Image Anal. 13(4), 543–563 (2009) CrossRefGoogle Scholar
  13. 13.
    Heimann, T., Münzing, S., Meinzer, H.P., Wolf, I.: A shape-guided deformable model with evolutionary algorithm initialization for 3D soft tissue segmentation. In: Information Processing in Medical Imaging, pp. 1–12 (2007) CrossRefGoogle Scholar
  14. 14.
    Himberg, J.: A som based cluster visualization and its application for false coloring. In: Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks, 2000, IJCNN 2000, vol. 3, pp. 587–592 (2000). doi: 10.1109/IJCNN.2000.861379 Google Scholar
  15. 15.
    Kohonen, T.: Self-Organizing Maps, 3rd. edn. Springer, Berlin (2001) zbMATHCrossRefGoogle Scholar
  16. 16.
    Maciejewski, R., Rudolph, S., Hafen, R., Abusalah, A., Yakout, M., Ouzzani, M., Cleveland, W., Grannis, S., Ebert, D.: A visual analytics approach to understanding spatiotemporal hotspots. IEEE Trans. Vis. Comput. Graph. 16(2), 205–220 (2010) CrossRefGoogle Scholar
  17. 17.
    Matkovic, K., Gracanin, D., Jelovic, M., Ammer, A., Lez, A., Hauser, H.: Interactive visual analysis of multiple simulation runs using the simulation model view: understanding and tuning of an electronic unit injector. IEEE Trans. Vis. Comput. Graph. 16(6), 1449–1457 (2010) CrossRefGoogle Scholar
  18. 18.
    Mayer, et al.: Java SOMToolbox. Online., accessed 8/2/2013
  19. 19.
    Preim, B., Bartz, D.: Visualization in Medicine: Theory, Algorithms, and Applications. Morgan Kaufmann, San Mateo (2007) Google Scholar
  20. 20.
    Rinzivillo, S., Pedreschi, D., Nanni, M., Giannotti, F., Andrienko, N., Andrienko, G.: Visually driven analysis of movement data by progressive clustering. Inf. Vis. 7(3–4), 225–239 (2008) CrossRefGoogle Scholar
  21. 21.
    Schreck, T., Bernard, J., von Landesberger, T., Kohlhammer, J.: Visual cluster analysis of trajectory data with interactive Kohonen maps. Inf. Vis. 8(1), 14–29 (2009) CrossRefGoogle Scholar
  22. 22.
    Silva, S., Madeira, J., Santos, B.: Polymeco—a polygonal mesh comparison tool. In: Proceedings Ninth International Conference on Information Visualisation, 2005, pp. 842–847. IEEE, New York (2005) Google Scholar
  23. 23.
    Tversky, B., Morrison, J.B., Betrancourt, M.: Animation: can it facilitate? Int. J. Hum.-Comput. Stud. 57, 247–262 (2002) CrossRefGoogle Scholar
  24. 24.
    Vesanto, J.: Som-based data visualization methods. Intell. Data Anal. 3(2), 111–126 (1999) zbMATHCrossRefGoogle Scholar
  25. 25.
    Willems, N., Van de Wetering, H., Van Wijk, J.J.: Visualization of vessel movements. Comput. Graph. Forum 28(3), 959–966 (2009) CrossRefGoogle Scholar
  26. 26.
    Zhou, L., Pang, A.: Metrics and visualization tools for surface mesh comparison. Ph.D. thesis, University of California, Santa Cruz (2001) Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Tatiana von Landesberger
    • 1
    Email author
  • Gennady Andrienko
    • 2
  • Natalia Andrienko
    • 2
  • Sebastian Bremm
    • 1
  • Matthias Kirschner
    • 1
  • Stefan Wesarg
    • 3
  • Arjan Kuijper
    • 3
  1. 1.TU DarmstadtDarmstadtGermany
  2. 2.Fraunhofer IAISBonnGermany
  3. 3.Fraunhofer IGDDarmstadtGermany

Personalised recommendations