Advertisement

Creating a Large-Scale Silver Corpus from Multiple Algorithmic Segmentations

  • Markus Krenn
  • Matthias Dorfer
  • Oscar Alfonso Jiménez del Toro
  • Henning Müller
  • Bjoern Menze
  • Marc-André Weber
  • Allan Hanbury
  • Georg Langs
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9601)

Abstract

Currently, increasingly large medical imaging data sets become available for research and are analysed by a range of algorithms segmenting anatomical structures automatically and interactively. While they provide segmentations on a much larger scale than possible to achieve with expert annotators, they are typically less accurate than experts. We present and compare approaches to estimate segmentations on large imaging data sets based on a small number of expert annotated examples, and algorithmic segmentations on a much larger data set. Results demonstrate that combining algorithmic segmentations is reliably outperforming the average individual algorithm. Furthermore, injecting organ specific reliability assessments of algorithms based on expert annotations improves accuracy compared to standard label fusion algorithms. The proposed methods are particularly relevant in putting the results of large image analysis algorithm benchmarks to long-term use.

Keywords

Segmentation Label fusion Silver corpus 

Notes

Acknowledgements

The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreements 318068 (VISCERAL) and 257528 (KHRESMOI). We furthermore acknowledge the support of NVIDIA Corporation with the donation of a Tesla K40 GPU used for this work and would like to thank all research groups contributing to this work by participating in the VISCERAL Anatomy 2 & 3 benchmarks [4, 5, 7, 9, 10, 12, 13, 14, 20, 23, 25].

References

  1. 1.
    Aljabar, P., Heckemann, R.A., Hammers, A., Hajnal, J.V., Rueckert, D.: Multi-atlas based segmentation of brain images: atlas selection and its effect on accuracy. NeuroImage 46(3), 726–738 (2009)CrossRefGoogle Scholar
  2. 2.
    Artaechevarria, X., Munoz-Barrutia, A., Ortiz-de Solórzano, C.: Combination strategies in multi-atlas image segmentation: application to brain mr data. IEEE Trans. Med. Imaging 28(8), 1266–1277 (2009)CrossRefGoogle Scholar
  3. 3.
    Dice, L.R.: Measures of the amount of ecologic association between species. Ecology 26(3), 297–302 (1945)CrossRefGoogle Scholar
  4. 4.
    Dicente Cid, Y., Depeursinge, A., Jiménez del Toro, O.A., Müller, H.: Efficient and fully automatic segmentation of the lungs in ct volumes. In: Goksel, O., Jiménez-del Toro, O.A., Foncubierta-Rodríguez, A., Müller, H. (eds.) Proceedings of the VISCERAL Challenge at ISBI, vol. 1390, p. 31, April 2015Google Scholar
  5. 5.
    Gass, T., Szekely, G., Goksel, O.: Multi-atlas segmentation and landmark localization in images with large field of view. In: Menze, B., Langs, G., Montillo, A., Kelm, M., Müller, H., Zhang, S., Cai, W.T., Metaxas, D. (eds.) MCV 2014. LNCS, vol. 8848, pp. 171–180. Springer, Heidelberg (2014)Google Scholar
  6. 6.
    Göksel, O., Jiménez-del Toro, O.A., Foncubierta-Rodríguez, A., Muller, H.: Overview of the VISCERAL challenge at ISBI 2015. In: Göksel, O., Jiménez-del Toro, O.A., Foncubierta-Rodríguez, A., Müller, H. (eds.) Proceedings of the VISCERAL Challenge at ISBI, New York, NY, May 2015Google Scholar
  7. 7.
    He, B., Huang, C., Jia, F.: Fully automatic multi-organ segmentation based on multi-boost learning and statistical shape model search. In: Goksel, O., Jiménez-del Toro, O.A., Foncubierta-Rodríguez, A., Müller, H. (eds.) Proceedings of the VISCERAL Challenge at ISBI, vol. 1390, pp. 18–21, April 2015Google Scholar
  8. 8.
    Heckemann, R.A., Hajnal, J.V., Aljabar, P., Rueckert, D., Hammers, A.: Automatic anatomical brain MRI segmentation combining label propagation and decision fusion. NeuroImage 33(1), 115–126 (2006)CrossRefGoogle Scholar
  9. 9.
    Heinrich, M.P., Maier, O., Handels, H.: Multi-modal multi-atlas segmentation using discrete optimisation and self-similarities. In: Goksel, O., Jiménez-del Toro, O.A., Foncubierta-Rodríguez, A., Müller, H. (eds.) Proceedings of the VISCERAL Challenge at ISBI, vol. 1390, p. 27, April 2015Google Scholar
  10. 10.
    Jiménez del Toro, O.A., Dicente Cid, Y., Depeursinge, A., Müller, H., Hierarchic anatomical structure segmentation guided by spatial correlations (anatseg-gspac): Visceral anatomy3. In: Goksel, O., Jiménez-del Toro, O.A., Foncubierta-Rodríguez, A., Müller, H. (eds.) Proceedings of the VISCERAL Challenge at ISBI, vol. 1390, pp. 22–66. CEUR-WS, April 2015. http://ceur-ws.org
  11. 11.
    Jiménez del Toro, O.A., Goksel, O., Menze, B., Müller, H., Langs, G., Weber, M.A., Eggel, I., Gruenberg, K., Holzer, M., Jakab, A., Kotsios-Kontokotsios, G., Krenn, M., Salas Fernandez, T., Schaer, R., Abdel Aziz, T., Winterstein, M., Hanbury, A.: Visceral-visual concept extraction challenge in radiology: ISBI 2014 challenge organization. In: Göksel, O. (ed.) Proceedings of the VISCERAL Challenge at ISBI. CEUR Workshop Proceedings, pp. 6–15 (2014)Google Scholar
  12. 12.
    Jiménez del Toro, O.A., Müller, H.: Hierarchic Multi–atlas based segmentation for anatomical structures: evaluation in the VISCERAL anatomy benchmarks. In: Menze, B., Langs, G., Montillo, A., Kelm, M., Müller, H., Zhang, S., Cai, W.T., Metaxas, D. (eds.) MCV 2014. LNCS, vol. 8848, pp. 189–200. Springer, Heidelberg (2014)Google Scholar
  13. 13.
    Kahl, F., Alvén, J., Enqvist, O., Fejne, F., Ulén, J., Fredriksson, J., Landgren, M., Larsson, V.: Good features for reliable registration in multi-atlas segmentation. In: Goksel, O., Jiménez-del Toro, O.A., Foncubierta-Rodríguez, A., Müller, H. (eds.) Proceedings of the VISCERAL Challenge at ISBI, vol. 1390, pp. 12–17, April 2015Google Scholar
  14. 14.
    Kéchichian, R., Valette, S., Sdika, M., Desvignes, M.: Automatic 3D multiorgan segmentation via clustering and graph cut using spatial relations and hierarchically-registered atlases. In: Menze, B., Langs, G., Montillo, A., Kelm, M., Müller, H., Zhang, S., Cai, W.T., Metaxas, D. (eds.) MCV 2014. LNCS, vol. 8848, pp. 201–209. Springer, Heidelberg (2014)Google Scholar
  15. 15.
    Kittler, J., Alkoot, F.M.: Sum versus vote fusion in multiple classifier systems. IEEE Trans. Pattern Anal. Mach. Intell. 25(1), 110–115 (2003)CrossRefGoogle Scholar
  16. 16.
    Kittler, J., Hatef, M., Duin, R.P.W., Matas, J.: On combining classifiers. IEEE Trans. Pattern Anal. Mach. Intell. 20(3), 226–239 (1998)CrossRefGoogle Scholar
  17. 17.
    Klein, S., Staring, M., Murphy, K., Viergever, M., Pluim, J.P.W.: Elastix: a toolbox for intensity-based medical image registration. IEEE Trans. Med. Imaging 29(1), 196–205 (2010)CrossRefGoogle Scholar
  18. 18.
    Langerak, T.R., Van der Heide, U.A., Kotte, A.N.T.J., Viergever, M.A., Van Vulpen, M., Pluim, J.P.W.: Label fusion in atlas-based segmentation using a selective and iterative method for performance level estimation (simple). IEEE Trans. Med. Imaging 29(12), 2000–2008 (2010)CrossRefGoogle Scholar
  19. 19.
    Langs, G., Hanbury, A., Menze, B., Müller, H.: VISCERAL: towards large data in medical imaging — challenges and directions. In: Greenspan, H., Müller, H., Syeda-Mahmood, T. (eds.) MCBR-CDS 2012. LNCS, vol. 7723, pp. 92–98. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  20. 20.
    Li, X., Huang, C., Jia, F., Li, Z., Fang, C., Fan, Y.: Automatic liver segmentation using statistical prior models and free-form deformation. In: Menze, B., Langs, G., Montillo, A., Kelm, M., Müller, H., Zhang, S., Cai, W.T., Metaxas, D. (eds.) MCV 2014. LNCS, vol. 8848, pp. 181–188. Springer, Heidelberg (2014)Google Scholar
  21. 21.
    Rohlfing, T., Brandt, R., Menzel, R., Maurer, C.R.: Evaluation of atlas selection strategies for atlas-based image segmentation with application to confocal microscopy images of bee brains. NeuroImage 21(4), 1428–1442 (2004)CrossRefGoogle Scholar
  22. 22.
    Roli, F., Kittler, J., Fumera, G., Muntoni, D.: An experimental comparison of classifier fusion rules for multimodal personal identity verification systems. In: Roli, F., Kittler, J. (eds.) MCS 2002. LNCS, vol. 2364, pp. 325–335. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  23. 23.
    Spanier, A.B., Joskowicz, L.: Rule-based ventral cavity multi-organ automatic segmentation in CT scans. In: Menze, B., Langs, G., Montillo, A., Kelm, M., Müller, H., Zhang, S., Cai, W.T., Metaxas, D. (eds.) MCV 2014. LNCS, vol. 8848, pp. 163–170. Springer, Heidelberg (2014)Google Scholar
  24. 24.
    Studholme, C., Hill, D.L.G., Hawkes, D.J.: An overlap invariant entropy measure of 3D medical image alignment. Pattern recognition 32(1), 71–86 (1999)CrossRefGoogle Scholar
  25. 25.
    Wang, C., Smedby, O.: Automatic multi-organ segmentation using fast model based level set method and hierarchical shape priors. In: Goksel, O., Jiménez-del Toro, O.A., Foncubierta-Rodríguez, A., Müller, H. (eds.) Proceedings of the VISCERAL Challenge at ISBI, vol. 1194, pp. 25–31 (2014)Google Scholar
  26. 26.
    Warfield, S.K., Zou, K.H., Wells III, W.M.: Validation of image segmentation and expert quality with an expectation-maximization algorithm. In: Dohi, T., Kikinis, R. (eds.) MICCAI 2002, Part I. LNCS, vol. 2488, pp. 298–306. Springer, Heidelberg (2002)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Markus Krenn
    • 1
  • Matthias Dorfer
    • 2
  • Oscar Alfonso Jiménez del Toro
    • 3
  • Henning Müller
    • 3
  • Bjoern Menze
    • 4
  • Marc-André Weber
    • 5
  • Allan Hanbury
    • 6
  • Georg Langs
    • 1
  1. 1.Computational Imaging Research (CIR) Lab, Department of Biomedical Imaging and Image-guided TherapyMedical University of ViennaViennaAustria
  2. 2.Department of Computational PerceptionJohannes Kepler University (JKU)LinzAustria
  3. 3.University of Applied Sciences Western Switzerland (HES-SO)SierreSwitzerland
  4. 4.Institute for Advanced Study and Department of Computer ScienceTechnische Universität MünchenMunichGermany
  5. 5.Department of Diagnostic and Interventional RadiologyUniversity of HeidelbergHeidelbergGermany
  6. 6.Institute of Software Technology and Interactive SystemsTU WienViennaAustria

Personalised recommendations