Advertisement

User-Guided Segmentation of Multi-modality Medical Imaging Datasets with ITK-SNAP

  • Paul A. Yushkevich
  • Artem Pashchinskiy
  • Ipek Oguz
  • Suyash Mohan
  • J. Eric Schmitt
  • Joel M. Stein
  • Dženan Zukić
  • Jared Vicory
  • Matthew McCormick
  • Natalie Yushkevich
  • Nadav Schwartz
  • Yang Gao
  • Guido Gerig
Software Original Article

Abstract

ITK-SNAP is an interactive software tool for manual and semi-automatic segmentation of 3D medical images. This paper summarizes major new features added to ITK-SNAP over the last decade. The main focus of the paper is on new features that support semi-automatic segmentation of multi-modality imaging datasets, such as MRI scans acquired using different contrast mechanisms (e.g., T1, T2, FLAIR). The new functionality uses decision forest classifiers trained interactively by the user to transform multiple input image volumes into a foreground/background probability map; this map is then input as the data term to the active contour evolution algorithm, which yields regularized surface representations of the segmented objects of interest. The new functionality is evaluated in the context of high-grade and low-grade glioma segmentation by three expert neuroradiogists and a non-expert on a reference dataset from the MICCAI 2013 Multi-Modal Brain Tumor Segmentation Challenge (BRATS). The accuracy of semi-automatic segmentation is competitive with the top specialized brain tumor segmentation methods evaluated in the BRATS challenge, with most results obtained in ITK-SNAP being more accurate, relative to the BRATS reference manual segmentation, than the second-best performer in the BRATS challenge; and all results being more accurate than the fourth-best performer. Segmentation time is reduced over manual segmentation by 2.5 and 5 times, depending on the rater. Additional experiments in interactive placenta segmentation in 3D fetal ultrasound illustrate the generalizability of the new functionality to a different problem domain.

Keywords

Image segmentation Semi-automatic segmentation Gliomas Software MRI 

Notes

Acknowledgements

This work was supported by NIH grants R01 EB014346, R01 EB017255, K01 ES026840, and U01 HD087180. We gratefully acknowledge the many researchers and developers who contributed to ITK-SNAP software over the past 20 years. A full list of contributors is at http://www.itksnap.org/credits. We thank the organizers of the MICCAI BRATS 2012 and 2013 challenges for providing this valuable public dataset, and specifically Prof. Dr. Bjoern Menze at TU München for his advice and assistance with regard to BRATS reference data evaluations.

References

  1. Abramoff, M., Magelhaes, P., Ram, S. (2004). Image processing with ImageJ. Biophotonics International, 11(7), 36–42.Google Scholar
  2. Arthur, D., & Vassilvitskii, S. (2007). K-means+ +: the advantages of careful seeding. In Proceedings of the eighteenth annual ACM-SIAM symposium on Discrete algorithms (pp. 1027–1035). Society for Industrial and Applied Mathematics.Google Scholar
  3. Ashburner, J. (2009). Computational anatomy with the SPM software. Magnetic Resonance Imaging, 27(8), 1163–1174.CrossRefPubMedGoogle Scholar
  4. Barboriak, D.P., Padua, A.O., York, G.E., Macfall, J.R. (2005). Creation of DICOM–aware applications using ImageJ. Journal of Digital Imaging, 18(2), 91–99.CrossRefPubMedPubMedCentralGoogle Scholar
  5. Bland, J., & Altman, D. (2007). Agreement between methods of measurement with multiple observations per individual. Journal of Biopharmaceutical Statistics, 17(4), 571–582.CrossRefPubMedGoogle Scholar
  6. Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32.CrossRefGoogle Scholar
  7. Caselles, V., Catte, F., Coll, T., Dibos, F. (1993). A geometric model for active contours. Numerische Mathematik, 66, 1–31.CrossRefGoogle Scholar
  8. Caselles, V., Kimmel, R., Sapiro, G. (1997). Geodesic active contours. International Journal of Computer Vision, 22, 61–79.CrossRefGoogle Scholar
  9. Collins, S.L., Stevenson, G.N., Noble, J.A., Impey, L. (2013). Rapid calculation of standardized placental volume at 11 to 13 weeks and the prediction of small for gestational age babies. Ultrasound in Medicine and Biology, 39(2), 253–260.CrossRefPubMedGoogle Scholar
  10. Criminisi, A., Shotton, J., Konukoglu, E. (2012). Decision forests: a unified framework for classification, regression, density estimation, manifold learning and semi-supervised learning. Foundations and Trends in Computer Graphics and Vision, 7(2–3), 81–227.Google Scholar
  11. Dempster, A.P., Laird, N.M., Rubin, D.B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society. Series B (Methodological), 39(1), 1–38.Google Scholar
  12. Dice, L.R. (1945). Measures of the amount of ecologic association between species. Ecology, 26(3), 297–302.CrossRefGoogle Scholar
  13. Duncan, J.S., Papademetris, X., Yang, J., Jackowski, M., Zeng, X., Staib, L.H. (2004). Geometric strategies for neuroanatomic analysis from MRI. Neuroimage, 23(Suppl 1), S34–S45.CrossRefPubMedPubMedCentralGoogle Scholar
  14. Egger, J., Kapur, T., Fedorov, A., Pieper, S., Miller, J.V., Veeraraghavan, H., Freisleben, B., Golby, A.J., Nimsky, C., Kikinis, R. (2013). GBM Volumetry using the 3D slicer medical image computing platform. Science Reports, 3, 1364.CrossRefGoogle Scholar
  15. Fedorov, A., Beichel, R., Kalpathy-Cramer, J., Finet, J., Fillion-Robin, J.-C., Pujol, S., Bauer, C., Jennings, D., Fennessy, F., Sonka, M., Buatti, J., Aylward, S., Miller, J.V., Pieper, S., Kikinis, R. (2012). 3D slicer as an image computing platform for the Quantitative Imaging Network. Magnetic Resonance Imaging, 30(9), 1323–1341.CrossRefPubMedPubMedCentralGoogle Scholar
  16. Fischl, B., Salat, D.H., Busa, E., Albert, M., Dieterich, M., Haselgrove, C., van der Kouwe, A., Killiany, R., Kennedy, D., Klaveness, S., Montillo, A., Makris, N., Rosen, B., Dale, A.M. (2002). Whole brain segmentation: automated labeling of neuroanatomical structures in the human brain. Neuron, 33(3), 341–355.CrossRefPubMedGoogle Scholar
  17. Free Software Foundation. (2007). GNU General Public License, version 3. http://www.gnu.org/licenses/gpl.html. Accessed 25 March 2017.
  18. Gao, Y., Kikinis, R., Bouix, S., Shenton, M., Tannenbaum, A. (2012). A 3D interactive multi-object segmentation tool using local robust statistics driven active contours. Medical Image Analysis, 16(6), 1216–1227.CrossRefPubMedPubMedCentralGoogle Scholar
  19. Gering, D., Nabavi, A., Kikinis, R., Hata, N., O’Donnell, L., Grimson, W.E.L., Jolesz, F., Black, P., Wells, W. III. (2001). An integrated visualization system for surgical planning and guidance using image fusion and an open MR. Journal of Magnetic Resonance Imaging, 13, 967–975.CrossRefPubMedGoogle Scholar
  20. Heimann, T., & Meinzer, H.-P. (2009). Statistical shape models for 3D medical image segmentation: a review. Medical Image Analysis, 13(4), 543–563.CrossRefPubMedGoogle Scholar
  21. Iglesias, J.E., & Sabuncu, M.R. (2015). Multi-atlas segmentation of biomedical images: a survey. Medical Image Analysis, 24(1), 205–219.CrossRefPubMedPubMedCentralGoogle Scholar
  22. Jakab, A. (2012). Segmenting brain tumors with the Slicer 3D software. Tech. rep., Technical Report.Google Scholar
  23. Litjens, G., Kooi, T., Bejnordi, B.E., Setio, A.A.A., Ciompi, F., Ghafoorian, M., van der Laak, J.A.W.M., van Ginneken, B., Sánchez, C.I. (2017). A survey on deep learning in medical image analysis. Medical Image Analysis, 42, 60–88.CrossRefPubMedGoogle Scholar
  24. McAuliffe, M.J., Lalonde, F.M., McGarry, D., Gandler, W., Csaky, K., Trus, B.L. (2001). Medical image processing, analysis & visualization in clinical research. In CBMS ’01: proceedings of the fourteenth IEEE symposium on computer-based medical systems (p. 381). Washington: IEEE Computer Society.Google Scholar
  25. Menze, B.H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., Burren, Y., Porz, N., Slotboom, J., Wiest, R., Lanczi, L., Gerstner, E., Weber, M.-A., Arbel, T., Avants, B.B., Ayache, N., Buendia, P., Collins, D.L., Cordier, N., Corso, J.J., Criminisi, A., Das, T., Delingette, H., Demiralp, Ç., Durst, C.R., Dojat, M., Doyle, S., Festa, J., Forbes, F., Geremia, E., Glocker, B., Golland, P., Guo, X., Hamamci, A., Iftekharuddin, K.M., Jena, R., John, N.M., Konukoglu, E., Lashkari, D., Mariz, J.A., Meier, R., Pereira, S., Precup, D., Price, S.J., Raviv, T.R., Reza, S.M.S., Ryan, M., Sarikaya, D., Schwartz, L., Shin, H.-C., Shotton, J., Silva, C.A., Sousa, N., Subbanna, N.K., Szekely, G., Taylor, T.J., Thomas, O.M., Tustison, N.J., Unal, G., Vasseur, F., Wintermark, M., Ye, D.H., Zhao, L., Zhao, B., Zikic, D., Prastawa, M., Reyes, M., Van Leemput, K. (2015). The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Transactions on Medical Imaging, 34(10), 1993–2024.CrossRefPubMedGoogle Scholar
  26. Oguz, I., Pouch, A.M., Yushkevich, N., Wang, H., Gee, J.C., Schwartz, N., Yushkevich, P.A. (2016). Automated placenta segmentation from 3D ultrasound images. In MICCAI workshop on perinatal, preterm and paediatric image analysis (PIPPI).Google Scholar
  27. Pohl, K.M., Bouix, S., Nakamura, M., Rohlfing, T., McCarley, R.W., Kikinis, R., Grimson, W.E.L., Shenton, M.E., Wells, W.M. (2007). A hierarchical algorithm for MR brain image parcellation. IEEE Transactions on Medical Imaging, 26(9), 1201–1212.CrossRefPubMedPubMedCentralGoogle Scholar
  28. Sethian, J.A. (1999). Level set methods and fast marching methods. Cambridge: Cambridge University Press.Google Scholar
  29. Shen, D., Wu, G., Suk, H.-I. (2017). Deep learning in medical image analysis. Annual Review of Biomedical Engineering, 19, 221–248.CrossRefPubMedPubMedCentralGoogle Scholar
  30. Shrout, P., & Fleiss, J. (1979). Intraclass correlations: uses in assessing rater reliability. Psychological Bulletin, 86, 420–428.CrossRefPubMedGoogle Scholar
  31. Smith, S.M., Jenkinson, M., Woolrich, M.W., Beckmann, C.F., Behrens, T.E.J., Johansen-Berg, H., Bannister, P.R., Luca, M.D., Drobnjak, I., Flitney, D.E., Niazy, R.K., Saunders, J., Vickers, J., Zhang, Y., Stefano, N.D., Brady, J.M., Matthews, P.M. (2004). Advances in functional and structural MR image analysis and implementation as FSL. Neuroimage, 23(Suppl 1), S208–S219.CrossRefPubMedGoogle Scholar
  32. Sommer, C., Straehle, C., Kothe, U., Hamprecht, F.A. (2011). ilastik: interactive learning and segmentation toolkit. In 2011 IEEE international symposium on Biomedical imaging: from nano to macro (pp. 230–233). IEEE.Google Scholar
  33. Stevenson, G.N., Collins, S.L., Ding, J., Impey, L., Noble, J.A. (2015). 3-D ultrasound segmentation of the placenta using the random walker algorithm: reliability and agreement. Ultrasound in Medicine and Biology, 41(12), 3182–3193.CrossRefPubMedGoogle Scholar
  34. Whitaker, R.T. (1998). A level-set approach to 3D reconstruction from range data. International Journal of Computer Vision, 29(3), 203–231.CrossRefGoogle Scholar
  35. Yushkevich, P.A., Piven, J., Hazlett, H.C., Smith, R.G., Ho, S., Gee, J.C., Gerig, G. (2006). User-guided 3D active contour segmentation of anatomical structures: significantly improved efficiency and reliability. NeuroImage, 31(3), 1116–1128.CrossRefPubMedGoogle Scholar
  36. Zhu, S., & Yuille, A. (1995). Region competition: unifying snakes, region growing, and Bayes/MDL for multi-band image segmentation. In International conference on computer vision (ICCV’95) (pp. 416–423). citeseer.nj.nec.com/zhu95region.html.
  37. Zhu, S.C., & Yuille, A. (1996). Region competition: Unifying snakes, region growing, and bayes/MDL for multiband image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(9), 884–900.CrossRefGoogle Scholar
  38. Zhu, L., Kolesov, I., Gao, Y., Kikinis, R., Tannenbaum, A. (2014). An effective interactive medical image segmentation method using fast growcut. In MICCAI workshop on interactive medical image computing.Google Scholar
  39. Zukić, D., McCormick, M., Gerig, G., Yushkevich, P. (2016a). RLEImage: run-length encoded memory compression scheme for an itk::Image. Insight Journal (published online). http://hdl.handle.net/10380/3562.
  40. Zukić, D., Vicory, J., McCormick, M., Wisse, L., Gerig, G., Yushkevich, P., Aylward, S. (2016b). ND morphological contour interpolation. Insight Journal (published online). http://hdl.handle.net/10380/3563.

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Paul A. Yushkevich
    • 1
  • Artem Pashchinskiy
    • 1
  • Ipek Oguz
    • 1
  • Suyash Mohan
    • 1
  • J. Eric Schmitt
    • 1
  • Joel M. Stein
    • 1
  • Dženan Zukić
    • 2
  • Jared Vicory
    • 2
  • Matthew McCormick
    • 2
  • Natalie Yushkevich
    • 1
  • Nadav Schwartz
    • 3
  • Yang Gao
    • 4
  • Guido Gerig
    • 5
  1. 1.Department of RadiologyUniversity of PennsylvaniaPhiladelphiaUSA
  2. 2.Kitware, IncCarrboroUSA
  3. 3.Department of Obstetrics and GynecologyUniversity of PennsylvaniaPhiladelphiaUSA
  4. 4.Department of Computer ScienceUniversity of UtahSalt Lake CityUSA
  5. 5.Department of Computer Science and EngineeringNYU Tandon School of EngineeringNew YorkUSA

Personalised recommendations