Skip to main content

User-Guided Segmentation of Multi-modality Medical Imaging Datasets with ITK-SNAP

Neuroinformatics volume 17, pages 83–102 (2019)Cite this article

Abstract

ITK-SNAP is an interactive software tool for manual and semi-automatic segmentation of 3D medical images. This paper summarizes major new features added to ITK-SNAP over the last decade. The main focus of the paper is on new features that support semi-automatic segmentation of multi-modality imaging datasets, such as MRI scans acquired using different contrast mechanisms (e.g., T1, T2, FLAIR). The new functionality uses decision forest classifiers trained interactively by the user to transform multiple input image volumes into a foreground/background probability map; this map is then input as the data term to the active contour evolution algorithm, which yields regularized surface representations of the segmented objects of interest. The new functionality is evaluated in the context of high-grade and low-grade glioma segmentation by three expert neuroradiogists and a non-expert on a reference dataset from the MICCAI 2013 Multi-Modal Brain Tumor Segmentation Challenge (BRATS). The accuracy of semi-automatic segmentation is competitive with the top specialized brain tumor segmentation methods evaluated in the BRATS challenge, with most results obtained in ITK-SNAP being more accurate, relative to the BRATS reference manual segmentation, than the second-best performer in the BRATS challenge; and all results being more accurate than the fourth-best performer. Segmentation time is reduced over manual segmentation by 2.5 and 5 times, depending on the rater. Additional experiments in interactive placenta segmentation in 3D fetal ultrasound illustrate the generalizability of the new functionality to a different problem domain.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

43,34 €

Price includes VAT (Finland)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Notes

  1. The number of articles citing the original ITK-SNAP paper (Yushkevich et al. 2006) reported by Scopus (scopus.com) as of 3/12/2018 was 1944; the number of citations reported by Google Scholar (scholar.google.com) was 2822.

  2. Note that active contour segmentations of individiual structures generated in the semi-automatic mode can be exported in a way that retains partial volume information.

  3. https://www.slicer.org/wiki/Documentation/4.8/Modules/SimpleRegionGrowingSegmentation

  4. https://www.slicer.org/wiki/Documentation/4.4/Extensions/Wasp

  5. https://www.slicer.org/wiki/Documentation/Nightly/Extensions/SegmentationWizard

References

  • Abramoff, M., Magelhaes, P., Ram, S. (2004). Image processing with ImageJ. Biophotonics International, 11(7), 36–42.

    Google Scholar 

  • Arthur, D., & Vassilvitskii, S. (2007). K-means+ +: the advantages of careful seeding. In Proceedings of the eighteenth annual ACM-SIAM symposium on Discrete algorithms (pp. 1027–1035). Society for Industrial and Applied Mathematics.

  • Ashburner, J. (2009). Computational anatomy with the SPM software. Magnetic Resonance Imaging, 27(8), 1163–1174.

    Article  Google Scholar 

  • Barboriak, D.P., Padua, A.O., York, G.E., Macfall, J.R. (2005). Creation of DICOM–aware applications using ImageJ. Journal of Digital Imaging, 18(2), 91–99.

    Article  Google Scholar 

  • Bland, J., & Altman, D. (2007). Agreement between methods of measurement with multiple observations per individual. Journal of Biopharmaceutical Statistics, 17(4), 571–582.

    Article  Google Scholar 

  • Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32.

    Article  Google Scholar 

  • Caselles, V., Catte, F., Coll, T., Dibos, F. (1993). A geometric model for active contours. Numerische Mathematik, 66, 1–31.

    Article  Google Scholar 

  • Caselles, V., Kimmel, R., Sapiro, G. (1997). Geodesic active contours. International Journal of Computer Vision, 22, 61–79.

    Article  Google Scholar 

  • Collins, S.L., Stevenson, G.N., Noble, J.A., Impey, L. (2013). Rapid calculation of standardized placental volume at 11 to 13 weeks and the prediction of small for gestational age babies. Ultrasound in Medicine and Biology, 39(2), 253–260.

    Article  Google Scholar 

  • Criminisi, A., Shotton, J., Konukoglu, E. (2012). Decision forests: a unified framework for classification, regression, density estimation, manifold learning and semi-supervised learning. Foundations and Trends in Computer Graphics and Vision, 7(2–3), 81–227.

    Google Scholar 

  • Dempster, A.P., Laird, N.M., Rubin, D.B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society. Series B (Methodological), 39(1), 1–38.

    Article  Google Scholar 

  • Dice, L.R. (1945). Measures of the amount of ecologic association between species. Ecology, 26(3), 297–302.

    Article  Google Scholar 

  • Duncan, J.S., Papademetris, X., Yang, J., Jackowski, M., Zeng, X., Staib, L.H. (2004). Geometric strategies for neuroanatomic analysis from MRI. Neuroimage, 23(Suppl 1), S34–S45.

    Article  Google Scholar 

  • Egger, J., Kapur, T., Fedorov, A., Pieper, S., Miller, J.V., Veeraraghavan, H., Freisleben, B., Golby, A.J., Nimsky, C., Kikinis, R. (2013). GBM Volumetry using the 3D slicer medical image computing platform. Science Reports, 3, 1364.

    CAS  Article  Google Scholar 

  • Fedorov, A., Beichel, R., Kalpathy-Cramer, J., Finet, J., Fillion-Robin, J.-C., Pujol, S., Bauer, C., Jennings, D., Fennessy, F., Sonka, M., Buatti, J., Aylward, S., Miller, J.V., Pieper, S., Kikinis, R. (2012). 3D slicer as an image computing platform for the Quantitative Imaging Network. Magnetic Resonance Imaging, 30(9), 1323–1341.

    Article  Google Scholar 

  • Fischl, B., Salat, D.H., Busa, E., Albert, M., Dieterich, M., Haselgrove, C., van der Kouwe, A., Killiany, R., Kennedy, D., Klaveness, S., Montillo, A., Makris, N., Rosen, B., Dale, A.M. (2002). Whole brain segmentation: automated labeling of neuroanatomical structures in the human brain. Neuron, 33(3), 341–355.

    CAS  Article  Google Scholar 

  • Free Software Foundation. (2007). GNU General Public License, version 3. http://www.gnu.org/licenses/gpl.html. Accessed 25 March 2017.

  • Gao, Y., Kikinis, R., Bouix, S., Shenton, M., Tannenbaum, A. (2012). A 3D interactive multi-object segmentation tool using local robust statistics driven active contours. Medical Image Analysis, 16(6), 1216–1227.

    Article  Google Scholar 

  • Gering, D., Nabavi, A., Kikinis, R., Hata, N., O’Donnell, L., Grimson, W.E.L., Jolesz, F., Black, P., Wells, W. III. (2001). An integrated visualization system for surgical planning and guidance using image fusion and an open MR. Journal of Magnetic Resonance Imaging, 13, 967–975.

    CAS  Article  Google Scholar 

  • Heimann, T., & Meinzer, H.-P. (2009). Statistical shape models for 3D medical image segmentation: a review. Medical Image Analysis, 13(4), 543–563.

    Article  Google Scholar 

  • Iglesias, J.E., & Sabuncu, M.R. (2015). Multi-atlas segmentation of biomedical images: a survey. Medical Image Analysis, 24(1), 205–219.

    Article  Google Scholar 

  • Jakab, A. (2012). Segmenting brain tumors with the Slicer 3D software. Tech. rep., Technical Report.

  • Litjens, G., Kooi, T., Bejnordi, B.E., Setio, A.A.A., Ciompi, F., Ghafoorian, M., van der Laak, J.A.W.M., van Ginneken, B., Sánchez, C.I. (2017). A survey on deep learning in medical image analysis. Medical Image Analysis, 42, 60–88.

    Article  Google Scholar 

  • McAuliffe, M.J., Lalonde, F.M., McGarry, D., Gandler, W., Csaky, K., Trus, B.L. (2001). Medical image processing, analysis & visualization in clinical research. In CBMS ’01: proceedings of the fourteenth IEEE symposium on computer-based medical systems (p. 381). Washington: IEEE Computer Society.

  • Menze, B.H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., Burren, Y., Porz, N., Slotboom, J., Wiest, R., Lanczi, L., Gerstner, E., Weber, M.-A., Arbel, T., Avants, B.B., Ayache, N., Buendia, P., Collins, D.L., Cordier, N., Corso, J.J., Criminisi, A., Das, T., Delingette, H., Demiralp, Ç., Durst, C.R., Dojat, M., Doyle, S., Festa, J., Forbes, F., Geremia, E., Glocker, B., Golland, P., Guo, X., Hamamci, A., Iftekharuddin, K.M., Jena, R., John, N.M., Konukoglu, E., Lashkari, D., Mariz, J.A., Meier, R., Pereira, S., Precup, D., Price, S.J., Raviv, T.R., Reza, S.M.S., Ryan, M., Sarikaya, D., Schwartz, L., Shin, H.-C., Shotton, J., Silva, C.A., Sousa, N., Subbanna, N.K., Szekely, G., Taylor, T.J., Thomas, O.M., Tustison, N.J., Unal, G., Vasseur, F., Wintermark, M., Ye, D.H., Zhao, L., Zhao, B., Zikic, D., Prastawa, M., Reyes, M., Van Leemput, K. (2015). The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Transactions on Medical Imaging, 34(10), 1993–2024.

    Article  Google Scholar 

  • Oguz, I., Pouch, A.M., Yushkevich, N., Wang, H., Gee, J.C., Schwartz, N., Yushkevich, P.A. (2016). Automated placenta segmentation from 3D ultrasound images. In MICCAI workshop on perinatal, preterm and paediatric image analysis (PIPPI).

  • Pohl, K.M., Bouix, S., Nakamura, M., Rohlfing, T., McCarley, R.W., Kikinis, R., Grimson, W.E.L., Shenton, M.E., Wells, W.M. (2007). A hierarchical algorithm for MR brain image parcellation. IEEE Transactions on Medical Imaging, 26(9), 1201–1212.

    Article  Google Scholar 

  • Sethian, J.A. (1999). Level set methods and fast marching methods. Cambridge: Cambridge University Press.

    Google Scholar 

  • Shen, D., Wu, G., Suk, H.-I. (2017). Deep learning in medical image analysis. Annual Review of Biomedical Engineering, 19, 221–248.

    CAS  Article  Google Scholar 

  • Shrout, P., & Fleiss, J. (1979). Intraclass correlations: uses in assessing rater reliability. Psychological Bulletin, 86, 420–428.

    CAS  Article  Google Scholar 

  • Smith, S.M., Jenkinson, M., Woolrich, M.W., Beckmann, C.F., Behrens, T.E.J., Johansen-Berg, H., Bannister, P.R., Luca, M.D., Drobnjak, I., Flitney, D.E., Niazy, R.K., Saunders, J., Vickers, J., Zhang, Y., Stefano, N.D., Brady, J.M., Matthews, P.M. (2004). Advances in functional and structural MR image analysis and implementation as FSL. Neuroimage, 23(Suppl 1), S208–S219.

    Article  Google Scholar 

  • Sommer, C., Straehle, C., Kothe, U., Hamprecht, F.A. (2011). ilastik: interactive learning and segmentation toolkit. In 2011 IEEE international symposium on Biomedical imaging: from nano to macro (pp. 230–233). IEEE.

  • Stevenson, G.N., Collins, S.L., Ding, J., Impey, L., Noble, J.A. (2015). 3-D ultrasound segmentation of the placenta using the random walker algorithm: reliability and agreement. Ultrasound in Medicine and Biology, 41(12), 3182–3193.

    Article  Google Scholar 

  • Whitaker, R.T. (1998). A level-set approach to 3D reconstruction from range data. International Journal of Computer Vision, 29(3), 203–231.

    Article  Google Scholar 

  • Yushkevich, P.A., Piven, J., Hazlett, H.C., Smith, R.G., Ho, S., Gee, J.C., Gerig, G. (2006). User-guided 3D active contour segmentation of anatomical structures: significantly improved efficiency and reliability. NeuroImage, 31(3), 1116–1128.

    Article  Google Scholar 

  • Zhu, S., & Yuille, A. (1995). Region competition: unifying snakes, region growing, and Bayes/MDL for multi-band image segmentation. In International conference on computer vision (ICCV’95) (pp. 416–423). citeseer.nj.nec.com/zhu95region.html.

  • Zhu, S.C., & Yuille, A. (1996). Region competition: Unifying snakes, region growing, and bayes/MDL for multiband image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(9), 884–900.

    Article  Google Scholar 

  • Zhu, L., Kolesov, I., Gao, Y., Kikinis, R., Tannenbaum, A. (2014). An effective interactive medical image segmentation method using fast growcut. In MICCAI workshop on interactive medical image computing.

  • Zukić, D., McCormick, M., Gerig, G., Yushkevich, P. (2016a). RLEImage: run-length encoded memory compression scheme for an itk::Image. Insight Journal (published online). http://hdl.handle.net/10380/3562.

  • Zukić, D., Vicory, J., McCormick, M., Wisse, L., Gerig, G., Yushkevich, P., Aylward, S. (2016b). ND morphological contour interpolation. Insight Journal (published online). http://hdl.handle.net/10380/3563.

Download references

Acknowledgements

This work was supported by NIH grants R01 EB014346, R01 EB017255, K01 ES026840, and U01 HD087180. We gratefully acknowledge the many researchers and developers who contributed to ITK-SNAP software over the past 20 years. A full list of contributors is at http://www.itksnap.org/credits. We thank the organizers of the MICCAI BRATS 2012 and 2013 challenges for providing this valuable public dataset, and specifically Prof. Dr. Bjoern Menze at TU München for his advice and assistance with regard to BRATS reference data evaluations.

Author information

Affiliations

  1. Department of Radiology, University of Pennsylvania, Philadelphia, PA, USA

    Paul A. Yushkevich, Artem Pashchinskiy, Ipek Oguz, Suyash Mohan, J. Eric Schmitt, Joel M. Stein & Natalie Yushkevich

  2. Kitware, Inc, Carrboro, NC, USA

    Dženan Zukić, Jared Vicory & Matthew McCormick

  3. Department of Obstetrics and Gynecology, University of Pennsylvania, Philadelphia, PA, USA

    Nadav Schwartz

  4. Department of Computer Science, University of Utah, Salt Lake City, UT, USA

    Yang Gao

  5. Department of Computer Science and Engineering, NYU Tandon School of Engineering, New York, NY, USA

    Guido Gerig

Authors
  1. Paul A. Yushkevich
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Artem Pashchinskiy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ipek Oguz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Suyash Mohan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. J. Eric Schmitt
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Joel M. Stein
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Dženan Zukić
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jared Vicory
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Matthew McCormick
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Natalie Yushkevich
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Nadav Schwartz
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Yang Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Guido Gerig
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paul A. Yushkevich.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Yushkevich, P.A., Pashchinskiy, A., Oguz, I. et al. User-Guided Segmentation of Multi-modality Medical Imaging Datasets with ITK-SNAP. Neuroinform 17, 83–102 (2019). https://doi.org/10.1007/s12021-018-9385-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12021-018-9385-x

Keywords

  • Image segmentation
  • Semi-automatic segmentation
  • Gliomas
  • Software
  • MRI