Abstract
ITK-SNAP is an interactive software tool for manual and semi-automatic segmentation of 3D medical images. This paper summarizes major new features added to ITK-SNAP over the last decade. The main focus of the paper is on new features that support semi-automatic segmentation of multi-modality imaging datasets, such as MRI scans acquired using different contrast mechanisms (e.g., T1, T2, FLAIR). The new functionality uses decision forest classifiers trained interactively by the user to transform multiple input image volumes into a foreground/background probability map; this map is then input as the data term to the active contour evolution algorithm, which yields regularized surface representations of the segmented objects of interest. The new functionality is evaluated in the context of high-grade and low-grade glioma segmentation by three expert neuroradiogists and a non-expert on a reference dataset from the MICCAI 2013 Multi-Modal Brain Tumor Segmentation Challenge (BRATS). The accuracy of semi-automatic segmentation is competitive with the top specialized brain tumor segmentation methods evaluated in the BRATS challenge, with most results obtained in ITK-SNAP being more accurate, relative to the BRATS reference manual segmentation, than the second-best performer in the BRATS challenge; and all results being more accurate than the fourth-best performer. Segmentation time is reduced over manual segmentation by 2.5 and 5 times, depending on the rater. Additional experiments in interactive placenta segmentation in 3D fetal ultrasound illustrate the generalizability of the new functionality to a different problem domain.
Similar content being viewed by others
Notes
The number of articles citing the original ITK-SNAP paper (Yushkevich et al. 2006) reported by Scopus (scopus.com) as of 3/12/2018 was 1944; the number of citations reported by Google Scholar (scholar.google.com) was 2822.
Note that active contour segmentations of individiual structures generated in the semi-automatic mode can be exported in a way that retains partial volume information.
References
Abramoff, M., Magelhaes, P., Ram, S. (2004). Image processing with ImageJ. Biophotonics International, 11(7), 36–42.
Arthur, D., & Vassilvitskii, S. (2007). K-means+ +: the advantages of careful seeding. In Proceedings of the eighteenth annual ACM-SIAM symposium on Discrete algorithms (pp. 1027–1035). Society for Industrial and Applied Mathematics.
Ashburner, J. (2009). Computational anatomy with the SPM software. Magnetic Resonance Imaging, 27(8), 1163–1174.
Barboriak, D.P., Padua, A.O., York, G.E., Macfall, J.R. (2005). Creation of DICOM–aware applications using ImageJ. Journal of Digital Imaging, 18(2), 91–99.
Bland, J., & Altman, D. (2007). Agreement between methods of measurement with multiple observations per individual. Journal of Biopharmaceutical Statistics, 17(4), 571–582.
Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32.
Caselles, V., Catte, F., Coll, T., Dibos, F. (1993). A geometric model for active contours. Numerische Mathematik, 66, 1–31.
Caselles, V., Kimmel, R., Sapiro, G. (1997). Geodesic active contours. International Journal of Computer Vision, 22, 61–79.
Collins, S.L., Stevenson, G.N., Noble, J.A., Impey, L. (2013). Rapid calculation of standardized placental volume at 11 to 13 weeks and the prediction of small for gestational age babies. Ultrasound in Medicine and Biology, 39(2), 253–260.
Criminisi, A., Shotton, J., Konukoglu, E. (2012). Decision forests: a unified framework for classification, regression, density estimation, manifold learning and semi-supervised learning. Foundations and Trends in Computer Graphics and Vision, 7(2–3), 81–227.
Dempster, A.P., Laird, N.M., Rubin, D.B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society. Series B (Methodological), 39(1), 1–38.
Dice, L.R. (1945). Measures of the amount of ecologic association between species. Ecology, 26(3), 297–302.
Duncan, J.S., Papademetris, X., Yang, J., Jackowski, M., Zeng, X., Staib, L.H. (2004). Geometric strategies for neuroanatomic analysis from MRI. Neuroimage, 23(Suppl 1), S34–S45.
Egger, J., Kapur, T., Fedorov, A., Pieper, S., Miller, J.V., Veeraraghavan, H., Freisleben, B., Golby, A.J., Nimsky, C., Kikinis, R. (2013). GBM Volumetry using the 3D slicer medical image computing platform. Science Reports, 3, 1364.
Fedorov, A., Beichel, R., Kalpathy-Cramer, J., Finet, J., Fillion-Robin, J.-C., Pujol, S., Bauer, C., Jennings, D., Fennessy, F., Sonka, M., Buatti, J., Aylward, S., Miller, J.V., Pieper, S., Kikinis, R. (2012). 3D slicer as an image computing platform for the Quantitative Imaging Network. Magnetic Resonance Imaging, 30(9), 1323–1341.
Fischl, B., Salat, D.H., Busa, E., Albert, M., Dieterich, M., Haselgrove, C., van der Kouwe, A., Killiany, R., Kennedy, D., Klaveness, S., Montillo, A., Makris, N., Rosen, B., Dale, A.M. (2002). Whole brain segmentation: automated labeling of neuroanatomical structures in the human brain. Neuron, 33(3), 341–355.
Free Software Foundation. (2007). GNU General Public License, version 3. http://www.gnu.org/licenses/gpl.html. Accessed 25 March 2017.
Gao, Y., Kikinis, R., Bouix, S., Shenton, M., Tannenbaum, A. (2012). A 3D interactive multi-object segmentation tool using local robust statistics driven active contours. Medical Image Analysis, 16(6), 1216–1227.
Gering, D., Nabavi, A., Kikinis, R., Hata, N., O’Donnell, L., Grimson, W.E.L., Jolesz, F., Black, P., Wells, W. III. (2001). An integrated visualization system for surgical planning and guidance using image fusion and an open MR. Journal of Magnetic Resonance Imaging, 13, 967–975.
Heimann, T., & Meinzer, H.-P. (2009). Statistical shape models for 3D medical image segmentation: a review. Medical Image Analysis, 13(4), 543–563.
Iglesias, J.E., & Sabuncu, M.R. (2015). Multi-atlas segmentation of biomedical images: a survey. Medical Image Analysis, 24(1), 205–219.
Jakab, A. (2012). Segmenting brain tumors with the Slicer 3D software. Tech. rep., Technical Report.
Litjens, G., Kooi, T., Bejnordi, B.E., Setio, A.A.A., Ciompi, F., Ghafoorian, M., van der Laak, J.A.W.M., van Ginneken, B., Sánchez, C.I. (2017). A survey on deep learning in medical image analysis. Medical Image Analysis, 42, 60–88.
McAuliffe, M.J., Lalonde, F.M., McGarry, D., Gandler, W., Csaky, K., Trus, B.L. (2001). Medical image processing, analysis & visualization in clinical research. In CBMS ’01: proceedings of the fourteenth IEEE symposium on computer-based medical systems (p. 381). Washington: IEEE Computer Society.
Menze, B.H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., Burren, Y., Porz, N., Slotboom, J., Wiest, R., Lanczi, L., Gerstner, E., Weber, M.-A., Arbel, T., Avants, B.B., Ayache, N., Buendia, P., Collins, D.L., Cordier, N., Corso, J.J., Criminisi, A., Das, T., Delingette, H., Demiralp, Ç., Durst, C.R., Dojat, M., Doyle, S., Festa, J., Forbes, F., Geremia, E., Glocker, B., Golland, P., Guo, X., Hamamci, A., Iftekharuddin, K.M., Jena, R., John, N.M., Konukoglu, E., Lashkari, D., Mariz, J.A., Meier, R., Pereira, S., Precup, D., Price, S.J., Raviv, T.R., Reza, S.M.S., Ryan, M., Sarikaya, D., Schwartz, L., Shin, H.-C., Shotton, J., Silva, C.A., Sousa, N., Subbanna, N.K., Szekely, G., Taylor, T.J., Thomas, O.M., Tustison, N.J., Unal, G., Vasseur, F., Wintermark, M., Ye, D.H., Zhao, L., Zhao, B., Zikic, D., Prastawa, M., Reyes, M., Van Leemput, K. (2015). The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Transactions on Medical Imaging, 34(10), 1993–2024.
Oguz, I., Pouch, A.M., Yushkevich, N., Wang, H., Gee, J.C., Schwartz, N., Yushkevich, P.A. (2016). Automated placenta segmentation from 3D ultrasound images. In MICCAI workshop on perinatal, preterm and paediatric image analysis (PIPPI).
Pohl, K.M., Bouix, S., Nakamura, M., Rohlfing, T., McCarley, R.W., Kikinis, R., Grimson, W.E.L., Shenton, M.E., Wells, W.M. (2007). A hierarchical algorithm for MR brain image parcellation. IEEE Transactions on Medical Imaging, 26(9), 1201–1212.
Sethian, J.A. (1999). Level set methods and fast marching methods. Cambridge: Cambridge University Press.
Shen, D., Wu, G., Suk, H.-I. (2017). Deep learning in medical image analysis. Annual Review of Biomedical Engineering, 19, 221–248.
Shrout, P., & Fleiss, J. (1979). Intraclass correlations: uses in assessing rater reliability. Psychological Bulletin, 86, 420–428.
Smith, S.M., Jenkinson, M., Woolrich, M.W., Beckmann, C.F., Behrens, T.E.J., Johansen-Berg, H., Bannister, P.R., Luca, M.D., Drobnjak, I., Flitney, D.E., Niazy, R.K., Saunders, J., Vickers, J., Zhang, Y., Stefano, N.D., Brady, J.M., Matthews, P.M. (2004). Advances in functional and structural MR image analysis and implementation as FSL. Neuroimage, 23(Suppl 1), S208–S219.
Sommer, C., Straehle, C., Kothe, U., Hamprecht, F.A. (2011). ilastik: interactive learning and segmentation toolkit. In 2011 IEEE international symposium on Biomedical imaging: from nano to macro (pp. 230–233). IEEE.
Stevenson, G.N., Collins, S.L., Ding, J., Impey, L., Noble, J.A. (2015). 3-D ultrasound segmentation of the placenta using the random walker algorithm: reliability and agreement. Ultrasound in Medicine and Biology, 41(12), 3182–3193.
Whitaker, R.T. (1998). A level-set approach to 3D reconstruction from range data. International Journal of Computer Vision, 29(3), 203–231.
Yushkevich, P.A., Piven, J., Hazlett, H.C., Smith, R.G., Ho, S., Gee, J.C., Gerig, G. (2006). User-guided 3D active contour segmentation of anatomical structures: significantly improved efficiency and reliability. NeuroImage, 31(3), 1116–1128.
Zhu, S., & Yuille, A. (1995). Region competition: unifying snakes, region growing, and Bayes/MDL for multi-band image segmentation. In International conference on computer vision (ICCV’95) (pp. 416–423). citeseer.nj.nec.com/zhu95region.html.
Zhu, S.C., & Yuille, A. (1996). Region competition: Unifying snakes, region growing, and bayes/MDL for multiband image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(9), 884–900.
Zhu, L., Kolesov, I., Gao, Y., Kikinis, R., Tannenbaum, A. (2014). An effective interactive medical image segmentation method using fast growcut. In MICCAI workshop on interactive medical image computing.
Zukić, D., McCormick, M., Gerig, G., Yushkevich, P. (2016a). RLEImage: run-length encoded memory compression scheme for an itk::Image. Insight Journal (published online). http://hdl.handle.net/10380/3562.
Zukić, D., Vicory, J., McCormick, M., Wisse, L., Gerig, G., Yushkevich, P., Aylward, S. (2016b). ND morphological contour interpolation. Insight Journal (published online). http://hdl.handle.net/10380/3563.
Acknowledgements
This work was supported by NIH grants R01 EB014346, R01 EB017255, K01 ES026840, and U01 HD087180. We gratefully acknowledge the many researchers and developers who contributed to ITK-SNAP software over the past 20 years. A full list of contributors is at http://www.itksnap.org/credits. We thank the organizers of the MICCAI BRATS 2012 and 2013 challenges for providing this valuable public dataset, and specifically Prof. Dr. Bjoern Menze at TU München for his advice and assistance with regard to BRATS reference data evaluations.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Yushkevich, P.A., Pashchinskiy, A., Oguz, I. et al. User-Guided Segmentation of Multi-modality Medical Imaging Datasets with ITK-SNAP. Neuroinform 17, 83–102 (2019). https://doi.org/10.1007/s12021-018-9385-x
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12021-018-9385-x