Skip to main content
Log in

FCN Based Label Correction for Multi-Atlas Guided Organ Segmentation

  • Original Article
  • Published:
Neuroinformatics Aims and scope Submit manuscript

Abstract

Segmentation of medical images using multiple atlases has recently gained immense attention due to their augmented robustness against variabilities across different subjects. These atlas-based methods typically comprise of three steps: atlas selection, image registration, and finally label fusion. Image registration is one of the core steps in this process, accuracy of which directly affects the final labeling performance. However, due to inter-subject anatomical variations, registration errors are inevitable. The aim of this paper is to develop a deep learning-based confidence estimation method to alleviate the potential effects of registration errors. We first propose a fully convolutional network (FCN) with residual connections to learn the relationship between the image patch pair (i.e., patches from the target subject and the atlas) and the related label confidence patch. With the obtained label confidence patch, we can identify the potential errors in the warped atlas labels and correct them. Then, we use two label fusion methods to fuse the corrected atlas labels. The proposed methods are validated on a publicly available dataset for hippocampus segmentation. Experimental results demonstrate that our proposed methods outperform the state-of-the-art segmentation methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Aljabar, P., Heckemann, R., Hammers, A., Hajnal, J. V., & Rueckert, D. (2009). Multi-atlas based segmentation of brain images: Atlas selection and its effect on accuracy. NeuroImage, 46(3), 726–738.

    Article  CAS  PubMed  Google Scholar 

  • Artaechevarria, X., Muñoz-Barrutia, A., and Ortiz-de-Solorzano, C.(2008). “Efficient classifier generation and weighted voting for atlas-based segmentation: Two small steps faster and closer to the combination oracle,” SPIE Medical Imaging, 69141W–69141W-9.

  • Artaechevarria, X., Munoz-Barrutia, A., & Ortiz-de-Solorzano, C. (2009). Combination strategies in multi-atlas image segmentation: Application to brain MR data. Medical Imaging, IEEE Transactions on, 28(8), 1266–1277.

    Article  Google Scholar 

  • Asman, A. J., & Landman, B. A. (2012). Formulating spatially varying performance in the statistical fusion framework. Medical Imaging, IEEE Transactions on, 31(6), 1326–1336.

    Article  Google Scholar 

  • Asman, A. J., & Landman, B. A. (2013). Non-local statistical label fusion for multi-atlas segmentation. Medical Image Analysis, 17(2), 194–208.

    Article  PubMed  Google Scholar 

  • Asman, A. J., & Landman, B. A. (2014). Hierarchical performance estimation in the statistical label fusion framework. Medical Image Analysis, 18(7), 1070–1081.

    Article  PubMed  PubMed Central  Google Scholar 

  • Avants, B. B., Epstein, C. L., Grossman, M., & Gee, J. C. (2008). Symmetric diffeomorphic image registration with cross-correlation: Evaluating automated labeling of elderly and neurodegenerative brain. Medical Image Analysis, 12(1), 26–41.

    Article  CAS  PubMed  Google Scholar 

  • Bai, W., Shi, W., O'Regan, D. P., et al. (2013). A probabilistic patch-based label fusion model for multi-atlas segmentation with registration refinement: Application to cardiac MR images. Medical Imaging, IEEE Transactions on, 32(7), 1302–1315.

    Article  Google Scholar 

  • Bai, W., Shi, W., Ledig, C., & Rueckert, D. (2015). Multi-atlas segmentation with augmented features for cardiac MR images. Medical Image Analysis, 19(1), 98–109.

    Article  PubMed  Google Scholar 

  • Benkarim, O. M., Piella, G., Ballester, M. A. G., et al. (2017). Discriminative confidence estimation for probabilistic multi-atlas label fusion. Medical Image Analysis, 42, 274–287.

    Article  PubMed  Google Scholar 

  • Boccardi, M., Bocchetta, M., Morency, F. C., Collins, D. L., Nishikawa, M., Ganzola, R., Grothe, M. J., Wolf, D., Redolfi, A., Pievani, M., Antelmi, L., Fellgiebel, A., Matsuda, H., Teipel, S., Duchesne, S., Jack CR Jr, Frisoni, G. B., & EADC-ADNI Working Group on The Harmonized Protocol for Manual Hippocampal Segmentation and for the Alzheimer's Disease Neuroimaging Initiative. (2015). Training labels for hippocampal segmentation based on the EADC-ADNI harmonized hippocampal protocol. Alzheimers Dement, 11(2), 175–183.

    Article  PubMed  Google Scholar 

  • Cao, Y., Yuan, Y., Li, X. et al. (2011). “Segmenting images by combining selected atlases on manifold,” International Conference on Medical Image Computing and Computer-Assisted Intervention, 272–279.

  • Chen, H., Dou, Q., Yu, L., Qin, J., & Heng, P. A. (2018). VoxResNet: Deep voxelwise residual networks for brain segmentation from 3D MR images. NeuroImage, 170, 446–455.

    Article  PubMed  Google Scholar 

  • Commowick, O., Akhondi-Asl, A., & Warfield, S. K. (2012). Estimating a reference standard segmentation with spatially varying performance parameters: Local MAP STAPLE. Medical Imaging, IEEE Transactions on, 31(8), 1593–1606.

    Article  Google Scholar 

  • Coupé, P., Manjón, J. V., Fonov, V., Pruessner, J., Robles, M., & Collins, D. L. (2011). Patch-based segmentation using expert priors: Application to hippocampus and ventricle segmentation. NeuroImage, 54(2), 940–954.

    Article  PubMed  Google Scholar 

  • Doshi, J., Erus, G., Ou, Y., Resnick, S. M., Gur, R. C., Gur, R. E., Satterthwaite, T. D., Furth, S., Davatzikos, C., & Alzheimer's Neuroimaging Initiative. (2016). MUSE: MUlti-atlas region segmentation utilizing ensembles of registration algorithms and parameters, and locally optimal atlas selection. NeuroImage, 127, 186–195.

    Article  PubMed  Google Scholar 

  • Dou, Q., Yu, L., Chen, H., Jin, Y., Yang, X., Qin, J., & Heng, P. A. (2017). 3D deeply supervised network for automated segmentation of volumetric medical images. Medical Image Analysis, 41, 40–54.

    Article  PubMed  Google Scholar 

  • A. K. H. Duc, M. Modat, K. K. Leung et al., “Manifold learning for atlas selection in multi-atlas-based segmentation of hippocampus,” Medical Imaging 2012: Image Processing, 8314, 83140Z (2012).

  • Fang, L., Zhang, L., Nie, D. et al. (2017). “Brain Image Labeling Using Multi-atlas Guided 3D Fully Convolutional Networks,” International Workshop on Patch-based Techniques in Medical Imaging, 12–19.

  • Gu, J., Wang, Z., Kuen, J., et al. (2017). Recent advances in convolutional neural networks. Pattern Recognition, 77, 354–377.

    Article  Google Scholar 

  • Haber, E., & Modersitzki, J. (2004). Numerical methods for volume preserving image registration. Inverse Problems, 20(5), 1621.

    Article  Google Scholar 

  • Hao, Y., Wang, T., Zhang, X., Duan, Y., Yu, C., Jiang, T., Fan, Y., & Alzheimer's Disease Neuroimaging Initiative. (2014). Local label learning (LLL) for subcortical structure segmentation: Application to hippocampus segmentation. Human Brain Mapping, 35(6), 2674–2697.

    Article  PubMed  Google Scholar 

  • Haom, Y., Liu, J., Duan, Y. et al. (2012). “Local label learning (L3) for multi-atlas based segmentation,” SPIE Medical Imaging, 83142E-83142E-8.

  • He, K., Zhang, X., Ren, S. et al. (2016a). “Identity mappings in deep residual networks,” European Conference on Computer Vision, 630–645.

  • He, K., Zhang, X., Ren, S. et al. (2016b). “Deep residual learning for image recognition,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 770–778.

  • Heckemann, R. A., Hajnal, J. V., Aljabar, P., Rueckert, D., & Hammers, A. (2006). Automatic anatomical brain MRI segmentation combining label propagation and decision fusion. NeuroImage, 33(1), 115–126.

    Article  PubMed  Google Scholar 

  • Huang, G., Liu, Z., Van Der Maaten, L. et al. (2017). “Densely connected convolutional networks,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4700–4708.

  • Iglesias, J. E., & Sabuncu, M. R. (2015). Multi-atlas segmentation of biomedical images: A survey. Medical Image Analysis, 24(1), 205–219.

    Article  PubMed  PubMed Central  Google Scholar 

  • Jack, C. R., Bernstein, M. A., Fox, N. C., et al. (2008). The Alzheimer's disease neuroimaging initiative (ADNI): MRI methods. Journal of Magnetic Resonance Imaging, 27(4), 685–691.

    Article  PubMed  Google Scholar 

  • Jafari-Khouzani, K., Elisevich, K. V., Patel, S., & Soltanian-Zadeh, H. (2011). Dataset of magnetic resonance images of nonepileptic subjects and temporal lobe epilepsy patients for validation of hippocampal segmentation techniques. Neuroinformatics, 9(4), 335–346.

    Article  PubMed  PubMed Central  Google Scholar 

  • Jia, Y., Shelhamer, E., Donahue, J. et al. (2014). “Caffe: Convolutional architecture for fast feature embedding,” Proceedings of the 22nd ACM international conference on Multimedia, 675–678.

  • Jorge Cardoso, M., Leung, K., Modat, M., Keihaninejad, S., Cash, D., Barnes, J., Fox, N. C., Ourselin, S., & Alzheimer’s Disease Neuroimaging Initiative. (2013). STEPS: Similarity and truth estimation for propagated segmentations and its application to hippocampal segmentation and brain parcelation. Medical Image Analysis, 17(6), 671–684.

    Article  CAS  PubMed  Google Scholar 

  • Langerak, T. R., Berendsen, F. F., Van der Heide, U. A., et al. (2013). Multiatlas-based segmentation with preregistration atlas selection. Medical Physics, 40(9), 091701.

    Article  PubMed  Google Scholar 

  • Liao, S., Gao, Y., & Shen, D. (2012). Sparse patch based prostate segmentation in CT images. Medical Image Computing and Computer-Assisted Intervention–MICCAI, 2012, 385–392.

    Google Scholar 

  • Liao, S., Gao, Y., Lian, J., et al. (2013). Sparse patch-based label propagation for accurate prostate localization in CT images. Medical Imaging, IEEE Transactions on, 32(2), 419–434.

    Article  Google Scholar 

  • Long, J., Shelhamer, E., and Darrell, T. (2015). “Fully convolutional networks for semantic segmentation,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3431–3440.

  • Lötjönen, J. M. P., Wolz, R., Koikkalainen, J. R., et al. (2010). Fast and robust multi-atlas segmentation of brain magnetic resonance images. NeuroImage, 49(3), 2352–2365.

    Article  PubMed  Google Scholar 

  • Milletari, F., Navab, N., and Ahmadi, S.-A. (2016). “V-net: Fully convolutional neural networks for volumetric medical image segmentation,” 3D Vision (3DV), 2016 Fourth International Conference on, 565–571.

  • Rohlfing, T., Brandt, R., Menzel, R., & Maurer CR Jr. (2004). Evaluation of atlas selection strategies for atlas-based image segmentation with application to confocal microscopy images of bee brains. NeuroImage, 21(4), 1428–1442.

    Article  PubMed  Google Scholar 

  • Ronneberger, O., Fischer, P., and Brox, T. (2015). “U-net: Convolutional networks for biomedical image segmentation,” International Conference on Medical Image Computing and Computer-Assisted Intervention, 234–241.

  • Rousseau, F., Habas, P. A., & Studholme, C. (2011). A supervised patch-based approach for human brain labeling. Medical Imaging, IEEE Transactions on, 30(10), 1852–1862.

    Article  Google Scholar 

  • Sabuncu, M. R., Yeo, B. T. T., Van Leemput, K., et al. (2010). A generative model for image segmentation based on label fusion. Medical Imaging, IEEE Transactions on, 29(10), 1714–1729.

    Article  Google Scholar 

  • Sanroma, G., Wu, G., Gao, Y., et al. (2014). Learning to rank atlases for multiple-atlas segmentation. Medical Imaging, IEEE Transactions on, 33(10), 1939–1953.

    Article  Google Scholar 

  • Shamsolmoali, P., Zhang, J., & Yang, J. (2019). Image super resolution by dilated dense progressive network. Image and Vision Computing, 88, 9–18.

    Article  Google Scholar 

  • Wang, H., Suh, J. W., Das, S. et al. (2011). “Regression-based label fusion for multi-atlas segmentation,” Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, 1113–1120.

  • Wang, H., Suh, J. W., Das, S. R., et al. (2013). Multi-atlas segmentation with joint label fusion. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 35(3), 611–623.

    Article  Google Scholar 

  • Warfield, S. K., Zou, K. H., & Wells, W. M. (2004). Simultaneous truth and performance level estimation (STAPLE): An algorithm for the validation of image segmentation. Medical Imaging, IEEE Transactions on, 23(7), 903–921.

    Article  Google Scholar 

  • Xu, B., Ye, H., Zheng, Y., Wang, H., Luwang, T., & Jiang, Y. G. (2019). Dense dilated network for video action recognition. IEEE Transactions on Image Processing, 28(10), 4941–4953.

    Article  PubMed  Google Scholar 

  • Yang, H., Sun, J., Li, H. et al. (2017). “Neural Multi-Atlas Label Fusion: Application to Cardiac MR Images,” arXiv preprint arXiv:1709.09641.

  • Yu, F., and Koltun, V. (2015). “Multi-scale context aggregation by dilated convolutions,” arXiv preprint arXiv:1511.07122.

  • Yu, L., Yang, X., Chen, H. et al. (2017). “Volumetric ConvNets with Mixed Residual Connections for Automated Prostate Segmentation from 3D MR Images,” AAAI, 66–72.

  • Zaffino, P., Ciardo, D., Raudaschl, P., et al. (2018). Multi atlas based segmentation: Should we prefer the best atlas group over the group of best atlases? Physics in Medicine & Biology, 63(12), 12NT01.

    Article  Google Scholar 

  • Zhu, H., Cheng, H., and Fan, Y. (2015). “Random local binary pattern based label learning for multi-atlas segmentation,” SPIE Medical Imaging, 94131B-94131B-8.

  • Zhu, H., Cheng, H., Yang, X., Fan, Y., & Alzheimer’s Disease Neuroimaging Initiative. (2017). Metric learning for multi-atlas based segmentation of Hippocampus. Neuroinformatics, 15(1), 41–50.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgments

This work was supported in part by National Natural Science Foundation of China [61602307, 61877039] and Natural Science Foundation of Zhejiang Province [LY19F020013, LY20F020011].

Data collection and sharing for this project was funded by the Alzheimer’s Disease Neuroimaging Initiative (ADNI) (National Institutes of Health Grant U01 AG024904) and DOD ADNI (Department of Defense award number W81XWH-12-2-0012). ADNI is funded by the National Institute on Aging, the National Institute of Biomedical Imaging and Bioengineering, and through generous contributions from the following: AbbVie, Alzheimer’s Association; Alzheimer’s Drug Discovery Foundation; Araclon Biotech; BioClinica, Inc.; Biogen; Bristol-Myers Squibb Company; CereSpir, Inc.; Cogstate; Eisai Inc.; Elan Pharmaceuticals, Inc.; Eli Lilly and Company; EuroImmun; F. Hoffmann-La Roche Ltd. and its affiliated company Genentech, Inc.; Fujirebio; GE Healthcare; IXICO Ltd.; Janssen Alzheimer Immunotherapy Research & Development, LLC.; Johnson & Johnson Pharmaceutical Research & Development LLC.; Lumosity; Lundbeck; Merck & Co., Inc.; Meso Scale Diagnostics, LLC.; NeuroRx Research; Neurotrack Technologies; Novartis Pharmaceuticals Corporation; Pfizer Inc.; Piramal Imaging; Servier; Takeda Pharmaceutical Company; and Transition Therapeutics. The Canadian Institutes of Health Research is providing funds to support ADNI clinical sites in Canada. Private sector contributions are facilitated by the Foundation for the National Institutes of Health (www.fnih.org). The grantee organization is the Northern California Institute for Research and Education, and the study is coordinated by the Alzheimer’s Therapeutic Research Institute at the University of Southern California. ADNI data are disseminated by the Laboratory for Neuro Imaging at the University of Southern California.

Author information

Authors and Affiliations

Authors

Consortia

Corresponding author

Correspondence to Dinggang Shen.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Data used in preparation of this article were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu). As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in analysis or writing of this report. A complete listing of ADNI investigators can be found at: http://adni.loni.usc.edu/wp-content/uploads/how_to_apply/ADNI_Acknowledgement_List.pdf

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, H., Adeli, E., Shi, F. et al. FCN Based Label Correction for Multi-Atlas Guided Organ Segmentation. Neuroinform 18, 319–331 (2020). https://doi.org/10.1007/s12021-019-09448-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12021-019-09448-5

Keywords

Navigation