Patch-Based Label Fusion with Structured Discriminant Embedding for Hippocampus Segmentation

Original Article


Automatic and accurate segmentation of hippocampal structures in medical images is of great importance in neuroscience studies. In multi-atlas based segmentation methods, to alleviate the misalignment when registering atlases to the target image, patch-based methods have been widely studied to improve the performance of label fusion. However, weights assigned to the fused labels are usually computed based on predefined features (e.g. image intensities), thus being not necessarily optimal. Due to the lack of discriminating features, the original feature space defined by image intensities may limit the description accuracy. To solve this problem, we propose a patch-based label fusion with structured discriminant embedding method to automatically segment the hippocampal structure from the target image in a voxel-wise manner. Specifically, multi-scale intensity features and texture features are first extracted from the image patch for feature representation. Margin fisher analysis (MFA) is then applied to the neighboring samples in the atlases for the target voxel, in order to learn a subspace in which the distance between intra-class samples is minimized and the distance between inter-class samples is simultaneously maximized. Finally, the k-nearest neighbor (kNN) classifier is employed in the learned subspace to determine the final label for the target voxel. In the experiments, we evaluate our proposed method by conducting hippocampus segmentation using the ADNI dataset. Both the qualitative and quantitative results show that our method outperforms the conventional multi-atlas based segmentation methods.


Multi-atlas based method Margin fisher analysis Structured discriminant embedding Subspace learning Patch-based label fusion 



This work is supported in part by NSFC project 61701324, Sience&Technology Department of Sichuan Province 2016JZ0014, Open Fund Project of Fujian Provincial Key Laboratory of Information Processing and Intelligent Control (Minjiang University) (No. MJUKF201715).

Compliance with Ethical Standards

Conflict of Interest

The authors declare no conflict of interest.


  1. Carmichael, O. T., Aizenstein, H. A., Davis, S. W., Becker, J. T., Thompson, P. M., Meltzer, C. C., & Liu, Y. (2005). Atlas-based hippocampus segmentation in Alzheimer's disease and mild cognitive impairment. NeuroImage, 27(4), 979–990.CrossRefPubMedPubMedCentralGoogle Scholar
  2. Chen, Z., Jie, B., Liu, M., Chen, S., Shen, D., & Zhang, D. (2015). Label-aligned multi-task feature learning for multimodal classification of Alzheimer’s disease and mild cognitive impairment. Brain Imaging & Behavior, 1–12.Google Scholar
  3. Chen, Z., Wang, Z., Zhang, D., Liang, P., Shi, Y., Shen, D., & Wu, G. (2017). Robust multi-atlas label propagation by deep sparse representation. Pattern Recognition, 63, 511–517.CrossRefGoogle Scholar
  4. Coupé, P., Manjón, J. V., Fonov, V., Pruessner, J., Robles, M., & Collins, D. L. (2011). Patch-based segmentation using expert priors: Application to hippocampus and ventricle segmentation. NeuroImage, 54, 940–954.CrossRefPubMedGoogle Scholar
  5. Dong, P., Wang, L., Lin, W., Shen, D., & Wu, G. (2017). Scalable joint segmentation and registration framework for infant brain images. Neurocomputing, 229, 54–62.Google Scholar
  6. He, X., Lum, A., Sharma, M., Brahm, G., Mercado, A., & Li, S. (2017). Automated segmentation and area estimation of neural foramina with boundary regression model. Pattern Recognition, 63, 625–641.CrossRefGoogle Scholar
  7. Jafari-Khouzani, K., Elisevich, K. V., Patel, S., & Soltanian-Zadeh, H. (2011). Dataset of magnetic resonance images of nonepileptic subjects and temporal lobe epilepsy patients for validation of hippocampal segmentation techniques. Neuroinformatics, 9(4), 335–346.CrossRefPubMedPubMedCentralGoogle Scholar
  8. Liao, S., Gao, Y., Lian, J., & Shen, D. (2013). Sparse patch-based label propagation for accurate prostate localization in CT images. IEEE Transactions on Medical Imaging, 32, 419–434.CrossRefPubMedGoogle Scholar
  9. Rand, W. M. (1971). Objective criteria for the evaluation of clustering methods. Journal of the American Statistical Association, 66(336), 846–850.CrossRefGoogle Scholar
  10. Rekik, I., Li, G., Wu, G., Lin, W., & Shen, D. (2015). Prediction of infant MRI appearance and anatomical structure evolution using sparse patch-based metamorphosis learning framework. In International Workshop on Patch-based Techniques in Medical Imaging (pp. 197–204). Springer, Cham.Google Scholar
  11. Rincón, M., Díaz-López, E., Selnes, P., Vegge, K., Altmann, M., Fladby, T., et al. (2017). Improved automatic segmentation of white matter hyperintensities in mri based on multilevel lesion features. Neuroinformatics, 1–15.Google Scholar
  12. Shen, D. (2007). Image registration by local histogram matching. Pattern Recognition, 40, 1161–1172.CrossRefGoogle Scholar
  13. Shi, F., Wang, L., Dai, Y., Gilmore, J. H., Lin, W., & Shen, D. (2012). LABEL: Pediatric brain extraction using learning-based meta-algorithm. NeuroImage, 62, 1975–1986.CrossRefPubMedPubMedCentralGoogle Scholar
  14. Sundar, H., Litt, H., & Shen, D. (2009). Estimating myocardial motion by 4D image warping. Pattern Recognition, 42, 2514–2526.CrossRefPubMedPubMedCentralGoogle Scholar
  15. Tong, T., Wolz, R., Coupé, P., Hajnal, J. V., & Rueckert, D. (2013). Initiative, A. D. N. & others segmentation of MR images via discriminative dictionary learning and sparse coding: Application to hippocampus labeling. NeuroImage, 76, 11–23.CrossRefPubMedGoogle Scholar
  16. Tustison, N. J., Avants, B. B., Cook, P. A., Zheng, Y., Egan, A., Yushkevich, P. A., & Gee, J. C. (2010). N4ITK: Improved N3 bias correction. IEEE Transactions on Medical Imaging, 29, 1310–1320.CrossRefPubMedPubMedCentralGoogle Scholar
  17. Wang, H., Suh, J. W., Das, S. R., Pluta, J. B., Craige, C., & Yushkevich, P. A. (2013). Multi-atlas segmentation with joint label fusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35, 611–623.CrossRefPubMedGoogle Scholar
  18. Wang, Y., Ma, G., An, L., Shi, F., Zhang, P., Wu, X., Zhou, J., & Shen, D. (2016a). Semi-Supervised Tripled Dictionary Learning for Standard-dose PET Image Prediction using Low-dose PET and Multimodal MRI. IEEE Transactions on Biomedical Engineering, 1–1.Google Scholar
  19. Wang, Y., Zhang, P., An, L., Ma, G., Kang, J., Shi, F., Wu, X., Zhou, J., Lalush, D. S., Lin, W., et al. (2016b). Predicting standard-dose PET image from low-dose PET and multimodal MR images using mapping-based sparse representation. Physics in Medicine and Biology, 61, 791.CrossRefPubMedGoogle Scholar
  20. Wu, G., Wang, Q., Lian, J., & Shen, D. (2013). Estimating the 4D respiratory lung motion by spatiotemporal registration and super-resolution image reconstruction. Medical Physics, 40(3), 532–539.Google Scholar
  21. Wu, G., Wang, Q., Zhang, D., Nie, F., Huang, H., & Shen, D. (2015a). A generative probability model of joint label fusion for multi-atlas based brain segmentation. Med Image Anal, 18(6), 881.CrossRefGoogle Scholar
  22. Wu, G., Kim, M., Wang, Q., Munsell, B. C., & Shen, D. (2015b). Scalable high-performance image registration framework by unsupervised deep feature representations learning. Deep Learning for Medical Image Analysis, 63(7), 1505–1516.Google Scholar
  23. Wu, G., Kim, M., Sanroma, G., Qian, W., Munsell, B. C., & Shen, D. (2015c). Hierarchical multi-atlas label fusion with multi-scale feature representation and label-specific patch partition. NeuroImage, 106, 34–46.CrossRefPubMedGoogle Scholar
  24. Zarei, M., Beckmann, C. F., Binnewijzend, M. A., Schoonheim, M. M., Oghabian, M. A., Sanz-Arigita, E. J., Scheltens, P., Matthews, P. M., & Barkhof, F. (2013). Functional segmentation of the hippocampus in the healthy human brain and in Alzheimer's disease. NeuroImage, 66, 28–35.CrossRefPubMedGoogle Scholar
  25. Zhou, L., Wang, L., & Ogunbona, P. (2014). Discriminative sparse inverse covariance matrix: Application in brain functional network classification. Computer Vision and Pattern Recognition, 3097–3104.Google Scholar
  26. Zhou, L., Wang, L., Liu, L., Ogunbona, P., & Shen, D. (2016). Learning discriminative bayesian networks from high-dimensional continuous neuroimaging data. IEEE Transactions on Pattern Analysis & Machine Intelligence, 38(11), 2269–2283.CrossRefGoogle Scholar
  27. Zhu, H., Cheng, H., Yang, X., & Fan, Y. (2017). Metric learning for multi-atlas based segmentation of hippocampus. Neuroinformatics, 15(1), 41–50.CrossRefPubMedPubMedCentralGoogle Scholar
  28. Zu, C., Wang, Z., Zhang, D., Liang, P., Shi, Y., Shen, D., & Wu, G. (2017). Robust multi-atlas label propagation by deep sparse representation. Pattern Recogn, 63, 511–517.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.College of Computer ScienceSichuan UniversityChengduChina
  2. 2.Fujian Provincial Key Laboratory of Information Processing and Intelligent Control (Minjiang University)FuzhouChina
  3. 3.Space Control and Inertial Technology Research CenterHarbin Institute of TechnologyHarbinChina
  4. 4.Department of Computer ScienceChengdu University of Information TechnologyChengduChina

Personalised recommendations