Abstract
Segmentation of thigh tissues (muscle, fat, inter-muscular adipose tissue (IMAT), bone, and bone marrow) from magnetic resonance imaging (MRI) scans is useful for clinical and research investigations in various conditions such as aging, diabetes mellitus, obesity, metabolic syndrome, and their associated comorbidities. Towards a fully automated, robust, and precise quantification of thigh tissues, herein we designed a novel semi-supervised segmentation algorithm based on deep network architectures. Built upon Tiramisu segmentation engine, our proposed deep networks use variational and specially designed targeted dropouts for faster and robust convergence, and utilize multi-contrast MRI scans as input data. In our experiments, we have used 150 scans from 50 distinct subjects from the Baltimore Longitudinal Study of Aging (BLSA). The proposed system made use of both labeled and unlabeled data with high efficacy for training, and outperformed the current state-of-the-art methods. In particular, dice scores of 97.52%, 94.61%, 80.14%, 95.93%, and 96.83% are achieved for muscle, fat, IMAT, bone, and bone marrow segmentation, respectively. Our results indicate that the proposed system can be useful for clinical research studies where volumetric and distributional tissue quantification is pivotal and labeling is a significant issue. To the best of our knowledge, the proposed system is the first attempt at multi-tissue segmentation using a single end-to-end semi-supervised deep learning framework for multi-contrast thigh MRI scans.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
The data sets generated during and/or analyzed during the current study are available from the BLSA with research agreement [2].
References
Goodpaster, B.H., Krishnaswami, S., Resnick, H., Kelley, D.E., Haggerty, C., Harris, T.B., Schwartz, A.V., Kritchevsky, S., & Newman, A.B. (2003). Association between regional adipose tissue distribution and both type 2 diabetes and impaired glucose tolerance in elderly men and women. Diabetes Care, 26(2), 372.
BLSA. (2009). Longitudinal study of muscle strength, quality, and adipose tissue infiltration. The American Journal of Clinical Nutrition, 90(6), 1579. https://doi.org/10.3945/ajcn.2009.28047.
Hamrick, M.W., McGee-Lawrence, M.E., & Frechette, D.M. (2016). Fatty infiltration of skeletal muscle: mechanisms and comparisons with bone marrow adiposity. Frontiers in Endocrinology, 7, 69.
Porter, M.M., Vandervoort, A.A., & Lexell, J. (1995). Aging of human muscle: structure, function and adaptability. Scandinavian Journal of Medicine & Science in Sports, 5(3), 129.
Loughran, T., Higgins, D.M., McCallum, M., Coombs, A., Straub, V., & Hollingsworth, K.G. (2015). Improving highly accelerated fat fraction measurements for clinical trials in muscular dystrophy: origin and quantitative effect of r2* changes. Radiology, 275(2), 570.
Gadermayr, M., Disch, C., Müller, M., Merhof, D., & Gess, B. (2018). A comprehensive study on automated muscle segmentation for assessing fat infiltration in neuromuscular diseases. Magnetic Resonance Imaging, 48, 20.
Morrow, J.M., Sinclair, C.D., Fischmann, A., Machado, P.M., Reilly, M.M., Yousry, T.A., Thornton, J.S., & Hanna, M.G. (2016). Mri biomarker assessment of neuromuscular disease progression: a prospective observational cohort study. The Lancet Neurology, 15(1), 65.
Mhuiris, Á.N., Volken, T., Elliott, J.M., Hoggarth, M., Samartzis, D., & Crawford, R.J. (2016). Reliability of quantifying the spatial distribution of fatty infiltration in lumbar paravertebral muscles using a new segmentation method for t1-weighted mri. BMC Musculoskeletal Disorders, 17(1), 234.
Díaz-Manera, J., Llauger, J., Gallardo, E., & Illa, I. (2015). Muscle mri in muscular dystrophies. Acta Myologica, 34(2-3), 95.
Anwar, S.M., Majid, M., Qayyum, A., Awais, M., Alnowami, M., & Khan, M.K. (2018). Medical image analysis using convolutional neural networks: a review. Journal of Medical Systems, 42(11), 226.
Hussain, S., Anwar, S.M., & Majid, M. (2018). Segmentation of glioma tumors in brain using deep convolutional neural network. Neurocomputing, 282, 248.
Hussein, S., Kandel, P., Bolan, C.W., Wallace, M.B., & Bagci, U. (2019). Lung and pancreatic tumor characterization in the deep learning era: novel supervised and unsupervised learning approaches. IEEE Transactions on Medical Imaging.
Gadermayr, M., Li, K., Müller, M., Truhn, D., Krämer, N., Merhof, D., & Gess, B. (2019). Domain-specific data augmentation for segmenting mr images of fatty infiltrated human thighs with neural networks. Journal of Magnetic Resonance Imaging.
Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In International conference on medical image computing and computer-assisted intervention (pp. 234–241): Springer.
Bocchieri, A.E., Parekh, V.S., Wagner, K.R., Braverman, S.A.V., Leung, D.G., & Jacobs, M.A. (2019). Multiparametric deep learning tissue signatures for muscular dystrophy: Preliminary results.
Makrogiannis, S., Fishbein, K.W., Moore, A., Spencer, R., & Ferrucci, L. (2016). Image-based tissue distribution modeling for skeletal muscle quality characterization. IEEE Transactions on Biomedical Engineering, 63(4), 805.
Irmakci, I., Hussein, S., Savran, A., Kalyani, R.R., Reiter, D., Chia, C.W., Fishbein, K., Spencer, R.G., Ferrucci, L., & Bagci, U. (2018). A novel extension to fuzzy connectivity for body composition analysis: Applications in thigh, brain, and whole body tissue segmentation. IEEE Transactions on Biomedical Engineering.
Ferrucci, L. (2008). The baltimore longitudinal study of aging (blsa): a 50-year-long journey and plans for the future. Journals of Gerontology - Series A Biological Sciences and Medical Sciences, 63(12), 1416.
Lugauer, F., Nickel, D., Wetzl, J., Kannengiesser, S.A., Maier, A., & Hornegger, J. (2015). Robust spectral denoising for water-fat separation in magnetic resonance imaging. In International conference on medical image computing and computer-assisted intervention (pp. 667–674): Springer.
Tustison, N.J., Avants, B.B., Cook, P.A., Zheng, Y., Egan, A., Yushkevich, P.A., & Gee, J.C. (2010). N4itk: improved n3 bias correction. IEEE Transactions on Medical Imaging, 29(6), 1310.
Aurich, V., & Weule, J. (1995). Non-linear gaussian filters performing edge preserving diffusion. In Mustererkennung 1995 (pp. 538–545): Springer.
Nyúl, L.G., Udupa, J.K., & Zhang, X. (2000). New variants of a method of mri scale standardization. IEEE Transactions on Medical Imaging, 19(2), 143.
Jégou, S., Drozdzal, M., Vazquez, D., Romero, A., & Bengio, Y. (2017). The one hundred layers tiramisu: Fully convolutional densenets for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops (pp. 11–19).
Cheplygina, V., de Bruijne, M., & Pluim, J.P. (2019). Not-so-supervised: A survey of semi-supervised, multi-instance, and transfer learning in medical image analysis. Medical Image Analysis, 54, 280. https://doi.org/10.1016/j.media.2019.03.009. http://www.sciencedirect.com/science/article/pii/S1361841518307588.
Orgiu, S., Lafortuna, C.L., Rastelli, F., Cadioli, M., Falini, A., & Rizzo, G. (2016). Automatic muscle and fat segmentation in the thigh from t1-weighted mri. Journal of Magnetic Resonance Imaging, 43 (3), 601.
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1), 1929.
Neelakantan, A., Vilnis, L., Le, Q.V., Sutskever, I., Kaiser, L., Kurach, K., & Martens, J. (2015). Adding gradient noise improves learning for very deep networks, arXiv:1511.06807.
Kingma, D.P., Salimans, T., & Welling, M. (2015). Variational dropout and the local reparameterization trick. In Advances in neural information processing systems (pp. 2575–2583).
Gomez, A.N., Zhang, I., Swersky, K., Gal, Y., & Hinton, G.E. (2019). Learning sparse networks using targeted dropout. arXiv:1905.13678.
Huttenlocher, D.P., Klanderman, G.A., & Rucklidge, W.J. (1993). Comparing images using the hausdorff distance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(9), 850.
Yao, J., Kovacs, W., Hsieh, N., Liu, C.Y., & Summers, R.M. (2017). Holistic segmentation of intermuscular adipose tissues on thigh mri. In MICCAI (pp. 737–745).
Acknowledgments
We would like to acknowledge the Baltimore Longitudinal Aging Study (BLAS) for providing the data set used in this study. We would also like to thank NVIDIA for donating a Titan X GPU for our experiments.
Author information
Authors and Affiliations
Contributions
UB is the PI of the project as a part of the larger effort for creating Musculoskeletal AI in multiple institutes, agreed by the participating institutes’ PIs: MA, JE, CA, SJ, and DAT. SMA structured the overall experiments and discussions while II, SJ, and SMA ran the image analysis pipelines and deep learning experiments. UB, SJ, DAT, JE, MA, GZP, and SMA analyzed the results, and evaluated the clinical relevance, reproducibility, and feasibility of the proposed method(s). DAT and JE participated to the study as radiologists while CA and SJ contributed as MRI physicists. All authors wrote and edited the manuscript, and agreed on the content prior to submission.
Corresponding author
Ethics declarations
Conflict of interests
The authors declare no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Anwar, S.M., Irmakci, I., Torigian, D.A. et al. Semi-Supervised Deep Learning for Multi-Tissue Segmentation from Multi-Contrast MRI. J Sign Process Syst 94, 497–510 (2022). https://doi.org/10.1007/s11265-020-01612-4
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11265-020-01612-4