Abstract
Deep learning and advancements in computer vision offer significant potential for analyzing medical images resulting in better healthcare and improved patient outcomes. Currently, the dominant approaches in the field of machine learning are supervised learning and transfer learning. These methods are not only prevalent in medicine and healthcare but also across various other industries. They rely on large datasets that have been manually annotated to train increasingly sophisticated models. However, the manual labeling process results in a wealth of untapped, unlabeled data that is accessible in both public and private data repositories. Self-supervised learning (SSL), an emerging field within machine learning, provides a solution by leveraging this untapped, unlabeled data. Unlike traditional machine learning paradigms, SSL algorithms pre-train models using artificial supervisory signals generated from the unlabeled data. This comprehensive review article explores the fundamental concepts, approaches, and advancements in self-supervised learning, with a particular emphasis on medical image datasets and their sources. By summarizing and highlighting the main contributions and findings from the article, this analysis and synthesis aim to shed light on the current state of research in self-supervised learning. Through these rigorous efforts, the existing body of knowledge is synthesized, and implementation recommendations are provided for future researchers interested in harnessing self-supervised learning to develop classification models for medical imaging.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Abbreviations
- AI:
-
Artificial intelligence
- AUC:
-
Area under curve
- BraTS:
-
Multimodal brain tumor segmentation
- CNN:
-
Convolutional neural network
- CPC:
-
Contrastive predictive coding
- CT:
-
Computed tomography
- CXR:
-
X-ray images of chest
- DL:
-
Deep learning
- GANs:
-
Generative adversarial networks
- InfoNCE:
-
Normalized mutual information neural estimation
- ISIC:
-
International Skin Imaging Collaboration
- LIDC–IDRI:
-
Lung image database consortium and image database resource initiative
- LLP:
-
Label proportions
- MICLe:
-
Multi-instance contrastive learning
- MIMIC-CXR:
-
Medical information mart for intensive care-chest X-ray
- MRI:
-
Magnetic resonance imaging
- MURA:
-
Musculoskeletal radiographs
- NAF:
-
Neural autoregressive flows
- NT-Xent:
-
Normalized temperature-scaled cross entropy
- OOD:
-
Out-of-distribution
- PARISMA:
-
Preferred reporting items for systematic reviews and meta-analyses
- PET:
-
Positron emission tomography
- SSL:
-
Self-supervised learning
- US:
-
Ultra sound
References
Albelwi S (2022) Survey on self-supervised learning: auxiliary pretext tasks and contrastive learning methods in imaging. Entropy (basel, Switzerland) 24(4):551. https://doi.org/10.3390/e24040551
Ali R, Khan MUK, Kyung CM (2020) Self-supervised representation learning for visual anomaly detection. arXiv:2006.09654
Anton J, Castelli L, Chan MF, Outters M, Tang WH, Cheung V, Shukla P, Walambe R, Kotecha K (2022) How well do self-supervised models transfer to medical imaging? J Imaging 8(12):320. https://doi.org/10.3390/jimaging8120320
Azizi S, Mustafa B, Ryan F, Beaver Z, Freyberg J, Deaton J, Loh A, Karthikesalingam A, Kornblith S, Chen T, Natarajan V, Norouzi M.(2021) Big self-supervised models advance medical image classification. In: 2021 IEEE/CVF international conference on computer vision (ICCV). https://doi.org/10.1109/iccv48922.2021.00346
Bai W, Chen C, Tarroni G, Duan J, Guitton F, Petersen SE, Rueckert D (2019) Self-supervised learning for cardiac MR image segmentation by anatomical position prediction. In: Medical image computing and computer assisted intervention–MICCAI 2019: 22nd international conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part II 22. Springer International Publishing, pp 541–549
Cao B, Zhang H, Wang N, Gao X, Shen D (2020) Auto-gan: self-supervised collaborative learning for medical image synthesis. Proc AAAI Conf Artif Intell 34(07):10486–10493. https://doi.org/10.1609/aaai.v34i07.6619
Chaitanya K, Erdil E, Karani N, Konukoglu E (2020) Contrastive learning of global and local features for medical image segmentation with limited annotations. Adv Neural Inf Process Syst 33:12546–12558
Chen W, Li K (2022) Self-supervised learning for medical image classification using imbalanced training data. Explor Novel Intell Optim Algorithms. https://doi.org/10.1007/978-981-19-4109-2_23
Chen L, Bentley P, Mori K, Misawa K, Fujiwara M, Rueckert D (2019) Self-supervised learning for medical image analysis using image context restoration. Med Image Anal 58:101539. https://doi.org/10.1016/j.media.2019.101539
Chowdhury A, Rosenthal J, Waring J, Umeton R (2021) Applying self-supervised learning to medicine: review of the state of the art and medical implementations. Informatics 8(3):59. https://doi.org/10.3390/informatics8030059
Codella N, Cai J, Abedini M, Garnavi R, Halpern A, Smith JR (2015) Deep learning, sparse coding, and SVM for melanoma recognition in Dermoscopy Images. Mach Learn Med Imaging. https://doi.org/10.1007/978-3-319-24888-2_15
Ding J, Xie E, Xu H, Jiang C, Li Z, Luo P, Xia G-S (2022) Deeply unsupervised patch re-identification for pre-training object detectors. IEEE Trans Pattern Anal Mach Intell. https://doi.org/10.1109/tpami.2022.3164911
Dominic J, Bhaskhar N, Desai AD, Schmidt A, Rubin E, Gunel B, Gold GE, Hargreaves BA, Lenchik L, Boutin R, Chaudhari AS (2023) Improving data-efficiency and robustness of medical imaging segmentation using inpainting-based self-supervised learning. Bioengineering 10(2):207. https://doi.org/10.3390/bioengineering10020207
Ericsson L, Gouk H, Loy CC, Hospedales TM (2022) Self-supervised representation learning: introduction, advances, and challenges. IEEE Signal Process Mag 39(3):42–62. https://doi.org/10.1109/msp.2021.3134634
Fashi PA, Hemati S, Babaie M, Gonzalez R, Tizhoosh HR (2022) A self-supervised contrastive learning approach for whole slide image representation in digital pathology. J Pathol Inform 13:100133. https://doi.org/10.1016/j.jpi.2022.100133
Fischer M, Hepp T, Gatidis S, Yang B (2023) Self-supervised contrastive learning with random walks for medical image segmentation with limited annotations. Comput Med Imaging Graph 104:102174. https://doi.org/10.1016/j.compmedimag.2022.102174
Gao X, Hu W, Qi GJ (2021) Self-supervised multi-view learning via auto-encoding 3D transformations. arXiv:2103.00787
Ghesu FC, Georgescu B, Mansoor A, Yoo Y, Neumann D, Patel P, Comaniciu D (2022) Self-supervised learning from 100 million medical images. arXiv:2201.01283.
Gidaris S, Bursuc A, Puy G, Komodakis N, Cord M, Perez P (2021) Obow: online bag-of-visual-words generation for self-supervised learning. In: 2021 IEEE/CVF conference on computer vision and pattern recognition (CVPR). https://doi.org/10.1109/cvpr46437.2021.00676
Hojjati H, Ho TKK, Armanfard N (2022) Self-supervised anomaly detection: a survey and outlook. arXiv:2205.05173
Huang Z, Lin J, Xu L, Wang H, Bai T, Pang Y, Meen T-H (2020) Fusion high-resolution network for diagnosing Chest X-ray images. Electronics 9(1):190. https://doi.org/10.3390/electronics9010190
Huang SC, Pareek A, Jensen M (2023) Self-supervised learning for medical image classification: a systematic review and implementation guidelines. Npj Digit Med 6:74. https://doi.org/10.1038/s41746-023-00811-0
Kasinathan G, Jayakumar S (2022) Cloud-based lung tumor detection and stage classification using deep learning techniques. Biomed Res Int 2022:1–17. https://doi.org/10.1155/2022/4185835
Krishnan R, Rajpurkar P, Topol EJ (2022) Self-supervised learning in medicine and Healthcare. Nat Biomed Eng 6(12):1346–1352. https://doi.org/10.1038/s41551-022-00914-1
Le-Khac PH, Healy G, Smeaton AF (2020) Contrastive representation learning: a framework and review. IEEE Access 8:193907–193934. https://doi.org/10.1109/access.2020.3031549
Li X, Jiang Y, Rodriguez-Andina JJ (2021) When medical images meet generative adversarial network: recent development and research opportunities. Discov Artif Intell 1:5. https://doi.org/10.1007/s44163-021-00006-0
Li Y, Mohamied Y, Bell P, Lai C (2023) Exploration of a self-supervised speech model: a study on emotional corpora. In: 2022 IEEE spoken language technology workshop (SLT). https://doi.org/10.1109/slt54892.2023.10023428
Liu S, Mallol-Ragolta A, Parada-Cabaleiro E, Qian K, Jing X, Kathan A, Schuller BW (2022) Audio self-supervised learning: a survey. Patterns 3(12):100616
Mascolini A, Cardamone D, Ponzio F (2022) Exploiting generative self-supervised learning for the assessment of biological images with lack of annotations. BMC Bioinform 23:295. https://doi.org/10.1186/s12859-022-04845-1
Muller AE, Berg RC, Meneses-Echavez JF (2023) The effect of machine learning tools for evidence synthesis on resource use and time-to-completion: protocol for a retrospective pilot study. Syst Rev 12:7. https://doi.org/10.1186/s13643-023-02171-y
Nguyen X, Lee G, Kim S, Yang H (2020) Self-supervised learning based on spatial awareness for medical image analysis. IEEE Access 8:162973–162981
Ohri K, Kumar M (2021) Review on self-supervised image recognition using Deep Neural Networks. Knowl Based Syst 224:107090. https://doi.org/10.1016/j.knosys.2021.107090
Page MJ, McKenzie JE, Bossuyt PM (2021) The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Syst Rev 10:89. https://doi.org/10.1186/s13643-021-01626-4
Qu L, Liu S, Liu X, Wang M, Song Z (2022) Towards label-efficient automatic diagnosis and analysis: a comprehensive survey of advanced deep learning-based weakly-supervised, semi-supervised and self-supervised techniques in histopathological image analysis. Phys Med Biol. https://doi.org/10.1088/1361-6560/ac910a
Ramapuram J, Busbridge D, Webb R (2021) Evaluating the fairness of fine-tuning strategies in self-supervised learning. arXiv:2110.00538
Rani V, Kumar M (2023a) Human gait recognition: a systematic review. Multimed Tools Appl. https://doi.org/10.1007/s11042-023-15079-5
Rani V, Kumar M (2023b) DeepNet-gait: Human identification by gait using convolutional neural network model. In: 2023 10th international conference on signal processing and integrated networks (SPIN). https://doi.org/10.1109/spin57001.2023.10117067
Rani V, Kumar M, Mittal A, Kumar K (2022a) Artificial Intelligence for cybersecurity: Recent advancements, challenges and opportunities. Robot AI Cybersecur Crit Infrastruct Smart Cities. https://doi.org/10.1007/978-3-030-96737-6_4
Rani V, Kumar M, Singh B (2022b) Handcrafted features for human gait recognition: CASIA-A dataset. Commun Comput Inf Sci. https://doi.org/10.1007/978-3-031-21385-4_7
Rani V, Nabi ST, Kumar M, Mittal A, Kumar K (2023) Self-supervised learning: a succinct review. Arch Comput Methods Eng 30(4):2761–2775. https://doi.org/10.1007/s11831-023-09884-2
Shurrab S, Duwairi R (2022) Self-supervised learning methods and applications in medical imaging analysis: a survey. PeerJ Comput Sci. https://doi.org/10.7717/peerj-cs.1045
Spitzer H, Kiwitz K, Amunts K, Harmeling S, Dickscheid T (2018) Improving cytoarchitectonic segmentation of human brain areas with self-supervised Siamese networks. In: Medical image computing and computer assisted intervention—MICCAI 2018, 663–671. https://doi.org/10.1007/978-3-030-00931-1_76
Suganyadevi S, Seethalakshmi V, Balasamy K (2022) A review on deep learning in medical image analysis. Int J Multimed Info Retr 11:19–38. https://doi.org/10.1007/s13735-021-00218-1
Taleb A, Rohrer C, Bergner B, De Leon G, Rodrigues JA, Schwendicke F, Lippert C, Krois J (2022) Self-supervised learning methods for label-efficient dental caries classification. Diagnostics 12(5):1237. https://doi.org/10.3390/diagnostics12051237
Tao X, Li Y, Zhou W, Ma K, Zheng Y (2020) Revisiting Rubik’s cube: Self-supervised learning with volume-wise transformation for 3D medical image segmentation. In: Medical image computing and computer assisted intervention—MICCAI 2020, pp 238–248. https://doi.org/10.1007/978-3-030-59719-1_24
Wang L, Jiang Z, Shao A, Liu Z, Gu R, Ge R, Jia G, Wang Y, Ye J (2022) Self-supervised learning mechanism for identification of eyelid malignant melanoma in pathologic slides with limited annotation. Front Med. https://doi.org/10.3389/fmed.2022.976467
Wang WC, Ahn E, Feng D, Kim J (2023a) A review of predictive and contrastive self-supervised learning for medical images. arXiv:2302.05043
Wang X, Cheng Y, Yang Y (2023b) Multitask joint strategies of self-supervised representation learning on biomedical networks for drug discovery. Nat Mach Intell 5:445–456. https://doi.org/10.1038/s42256-023-00640-6
Xu J (2021) A review of self-supervised learning methods in the field of medical image analysis. Int J Image Graph Signal Process 13(4):33–46. https://doi.org/10.5815/ijigsp.2021.04.03
Zeng G, Guo Y, Zhan J (2021) A review on deep learning MRI reconstruction without fully sampled k-space. BMC Med Imaging 21:195. https://doi.org/10.1186/s12880-021-00727-9
Zhang Z, Wang R, Zhang H, Chen Y, Zuo W (2022) Self-supervised learning for real-world super-resolution from dual zoomed observations. Lect Notes Comput Sci. https://doi.org/10.1007/978-3-031-19797-0_35
Zhang P, Li Y, Zhuang Y, Kuang J, Niu X, Chen R (2023) Multi-level information fusion with motion constraints: key to achieve high-precision gait analysis using low-cost inertial sensors. Inf Fusion 89:603–618
Zhao L, Jia C, Ma J, Shao Y, Liu Z, Yuan H (2023) Medical image segmentation based on self-supervised hybrid fusion network. Front Oncol. https://doi.org/10.3389/fonc.2023.1109786
Zhou H-Y, Lu C, Yang S, Han X, Yu Y (2021) Preservational learning improves self-supervised medical image models by reconstructing diverse contexts. In: 2021 IEEE/CVF international conference on computer vision (ICCV). https://doi.org/10.1109/iccv48922.2021.00348
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Rani, V., Kumar, M., Gupta, A. et al. Self-supervised learning for medical image analysis: a comprehensive review. Evolving Systems (2024). https://doi.org/10.1007/s12530-024-09581-w
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s12530-024-09581-w