Abstract
Recent contrastive learning methods achieved state-of-the-art in low label regimes. However, the training requires large batch sizes and heavy augmentations to create multiple views of an image. With non-contrastive methods, the negatives are implicitly incorporated in the loss, allowing different images and modalities as pairs. Although the meta-information (i.e., age, sex) in medical imaging is abundant, the annotations are noisy and prone to class imbalance. In this work, we exploited already existing temporal information (different visits from a patient) in a longitudinal optical coherence tomography (OCT) dataset using temporally informed non-contrastive loss (TINC) without increasing complexity and need for negative pairs. Moreover, our novel pair-forming scheme can avoid heavy augmentations and implicitly incorporates the temporal information in the pairs. Finally, these representations learned from the pretraining are more successful in predicting disease progression where the temporal information is crucial for the downstream task. More specifically, our model outperforms existing models in predicting the risk of conversion within a time frame from intermediate age-related macular degeneration (AMD) to the late wet-AMD stage.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
The other eye that is not part of the interventional study.
- 2.
NCT00891735. https://clinicaltrials.gov/ct2/show/NCT00891735.
References
Azizi, S., et al.: Big self-supervised models advance medical image classification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3478–3488 (2021)
Banerjee, I., et al.: Prediction of age-related macular degeneration disease using a sequential deep learning approach on longitudinal sd-oct imaging biomarkers. Sci. Rep. 10(1), 15434 (2020). https://doi.org/10.1038/s41598-020-72359-y
Bardes, A., Ponce, J., LeCun, Y.: VICReg: variance-invariance-covariance regularization for self-supervised learning. In: International Conference on Learning Representations (2022)
Bressler, N.M.: Age-related macular degeneration is the leading cause of blindness. JAMA 291(15), 1900–1901 (2004). https://doi.org/10.1001/jama.291.15.1900
Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., Joulin, A.: Unsupervised learning of visual features by contrasting cluster assignments. Adv. Neural. Inf. Process. Syst. 33, 9912–9924 (2020)
Chen, X., He, K.: Exploring simple siamese representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15750–15758 (2021)
Chen, Y., et al.: USCL: Pretraining deep ultrasound image diagnosis model through video contrastive representation learning. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12908, pp. 627–637. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87237-3_60
Ermolov, A., Siarohin, A., Sangineto, E., Sebe, N.: Whitening for self-supervised representation learning. In: International Conference on Machine Learning, pp. 3015–3024. PMLR (2021)
Grill, J.B., et al.: Bootstrap your own latent-a new approach to self-supervised learning. Adv. Neural. Inf. Process. Syst. 33, 21271–21284 (2020)
Jing, L., Vincent, P., LeCun, Y., Tian, Y.: Understanding dimensional collapse in contrastive self-supervised learning. In: International Conference on Learning Representations (2022)
Li, H., et al.: Imbalance-aware self-supervised learning for 3D radiomic representations. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12902, pp. 36–46. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87196-3_4
van den Oord, A., Li, Y., Vinyals, O.: Representation learning with contrastive predictive coding (2019)
Rivail, A., et al.: Modeling disease progression in Retinal OCTs with longitudinal self-supervised learning. In: Rekik, I., Adeli, E., Park, S.H. (eds.) PRIME 2019. LNCS, vol. 11843, pp. 44–52. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32281-6_5
Russakoff, D.B., Lamin, A., Oakley, J.D., Dubis, A.M., Sivaprasad, S.: Deep learning for prediction of amd progression: a pilot study. Invest. Ophthalmol. Visual Sci. 60(2), 712–722 (2019)
Schmidt-Erfurth, U., et al.: Prediction of individual disease conversion in early amd using artificial intelligence. Invest. Ophthalmol. Visual Sci. 59(8), 3199–3208 (2018)
Wu, Z., Bogunović, H., Asgari, R., Schmidt-Erfurth, U., Guymer, R.H.: Predicting progression of age-related macular degeneration using oct and fundus photography. Ophthalmol. Retina 5(2), 118–125 (2021). https://doi.org/10.1016/j.oret.2020.06.026
Yan, Q., et al.: Deep-learning-based prediction of late age-related macular degeneration progression. Nat. Mach. intell. 2(2), 141–150 (2020)
Yang, J., et al.: Two-year risk of exudation in eyes with nonexudative age-related macular degeneration and subclinical neovascularization detected with swept source optical coherence tomography angiography. Am. J. Ophthalmol. 208, 1–11 (2019). https://doi.org/10.1016/j.ajo.2019.06.017
Yim, J., et al.: Predicting conversion to wet age-related macular degeneration using deep learning. Nat. Med. 26(6), 892–899 (2020)
Zbontar, J., Jing, L., Misra, I., LeCun, Y., Deny, S.: Barlow twins: self-supervised learning via redundancy reduction. In: International Conference on Machine Learning, pp. 12310–12320. PMLR (2021)
Zhang, L., et al.: Validity of Automated Choroidal Segmentation in SS-OCT and SD-OCT. Investigative Ophthalmol. Visual Sci. 56(5), 3202–3211 (2015). https://doi.org/10.1167/iovs.14-15669
Acknowledgements
The work has been partially funded by FWF Austrian Science Fund (FG 9-N), and a Wellcome Trust Collaborative Award (PINNACLE Ref. 210572/Z/18/Z).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Emre, T., Chakravarty, A., Rivail, A., Riedl, S., Schmidt-Erfurth, U., Bogunović, H. (2022). TINC: Temporally Informed Non-contrastive Learning for Disease Progression Modeling in Retinal OCT Volumes. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. MICCAI 2022. Lecture Notes in Computer Science, vol 13432. Springer, Cham. https://doi.org/10.1007/978-3-031-16434-7_60
Download citation
DOI: https://doi.org/10.1007/978-3-031-16434-7_60
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16433-0
Online ISBN: 978-3-031-16434-7
eBook Packages: Computer ScienceComputer Science (R0)