Skip to main content

Advertisement

SpringerLink
Log in

Continuous image anomaly detection based on contrastive lifelong learning

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

With the development of deep learning techniques, an increasing number of anomaly detection methods based on deep neural networks have been proposed during the last decade. Nevertheless, these methods often suffer from catastrophic forgetting when trained on continuously arriving data samples, as deep neural networks quickly forget the knowledge obtained from previous training while adjusting to learning new information. In this work, we propose a contrastive lifelong learning model for image anomaly detection. Rather than adopting CNN-based neural networks as in other anomaly detection approaches to learn representations from training samples, we propose a contrastive learning framework for anomaly detection in which Vision Transformer (VIT) is adopted for extracting promising representations. Two nonlinear structures (projector and predictor) are integrated into our model, which is helpful in improving the performance of anomaly detection. Moreover, a lifelong learning framework that contains teacher and student networks is deployed in our model, which is able to mitigate the problem of catastrophic forgetting in image anomaly detection. By leveraging both lifelong learning and contrastive learning frameworks, our model is able to progressively perform image anomaly detection where the problem of catastrophic forgetting can be greatly mitigated. We demonstrate the effectiveness of the proposed anomaly detection method by conducting experiments on multiple image data sets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data Availability

The data sets analysed during the current study are available at:

– CIFAR10 and CIFAR100: http://www.cs.toronto.edu/~kriz/cifar.html

– BreakHis: https://web.inf.ufpr.br/vri/databases

– BACH: https://iciar2018-challenge.grand-challenge.org/Dataset

– MNIST: http://yann.lecun.com/exdb/mnist

– MVTec: https://www.mvtec.com/company/research/datasets/mvtec-ad

Notes

  1. The source code of our proposed model is available at https://github.com/sutusutu/lifelong-anomaly-detection.

References

  1. Akcay S, Atapour-Abarghouei A, Breckon TP (2018) Ganomaly: Semi-supervised anomaly detection via adversarial training. In: Asian conference on computer vision, Springer, pp 622–637

  2. Aresta G, Araújo T, Kwok S, Chennamsetty SS, Safwan M, Alex V, Marami B, Prastawa M, Chan M, Donovan M et al (2019) Bach: Grand challenge on breast cancer histology images. Med Image Anal 56:122–139

    Article  Google Scholar 

  3. Bergmann P, Fauser M, Sattlegger D, Steger C (2019) Mvtec ad–a comprehensive real-world dataset for unsupervised anomaly detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 9592–9600

  4. Bottou L (2010) Large-scale machine learning with stochastic gradient descent. In: Proceedings of COMPSTAT’2010, Springer, pp 177–186

  5. Brown CD, Davis HT (2006) Receiver operating characteristics curves and related decision measures: a tutorial. Chemometr Intell Lab Syst 80(1):24–38

    Article  Google Scholar 

  6. Chen T, Kornblith S, Norouzi M, Hinton G (2020a) A simple framework for contrastive learning of visual representations. In: International conference on machine learning, PMLR, pp 1597–1607

  7. Chen X, He K (2021) Exploring simple siamese representation learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 15,750–15,758

  8. Chen X, Fan H, Girshick R, He K (2020b) Improved baselines with momentum contrastive learning. arXiv:200304297

  9. Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) Imagenet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, pp 248–255

  10. Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S et al (2020) An image is worth 16x16 words: Transformers for image recognition at scale. arXiv:201011929

  11. Fini E, da Costa VGT, Alameda-Pineda X, Ricci E, Alahari K, Mairal J (2021) Self-supervised models are continual learners. arXiv:211204215

  12. Fujioka T, Kubota K, Mori M, Kikuchi Y, Katsuta L, Kimura M, Yamaga E, Adachi M, Oda G, Nakagawa T et al (2020) Efficient anomaly detection with generative adversarial network for breast ultrasound imaging. Diagnostics 10(7):456

    Article  Google Scholar 

  13. He J, Zhu F (2021) Online continual learning for visual food classification. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 2337–2346

  14. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  15. Hendrycks D, Mazeika M, Kadavath S, Song D (2019) Using self-supervised learning can improve model robustness and uncertainty. In: Advances in neural information processing systems, vol 32

  16. van Hespen KM, Zwanenburg JJ, Dankbaar JW, Geerlings MI, Hendrikse J, Kuijf HJ (2021) An anomaly detection approach to identify chronic brain infarcts on mri. Sci Rep 11(1):1–10

    Google Scholar 

  17. Kingma DP, Welling M (2013) Auto-encoding variational bayes. arXiv:13126114

  18. Krizhevsky A (2009) Learning multiple layers of features from tiny images. Tech. rep.

  19. LeCun Y (1998) The mnist database of handwritten digits. http://yannlecuncom/exdb/mnist/

  20. Lee S, Behpour S, Eaton E (2021) Sharing less is more: Lifelong learning in deep networks with selective layer transfer. In: International conference on machine learning, PMLR, pp 6065–6075

  21. Lu Y, Xu P (2018) Anomaly detection for skin disease images using variational autoencoder. arXiv:180701349

  22. Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, et al. (2017) Imagenet large scale visual recognition challenge. Int J Comput Vis 115(3):211–252

    MathSciNet  Google Scholar 

  23. Nunnari F, Alam HMT, Sonntag D (2021) Anomaly detection for skin lesion images using replicator neural networks. In: International cross-domain conference for machine learning and knowledge extraction, Springer, pp 225–240

  24. Pinaya WHL, Tudosiu PD, Gray R, Rees G, Nachev P, Ourselin S, Cardoso MJ (2021) Unsupervised brain anomaly detection and segmentation with transformers. arXiv:210211650

  25. Quellec G, Lamard M, Cozic M, Coatrieux G, Cazuguel G (2016) Multiple-instance learning for anomaly detection in digital mammography. Ieee Trans Med Imaging 35(7):1604–1614

    Article  Google Scholar 

  26. Raghu M, Unterthiner T, Kornblith S, Zhang C (2021) Do vision transformers see like convolutional neural networks?. In: Advances in Neural Information Processing Systems

  27. Reiss T, Hoshen Y (2021) Mean-shifted contrastive loss for anomaly detection. arXiv:210603844

  28. Reiss T, Cohen N, Bergman L, Hoshen Y (2020) Panda–adapting pretrained features for anomaly detection. arXiv:201005903

  29. Ruff L, Vandermeulen R, Goernitz N, Deecke L, Siddiqui SA, Binder A, Müller E, Kloft M (2018) Deep one-class classification. In: International conference on machine learning, PMLR, pp 4393–4402

  30. Sánchez-Cauce R, Pérez-martín J, Luque M (2021) Multi-input convolutional neural network for breast cancer detection using thermal images and clinical data. Comput Methods Programs Biomed 204:106,045

    Article  Google Scholar 

  31. Seeböck P, Orlando JI, Schlegl T, Waldstein SM, Bogunović H, Klimscha S, Langs G, Schmidt-Erfurth U (2019) Exploiting epistemic uncertainty of anatomy segmentation for anomaly detection in retinal oct. IEEE Trans Med Imaging 39(1):87–98

    Article  Google Scholar 

  32. Sohn K, Li CL, Yoon J, Jin M, Pfister T (2021) Learning and evaluating representations for deep one-class classification. In: International conference on learning representations

  33. Spanhol FA, Oliveira LS, Petitjean C, Heutte L (2015) A dataset for breast cancer histopathological image classification. Ieee Trans Biomed Eng 63(7):1455–1462

    Article  Google Scholar 

  34. Tack J, Mo S, Jeong J, Shin J (2020) Csi: Novelty detection via contrastive learning on distributionally shifted instances. In: 34th Conference on neural information processing systems

  35. Tan J, Hou B, Day T, Simpson J, Rueckert D, Kainz B (2021) Detecting outliers with poisson image interpolation. In: International conference on medical image computing and computer-assisted intervention, Springer, pp 581–591

  36. Thrun S, Mitchell TM (1995) Lifelong robot learning. Rob Auton Syst 15(1-2):25–46

    Article  Google Scholar 

  37. Tian Y, Pang G, Liu F, Chen Y, Shin SH, Verjans JW, Singh R, Carneiro G (2021) Constrained contrastive distribution learning for unsupervised anomaly detection and localisation in medical images. In: International conference on medical image computing and computer-assisted intervention, Springer, pp 128–140

  38. Ye F, Bors AG (2021) Lifelong mixture of variational autoencoders. IEEE Transactions on Neural Networks and Learning Systems, 1–14, https://doi.org/10.1109/TNNLS.2021.3096457

  39. Zagoruyko S, Komodakis N (2015) Learning to compare image patches via convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4353–4361

  40. Zhao H, Wang H, Fu Y, Wu F, Li X (2021) Memory efficient class-incremental learning for image classification. IEEE Transactions on Neural Networks and Learning Systems, 1–12, https://doi.org/10.1109/TNNLS.2021.3072041

  41. Zhao Q, Karray F (2020) Anomaly detection for images using auto-encoder based sparse representation. In: International conference on image analysis and recognition, Springer, pp 144–153

  42. Zhou K, Xiao Y, Yang J, Cheng J, Liu W, Luo W, Gu Z, Liu J, Gao S (2020) Encoding structure-texture relation with p-net for anomaly detection in retinal images. In: European conference on computer vision, Springer, pp 360–377

  43. Zhou W, Gao Y, Ji J, Li S, Yi Y (2021) Unsupervised anomaly detection for glaucoma diagnosis. Wirel Commun Mob Comput, 2021

Download references

Acknowledgements

The completion of this work was supported in part by the National Natural Science Foundation of China (62276106,61876068), the UIC Start-up Research Fund (UICR0700056-23), the Guangdong Provincial Key Laboratory of Interdisciplinary Research and Application for Data Science(2022B1212010006), the Guangdong Higher Education Upgrading Plan (2021-2025) of “Rushing to the Top, Making Up Shortcomings and Strengthening Special Features” (R0400001-22) and the Artificial Intelligence and Data Science Research Hub (AIRH) of BNU-HKBU United International College.

Author information

Authors and Affiliations

  1. Department of Computer Science, Beijing Normal University-Hong Kong Baptist University United International College, Zhuhai, China

    Wentao Fan

  2. Department of Computer Science and Technology, Huaqiao University, Xiamen, China

    Wentao Fan & Weimin Shangguan

  3. Guangdong Provincial Key Laboratory of Interdisciplinary Research and Application for Data Science, BNU-HKBU United International College, Zhuhai, China

    Wentao Fan

  4. Concordia Institute for Information Systems Engineering (CIISE), Concordia University, Montreal, QC, Canada

    Nizar Bouguila

Authors
  1. Wentao Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Weimin Shangguan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nizar Bouguila
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wentao Fan.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflicts of interest to this paper.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fan, W., Shangguan, W. & Bouguila, N. Continuous image anomaly detection based on contrastive lifelong learning. Appl Intell 53, 17693–17707 (2023). https://doi.org/10.1007/s10489-022-04401-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-04401-7

Keywords

Search

Navigation