Skip to main content
SpringerLink
Log in

A Regularization Factor-Based Approach to Anomaly Detection Using Contrastive Learning

  • Research Article-Computer Engineering and Computer Science
  • Published:
Arabian Journal for Science and Engineering Aims and scope Submit manuscript

Abstract

Anomaly detection methods are introduced to remove abnormalities from specific datasets. In the paper, mean shifted contrastive loss for anomaly detection, a new objective function named mean shifted contrastive loss (MSCL), was created to retain the pre-trained features over transfer learning which were lost in previous anomaly detection methods. MSCL stops learning after a certain epoch as the points get closer to the normalized center. The proposed work investigated that due to some noise in the extracted features from images, the points revolve around the normalized center, and the accuracy freezes after some epochs. To resolve this issue, a new loss function is proposed which has better performance. The proposed methodology obtains a highly effective AUROC score of 98.6% and 97.6% on the CIFAR-100 and CIFAR-10 datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Chen, T.; Kornblith, S.; Norouzi, M.; Hinton, G.: A Simple Framework for Contrastive Learning of Visual Representations. arXiv:2002.05790v3 (2020).

  2. Tack, J.; Mo, S.; Jeong, J.; Shin, J.: CSI: Novelty Detection Via Contrastive Learning on Distributionally Shifted Instances. arXiv:2007.08176v2 (2020).

  3. Scholkopf, B.; Platt, J.C.; Shawe-Taylor, J.; Smola, A.J.; Williamson, R.C.: Estimating the support of a high dimensional distribution. Neural Comput. 13(7), 1443–1471 (2001)

    Article  CAS  PubMed  Google Scholar 

  4. Parzen, E.: On estimation of a probability density function and mode. Ann. Math. Stat. 33(3), 1065–1076 (1962)

    Article  MathSciNet  Google Scholar 

  5. Ruff, L.; Vandermeulen, R.A.; Gornitz, N.; Deecke, L.; Siddiqui, S.A.; Binder, A.; Mullera, E.; Kloft, M.: Deep one-class classification. In: \(35^{th}\) International Conference on Machine Learning, Stockholm, Sweden, PMLR 80. (2018)

  6. Reiss, T.; Cohen, N.; Bergman, L.; Hoshen, Y.: PANDA: Adapting Pretrained Features for Anomaly Detection and Segmentation. arXiv:2010.05903v3 (2021).

  7. Reiss, T.; Hoshen, Y.: Mean-Shifted Contrastive Loss for Anomaly Detection. arXiv:2106.03844v1 (2021).

  8. Aronszajn, N.: Theory of reproducing kernels. Trans. Am. Math. Soc. 68(3), 337–404 (1950)

    Article  MathSciNet  Google Scholar 

  9. Chen, T.; Kornblith, S.; Norouzi, M.; Hinton, G.: A simple framework for contrastive learning of visual representations. In: International Conference on Machine Learning (2020)

  10. Liang, S.; Li, Y.; Srikant, R.: Enhancing the reliability of out-of-distribution image detection in neural networks. In: The International Conference on Learning Representations (2018).

  11. Perera, P.; Patel, V.M.: Learning Deep Features for One-Class Classification. arXiv:1801.05365 (2019)

  12. He, K.; Zhang, X.; Ren, S.; Sun, J.: Deep Residual Learning for Image Recognition. arXiv:1512.03385 (2015).

  13. Hendrycks, D.; Mazeika, M.; Kadavath, S.; Song, D.: Using self-supervised learning can improve model robustness and uncertainty. In: NeurIPS (2019)

  14. Sohn, K.; Li, C.-L.; Yoon, J.; Jin, M.; Pfister, T.: Learning and Evaluating Representations for Deep One-Class Classification. arXiv:2011.02578 (2020)

  15. Wenpeng, H.; Wang, M.; Qin, Q.; Ma, J.; Liu, B.: Hrn: A holistic approach to one class learning. Adv. Neural Inf. Process. Syst. 33, 19111–19124 (2020)

  16. Hou, J.; Zhang, Y.; Zhong, Q.; Xie, D.; Pu, S.; Zhou, H.: Divide-and-assemble: learning block-wise memory for unsupervised anomaly detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 8791–8800 (2021)

  17. Salehi, M.; Sadjadi, N.; Baselizadeh, S.; Rohaban, M.H.; Rabiee, H.R.: Multiresolution knowledge distillation for anomaly detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 14902–14912 (2021)

  18. Deng, H.; Li, X.: Anomaly detection via reverse distillation from one-class embedding. In: IEEE/CVF Computer Vision and Pattern Recognition Conference (CVPR) (2022)

  19. Bergman, L.; Hoshen, Y.: Classification-based anomaly detection for general data. In: ICLR (2020).

  20. Bergmann, P.; Fauser, M.; Sattlegger, D.; Steger, C.: Mvtec ad-a comprehensive real-world dataset for unsupervised anomaly detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9592–9600 (2019)

  21. Chen, T.; Kornblith, S.; Norouzi, M.; Hinton, G.: A Simple Framework for Contrastive Learning of Visual Representations. arXiv:2002.05709 (2020)

  22. Chen, X.; He, K.: Exploring Simple Siamese Representation Learning. arXiv:2011.10566 (2020)

  23. Chen, X.; Fan, H.; Girshick, R.; He, K.: Improved Baselines with Momentum Contrastive Learning. arXiv:2003.04297 (2020)

  24. Gidaris, S.; Singh, P.; Komodakis, N.: Unsupervised Representation Learning by Predicting Image Rotations. arXiv:1803.07728 (2018)

  25. Bergmann, P.; Fauser, M.; Sattlegger, D.; Steger, C.: Uninformed students: student teacher anomaly detection with discriminative latent embeddings (2020)

  26. Goodfellow, I.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y.: Generative Adversarial Networks (2014)

  27. Selvaraju, R.R.; Cogswell, M; Das, A.; Vedantam, R.; Parikh, D.; Batra, D.: Grad-cam: Visual Explanations from Deep Networks Via Gradient-Based Localization (2017)

  28. Venkataramanan, S.; Peng, K.-C., Singh, R.V.; Mahalanobis, A.: Attention Guided Anomaly Localization in Images (2020)

  29. Elson, J.; Douceur, J.R.; Howell, J.; Saul, J.: Asirra: a captcha that exploits interest aligned manual image categorization. In: ACM Conference on Computer and Communications Security, vol. 7, pp. 366–374 (2007).

  30. Eskin, E.; Andrew, A.; Prerau, M.; Portnoy, L.; Stolfo, S.: A geometric framework for unsupervised anomaly detection. In: Applications of Data Mining in Computer Security, pp. 77–101. Springer (2002)

  31. Schmidhuber, J.: Deep learning in neural networks: an overview. Neural Netw. 61, 85–117 (2015)

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Science and Engineering Department, REC, Sonbhadra, UP, India

    Abhinav Maurya, Mainejar Yadav, Geetanjali Yadav, Peehu Saxena & Saksham Giri

Authors
  1. Abhinav Maurya
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mainejar Yadav
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Geetanjali Yadav
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Peehu Saxena
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Saksham Giri
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Abhinav Maurya.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Maurya, A., Yadav, M., Yadav, G. et al. A Regularization Factor-Based Approach to Anomaly Detection Using Contrastive Learning. Arab J Sci Eng 49, 3371–3381 (2024). https://doi.org/10.1007/s13369-023-07959-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13369-023-07959-7

Keywords

Search

Navigation