Skip to main content
SpringerLink
Log in

ARES: Locally Adaptive Reconstruction-Based Anomaly Scoring

  • Conference paper
  • First Online:
Machine Learning and Knowledge Discovery in Databases (ECML PKDD 2022)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13713))

  • 1019 Accesses

Abstract

How can we detect anomalies: that is, samples that significantly differ from a given set of high-dimensional data, such as images or sensor data? This is a practical problem with numerous applications and is also relevant to the goal of making learning algorithms more robust to unexpected inputs. Autoencoders are a popular approach, partly due to their simplicity and their ability to perform dimension reduction. However, the anomaly scoring function is not adaptive to the natural variation in reconstruction error across the range of normal samples, which hinders their ability to detect real anomalies. In this paper, we empirically demonstrate the importance of local adaptivity for anomaly scoring in experiments with real data. We then propose our novel Adaptive Reconstruction Error-based Scoring approach, which adapts its scoring based on the local behaviour of reconstruction error over the latent space. We show that this improves anomaly detection performance over relevant baselines in a wide variety of benchmark datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/agoodge/ARES.

References

  1. Abati, D., Porrello, A., Calderara, S., Cucchiara, R.: Latent space autoregression for novelty detection. In: ICCV, pp. 481–490 (2019)

    Google Scholar 

  2. Akcay, S., Atapour-Abarghouei, A., Breckon, T.P.: GANomaly: semi-supervised anomaly detection via adversarial training. In: Jawahar, C.V., Li, H., Mori, G., Schindler, K. (eds.) ACCV 2018. LNCS, vol. 11363, pp. 622–637. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20893-6_39

    Chapter  Google Scholar 

  3. Amarbayasgalan, T., Jargalsaikhan, B., Ryu, K.H.: Unsupervised novelty detection using deep autoencoders with density based clustering. Appl. Sci. 8(9), 1468 (2018)

    Article  Google Scholar 

  4. An, J.: Variational autoencoder based anomaly detection using reconstruction probability. In: SNU Data Mining Center 2015–2 Special Lecture on IE (2015)

    Google Scholar 

  5. Bergman, L., Cohen, N., Hoshen, Y.: Deep nearest neighbor anomaly detection. arXiv preprint arXiv:2002.10445 (2020)

  6. Beyer, K., Goldstein, J., Ramakrishnan, R., Shaft, U.: When is “nearest neighbor’’ meaningful? In: Beeri, C., Buneman, P. (eds.) ICDT 1999. LNCS, vol. 1540, pp. 217–235. Springer, Heidelberg (1999). https://doi.org/10.1007/3-540-49257-7_15

    Chapter  Google Scholar 

  7. Bo, Z., Song, Q., Chen, H.: Deep autoencoding gaussian mixture model for unsupervised anomaly detection (2018)

    Google Scholar 

  8. Breunig, M.M., Kriegel, H.P., Ng, R.T., Sander, J.: LOF: identifying density-based local outliers. In: SIGMOD, vol. 29, pp. 93–104. ACM (2000)

    Google Scholar 

  9. Chen, J., Sathe, S., Aggarwal, C., Turaga, D.: Outlier detection with autoencoder ensembles. In: SDM, pp. 90–98. SIAM (2017)

    Google Scholar 

  10. Chen, Y., Zhou, X.S., Huang, T.S.: One-class SVM for learning in image retrieval. In: ICIP pp. 34–37. Citeseer (2001)

    Google Scholar 

  11. Deng, A., Goodge, A., Lang, Y.A., Hooi, B.: CADET: calibrated anomaly detection for mitigating hardness bias. In: IJCAI (2022)

    Google Scholar 

  12. Dinh, L., Sohl-Dickstein, J., Bengio, S.: Density estimation using real NVP. arXiv preprint arXiv:1605.08803 (2016)

  13. Feng, W., Han, C.: A novel approach for trajectory feature representation and anomalous trajectory detection. In: ISIF, pp. 1093–1099 (2015)

    Google Scholar 

  14. Goodge, A., Hooi, B., Ng, S.K., Ng, W.S.: Robustness of autoencoders for anomaly detection under adversarial impact. In: IJCAI (2020)

    Google Scholar 

  15. Goodge, A., Hooi, B., Ng, S.K., Ng, W.S.: Lunar: Unifying local outlier detection methods via graph neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence (2022)

    Google Scholar 

  16. inIT: Tool wear detection in CNC mill (2018). https://www.kaggle.com/init-owl/high-storage-system-data-for-energy-optimization

  17. Kim, K.H., et al.: RaPP: novelty detection with reconstruction along projection pathway. In: ICLR (2019)

    Google Scholar 

  18. Kirichenko, P., Izmailov, P., Wilson, A.G.: Why normalizing flows fail to detect out-of-distribution data. arXiv preprint arXiv:2006.08545 (2020)

  19. Lecun, Y.: Mnist (2012). http://yann.lecun.com/exdb/mnist/

  20. Liu, F.T., Ting, K.M., Zhou, Z.H.: Isolation-based anomaly detection. ACM Trans. Knowl. Discov. Data (TKDD) 6(1), 1–39 (2012)

    Article  Google Scholar 

  21. Maaten, L.V.D., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008)

    MATH  Google Scholar 

  22. Otto, G.: Otto group product classification challenge (2015). https://www.kaggle.com/c/otto-group-product-classification-challenge

  23. Papamakarios, G., Pavlakou, T., Murray, I.: Masked autoregressive flow for density estimation. In: NeurIPS, pp. 2338–2347 (2017)

    Google Scholar 

  24. Rezende, D.J., Mohamed, S.: Variational inference with normalizing flows. arXiv preprint arXiv:1505.05770 (2015)

  25. Ruff, L., et al.: Deep one-class classification. In: ICML, pp. 4393–4402 (2018)

    Google Scholar 

  26. Sakurada, M., Yairi, T.: Anomaly detection using autoencoders with nonlinear dimensionality reduction. In: MLSDA, p. 4. ACM (2014)

    Google Scholar 

  27. Shyu, M.L., Chen, S.C., Sarinnapakorn, K., Chang, L.: A novel anomaly detection scheme based on principal component classifier. Technical report, Miami Univ Coral Gables FL Dept of Electric and Computer Engineering (2003)

    Google Scholar 

  28. Siffer, A., Fouque, P.A., Termier, A., Largouet, C.: Anomaly detection in streams with extreme value theory. In: SIGKDD, pp. 1067–1075 (2017)

    Google Scholar 

  29. SMART: Tool wear detection in CNC mill (2018). https://www.kaggle.com/shasun/tool-wear-detection-in-cnc-mill

  30. Tax, D.M., Duin, R.P.: Support vector data description. Mach. Learn. 54(1), 45–66 (2004)

    Article  MATH  Google Scholar 

  31. UCI: Sensorless drive diagnosis (2015)

    Google Scholar 

  32. Xiao, H., Rasul, K., Vollgraf, R.: Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms (2017)

    Google Scholar 

  33. Zenati, H., Foo, C.S., Lecouat, B., Manek, G., Chandrasekha, V.R.: Efficient GAN-based anomaly detection (2019)

    Google Scholar 

  34. Zhai, S., Cheng, Y., Lu, W., Zhang, Z.: Deep structured energy based models for anomaly detection. ICML 48, 1100–1109 (2016)

    Google Scholar 

  35. Zimek, A., Gaudet, M., Campello, R.J.G.B., Sander, J.: Subsampling for efficient and effective unsupervised outlier detection ensembles. In: SIGKDD, pp. 428–436 (2013)

    Google Scholar 

Download references

Acknowledgements

This work was supported in part by NUS ODPRT Grant R252-000-A81-133.

Author information

Authors and Affiliations

  1. School of Computing, National University of Singapore, Singapore, Singapore

    Adam Goodge, Bryan Hooi & See Kiong Ng

  2. Institute of Data Science, National University of Singapore, Singapore, Singapore

    Bryan Hooi & See Kiong Ng

  3. Institute for Infocomm Research, A*STAR, Singapore, Singapore

    Adam Goodge & Wee Siong Ng

Authors
  1. Adam Goodge
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bryan Hooi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. See Kiong Ng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wee Siong Ng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Adam Goodge .

Editor information

Editors and Affiliations

  1. Grenoble Alpes University, Saint Martin d’Hères, France

    Massih-Reza Amini

  2. INSA Rouen Normandy, Saint Etienne du Rouvray, France

    Stéphane Canu

  3. Ruhr-Universität Bochum, Bochum, Germany

    Asja Fischer

  4. KU Leuven, Leuven, Belgium

    Tias Guns

  5. Central European University, Vienna, Austria

    Petra Kralj Novak

  6. Aristotle University of Thessaloniki, Thessaloniki, Greece

    Grigorios Tsoumakas

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 224 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Goodge, A., Hooi, B., Ng, S.K., Ng, W.S. (2023). ARES: Locally Adaptive Reconstruction-Based Anomaly Scoring. In: Amini, MR., Canu, S., Fischer, A., Guns, T., Kralj Novak, P., Tsoumakas, G. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2022. Lecture Notes in Computer Science(), vol 13713. Springer, Cham. https://doi.org/10.1007/978-3-031-26387-3_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-26387-3_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-26386-6

  • Online ISBN: 978-3-031-26387-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation