Advertisement

Attention Guided Anomaly Localization in Images

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12362)

Abstract

Anomaly localization is an important problem in computer vision which involves localizing anomalous regions within images with applications in industrial inspection, surveillance, and medical imaging. This task is challenging due to the small sample size and pixel coverage of the anomaly in real-world scenarios. Most prior works need to use anomalous training images to compute a class-specific threshold to localize anomalies. Without the need of anomalous training images, we propose Convolutional Adversarial Variational autoencoder with Guided Attention (CAVGA), which localizes the anomaly with a convolutional latent variable to preserve the spatial information. In the unsupervised setting, we propose an attention expansion loss where we encourage CAVGA to focus on all normal regions in the image. Furthermore, in the weakly-supervised setting we propose a complementary guided attention loss, where we encourage the attention map to focus on all normal regions while minimizing the attention map corresponding to anomalous regions in the image. CAVGA outperforms the state-of-the-art (SOTA) anomaly localization methods on MVTec Anomaly Detection (MVTAD), modified ShanghaiTech Campus (mSTC) and Large-scale Attention based Glaucoma (LAG) datasets in the unsupervised setting and when using only 2% anomalous images in the weakly-supervised setting. CAVGA also outperforms SOTA anomaly detection methods on the MNIST, CIFAR-10, Fashion-MNIST, MVTAD, mSTC and LAG datasets.

Keywords

Guided attention Anomaly localization Convolutional adversarial variational autoencoder 

Notes

Acknowledgments

This work was done when Shashanka was an intern and Kuan-Chuan was a Staff Scientist at Siemens. Shashanka’s effort was partially supported by DARPA under Grant D19AP00032.

Supplementary material

504472_1_En_29_MOESM1_ESM.pdf (40.8 mb)
Supplementary material 1 (pdf 41783 KB)

References

  1. 1.
    Code for iterative energy-based projection on a normal data manifold for anomaly localization. https://qiita.com/kogepan102/items/122b2862ad5a51180656. Accessed 29 Feb 2020
  2. 2.
    Abati, D., Porrello, A., Calderara, S., Cucchiara, R.: Latent space autoregression for novelty detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 481–490 (2019)Google Scholar
  3. 3.
    Akcay, S., Atapour-Abarghouei, A., Breckon, T.P.: GANomaly: semi-supervised anomaly detection via adversarial training. In: Jawahar, C.V., Li, H., Mori, G., Schindler, K. (eds.) ACCV 2018. LNCS, vol. 11363, pp. 622–637. Springer, Cham (2019).  https://doi.org/10.1007/978-3-030-20893-6_39CrossRefGoogle Scholar
  4. 4.
    Baur, C., Wiestler, B., Albarqouni, S., Navab, N.: Deep autoencoding models for unsupervised anomaly segmentation in brain MR images. In: Crimi, A., Bakas, S., Kuijf, H., Keyvan, F., Reyes, M., van Walsum, T. (eds.) BrainLes 2018. LNCS, vol. 11383, pp. 161–169. Springer, Cham (2019).  https://doi.org/10.1007/978-3-030-11723-8_16CrossRefGoogle Scholar
  5. 5.
    Bergmann, P., Fauser, M., Sattlegger, D., Steger, C.: MVTec AD-a comprehensive real-world dataset for unsupervised anomaly detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9592–9600 (2019)Google Scholar
  6. 6.
    Bergmann, P., Löwe, S., Fauser, M., Sattlegger, D., Steger, C.: Improving unsupervised defect segmentation by applying structural similarity to autoencoders. In: International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP), vol. 5 (2019)Google Scholar
  7. 7.
    Bian, J., Hui, X., Sun, S., Zhao, X., Tan, M.: A novel and efficient CVAE-GAN-based approach with informative manifold for semi-supervised anomaly detection. IEEE Access 7, 88903–88916 (2019)CrossRefGoogle Scholar
  8. 8.
    Böttger, T., Ulrich, M.: Real-time texture error detection on textured surfaces with compressed sensing. Pattern Recogn. Image Anal. 26(1), 88–94 (2016)CrossRefGoogle Scholar
  9. 9.
    Brock, A., Donahue, J., Simonyan, K.: Large scale GAN training for high fidelity natural image synthesis. In: International Conference on Learning Representations (2019)Google Scholar
  10. 10.
    Cheng, K.W., Chen, Y.T., Fang, W.H.: Abnormal crowd behavior detection and localization using maximum sub-sequence search. In: Proceedings of the 4th ACM/IEEE International Workshop on Analysis and Retrieval of Tracked Events and Motion in Imagery Stream, pp. 49–58. ACM (2013)Google Scholar
  11. 11.
    Daniel, T., Kurutach, T., Tamar, A.: Deep variational semi-supervised novelty detection. arXiv preprint arXiv:1911.04971 (2019)
  12. 12.
    Deecke, L., Vandermeulen, R., Ruff, L., Mandt, S., Kloft, M.: Image anomaly detection with generative adversarial networks. In: Berlingerio, M., Bonchi, F., Gärtner, T., Hurley, N., Ifrim, G. (eds.) ECML PKDD 2018. LNCS (LNAI), vol. 11051, pp. 3–17. Springer, Cham (2019).  https://doi.org/10.1007/978-3-030-10925-7_1CrossRefGoogle Scholar
  13. 13.
    Dehaene, D., Frigo, O., Combrexelle, S., Eline, P.: Iterative energy-based projection on a normal data manifold for anomaly localization. In: International Conference on Learning Representations (2020)Google Scholar
  14. 14.
    Dieng, A.B., Kim, Y., Rush, A.M., Blei, D.M.: Avoiding latent variable collapse with generative skip models. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 2397–2405 (2019)Google Scholar
  15. 15.
    Dimokranitou, A.: Adversarial autoencoders for anomalous event detection in images. Ph.D. thesis (2017)Google Scholar
  16. 16.
    Gong, D., et al.: Memorizing normality to detect anomaly: memory-augmented deep autoencoder for unsupervised anomaly detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1705–1714 (2019)Google Scholar
  17. 17.
    Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)Google Scholar
  18. 18.
    Gutoski, M., Aquino, N.M.R., Ribeiro, M., Lazzaretti, E., Lopes, S.: Detection of video anomalies using convolutional autoencoders and one-class support vector machines. In: XIII Brazilian Congress on Computational Intelligence, 2017 (2017)Google Scholar
  19. 19.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  20. 20.
    Hendrycks, D., Mazeika, M., Dietterich, T.G.: Deep anomaly detection with outlier exposure. In: International Conference on Learning Representations (2019)Google Scholar
  21. 21.
    Higgins, I., et al.: beta-VAE: learning basic visual concepts with a constrained variational framework. In: International Conference on Learning Representations, vol. 2, no. 5, p. 6 (2017)Google Scholar
  22. 22.
    Kimura, D., Chaudhury, S., Narita, M., Munawar, A., Tachibana, R.: Adversarial discriminative attention for robust anomaly detection. In: The IEEE Winter Conference on Applications of Computer Vision (WACV), March 2020Google Scholar
  23. 23.
    Kingma, D.P., Welling, M.: Auto-encoding variational Bayes. In: International Conference on Learning Representations (2014)Google Scholar
  24. 24.
    Kiran, B., Thomas, D., Parakkal, R.: An overview of deep learning based methods for unsupervised and semi-supervised anomaly detection in videos. J. Imaging 4(2), 36 (2018)CrossRefGoogle Scholar
  25. 25.
    Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images. Technical report, Citeseer (2009)Google Scholar
  26. 26.
    Larsen, A.B.L., Sønderby, S.K., Larochelle, H., Winther, O.: Autoencoding beyond pixels using a learned similarity metric. In: International Conference on Machine Learning (2016)Google Scholar
  27. 27.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P., et al.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  28. 28.
    Li, K., Wu, Z., Peng, K.C., Ernst, J., Fu, Y.: Tell me where to look: guided attention inference network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9215–9223 (2018)Google Scholar
  29. 29.
    Li, L., Xu, M., Wang, X., Jiang, L., Liu, H.: Attention based glaucoma detection: a large-scale database and CNN model. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019Google Scholar
  30. 30.
    Li, X., Kiringa, I., Yeap, T., Zhu, X., Li, Y.: Exploring deep anomaly detection methods based on capsule net. In: International Conference on Machine Learning 2019 Workshop on Uncertainty and Robustness in Deep Learning (2019)Google Scholar
  31. 31.
    Liu, W., Luo, W., Lian, D., Gao, S.: Future frame prediction for anomaly detection-a new baseline. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6536–6545 (2018)Google Scholar
  32. 32.
    Liu, W., et al.: Towards visually explaining variational autoencoders. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2020)Google Scholar
  33. 33.
    Liu, Z., Luo, P., Wang, X., Tang, X.: Deep learning face attributes in the wild. In: Proceedings of International Conference on Computer Vision (ICCV), December 2015Google Scholar
  34. 34.
    Makhzani, A., Shlens, J., Jaitly, N., Goodfellow, I., Frey, B.: Adversarial autoencoders. In: International Conference on Learning Representations (2016)Google Scholar
  35. 35.
    Masana, M., Ruiz, I., Serrat, J., van de Weijer, J., Lopez, A.M.: Metric learning for novelty and anomaly detection. In: British Machine Vision Conference (BMVC) (2018)Google Scholar
  36. 36.
    Matteoli, S., Diani, M., Theiler, J.: An overview of background modeling for detection of targets and anomalies in hyperspectral remotely sensed imagery. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 7(6), 2317–2336 (2014)CrossRefGoogle Scholar
  37. 37.
    Napoletano, P., Piccoli, F., Schettini, R.: Anomaly detection in nanofibrous materials by CNN-based self-similarity. Sensors 18(1), 209 (2018)CrossRefGoogle Scholar
  38. 38.
    Nguyen, P., Liu, T., Prasad, G., Han, B.: Weakly supervised action localization by sparse temporal pooling network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6752–6761 (2018)Google Scholar
  39. 39.
    Oquab, M., Bottou, L., Laptev, I., Sivic, J.: Is object localization for free?-weakly-supervised learning with convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 685–694 (2015)Google Scholar
  40. 40.
    Pawlowski, N., et al.: Unsupervised lesion detection in brain CT using bayesian convolutional autoencoders. In: Medical Imaging with Deep Learning (2018)Google Scholar
  41. 41.
    Perera, P., Nallapati, R., Xiang, B.: OCGAN: one-class novelty detection using GANs with constrained latent representations. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2898–2906 (2019)Google Scholar
  42. 42.
    Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. In: International Conference on Learning Representations (2016)Google Scholar
  43. 43.
    Ravanbakhsh, M., Sangineto, E., Nabi, M., Sebe, N.: Training adversarial discriminators for cross-channel abnormal event detection in crowds. In: 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1896–1904. IEEE (2019)Google Scholar
  44. 44.
    Ruff, L., et al.: Deep semi-supervised anomaly detection. In: International Conference on Learning Representations (2020)Google Scholar
  45. 45.
    Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vision 115(3), 211–252 (2015)MathSciNetCrossRefGoogle Scholar
  46. 46.
    Sabokrou, M., Khalooei, M., Fathy, M., Adeli, E.: Adversarially learned one-class classifier for novelty detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3379–3388 (2018)Google Scholar
  47. 47.
    Sabokrou, M., et al.: AVID: adversarial visual irregularity detection. In: Jawahar, C.V., Li, H., Mori, G., Schindler, K. (eds.) ACCV 2018. LNCS, vol. 11366, pp. 488–505. Springer, Cham (2019).  https://doi.org/10.1007/978-3-030-20876-9_31CrossRefGoogle Scholar
  48. 48.
    Schlegl, T., Seeböck, P., Waldstein, S.M., Schmidt-Erfurth, U., Langs, G.: Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In: Niethammer, M., et al. (eds.) IPMI 2017. LNCS, vol. 10265, pp. 146–157. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-59050-9_12CrossRefGoogle Scholar
  49. 49.
    Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-CAM: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 618–626 (2017)Google Scholar
  50. 50.
    Smilkov, D., Thorat, N., Kim, B., Viégas, F., Wattenberg, M.: SmoothGrad: removing noise by adding noise. arXiv preprint arXiv:1706.03825 (2017)
  51. 51.
    Springenberg, J.T., Dosovitskiy, A., Brox, T., Riedmiller, M.: Striving for simplicity: the all convolutional net. arXiv preprint arXiv:1412.6806 (2014)
  52. 52.
    Steger, C.: Similarity measures for occlusion, clutter, and illumination invariant object recognition. In: Radig, B., Florczyk, S. (eds.) DAGM 2001. LNCS, vol. 2191, pp. 148–154. Springer, Heidelberg (2001).  https://doi.org/10.1007/3-540-45404-7_20CrossRefzbMATHGoogle Scholar
  53. 53.
    Vu, H.S., Ueta, D., Hashimoto, K., Maeno, K., Pranata, S., Shen, S.M.: Anomaly detection with adversarial dual autoencoders. arXiv preprint arXiv:1902.06924 (2019)
  54. 54.
    Wang, X., Xu, M., Li, L., Wang, Z., Guan, Z.: Pathology-aware deep network visualization and its application in glaucoma image synthesis. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11764, pp. 423–431. Springer, Cham (2019).  https://doi.org/10.1007/978-3-030-32239-7_47CrossRefGoogle Scholar
  55. 55.
    Wang, Z., Fan, M., Muknahallipatna, S., Lan, C.: Inductive multi-view semi-supervised anomaly detection via probabilistic modeling. In: 2019 IEEE International Conference on Big Knowledge (ICBK), pp. 257–264. IEEE (2019)Google Scholar
  56. 56.
    Wolf, L., Benaim, S., Galanti, T.: Unsupervised learning of the set of local maxima. In: International Conference on Learning Representations (2019)Google Scholar
  57. 57.
    Xiao, H., Rasul, K., Vollgraf, R.: Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747 (2017)
  58. 58.
    Zagoruyko, S., Komodakis, N.: Paying more attention to attention: improving the performance of convolutional neural networks via attention transfer. In: International Conference on Learning Representations (2017)Google Scholar
  59. 59.
    Zenati, H., Foo, C.S., Lecouat, B., Manek, G., Chandrasekhar, V.R.: Efficient GAN-based anomaly detection. arXiv preprint arXiv:1802.06222 (2018)
  60. 60.
    Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2921–2929 (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Center for Research in Computer VisionUniversity of Central FloridaOrlandoUSA
  2. 2.Mitsubishi Electric Research LaboratoriesCambridgeUSA
  3. 3.Siemens Corporate TechnologyPrincetonUSA

Personalised recommendations