Key-Region Representation Learning for Anomaly Detection

  • Wenfei Yang
  • Bin LiuEmail author
  • Nenghai Yu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10666)


Anomaly detection and localization is of great importance for public safety monitoring. In this paper we focus on individual behavior anomaly detection, which remains a challenging problem due to complicated dynamics of video data. We try to solve this problem in a way based on feature extraction, we believe that patterns are easier to classify in feature space. However, different from many works in video analysis, we only extract features from small key-region patches, which allows our feature extraction module to have a simple architecture and be more targeted at anomaly detection. Our anomaly detection framework consists of three parts, the main part is an auto-encoder based representation learning module, and the other two parts, key-region extracting module and Mahalanobis distance based classifier, are specifically designed for anomaly detection in video. Our work has the following advantages: (1) our anomaly detection framework focus only on suspicious regions, and can detect anomalies with high accuracy and speed. (2) Our anomaly detection classifier has a stronger power to capture data distribution for anomaly detection.


Anomaly detection Auto-encoder Mahalanobis distance 



This work is supported by the National Natural Science Foundation of China (Grant No. 61371192), the Key Laboratory Foundation of the Chinese Academy of Sciences (CXJJ-17S044) and the Fundamental Research Funds for the Central Universities (WK2100330002).


  1. 1.
    Bertini, M., Del Bimbo, A., Seidenari, L.: Multi-scale and real-time non-parametric approach for anomaly detection and localization. Comput. Vis. Image Underst. 116(3), 320–329 (2012)CrossRefGoogle Scholar
  2. 2.
    Bottou, L.: Large-scale machine learning with stochastic gradient descent. In: Lechevallier, Y., Saporta, G. (eds.) Proceedings of COMPSTAT 2010, pp. 177–186. Springer, Heidelberg (2010). CrossRefGoogle Scholar
  3. 3.
    Cheng, K.W., Chen, Y.T., Fang, W.H.: Video anomaly detection and localization using hierarchical feature representation and Gaussian process regression. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2909–2917 (2015)Google Scholar
  4. 4.
    Chong, Y.S., Tay, Y.H.: Abnormal event detection in videos using spatiotemporal autoencoder. arXiv preprint arXiv:1701.01546 (2017)
  5. 5.
    Cong, Y., Yuan, J., Liu, J.: Sparse reconstruction cost for abnormal event detection. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3449–3456. IEEE (2011)Google Scholar
  6. 6.
    Hasan, M., Choi, J., Neumann, J., Roy-Chowdhury, A.K., Davis, L.S.: Learning temporal regularity in video sequences. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 733–742 (2016)Google Scholar
  7. 7.
    Hu, X., Hu, S., Zhang, X., Zhang, H., Luo, L.: Anomaly detection based on local nearest neighbor distance descriptor in crowded scenes. Sci. World J. 2014 (2014)Google Scholar
  8. 8.
    Leyva, R., Sanchez, V., Li, C.T.: Video anomaly detection with compact feature sets for online performance. IEEE Trans. Image Process. (2017)Google Scholar
  9. 9.
    Li, K.L., Huang, H.K., Tian, S.F., Xu, W.: Improving one-class SVM for anomaly detection. In: International Conference on Machine Learning and Cybernetics, vol. 5, pp. 3077–3081. IEEE (2003)Google Scholar
  10. 10.
    Li, W., Mahadevan, V., Vasconcelos, N.: Anomaly detection and localization in crowded scenes. IEEE Trans. Pattern Anal. Mach. Intell. 36(1), 18–32 (2014)CrossRefGoogle Scholar
  11. 11.
    Lu, C., Shi, J., Jia, J.: Abnormal event detection at 150 fps in MATLAB. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2720–2727 (2013)Google Scholar
  12. 12.
    Medel, J.R., Savakis, A.: Anomaly detection in video using predictive convolutional long short-term memory networks. arXiv preprint arXiv:1612.00390 (2016)
  13. 13.
    Mensink, T., Verbeek, J., Perronnin, F., Csurka, G.: Distance-based image classification: generalizing to new classes at near-zero cost. IEEE Trans. Pattern Anal. Mach. Intell. 35(11), 2624–2637 (2013)CrossRefGoogle Scholar
  14. 14.
    Popoola, O.P., Wang, K.: Video-based abnormal human behavior recognitiona review. IEEE Trans. Syst. Man. Cybern. Part C (Appl. Rev.) 42(6), 865–878 (2012)CrossRefGoogle Scholar
  15. 15.
    Sabokrou, M., Fathy, M., Hoseini, M., Klette, R.: Real-time anomaly detection and localization in crowded scenes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 56–62 (2015)Google Scholar
  16. 16.
    Sabokrou, M., Fayyaz, M., Fathy, M., et al.: Fully convolutional neural network for fast anomaly detection in crowded scenes. arXiv preprint arXiv:1609.00866 (2016)
  17. 17.
    Simonyan, K., Vedaldi, A., Zisserman, A.: Deep inside convolutional networks: visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034 (2013)
  18. 18.
    Vincent, P., Larochelle, H., Bengio, Y., Manzagol, P.A.: Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th International Conference on Machine Learning, pp. 1096–1103. ACM (2008)Google Scholar
  19. 19.
    Welford, B.: Note on a method for calculating corrected sums of squares and products. Technometrics 4(3), 419–420 (1962)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Wu, C., Guo, Y., Ma, Y.: Adaptive anomalies detection with deep network, pp. 181–186 (2015)Google Scholar
  21. 21.
    Xu, D., Ricci, E., Yan, Y., Song, J., Sebe, N.: Learning deep representations of appearance and motion for anomalous event detection. arXiv preprint arXiv:1510.01553 (2015)
  22. 22.
    Zhang, D., Lu, G.: Evaluation of similarity measurement for image retrieval. In: Proceedings of the 2003 International Conference on Neural Networks and Signal Processing, vol. 2, pp. 928–931. IEEE (2003)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Key Laboratory of Electromagnetic Space InformationChinese Academy of ScienceHefeiChina
  2. 2.School of Information Science and TechnologyUniversity of Science and Technology of ChinaHefeiChina

Personalised recommendations