Anomaly from Motion: Unsupervised Extraction of Visual Irregularity via Motion Prediction

  • Avishek MajumderEmail author
  • R. Venkatesh BabuEmail author
  • Anirban ChakrabortyEmail author
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 841)


The problem of automatically extracting anomalous events from any given video is a problem that has been researched from the early days of computer vision. It has still not been fully solved, showing that it is indeed not a trivial problem. The various challenges involved are lack of proper definition, varying scene structure and objects of interest in the scene, just a few to name.

In this paper we propose a novel method to extract outliers from motion alone. We employ a stacked LSTM encoder-decoder structure to model the regular motion patterns of the given video sequence. The discrepancy between the motion predicted using the model and the actual observed motion in the scene is analyzed to detect anomalous activities. We perform extensive experimentation on the benchmark datasets of crowd anomaly analysis. We report State of the Art results across all the datasets.


RNN Anomaly Crowded scenes LSTMs Video surveillance Machine learning Computer vision 



This work is supported by Science and Engineering Research Board (SERB), Department of Science and Technology (DST), Government of India (Proj No. SB/S3/EECE/0127/2015).


  1. 1.
    Adam, A., Rivlin, E., Shimshoni, I., Reinitz, D.: Robust real-time unusual event detection using multiple fixed-location monitors. IEEE Trans. Pattern Anal. Mach. Intell. 30(3), 555–560 (2008)CrossRefGoogle Scholar
  2. 2.
    Chaker, R., Al Aghbari, Z., Junejo, I.N.: Social network model for crowd anomaly detection and localization. Pattern Recognit. 61, 266–281 (2017)CrossRefGoogle Scholar
  3. 3.
    Chen, B., Marlin, B., de Freitas, N., et al.: Deep learning of invariant spatio-temporal features from video. In: Deep Learning and Unsupervised Feature Learning Workshop, NIPS 2010 (2010)Google Scholar
  4. 4.
    Chong, Y.S., Tay, Y.H.: Modeling representation of videos for anomaly detection using deep learning: a review. arXiv preprint arXiv:1505.00523 (2015)
  5. 5.
    Chong, Y.S., Tay, Y.H.: Abnormal event detection in videos using spatiotemporal autoencoder. In: Cong, F., Leung, A., Wei, Q. (eds.) ISNN 2017. LNCS, vol. 10262, pp. 189–196. Springer, Cham (2017). Scholar
  6. 6.
    Farnebäck, G.: Two-frame motion estimation based on polynomial expansion. In: Bigun, J., Gustavsson, T. (eds.) SCIA 2003. LNCS, vol. 2749, pp. 363–370. Springer, Heidelberg (2003). Scholar
  7. 7.
    Gers, F.A., Schmidhuber, J., Cummins, F.: Learning to forget: continual prediction with LSTM (1999)Google Scholar
  8. 8.
    Hasan, M., Choi, J., Neumann, J., Roy-Chowdhury, A.K., Davis, L.S.: Learning temporal regularity in video sequences. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 733–742 (2016)Google Scholar
  9. 9.
    Hu, X., Hu, S., Huang, Y., Zhang, H., Wu, H.: Video anomaly detection using deep incremental slow feature analysis network. IET Comput. Vis. 10(4), 258–265 (2016)CrossRefGoogle Scholar
  10. 10.
    Kim, J., Grauman, K.: Observe locally, infer globally: a space-time MRF for detecting abnormal activities with incremental updates. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 2921–2928. IEEE (2009)Google Scholar
  11. 11.
    Kozlov, Y., Weinkauf, T.: Persistence1D: extracting and filtering minima and maxima of 1D functions (2015). Accessed 11 Jan 2015
  12. 12.
    Lu, C., Shi, J., Jia, J.: Abnormal event detection at 150 FPS in MATLAB. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2720–2727 (2013)Google Scholar
  13. 13.
    Mahadevan, V., Li, W., Bhalodia, V., Vasconcelos, N.: Anomaly detection in crowded scenes. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 1975–1981. IEEE (2010)Google Scholar
  14. 14.
    Medel, J.R., Savakis, A.: Anomaly detection in video using predictive convolutional long short-term memory networks. arXiv preprint arXiv:1612.00390 (2016)
  15. 15.
    Mehran, R., Oyama, A., Shah, M.: Abnormal crowd behavior detection using social force model. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 935–942. IEEE (2009)Google Scholar
  16. 16.
    Mousavi, H., Nabi, M., Kiani, H., Perina, A., Murino, V.: Crowd motion monitoring using tracklet-based commotion measure. In: 2015 IEEE International Conference on Image Processing, ICIP, pp. 2354–2358. IEEE (2015)Google Scholar
  17. 17.
    Ravanbakhsh, M., Nabi, M., Mousavi, H., Sangineto, E., Sebe, N.: Plug-and-play CNN for crowd motion analysis: an application in abnormal event detection. arXiv preprint arXiv:1610.00307 (2016)
  18. 18.
    Wu, S., Moore, B.E., Shah, M.: Chaotic invariants of Lagrangian particle trajectories for anomaly detection in crowded scenes. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 2054–2060. IEEE (2010)Google Scholar
  19. 19.
    Xu, D., Song, J., Yan, Y., Ricci, E., Sebe, N., et al.: Learning deep representations of appearance and motion for anomalous event detection. In: BMVC. BMVA Press (2015)Google Scholar
  20. 20.
    Yuan, Y., Fang, J., Wang, Q.: Online anomaly detection in crowd scenes via structure analysis. IEEE Trans. Cybern. 45(3), 548–561 (2015)CrossRefGoogle Scholar
  21. 21.
    Zhou, S., Shen, W., Zeng, D., Fang, M., Wei, Y., Zhang, Z.: Spatial-temporal convolutional neural networks for anomaly detection and localization in crowded scenes. Signal Process. Image Commun. 47, 358–368 (2016)CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  1. 1.Indian Institute of ScienceBangaloreIndia

Personalised recommendations