Detecting Anomalous Trajectories via Recurrent Neural Networks

  • Cong MaEmail author
  • Zhenjiang Miao
  • Min Li
  • Shaoyue Song
  • Ming-Hsuan Yang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11364)


Detecting anomalies from trajectory data is an important task in video surveillance. However, it is difficult to give a precise definition of this term since trajectory data obtained from different camera views may vary in shape, direction, and spatial distribution. In this paper, we propose trajectory distance metrics based on a recurrent neural network to measure similarities and detect anomalies from trajectory data. First, we use an autoencoder to capture the dynamic features of a trajectory. The distance between two trajectories is defined by the reconstruction errors based on the learned models. We then detect anomalies based on the nearest neighbors using the proposed metric. As such, we can deal with various kinds of anomalies in different scenes and detect anomalous trajectories in either a supervised or unsupervised manner. Experiments show that the proposed algorithm performs favorably against the state-of-the-art anomaly detections on the benchmark datasets.


  1. 1.
    Alt, H., Godau, M.: Computing the fréchet distance between two polygonal curves. Int. J. Comput. Geom. Appl. 5, 75–91 (1995)CrossRefGoogle Scholar
  2. 2.
    Berndt, D.J., Clifford, J.: Using dynamic time warping to find patterns in time series. In: Workshop on Knowledge Discovery in Databases (1994)Google Scholar
  3. 3.
    Besse, P., Guillouet, B., Loubes, J.M., François, R.: Review and perspective for distance based trajectory clustering. arXiv preprint arXiv:1508.04904 (2015)
  4. 4.
    Chen, L., Ng, R.: On the marriage of LP-norms and edit distance. In: International Conference on Very Large Data Bases (2004)Google Scholar
  5. 5.
    Chen, L., Ozsu, M.T., Oria, V.: Robust and fast similarity search for moving object trajectories. In: ACM SIGMOD International Conference on Management of Data (2005)Google Scholar
  6. 6.
    Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014)
  7. 7.
    Chong, Y.S., Tay, Y.H.: Abnormal event detection in videos using spatiotemporal autoencoder. arXiv preprint arXiv:1701.01546 (2017)
  8. 8.
    Chung, J., Gulcehre, C., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 (2014)
  9. 9.
    Ergezer, H., Leblebicioğlu, K.: Anomaly detection and activity perception using covariance descriptor for trajectories. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9914, pp. 728–742. Springer, Cham (2016). Scholar
  10. 10.
    Hasan, M., Choi, J., Neumann, J., Roy-Chowdhury, A.K., Davis, L.S.: Learning temporal regularity in video sequences. In: IEEE Conference on Computer Vision and Pattern Recognition (2016)Google Scholar
  11. 11.
    Hausdorff, F.: Grundz uge der mengenlehre (1914)Google Scholar
  12. 12.
    Hu, W., Li, X., Tian, G., Maybank, S., Zhang, Z.: An incremental DPMM-based method for trajectory clustering, modeling, and retrieval. IEEE Trans. Pattern Mach. Intell. 35(5), 1051–1065 (2013)CrossRefGoogle Scholar
  13. 13.
    Hu, W., Xiao, X., Fu, Z., Xie, D., Tan, T., Maybank, S.: A system for learning statistical motion patterns. IEEE Trans. Pattern Mach. Intell. 28(9), 1450–1464 (2006)CrossRefGoogle Scholar
  14. 14.
    Jiang, F., Wu, Y., Katsaggelos, A.K.: A dynamic hierarchical clustering method for trajectory-based unusual video event detection. IEEE Trans. Image Process. 18(4), 907–913 (2009)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Kumar, D., Bezdek, J.C., Rajasegarar, S., Leckie, C., Palaniswami, M.: A visual-numeric approach to clustering and anomaly detection for trajectory data. Vis. Comput. 33(3), 265–281 (2017)CrossRefGoogle Scholar
  16. 16.
    Laxhammar, R., Falkman, G.: Online learning and sequential anomaly detection in trajectories. IEEE Trans. Pattern Mach. Intell. 36(6), 1158–1173 (2014)CrossRefGoogle Scholar
  17. 17.
    Li, C., Han, Z., Ye, Q., Jiao, J.: Visual abnormal behavior detection based on trajectory sparse reconstruction analysis. Neurocomputing 119, 94–100 (2013)CrossRefGoogle Scholar
  18. 18.
    Lin, W., et al.: A tube-and-droplet-based approach for representing and analyzing motion trajectories. IEEE Trans. Pattern Mach. Intell. 39(8), 1489–1503 (2017)CrossRefGoogle Scholar
  19. 19.
    Morris, B.T., Trivedi, M.M.: Trajectory learning for activity understanding: Unsupervised, multilevel, and long-term adaptive approach. IEEE Trans. Pattern Mach. Intell. 33(11), 2287–2301 (2011)CrossRefGoogle Scholar
  20. 20.
    Piciarelli, C., Micheloni, C., Foresti, G.L.: Trajectory-based anomalous event detection. IEEE Trans. Circ. Syst. Video Technol. 18(11), 1544–1554 (2008)CrossRefGoogle Scholar
  21. 21.
    Porikli, F.: Trajectory distance metric using hidden Markov model based representation. In: ECCV 2004, PETS Workshop (2004)Google Scholar
  22. 22.
    Sillito, R.R., Fisher, R.B.: Semi-supervised learning for anomalous trajectory detection. In: British Machine Vision Conference (2008)Google Scholar
  23. 23.
    Srivastava, N., Mansimov, E., Salakhudinov, R.: Unsupervised learning of video representations using LSTMs. In: International Conference on Machine Learning (2015)Google Scholar
  24. 24.
    Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Neural Information Processing Systems (2014)Google Scholar
  25. 25.
    Vlachos, M., Kollios, G., Gunopulos, D.: Discovering similar multidimensional trajectories. In: IEEE International Conference on Data Engineering (2002)Google Scholar
  26. 26.
    Wen, L., et al.: UA-DETRAC: a new benchmark and protocol for multi-object detection and tracking. arXiv preprint arXiv:1511.04136 (2015)
  27. 27.
    Yang, H., Wang, B., Lin, S., Wipf, D., Guo, M., Guo, B.: Unsupervised extraction of video highlights via robust recurrent auto-encoders. In: IEEE International Conference on Computer Vision (2015)Google Scholar
  28. 28.
    Yankov, D., Keogh, E., Rebbapragada, U.: Disk aware discord discovery: finding unusual time series in terabyte sized datasets. In: IEEE International Conference on Data Mining, pp. 381–390 (2007)Google Scholar
  29. 29.
    Yao, D., Zhang, C., Zhu, Z., Huang, J., Bi, J.: Trajectory clustering via deep representation learning. In: International Joint Conference on Neural Networks (2017)Google Scholar
  30. 30.
    Zhang, Z., Huang, K., Tan, T.: Comparison of similarity measures for trajectory clustering in outdoor surveillance scenes. In: International Conference on Pattern Recognition (2006)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Cong Ma
    • 1
    Email author
  • Zhenjiang Miao
    • 1
  • Min Li
    • 1
  • Shaoyue Song
    • 1
  • Ming-Hsuan Yang
    • 2
  1. 1.Beijing Jiaotong UniversityBeijingChina
  2. 2.University of CaliforniaMercedUSA

Personalised recommendations