The Visual Computer

, Volume 33, Issue 3, pp 265–281 | Cite as

A visual-numeric approach to clustering and anomaly detection for trajectory data

  • Dheeraj Kumar
  • James C. Bezdek
  • Sutharshan Rajasegarar
  • Christopher Leckie
  • Marimuthu Palaniswami
Original Article


This paper proposes a novel application of Visual Assessment of Tendency (VAT)-based hierarchical clustering algorithms (VAT, iVAT, and clusiVAT) for trajectory analysis. We introduce a new clustering based anomaly detection framework named iVAT+ and clusiVAT+ and use it for trajectory anomaly detection. This approach is based on partitioning the VAT-generated Minimum Spanning Tree based on an efficient thresholding scheme. The trajectories are classified as normal or anomalous based on the number of paths in the clusters. On synthetic datasets with fixed and variable numbers of clusters and anomalies, we achieve 98 % classification accuracy. Our two-stage clusiVAT method is applied to 26,039 trajectories of vehicles and pedestrians from a parking lot scene from the real life MIT trajectories dataset. The first stage clusters the trajectories ignoring directionality. The second stage divides the clusters obtained from the first stage by considering trajectory direction. We show that our novel two-stage clusiVAT approach can produce natural and informative trajectory clusters on this real life dataset while finding representative anomalies.


Trajectory clustering Anomaly detection ClusiVAT hierarchical clustering MIT trajectory dataset 



We acknowledge the support from the Australian Research Council (ARC) Linkage Project grant (LP120100529), the ARC Linkage Infrastructure, Equipment and Facilities scheme (LIEF) grant (LF120100129), the EU-FP7 SOCIOTAL grant and National ICT Australia (NICTA).


  1. 1.
    Synthetic trajectory dataset. Accessed 23 Dec 2014
  2. 2.
    Ali, I., Dailey, M.N.: Multiple human tracking in high-density crowds. Image Vis. Comput. 30(12), 966–977 (2012)CrossRefGoogle Scholar
  3. 3.
    Arandjelović, O.: Contextually learnt detection of unusual motion-based behaviour in crowded public spaces. In: Computer and Information Sciences II, pp 403–410. Springer, London (2012)Google Scholar
  4. 4.
    Benezeth, Y., Jodoin, P., Saligrama, V., Rosenberger, C.: Abnormal events detection based on spatio-temporal co-occurences. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2009, pp. 2458–2465 (2009)Google Scholar
  5. 5.
    Bezdek, J., Hathaway, R.: VAT: a tool for visual assessment of (cluster) tendency. In: International Joint Conference on Neural Networks (IJCNN), pp. 2225–2230 (2002)Google Scholar
  6. 6.
    Bousetouane, F., Dib, L., Snoussi, H.: Improved mean shift integrating texture and color features for robust real time object tracking. Vis. Comput. 29(3), 155–170 (2013)CrossRefGoogle Scholar
  7. 7.
    Brun, L., Saggese, A., Vento, M.: Learning and classification of car trajectories in road video by string kernels. Int. Conf. Comput. Vis. Theory Appl. 1, 709–714 (2013)Google Scholar
  8. 8.
    Chan, A., Mahadevan, V., Vasconcelos, N.: Generalized stauffer-grimson background subtraction for dynamic scenes. Mach. Vis. Appl. 22(5), 751–766 (2011)CrossRefGoogle Scholar
  9. 9.
    Cong, Y., Yuan, J., Liu, J.: Sparse reconstruction cost for abnormal event detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2011, pp. 3449–3456 (2011)Google Scholar
  10. 10.
    Cong, Y., Yuan, J., Tang, Y.: Video anomaly search in crowded scenes via spatio-temporal motion context. IEEE Trans. Inf. Forensics Secur. 8(10), 1590–1599 (2013)CrossRefGoogle Scholar
  11. 11.
    Cui, X., Liu, Q., Gao, M., Metaxas, D.: Abnormal detection using interaction energy potentials. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2011, pp. 3161–3167 (2011)Google Scholar
  12. 12.
    Fu, Z., Hu, W., Tan, T.: Similarity based vehicle trajectory clustering and anomaly detection. In: IEEE International Conference on Image Processing (ICIP), Sept 2005, vol. 2, pp. II-602–605 (2005)Google Scholar
  13. 13.
    Hathaway, R., Bezdek, J., Huband, J.: Scalable visual assessment of cluster tendency for large data sets. Pattern Recognit. 39, 1315–1324 (2006)CrossRefzbMATHGoogle Scholar
  14. 14.
    Havens, T., Bezdek, J.: An efficient formulation of the improved visual assessment of cluster tendency (iVAT) algorithm. IEEE Trans. Knowl. Data Eng. 24(5), 813–822 (2012)CrossRefGoogle Scholar
  15. 15.
    Hoare, C.A.R.: Algorithm 64: Quicksort. Commun. ACM 4(7), 321 (1961)CrossRefGoogle Scholar
  16. 16.
    Jiang, F., Wu, Y., Katsaggelos, A.: A dynamic hierarchical clustering method for trajectory-based unusual video event detection. IEEE Trans. Image Process. 18(4), 907–913 (2009)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Jiang, F., Yuan, J., Tsaftaris, S.A., Katsaggelos, A.K.: Anomalous video event detection using spatiotemporal context. Comput. Vis. Image Underst. 115(3), 323–333 (2011)CrossRefGoogle Scholar
  18. 18.
    Keogh, E., Lin, J., Fu, A.: Hot sax: Efficiently finding the most unusual time series subsequence. In: IEEE International Conference on Data Mining, ICDM ’05, pp. 226–233 (2005)Google Scholar
  19. 19.
    Kim, J., Grauman, K.: Observe locally, infer globally: A space-time mrf for detecting abnormal activities with incremental updates. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2009, pp. 2921–2928 (2009)Google Scholar
  20. 20.
    Kumar, D. Bezdek, J., Palaniswami, M., Rajasegarar, S., Leckie, C., Havens, T.: A hybrid approach to clustering in big data. IEEE Trans. Cybern. PP(99):1 (2015)Google Scholar
  21. 21.
    Kumar, D., Palaniswami, M., Rajasegarar, S., Leckie, C., Bezdek, J., Havens, T.: clusiVAT: A mixed visual/numerical clustering algorithm for big data. In: IEEE International Conference on Big Data, Oct 2013, pp. 112–117Google Scholar
  22. 22.
    Laxhammar, R., Falkman, G.: Sequential conformal anomaly detection in trajectories based on hausdorff distance. In: International Conference on Information Fusion (FUSION), July 2011, pp. 1–8 (2011)Google Scholar
  23. 23.
    Laxhammar, R., Falkman, G.: Online learning and sequential anomaly detection in trajectories. IEEE Trans. Pattern Anal. Mach. Intell. 36(6), 1158–1173 (2014)CrossRefzbMATHGoogle Scholar
  24. 24.
    Li, W., Mahadevan, V., Vasconcelos, N.: Anomaly detection and localization in crowded scenes. IEEE Trans. Pattern Anal. Mach. Intell. 36(1), 18–32 (2014)CrossRefGoogle Scholar
  25. 25.
    Li, X., Han, J., Kim, S., Gonzalez, H.: ROAM: rule and motif-based anomaly detection in massive moving object data sets. In: SIAM International Conference on Data Mining (2007)Google Scholar
  26. 26.
    Martin, R., Arandjelović, O.: Multiple-object tracking in cluttered and crowded public spaces. In: Bebis, G., Boyle, R., Parvin, B., Koracin, D., Chung, R., Hammound, R., Hussain, M., Kar-Han, T., Crawfis, R., Thalmann, D., Kao, D., Avila, L. (eds.) Advances in Visual Computing, Lecture Notes in Computer Science, vol. 6455, pp. 89–98. Springer, Berlin, Heidelberg (2010)Google Scholar
  27. 27.
    Mehran, R., Oyama, A., Shah, M.: Abnormal crowd behavior detection using social force model. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2009, pp. 935–942 (2009)Google Scholar
  28. 28.
    Morris, B., Trivedi, M.: A survey of vision-based trajectory learning and analysis for surveillance. IEEE Trans. Circuits Syst. Video Technol. 18(8), 1114–1127 (2008)CrossRefGoogle Scholar
  29. 29.
    Morris, B., Trivedi, M.: Learning trajectory patterns by clustering: Experimental studies and comparative evaluation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2009, pp. 312–319 (2009)Google Scholar
  30. 30.
    Naftel, A., Khalid, S.: Classifying spatiotemporal object trajectories using unsupervised learning in the coefficient feature space. Multimed. Syst. 12(3), 227–238 (2006)CrossRefGoogle Scholar
  31. 31.
    Pham, D.S., Arandjelovic, O., Venkatesh, S.: Detection of dynamic background due to swaying movements from motion features. IEEE Trans. Image Process. 24(1), 332–344 (2015)MathSciNetCrossRefGoogle Scholar
  32. 32.
    Piciarelli, C., Foresti, G.: Anomalous trajectory detection using support vector machines. In: IEEE Conference on Advanced Video and Signal Based Surveillance (AVSS), Sept 2007, pp. 153–158 (2007)Google Scholar
  33. 33.
    Piciarelli, C., Micheloni, C., Foresti, G.: Trajectory-based anomalous event detection. IEEE Trans. Circuits Syst. Video Technol. 18(11), 1544–1554 (2008)CrossRefGoogle Scholar
  34. 34.
    Rabiner, L., Juang, B.H.: Fundamentals of Speech Recognition. Prentice-Hall Inc, Upper Saddle River (1993)zbMATHGoogle Scholar
  35. 35.
    Rao, Y.: Automatic vehicle recognition in multiple cameras for video surveillance. Vis. Comput. 31(3), 271–280 (2015)CrossRefGoogle Scholar
  36. 36.
    Rodriguez, M., Sivic, J., Laptev, I. and Audibert, J.Y.: Density-aware person detection and tracking in crowds. In: International Conference on Computer Vision (ICCV) (2011)Google Scholar
  37. 37.
    Roshtkhari, M., Levine, M.: Online dominant and anomalous behavior detection in videos. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2013, pp. 2611–2618 (2013)Google Scholar
  38. 38.
    Saleemi, I., Shafique, K., Shah, M.: Probabilistic modeling of scene dynamics for applications in visual surveillance. IEEE Trans. Pattern Anal. Mach. Intell. 31(8), 1472–1485 (2009)CrossRefGoogle Scholar
  39. 39.
    Saligrama, V., Konrad, J., Jodoin, P.: Video anomaly identification. IEEE Signal Process. Mag. 27(5), 18–33 (2010)CrossRefGoogle Scholar
  40. 40.
    Salvador, S., Chan, P.: Toward accurate dynamic time warping in linear time and space. Intell. Data Anal. 11(5), 561–580 (2007)Google Scholar
  41. 41.
    Vishwakarma, S., Agrawal, A.: A survey on activity recognition and behavior understanding in video surveillance. Vis. Comput. 29(10), 983–1009 (2013)CrossRefGoogle Scholar
  42. 42.
    Wang, X., Ma, K., Ng, G.W., Grimson, W.: Trajectory analysis and semantic region modeling using nonparametric hierarchical bayesian models. Int. J. Comput. Vis. 95(3), 287–312 (2011)CrossRefGoogle Scholar
  43. 43.
    Wang, X., Ma, K.T., Ng, G.W., Grimson, W.: Trajectory analysis and semantic region modeling using a nonparametric bayesian model. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2008, pp. 1–8 (2008)Google Scholar
  44. 44.
    Zhao, B., Fei, L.F., Xing, E.P.: Online detection of unusual events in videos via dynamic sparse coding. In; IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3313–3320. IEEE Computer Society (2011)Google Scholar
  45. 45.
    Zhou, Y., Yan, S., Huang, T.: Detecting anomaly in videos from trajectory similarity analysis. In: IEEE International Conference on Multimedia and Expo, July 2007, pp. 1087–1090 (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  • Dheeraj Kumar
    • 1
  • James C. Bezdek
    • 2
  • Sutharshan Rajasegarar
    • 2
    • 3
  • Christopher Leckie
    • 2
    • 3
  • Marimuthu Palaniswami
    • 1
  1. 1.Department of Electrical and Electronic EngineeringThe University of MelbourneMelbourneAustralia
  2. 2.Department of Computing and Information SystemsThe University of MelbourneMelbourneAustralia
  3. 3.National ICT AustraliaMelbourneAustralia

Personalised recommendations