Pattern Analysis and Applications

, Volume 21, Issue 3, pp 829–846 | Cite as

Unsupervised detection of ruptures in spatial relationships in video sequences based on log-likelihood ratio

  • Abdalbassir Abou-ElailahEmail author
  • Isabelle Bloch
  • Valerie Gouet-Brunet
Industrial and Commercial Application


In this work, we propose a new approach to automatically detect ruptures in spatial relationships in video sequences, based on low-level primitives, in unsupervised manner. The spatial relationships between two objects of interest are modeled using angle and distance histograms as examples. The evolution of the spatial relationships during time is estimated from the distances between two successive angle or distance histograms and then considered as a temporal signal. The evolution of a spatial relationship is modeled by a linear Gaussian model. Then, two hypotheses “without change” and “with change” are considered, and a log-likelihood ratio is computed. The distribution of the log-likelihood ratio, given that \(H_0\) is true, is estimated and used to compute the p value. The comparison of this p value to a significance level \(\alpha \), expressing the probability of false alarms, allows us to detect significant ruptures in spatial relationships during time. In addition, this approach is generalized to detect multiple object events such as merging, splitting, and other events that contain ruptures in their spatial relationships evolution. This work shows that the description of spatial relationships across time is a promising step toward event detection.


Spatial relationships Distances between histograms Detection of ruptures Hypotheses testing Log-likelihood ratio Significance level 


  1. 1.
    Visam Project (1997)
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
    Piciarelli C, Micheloni C, Foresti G (2008) Trajectory-based anomalous event detection. IEEE Trans Circ Syst Video Technol 18:1544–1554CrossRefGoogle Scholar
  9. 9.
    Saleemi I, Shafique K, Shah M (2009) Probabilistic modeling of scene dynamics for applications in visual surveillance. IEEE Trans Pattern Anal Mach Intell 31(8):1472–1485CrossRefGoogle Scholar
  10. 10.
    Hu W, Xiao X, Fu Z, Xie D, Tan T, Maybank S (2006) A system for learning statistical motion patterns. IEEE Trans Pattern Anal Mach Intell 28:1450–1464CrossRefGoogle Scholar
  11. 11.
    Wang T, Snoussi H (2014) Detection of abnormal visual events via global optical flow orientation histogram. IEEE Trans Inf Forensics Secur 9(6):988–998CrossRefGoogle Scholar
  12. 12.
    Li A, Miao Z, Cen Y, Wang T, Voronin V (2015) Histogram of maximal optical flow projection for abnormal events detection in crowded scenes. Int J Distrib Sens Netw 11:1–11Google Scholar
  13. 13.
    Adam A, Rivlin E, Shimshoni I, Reinitz D (2008) Robust real-time unusual event detection using multiple fixed-location monitors. IEEE Trans Pattern Anal Mach Intell 30:555–560CrossRefGoogle Scholar
  14. 14.
    Kratz L, Nishino K (2009) Anomaly detection in extremely crowded scenes using spatio-temporal motion pattern models. In: IEEE conference on computer vision and pattern recognition, pp 1446–1453Google Scholar
  15. 15.
    Jiang F, Wu Y, Katsaggelos AK (2009) Detecting contextual anomalies of crowd motion in surveillance video. In: 16th IEEE international conference on image processing, pp 1117–1120Google Scholar
  16. 16.
    Mehran R, Oyama A, Shah M (2009) Abnormal crowd behavior detection using social force model. In: IEEE conference on computer vision and pattern recognition, pp 935–942Google Scholar
  17. 17.
    Cong Y, Yuan J, Tang Y (2013) Video anomaly search in crowded scenes via spatio-temporal motion context. IEEE Trans Inf Forensics Secur 8:1590–1599CrossRefGoogle Scholar
  18. 18.
    Cong Y, Yuan J, Liu J (2013) Abnormal event detection in crowded scenes using sparse representation. Pattern Recognit 46:1851–1864CrossRefGoogle Scholar
  19. 19.
    Hu X, Hu S, Zhang X, Zhang H, Luo L (2014) Anomaly detection based on local nearest neighbor distance descriptor in crowded scenes. Sci World J 2014:1–12 Google Scholar
  20. 20.
    Tran D, Yuan J, Forsyth D (2014) Video event detection: from subvolume localization to spatio-temporal path search. IEEE Trans Pattern Anal Mach Intell 36(12):404–416CrossRefGoogle Scholar
  21. 21.
    Dan DX, Ricci E, Yan Y, Song J, Sebe N (2015) Learning deep representations of appearance and motion for anomalous event detection. British Machine Vision ConferenceGoogle Scholar
  22. 22.
    Ren H, Liu W, Olsen SI, Escalera S, Moeslund TB (2015) Unsupervised behavior-specific dictionary learning for abnormal event detection. British Machine Vision Conference, pp 1–28Google Scholar
  23. 23.
    Lu C, Shi J, Jia J (2013) Abnormal event detection at 150 fps in matlab. In: IEEE international conference on computer vision, pp 2720–2727Google Scholar
  24. 24.
    Zhao B, Fei-Fei L, Xing E (2001) Online detection of unusual events in videos via dynamic sparse coding. In: IEEE conference on computer vision and pattern recognition, pp 3313–3320Google Scholar
  25. 25.
    Cheng K, Chen Y, Fang W (2015) Video anomaly detection and localization using hierarchical feature representation and Gaussian process regression. In: IEEE conference on computer vision and pattern recognition, pp 2909–2917Google Scholar
  26. 26.
    Basharat A, Gritai A, Shah M (2008) Learning object motion patterns for anomaly detection and improved object detection. In: IEEE conference on computer vision and pattern recognition, pp 1–8Google Scholar
  27. 27.
    Solmaz B, Moore B, Shah M (2012) Identifying behaviors in crowd scenes using stability analysis for dynamical systems. IEEE Trans Pattern Anal Mach Intell 34:2064–2070CrossRefGoogle Scholar
  28. 28.
    Saleemi I, Hartung L, Shah M (2010) Scene understanding by statistical modeling of motion patterns. In: IEEE conference on computer vision and pattern recognition, pp 2069–2076Google Scholar
  29. 29.
    Tzelepis C, Mezaris V, Patras I (2016) Video event detection using kernel support vector machine with isotropic gaussian sample uncertainty KSVM-iGSU. In: International conference on multimedia modeling, pp 3–15Google Scholar
  30. 30.
    Mazloom M, Li X, Snoek CG (2016) TagBook: a semantic video representation without supervision for event detection. IEEE Trans Multimed 18:1378–1388CrossRefGoogle Scholar
  31. 31.
    Li Z, Liu J, Tang J, Lu H (2015) Robust structured subspace learning for data representation. IEEE Trans Pattern Anal Mach Intell 37:2085–2098CrossRefGoogle Scholar
  32. 32.
    Li Z, Tang J (2015) Weakly supervised deep metric learning for community-contributed image retrieval. IEEE Trans Multimed 17:1989–1999CrossRefGoogle Scholar
  33. 33.
    Abou-Elailah A, Gouet-Brunet V, Bloch I (2015) Detection of ruptures in spatial relationships in video sequences. In: International conference on pattern recognition applications and methods, pp 110–120Google Scholar
  34. 34.
    Tissainayagam P, Suter D (2005) Object tracking in image sequences using point features. Pattern Recognit 38:105–113CrossRefGoogle Scholar
  35. 35.
    Zhou H, Yuan Y, Shi C (2009) Object tracking using SIFT features and mean shift. Comput Vis Image Underst 113(3):345–352CrossRefGoogle Scholar
  36. 36.
    Miyajima K, Ralescu A (1994) Spatial organization in 2D images. In: Third IEEE conference on fuzzy systems, pp 100–105Google Scholar
  37. 37.
    Hafner J, Sawhney H, Equitz W, Flickner M, Niblack W (1995) Efficient color histogram indexing for quadratic form distance functions. IEEE Trans Pattern Anal Mach Intell 17:729–736CrossRefGoogle Scholar
  38. 38.
    Bloch I, Atif J (2015) Hausdorff distances between distributions using optimal transport and mathematical morphology. In: Mathematical morphology and its applications to signal and image processing, pp 522–534Google Scholar
  39. 39.
    Bloch I, Atif J (2016) Defining and computing Hausdorff distances between distributions on the real line and on the circle: link between optimal transport and morphological dilations. Math Morphol Theory Appl 1:79–99Google Scholar
  40. 40.
    Zhang L, van der Maaten L (2013) Structure preserving object tracking. In: IEEE conference on computer vision and pattern recognition, pp 1838–1845Google Scholar
  41. 41.
    Widynski N, Dubuisson S, Bloch I (2012) Fuzzy spatial constraints and ranked partitioned sampling approach for multiple object tracking. Comput Vis Image Underst 116:1076–1094CrossRefGoogle Scholar
  42. 42.
    Morimitsu H, Roberto M, Bloch I (2014) A spatio-temporal approach for multiple object detection in videos using graphs and probability maps. In: International conference on image analysis and recognition, pp 421–428Google Scholar
  43. 43.
    Morimitsu H, Bloch I, Cesar RM (2017) Exploring structure for long-term tracking of multiple objects in sports videos. Comput Vis Image Underst 159:89–104CrossRefGoogle Scholar
  44. 44.
    Harris C, Stephens M (1988) A combined corner and edge detector. In: Fourth Alvey vision conference, pp 147–151Google Scholar
  45. 45.
    Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60:91–110CrossRefGoogle Scholar
  46. 46.
    Basseville M, Nikiforov IV (1993) Detection of abrupt changes: theory and application. Prentice Hall, Englewood Cliffs, p 104Google Scholar
  47. 47.
  48. 48.
  49. 49.
    Bazzani L, Cristani M, Murino V (2012) Decentralized particle filter for joint individual-group tracking. In: IEEE conference on computer vision and pattern recognition, pp 1886–1893Google Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2017

Authors and Affiliations

  • Abdalbassir Abou-Elailah
    • 1
    Email author
  • Isabelle Bloch
    • 1
  • Valerie Gouet-Brunet
    • 2
  1. 1.LTCI, Télécom ParisTechUniversité Paris-SaclayParisFrance
  2. 2.LaSTIG MATIS, IGN, ENSGUniv. Paris-EstSaint-MandeFrance

Personalised recommendations