Multi-layered Decomposition of Recurrent Scenes

  • David Russell
  • Shaogang Gong
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5304)


There is considerable interest in techniques capable of identifying anomalies and unusual events in busy outdoor scenes, e.g. road junctions. Many approaches achieve this by exploiting deviations in spatial appearance from some expected norm accumulated by a model over time. In this work we show that much can be gained from explicitly modelling temporal aspects in detail. Specifically, many traffic junctions are regulated by lights controlled by a timing device of considerable precision, and it is in these situations that we advocate a model which learns periodic spatio-temporal patterns with a view to highlighting anomalous events such as broken-down vehicles, traffic accidents, or pedestrians jay-walking. More specifically, by estimating autocovariance of self-similarity, used previously in the context gait recognition, we characterize a scene by identifying a global fundamental period. As our model, we introduce a spatio-temporal grid of histograms built in accordance with some chosen feature. This model is then used to classify objects found in subsequent test data. In particular we demonstrate the effect of such characterization experimentally by monitoring the bounding box aspect ratio and optical flow field of objects detected on a road traffic junction, enabling our model to discriminate between people and cars sufficiently well to provide useful warnings of adverse behaviour in real time.


Fundamental Period Recurrence Plot Dynamic Background Motion History Image Scene Activity 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Stauffer, C., Grimson, W.: Adaptive background mixture models for real-time tracking. In: IEEE CVPR, Colorado, pp. 246–252 (1999)Google Scholar
  2. 2.
    Oliver, N., Rosario, B., Pentland, A.: A bayesian computer vision system for modelling human interactions. IEEE PAMI 22(8), 831–843 (2000)CrossRefGoogle Scholar
  3. 3.
    Natschläger, T., Ruf, B.: Spatial and temporal pattern analysis via spiking neurons. Network: Computation in Neural Systems 9(3), 319–332 (1998)CrossRefzbMATHGoogle Scholar
  4. 4.
    Ng, J., Gong, S.: On the binding mechanism of synchronised visual events. In: IEEE Workshop on Motion and Video Computing (December 2002)Google Scholar
  5. 5.
    Bobick, A.F., Davis, J.W.: The recognition of human movement using temporal templates. IEEE PAMI 23(3), 257–267 (2001)CrossRefGoogle Scholar
  6. 6.
    Xiang, T., Gong, S.: Beyond tracking: Modelling activity and understanding behaviour. IJCV 67(1), 21–51 (2006)CrossRefGoogle Scholar
  7. 7.
    Szummer, M.: Temporal texture modeling. Technical Report 346, MIT Media Lab Perceptual Computing (1995)Google Scholar
  8. 8.
    Liu, F., Picard, R.W.: Finding periodicity in space and time. In: ICCV, pp. 376–383 (1998)Google Scholar
  9. 9.
    Polana, R., Nelson, R.C.: Detection and recognition of periodic, nonrigid motion. IJCV 23(3), 261–282 (1997)CrossRefGoogle Scholar
  10. 10.
    Cutler, R., Davis, L.S.: Robust real-time periodic motion detection, analysis, and applications. IEEE PAMI 22(8), 781–796 (2000)CrossRefGoogle Scholar
  11. 11.
    Casdagli, M.: Recurrence plots revisited. Physica D 108, 12–44 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Boyd, J.: Synchronization of oscillations for machine perception of gaits. CVIU 96(1), 35–59 (2004)Google Scholar
  13. 13.
    Russell, D., Gong, S.: Minimum cuts of a time-varying background. In: BMVC, pp. 809–818 (September 2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • David Russell
    • 1
  • Shaogang Gong
    • 1
  1. 1.Department of Computer ScienceQueen Mary, University of LondonLondonUK

Personalised recommendations