Modelling Scenes Using the Activity within Them

  • Hannah M. Dee
  • Roberto Fraile
  • David C. Hogg
  • Anthony G. Cohn
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5248)


This paper describes a method for building visual “maps” from video data using quantized descriptions of motion. This enables unsupervised classification of scene regions based upon the motion patterns observed within them. Our aim is to recognise generic places using a qualitative representation of the spatial layout of regions with common motion patterns. Such places are characterised by the distribution of these motion patterns as opposed to static appearance patterns, and could include locations such as train platforms, bus stops, and park benches. Motion descriptions are obtained by tracking image features over a temporal window, and are then subjected to normalisation and thresholding to provide a quantized representation of that feature’s gross motion. Input video is quantized spatially into N ×N pixel blocks, and a histogram of the frequency of occurrence of each vector is then built for each of these small areas of scene. Within these we can therefore characterise the dominant patterns of motion, and then group our spatial regions based upon both proximity and local motion similarity to define areas or regions with particular motion characteristics. Moving up a level we then consider the relationship between the motion in adjacent spatial areas, and can characterise the dominant patterns of motion expected in a particular part of the scene over time. The current paper differs from previous work which has largely been based on the paths of moving agents, and therefore restricted to scenes in which such paths are identifiable. We demonstrate our method in three very different scenes: an indoor room scenario with multiple chairs and unpredictable unconstrained motion, an underground station featuring regions where motion is constrained (train tracks) and regions with complicated motion and difficult occlusion relationships (platform), and an outdoor scene with challenging camera motion and partially overlapping video streams.


Learning Spatial relations Computer vision Modeling behaviour Qualitative reasoning 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Fernyhough, J.H., Cohn, A.G., Hogg, D.C.: Generation of semantic regions from image sequences. In: Proc. European Conference on Computer Vision (ECCV), Cambridge, UK, pp. 475–484 (1996)Google Scholar
  2. 2.
    Laptev, I.: On space-time interest points. Journal of Computer Vision 64(2/3), 107–123 (2005)CrossRefGoogle Scholar
  3. 3.
    Johnson, N., Hogg, D.C.: Learning the distribution of object tractories for event recognition. Image and Vision Computing 14(8), 609–615 (1996)CrossRefGoogle Scholar
  4. 4.
    Stauffer, C., Grimson, E.: Learning patterns of activity using real-time tracking. IEEE transactions on Pattern Analysis and Machine Intelligence (PAMI) 22(8), 747–757 (2000)CrossRefGoogle Scholar
  5. 5.
    Makris, D., Ellis, T.: Learning semantic scene models from observing activity in visual surveillance. IEEE Transactions on Systems, Man and Cybernetics 35(3), 397–408 (2005)CrossRefGoogle Scholar
  6. 6.
    McKenna, S.J., Charif, H.N.: Summarising contextual activity and detecting unusual inactivity in a supportive home environment. Pattern Analysis and Applications 7(4), 386–401 (2004)CrossRefMathSciNetGoogle Scholar
  7. 7.
    KaewTraKulPong, P., Bowden, R.: Probabilistic learning of salient patterns across spatially separated, uncalibrated views. In: Intelligent Distributed Surveillance Systems, pp. 36–40 (2004)Google Scholar
  8. 8.
    Xiang, T., Gong, S.: Beyond tracking: Modelling activity and understanding behaviour. International Journal of Computer Vision 67(1), 21–51 (2006)CrossRefGoogle Scholar
  9. 9.
    Bicego, M., Cristiani, M., Murino, V.: Unsupervised scene analysis: a hidden Markov model approach. Computer Vision and Image Understanding (CVIU) 102, 22–41 (2006)CrossRefGoogle Scholar
  10. 10.
    Efros, A.A., Berg, A.C., Mori, G., Malik, J.: Recognizing action at a distance. In: Proc. International Conference on Computer Vision (ICCV), Nice, France (2003)Google Scholar
  11. 11.
    Laptev, I., Pérez, P.: Retrieving actions in movies. In: Proc. International Conference on Computer Vision (ICCV) (2007)Google Scholar
  12. 12.
    Dalal, N., Triggs, B., Schmid, C.: Human detection using oriented histograms of flow and appearance. In: Proc. European Conference on Computer Vision (ECCV), pp. 428–441 (2006)Google Scholar
  13. 13.
    Gryn, J.M., Wildes, R.P., Tsotsos, J.: Detecting motion patterns via direction maps with application to surveillance. In: Workshop on Applications of Computer Vision, pp. 202–209 (2005)Google Scholar
  14. 14.
    Colombo, A., Leung, V., Orwell, J., Velastin, S.A.: Markov models of periodically varying backgrounds for change detection. In: Visual Information Engineering, London, UK (2007)Google Scholar
  15. 15.
    Shi, J., Tomasi, C.: Good features to track. In: Proc. Computer Vision and Pattern Recognition (CVPR), pp. 593–600 (1994)Google Scholar
  16. 16.
    Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: International Joint Conference on Artificial Intelligence, pp. 674–679 (1981)Google Scholar
  17. 17.
    Tomasi, C., Kanade, T.: Detection and tracking of point features. Technical Report CMU-CS-91-132, Carnegie Mellon (1991)Google Scholar
  18. 18.
    Home Office Scientific Development Branch U.i-LIDS: Imagery library for intelligent detection systems ,
  19. 19.
    Boykov, Y., Veksler, O., Zabih, R.: Efficient approximate energy minimization via graph cuts. IEEE transactions on Pattern Analysis and Machine Intelligence (PAMI) 20(12), 1222–1239 (2001)CrossRefGoogle Scholar
  20. 20.
    Kolmogorov, V., Zabih, R.: What energy functions can be minimized via graph cuts? IEEE transactions on Pattern Analysis and Machine Intelligence (PAMI) 26(2), 147–159 (2004)CrossRefGoogle Scholar
  21. 21.
    Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE transactions on Pattern Analysis and Machine Intelligence (PAMI) 26(9), 1124–1137 (2004)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Hannah M. Dee
    • 1
  • Roberto Fraile
    • 1
  • David C. Hogg
    • 1
  • Anthony G. Cohn
    • 1
  1. 1.School of ComputingUniversity of LeedsLeedsUnited Kingdom

Personalised recommendations