Modelling Scenes Using the Activity within Them
This paper describes a method for building visual “maps” from video data using quantized descriptions of motion. This enables unsupervised classification of scene regions based upon the motion patterns observed within them. Our aim is to recognise generic places using a qualitative representation of the spatial layout of regions with common motion patterns. Such places are characterised by the distribution of these motion patterns as opposed to static appearance patterns, and could include locations such as train platforms, bus stops, and park benches. Motion descriptions are obtained by tracking image features over a temporal window, and are then subjected to normalisation and thresholding to provide a quantized representation of that feature’s gross motion. Input video is quantized spatially into N ×N pixel blocks, and a histogram of the frequency of occurrence of each vector is then built for each of these small areas of scene. Within these we can therefore characterise the dominant patterns of motion, and then group our spatial regions based upon both proximity and local motion similarity to define areas or regions with particular motion characteristics. Moving up a level we then consider the relationship between the motion in adjacent spatial areas, and can characterise the dominant patterns of motion expected in a particular part of the scene over time. The current paper differs from previous work which has largely been based on the paths of moving agents, and therefore restricted to scenes in which such paths are identifiable. We demonstrate our method in three very different scenes: an indoor room scenario with multiple chairs and unpredictable unconstrained motion, an underground station featuring regions where motion is constrained (train tracks) and regions with complicated motion and difficult occlusion relationships (platform), and an outdoor scene with challenging camera motion and partially overlapping video streams.
KeywordsLearning Spatial relations Computer vision Modeling behaviour Qualitative reasoning
Unable to display preview. Download preview PDF.
- 1.Fernyhough, J.H., Cohn, A.G., Hogg, D.C.: Generation of semantic regions from image sequences. In: Proc. European Conference on Computer Vision (ECCV), Cambridge, UK, pp. 475–484 (1996)Google Scholar
- 7.KaewTraKulPong, P., Bowden, R.: Probabilistic learning of salient patterns across spatially separated, uncalibrated views. In: Intelligent Distributed Surveillance Systems, pp. 36–40 (2004)Google Scholar
- 10.Efros, A.A., Berg, A.C., Mori, G., Malik, J.: Recognizing action at a distance. In: Proc. International Conference on Computer Vision (ICCV), Nice, France (2003)Google Scholar
- 11.Laptev, I., Pérez, P.: Retrieving actions in movies. In: Proc. International Conference on Computer Vision (ICCV) (2007)Google Scholar
- 12.Dalal, N., Triggs, B., Schmid, C.: Human detection using oriented histograms of flow and appearance. In: Proc. European Conference on Computer Vision (ECCV), pp. 428–441 (2006)Google Scholar
- 13.Gryn, J.M., Wildes, R.P., Tsotsos, J.: Detecting motion patterns via direction maps with application to surveillance. In: Workshop on Applications of Computer Vision, pp. 202–209 (2005)Google Scholar
- 14.Colombo, A., Leung, V., Orwell, J., Velastin, S.A.: Markov models of periodically varying backgrounds for change detection. In: Visual Information Engineering, London, UK (2007)Google Scholar
- 15.Shi, J., Tomasi, C.: Good features to track. In: Proc. Computer Vision and Pattern Recognition (CVPR), pp. 593–600 (1994)Google Scholar
- 16.Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: International Joint Conference on Artificial Intelligence, pp. 674–679 (1981)Google Scholar
- 17.Tomasi, C., Kanade, T.: Detection and tracking of point features. Technical Report CMU-CS-91-132, Carnegie Mellon (1991)Google Scholar
- 18.Home Office Scientific Development Branch U.i-LIDS: Imagery library for intelligent detection systems , http://scienceandresearch.homeoffice.gov.uk/hosdb/cctv-imaging-technology/video-based-detection-systems/i-lids/