Spatio-temporal Discovery: Appearance + Behavior = Agent
Experiments in infant category formation indicate a strong role for temporal continuity and change in perceptual categorization. Computational approaches to model discovery in vision have traditionally focused on static images, with appearance features such as shape playing an important role. In this work, we consider integrating agent behaviors with shape for the purpose of agent discovery. Improved algorithms for video segmentation and tracking under occlusion enable us to construct models that characterize agents in terms of motion and interaction with other objects. We present a preliminary approach for discovering agents based on a combination of appearance and motion histories. Using uncalibrated camera images, we characterize objects discovered in the scene by their shape and motion attributes, and cluster these using agglomerative hierarchical clustering. Even with very simple feature sets, initial results suggest that the approach forms reasonable clusters for diverse categories such as people, and for very distinct clusters (animals), and performs above average on other classes.
KeywordsAgglomerative Hierarchical Cluster Motion History Video Segmentation Agent Categorization Variable Length Sequence
Unable to display preview. Download preview PDF.
- 4.Mandler, J.M.: Foundations of Mind. Oxford University Press, New York (2004)Google Scholar
- 6.Blind1: Details withheld. In: for Blind Review (2006)Google Scholar
- 7.Zivkovic, Z.: Improved adaptive gaussian mixture model for background subtraction. In: Proceedings of the 17th International Conference on Pattern Recognition, vol. 2, pp. 28–31 (2004)Google Scholar
- 8.Proesmans, M., Gool, L.V., Pauwels, E., Osterlinck, A.: Determination of optical flow and its discontinuities using non-linear diffusion. In: The 3rd Eurpoean Conference on Computer Vision, vol. 2, pp. 295–304 (1994)Google Scholar
- 9.Collins, Lipton, Kanade, Fujiyoshi, Duggins, Tsin, Tolliver, Enomoto, Hasegawa: A system for video surveillance and monitoring: Vsam final report. Technical Report CMU-RI-TR-00-12, Robotics Institute, Carnegie Mellon University (2000)Google Scholar
- 10.Duda, R., Hart, P., Stork, D.: Pattern Recognition, 2nd edn. John Wiley and Sons, Chichester (2003)Google Scholar
- 11.Johnson, N., Hogg, D.: Learning the distribution of object trajectories for event recognition. In: Proceedings of the 6th British conference on Machine vision, vol. 2, pp. 583–592. BMVA Press (1995)Google Scholar
- 14.Johnson, N., Galata, A., Hogg, D.: The acquisition and use of interaction behavior models. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 866–871. IEEE Computer Society, Los Alamitos (1998)Google Scholar
- 15.Galata, A., Cohn, A.G., Magee, D., Hogg, D.: Modeling interaction using learnt qualitative spatio-temporal relations and variable length markov models. In: van Harmelen, F. (ed.) Proceedings of European Conference on Artificial Intelligence, pp. 741–745 (2002)Google Scholar