Towards an Architecture for Cognitive Vision Using Qualitative Spatio-temporal Representations and Abduction

  • Anthony G. Cohn
  • Derek R. Magee
  • Aphrodite Galata
  • David C. Hogg
  • Shyamanta M. Hazarika
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2685)


In recent years there has been increasing interest in constructing cognitive vision systems capable of interpreting the high level semantics of dynamic scenes. Purely quantitative approaches to the task of constructing such systems have met with some success. However, qualitative analysis of dynamic scenes has the advantage of allowing easier generalisation of classes of di.erent behaviours and guarding against the propagation of errors caused by uncertainty and noise in the quantitative data. Our aim is to integrate quantitative and qualitative modes of representation and reasoning for the analysis of dynamic scenes. In particular, in this paper we outline an approach for constructing cognitive vision systems using qualitative spatial-temporal representations including prototypical spatial relations and spatio-temporal event descriptors automatically inferred from input data. The overall architecture relies on abduction: the system searches for explanations, phrased in terms of the learned spatio-temporal event descriptors, to account for the video data.


Abstract Point Location Dynamic Scene Computer Vision System Memory Length Scene Description 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Yanai, K., Deguchi, K.: Recognition of indoor images employing qualitative model fitting and supporting relation between objects. In Sanfeliu, A., Villanueva, J., Vanrell, M., Alquezar, R., Eklundh, J.O., Aloimonos, Y., eds.: Proceedings 15th International Conference on Pattern Recognition. Volume 1., Barcelona, Spain, IEEE Press (2000) 964–967Google Scholar
  2. 2.
    Howarth, R.: Interpreting a dynamic and uncertain world: High-level vision. Artificial Intelligence Review 9 (1995) 37–63CrossRefGoogle Scholar
  3. 3.
    Buxton, H., Howarth, R.: Spatial and temporal reasoning in the generation of dynamic scene descriptions. In Rodríguez, R.V., ed.: Proceedings on Spatial and Temporal Reasoning, Montréal, Canada, IJCAI-95 Workshop (1995) 107–115Google Scholar
  4. 4.
    Fernyhough, J., Cohn, A., Hogg, D.: Constructing qualitative event models automatically from video input. Image and Vision Computing 18 (2000) 81–103CrossRefGoogle Scholar
  5. 5.
    Cootes, T., Taylor, C., Cooper, D., Graham, J.: Training models of shape from sets of examples. In: Proc. British Machine Vision Conference. (1992) 9–18Google Scholar
  6. 6.
    Baumberg, A., Hogg, D.: Learning flexible models from image sequences. In: European Conference on Computer Vision, Springer Verlag (1994) 299–308Google Scholar
  7. 7.
    Kass, M., Witkin, A., Terzopoulos, D.: Snakes: Active contour models. In: Proc. First International Conference on Computer Vision. (1989) 259–268Google Scholar
  8. 8.
    Blake, A., Curwen, R., Zisserman, A.: A framework for spatiotemporal control in the tracking of visual contours. International Journal of Computer Vision 11 (1993) 127–145CrossRefGoogle Scholar
  9. 9.
    Turk, M., Pentland, A.: Eigenfaces for recognition. Journal of Cognitive Neuroscience 3 (1991) 71–86CrossRefGoogle Scholar
  10. 10.
    Rabiner, L.: A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE 77 (1989) 257–286CrossRefGoogle Scholar
  11. 11.
    Starner, T., Pentland, A.: Real-time american sign language recognition from video using hidden markov models. In: Int. Symposium on Computer Vision. (1995)Google Scholar
  12. 12.
    Cohn, A.G., Hazarika, S.M.: Qualitative spatial representation and reasoning: An overview. Fundamenta Informaticae 46 (2001) 1–29zbMATHMathSciNetGoogle Scholar
  13. 13.
    Cohn, A.G., Bennett, B., Gooday, J., Gotts, N.: RCC: a calculus for region based qualitative spatial reasoning. GeoInformatica 1 (1997) 275–316CrossRefGoogle Scholar
  14. 14.
    Egenhofer, M., Franzosa, R.: Point-set topological spatial relations. International Journal of Geographical Information Systems 5 (1991) 161–174CrossRefGoogle Scholar
  15. 15.
    Cohn, A.G., Gotts, N.M.: Representing spatial vagueness: a mereological approach. In L C Aiello, J.D., Shapiro, S., eds.: Proceedings of the 5th conference on principles of knowledge representation and reasoning (KR-96), Morgan Kaufmann (1996) 230–241Google Scholar
  16. 16.
    Clementini, E., Di Felice, P.: Approximate topological relations. International Journal of Approximate Reasoning 16 (1997) 173–204zbMATHCrossRefMathSciNetGoogle Scholar
  17. 17.
    Schlieder, C.: Reasoning about ordering. In A Frank, W.K., ed.: Spatial Information Theory: a theoretical basis for GIS. Number 988 in Lecture Notes in Computer Science, Berlin, Springer Verlag (1995) 341–349Google Scholar
  18. 18.
    Isli, A., Cohn, A.: A new approach to cyclic ordering of 2d orientations using ternary relation algebras. Artificial Intelligence 122 (2000) 137–187zbMATHCrossRefMathSciNetGoogle Scholar
  19. 19.
    Frank, A.U.: Qualitative spatial reasoning about distance and directions in geograp hic space. Journal of Visual Languages and Computing 3 (1992) 343–373CrossRefGoogle Scholar
  20. 20.
    Meathrel, R.C., Galton, A.P.: A heirarchy of boundary-based shape descriptors. In Nebel, B., ed.: Proc. 17th IJCAI, Morgan Kaufmann (2001) 1359–1364Google Scholar
  21. 21.
    Bennett, B., Cohn, A.G., Torrini, P., Hazarika, S.M.: Describing rigid body motions in a qualitative theory of spatial regions. In Kautz, H.A., Porter, B., eds.: Proceedings of AAAI-2000. (2000) 503–509Google Scholar
  22. 22.
    Cristani, M., Cohn, A., Bennett, B.: Spatial locations via morpho-mereology. In: Proc. KR’2000, Morgan Kaufmann (2000)Google Scholar
  23. 23.
    Cohn, A.G., Bennett, B., Gooday, J., Gotts, N.: Representing and reasoning with qualitative spatial relations about regions. In Stock, O., ed.: Temporal and spatial reasoning, Kluwer (1997)Google Scholar
  24. 24.
    Wolter, F., Zakharyaschev, M.: Spatio-temporal representation and reasoning based on RCC-8. In: Proceedings of the seventh Conference on Principles of Knowledge Representation and Reasoning, Morgan Kaufman (2000) 3–14Google Scholar
  25. 25.
    Wolter, F., Zakharyaschev, M.: Qualitative spatio-temporal representation and reasoning: a computational perspective. In: Exploring Artifitial Intelligence in the New Millenium. Morgan Kaufmann (To appear)Google Scholar
  26. 26.
    Bennett, B., Cohn, A., Wolter, F., Zakharyaschev, M.: Multi-dimensional modal logic as a framework for spatio-temporal reasoning. Applied Intelligence (2002) To appear.Google Scholar
  27. 27.
    Hayes, P.J.: Naive physics I: Ontology for liquids. In Hobbs, J.R., Moore, B., eds.: Formal Theories of the Commonsense World. Ablex (1985) 71–89Google Scholar
  28. 28.
    Muller, P.: A qualitative theory of motion based on spatio-temporal primitives. In Cohn, A.G., Schubert, L.K., Shapiro, S., eds.: Principles of Knowledge Representation and Reasoning: Proceedings of the 6th International Conference (KR-98), Morgan Kaufman (1998) 131–141Google Scholar
  29. 29.
    Muller, P.: Space-time as a primitive for space and motion. In Guarino, N., ed.: Formal ontology in information systems: Proceedings of the 1st international conference (FOIS-98). Volume 46 of Frontiers in Artificial Intelligence and Applications., Trento, Italy, Ios Press (1998) 63–76Google Scholar
  30. 30.
    Hazarika, S.M., Cohn, A.G.: Qualitative spatio-temporal continuity. In Montello, D.R., ed.: Spatial Information Theory: Foundations of Geographic Information Science; Proceedings of COSIT’01. Volume 2205 of LNCS., Morro Bay,CA, Springer (2001) 92–107Google Scholar
  31. 31.
    Cui, Z., Cohn, A.G., Randell, D.A.: Qualitative simulation based on a logical formalism of space and time. In: Proceedings of AAAI-92, Menlo Park, California, AAAI Press (1992) 679–684Google Scholar
  32. 32.
    Shanahan, M.: Noise, non-determinism and spatial uncertainty. In: Proceedings of AAAI-97. (1997) 153–158Google Scholar
  33. 33.
    Shanahan, M.: A logical account of the common sense informatic situation for a mobile robot. Electronic Transactions on Artificial Intelligence (1999)Google Scholar
  34. 34.
    Remolina, E., Kuipers, B.: A logical account of causal and topological maps. In: Proceedings of Seventeenth International Conference on Artificial Intelligence (IJCAI-01). Volume I., Seattle, Washington, USA (2001) 5–11Google Scholar
  35. 35.
    Shanahan, M.: A logical account of perception incorporating feedback and expectation. In: Proc. 8th Int. Conf. on Knowledge Representation and Reasoning, San Mateo, Morgan Kaufmann (2002)Google Scholar
  36. 36.
    Galata, A., Cohn, A.G., Magee, D., Hogg, D.: Modelling interaction using learnt qualitative spatio-temporal relations and variable length markov models. In: Proc. European Conference on AI (ECAI). (2002)Google Scholar
  37. 37.
    Galata, A., Johnson, N., Hogg, D.: Learning behaviour models of human activities. In: British Machine Vision Conference, BMVC’99. (1999)Google Scholar
  38. 38.
    Galata, A., Johnson, N., Hogg, D.: Learning Variable Length Markov Models of Behaviour. Computer Vision and Image Understanding (CVIU) Journal 81 (2001) 398–413zbMATHCrossRefGoogle Scholar
  39. 39.
    Ron, D., Singer, S., Tishby, N.: The Power of Amnesia. In: Advances in Neural Information Processing Systems. Volume 6. Morgan Kauffmann (1994) 176–183Google Scholar
  40. 40.
    Guyon, I., Pereira, F.: Design of a Linguistic Postprocessor using Variable Memory Length Markov Models. In: International Conference on Document Analysis and Recognition. (1995) 454–457Google Scholar
  41. 41.
    Cormack, G., Horspool, R.: Data Compression using Dynamic Markov Modelling. Computer Journal 30 (1987) 541–550MathSciNetGoogle Scholar
  42. 42.
    Bell, T., Cleary, J., Witten, I.: Text Compression. Prentice Hall (1990)Google Scholar
  43. 43.
    Hu, J., Turin, W., Brown, M.: Language Modelling using Stochastic Automata with Variable Length Contexts. Computer Speech and Language 11 (1997) 1–16CrossRefGoogle Scholar
  44. 44.
    Magee, D.: Tracking multiple vehicles using foreground, background and motion models. In: Proc. ECCV Workshop on Statistical Methods in Video Processing. (2002)Google Scholar
  45. 45.
    Johnson, N., Hogg, D.: Learning the Distribution of Object Trajectories for Event Recognition. Image and Vision Computing 14 (1996) 609–615CrossRefGoogle Scholar
  46. 46.
    Wren, C., Azarbayejani, A., Darrell, T., Pentland, A.: Pfinder: Real-time tracking of the human body. IEEE Transactions on PAMI 19(7) (1997) 780–785Google Scholar
  47. 47.
    Cohn, A.G., Hazarika, S.M.: Continuous transitions in mereotopology. In: Commonsense-2001: 5th Symposium on Logical Formalizations of Commonsense Reasoning. (2001)Google Scholar
  48. 48.
    Hazarika, S.M., Cohn, A.G.: Abducing qualitative spatio-temporal histories from partial observations. In: Proc. 8th Int. Conf. on Knowledge Representation and Reasoning, San Mateo, Morgan Kaufmann (2002)Google Scholar
  49. 49.
    Randell, D., Witkowski, M., Shanahan, M.: From images to bodies: Modelling and exploiting spatial occlusion and motion parallax. In: Proc. IJCAI, Morgan Kaufmann (2001)Google Scholar
  50. 50.
    Freksa, C.: Using orientation information for qualitative spatial reasoning. In Frank, A.U., Campari, I., Formentini, U., eds.: Proc. Int. Conf. on Theories and Methods of Spatio-Temporal Reasoning in Geographic Space, Berlin, Springerverlag (1992)Google Scholar
  51. 51.
    Meathrel, R.C., Galton, A.: A hierarchy of boundary-based shape descriptors. In: Proc. IJCAI. (2001) 1359–1364Google Scholar
  52. 52.
    Jungert, E.: Symbolic spatial reasoning on object shapes for qualitative matching. In Frank, A.U., Campari, L., eds.: Spatial Information Theory: A Theoretical Basis for GIS. Lecture Notes in Computer Science No.716, COSIT’93, Springer-Verlag (1993) 444–462Google Scholar
  53. 53.
    Clementini, E., Di Felice, P.: A global framework for qualitative shape description. Geoinformatica 1 (1997) 1–17CrossRefGoogle Scholar
  54. 54.
    Davis, E., Gotts, N.M., Cohn, A.G.: Constraint networks of topological relations and convexity. Constraints 4 (1999) 241–280zbMATHCrossRefMathSciNetGoogle Scholar
  55. 55.
    Kaelbling, L.P., Oates, T., Hernandez, N., Finney, S.: Learning in worlds with objects. In Cohen, P.R., Oates, T., eds.: Learning Grounded Representations. Number Technical Report SS-01-05, AAAI Press (2001) 31–36Google Scholar
  56. 56.
    Zacks, J., Tversky, B., Iyer, G.: Perceiving, remembering and communicating structure in events. Journal of Experimental Psychology: General 136 (2001) 29–58CrossRefGoogle Scholar
  57. 57.
    Zacks, J., Tversky, B.: Event structure in perception and conception. Psychological Bulletin 127 (2001) 3–21CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2003

Authors and Affiliations

  • Anthony G. Cohn
    • 1
  • Derek R. Magee
    • 1
  • Aphrodite Galata
    • 1
  • David C. Hogg
    • 1
  • Shyamanta M. Hazarika
    • 1
  1. 1.School of ComputingUniversity of LeedsLeedsUK

Personalised recommendations