Abstract
This paper describes a project currently being undertaken at the National Research Council. The project addresses itself to two much-discussed topics:
-
• the relation between the semantics of language and vision
-
• the notion of symbol-grounding.
We discuss some of the parallels and differences between the linguistic and visual channels of perception, and then describe a computer system we are designing that exploits both. This system accepts verbal scene descriptions and re-constructs a three-dimensional display of the virtual model of the world that it builds up in the process of interpreting the input.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Alshawi, H., Carter, D. M., Crouch, R., Pulman, S. G., Rayner, M. & Smith, A. G. (1992). CLARE: A Contextual Reasoning and Cooperative Response Framework for the Core Language Engine — Final Report. SRI International: Cambridge, England.
Barnden, J. A. & Pollock, J. B. (eds.) (1991).High Level Connectionist Models. Ablex: Norwood, N.J.
Best, K. (1994).The Idiot's Guide to Virtual World Design. Little Star Press: Seattle.
Gombrich, E. (1962).Art and Illusion. Phaidon: London.
Grover, C., Carroll, J. & Briscoe, E. (1993). The Alvey Natural Language Tools Grammar 4th Release. Technical Report, Cambridge University Computer Laboratory: Cambridge, England.
Herskovits, A. (1986).Language and Spatial Cognition. Cambridge University Press: Cambridge.
Hayes, P. (1985). The Second Naive Physics Manifesto. In Hobbs, J. and Moore, B. (eds.)Formal Theories of the Commonsense World. Ablex: Norwood, N.J.
Hobbs, J. & Moore, R. (eds.) (1985)Formal Theories of the Commonsense World. Albex Publishing: Norwood, N.J.
Jackendoff, R. (1987).Consciousness and the Computational Mind. MIT Press: Cambridge, Mass.
Johnson-Laird, P. N. (1983).Mental Models: Towards a Cognitive Science of Language, Inference and Consciousness. Cambridge University Press: Cambridge, England.
Johnson-Laird, P. N. (1988).tThe Computer and the Mind. Fontana: London, England.
Luria, A. R. (1970).Une Prodigieuse Mémoire; Étude Psychobiographique. Neuchâtelt, Delachaux et Niestlé.
Ó Nualláin, S. (in press).The Search for Mind: A New Foundation for Cognitive Science. Norwood, N.J.: Ablex.
Ó Nualláin, S., Smith, A. & Farley, B. (in press). The Spoken Image System. AAAI workshop on the integration of Natural Language and Vision Processing, Seattle.
Polanyi, M. (1958).Personal Knowledge. Routledge and Kegan Paul: London, England.
Rayner, M. & Alshawi, H. (1992). Deriving Database Queries from Logical Forms by Abductive Definition Expansion. Third Applied ACL, Trento, Italy.
Rayner, M. (1993).Abductive Equivalential Translation and Its Application to Natural Language Database Interfacing. PhD Thesis, Stockholm University.
Schank, R. (1972). Conceptual Dependency.Cognitive Psychology 3: 552–631.
Schank, R. & Abelson, R. (1977).Scripts, Plans, Goals and Understanding. Lawrence Erlbaum: Hillsdale, NJ.
Vygotsky, V. (1962).Thought and Language. MIT Press: Cambridge, Mass.
Wittgenstein, L. (1961).Tractatus Logico-Philosophicus. Translated by D. F. Pears & B. F. McGuinness Routledge and Kegan Paul: London. Original work published 1921.
Wittgenstein, L. (1968).Philosophische Untersuchungen; Philosophical Investigations. Translated by GEM Anscombe. Macmillan: New York. Original work published 1953.
Author information
Authors and Affiliations
Additional information
The Canadian Government's right to retain a non-exclusive, royalty-free licence in and to any copyright is acknowledged.
Rights and permissions
About this article
Cite this article
Nualláin, S.ó., Smith, A.G. An investigation into the common semantics of language and vision. Artif Intell Rev 8, 113–122 (1994). https://doi.org/10.1007/BF00849069
Issue Date:
DOI: https://doi.org/10.1007/BF00849069