Abstract
Formal models of natural language semantics using TTR (Type Theory with Records) attempt to relate natural language to perception, modelled as classification of objects and events by types which are available as resources to an agent. We argue that this is better suited for representing the meaning of spatial descriptions in the context of agent modelling than traditional formal semantic models which do not relate spatial concepts to perceptual apparatus. Spatial descriptions include perceptual, conceptual and discourse knowledge which we represent all in a single framework. Being a general framework for modelling both linguistic and non-linguistic cognition, TTR is more suitable for the modelling of situated conversational agents in robotics and virtual environments where interoperability between language, action and perception is required. The perceptual systems gain access to abstract conceptual meaning representations of language while the latter can be justified in action and perception.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Blackburn, P., Bos, J.: Representation and inference for natural language. A first course in computational semantics. CSLI (2005)
Winograd, T.: Understanding Natural Language. Edinburgh University Press (1976)
Miller, G.A., Johnson-Laird, P.N.: Language and perception. Cambridge University Press, Cambridge (1976)
Gapp, K.P.: Basic meanings of spatial relations: Computation and evaluation in 3d space. In: Hayes-Roth, B., Korf, R.E. (eds.) AAAI, pp. 1393–1398. AAAI Press/The MIT Press (1994)
Logan, G.D., Sadler, D.D.: A computational analysis of the apprehension of spatial relations. In: Bloom, P., Peterson, M.A., Nadel, L., Garrett, M.F. (eds.) Language and Space, pp. 493–530. MIT Press, Cambridge (1996)
Siskind, J.M.: Grounding the lexical semantics of verbs in visual perception using force dynamics and event logic. Journal of Artificial Intelligence Research 15, 31–90 (2001)
Roy, D.: Semiotic schemas: a framework for grounding language in action and perception. Artificial Intelligence 167, 170–205 (2005)
Guerra-Filho, G., Aloimonos, Y.: A language for human action. Computer 40, 42–51 (2007)
Matuszek, C., Herbst, E., Zettlemoyer, L., Fox, D.: Learning to parse natural language commands to a robot control system. In: Proc. of the 13th International Symposium on Experimental Robotics, ISER (2012)
Dobnik, S.: Teaching mobile robots to use spatial words. PhD thesis, University of Oxford: Faculty of Linguistics, Philology and Phonetics and The Queen’s College, Oxford, United Kingdom (2009)
Cooper, R.: Austinian truth, attitudes and type theory. Research on Language and Computation 3, 333–362 (2005)
Cooper, R.: Type theory and semantics in flux. In: Kempson, R., Asher, N., Fernando, T. (eds.) Handbook of the Philosophy of Science. General editors: Gabbay, D.M., Thagard, P., Woods, J., vol. 14. Elsevier BV (2012)
Davidson, D.: The logical form of action sentences. In: Rescher, N., Anderson, A.R. (eds.) The Logic of Decision and Action. University of Pittsburgh Press, Pittsburgh (1967)
Barwise, J.: The situation in logic. CSLI Publications, Stanford (1989)
Cooper, R.: Records and record types in semantic theory. Journal of Logic and Computation 15, 99–112 (2005)
Larsson, S., Cooper, R.: Towards a formal view of corrective feedback. In: Alishahi, A., Poibeau, T., Villavicencio, A. (eds.) Proceedings of the Workshop on Cognitive Aspects of Computational Language Acquisition, EACL, pp. 1–9 (2009)
Cooper, R., Larsson, S.: Compositional and ontological semantics in learning from corrective feedback and explicit definition. In: Edlund, J., Gustafson, J., Hjalmarsson, A., Skantze, G. (eds.) Proceedings of SemDial 2009 (DiaHolmia): Workshop on the Semantics and Pragmatics of Dialogue, Stockholm, Sweden, Royal Institute of Technology (KTH), pp. 59–66 (2009)
Ginzburg, J.: The interactive stance: meaning for conversation. Oxford University Press, Oxford (2012)
Larsson, S.: The TTR perceptron: Dynamic perceptual meanings and semantic coordination. In: Artstein, R., Core, M., DeVault, D., Georgila, K., Kaiser, E., Stent, A. (eds.) SemDial 2011 (Los Angelogue): Proceedings of the 15th Workshop on the Semantics and Pragmatics of Dialogue, Los Angeles, California, pp. 140–148 (2011)
Farhadi, A., Endres, I., Hoiem, D., Forsyth, D.: Describing objects by their attributes. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1778–1785 (2009)
Herskovits, A.: Language and spatial cognition: an interdisciplinary study of the prepositions in English. Cambridge University Press, Cambridge (1986)
Talmy, L.: Toward a cognitive semantics: concept structuring systems, vol. 1 and 2. MIT Press, Cambridge (2000)
Regier, T., Carlson, L.A.: Grounding spatial language in perception: an empirical and computational investigation. Journal of Experimental Psychology: General 130, 273–298 (2001)
Coventry, K.R., Prat-Sala, M., Richards, L.: The interplay between geometry and function in the apprehension of Over, Under, Above and Below. Journal of Memory and Language 44, 376–398 (2001)
Levinson, S.C.: Space in language and cognition: explorations in cognitive diversity. Cambridge University Press, Cambridge (2003)
Maillat, D.: The semantics and pragmatics of directionals: a case study in English and French. PhD thesis, Committee for Comparative Philology and General Linguistics, University of Oxford, Oxford, UK (2003)
Steels, L., Loetzsch, M.: Perspective alignment in spatial language. In: Coventry, K.R., Tenbrink, T., Bateman, J.A. (eds.) Spatial Language and Dialogue. Oxford University Press (2009)
Dobnik, S.: Spatial descriptions in discourse: choosing a perspective. In: Brown-Schmidt, S., Jonathan Ginzburg, S.L. (eds.) Proceedings of SemDial 2012 (SeineDial): The 16th Workshop on the Semantics and Pragamtics of Dialogue, Université Paris-Diderot (Paris 7), Paris Sorbonne-Cité, pp. 151–152 (2012)
Dissanayake, M.W.M.G., Newman, P.M., Durrant-Whyte, H.F., Clark, S., Csorba, M.: A solution to the simultaneous localization and map building (SLAM) problem. IEEE Transactions on Robotic and Automation 17, 229–241 (2001)
Shieber, S.: An Introduction to Unification-Based Approaches to Grammar. CSLI Publications, Stanford (1986)
Fernando, T.: A finite-state approach to events in natural language semantics. Journal of Logic and Computation 14, 79–92 (2004)
Lauria, S., Bugmann, G., Kyriacou, T., Bos, J., Klein, E.: Training personal robots using natural language instruction. IEEE Intelligent Systems 16, 38–45 (2001)
Witten, I.H., Frank, E., Hall, M.A.: Data mining: practical machine learning tools and techniques, 3rd edn. Morgan Kaufmann, Burlington (2011)
Clark, A., Lappin, S.: Linguistic nativism and the poverty of the stimulus. Wiley-Blackwell, Chichester (2011)
Tenenbaum, J.B., Kemp, C., Griffiths, T.L., Goodman, N.D.: How to grow a mind: Statistics, structure, and abstraction. Science 331, 1279–1285 (2011)
Pulman, S.G., Liakata, M.: Learning domain theories. In: Nicolov, N., Bontcheva, K., Angelova, G., Mitkov, R. (eds.) RANLP. Current Issues in Linguistic Theory (CILT), vol. 260, pp. 29–44. John Benjamins, Amsterdam/Philadelphia (2003)
Baroni, M., Murphy, B., Barbu, E., Poesio, M.: Strudel: A corpus-based semantic model based on properties and types. Cognitive Science 34, 222–254 (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Dobnik, S., Cooper, R., Larsson, S. (2013). Modelling Language, Action, and Perception in Type Theory with Records. In: Duchier, D., Parmentier, Y. (eds) Constraint Solving and Language Processing. CSLP 2012. Lecture Notes in Computer Science, vol 8114. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-41578-4_5
Download citation
DOI: https://doi.org/10.1007/978-3-642-41578-4_5
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-41577-7
Online ISBN: 978-3-642-41578-4
eBook Packages: Computer ScienceComputer Science (R0)