Abstract
In our opinion the exuberance surrounding the relative success of data-driven large language models (LLMs) is slightly misguided and for several reasons (i) LLMs cannot be relied upon for factual information since for LLMs all ingested text (factual or non-factual) was created equal; (ii) due to their subsymbolic nature, whatever ‘knowledge’ these models acquire about language will always be buried in billions of microfeatures (weights), none of which is meaningful on its own; and (iii) LLMs will often fail to make the correct inferences in several linguistic contexts (e.g., nominal compounds, copredication, quantifier scope ambiguities, intensional contexts). Since we believe the relative success of data-driven large language models (LLMs) is not a reflection on the symbolic vs. subsymbolic debate but a reflection on applying the successful strategy of a bottom-up reverse engineering of language at scale, we suggest in this paper applying the effective bottom-up strategy in a symbolic setting resulting in symbolic, explainable, and ontologically grounded language models.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
References
Aitchison, J.: Words in the Mind – An Introduction to the Mental Lexicon, Wiley (2012)
Asher, N.: Lexical Meaning in Context, a Web of Words. Cambridge University Press (2011)
Asher, N., Pustejovsky, J.: A type composition logic for generative lexicon. Journal of Cognitive Science 6, 1–38 (2011)
Boleda, G.: Distributional Semantics and Linguistic Theory. Annual Review of Linguistics 6, 213–234 (2020)
Dummett, M.: Frege: Philosophy of Language. Harvard University Press (1981)
Fodor, J. A. and Pylyshyn, Zenon W.: Connectionism and cognitive architecture: A critical analysis, Cognition, 28 (1), pp. 3–71 (1988)
Hobbs, J.: Ontological promiscuity. In Proc. of the 23rd Annual Meeting of the Assoc. for Computational Linguistics, Chicago, Illinois, 1985, pp. 61–69 (1985)
Harris, Z.S.: Distributional Structure. Word 10, 146–162 (1954)
Kiss, K. E. and Pafel, J.: Quantifier Scope Ambiguities, In Martin Everaert and Henk C. van Riemsdijk (Eds.), The Wiley Blackwell Companion to Syntax (2017)
Larson, R.: Events and Modification in Nominals, In Devon Strolovitch and Aaron Lawson (eds), SALT VIII, 145- 168, Ithaca, NY (1998)
Lidz, J.: Children’s Use of Syntax in Word Learning, In Anna Papafragou, John C. Trueswell & Lila R. Gleitman (eds.), The Oxford Handbook of the Mental Lexicon, Oxford University Press (2022)
Lopes, J.: Can Deep CNNs Avoid Infnite Regress/Circularity in Content Constitution?, Minds and Machines, https://doi.org/10.1007/s11023-023-09642-0 (2023)
McShane, M., Beale, S. and Babkin, P.: Nominal Compound Interpretation by Intelligent Agents, Linguistic Issues in Language Technology (LiLT), vol. 10, No 1 (2014)
Milne, P.: Frege’s Context Principle. Mind 95(380), 491–495 (1986)
Moltmann, F.: Abstract Objects and the Semantics of Natural Language, Oxford University Press (2013)
Peckenpaugh, T.: Prepositional phrase attachment ambiguities in declarative and interrogative contexts: Oral reading data, PhD Thesis, The City University of New York (2019)
Piñango, M.M., Zhang, M., et al.: Metonymy as Referential Dependency: Psycholinguistic and Neurolinguistic Arguments for a Unified Linguistic Treatment. Cogn. Sci.. Sci. 41(S2), 351–378 (2017)
Saba, W.: New Research Vindicates Fodor and Pylyshyn: No Explainable AI Without ‘Structured Semantics, Blog of the Communications of the ACM, September 14 (2022)
Saba, W.: Language, Knowledge and Ontology: Where Formal Semantics Went Wrong, and How to Go Forward, Again. Journal of Knowledge Structures and Systems (JKSS) 1(1), 40–62 (2020)
Saba, W., Corriveau, J.-P.: Plausible Reasoning and the Resolution of Quantifier Scope Ambiguities, Studia Logica – Int. J. Symb. Log.Symb. Log. 67, 271–289 (2001)
Saba, W.: Language, logic and ontology: Uncovering the structure of commonsense knowledge. Int. J. of Human Computer Studies 7(65), 610–623 (2007)
Shelestiuk, H. V.: Metonymy as a tool of cognition and representation: A natural language analysis, Semiotica, pp. 1–20 (2005)
Sommers, F.: Types and ontology. Philos. Rev. 72(3), 327–363 (1963)
Sugawara, S. and Tsugita. S.: On Degrees of Freedom in Defining and Testing Natural Language Understanding, In Findings of the Association for Computational Linguistics: ACL, pp. 13625–13649 (2023)
Vaswani, A., Shazeer, N., et. al.: Attention is All You Need, In NIPS’17: Proceedings of the 31st Int. Conference on Neural Information Processing Systems. pp. 6000–6010, (2017)
Viebahn, E.: Copredication, polysemy and context-sensitivity, Inquiry, Volume 65 (2020)
von Fintel, K. and Heim, I. Lecture Notes on Intensional Semantics, available online here https://www.phil-fak.uni-duesseldorf.de/summerschool2002/fintel.pdf, (2002)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Saba, W.S. (2023). Stochastic LLMs do not Understand Language: Towards Symbolic, Explainable and Ontologically Based LLMs. In: Almeida, J.P.A., Borbinha, J., Guizzardi, G., Link, S., Zdravkovic, J. (eds) Conceptual Modeling. ER 2023. Lecture Notes in Computer Science, vol 14320. Springer, Cham. https://doi.org/10.1007/978-3-031-47262-6_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-47262-6_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-47261-9
Online ISBN: 978-3-031-47262-6
eBook Packages: Computer ScienceComputer Science (R0)