Language and Complexity: Neurolinguistic Perspectives

  • Bernard Scott
Part of the Machine Translation: Technologies and Applications book series (MATRA, volume 2)


In the previous chapter we dealt with the problem of ambiguity and how simulation of input-driven, psycholinguistic processes enables a semantico-syntactic translation model to deal effectively with ambiguity. In the present chapter we deal with the issue of complexity, focusing in particular on the constraining effect that cognitive complexity has on MT development as it attempts to cope with the ambiguities of langauge. We define cognitive complexity as the difficulty developers experience in maintaining complex systems. We show how the associative nature of Logos Model’s neural-like translation paradigm allows it to deal more effectively with cognitive complexity than is possible with rule-based technology, or indeed any other MT paradigm. We attribute the reasons for this to Logos Model’s serendipitous correspondence to findings of neuroscience on the brain’s processing of language, citing the brain’s evident freedom from complexity in processing language as a motivation for this direction in Logos Model design. We focus on two regions of the brain that are involved with language: (1) the prefrontal temporal cortex designated as the Broca area, commonly connected with rule-based processes, and (2) the hippocampus, a well-defined reticulum in the medial temporal lobe distinguished for its declarative, associative memory processes, and whose connection with language processes has only recently been proposed by neuroscientists. We provide illustrations and examples of how the associative processes of the hippocampus have been simulated in Logos Model, and how Logos Model has benefited from this simulation.

Supplementary material


  1. Bonhage CE, Mueller JL, Friederici AD, Fiebach CJ (2015) Combined eye tracking and fMRI reveals neural basis of linguistic predictions during sentence comprehension. Cortex 68:33–47CrossRefGoogle Scholar
  2. Breitenstein C, Jansen A, Deppe M, Foerster AF, Somme J, Wolbers T, Knecht S (2004) Hippocampus activity differentiates good from poor learners of a novel lexicon. NeuroImage 25(3):958–968. S1053-8119(04)00770-0 [pii]CrossRefGoogle Scholar
  3. Chan D, Fox NC, Scahill RI, Crum WR, LWhitwell J, Leschziner G, Rosser AM, Stevens JM, Cipolotti L, Rosser MN (2001) Patterns of temporal lobe atrophy in semantic dementia and Alzheimer’s disease. Ann Neurol 49(4):433–442CrossRefGoogle Scholar
  4. Chomsky N (1957) Syntactic structures. Mouton, The Hague/PariszbMATHGoogle Scholar
  5. Chomsky N (1965) Aspects of the theory of syntax. MIT Press, CambridgeGoogle Scholar
  6. Crick F, Asanuma C (1986) Certain aspects of the anatomy and physiology of the cerebral cortex. In: McClelland JL, Rumelhart DE (eds) Distributed parallel processing, vol 2. MIT Press, Cambridge, pp 333–371Google Scholar
  7. de Vries MH, Petersson KM, Geukes S, Zwitserlood P, Christiansen CH (2012) Processing multiple non-adjacent dependencies: evidence from sequence learning. Philos Trans R Soc B 367:2065–2076CrossRefGoogle Scholar
  8. DeKeyser R (2008) Chapter 11: Implicit and explicit learning. In: Doughty CJ, Long MH (eds) The handbook of second langauge acquisition. Wiley Online Library. Accessed 21 July 2016
  9. Douglas R, Martin K (1990) Neocortex. In: Shepherd G (ed) The synapatic organization of the brain, 2nd edn. Oxford University Press, Oxford/New York, pp 389–438Google Scholar
  10. Duff MC, Brown-Schmidt S (2012) The Hippocampus and the flexible use and processing of language. Front Hum Neurosci 6:69. Accessed 23 Sept 2016CrossRefGoogle Scholar
  11. Duff MC, Brown-Schmidt S (2017) Hippocampal contributions to language use and processing. In: Hannula DE, Duff MC (eds) The hippocampus from cells to systems. Springer, Cham, pp 503–536CrossRefGoogle Scholar
  12. Elman J (2001) Connectionism and the acquisition of language. In: Tomasello M, Bates E (eds) Essential readings in language acquisition. Basil Blackwell, Oxford, pp 295–306Google Scholar
  13. Fanselow MS, Dong H-W (2010) Are the dorsal and ventral hippocampus functionally distinct structures? Neuron 65(1):7. Accessed 9 June 2016CrossRefGoogle Scholar
  14. Fauconnier G (1997) Mappings in thought and language. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  15. Fiebach CJ, Friederici AD, Mûller K, Yves von Cramon D (2002) fMRI evidence for dual routes to the mental lexicon in visual word recognition. J Cogn Neurosci 14(1):11–23CrossRefGoogle Scholar
  16. Fillmore C, Kay P, O’Connor C (1988) Regularity and Idiomaticity in grammatical constructions: the case of let alone. Language 64:501–538CrossRefGoogle Scholar
  17. Fitch WT, Friederici AD, Hagoort P (2012) Pattern perception and computational complexity: introduction to the special issue. Philos Trans R Soc 367:1925–1932. Accessed 22 Jan 2016CrossRefGoogle Scholar
  18. Glaser YG, Martin RC, Van Dyke JA, Chris Hamilton A, Tan Y (2013) Neural basis of semantic and syntactic interference in sentence comprehension. Brain Lang 126:314–326CrossRefGoogle Scholar
  19. Henke K, Treyer V, Nagy ET, Kneifel S, Dursteler M, Nitsch RM, Buck A (2003) Active hippocampus during nonconscious memories. Conscious Cogn 12(1):31–48CrossRefGoogle Scholar
  20. Kay M, Hays DG (2000) In: John Hutchins W (ed) Early years in machine translation. John Benjamin Publishing, Amsterdam/Philadelphia Company, pp 165–170CrossRefGoogle Scholar
  21. Koehn P (2011) Statistical machine translation. Cambridge University Press, CambridgezbMATHGoogle Scholar
  22. Koehn P, Knowles R (2017) Six challenges for neural machine translation. In: Proceedings of the first workshop on neural machine translation. Vancouver, pp 26–39. http://arXiv:1706.03872v1. Accessed 13 Dec 2017Google Scholar
  23. Kumaran D, Hassabis D, McClelland JL (2016) What learning systems do intelligent agents need? Complementary learning systems theory updated. Trends Cogn Sci 20(7):512. Accessed 12 Jan 2017CrossRefGoogle Scholar
  24. Kurczek J, Brown-Schmidt S, Duff M (2013) Hippocampal contributions to language: evidence of referential processing deficits in amnesia. J Exp Psychol Gen 142(4):1346–1354CrossRefGoogle Scholar
  25. Lakoff G (1987) Women, fire, and dangerous things: what categories reveal about the mind. CSLI, ChicagoCrossRefGoogle Scholar
  26. Liu S, Yang N, Li M, Zhou M (2014) A recursive recurrent neural network for statistical machine translation. In: Proceedings of the 52nd annual meeting of the association for computational linguistics. Maryland, Baltimore, pp 1491–1500Google Scholar
  27. Marr D (1971) Simple memory: a theory for archicortex. Philos Trans R Soc B Biol Sci 262:23–81. CrossRefGoogle Scholar
  28. Mårtensson J, Eriksson J, Bodammer NC, Lindgren M, Johansson M, Nyberg L, Lövdén M (2012) Growth of language-related brain areas after foreign language learning. NeuroImage 63(1):240–244CrossRefGoogle Scholar
  29. McClelland JL, McNaughton BL, O’Reilly RC (1995) Why there are complementary learning systems in the Hippocampus and Neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychol Rev 102(3):419–457CrossRefGoogle Scholar
  30. McCormick DA (1990) Membrane properties and neurotransmitter action. In: Shepherd G (ed) The synaptic organization of the brain. Oxford University Press, New York/Oxford, pp 39–78Google Scholar
  31. O’Keefe J, Dostrovsky J (1971) The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. Brain Res 34(1):171–175CrossRefGoogle Scholar
  32. Optiz B (2010) Neural binding mechanisms in learning and memory. Neurosci Biobehav Rev 34:1036–1046CrossRefGoogle Scholar
  33. Optiz B, Friederici AD (2003) Interactions of the hippocampal system and the prefrontal cortex in learning language-like rules. NeuroImage 19:1730–1737. Accessed 25 Jan 2016CrossRefGoogle Scholar
  34. Palmer DC (2006) On Chomsky’s appraisal of Skinner’s verbal behavior: a half-century of misunderstanding. Behav Anal 29(2):253–267CrossRefGoogle Scholar
  35. Pinker S, Prince A (1988) On language and connectionism: analysis of a parallel distributed processing model of language acquisition. Cognition 23:73–193CrossRefGoogle Scholar
  36. Race E, Keane MM, Verfaellie M (2015) Sharing mental simulations and stories: hippocampal contributions to discourse integration. Cortex 63:271–281CrossRefGoogle Scholar
  37. Robinson P (2008) In: Ellis NC (ed) Handbook of cognitive linguistics and second language acquisition. Routledge, New YorkGoogle Scholar
  38. Rumelhart DE, McClelland JL (1986) Chapter 18: On learning the past tenses of English verbs. In: Parallel distributed processing: explorations in the microstructure of cognition, vol 2. MIT Press, Cambridge, pp 216–271Google Scholar
  39. Scott B (1989) The logos system. In: Proceedings of MT Summit II, Munich, pp 137–142Google Scholar
  40. Scott B (1990) Biological neural net for parsing long, complex sentences. Logos Corporation Publication. (Available from the author ( Scholar
  41. Scott B (2000) Logos model as a metaphorical biological neural net. Logos Corporation Publication.
  42. Scott B (2003) Logos model: an historical perspective. Mach Transl 18(1):1–72MathSciNetCrossRefGoogle Scholar
  43. Sejnowski TJ (1986) Chapter 21: Open questions about computation in cerebral cortex. In: Parallel distributed processing, vol II. MIT Press, Cambridge, pp 372–389Google Scholar
  44. Sherrington C (1941) Man on his nature. The Macmillian Company, New YorkGoogle Scholar
  45. Sidiropoulou K, Lu FM, Fowler MA, Xiao R, Phillips C, Ozkan ED, Zhu MX (2009) Dopamine modulates an mGluR5-mediated depolarization underlying prefrontal persistent activity. Nat Neurosci 12(2):190–199CrossRefGoogle Scholar
  46. Socher R, Lin CC-Y, Ng AY, Manning CD (2011) Parsing natural scenes and natural language with recursive neural networks. In: Proceedings of the 28th international conference on machine learning, Bellevue, pp 129–136. Accessed 17 Apr 2016
  47. Squire LR, Sola-Morgan S (1991) The medial temporal lobe memory system. Science 253(5026):1380–1386CrossRefGoogle Scholar
  48. Thorpe SJ (1995) Localized versus distributed representation. In: Arbib MA (ed) The handbook of brain theory and neural networks. MIT Press, Cambridge, pp 643–646Google Scholar
  49. Verfaellie MK, Bousquet MK, Keane MM (2014) Medial temporal and neocortical contributions to remote memory for semantic narratives: evidence from amnesia. Neuropsychologia 61:105–112CrossRefGoogle Scholar
  50. Wang S-H, Morris RGM (2010) Hippocampal-neocortical interactions in memory formation, consolidation, and reconsolidation. Science 345(6200):1054–1057CrossRefGoogle Scholar
  51. Wang JX, Rogers LM, Gross EZ, Ryals AJ, Dokucu ME, Brandstatt KL, Hermiller MS, Voss JL (2014) Targeted enhancement of cortical-hippocampal brain networks and associative memory. Science 345(6200):1054–1057CrossRefGoogle Scholar
  52. Wintzer ME, Boehringer R, Polygalov D, McHugh TJ (2014) The hippocampal CA2 ensemble is sensitive to contextual change. J Neurosci 34(8):3056–3066CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Bernard Scott
    • 1
  1. 1.Tarpon SpringsUSA

Personalised recommendations