Skip to main content
SpringerLink
Log in

Complexity and Universality in the Long-Range Order of Words

  • Chapter
  • First Online:
Creativity and Universality in Language

Part of the book series: Lecture Notes in Morphogenesis ((LECTMORPH))

Abstract

As is the case of many signals produced by complex systems, language presents a statistical structure that is balanced between order and disorder. Here we review and extend recent results from quantitative characterisations of the degree of order in linguistic sequences that give insights into two relevant aspects of language: the presence of statistical universals in word ordering, and the link between semantic information and the statistical linguistic structure. We first analyse a measure of relative entropy that assesses how much the ordering of words contributes to the overall statistical structure of language. This measure presents an almost constant value close to 3.5 bits/word across several linguistic families. Then, we show that a direct application of information theory leads to an entropy measure that can quantify semantic structures and extract keywords from linguistic samples, even without prior knowledge of the underlying language.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 54.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    All but the Old Egyptian and Sumerian text sources were obtained from the Project Gutenberg e-text repository (www.gutenberg.org). The Old Egyptian texts were obtained from the page maintained by Dr. Mark-Jan Nederhof at the University of St Andrews (www.cs.st-andrews.ac.uk/mjn/egyptian/texts/) as transliterations from the original hieroglyphs. The Sumerian texts were downloaded from The Electronic Text Corpus of Sumerian Literature (www-etcsl.orient.ox.ac.uk/) and consisted of transliterations of the logo-syllabic symbols.

  2. 2.

    The parallel Bible corpus was compiled by Christos Christodoulopoulos from the Cognitive Computation Group at the University of Illinois, and downloaded from http://homepages.inf.ed.ac.uk/s0787820/bible/. Only texts that were encoded in Latin characters were used.

References

  1. Von Frisch, K.: The Dance Language and Orientation of Bees. Harvard University Press, Cambridge (1967)

    Google Scholar 

  2. Ouattara, K., Lemasson, A., Zuberbühler, K.: Proc. Natl. Acad. Sci. 106(51), 22026 (2009)

    Article  Google Scholar 

  3. Kershenbaum, A., Bowles, A.E., Freeberg, T.M., Jin, D.Z., Lameira, A.R., Bohn, K.: Proc. R. Soc. B: Biol. Sci. 281(1792), 20141370 (2014)

    Article  Google Scholar 

  4. Maynard Smith, J., Szathmáry, E.: The Major Transitions in Evolution. W.H. Freeman Spektrum, Oxford, New York (1995)

    Google Scholar 

  5. Deacon, T.W.: The Symbolic Species: The Co-Evolution of the Brain and Language. WW Norton & Co, New York (1997)

    Google Scholar 

  6. Zanette, D.H.: arXiv preprint arXiv:1412.3336 (2014)

  7. Greenberg, J.H.: Universals of Language. M.I.T. Press, Cambridge (1963)

    Google Scholar 

  8. Chomsky, N.: Aspects of the Theory of Syntax. M.I.T. Press, Cambridge (1965)

    Google Scholar 

  9. Hawkins, J.A.: Word Order Universals. Academic Press, New York (1983)

    Google Scholar 

  10. Hawkins, J.A.: Efficiency and Complexity in Grammars. Oxford University Press, Oxford (2004)

    Book  Google Scholar 

  11. Evans, N., Levinson, S.C.: Behav. Brain Sci. 32(05), 429 (2009)

    Article  Google Scholar 

  12. Dunn, M., Greenhill, S.J., Levinson, S.C., Gray, R.D.: Nature 473(7345), 79 (2011)

    Article  Google Scholar 

  13. Gell-Mann, M., Ruhlen, M.: Proc. Natl. Acad. Sci. 108(42), 17290. doi:10.1073/pnas.1113716108. http://www.pnas.org/content/108/42/17290.abstract (2011)

    Google Scholar 

  14. Zipf, G.K.: The Psycho-Biology of Language; An Introduction to Dynamic Philology. Houghton Mifflin Company, Boston (1935)

    Google Scholar 

  15. Heaps, H.S.: Information Retrieval, Computational and Theoretical Aspects. Academic Press, New York (1978)

    Google Scholar 

  16. Montemurro, M.A.: Phys. A 300(3–4), 567 (2001)

    Article  Google Scholar 

  17. Montemurro, M.A., Zanette, D.H.: Glottometrics 4, 87 (2002)

    Google Scholar 

  18. Zanette, D.H., Montemurro, M.A.: J. Quant. Linguist. 12, 29 (2005)

    Article  Google Scholar 

  19. Altmann, E.G., Gerlach, M.: arXiv preprint arXiv:1502.03296 (2015)

  20. Schenkel, A., Zhang, J., Zhang, Y.C.: Fractals-Complex Geom. Patterns Scaling Nat. Soc. 1(1), 47 (1993)

    Google Scholar 

  21. Ebeling, W., Neiman, A.: Phys. A 215(3), 233 (1995)

    Article  Google Scholar 

  22. Montemurro, M.A., Pury, P.: Fractals 10, 451 (2002)

    Article  Google Scholar 

  23. Alvarez-Lacalle, E., Dorow, B., Eckmann, J.P., Moses, E.: Proc. Natl. Acad. Sci. U.S.A. 103(21), 7956 (2006)

    Article  Google Scholar 

  24. Altmann, E.G., Cristadoro, G., Esposti, M.D.: Proc. Natl. Acad. Sci. U.S.A. 109(29), 11582 (2012)

    Article  Google Scholar 

  25. Shannon, C.E.: Bell Syst. Tech. J. 30(1), 50 (1951)

    Article  Google Scholar 

  26. Cover, T.M., King, R.C.: IEEE Trans. Inf. Theory 24(4), 413 (1978)

    Article  Google Scholar 

  27. Cover, T.M., Thomas, J.A.: Elements of Information Theory, 2nd edn. Wiley-Interscience, Hoboken (2006)

    Google Scholar 

  28. Lempel, A., Ziv, J.: IEEE Trans. Inf. Theory 22(1), 75 (1976)

    Article  Google Scholar 

  29. Ziv, J., Lempel, A.: EEE Trans. Inf. Theory 23(3), 337 (1977)

    Article  Google Scholar 

  30. Ziv, J., Lempel, A.: IEEE Trans. Inf. Theory 24(5), 530 (1978)

    Article  Google Scholar 

  31. Wyner, A.D., Ziv, J.: IEEE Trans. Inf. Theory 35(6), 1250 (1989)

    Article  Google Scholar 

  32. Kontoyiannis, I., Suhov, Y.M.: In: IEEE International Symposium on Information Theory, pp. 194–194. IEEE (1994)

    Google Scholar 

  33. Schurmann, T., Grassberger, P.: Chaos 6(3), 414 (1996)

    Article  Google Scholar 

  34. Kontoyiannis, I., Algoet, P.H., Suhov, Y.M., Wyner, A.J.: IEEE Trans. Inf. Theory 44(3), 1319 (1998)

    Article  Google Scholar 

  35. Puglisi, A., Benedetto, D., Caglioti, E., Loreto, V., Vulpiani, A.: Phys. D: Nonlinear Phenom. 180(1), 92 (2003)

    Article  Google Scholar 

  36. Gao, Y., Kontoyiannis, I., Bienenstock, E.: Entropy 10(2), 71 (2008)

    Article  Google Scholar 

  37. Lesne, A., Blanc, J.L., Pezard, L.: Phys. Rev. E 79(4), 046208 (2009)

    Article  Google Scholar 

  38. Montemurro, M.A., Zanette, D.H.: PLOS ONE 6(5), e19875 (2011)

    Article  Google Scholar 

  39. Harter, S.P.: J. Am. Soc. Inf. Sci. 26(4), 197 (1975)

    Article  Google Scholar 

  40. Church, K.W., Gale, W.A.: Nat. Lang. Eng. 1(2), 163 (1995)

    Article  Google Scholar 

  41. Ortuño, M., Carpena, P., Bernaola-Galvan, P., Munoz, E., Somoza, A.M.: Europhys. Lett. 57(5), 759 (2002)

    Article  Google Scholar 

  42. Herrera, J.P., Pury, P.A.: Eur. Phys. J. B - Condens. Matter Complex Syst. 63(1), 135 (2008)

    Article  Google Scholar 

  43. Altmann, E.G., Pierrehumbert, J.B., Motter, A.E.: PLOS ONE 4(11), e7678 (2009)

    Article  Google Scholar 

  44. Montemurro, M.A., Zanette, D.H.: Adv. Complex Syst. 13(2), 135 (2010)

    Article  Google Scholar 

  45. Carretero-Campos, C., Montemurro, M.A., Bernaola-Galván, P., Coronado, A.V., Carpena, P.: In: Proceedings of the European Conference on Complex Systems, pp. 241–249. Springer (2013)

    Google Scholar 

  46. Montemurro, M.A., Zanette, D.H.: Adv. Complex Syst. 5(1), 7 (2002)

    Article  Google Scholar 

  47. Montemurro, M.A., Zanette, D.H.: PLOS ONE 8(6), e66344 (2013)

    Article  Google Scholar 

  48. Montemurro, M.A.: Cortex 55, 5 (2014)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Life Sciences, The University of Manchester, Manchester, UK

    Marcelo A. Montemurro

  2. Centro Atómico Bariloche e Insituto Balseiro, San Carlos de Bariloche, Río Negro, Argentina

    Damián H. Zanette

Authors
  1. Marcelo A. Montemurro
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Damián H. Zanette
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marcelo A. Montemurro .

Editor information

Editors and Affiliations

  1. Dipartimento di Matematica, Università di Bologna, Bologna, Italy

    Mirko Degli Esposti

  2. the Physics of Complex Systems, Max Planck Institute for, Dresden, Germany

    Eduardo G. Altmann

  3. Sony Computer Science Laboratory, Paris, France

    François Pachet

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Montemurro, M.A., Zanette, D.H. (2016). Complexity and Universality in the Long-Range Order of Words. In: Degli Esposti, M., Altmann, E., Pachet, F. (eds) Creativity and Universality in Language. Lecture Notes in Morphogenesis. Springer, Cham. https://doi.org/10.1007/978-3-319-24403-7_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-24403-7_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-24401-3

  • Online ISBN: 978-3-319-24403-7

  • eBook Packages: Social SciencesSocial Sciences (R0)

Publish with us

Policies and ethics

Search

Navigation