Abstract
Information theory is at the core of the sciences of war. Although the scientific notion of information had existed since the 1920s, it really took off with the publication of Cybernetics by Norbert Wiener in 1948 and with the articles by Claude Shannon in the Bell System Technical Journal. These articles were republished in 1949 with a preface by Warren Weaver in The Mathematical Theory of Communication co-authored by Shannon and Weaver. Information theory then aroused a huge interest among scientists from all disciplines, leading to numerous and very often disparate works. It offered a large range of new methods and notions that tempted many linguists. In this chapter, after a survey of the use of information as a term and notion, we will examine three modes of how information theory was integrated in linguistics: its adaptation by the Neo-Bloomfieldians, especially by Charles F. Hockett and Zellig S. Harris; the convergence of information theory, engineering and linguistics in Roman Jakobson’s Distinctive Feature Theory; and its use by two French linguists, André Martinet and Jean Dubois.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
See Chap. 6. §6.5.
- 2.
Andrey Andreyevich Markov (1856–1922) studied literary texts, among them Eugene Onegin by Pushkin, as stochastic strings of characters. A Markov chain is a finite-state automaton whose transitions from one state to another are ruled by probabilities. In 1948, Shannon put forward a statistical model based on Markov chains to study the strings of letters in an English text.
- 3.
According to Segal (2003), Zipf, in his 1946 article entitled “Some determinants of the circulation of information”, developed a mathematical definition of “quantity of information” close to Shannon’s.
- 4.
The term “code” had already been used by Saussure. Since then, the notion of code has been at the core of the linguists’ reflections on the relationship between spoken and written language, between speech and writing (Fehr 2003).
- 5.
Hockett (1953) dedicated an imposing review of thirty pages to Shannon and Weaver’s book in Language.
- 6.
“It [language] can be objectively studied if one considers speech and writing not as an expression of the speaker which has particular, introspectively recognised, meanings to the hearer; but rather as a set of events – sound waves or ink marks – which may be said, if we wish, to serve as tool or vehicle for communication and expression. This set of events, the many occurrences of speaking and of writing, can be described in terms of a structural model” (Harris 1959, p. 458).
- 7.
Phonemes are the linguistically contrastive or significant sounds (or sets of sounds) of a language: these sounds cannot be substituted for each other without a change in meaning. For example, in English /a/ /i/ /e/ /u/ /o/ are different phonemes because the words /pat/ /pit/ /pet/ /put/ /pot/ have different meanings. Allophones are the linguistically non-significant variants of each phoneme. In other words, a phoneme may be realised by more than one speech sound. Allophones can be substituted for each other without a change in meaning. For example, in French, “père” can be pronounced in three different ways: the consonant [r] can be realised as an alveolar trill, an uvular trill or a uvular fricative; they are different phonetic sounds, but their difference is not relevant from the point of view of meaning. They are interpreted as the variants of pronunciation of the same phoneme /r/.
- 8.
For a detailed study of the significance of information theory in Harris’s work, see Léon (2011a).
- 9.
For Harris, sublanguages of sciences are formal artificial languages, identical for a given science whatever (the natural) language (see Chap. 10).
- 10.
Phonetics is about the physical aspect of sounds. It deals with the description and classification of speech sounds, particularly how sounds are produced, transmitted and received, often without prior knowledge of the language being spoken. Phonology is about the abstract aspect of sounds. It establishes what are the phonemes in a given language, i.e. those sounds that can bring a difference in meaning between two words (see note 7 above).
- 11.
Waugh et al. (1990) indicate the antinomies of post-Hegelian Russian dialectic tradition as being also at the source of the notion of opposition.
- 12.
According to Anderson (1985), it is difficult to distinguish which was specific to Trubetzkoy or to Jakobson during the Prague period. As their correspondence attests, there were very few points of disagreement between them (Sériot 2006). It was only after Trubetzkoy’s death in 1938 that Jakobson’s views began to really diverge.
- 13.
- 14.
Homophony is the linguistic phenomenon whereby two or more different written forms have the same pronunciation: bare/ bear; flour/flower.
- 15.
Cherry et al. (1953) give the following example, p. 39: a redundant feature helps the hearer to solve some of the ambiguities caused by signal distortion. For example, the nasal feature is marked zero for all the Russian vowels. If these zeros were changed into (+), the new symbols would not mean that a Russian speaker nasalises all his/her vowels; normally he/she would not do it; even if he/she would do it, nasality would not have any phonemic signification. It is a redundant feature.
- 16.
“The necessity of a strict distinction between different types of redundancy is now realised in the theory of communication as well as in linguistics, where the concept of redundancy encompasses on the one hand pleonastic means as opposed to explicit conciseness (brevitas in the tradition nomenclature of rhetoric) and on the other hand explicitness in contradistinction to ellipsis. On the phonological level, linguists have been accustomed to delimit phonemic, distinctive units from contextual, combinatory, allophonic variants, but the treatment of such interconnected problems as redundancy, predictability, and conditional probabilities in communication theory furthered a clarification of the relationship between the two basic linguistic classes of sound-properties – the distinctive features and the redundant features” (Jakobson 1971b [1961], p. 571–72).
- 17.
In generative theory, an explanatory model is a formal grammar based on explanatory principles, which must take into account the intuition of the native speaker. See the definition given by Chomsky in 1962: “What we seek, then, is a formalised grammar that specifies the correct structural descriptions with a fairly small number of general principles of sentence formation and that is embedded within a theory of linguistic structure that provides a justification for the choice of this grammar over alternatives. Such a grammar could properly be called an explanatory model, a theory of the linguistic intuition of a native speaker” (Chomsky 1962, p. 533).
- 18.
Waugh et al. (1990) note that the theory of relativity was very attractive for Jakobson as well as every notion common to physics, mathematics and linguistics.
- 19.
The journal Word was founded in 1943 together with the Linguistic Circle of New York by linguists, some of whom were recent immigrants fleeing from the Nazis, such as Roman Jakobson.
- 20.
In his study on Martinet and the Prague School, Verleyen (2007) insists on the influence of Zipf as a psychologist rather than as a statistician. He shows that Jakobson and Trubetzkoy conceived of language in diachrony as an organic totality where the speakers’ influence is rather low, while, on the contrary, Martinet thinks of the systematicity of changes in terms of the speaker’s properties. This explains, Verleyen argues, why Martinet has recourse to the psychologist Zipf whose principle of the least effort attempts to account for human behaviour in general.
- 21.
We do not know whether Martinet knew the article published in the Scientific American in 1952 (see Sect. 5.3.3 above) by Joshua Whatmough, a Harvard philologist.
- 22.
In the brief bibliography of the 1970 edition, Jakobson and Halle (1956) are only mentioned in connection with their binaristic and aprioristic conceptions of phonology, which Martinet criticises.
- 23.
- 24.
Whereas Dubois, in his PhD, mentions G.A. Miller’s Language and Communication (1951, French translation 1956), which constitutes an anti-behaviourist introduction to information theory for psychology students, he does not mention it in his later works.
Bibliography
Anderson, S.R. 1985. Phonology in the twentieth century. Chicago: The Chicago Press.
Auroux, S. 2009. Mathématisation de la linguistique et nature du langage. Histoire Épistémologie Langage 31 (1): 5–45.
Bar-Hillel, Y. 1953b. A Quasi-Arithmetic Notation for Syntactic Description. Language 29: 47–58.
Carnap, R., and Y. Bar-Hillel. 1952. An outline of a theory of semantic information Massachusetts Institute of Technology, Research Laboratory of Electronics, Technical Report 247, Oct. 27,1952. Reprinted in Bar-Hillel, Y. 1964. Language and Information, 221–274. Reading: Addison-Wesley.
Cherry, C. 1957. On Human Communication. Cambridge: The MIT Press.
Cherry, C., M. Halle, and R. Jakobson. 1953. Toward the Logical Description of Languages in their Phonemic Aspect. Language 29: 34–46.
Chevalier, J.-C. 2006. Combats pour la linguistique, de Martinet à Kristeva. Lyon: ENS Éditions.
Chomsky, N. 1955. The Logical Structure of Linguistic Theory. MIT.
———. 1962. Explanatory models in Linguistic. In Logic, Methodology and Philosophy of Science, ed. E. Nagel, P. Suppes, and A. Tarski, 528–550. Stanford: Stanford University Press.
Dubois, J. 1964a. La traduction de l’aspect et du temps dans le code français (structure du verbe). Le Français moderne 32 (1): 1–26.
———. 1964b. Compte-rendu de Roman Jakobson Essais de linguistique générale Les Éditions de Minuit. Le Français moderne 32 (4): 303–307.
———. 1965. Grammaire structurale du français, nom et pronom. Paris: Larousse.
———. 1967. Grammaire structurale du français, le verbe. Paris: Larousse.
———. 1969a. Grammaire structurale du français, la phrase et les transformations. Paris: Larousse.
Fehr, J. 2000. Visible Speech and Linguistic Insight. In Shifting Boundaries of the Real, Making the Invisible Visible, ed. H. Nowotny and M. Weiss, 31–47. Zürich: Hochschulverlag AG an der ETH.
———. 2003. Interceptions et interférences: la notion de ’code’ entre cryptologie, télécommunications et les sciences du langage. In History of Linguistics 1999, ed. S. Auroux, 363–372. Amsterdam: Benjamins (SiHoLS 99).
Guiraud, P. 1954. Les Caractères statistiques du vocabulaire. Paris: PUF.
Harris, Z.S. 1955. From phoneme to morpheme. Language 31: 190–222.
———. 1959. Linguistic Transformations for information retrieval. Proceedings of the International Conference on Scientific Information, Washington, DC, 16–21 November 1958, 937–950. Washington, DC: National Academy of Sciences, National Research Council.
———. 1968. Mathematical Structures of Language. New York: Wiley.
———. 1988. Language and Information. New York: Columbia University Press.
———. 1991. A Theory of Language and Information: A mathematical approach. Oxford/New York: Clarendon Press.
Hartley, R.V.L. 1928. Transmission of Information. Bell System Technical Journal 7: 535–563.
Hockett, C.F. 1953. Review: The Mathematical Theory of Communication by Claude L. Shannon and Warren Weaver. Language 29 (1): 69–93.
Jakobson, R. 1953. Patterns in linguistics. In Results of the conference of anthropologists and linguists, ed. C. Levi-Strauss, R. Jakobson, C.F. Voegelin, T. Sebeok. Memoir 8 of the International Journal of American Linguistics. Indiana University Publications in Anthropology and Linguistics:11–21.
———. 1958. Typological Studies and Their Contribution to Historical Comparative Linguistics. In Actes du 8e Congrès international des linguistes, ed. E. Sivertsen, 17–35. Oslo: Presses Universitaires d’Oslo.
———. 1971a [1959]. Boas’view of grammatical meaning. Selected Writings II, 489–496. s-Gravenhage: Mouton.
———. 1971b [1961]. Linguistics and Communication Theory. Selected Writings II, 570–579. s-Gravenhage: Mouton.
Jakobson, R., and M. Halle. 1956. Fundamentals of Language. The Hague: Mouton.
Jakobson, R., C.G. Fant, and M. Halle 1952. Preliminaries to Speech Analysis. The Distinctive Features and their Correlates. MIT Acoustic Laboratory, Technical report 13.
Léon, J. 2008a. Théorie de l’information, information et linguistes français dans les années 1960. Un exemple de transfert entre mathématiques et sciences du langage. In Actes du Congrès Mondial de Linguistique Française, ed. J. Durand, B. Habert, B. Laks. Paris, 9–12 juillet 2008. pp 923–938 https://doi.org/10.1051/cmlf08142
———. 2011a. S.Z. Harris and the semantic turn of Mathematical information theory. In: Hassler G (ed) History of Linguistics 2008. Selected Papers from the Eleventh International Conference on the History of Language Sciences, 28 August–2 September 2008 Postdam. Benjamins (SiHoLS 115), Amsterdam, pp 449–458.
Mandelbrot, B. 1954a. Structure formelle des textes et communication. Word 10 (3): 1–27.
Markov, A.A. 1913. Exemple d’une étude statistique d’un texte extrait de ‘Eugène Oniguine’ illustrant les probabilités liées. Bulletin de l’Académie Impériale des Sciences de St Pétersbourg: 153–162.
Martinet, A. 1955. Économie des changements phonétiques. Berne: Francke.
———. 1957a. Compte-rendu de George A. Miller Langage et communication PUF 1956. Bulletin de la Société de Linguistique 53: 25–26.
———. 1957b. Compte-rendu de Vitold Belevitch Langage des machines et langage humain Paris, Hermann 1956. Bulletin de la Société de Linguistique 53: 27–29.
———. 1957c. Substance phonique et traits distinctifs. Bulletin de la Société de Linguistique 53: 72–85.
———. 1970 [1960]. Eléments de linguistique générale. Paris: Armand Colin.
———. 1993. Mémoires d’un linguiste. Paris: Quai Voltaire.
Miller, G.A. 1951. Language and Communication. New York: McGraw Hill.
Petruszewycz, M. 1981. Les Chaînes de Markov dans le domaine linguistique. Genève/Paris: Éditions Slatkine.
Romashko, S. 2000. Vers l’analyse du dialogue en Russie. Histoire Épistémologie Langage 22 (1): 83–98.
Segal, J. 2003. Le Zéro et le Un. Histoire de la notion scientifique d’information au XXe siècle. Paris: Éditions Syllepse.
Sériot, P., ed. 2006. Nicolaï Troubetzkoy, Correspondance avec Roman Jakobson et autres écrits (traduction du russe par Patrick Sériot et Margarita Schoenenberger). Lausanne: Payot Lausanne.
Shannon, C.E. 1956. The Bandwagon. Institute of Radio Engineers, Transactions on Information Theory IT-2: 3.
Van de Walle, J. 2008. Roman Jakobson, Cybernetics and Information Theory: A Critical Assessment. Folia Linguistica Historica 29 (1): 87–124.
Verleyen, S. 2007. Le fonctionnalisme entre système linguistique et sujet parlant: Jakobson et Troubetzkoy face à Martinet. Cahiers Ferdinand de Saussure 60: 163–188.
Verleyen, S., and P. Swiggers. 2006. Causalité et conditionnement dans le fonctionnalisme diachronique. Folia Linguistica Historica 27 (1–2): 171–195.
Waugh, L.R., and M. Monville-Burston. 1990. On Language. Roman Jakobson. Cambridge: Harvard University Press.
Whatmough, J. 1952. Natural Selection in Language. Scientific American 186: 82–86.
Wiener, N. 1948. Cybernetics, or Control and Communication in the Animal and the Machine Paris. Hermann & Cie/The MIT Press/Wiley: New York.
Zipf, G.K. 1949. Human Behavior and the Principle of Least Effort. Cambridge: Addison Wesley.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Léon, J. (2021). Information Theory: Transfer of Terms, Concepts and Methods. In: Automating Linguistics. History of Computing. Springer, Cham. https://doi.org/10.1007/978-3-030-70642-5_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-70642-5_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-70641-8
Online ISBN: 978-3-030-70642-5
eBook Packages: Computer ScienceComputer Science (R0)