Advertisement

A Survey on Modeling Language Evolution in the New Millennium

  • Arianna D’Ulizia
  • Fernando Ferri
  • Patrizia GrifoniEmail author
Open Access
Article
  • 77 Downloads

Abstract

Language is a complex evolving system and it is not a trivial task to model the dynamics of processes occurring during its evolution. Therefore, modeling language evolution has attracted the interest of several researchers giving rise to a lot of models in the literature of the last millennium. This work reviews the literature devoted to computationally represent the evolution of human language through formal models and provides an analysis of the bibliographic production and scientific impact of the surveyed language evolution models to give some conclusions about current trends and future perspectives of this research field. The survey provides also an overview of the strategies for validating and comparing the different language evolution models and how these techniques have been applied by the surveyed models.

Keywords

Language evolution Grammatical evolution Evolutionary computation Agent-based models Game-theoretic models 

Introduction

Over the past years, the fascinating question “How did human language evolve?” has received a lot of answers from the research community. Several contributions have been proposed by researchers of many disciplines, ranging from anthropology and biology to linguistics, psychology, and computer science. In this interdisciplinary perspective of the language evolution study, anthropologists were mainly devoted to investigating how the nature of language and its functions are evolved and how this evolution has influenced other aspects of cultural life (e.g., interactions within societies, social identity, and group membership); biologists studied the evolution of language starting from the fundamentals of the language faculty and focusing on the distinctive behavioral mechanisms that enable the emergence of language; linguists focused on the language properties (e.g., morphemes, words, syntactic patterns, semantic structures, etc.) that can emerge and co-evolve within cultures; psychologists mainly addressed the mental processes and structures underlying the language use and evolution; computer scientists mainly aimed to better understand how specific computational mechanisms affect the outcome of observed linguistic phenomena. These perspectives are strongly connected with the representation of the complexity of the phenomenon of language evolution. Language, indeed, is a complex and non-linear dynamic system [48] and it is not a trivial task to provide a formal representation of the dynamics of processes occurring during its evolution. To this aim, computational modeling has become fundamental for investigating and simulating the behavior and long-term dynamics of human language [8, 11, 12]. Computational modeling has been notably applied in the new millennium giving rise to a great amount of language evolution models. These models have been surveyed by several authors [21, 23, 27, 55] in the literature. To advance the field of language evolution modeling, it is useful to consider the new developments by carrying out a bibliographic analysis of the most relevant models developed in this new millennium. Due to the ongoing interest in this research topic, we think that such an analysis is valuable to many researchers to reveal the developments in the field and to plan future research directions.

Therefore, the goal of the paper is to analyze bibliographic production and scientific impact of these language evolution models and the future trends and perspectives of this research field. In this analysis, we adopt the classification of language evolution models proposed by Grifoni et al. [23] and based on the computational method (agent-based, evolutionary computation-based, and game-theoretic models) and the grammatical formalism (context-free grammar-based, attribute grammar-based, Christiansen grammar-based, fluid construction grammar-based, and universal grammar-based models). We extended the analysis for papers in the period 2001–2017, related to the models identified. Specifically, we started from the ten language evolution models surveyed by Grifoni et al. [23], and we observed their bibliographic production for identifying computational methods and grammatical formalisms with the highest scientific impact over the years. Moreover, we discuss the strategies for validating the language evolution models usually applied in the literature and we give some results obtained by the authors of the models during their evaluation. Finally, we outline the most promising directions in this research field resulting from a brief interview with the authors of the surveyed models, the future work section of the surveyed papers, and the literature.

The remainder of the paper is organized as follows. The section “Analyzed Language Evolution Models” gives some information about language evolution models developed in the years 2001–2017, categorizing them according to grammatical representations and computational approaches. In the section “Bibliographic Production of Language Evolution Models”, an analysis of bibliographic production and citations of language evolution models is provided. The section “Validation of Language Evolution Models” overviews the validation strategies, usually applied in language evolution, and describes how they have been applied to evaluate and compare the surveyed language evolution models. In the section “Trends and Future Perspectives, a discussion about future trends and perspectives of language evolution models is given. The section “Conclusions” concludes the paper.

Analyzed Language Evolution Models

Many researchers of language evolution, mainly linguists and computer scientists, have paid considerable attention to understanding how the evolution of language can be computationally represented through a formal model [9, 54]. In the 17 years between 2001 and 2017, several models of language evolution have been produced. In this survey, we aim at analyzing trends of language evolution models developed in the new millennium. The analyzed models are taken starting from the survey of Grifoni et al. [23] that classifies these models according to a twofold point of view, representational and computational, and extended for the years until 2017.

The representational point of view investigates the grammatical representations that are used in language evolution models to represent the language. We are, therefore, interested in understanding how linguistic knowledge can be represented in formal computational models of human-language evolution. According to this point of view, language evolution models have been classified in the following five main categories: context-free grammar-based (CFG-based), attribute grammar-based (AG-based), Christiansen grammar-based (CG-based), fluid construction grammar-based (FCG-based), and universal grammar-based (UG-based). Although there are also non-grammatical models in the literature, here, we focused only on language evolution models that have a grammatical representation, by maintaining the perspective introduced in the survey of Grifoni et al. [23]. The reason for that lies primarily in the growing interest in the literature investigating the emergence of grammars and in the use of sophisticated and more realistic grammatical representations underlying the evolution of language [3, 50]. In our current research, we are particularly interested in investigating the emergence of grammars as we are going to develop new models based on grammatical representations.

The computational point of view investigates the computational methods that are used to process language evolution. According to this point of view, language evolution models have been classified into three main categories that are: agent-based, evolutionary computation-based, and game theoretic. A brief description of the categories of language evolution models discussed in the paper is given in Online Resource 1.

The adopted classification is depicted in Fig. 1.
Fig. 1

Classification of language evolution models by Grifoni et al. [23]

Hereafter, we rely on this classification when we refer to the categories of language evolution models. Table 1 summarizes the analyzed models according to the provided classification. Note that blank boxes in the table represent possibilities to combine computational methods and grammatical representations unexplored by the current literature, and could be the scope of future research works.
Table 1

Surveyed models of language evolution, their computational modeling paradigm, and grammatical representation

Grammatical representation

Computational methods

Agent-based

Evolutionary computation

Game theoretic

Iterated learning

Naming game

Genetic algorithm

Grammatical evolution

Context-free grammar

Cultural grammar system (CGS) [28]

GRAmmar EvoLution (GRAEL) [15]

GRAmmar EvoLution (GRAEL) [15]

Grammatical evolution by grammatical evolution (GE)2 [40]

 

Attribute grammar

   

Language Evolver (LEVER) [29]

Attribute Grammar Evolution (AGE) [13]

 

Christiansen grammar

   

Christiansen Grammar Evolution (CGE) [41]

 

Fluid construction grammar

 

FCGlight [44]

   

Universal grammar

Iterated Learning Model (ILM) [47]

   

Game dynamics (GD) [35]

Evolutionary game theory (EGT) [26]

In the remainder of this section, we provide some generic information about these models.

GRAEL (GRAmmar EvoLution) Framework [14, 15] provides an evolutionary computing approach to natural language grammar optimization and induction. GRAEL works with a population of agents, each of which holds a set of linguistic structures, represented using a CFG formalism that allows formulating sentences and analyzing other agents’ sentences.

LEVER (Language Evolver) [29] provides a tool for the evolutionary development and adaptation of the syntax, parser, and vocabulary of domain-specific languages (DSLs). LEVER uses attribute grammars as specification formalism for both syntax and semantics of a DSL.

Grammatical evolution by grammatical evolution (GE)2 [25] provides an evolutionary computing approach in which an input grammar, expressed using CFG notation, is used to specify the construction of another syntactically correct grammar.

Attribute Grammar Evolution (AGE) [13] is an evolutionary computation approach that extends the grammatical evolution, proposed by O’Neill and Ryan [39], by using AGs instead of CFGs.

Christiansen Grammar Evolution (CGE) [2, 41] is an automatic modeling tool that extends the grammatical evolution, proposed by O’Neill and Ryan [39], by using CGs instead of CFGs.

FCGlight [44] provides a framework for studying the evolution of natural language. It is based on FCGs, which allow the grammar to change, and on multi-agent language games.

Cultural Grammar Systems (CGS) [28] is a framework to formalize cultural dynamics of language evolution. It provides a syntactical framework based on CFGs and a population of agents.

Game Dynamics (GD) [6, 35] is a model of language evolution based on a mixed population, where each member has a genetically determined universal grammar and learns to speak one new grammar. The dynamics of language evolution is driven by a communication game.

Iterated Learning Model (ILM) [47] is a tool for investigating the cultural evolution of language. As suggested by the name, ILM applies iterated learning [31] to a population of agents that try to reconstruct a universal grammar through an inference process based on observation.

Evolutionary Game Theory (EGT) [46] was first developed by a theoretical biologist, Maynard Smith, in 1982. EGT has been applied to study language evolution by several authors, such as Jäger [26]. As suggested by the name, EGT relies on game theory for modeling the evolution of language structure, formalized using universal grammars.

Bibliographic Production of Language Evolution Models

In an attempt to gain a better understanding of the current trends regarding language evolution models, the analysis of the temporal evolution and scientific impact of articles being published in 2001–2014 and dealing with each of the models, previously introduced in the section “Analyzed Language Evolution Models”, has been carried out. The analysis of scientific production, based on bibliographic data, is one of the most widely used methods for obtaining indicators about temporal evolution, variations, and trends in a specific field of research. Several works [7, 58] have applied this kind of analysis in the study of research trends.

Consistent with the approach applied in these works, to find the most relevant papers to be included in our analysis, we start from the language evolution models surveyed by Grifoni et al. [23] and summarized in Table 1. Therefore, for each of the ten analyzed language evolution models, we have considered the number of published papers in the 14 year period 2001–2014 that we have obtained by the authors themselves by asking them for the bibliographic production of their language evolution models. This process yielded 52 papers to be included in our bibliographic analysis (see “Appendix A”). Moreover, we have integrated these papers with those resulted from a systematic search (using two relevant search engines, i.e. Web of Science (WoS) and Scopus) for scientific papers published from 2001 to 2017 (end of June) and dealing with the ten analyzed language evolution models surveyed by Grifoni et al. [23]. This process yielded further 32 papers to be included in our bibliographic analysis (see “Appendix A”, orange rows for papers retrieved from Scopus and green rows for papers retrieved from WoS).

Moreover, we have considered the number of citations and self-citations (retrieved from Google scholar at the end of July 2017) of these papers that give a measure of the scientific impact of these models. The analysis of the number of publications and the number of citations of each class of language evolution models is provided in the following sub-sections.

Analysis of the Scientific Production

As shown in Fig. 2, the total number of published papers (84 papers) in 2001–2017 varies from a minimum of 1 published paper in 2001 and 2017 to 14 published papers in 2007.
Fig. 2

Total number of published papers on surveyed language evolution models from 2001 to 2017 (end of June)

In particular, considering the classification of methods based on the computational modeling paradigm, the scientific production of agent-based models is distributed across all 17 reference years (see Fig. 3a), with a peak of 6 papers in 2003.
Fig. 3

Bibliographic production of language evolution models classified according to the computational modeling approach

The scientific production of evolutionary computation-based models has been concentrated in the period from 2002 to 2012 (see Fig. 3b), growing from 1 paper in 2002, reaching peaks of 5 papers in 2007, and concluding with 2 papers in 2012. Finally, game-theoretic models (see Fig. 3c) had the less continuous scientific production, with the first publications in 2003 and the last one in 2015, reaching a peak of four published papers in 2007 and no papers in the period 2009–2010.

Moreover, we can observe that half of the models were based on evolutionary computation and almost the half was based on agents and, only two models are based on game theory.

Agent-based models have the highest scientific production with 45 published papers, followed by evolutionary computation-based models with 25 published papers, and game-theoretic models with 21 published papers. However, the models belonging to the game-theoretic class are only two, compared with four models of the agent-based group and five models of the evolutionary computation-based group. Therefore, considering the average production per model, the agent-based models remain the one with the highest scientific production (11.25 papers/model), followed by game-theoretic models (10.5 papers/model), and by evolutionary computation-based models (5 papers/model). This analysis shows that the agent-based models are the most prolific in terms of published papers and they have the most continuous bibliographic production throughout the period 2001–2017.

Comparing the temporal evolution of the bibliographic production of language evolution models, as shown in Fig. 4, we can observe that agent-based models were the most applied models in the first years of the observed period (2001–2004). Subsequently, they were outclassed by evolutionary computation-based models, which were the most prolific from 2004 to 2007. Finally, agent-based models returned to being the most widely used from 2009 to 2017. This trend reflects perfectly the evolution that occurs in language evolution research. Since the 90s, indeed, several studies were developed for simulating language evolution in a bottom-up fashion using populations of agents. The developed agent-based models allow compensating for the lack of empirical evidence present in many language evolution theories developed during the 80s and 90s based on incomplete or absent evidence.
Fig. 4

Comparison of the bibliographic production of language evolution models

In the early twenty-first century, the majority of modeling efforts were concentrated to study the evolutionary dynamics of language transmission by applying various biological principles [e.g., reproduction, mutation, selection, recombination (crossover), and survival of the fittest].

These evolutionary computation-based models arise from the need to simplify complex agent-based models relying on sets of equations whose complexity grows exponentially with the complexity of the language to be modeled.

In the first 80s, a game-theoretic perspective was developed by Smith [46] for modeling the evolution of behavior. In 2003–2015, this perspective has been applied in game-theoretic models to study the evolution of language with the aim of aggregating the behavior of a population and defining general mathematical equations that model the evolution of this behavior. These models remain mainly conceptual and not largely applied, probably for the problems highlighted by Watumull and Hauser [57] concerning conceptual confusions and empirical deficiencies.

Afterward, we have considered the classification of models based on the grammatical representation and we have analyzed the scientific production of the five classes of models. CFG-based models have a scientific production distributed across 11 years (from 2002 to 2012), with peaks of three papers in 2003, 2005, 2007, and 2009 (see Fig. 5a). The scientific production of AG-based models has been concentrated in the period from 2005 to 2007 (see Fig. 5b), reaching a peak of 2 papers in 2007. Papers on CG-based models have been published during 2 years, 2007 and 2011, with two papers per year (see Fig. 5c). The scientific production of FCG-based models has been concentrated in the period from 2006 to 2017 (see Fig. 5d), with a peak of three papers in 2011. Finally, UG-based models (see Fig. 5e) had the most continuous scientific production, with the first publications in 2001 and the last one in 2016, reaching a peak of seven published papers in 2007.
Fig. 5

Bibliographic production of language evolution models classified according to the grammatical representation

UG-based models have the highest scientific production with 43 published papers, followed by CFG-based models with 21 published papers, FCG-based models with 12 papers, and AG-based and CG-based models with 4 published papers. However, only one model belongs to the CG-based and FCG-based classes, compared with three models of CFG-based and UG-based groups. Therefore, considering the average production per model, the UG-based models remain the class with the highest scientific production (14.33 papers/model), followed by FCG-based models (12 papers/model), CFG-based models (7 papers/model), CG-based models (4 papers/model), and AG-based models (2 papers/model).

This analysis shows that the UG-based models are the most prolific in terms of published papers and they have the most continuous bibliographic production throughout the period 2001–2017. This fact can be justified by the fact that UGs, as well as CFGs, provide a general theoretic grammatical framework with very few constraints that can be easily adapted to represent linguistic evolution. On the contrary, CG-based, AG-based, and FCG-based models provide an attempt to apply specialized grammars for representing the evolution of domain-specific languages, and therefore, they have not had a large following.

Analysis of Citations

Analyzing the citation count, agent-based models had the highest scientific impact with 3865 citations (retrieved from Google scholar at the end of July 2017). In the number of citations of agent-based models, a considerable weight is represented by ILM that alone has 3673 citations (including 367 self-citations). Agent-based models are followed by game-theoretic models that received 851 citations, and finally evolutionary computation-based models with 322 citations (see Fig. 6). However, since the models belonging to the game-theoretic class are only two, compared with four models of the agent-based group and five models of the evolutionary computation-based group, we consider the average citations per model. According to that, the agent-based models remain the one with the highest citations (966.25 citations/model), followed by game-theoretic models (425.5 citations/model), and by evolutionary computation-based models (64.4 citations/model).
Fig. 6

Number of citations of papers published about the analyzed language evolution models classified according to the computational modeling approach

We have also considered the self-citations of the language evolution models that are depicted in Fig. 7. Note that for evaluating self-citations, we considered the papers listed in Appendix A, and for each paper, we count the citing papers that have at least one author in common. For the agent-based models, the percentage of self-citing out of the overall citations tends to be lower (10.71%), while it increases for game-theoretic models (12.57%) and it is the highest for evolutionary computation-based models (31.36%). Therefore, even if we exclude self-citations in the citation count, the result remains unchanged.
Fig. 7

Percentage of citations/self-citations of papers published about the ten analyzed language evolution models classified according to the computational modeling approach

Analogously, the analysis of the citations considering the grammatical representation showed that UG-based models had the highest scientific impact with 4524 citations, followed by CFG-based models that received 212 citations, FCG-based models with 142 citations, AG-based models with 72 citations, and finally CG-based models that received 54 citations (see Fig. 8). Considering the average value of citations per model, the UG-based models remain the class with the highest citations (1508 citations/model), followed by FCG-based models (142 citations/model), CFG-based models (70.6 citations/model), CG-based models (54 citations/model), and AG-based models (36 citations/model). We have also considered the self-citations of the language evolution models that are depicted in Fig. 9. For the UG-based models, the percentage of self-citing out of the overall citations tend to be very low (10.48%), followed by AG-based models (15.28%), FCG-based models (24.65%), CG-based models (27.78%), and CFG-based models (41.04%). Therefore, if we exclude self-citations in the citation count, UG-based models remain the class with the highest citation impact.
Fig. 8

Number of citations of papers published about the ten analyzed language evolution models classified according to the grammatical representation

Fig. 9

Percentage of citations/self-citations of papers published about the ten analyzed language evolution models classified according to the grammatical representation

From the analysis of these results, we can observe that the current trend in language evolution models is oriented toward the use of agent-based and UG-based models, both for the highest number of published papers and the highest citation count. The reason for that relies on the fact that agent-based models are more suited for simulating the evolution of complex systems composed of behavioral entities (and human-language evolution falls in this category) and they make the model closer to the real behavior. Moreover, the general theoretic grammatical framework provided by UGs turns out to be easily adapted to represent linguistic evolution.

Validation of Language Evolution Models

Language evolution models are notoriously difficult to validate empirically, mainly because of the lack of valid data on the earliest human languages. Despite that, various approaches have been developed in the literature based on analytic techniques, computer simulation, and experiments. The section “Validation Strategies” discusses the main features of these three validation strategies usually applied in language evolution. Next, a summary of how these techniques have been applied to the ten language evolution models, analyzed in this paper, is given in the section “Validating language evolution models”, along with some results of the validation, obtained by the authors in their original works. The analysis of these results will allow giving some indications of the advantages and shortcomings of each language evolution model.

Validation Strategies

Generally, the validation of language evolution models is carried out by comparing the outcomes of the model with the reality it exemplifies. The different validation techniques applied in the literature can be grouped into three strategies: analytic techniques, computer simulation, and experiments.

Analytic techniques use mathematical equations that typically describe the evolving system. Solving these equations allows predicting the global evolution of the system. The global quantities used in analytic models are normally measured by empirical observation. If these data cannot be observed, as in language evolution, this strategy can be applied to the outcome of computer simulation. The main disadvantage of this strategy stays in the fact that finding the global quantities and the mathematical equations that describe language evolution is not a trivial task, even for a large number of non-linear dynamical systems, no solution can be found. As language evolution falls into this category, it is hard to formulate equations that are powerful enough to produce verifiable predictions.

Computer simulation allows studying the dynamics of language evolution, reconstructing the trajectories of changes, and recapitulating the effect of relevant factors on evolution [22]. It attempts to simulate the conceptual model of a system by a computer program [4]. This validation strategy also includes embodied simulations that use hardware-based models, such as robots. The computer program contains a set of hypotheses on the causes, mechanisms, and processes that govern the analyzed phenomenon represented by the model. Running the program allows observing and manipulating the parameters, conditions, and variables that control the phenomenon represented by the model, and to observe the responses to these manipulations. Computer simulation is particularly useful in cases where analytical models are not applicable due to the high complexity or non-linearity of the modeled system. In these cases, simulation allows testing language evolution models in a virtual experimental laboratory. Simulation also provides a more practical way of discovering new predictions that can be derived from the model. On the contrary, the main disadvantage of computer simulation is that the results could vary greatly in the real world due to unforeseen factors. Moreover, it can be quite expensive in terms of time and necessary resources.

Experiments consist of the examination of the real system that has been modeled and in the demonstration that specific outcomes occur when certain environmental parameters or system condition is changed. In the specific field of language evolution, natural experiments should involve humans and their brain reactions for observing how the language evolves. First attempts of natural experiments for validating language emergence and evolution were reviewed by Steels [49]; he argued that natural experiments are not sufficiently controllable for being a solid experimental method for language evolution. Afterward, Scott-Phillips and Kirby [45] reviewed laboratory-based experiments that use human participants to observe both the cognitive capacities required for language and the ways symbolic communication systems emerge and evolve. In addition to natural experiments, also artificial experiments may be performed, which use robots for reproducing human perceptive, cognitive, and linguistic abilities, and manipulate them for observing the emergence and evolution of language. Although artificial experiments have some characteristics in common with computer simulation, experiments require both more rigorous assumptions that need to be implemented in the robot and a more stringent way to test the realism of these assumptions [49]. For instance, if we want to validate the capacity to learn a new vocabulary word, using artificial experiments, we have to implement the perception and memory capacities of the robot, while it is not necessary using computer simulation. This is the main reason that prevents the use of artificial experiments towards computer simulation.

Validating Language Evolution Models

The ten language evolution models have been evaluated by the authors of the models using different validation techniques, described in the previous section.

Table 2 summarizes the validation strategy applied to each language evolution model. Two models (LEVER and CGS) have not been validated by their authors; therefore, we do not take them into account in the following discussion.
Table 2

Validation strategies of surveyed models of language evolution

 

Validation strategies

Analytic techniques

Computer simulation

Experiments

GRAEL

 

X

 

(GE)2

X

  

AGE

X

  

CGE

X

  

Game Dynamics

 

X

 

ILM

 

X

 

EGT

 

X

 

FCGligth

  

X

LEVER

Not validated

CGS

Not validated

Table 2 shows that the majority of models have been evaluated using computer/robotic simulation. Specifically, it is used mainly by agent-based models and game-theoretic models, while analytic techniques are used mainly by evolutionary computation-based models. The reason for that relies on the more theoretic nature of this last kind of language evolution model. On the contrary, agent-based models are better suited to be validated by simulation due to their more empirical nature. Moreover, game-theoretic models apply computer simulation for validating the theoretical hypotheses of stability and equilibrium.

The validation process of each of the seven validated language evolution models and obtained results are discussed in the following sub-sections.

Language Evolution Models Validated Through Analytic Techniques

Analytic techniques have been used to evaluate three of the ten language evolution models, belonging to the class of evolutionary computation-based models, as shown in the first column of Table 2.

Generally, the metrics used for validating evolutionary computation-based models (in particular, GE, AGE, and CGE) is the cumulative frequency of success, which is usefully applied in evolutionary computation for measuring the probability of finding a solution to a problem in a specific number of generations. It is defined as the number of runs where the solution to the problem was found [24]. GE, AGE, and CGE have been validated and compared using this metrics. Specifically, Table 3 shows the parameters that the authors of these three language evolution models have adopted for the experiments, i.e., the population size in terms of number of individuals used for the genetic algorithm, and crossover and mutation ratio, which are the probabilities of generating new individuals by crossover and mutation operations, and, finally, the number of generations, which represents the maximum number of runs of the algorithm. The last column of Table 3 shows the performance of the models in terms of cumulative frequency of success.
Table 3

Results of the validation of some language evolution models through analytic techniques

 

Parameters for the experiments

Metrics

Population size (individuals)

Crossover ratio

Mutation ratio

Number of generations

Cumulative frequency of success (%)

GE

500

0.9

0.7

100

97

500

0.9

0.9

100

79

AGE

500

0.9

0.1

100

82

500

0.9

0.7

100

95

CGE

500

0.9

0.9

324

100

500

0.9

0.9

200

97

500

0.9

0.9

100

76

Comparing the results of the validation is not a feasible task, due to the differences in mutation ratio. However, the authors of the models provided some comparative results in their works. Ortega et al. [41] compared GE and CGE performance, showing that the cumulative success frequency of GE is 79% and CGE is 76% after 100 runs of the algorithms. Moreover, a comparison between GE and AGE is performed in [13] showing a cumulative success frequency of 97% for GE and 95% for AGE after 100 runs of the algorithms. Therefore, GE results in higher performance than AGE and CGE.

Language Evolution Models Validated Through Computer Simulation

Computer simulation has been used to validate the following four language evolution models: GRAEL, ILM, GD, and EGT.

GRAEL is validated using the F1 score (or F score), which provides a measure of the experiment’s accuracy and can be interpreted as a weighted average of the precision and recall metrics. F1 score is formally defined as:
$$f_{1} {\text{score}} = 2 \times \frac{{{\text{precision}} \times {\text{recall}}}}{{{\text{precision}} + {\text{recall}}}},$$
where the recall gives a measure of the completeness of the model as it is the number of correct results divided by the number of results that should have been returned, and the precision shows how correct the model is as it measures the number of correct results divided by the number of all returned results.

The Wall Street Journal (WSJ) [33] is used as a corpus of annotated sentences for training and testing the population of 100 agents engaged in a series of language games. The obtained value of F1 score (around 81% [15]) indicates that the GRAEL model performs quite well; that is, the mutated grammar is able to create new evolved parses for understanding more difficult constructions.

ILM is validated using compositionality and (communicative) accuracy. The former is defined as a representation of how the meaning of the whole can be described as a function of the meaning of its parts. Compositionality is calculated as the proportion between the number of compositional rules used (both encoded and decoded) and the total number of utterances produced and interpreted [52]. A high value of compositionality means the emergence of linguistic structures in the language evolution process. The latter is calculated as the fraction of agents that could successfully interpret the produced utterances of the other agents in the population, averaged over the number of games played during the testing phase [53]. The ILM experiment consists of language games run with a population of 2 agents (1 adult and 1 learner) for 250 iterations. In each game, the adult encodes an utterance to convey the meaning of one of 120 objects, while the learner decodes this utterance constructing its private grammar ontogenetically. At the end of the 250 iterations, the results showed a compositionality that is around 0.89 and an accuracy of around 0.85 [53]. This means that ILM performs well in modeling the emergence and evolution of compositional languages.

Game-theoretic models (i.e., GD and EGT) do not apply specific metrics, because they do not compare the results of the simulation against the expected results, but they use computer simulation for addressing trajectories of evolutionary change, revealing how the language of modeled populations changes over evolutionary time (e.g., one equilibrium is reached, the system cycles endlessly, etc.). Specifically, GD is validated simulating a communication game between two speakers with various probabilities of understanding each other and analyzing under which conditions the UGs learned by the two speakers are evolutionarily stable against invasion by each other. EGT is validated simulating the emergence of a protolanguage in an initially prelinguistic society consisting of 100 individuals. Each of them plays a round of the game, which consists of interacting with every other individual with the aim of associating five objects to five signals (sounds). At the end of each round, the total payoff of all individuals is calculated, according to the communication success, and a proportional number of offspring is generated. After 20 rounds, EGT reaches an evolutionarily stable solution.

Computer simulation strategy does not provide a unique metrics of the performance of the language evolution models. GRAEL, indeed, used the F1 score, ILM adopted the compositionality and accuracy, and finally, GD and EGT evaluated the evolutionary stability. This is the main reason that makes these models incomparable.

Language Evolution Models Validated Through Experiments

Experiments using artificial agents are used to validate the FCGligth model. Specifically, the learning strategy is embodied in a language game that is played by a population of robotic agents. The learning strategy consists of the following three-step learning process: (i) use of holophrases for imitating the teacher; (ii) learning of item-based construction; (iii) acquisition of abstract construction.

The main difficulty of this validation strategy stays in implementing really in the robotic agents all the necessary formalisms for representing linguistic knowledge and for orchestrating the parsing, production, and learning processes.

No metrics to evaluate the performance of the language evolution model are provided by the authors.

Trends and Future Perspectives

From the analysis of the bibliographic production and scientific impact of the ten analyzed language evolution models, we can observe that their current trend is oriented toward the use of agent-based and UG-based models, both for the highest number of published papers and the highest citation count. The reason for that relies on the fact that agent-based models are more suited for simulating the evolution of complex systems composed of behavioral entities (and human-language evolution falls in this category) and they make the model closer to the real behavior thanks to the support of robust empirical evidence.

The need for empirical evidence comes from the necessity to go beyond idealizations and approximations of the language evolution phenomena. Without empirical evidence, indeed, the language evolution process is only numerically determined and, consequently, that can lead to an unrealistic representation of the reality. Therefore, from 2001 to 2017, the research on language evolution is evolved toward the use of agent-based models because supported by empirical evidence compared to evolutionary computation-based and game-theoretic models.

The motivation for the use of UG-based models relies on the fact that the general theoretic grammatical framework provided by UGs turns out to be easily adapted to represent linguistic evolution. Despite that, the necessity of grammatical formalisms equipped with structures and constructions able to represent semantic features of the language is also emerged during the surveyed period (see Fig. 10). In particular, models developed after 2005 (AGE, LEVER, CGE, and FCGlight) were oriented towards adding semantics and adaptability to the language representation.
Fig. 10

Evolution of the ten surveyed language evolution models with respect to the semantic representation

With regard to the future trend of language evolution, we asked the authors of the ten analyzed language evolution models whether and how their research on language evolution models is evolving in recent years. In Table 4, a summary of the answers received from authors about the evolution of their research is given. Most of the authors did not continue this research after the development of the model due to various reasons, mainly the end of project funding and a different research agenda (GRAEL, FCGLight, ILM, and EGT). Some authors (GE2) have focused their research on alternative grammatical formalisms by experimenting on how the language evolution model performs with different kinds of context-free and context-sensitive grammars. Some authors (LEVER) have worked on a further abstraction of the language evolution process using metamodels and representing the language evolution as a transformation between metamodels of language. Finally, the authors of GD have focused their research on the cognitive aspects of language evolution studying the phenomenon at the neural synaptic level and trying to simulate through neural networks the evolution that happens in human language.
Table 4

Answers from authors about the evolution of their research on language evolution models

 

Research evolution

GRAEL

The authors did not continue research on language evolution, due to reasons of time and a different research agenda

LEVER

The authors evolved their research on language evolution towards modeling languages tailored to a specific domain and defined by a metamodel. They faced the problem of migrating existing models to a new version of their metamodel and proposed an approach, named COPE, that specifies the coupled evolution of metamodels and models to reduce migration effort

(GE)2

After the publication of the (GE)2 model, the authors have explored the use of different types of grammars ranging from context-free to context-sensitive including Tree-adjunct Grammars [36], Attribute Grammars [37], and Shape Grammars [34, 38]

AGE

The authors have explored the use of grammatical evolution to obtain an ecology of artificial beings associated with mathematical functions [1] to generate artificial ecologies that exhibit some of the features of natural evolution

CGE

The authors did not have new projects in this direction

FCGlight

The authors did not continue to work on the FCG light topic, due to the end of project funding and involvement in other research areas

GD

The authors have explored how to set up stochastic dynamics to represent a population of language learners that can spontaneously move from one equilibrium point to another in a way that resembles documented language change

They worked also on a simulation of the evolution of neural synaptic coding, to evolve neural networks and manipulate information in something vaguely like what happens in language

ILM

The authors did not continue to work on the ILM model for language evolution

EGT

The authors did not continue to work on the EGT model for language evolution

Looking at possible future perspectives in language evolution research, in our opinion, one of the main open challenges consists in the necessity of advancements in neuroscientific research, as expressed also by several authors in their scientific works [30]. Neuroscience, indeed, represents the gateway to understanding the biological mechanisms of language and, consequently, can provide the empirical evidence of the neural processes allowing the formulation of new hypotheses about language evolution. This challenge matches also with the research undertaken by the authors of GD (see Table 4).

As a further future perspective, language evolution models should take into account multimodal aspects of language. Many models developed in the literature have followed a unimodal approach, according to which language is expressed in a single modality, mainly speech and/or text, “thus ignoring the wealth of additional information available in face-to-face communication” [51]. However, other significant research has been conducted in recent years [23, 32, 51, 56] that highlights the importance of abandoning the traditional distinctions among modalities in language evolution research and pursuing, instead, an integrated vision that combines all modalities (such as gestures, facial expressions, etc.) into a multimodal language [10, 16, 17, 18, 19, 20, 30, 42, 43]. Caschera et al. [5] also highlighted the necessity of tools for modeling the evolution of multimodal dialog in long-term changing situations. Therefore, we envision for the next years a research effort towards the development of multimodal approaches to modeling language evolution.

Conclusions

In this survey, computational modeling approaches and grammatical representations applied in language evolution models have been taken into account. The surveyed models have been analyzed considering their scientific production, obtaining indicators about temporal evolution, scientific impact, and trends in the field of language evolution. Agent-based and UG-based models turn out to be the most published and cited classes of language evolution models.

The great number of proposed language evolution models has given rise to the need for systematic validation and comparison. The paper has reviewed three main validation strategies proposed in the literature: analytic techniques, computer simulation, and experiments. The majority of language evolution models have been validated using computer simulation, mainly due to their ability to conduct more practical tests in a virtual experimental laboratory. Simulation, indeed, is useful to identify interesting experimental setups, which experiments are costly and hard to identify.

Notes

Supplementary material

354_2019_79_MOESM1_ESM.docx (24 kb)
Supplementary material 1 (DOCX 24 kb)

References

  1. 1.
    Alfonseca, M., Soler Gil, F.J.: Evolving an ecology of mathematical expressions with grammatical evolution. Biosystems 111(2), 111–119 (2013)CrossRefGoogle Scholar
  2. 2.
    Bel-Enguix, G., Christiansen, H., Jiménez-López, M.D.: A grammatical view of language evolution. In: Proceedings of the 1st International Workshop on AI Methods for Interdisciplinary Research in Language and Biology—BILC 2011, Rome, January 29, 2011, pp. 57–66 (2011)Google Scholar
  3. 3.
    Beuls, K., Steels, L.: Agent-based models of strategies for the emergence and evolution of grammatical agreement. PLoS One 8(3), e58960 (2013)CrossRefGoogle Scholar
  4. 4.
    Cangelosi, A., Parisi, D.: Computer simulation: a new scientific approach to the study of language evolution. In: Cangelosi A., Parisi D. (eds.) Simulating the Evolution of Language. Springer, London, pp. 3–28 (2002)zbMATHCrossRefGoogle Scholar
  5. 5.
    Caschera, M.C., D’Ulizia, A., Ferri, F.: Grifoni, P.: Towards evolutionary multimodal interaction. In: On the Move to Meaningful Internet Systems: OTM 2012 Workshops, Springer Berlin Heidelberg, pp. 608-616 (2002)CrossRefGoogle Scholar
  6. 6.
    Chatterjee, K., Zufferey, D., Nowak, M.A.: Evolutionary game dynamics in populations with different learners. J. Theor. Biol. 301, 161–173 (2012)MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    Chen, H., Jiang, W., Yang, Y., Yang, Y., Man, X.: Global trends of municipal solid waste research from 1997 to 2014 using bibliometric analysis. J. Air Waste Manag. Assoc. 65(10), 1161–1170 (2015)CrossRefGoogle Scholar
  8. 8.
    Christiansen, M.H., Kirby, S.: Language evolution: consensus and controversies. Trends Cognit. Sci. 7(7), 300–307 (2003)CrossRefGoogle Scholar
  9. 9.
    Croft, W.: Evolutionary linguistics. Annu. Rev. Anthropol. (Annual Reviews) 37, 219–234 (2008)CrossRefGoogle Scholar
  10. 10.
    D’Andrea, A., D’Ulizia, A., Ferri, F., Grifoni, P.: EMAG: an extended multimodal attribute grammar for behavioural features. Dig. Schol. Hum. 32(2), 251–275 (2017)Google Scholar
  11. 11.
    de Boer, B.: Emergence of vowel systems through self-organisation. AI Commun. 13(1), 27–39 (2000)Google Scholar
  12. 12.
    de Boer, B.: Computer modelling as a tool for understanding language evolution. Evolutionary Epistemology, Language and Culture—A Non-adaptationist, Systems Theoretical Approach, pp. 381–406. Springer, Dordrecht (2006)CrossRefGoogle Scholar
  13. 13.
    de la Cruz, M., de la Puente, A.O., Alfonseca, M.: Attribute grammar evolution, artificial intelligence and knowledge engineering applications: a bioinspired approach. In: First International Work-Conference on the Interplay Between Natural and Artificial Computation, IWINAC 2005, Las Palmas, Canary Islands, Spain, June 2005, pp. 182–191 (2005)Google Scholar
  14. 14.
    De Pauw. G.: Evolutionary computing as a tool for grammar development. In: Proceedings of GECCO 2003, Chicago, IL, USA, July 12–16 2003, LNCS 2723. Berlin Heidelberg, pp. 549–560 (2003)Google Scholar
  15. 15.
    De Pauw, G:. GRAEL: an agent-based evolutionary computing approach for natural language grammar development. In: Proceedings of the 18th international joint conference on Artificial intelligence, August 09–15, 2003, Acapulco, Mexico, pp. 823–828 (2003)Google Scholar
  16. 16.
    D’Ulizia, A., Ferri, F.: Formalization of multimodal languages in pervasive computing paradigm. In: Advanced Internet Based Systems and Applications, Second International Conference on Signal-Image Technology and Internet-Based Systems (SITIS 2006), Revised Selected Papers, Springer, Lecture Notes in Computer Science 4879, pp. 126–136 (2006)Google Scholar
  17. 17.
    D’Ulizia, A., Ferri, F., Grifoni, P.: A Hybrid grammar-based approach to multimodal languages specification. In: OTM 2007 Workshop Proceedings, 25–30 November 2007, Vilamoura, Portugal, Springer-Verlag, Lecture Notes in Computer Science 4805, pp 367–376 (2007)Google Scholar
  18. 18.
    D’Ulizia, A., Ferri, F., Grifoni, P.: Toward the Development of an Integrative Framework for Multimodal Dialogue Processing. In: On the Move to Meaningful Internet Systems: OTM 2008 Workshops Springer Berlin Heidelberg, pp. 509-518 (2008)CrossRefGoogle Scholar
  19. 19.
    D’Ulizia, A., Ferri, F., Grifoni, P.: A learning algorithm for multimodal grammar inference. IEEE Trans. Syst. Man Cybern. Part B (Cybernetics) 41(6), 1495–1510 (2011)CrossRefGoogle Scholar
  20. 20.
    Ferri. F., D’Ulizia, A., Grifoni, P.: Multimodal language specification for human adaptive mechatronics. arXiv preprint arxiv:1703.05616 (2017)
  21. 21.
    Ferri, F., D’Ulizia, A., Grifoni, P.: Computational models of language evolution: challenges and future perspectives. J. Univ. Comput. Sci. 24(10), 1345–1377 (2018)Google Scholar
  22. 22.
    Gong, T., Shuai, L., Comrie, B.: Evolutionary linguistics: theory of language in an interdisciplinary space. Lang. Sci. 41, 243–253 (2014)CrossRefGoogle Scholar
  23. 23.
    Grifoni, P., D’Ulizia, A., Ferri, F.: Computational methods and grammars in language evolution: a survey. Artif. Intell. Rev. 45(3), 369–403 (2016)CrossRefGoogle Scholar
  24. 24.
    Heder, B., Helio, B.: Grammar-based immune programming. Nat. Comput. 10(1), 209–241 (2011)MathSciNetzbMATHCrossRefGoogle Scholar
  25. 25.
    Hemberg, E., O’Neill, M., Brabazonm A.: Grammatical bias and building blocks in meta-grammar grammatical evolution. In: J. Wang (Ed.), 2008 IEEE World Congress on Computational Intelligence, Hong Kong, June 1-6, 2008, pp 3776–3783 (2008)Google Scholar
  26. 26.
    Jäger, G.: Evolutionary Game Theory and Typology: a Case Study. Language 83(1), 74–109 (2007)CrossRefGoogle Scholar
  27. 27.
    Jaeger, H., Baronchelli, A., Briscoe, E., Christiansen, M.H., Griffiths, T., Jäger, G., Kirby, S., Komarova, N., Richerson, P.J., Steels, L., Triesch, J.: What can mathematical, computational and robotic models tell us about the origins of syntax?, book chapter in “Biological Foundations and Origin of Syntax”. In: Bickerton, D., Szathmáry, E. (eds.) Strüngmann Forum Reports, vol. 3, pp. 385–410. MIT Press, Cambridge (2009)Google Scholar
  28. 28.
    Jimenez-Lopez, M.D.: A grammar-based multi-agent system for language evolution. Highlights PAAMS, AISC 156, 45–52 (2012)Google Scholar
  29. 29.
    Juergens, E., Pizka, M.: The language evolver lever—tool demonstration. Electron. Notes Theoret. Comput. Sci. 164(2), 55–60 (2006)CrossRefGoogle Scholar
  30. 30.
    Kanero, J.: The gesture theory of language origins: current issues and beyond. In: McCrohon, L., Thompson, B., Verhoef, T., Yamauchi, B. (eds.) The Past, Present and Future of Language Evolution Research, pp. 1–7. EvoLang9 Organising Committee, Tokyo (2014)Google Scholar
  31. 31.
    Kirby, S.: Spontaneous evolution of linguistic structure-an iterated learning model of the emergence of regularity and irregularity. IEEE Trans. Evol. Comput. 5(2), 102–110 (2001)CrossRefGoogle Scholar
  32. 32.
    Levinson, S.C., Holler, J.: The origin of human multi-modal communication. Philos. Trans. R. Soc. B 369, (2014)CrossRefGoogle Scholar
  33. 33.
    Marcus, M., Santorini, B., Marcinkiewicz, M.: Building a large annotated corpus of English: the Penn treebank. Comput. Linguist. 19(2), 313–330 (1993)Google Scholar
  34. 34.
    McDermott, J., Swafford, J.M., Hemberg, M., Byrne, J., Hemberg, E., Fenton, M., McNally, C., Shotton, E., O’Neill, M.: An assessment of string-rewriting grammars for evolutionary architectural design. Environ. Plan. B 39(4), 713–731 (2012)CrossRefGoogle Scholar
  35. 35.
    Mitchener, W.G.: Game dynamics with learning and evolution of universal grammar. Bull. Math. Biol. 69(3), 1093–1118 (2007)MathSciNetzbMATHCrossRefGoogle Scholar
  36. 36.
    Murphy, E., O’Neill, M., Galvan-Lopez, E., Brabazon, A.: Tree-adjunct grammatical evolution. In: IEEE Congress on Evolutionary Computation 2010 IEEE Press Barcelona, Spain (2010)Google Scholar
  37. 37.
    O’Neill, M., Cleary, R., Nikolov, N.: Solving knapsack problems with attribute grammars. Proceedings of the Third Grammatical Evolution Workshop GEWS 2004, WA, USA (2004)Google Scholar
  38. 38.
    O’Neill, M., McDermott, J., Swafford, J.M., Byrne, J., Hemberg, E., Shotton, E., McNally, C., Brabazon, A., Hemberg, M.: evolutionary design using grammatical evolution and shape grammars: designing a shelter. Int. J. Des. Eng. 3(1), 4–24 (2010)Google Scholar
  39. 39.
    O’Neill, M., Ryan, C.: Grammatical Evolution: Evolutionary Automatic Programming in an Arbitrary Language. Kluwer Academic Publishers, Norwell (2003)zbMATHCrossRefGoogle Scholar
  40. 40.
    O’Neill, M., Ryan, C.: Grammatical evolution by grammatical evolution: the evolution of grammar and Genetic Code. LNCS 3003, 138–149 (2004)Google Scholar
  41. 41.
    Ortega, A., De La Cruz, M., Alfonseca, M.: Christiansen grammar evolution: grammatical evolution with semantics. Evol. Comput. IEEE Trans. 11(1), 77–90 (2007)CrossRefGoogle Scholar
  42. 42.
    Paulmann, S., Jessen, S., Kotz, S.A.: Investigating the multimodal nature of human communication: insights from ERPs. J. Psychophysiol. 23(2), 63–76 (2009)CrossRefGoogle Scholar
  43. 43.
    Regenbogen, C., Schneider, D.A., Gur, R.E., Schneider, F., Habel, U., Kellermann, T.: Multimodal human communication—targeting facial expressions, speech content and prosody. NeuroImage 60(4), 2346–2356 (2012)CrossRefGoogle Scholar
  44. 44.
    Saveluc, V., Ciortuz, L.: FCGlight: a system for studying the evolution of natural language. In: 12th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing SYNASC 2010, Timisoara, Romania, 23–26 September 2010, IEEE, pp. 188–193, (2010)Google Scholar
  45. 45.
    Scott-Phillips, T.C., Kirby, S.: Language evolution in the laboratory. Trends Cognit. Sci. 14(9), 411–417 (2010)CrossRefGoogle Scholar
  46. 46.
    Smith, J.M.: Evolution and the Theory of Games. Cambridge University Press, Cambridge (1982)zbMATHCrossRefGoogle Scholar
  47. 47.
    Smith, K., Kirby, S., Brighton, H.: Iterated learning: a framework for the emergence of language. Artif. Life 9(4), 371–386 (2003)CrossRefGoogle Scholar
  48. 48.
    Steels, L.: The synthetic modelling of language origins. Evol. Commun. 1, 1–34 (1997)CrossRefGoogle Scholar
  49. 49.
    Steels, L.: how to do experiments in artificial language evolution and why. In: The Evolution of Language Proceedings of the 6th International Conference EVOLANG6, pp 323-332 (2006)Google Scholar
  50. 50.
    Steels, L., Szathmáry, E.: The evolutionary dynamics of language. BioSystems 164, 128–137 (2018)CrossRefGoogle Scholar
  51. 51.
    Vigliocco, G., Perniss, P., Vinson, D.: Language as a multimodal phenomenon: implications for language learning, processing and evolution. Philos. Trans. R. Soc. B. 369, 1–7 (2014)CrossRefGoogle Scholar
  52. 52.
    Vogt, P.: Meaning development versus predefined meanings in language evolution models. Int. Jt. Conf. Artif. Intell. 19, 1154–1159 (2005)Google Scholar
  53. 53.
    Vogt, P.: The emergence of compositional structures in perceptually grounded language games. Artif. Intell. 167(1–2), 206–242 (2005)CrossRefGoogle Scholar
  54. 54.
    Vogt, P.: Language evolution and robotics: issues in symbol grounding and language acquisition. In: Loula, A., Gudwin, R., Queiroz, J. (eds.) Artificial Cognition Systems, pp. 176–209. Idea Group Publishing, Hershey, PA (2006)Google Scholar
  55. 55.
    Vogt, P.: Modeling interactions between language evolution and demography. Hum. Biol. 81(2), 237–258 (2009)CrossRefGoogle Scholar
  56. 56.
    Waller, B., Liebal, K., Burrows, A., Slocombe, K.: How can a multimodal approach to primate communication help us understand the evolution of communication? Evol. Psychol. 11(3), 538–549 (2013)CrossRefGoogle Scholar
  57. 57.
    Watumull, J., Hauser, M,D.: Conceptual and empirical problems with game theoretic approaches to language evolution. Front. Psychol. 5, 1–4 (2014)CrossRefGoogle Scholar
  58. 58.
    Xie, S., Zhang, J., Ho, Y.S.: Assessment of world aerosol research trends by bibliometric analysis. Scientometrics 77(1), 113–130 (2008)CrossRefGoogle Scholar

Copyright information

© The Author(s) 2019

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  • Arianna D’Ulizia
    • 1
  • Fernando Ferri
    • 1
  • Patrizia Grifoni
    • 1
    Email author
  1. 1.Istituto di Ricerche sulla Popolazione e le Politiche Sociali, Consiglio Nazionale delle RicercheRomeItaly

Personalised recommendations