Advertisement

Journal of Computational Social Science

, Volume 1, Issue 1, pp 115–146 | Cite as

Survey of evolutionary computation methods in social agent-based modeling studies

  • Peter RevayEmail author
  • Claudio Cioffi-Revilla
Survey Article

Abstract

Agent-based modeling is a well-established discipline today with a rich and vibrant research community. The field of evolutionary computation (EC) is also well recognized within the larger family of computational sciences. In the past decades many agent-based modeling studies of social systems have used EC methods to tackle various research questions. Despite the relative frequency of such efforts, no systematic review of the use of evolutionary computation in agent-based modeling has been put forth. Here, we review a number of prominent agent-based models of social systems that employ evolutionary algorithms as a method. We comment on some theoretical considerations, the state of current practice, and suggest some best practices for future work.

Keywords

Agent-based modeling Evolutionary computation Evolutionary algorithms Survey 

Introduction

The purpose of this paper is to give an overview of the current state of evolutionary computation techniques in the field of agent-based modeling of social systems, and to confront the practices with the theory of evolutionary computation. The history of agent-based modeling is over four decades long today, and the topic has blossomed from experimental method to a full-fledged discipline with its own conferences, journals, and institutions. The literature on ABM studies is both numerous and broad in scope. Agent-based simulation is used to explain phenomena in sociology (see [10] for a review of relevant studies), as well as in archaeology, geography, demographics, human biology, political science and economics (see [14, 17, 51] for reviews). The field of agent-based modeling has established itself by publishing works in leading scientific journals (e.g. [12, 22, 53, 61]), by organizing annual conferences such as the Social Simulation Conference, and by sustaining quality field-specific journals such as JASSS. The very nature of agent-based simulations as powerful explanatory tools capable of modeling change and emergent phenomena made it a suitable method for tackling evolutionary questions. In fact at least a few dozen of “evolutionary ABM” studies have been published in the past two decades. Most of these studies are at least loosely inspired by the evolutionary computation (EC) approach and employ its tools to a certain degree. Evolutionary computations itself is an established discipline dating back to the 1960s, situated within the broader field of Artificial Intelligence in the computer science community ([31], ch. 10). The fundamental principle of the field lies in the harnessing of mechanisms of biological evolution as a powerful problem-solving tool. EC practitioners implement computational algorithms that mimic the processes of biological evolution to tackle various optimization problems as well as to study evolutionary dynamics. However, despite the significant histories of both fields, and the continuing intertwining and borrowing of concepts from both camps, there has been very little explicit interaction and collaboration between agent-based modelers and EC researchers. This work aims to survey the rich ecosystem of evolutionary ABM studies and to compare their approaches to established theory and practice in the domain of EC.

To carry out such a review, we must first define what we mean, when we say “evolution”. The term evolution has at least two distinct, commonly used meanings (along with a number of other, rarely used meanings). The first, colloquial meaning, refers to any kind of accumulation of change, or gradual directional change. The second, formal term, refers to biological evolution, which is defined as the change in the heritable characteristics of biological populations over successive generations [36]. Because the first meaning is overly broad, and the second meaning too restrictive for the purposes of the study of social systems, we introduce a third definition, which we will use throughout this chapter. This definition is neither as relaxed as the colloquial use, nor as narrow as the biological definition. We define evolution as the change in characteristics of populations of individual units over time, as a result of selective pressure brought upon by changes in the environment. Note, that unlike in biological evolution the individuals need not be biological organisms and furthermore the traits need not necessarily be heritable (at least not in the strict genetic sense). On the other hand, this definition is unlike its colloquial counterpart in the sense that any population-level change must be a result of the direct relationship between individuals’ fitness levels and the suitability of their traits to current environmental conditions. This allows us to study other crucial categories, such as cultural evolution. Thus, when we write about “Evolutionary ABMs” we mean agent-based models that simulate processes included under this definition of evolution.

In the sections that follow we begin with an overview of the history and development of the field of EC. We continue by outlining a simple prototype of an EA. Then, we proceed by giving brief overviews of numerous examples of Evolutionary ABMs. We divide these into several categories, such as evolution of cooperation, ecological models, etc. Next, we address each integral design component of an EA in its own subsection and contrast relevant theoretical considerations with actual implementations found in the surveyed studies. We conclude by assessing the current state of evolutionary ABM literature.

Overview of evolutionary algorithms: history, purpose and applications

An EA is a stochastic, population-based metaheuristic that relies on random variation, selection, and other mechanisms with analogies in biological evolution [8], p. xxv]. The central idea of evolutionary algorithms is that of harnessing the power of biological evolution and translating it into an efficient computer-based optimization algorithm. EAs have found their place in ABM literature as well. For example Axelrod [4] has used one in his study on social norms. In the model each agent possessed a pair of alleles tied to their propensity to cooperate with others. The agents’ fitness was modulated in different ways based on the actions they took, whether selfish or altruistic. Only the fittest agents were then selected to seed the next generation of agents. In this way, Axelrod was able to track which alleles related to cooperation were able to survive and eventually dominate the population.

Evolutionary algorithms (EAs) are the subject of the field of Evolutionary computation (EC), which traces its history back to the 1960s. The notion that evolutionary processes are able to find local (and potentially also global) optima of solutions to problems defined by environmental constraints was familiar since the times of Charles Darwin. The proposition that such processes could be represented in the form of computational operations and that their power could thus be harnessed by computers to solve various optimization problems was initially posed in the first half of the 20th century [20], p. 24]. It was not until the availability of inexpensive digital computers reached a sufficient threshold in the 1960s that these ideas began to be exploited in practice. Holland [41] envisioned the role of evolutionary processes as crucial tools for the design of robust adaptive systems capable of navigating through complex dynamic environments. Rechenberg [64] developed his idea of how evolutionary processes could be utilized to solve difficult parameter optimization problems into the family of tools now known as evolution strategies. Fogel et al. [29] saw evolutionary computational techniques as a means of developing artificial intelligence.

From these different visions emerged the three techniques which enjoy the most interest in the EC community today: Evolutionary Strategies (ES), used mainly for real-valued parameter optimization, Genetic algorithms (GA), focusing on the genotype-phenotype mapping and the idea of genetic building-blocks, and finally Genetic Programming (GP), which is used for evolving formal symbolic statements, such as chunks of computer code, decision-making routines or mathematical equations, represented as nested tree-like structures. EC and the formal study of EAs is well established today. There are several recognized archival journals (such as Evolutionary Computation or IEEE Transactions on Evolutionary Computation), as well as a number of peer-reviewed conferences (such as the Genetic and Evolutionary Computation Conference or the Congress of Evolutionary Computation), and numerous high-quality monographs (e.g. [7, 20, 27, 43, 49, 56]).

Although EAs are used for a wide range of purposes, De Jong [20], p. 27] notes that perhaps the most natural application is as an optimizer, which is why significant effort was put into applying these techniques towards various types of optimization problems. From a systems analysis perspective we may identify three main components of a working system: its inputs, outputs, and its internal model which transforms the former into the latter. Armed with this interpretation we may define an optimization problem as one where the model of the system is known along with a series of outputs and it is left to identify the inputs which best reproduce the outputs [25], p. 9]. Among some of the concrete applications of optimization problems one can count route planning (such as in the Traveling Salesman Problem) and scheduling, control (of robotic movement, etc.), design (e.g. aircraft parts) and classification (e.g. game-playing systems or image processing).

However, De Jong [20], p. 28] also cites other uses of EAs, such as models of cognition, the design of neural networks, but perhaps most importantly the potential of such techniques to provide significant insights into complex adaptive systems (CAS). He claims that even simple computational models can provide better understanding in contexts in which humans are known to make notoriously poor predictions (p. 231). In fact there is a desire among part of the EC community to capture the essence of evolution in a computer simulation and use the simulations to gain new insight into the physics of natural evolutionary processes [8], p. 2]. From a systems point of view, this type of problem falls under the modeling and system identification heading. Eiben and Smith [25] define system identification problems as those where both the inputs and outputs are known and a suitable model that translates between the two is to be found. A related type of problem is that of simulation; in this case the inputs and the model is known, and the goal is to identify the associated outputs. These kinds of problems are usually encountered in models of biological evolution, economic models and in models of social and cultural systems. In these cases evolutionary computing allows the researcher to design and perform experiments that fundamentally differ from the methods of classical biology, economics, or social science and offers possibilities that go far beyond the capabilities of studies based on such traditional approaches. There are, however, several caveats to this approach. One of these is the trade-off between biological (social, economic, etc.) fidelity and computational effectiveness. These pose conflicting design objectives in evolutionary algorithms (De Jong, 2005). Tightly related to this is the issue of how conclusions drawn in the computer simulations can be transferred to the biological (social, economic, etc.) medium [25], p. 8]. Finally, Eiben and Smith [25], p. 8] also note the lack of mutual awareness between computer scientists and other disciplines interested in modeling evolutionary processes as the reason why such experimental studies are still scarce.
Fig. 1

Evolutionary algorithm cycle

A simple evolutionary algorithm

Although the particulars of different implementations of EAs vary, each consists of at least a subset of canonical components. These are:
  • A population of individuals and their representation

  • Fitness evaluation

  • Parent selection mechanism

  • Survivor selection mechanism

  • Recombination operator

  • Mutation operator

A detailed discussion of each of these components along with an overview of its use by evolutionary agent-based modelers in practice will be given in section 5. For now, we will present a brief outline. Each EA is first initialized with a population of individuals (also “solutions” or “search points”). These are usually created randomly, although other strategies can be used as well. It should be noted that “individuals” in EAs need not be “individuals” in the biological or social sense. Most often these individuals are simply sets of parameters which pertain to the problem at hand (for example a set of solutions to a system of equations). In the most abstract sense, an “individual” is simply a point in a multi-dimensional space, to which we are able to assign a score (or a measure of fitness) post hoc. The set of all possible combinations of parameters is usually referred to as the search space. There are many options regarding the representation of the parameters in the computer. The most straight-forward way is the so-called phenotypic approach, where the representation space is identical with the search space. An example of this is when the parameters are all real-valued and they are also represented as real values in the computer program. This need not necessarily be the case, as is illustrated by the genotypic approach in which the search space is different from the representation space and a mapping between the two is necessary. The most common example of this is when integer- or real-valued parameters are represented as bit-strings. Although this intuitively seems as an unnecessary practice, it has certain advantages (as well as shortcomings). Moreover, the genetic approach has its analogy in biological evolution.

Each of these individuals is then evaluated for its fitness. Here EAs can be differentiated into those with an objective fitness function, and those where the function is only implicit. In the first case the performance of each individual is solely the result of its position in the search space and can be objectively measured independent of the performance of other individuals. The latter case is usually found in co-evolutionary systems, that is, systems where performance of an individual can depend on the state of the remaining individuals in the population.

Once the individuals are evaluated a selection mechanism is used to identify a subset of the fittest individuals (usually called the “parents”) who will reproduce and create additional offspring. This is done via the recombination operator, which dictates how the genetic material of two or more parents is combined in a deterministic or stochastic manner to create a new individual. The mutation operator is then responsible for making small, stochastic changes to the offspring’s genetic material.

Once the offspring have been created the survival mechanism determines which individuals proceed to the next time step. The simplest format is the so-called generational or age-based EA in which all the individuals from the previous generation are removed from the population and only the offspring are allowed to proceed. The complementary approach to this are steady-state or non-overlapping systems in which only a part of the previous generation is removed and replaced by offspring at any given step. The survival mechanism in this case can be a function of either age or fitness. In most EAs the selection, reproduction and survival phases are implemented in a way that keep the population size constant, although formats with variable population sizes are possible, and certainly more faithful analogues of biological and social systems.

This completes a single cycle of the EA. This completes a single cycle of the EA. A simple illustration of the EA scheme is given in Fig. 1. The individual stages are then repeated many times until some stopping criterion is met. Although there is a number of nuances and possible additions to this scheme this gives an introductory sketch of a basic EA design.

Evolutionary algorithms in agent-based models

Agent-based simulation is a powerful methodological tool because it is (1) inherently dynamic, and thus capable of providing a lucid account of change; (2) able to represent heterogeneity, (3) path-dependent and (4) focused on non-teleological, bottom-up explanations [26]. These qualities make it uniquely positioned to answer questions regarding evolutionary processes. Because of this, the history of evolutionary ABMs is almost as long as that of agent-based modeling itself. Only a decade after Schelling’s [71] ground-breaking study on patterns of segregation, Axelrod and Hamilton [6] analyzed the dynamics of iterative Prisoners’ Dilemma tournaments from an evolutionary perspective. Although their work did not explicitly employ EAs to study what the authors called the “evolutionary robustness and viability” of different strategies, it was perhaps the sign of an increasing interest of tackling the problem of evolution in artificial agent societies.

While evolutionary ABM studies have been published sporadically throughout the 1980s and the early 1990s, the frequency of such research efforts has increased greatly since the late 1990s and throughout the 2000s. The scholars who contribute to the emerging niche come from a variety of institutions across North America, Europe, and Asia, as well as from a wide range of disciplines. ABMs utilizing evolutionary algorithms have been designed to answer questions in sociology, economics, anthropology, archaeology, ecology, etc., and many of them touch on aspects of psychology, cognitive sciences and neuroscience as well.

In the following parts of this chapter we present what constitutes a representative cross-section with respect to discipline, research topic and methodology. The surveyed studies are summarized in Table 1. We begin by outlining the research categories under which evolutionary ABMs most commonly fall. We identified five common themes in such models, which may overlap to a certain degree:
  1. 1.

    Evolution of cooperation

     
  2. 2.

    Ecological models

     
  3. 3.

    Evolution of cultural practice

     
  4. 4.

    Evolution of bounded rationality

     
  5. 5.

    Evolution of sign systems

     
  6. 6.

    Adaptation

     
Table 1

Evolutionary ABM study designs

#

Study

Domain of evolution

Co-evolution

Parameter space domain

Genotype-phenotype mapping

Phenotype

Phenotype change

1

Axelrod

Populations

Yes

Integer (cardinal)

Binary code

Boldness,vengefulness

Via genotype change

2

Bowles and Gintis

Populations

Yes

Categorical

None

Public goods game strategy

Vertical transmission

3

Chiang

Populations

Yes

Real-valued

None

Thresholds of cooperation

Vertical transmission

4

de Vos et al.

Populations

Yes

Categorical

None

Indirect reciprocity strategy

Vertical transmission

5

Dosi et al.

Mind

Both scenarios

Real-valued,operators

None

Estimate of utility function

Vertical transmission

6

Edmonds

Mind

Both scenarios

Real-valued,operators

None

Estimate of utility function

Vertical transmission

7

Hales

Populations

Yes

Boolean

Binary code\(^1\)

PD strategy, tag

Via genotype change

8

Hammond & Axelrod

Populations

Yes

Categorical

None

Cooperation/defection (PD), tag

Vertical transmission

9

Janssen

Populations

Yes

Categorical,real-valued

None

Cooperation/defection (PD), tag,probability of feedback, trust

Learning (neural networks)

10

Kachel et al.

Populations

Yes

Real-valued

Arithmetic\(^2\)

Longevity, length of reproductive period

Via genotype change

11

Kohler et al.

Populations

Yes

Categorical

None

Public goods game strategy, hierarchical/non-hierarchical

Vertical transmission

12

Klos

Populations

Yes

Boolean Binary

code

Iterated PD strategy

Via genotype change

13

Lake & Crema

Populations

Yes

Integer (ordinal)

None

Effciency of resource extraction

Vertical and horizontal transmisison, innovation

14

Lake

Populations

Yes

Categorical

None

Cultural/individual learning

Vertical transmission

15

Macy & Skvoretz

Populations

Yes

Categorical

None

PD strategy, tags

Vertical transmission

16

Pepper and Smuts

Populations

Yes

Categorical

None

Presence/absence of alarm calling/food restraint

Vertical transmission

17

Perez-Losada & Fort

Populations

Yes

Boolean

None

Types of crops cultivated

Vertical transmission

18

Powell et al.

Populations

Yes

Real-valued

None

Degree of skill in abstract cultural domain

Vertical and horizontal transmission

19

Premo & Hublin

Populations

Yes

Integer (cardinal)

Arithmetic\(^3\)

Abstract genetic/cultural traits

Vertical and horizontal transmission

20

Saam & Harrer

Populations

Yes

Real-valued

None

Resource acquisition strategy

Vertical transmission

21

Santos & Pacheco

Populations

Yes

Categorical

None

Cooperation/defection (PD, stag hunt, snowdrift)

Vertical transmission

22

Takahashi

Populations

Yes

Real-valued

None

Indirect reciprocity strategy

Vertical transmission

23

Xue et al.

Populations

No

Real-valued

None

Degree of adaptation to environ-ment

Vertical transmission, innovation

#

Fitness

Survival

Selection

Recombination

Mutation

1

Payoffs from public goods game

Generational

Truncation

Cloning

Bit-flipping, 1%

2

Payoffs from public goods game

Steady-state; stochastic death, fitness-based

Unclear\(^4\)

Cloning

Random resetting, 1%

3

Payoffs from PD

Generational

Tournament\(^{5}\)

Cloning

Not specified

4

Dead/alive

Steady-state; stochastic death, averted by help from others

Fitness-neutral

Cloning

None

5

Distance from max. utility

Generational

Unclear\(^6\)

Sub-tree crossover

None

6

Distance from max. utility\(^7\)

Generational

Unclear\(^8\)

Unclear\(^9\)

Unclear\(^{10}\)

7

Payoffs from PD

Generational

Fitness-proportionate

Cloning

Bit-flipping, 0.1%

8

Payoffs from PD

Steady-state; fixed death rate

Fitness-proportionate

Cloning

Random resetting, 0.5%

9

Payoffs from PD

Steady-state; agents die if reputa-tion/payoffs below threshold

None\(^{11}\)

None

None

10

Dead/alive

Steady-state; stochastic death, based on longevity trait

Age-dependent

One-point crossover

Truncated Gaussian perturbation, 5%

11

Energy from resources

Age-specific mortality

Age-specific fertility

Not specified

Not specified

12

Payoffs from PD\(^{12}\)

Generational

Tournament\(^{13}\)

Two-point crossover

Bit-flipping, 0.5%

13

Energy from resources

Generational

Truncation

Cloning

None

14

Energy from resources

Generational

Tournament

Cloning

None

15

Payoffs from PD

Generational

Fitness-neutral, local

Fitness-biased uniform crossover

Bit-flipping, 1%

16

Energy from resources, death by predation

Generational

Fitness-neutral, local

Cloning

None

17

Age-based

Steady-state; fixed death rate

Fitness-neutral

Cloning

None

18

Degree of skill in abstract cul-tural domain

Generational

Fitness-neutral

Cloning

Gumbel distributed perturbation, 100\(^{14}\)%

19

Fully determined by genotype

Steady-state; stochastic death, based on current population size

Fitness-proportionate

One-point crossover

Incremental, 0.001%

20

Energy from resources

Generational

Fitness-neutral

Additive

None

21

Payoffs from games

Generational

Fitness-proportionate

Cloning

None

22

Energy from resources

Generational

Truncation

Cloning

Perturbation, distribution not specified, probability varied

23

Distance from optimal fitness

Steady-state; stochastic death, fitness-based

Fitness-neutral\(^{15}\)

Cloning

Truncated uniform perturbation, 100%\(^{16}\)

\({^1}\)Only the tag part of the genotype is represented in binary.

\({^2}\)Mean of two real-valued alleles.

\({^3}\)Each individual allele in a given locus has a fixed fitness contribution to the phenotype.

\({^4}\)“By the fitness of an agent, we mean the expected number offspring produced by the agent in one period minus the probability the agent dies in that period”. (Bowles and Gintis 2004, p.18)

\({^5}\)Agents imitate fittest neighbor.

\({^6}\)“Selection consists of preserving the fittest rules, and discarding the less fit ones”. (Dosi et al. 1999, p. 15)

\({^7}\)A penalty for model complexity (tree size) is also assessed.

\({^8}\)“[The agent] then selects the best models in terms of fitness for carrying over in the next period”. (Edmonds 1997, p.6)

\({^9}\)“[The agent] produces some new ones by either combining the previous models with a new operator or by growing a new random one”. (Edmonds 1997, p.6)

\(^{10}\)See above.

\(^{11}\)There is no reproduction or inheritance in Janssen’s model. Unfit agents are simply removed and replaced by new random ones.

\(^{12}\)Relative to neighbors’ payoffs.

\(^{13}\)Candidates sampled from Moore neighborhood.

\(^{14}\)Represents imperfect imitation.

\(^{15}\)Random surviving agents is chosen to create a new clone in place of a recently deceased agent

\(^{16}\)Represents imperfect imitation.

It should be further noted that the concept of individual as understood in EC, need not overlap with the concept of individual, or agent as understood in agent-based modeling. While this is often the case, there are also ABMs where the agents and the evolved individuals are two different entities. For example, some ABMs use EAs to evolve sets of decision-making strategies from which the agents choose and apply the fittest ones [23, 24]. Thus, when I speak of “individuals” in the following sections, I am referring to the EC concept, while “agents” refer to the ABM concept.

Finally, there are many ABMs that either include the term “evolution” or “evolutionary” in their descriptions or seem to be evolutionary at first glance, but do not exactly fit the proposed definition. Examples of this include Axelrod’s [5] study of cultural evolution or the Kohler et al’s [48] original Village simulation. The former was not included because there is no selective pressure or concept of fitness, while the latter, despite modeling death and reproduction, does not allow for any change in the inherent characteristics of its agents.

Evolution of cooperation

At first glance the existence of cooperative or altruistic behaviors, which lead the agent to temporarily defer its own well-being for the well-being of others, poses a logical paradox. Why would any agent act contrary to its self-interest if acting out of their own free will? These questions have led philosophers to reason that agents must be compelled to follow the laws of “covenants” [40] or social contracts [66] by the fear of punishment from some greater force. The already tentative position of cooperation was certainly not improved by Darwin’s theory of evolution, expressed in competitive terms of survival of the fittest individuals. After World War II, game theory emerged as a viable approach to formalizing the multi-objective nature of social behavior. Especially the formulation of Prisoner’s Dilemma and its subsequent analysis helped elucidate the elementary dynamics of self-interest and altruism (see [3, 60] for explanations of the dilemma and some theoretical considerations). For this reason it became a popular tool of researchers hoping to demonstrate conditions under which cooperation can emerge.

Axelrod [4] used a modified n-person version of the Prisoner’s Dilemma to explore what conditions lead to the development of norms in societies. In this model agents take turns during which they have an opportunity to cooperate or defect. Defections give a boost to the agent, but they hurt the other players. There is also a possibility of each defection being seen by other agents. If agents see a defection they can either choose to let it go, or they can punish the defector. Such a punishment costs the defector as well as the agent who executes the punishment. Axelrod then allowed for heterogeneity in strategies by giving each agent two parameters which he termed “boldness” and “vengefulness”. While boldness controlled the agent’s willingness to defect given a certain probability of being seen, vengefulness determined its probability of punishing a defection of other agents. In simulations of this model Axelrod failed to observe any sort of cooperative norm. This prompted him to introduces so-called “metanorms” in which agents could punish not only defectors but also those who see a defection and do not punish it. Under these new conditions Axelrod was able to observe the emergence of a cooperative norm. It should be noted however, that later efforts to replicate the model failed to reproduce Axelrod’s results [30]. This was mostly ascribed to the small number of runs, the small population sizes and the arbitrary choice of parameters.

Macy and Skvoretz [53] created an evolutionary ABM to study how cooperation emerges in a society where repeated interaction is uncertain. Once again, they achieve this by having the agents play the Prisoner’s Dilemma. The agents are modeled as either unconditional cooperators or unconditional defectors. Furthermore, they can signal certain external markers and possess rules on how to evaluate others (whether based on markers, fitness, projection of ego etc.). These rules can be based on the possession of certain markers or agents’ fitness. They position these agents into neighborhoods. Agents have a higher probability of interacting with neighbors rather than strangers. The agents also have the option to exit the game (i.e. refuse to interact, or distrust the partner). Under these assumptions, the authors allow the agents to evolve and track the model’s sensitivity to the cost of exit, the neighborhood size, and the propensity to favor neighbors over strangers. They show that cooperation can emerge (first locally in a single vanguard neighborhood and then globally by spreading to other neighborhoods) when exiting is fairly cheap and neighborhoods are small, and there is a large share of within-neighborhood interaction. The authors note that this is perhaps how Protestantism could have succeeded in the U.S.: churches were small and tightly knit and Protestantism thus became a marker of trust spreading to other communities.

Klos [47] extended an earlier evolutionary ABM by Miller [55]. In the original model agents play one of 16 different iterated Prisoner’s Dilemma strategies which replicate, recombine and mutate over the course of generations. In the extended version the agents are placed on a toroidal grid and they compete against their neighbors. This changes the selection mechanism, which becomes local: agents only imitate strategies of their neighbor. Similarly as in [55], a cooperative regime emerges. However, Klos introduces a second tweak by which fitness becomes subjective: the agents only know the standing of those neighbors who they have played in the current generation. With this modification implemented, cooperation ceases to emerge in the model.

Another adjustment of the classic PD design was explored by Takahashi [73], who explores conditions under which generalized exchange (i.e. indirect reciprocity) is globally sustainable. Thus, instead of mutual cooperation, agents offer help unilaterally in hopes that a third party will return the favor at some point in the future. In the model agents employ a particular strategy according to which they give another randomly chosen agent resources, while holding the receiving agent to a certain standard of giving to others. If the standard is not met, the giver ceases to donate to that agent, and chooses another random agent who meets the standard. The agents are heterogeneous in that they have different standards (thresholds) of giving and different shares of resources that they give out. Subsequently, the assumption of perfect information possessed by the agents is relaxed by placing them on a grid, where the agents only know how charitable their Moore neighbors are. Even under this assumption generalized exchange emerges. It should be noted that the central assumption in the form of the general strategy employed requires a certain pre-developed sense of fairness on the part of the agents.

The question of the evolution of direct delayed reciprocity was tackled by de Vos et al. [75]. In their model agents can become distressed with an exogenous probability (representing “harshness of conditions”) each round. If agents become distressed they will seek help from others; if they do not receive it in time they die. However, if an agent helps another one, this results in an increased risk (controlled by a cost parameter) of the helper becoming distraught in the next round. Agents possess one of three strategies. Defectors ask different agents for help every time, while never reciprocating it. Committal agents only ask for help those they have helped in the past; they always answer to requests for help, as long as those requesting it have helped the agent at some point in the past or have not received help from the agent before. The third strategy is a book-balancing one in which agents ask for help those who have received help from them and not reciprocated yet, and provide help only to those who are not indebted to them. The agents remember all interactions and their outcomes, but do not know anything about other interactions (i.e. they do not exchange information). The authors then simulate the evolution of mixed strategy populations under different conditions of harshness, population size, and cost of helping. They conclude that commitment strategies are able to invade small populations in harsh conditions as soon as two committal mutants appear. Overall the commitment strategies fared better than strict book-balancing agents, even when costs of helping are high.

Hales [35] explored another dimension of the evolution of cooperation. Unsatisfied with kin selection, group selection or reciprocity as explanations for emergence of cooperation, he set out to explore how biased interaction affects the onset of wide-spread cooperation in societies. In the model agents, who are either unconditional defectors or cooperators, possess one from a wide range of tags. Agents then play one-shot PD games in pairs. Agents pick an agent at random and if they have the same tag they play the game. In the other case they keep picking agents until they find one with the same tag. Hales shows that over many generations sustained cooperation emerges when the range of possible tags is high enough, i.e. the possibilities far outnumber the agent population.

Hammond and Axelrod [37] used this tag-based approach to test the hypothesis that social dilemmas such as the PD game promote in-group favoritism. In their model agents possess three traits: an observable tag, an in-group PD strategy, and an out-group PD strategy (either unconditional defection or cooperation). They then play one-shot PD games with Von Neumann neighbors on a toroidal grid and reproduce locally. The authors show that this leads to the evolution of ethnocentric strategies, i.e. in-group favoritism where agents cooperate within their group but defect outside of their groups. External markers also appear in Janssen’s [44] work on cooperation in reputation systems. The motivation of the research was to explain how reputation systems with voluntary feedback (such as eBay user ratings) effectively work. In the ABM agents play PD games repeatedly with an option to exit. Agents also possess reputation, which is modified based on feedback provided stochastically by their partners in the PD. Agents leave the simulation if their reputation or payoffs become too low and are replaced by new agents possessing random strategies. In an extension of the model, agents also signal a number of different symbols, which can eventually become recognized by others through a learning process as signs standing for specific strategies. This allows the agents to be strategic about playing and exiting the PD. The results show that in the absence of signaling symbols cooperation emerges only above a certain threshold of propensity for giving negative feedback. With signaling this threshold disappears and cooperation emerges in all cases.

Several articles have also focused on the co-evolution of cooperation and network structure. Santos and Pacheco [68] give agents in their model the option to update their game-playing strategies as well as their neighborhood. The sensitivity of the model is tested with respect to the ratio between the frequencies of strategy and network updates and the overall connectivity of the network. The authors show that cooperation emerges as the network update becomes more frequent relative to strategy update. Moreover, the threshold ratio necessary for cooperation to evolve increases with the network’s connectivity, i.e. highly connected networks require more agile adaptability.

Chiang [16] also focuses on the interplay of cooperation and network structure. The question is how is cooperation able to spread, if it is assumed to be conditional on certain nodal attributes (such as degree or betweenness centrality). Agents’ strategies are defined as ranges of nodal attributes, and they only cooperate with agents whose attribute values fall within that range. A wide variety of populations with mixed strategies are simulated and allowed to evolve. Chiang observes that under such assumptions the most robust strategies are those that cooperate with either distinctly lower valued or higher valued nodes.

Evolutionary ABMs inspired by anthropological motivations have also explored the issue of cooperation. Bowles and Gintis [11] simulate a hunter-gatherer society divided into distinct groups with migration between them. Members of the groups have an option to contribute to resource acquisition efforts at a certain cost. The acquired resources are then distributed evenly regardless of contribution. However, free-riders can be punished as in Axelrod’s norm model. Assuming agents possess imperfect information about the system, the authors show that cooperative mutants are able to survive and replicate even in populations initially composed entirely of opportunistic free-riders.

Other anthropological ABMs also touch on the evolution of cooperation, such as Pepper and Smuts’ [58] research on alarm signals and resource sharing and their dependence on kin selection, or Kohler et al’s (2012) [48] model of hierarchical structure formation in public goods games.

Ecological models

The models listed in the previous subsection focused mostly on the evolution of agent behaviors and their dependence on the nature of social interactions between the agents. However, societies do not evolve in a vacuum separated from the natural world. The surrounding environment constrains the actions of individual agents and affects their results as well. Thus the relationship between the population and its milieu shapes the co-evolution of the ecosystem as a whole. A number of articles on evolution in ABMs reflect this reality and employ an ecological approach to modeling.

One of the first such models was introduced and analyzed in a paper by Conte and Castelfranchi [13]. They were interested in the evolutionary dynamics of normative strategies for resource gathering. Agents move around a toroidal grid where food is distributed randomly. This food can be marked as their property and eventually eaten to increase their strength. Agents possess one of several strategies which dictate conditions under which they attack others for food. The authors then analyzed the resulting strength inequality within the population as a function of the initial make-up of the populations in terms of strategies. Saam and Harrer [67] subsequently added an evolutionary mechanism to the model whereby the agents were able reproduce by pooling their strength between offspring.

Pepper and Smuts [58] explored how ecological variation contributes to the emergence of within-group cooperation using a similar simulation environment in which agents expend energy to locate food. The authors modeled food restraint and alarm calling as two qualitatively different dimensions of cooperation. Alarm calling was useful in notifying nearby agents of predators, thus reducing their chance of being killed while simultaneously increasing the caller’s chances of being preyed upon. Both forms of cooperation emerged from mixed populations in patchy environments, which forced agents to concentrate in small cohesive groups, as relocation proved costly. However the two cooperative behaviors differed in their dependence on kin selection. Unlike food restraint, alarm calling was only able to spread when newly-born offspring were placed near their parents.

Another use of the same artificial ecosystem was motivated by the question of the onset of cultural learning in early hominids and its dependence on the variance of resource availability [50]. In Lake’s model agents possess different propensities towards individual and cultural learning. When agents learn individually they recall (possibly out-dated) information regarding resource availability only at cells that they have personally visited. When agents learn culturally they are able to communicate their experiences with others. However, there is an inherent trade-off as better knowledge by the group leads to more over-exploitation of the slowly regenerating resources. Lake’s simulations show an evolutionary advantage of cultural learning in environments where resource patchiness is low.

Lake and Crema [52] revisited this topic and studied the link between adaptation and resource exploitation in more depth. The authors simulated a population of agents mining resources from a common pool. The agents are equipped with different cultural traits, which determine how much of the resource they can extract. Crucially, the extraction payoffs show some degree of variance independent of the traits. Agents reproduce selectively based on their payoffs, inherit traits from parents and innovate these traits by imitating other successful agents. The simulations show that higher rates of innovation are only beneficial when payoff variance is low, otherwise the resource pool is over-exploited and adaptation dwindles. Moreover, the authors were able to identify a “sweet spot” such that population-wide adaptation is faster when the innovation rate is neither too low, nor too high.

Finally Kohler et al. [48] studied how a public goods game in a landscape with variable resources contributes to the evolution of social hierarchies. They specifically focus on modeling pre-hispanic Pueblo societies in the US Southwest. In the simulation agents choose between contributing to the public good and defecting, while receiving a fair share either way. Once again free-riding can be punished at a cost. Kohler et al. then introduce two types of agents–hierarchical and non-hierarchical–who occupy the same landscape. Hierarchical agents willingly elect leaders who tax the followers and who are entirely responsible for punishing free-riders while bearing the costs. The results show that over time there is a large increase in the ratio of agents living in hierarchical communities. These communities evolve to be significantly larger than those of the non-hierarchical type, which is in line with the archaeological record.

Migratory phenomena are another example of ecological factors in evolutionary dynamics. Premo and Hublin [62] built and ABM to test the hypothesis that culturally mediated migration was crucial to the loss of genetic diversity in hominid populations. agents in the model are stochastically forced to migrate, however they will only move to cells that are culturally similar enough. Each agent then possesses a series of genes and cultural traits which can be either selective or selectively neutral. The simulations show that the extent of genetic diversity loss is sensitive to increasing the cultural similarity threshold. The authors also found that larger innovation rates and proportions of culturally neutral traits result in a lower similarity threshold necessary to trigger loss of diversity.

The effect of population density and its structure in space was tackled by Powell et al. [61] when they agentized Henrich’s [39] mathematical model of cumulative cultural evolution during the Upper Pleistocene transition. Agents are defined by their skill level in an abstract cultural domain and placed in sub-populations, which are linked by stochastic migration events. The skill value is modified through unbiased vertical transmission from parents to offspring and by biased horizontal transmission which acts by copying the best individual in the agent’s current sub-population. In both cases transmission is imperfect and “noisy”. As in Henrich’s original model, the authors conclude that for any given level of transmission noise there exists a critical population size above which cultural innovations start spreading, and that smaller communities can only innovate through migration from and into denser sub-populations.

Evolution of cultural practice

Cultural practices have a strong grasp on the behaviors of individual humans, even entire societies, yet the origins of many of them remain largely unknown. Consider for example religion and ritual, which have been the subject of fierce sociological and anthropological debates for over a century now. On one end, under the Durkheimian interpretation, religion is considered a crucial adaptation of societies which fosters a feeling of solidarity among its members by resolving cognitive dissonance, satisfying emotional needs, or infusing action with meaning [46]. Other explanations view rituals as an unintended consequence of adaptations serving entirely different functions, such as regulating the impact of environmental conditions [63]. Gould and Lewontin [33] compared the latter phenomenon to the work of mosaic artists who fill in the empty spaces in the spandrels below the arches of a cathedral’s dome, itself a design aspect necessary from the perspective of structural stability. In fact, Gould and Lewontin dismissed post hoc explanations of specific traits as meaningful evolutionary adaptation for being “just-so stories”; offering enticing, but ultimately untestable narratives. However, the explanatory and revealing nature of agent-based simulations proves, yet again, especially fitting for the purpose of testing hypotheses regarding the emergence of cultural phenomena, be they adaptive or serendipitous in nature.

Kachel et al. [45] have used an evolutionary ABM for this reason, when testing the grandmother hypothesis. The hypothesis states that the lengthening of the post-menopausal period and thus the increasing longevity is the result of the inclusive fitness benefits of grandmothering. Each agent is defined by its longevity, fertility and reproductive age length, while the environment is defined by the exogenous parameters of maximum weaning age of offspring and the age of maturation. The critical assumption is that mothers who are weaning cannot have offspring. Two scenarios were tested: one in which grandmothers take over child-rearing duties, which allows mothers to become fertile again earlier, and another in which grandmothering increases the chances of the offspring’s survival to maturity. Results showed that grandmothering behavior does not lead to an increase in longevity in either of the scenarios. However, in scenarios where grandmothering behavior had a direct effect on decreasing the weaning age, selection did favor shorter reproductive periods in women.

Perez-Losada and Fort [59] set out to explain the loss of cultural diversity in farming during the neolithic in Central Europe through simulation methods. They model a landscape which is occupied by farming households who must share finite resources and take part in stochastic migration, as determined by the archaeological record. Each farm is defined by its strategy to cultivate any combination of eight possible crops. The households’ cropping practices are inherited by descendant households. From the simulated scenarios the authors observe that loss of diversity occurs in the most recently occupied cells and propagates “backwards”. The authors conclude that this is because cropping practices on the moving front of the migration have a better chance to catch on as a result of a lesser degree of competition.

Adaptation

Other evolutionary modeling efforts focus on explaining adaptation of cognitive agents to environmental constraints strictly during their lifetimes. This view of evolutionary processes in which individual phenotypes are plastic and amenable to change after birth is known as Baldwinian or Lamarckian evolution. While in the Baldwinian approach changes in the phenotype are not retained in subsequent generations, the Lamarckian position holds that phenotypical adaptations are reflected in the genotype and passed onto offspring. While these paradigms are contentious, and particularly Lamarckian inheritance has been largely disproved as a mechanism of biological evolution, both have their place in general evolutionary theory. In fact, the existence of cultural transmission suggests that if cultural genotypes are constituted of socially constructed representations such as language, rather than genes, Lamarckian inheritance may now dominate genetic inheritance in human societies [57]. Even if the role of Lamarckian processes in nature and society is disputed, nothing precludes researchers from using evolutionary algorithms to represent such mechanisms. On the contrary, computer simulations devised in this way might even speak to the validity of such evolutionary models. For example, even though the Klos [47] study uses a genetic algorithm complete with inheritance of strategies, its intention was never to model evolution over the course of many generations. Rather, the approach was to understand the algorithm as a model for the adaptation of agents through abandoning and acquiring different social strategies over the course of their lifetime.

A good example of an evolutionary simulation model employing Lamarckian adaptation is Xue’s et al. [76] effort to properly identify conditions under which short-term fitness-enhancing innovations are advantageous in the long term. The authors argue that the long-standing intuition that the ability to predict consequences of one’s own actions is beneficial to fitness has never been thoroughly tested. To carry out such a test, they design an agent-based model in which agents possess a single phenotypical trait representing their adaptation to the environment, i.e. a measure of how close they are to the optimal fitness value. Then at each time-step innovations become available and agents must decide whether to adopt them or not. Each agent has a probability of determining correctly whether the innovation will be beneficial or not. The environment is fully determined by the optimal fitness as a function of time. The function possesses a strong central tendency throughout the first phase of the simulation, after which the value precipitates dramatically. Agents that are closer to the optimum have a higher chance of survival, and their offspring inherit the innovations adapted throughout their lifetimes. Several different types of environments were tested, distinguished by the rate at which the fitness value precipitates during reversal events. The results show that the value of innovation significantly decreases in scenarios with faster environmental changes.

Evolution of sign systems

A crucial part of human cultural dynamics is the evolution of meaning: the forming of more or less arbitrary yet salient relationships between mental representations and physical phenomena. Complex sign systems such as languages convey references to physical objects or abstract concepts and their relations to one another through various means, such as the modulation of sound waves in the vocal chords, or the imprinting of various geometrical shapes on suitable materials. However, the assignment of specific signs to their representations is arbitrary. It is only a matter of convention that members of the feline family are in English referred to as cats and canines are referred to as dogs. Had it been the other way around, it would pose no impediment to the function of language whatsoever. The formal study of meaning in sign systems dates back to early 20th century, with the emergence of semiotics and the efforts of Saussure [70] to deconstruct meaning into its constituent parts. Since then significant effort was put into the analytical study of sign systems, while considerations of mechanisms responsible for the emergence of meaning were given less attention. Agent-based modeling certainly offers a way to test hypotheses regarding the evolution of sign systems.

A handful of the models described in previous subsections make use of external markers (or tags) which can be observed by other agents. Through repeated interactions and their subsequent evaluation or classification, agents come to associate these markers with certain patterns of behavior (should any be present). Thus, the markers can stand in as signs for a behavior that cannot be a priori derived by any other means. In Hales’ [35] model of conditional cooperation agents possessed a high degree of homophily, interacting only with those agents who shared the same tag. The simulations showed that the proliferation of cooperation within tag groups was only possible when the discriminatory ability of the tags was minute enough, that is, when the tag alphabet was large enough to represent a wide range of small communities. Similarly Hammond and Axelrod [37] showed that when agents are able to employ different strategies when encountering agents who possess different markers, the evolutionary dynamics of kin selection lead to the association of cooperation with sameness and defection with difference. Finally, Janssen [44] showed that agent populations equipped with simple neural networks coupled with fitness-based selection were able to learn existing correlations between external markers and cooperative behaviors.

Evolution of bounded rationality

A special niche of evolutionary modeling is concerned with the evolution of cognitive models of reality in the minds of individual actors. While classical economists assumed that agents always act rationally to maximize their utility, modern approaches attempt to relax these often unrealistic assumptions. Simon [72] pioneered the concept of procedural, or bounded rationality which posits that agents form estimates of unknown utility functions, and that these estimates are periodically updated to reflect feedback to most recent actions and (imperfect) observations of a changing environment. Tree-based genotype representations and genetic programming algorithms are particularly well-suited for simulating the selection of such boundedly-rational decision-making processes.

Edmonds [24] uses this approach to simulate cognitive frameworks of agents. In his case a mental model is composed of a subset of possible elementary operations, variables, constants, and the relationships between them which lead to different outcomes based on the inputs. Agents continuously evaluate the performance of their current models, create new ones by recombining older instances, and select the best ones to guide their next decision. Edmonds tests his agents in two contexts. The first is the management of a simple investment portfolio composed of two goods under an unknown utility function. Agents must estimate the function using their own models. By utilizing different utility functions, Edmonds is able to explore the limits of such a boundedly-rational cognitive structure. As a subsequent test Edmonds applies the same architecture to the El Farol crowding problem [2]. In this version the bar patrons continuously evolve a population of strategies for signaling their intentions and deciding on their true intention of whether or not to go to the bar (they only want to go, if it is not too crowded). The agents’ models effectively co-evolve because outputs of one model can be used as inputs for another. This interaction ultimately leads to the emergence of heterogeneous strategies which the original El Farol study lacked.

Dosi et al. [23] also argue that decision-making must follow some form of procedural rationality by pointing to the existence of a class of problems that cannot be algorithmically solved in optimal fashion. Just as Edmonds [24], they use genetic programming to test the viability of their hypothesis. Once again, agents possess a set of strategies defined by recursive tree-like structures which help them learn an unknown supply-demand curve. Agents have access to variables such as prices, quantities and market shares from a pre-determined number of previous steps. The authors show that in monopoly cases a single agent is able to learn the optimal strategy fairly quickly. In cases with several agents interesting behaviors such as tit-for-tat appear. However, the authors note that in cases of more complicated strategies their semantic interpretation becomes difficult. This points to the question of the realism of such a model and the degree of isomorphism to human cognitive architectures.

Design of evolutionary algorithms and practice in agent-based modeling studies

Representation

Usually the first important choice a modeler must make when designing an evolutionary algorithm is the choice of how to represent the individuals, or the candidate solutions, that make up the population of the EA. Because most problems tackled by EAs are simply parameter estimation problems, this question is reduced to the representation of the parameter values themselves. Depending on the nature of the parameters it is perhaps natural to represent them directly as what they are: Boolean variables, integers or real values. This is what De Jong [20] calls the phenotypic approach. In this case, individuals correspond directly to points in the solution space and the parameter values–the phenotypes-are the basic heritable units.
Fig. 2

An example of a binary phenotype representation (bottom) encoding for a genotype consisting of a list of real-valued parameters (top)

A different approach is to separate the representation space and the solution space and relate the two with some type of mapping. This approach, which De Jong [20] dubs the genotypic approach, consists of encoding the parameter values using a universal code. In the case of computers it is natural to choose binary code (as shown in Fig. 2). This type of representation is essentially analogous to the concept of genetic inheritance, in that individual traits (points in the solutions space) are made of a combination of multiple genes (points in a multi-dimensional representation space). Evolution then operates on single bits, or individual genes rather than the actual phenotype. There has been considerable debate regarding the utility of binary representation. Holland [42] and Goldberg [32] argue that it is desirable from a theoretical point of view that genetic algorithms act on distinct building-blocks or schemata that represent salient substructures of the larger structures being evolved. De Jong [20] offers a different perspective, claiming that neither approach can be globally declared to be better and that each strategy has its own strengths and weaknesses. Certainly one weakness is the existence of hamming cliffs in binary representations. This refers to the fact that a single small change in the representation space (i.e. a single bit-flip) may produce a disproportionately large change in the solution space, depending on the salience of the bit being flipped. Effective strategies, such as Gray coding, have been proposed and analyzed to deal with this issue [54]. However, even hamming cliffs aside, others have claimed it to be a mistake to encode real-valued parameters in binary [25], p. 40] and that maximizing the number of schemata being processed is not necessarily useful, or may even be harmful [28], p. 137].

Only a few of the reviewed ABM studies have employed the genotypic approach. Klos [47], based on Miller [55], represented agents’ PD strategies as finite state machines, which were in turn represented in binary. Here each bit marked the presence or absence of each component of the finite state machine. Other authors have utilized the genotypic approach without relying on binary representations. For example, Premo and Hublin [62] represented agent’s traits as a series of nominal, integer-valued loci. The nature of the trait was defined by the proportions of the different values in the loci. Kachel et al. [45], also striving for a greater degree of biological fidelity, represented their agents’ traits as a result of averaging two real-valued alleles. Each allele was inherited from a different parent. Finally, Hales [35] used a mixed approach in which part of the phenotype, namely the external marker, was represented with a binary genotype. This multi-dimensional tag space, and the way in which agents traverse it, dramatically changes the dynamics of the model when compared to a one-dimensional tag representation (cf. [65].

However, most studies used the phenotypic approach. The domain of the parameter space has varied depending on the nature of the problem. Macy and Skvoretz [53], as well as Perez-Losada and Fort [59] have used binary phenotypes to represent the presence or absence of certain behaviors. More commonly, ordinal scales have been used (e.g. [52]), as well as real-valued ones (e.g. [16, 61]).

Selection and survival

Darwinian evolution is driven by the natural selection of genotypes in the form of the survival of the fittest. Evolutionary algorithms acknowledge this by implementing selection and survival mechanisms that determine which individuals are allowed to reproduce and create new phenotypic combinations (as illustrated in Fig. 3). This is done by first assessing the individuals’ fitness. Fitness can be objective, as a measure of how well an individual is able to solve a problem within environmental constraints. It can also be implicit, that is, dependent on the behavior of the system as a whole. The ways in which fitness evaluations are conceptualized in ABM studies varies. In some studies this is done by representing payoffs from social dilemma’s through Prisoner’s Dilemma games (e.g. [16, 37, 53]) or public goods games (e.g. [3, 11]). In others agents gain fitness by collecting geographically dsitrbuted resources (e.g. [48, 52, 58]), and some combine multiple elements into a multi-objective fitness evaluation (e.g. [44]). Once every individual is associated with a fitness value, the population can be ranked accordingly. There are several ways how to select the subset of the fittest. The simplest one is truncation selection, in which the n highest ranked individuals are selected. In rank-proportional selection each individual is assigned a probability of selection according to its rank; the probability ratio between subsequent ranks can be tuned with a parameter. In tournament selection a subset of individuals of size n is first selected randomly and the most fit individual from it is selected; this process is then repeated many times. Finally, in fitness-proportionate selection every individual’s probability of selection is directly proportional to its fitness value.
Fig. 3

A general schema of survivor and parent selection

The consideration of the selection mechanism is important because it greatly affects the selective pressure exerted on the population throughout the simulation. This is in turn related to the phenomenon of genetic drift. Drift is the process through which, under neutral selection, trait variants that are by chance initially over-represented by even the slightest amount compared to others, will eventually take over the entire population. Any form of fitness-biased selection will lessen the effects of genetic drift, however it cannot eradicate it completely. Selective pressure then refers to the usual takeover time in the population. When the takeover time is low, selective pressure is high and vice versa.

There are various reasons why significant drift effects should be avoided. In optimization applications this is known as premature convergence, as it leads to suboptimal solutions. However, in simulations of evolutionary dynamics preventing drift phenomena is important as well, especially if the target system is known to show negligible effects of drift. Other simulation efforts call for the exploration of the degree to which drift affects the system, in which case it can be tuned via parameters of the selection mechanism.

Truncation selection shows the highest rates of selective pressure, leading to extremely rapid convergence, although it can be somewhat controlled by the selection of the truncation point; rank-proportional and tournament selection have a slower convergence overall, but there too the rate can be controlled by choice of their respective parameters (see [20], sec. 6.3.1.2]). Fitness-proportionate selection is certainly not without issues either. Once again, in practical applications, it comes with high rates of premature convergence. However, notably in larger populations, selective pressure tapers off as the share of the fittest variants increases (because these variants will have more or less equal fitness). Too low of a selection pressure can also pose a problem, especially in co-evolutionary systems (such as those usually modeled by ABMs), where relative fitness is more important that absolute fitness [15]. Eiben and Smith [25], p. 62] point out that tournament selection does in fact select in terms of relative rather than absolute fitness, and is therefore invariant towards any translations of fitness functions, whether objective or implicit.

The second concept complementary to selection is the survival mechanism. The most straight-forward and most commonly used is the generational GA. Under such a configuration, all of the parent individuals who are selected to reproduce are removed from the simulation and fully replaced by their offspring, while usually keeping population size constant. On the other hand there are so-called steady-state systems in which only a small subset of individuals are removed and replaced by new agents in every step. Death in these cases can be either age-based or fitness-based. Arguably, generational methods are prone to higher drift rates, which must be offset by larger population sizes [69]. Hancock [38] also notes that steady-state systems are more adept at dealing with noisy systems.

Not all evolutionary models include both selection and survival methods. In fact, almost a half of the articles surveyed here use no proper selection mechanism, and rely solely on survival mechanisms coupled with fitness-neutral selection to control the parent pool. This not necessarily a detriment to the quality of the studies, as it should be kept in mind that the goal of such ABMs is to model the dynamics of evolution and achieving a good fit with underlying empirical data, rather than search for sufficiently good solutions, or optimization. Naturally, these models are steady-state systems. Others utilize fitness-biased selection mechanisms. For example Klos [47] as well as Lake [50] use tournament selection. However a surprising number use methods that can be problematic with respect to drift effects. Axelrod [4], Takahashi [73] and Lake and Crema [52] all use some version of truncation selection. Examples of the use of fitness-proportionate selection include the studies of Hales [35], Santos and Pacheco [68] and Hammond and Axelrod [37].

It is surprising that there is little discussion or explanation regarding choice of selection mechanisms in a number of the surveyed studies, considering the confounding and often dramatic role that drift can play in evolutionary simulations. The lack of fitness-biased selection in steady-state systems can be somehow offset by the non-overlapping nature of the population’s generations. However in non-overlapping configurations which use fitness-proportionate or truncation methods, undesirable selective pressure should ideally be offset either by highly disruptive recombination operators or large population size [21]. Unfortunately, this is often not the case.

Another interesting aspect of selection is the effect of modeling physical space. Optimization applications recognize this in a number of techniques such as island models, speciation, fitness-sharing, or crowding (see e.g. [25], sec. 9.3-9.4] for descriptions). Several of the surveyed ABM studies take geographical effects into consideration as well. For example Pepper and Smuts [58] test the hypothesis that cooperation can emerge as a result of kin selection (the selection of behaviors which help genetically similar individuals at one’s own expense) and the local clustering of trait groups. In other cases, such implicit effects were brought to light post hoc, as was the case of Bausch [9] who showed that the model of Hammond and Axelrod [37] is sensitive to the practice of placing offspring only in the neighborhoods of their parents. The insight was that the cooperation behavior did not emerge because of the tags as Hammond and Axelrod argued, but rather simply because agents interacted with their close kin most of the time.

Recombination

Recombination operators control how genes or traits get transmitted from parents to their offspring and how they are modified in the process. The role of recombination is the exploration of the parameter space by creating new solutions from old ones. It can also help maintain diversity. Not all evolutionary algorithms utilize recombination, in which case offspring are created by (asexually) cloning their parents. However De Jong [20] notes that cloning is yet another example of a method susceptible to drift. In sexual reproduction models several different recombination techniques are commonly used. The simplest method is one-point crossover in which the parameter vectors of the two parents are split at a randomly chosen position and the complementary parts from opposite parents are then “re-glued” together to create two offspring (as shown in Fig. 4). This can be generalized to n-point crossover in which case there are n cut points. Uniform crossover is another operator which steps through the parameters one by one, while a random trial determines which of the two parent contributes its value in the current position (see Fig. 5 for an illustration).. It is important to note that there are trade-offs between the different types of crossover. The n-point version of crossover possesses positional bias, meaning that alleles that are close together have a higher chance of staying together. This has important implications for genotypic representations [25], p. 49]. On the other hand, uniform crossover has distributional bias, meaning that most of the time the ratio of genes inherited from the two parents will be close to even (unless the random trials are explicitly biased). Other possible recombination operators include arithmetic crossover (e.g. averaging of the parents’ values of real-valued parameters) or multi-parent recombination, which is easily accomplished in the computer and especially useful when simulating cultural dynamics. Genetic programming with its tree-based representations uses a specific set of crossover techniques. The most common one is sub-tree crossover, which acts by switching sub-trees of the two parents at randomly selected nodes [25], p. 148]. Often, GP algorithms would impose some type of limit on the tree size, since this method can lead to undesirable bloating, expending precious computational power [1], p. 287]. Ultimately, what matters most for recombination according to De Jong [20], p. 185] is that one should choose an internal representation that “in some sense reflects or preserves the meaningful application-dependent sub-components”.
Fig. 4

One-point crossover recombination

Fig. 5

Uniform crossover recombination

Once again, most of the surveyed ABM studies choose the path of least resistance and model inheritance via cloning. In some cases this is motivated by the goal of modeling imitation of behaviors rather than true sexual reproduction (e.g. [4, 52, 61, 76]). In many others the phenotype consists of a single value (e.g. a choice between cooperation and defection) and thus cloning is the only possible approach (e.g. [11, 50, 68, 75]). In some studies no justification for this choice is provided (e.g. [16, 35, 37, 73]).

A few of the studies utilize classic crossover techniques. Kachel et al.[45] use one point crossover, while Klos [47] implements two point crossover in his model, and Macy and Skvoretz [53]use a version of uniform crossover. Yet others use more exotic diversity maintenance techniques. Janssen [44] does not implement inheritance in his model at all. Instead removed agents are replaced by randomly created agents. This is a specific choice related to the nature of reputation systems which he sets out to investigate in this model. In Saam and Harrer’s [67] study the focus is on the flow of resources between agents. It is therefore natural that inheritance would act in the form of pooling of the parents’ resources. The GP models of Edmonds [24] and Dosi et al. [23] both use traditional sub-tree crossover.

Mutation

While recombination serves the purpose of exploration of the solution space, the role of mutation is exploitation, or local search within small regions of the solution space. Whereas recombination operators can have more or less arbitrary arity, mutation operators are usually unary, that is, they act on a single individual at a time, by introducing small stochastic changes to parts of their genotype. The goal is to define mutation in a way that a small change of an internal representation of an object results into a small change in the corresponding external object [20], p. 87]. This is mostly straight-forward in phenotypic representations, but crucial to understand when dealing with genotypic representations. For real-valued parameters the traditional approach is to introduce small Gaussian perturbations. Integer-valued representations make us of random resetting, which is especially well-suited for nominal variables, or gradual incrementing/decrementing, which is appropriate for ordinal and cardinal variables [25], p. 43-44]. Once again, GP algorithms make use of a set of idiosyncratic mutation strategies such as random leaf growth, random sub-tree shrinkage, or the switching of two random subtrees within a single tree [1], p. 249-250]. The chance of a mutation occurring in any given part of an individual’s genotype is usually fairly low, traditionally set at 1% or less. In fact, De Jong [20], p. 173] notes that in GA-like designs mutation rates higher than 1% result in the system becoming very noisy, and confound other underlying evolutionary dynamics.

Past evolutionary ABM studies have employed a variety of mutation techniques. Macy and Skvoretz [53], Hales [35] and Axelrod [4] flip bits in the agent’s genotype (without addressing hamming cliff phenomena). Models with nominal phenotypes, such as PD strategies, usually use random restting (e.g. [11, 37]). Real-valued problems appropriately utilize perturbation techniques, however, the range of distributions used is broad: from truncated Gaussian [45] and uniform perturbations [76] to Gumbel distributions [61]. Many models, however, do not implement mutation operators at all (e.g. [23, 50, 52, 58, 67, 68, 75]). A specific application of mutation techniques in certain ABMs is the representation of imperfect imitation. In this case mutation occurs every time imitation is attempted (see [61, 76]).

Lamarckian methods

As noted above, Lamarckian evolution is arguably an important driver of human cultural dynamics. From a practical point of view nothing prevents a modeler to implement Lamarckian inheritance into an evolutionary computation. For this reason some of the evolutionary ABMs in fact do this to achieve their objectives. Lamarckian effects can be modeled in two flavors: either as horizontal transmission between two specific agents (i.e. imitation or learning) which is propagated into subsequent generations or as an abstract innovation force (essentially a type of in-life mutation which is inherited by offspring). In the case of horizontal transmission, the mechanisms can differ in the way in which individuals select the models that they imitate. This choice can be done in a fitness-biased fashion (by selecting from some subset of fittest individuals) or in a selectively neutral fashion (e.g. based on the possession of certain tags).

Xue [76] makes use of an innovation operator, while others model horizontal transmission (e.g. [50, 61, 62]), and in some cases both techniques are combined [52]. Finally, in some instances the presence of Lamarckian inheritance is implicit, mainly in cases where evolutionary processes are interpreted as imitation and the intended target system has no genetic basis (e.g. [4, 47]).

It is important to note, that the specific choice of a Lamarckian operator also bears consequences with respect to the behavior of the model. Notably, Vaesen [74] has shown that the conclusion’s of Powell’s et al. [61] study do not hold when we relax the assumption that agents will always be able to identify the fittest agent to imitate.

Discussion

The above survey of ABM studies that use evolutionary computation techniques shows that there has been great interest in modeling evolutionary dynamics in a rigorous fashion. These efforts have been present for over three decades now and the frequency of such studies has only increased in recent years. The methods of evolutionary computation have appeared in ABMs of various target systems. Evolution of biological as well as cultural systems has been modeled. Cultural evolution itself has been studied from the perspective of various practices such as reciprocal cooperation norms, alarm-calling behaviors, grandmothering, or crop diversity. Although these concepts are multi-faceted, they are all studied through the prism of memetic transmission, which stems from sociobiological paradigms emerging in the second half of the twentieth century as a loose analogy to genetic transmission (e.g. [12, 19]. Although contentious, it appears that the popularity of such approaches is not fading in the foreseeable future, and thus an influx of new studies utilizing evolutionary algorithms coupled with agent-based simulation is inevitable.

To assure the highest possible quality of such future efforts the adoption of a few guiding principles is encouraged. As a start, both agent-based modelers as well as EC theorists would benefit from extended interaction between their respective communities. It has already been pointed out that the EC community has regretfully little to none interaction with those reasearchers who focus on the study of evolutionary dynamics [25]. Other leading figures of evolutionary computation have explicitly called for the development of a unifying EC framework which would be open and easily accessible to outsiders, in the hopes of fostering further collaboration [20]. As far as we know, there has also been little outreach from the ABM community, apart from rare exceptions [18]. Continued collaboration between EC experts and ABM researchers will only benefit the design of agentized simulations of evolution.

A second, closely related principle is a deeper focus on the theoretical considerations of the choice of specific EC mechanisms. In previous sections we have elaborated on how the implementation of different mechanisms has to be carefully adjudicated to avoid undesirable side-effects. It is therefore surprising to see how so many ABM studies forego the justification of some of the necessary mechanisms completely. In certain cases it has been proven that the lack of such reflection leads to very idiosyncratic results. For example Axelrod’s [4] norms game has been replicated by Galan and Izquierdo [30] using different evolutionary operators with striking results. The outcomes of the simulations were found to be highly sensitive to choice of different selection mechanism, fitness regimes and population sizes. Other studies also defy what is considered good practice in the EC community. The greatest danger seems to be the potentially confounding effect of genetic drift. A number of models use drift-prone selection mechanisms such as roulette-wheel selection, without off-setting its effects with disruptive recombination or mutation methods. Instead, modelers often combine this with other operators which are themselves susceptible to drift phenomena, such as cloning or the complete absence of mutation. In other cases, the exploitative nature of mutation is distorted with the use of bit-flipping in binary genotype representation without addressing hamming cliff issues. Finally, there are a number of models which simulate insufficiently small populations, numbering in the dozens, or in one extreme case only a handful of agents.

Finally, just as with other important aspects of ABM design, it is necessary to provide all necessary details of the precise implementation of evolutionary mechanisms within the model. For example the ODD protocol [34], has long been recognized as the standard for describing ABM design. In the light of this, it is surprising that some of the surveyed studies have offered only vague descriptions of the evolutionary algorithms powering their simulations. As reviewed above, these details are important when considering the sensitivity of a model, or even the possibility of its replication.

Even despite some of these shortcomings, the usefulness of such models in offering fresh explanations for complex phenomena and their ability to provide insight into the working of dynamic adaptive systems is invaluable. The design of new agent-based studies following the evolutionary computation approach should be welcomed with enthusiasm, while simultaneously promoting the quest for even sounder designs supported by good practice and theoretical rigor through collaboration with the EC community.

Notes

Acknowledgements

This study was funded in part by the Center for Social Complexity at GMU, by the US National Science Foundation, CDI Program, Grant no. IIS-1125171, and by ONR-Minerva Grant no. N00014130054.

References

  1. 1.
    Angeline, P. J. (2000). Parse trees. In T. Bäck, D. Fogel, & T. Michalewicz (Eds.), Evolutionary computation 1: basic algorithms and operators. Bristol: Institute of Physics Publishing.Google Scholar
  2. 2.
    Arthur, B. W. (1994). Inductive reasoning and bounded rationality. American Economic Review, 84, 406–411.Google Scholar
  3. 3.
    Axelrod, R. (1984). The evolution of cooperation. New York: Basic Books.Google Scholar
  4. 4.
    Axelrod, R. (1986). An evolutionary approach to norms. The American Political Science Review, 80(4), 1095–1111.CrossRefGoogle Scholar
  5. 5.
    Axelrod, R. (1997). The dissemination of culture: a model with local convergence and global polarization. Journal of Conflict Resolution, 41(2), 203–226.CrossRefGoogle Scholar
  6. 6.
    Axelrod, R., & Hamilton, W. D. (1981). The evolution of cooperation. Science, 211, 1390–96.CrossRefGoogle Scholar
  7. 7.
    Bäck, T. (1996). Evolutionary algorithms in theory and practice. New York, New York: Oxford University Press.Google Scholar
  8. 8.
    Bäck, T., Fogel, D., & Michalewicz, T. (2000). Evolutionary computation 1: basic algorithms and operators. Bristol: Institute of Physics Publishing.CrossRefGoogle Scholar
  9. 9.
    Bausch, W. (2015). The geography of ethnocentrism. Journal of Conflict Resolution, 59(3), 510–527.CrossRefGoogle Scholar
  10. 10.
    Bianchi, F., & Squazzoni, F. (2015). Agent-based models in sociology. Wiley Interdisciplinary Reviews: Computational Statistics, 7, 284306.CrossRefGoogle Scholar
  11. 11.
    Bowles, S., & Gintis, H. (2004). The evolution of strong reciprocity: cooperation in heterogeneous populations. Theoretical Population Biology, 65, 17–28.CrossRefGoogle Scholar
  12. 12.
    Boyd, R., & Richerson, P. J. (1985). Culture and the evolutionary process. Chicago: University of Chicago Press.Google Scholar
  13. 13.
    Castelfranchi, C., Conte, R., & Paolucci, M. (1998). Normative reputation and the costs of compliance. Journal of Artificial Societies and Social Simulation, 1(3), 3.Google Scholar
  14. 14.
    Cegielski, W. H., & Rogers, J. D. (2016). Rethinking the role of agent-based modeling in archaeology. Journal of Anthropological Archaeology, 41, 283–298.CrossRefGoogle Scholar
  15. 15.
    Chattoe, E. (1998). Just how (un)realistic are evolutionary algorithms as representations of social processes? Journal of Artificial Societies and Social Simulation, 1(3), 2.Google Scholar
  16. 16.
    Chiang, Y. S. (2013). Cooperation could evolve in complex networks when activated conditionally on network characteristics. Journal of Artificial Societies and Social Simulation, 16(2), 6.CrossRefGoogle Scholar
  17. 17.
    Cioffi-Revilla, C. (2017). Introduction to computational social science. New York: Springer.CrossRefGoogle Scholar
  18. 18.
    Cioffi-Revilla, C., Luke, S., Parker, D., Rogers, J. D., Fitzhugh, W. W., Honeychurch, W., et al. (2006). Agent-based modeling simulation of social adaptation and long-term change in inner Asia. In S. Takahashi, D. Sallach, & J. Rouchier (Eds.), Advancing social simulation: the first world congress (pp. 189–200). New York: Springer.Google Scholar
  19. 19.
    Dawkins, R. (1976). The selfish gene. Oxford: Oxford University Press.Google Scholar
  20. 20.
    De Jong, K. A. (2005). Evolutionary computation: a unified approach. Cambridge: MIT Press.Google Scholar
  21. 21.
    Deb, K. (2000). Introduction to selection. In T. Bäck, D. Fogel, & T. Michalewicz (Eds.), Evolutionary computation 1: basic algorithms and operators. Bristol: Institute of Physics Publishing.Google Scholar
  22. 22.
    Deffuant, G., & Huet, S. F. A. (2005). An individual-based model of innovation diffusion mixing social value and individual benefit. American Journal of Sociology, 110, 10411069.CrossRefGoogle Scholar
  23. 23.
    Dosi, G., Marengo, L., Bassanini, A., & Valente, M. (1999). Norms as emergent properties of adaptive learning: the case of economic routines. Journal of Evolutionary Economics, 9, 5–26.CrossRefGoogle Scholar
  24. 24.
    Edmonds, B.: Modelling bounded rationality using evolutionary techniques. Discussion paper, Centre for Policy Modelling, Metropolitan University of Manchester. http://www.cfpm.org/papers/mbruet.pdf.Google Scholar
  25. 25.
    Eiben, A. E., & Smith, J. E. (2007). Introduction to evolutionary computing. New York: Springer.Google Scholar
  26. 26.
    Epstein, J., & Axtell, R. (1996). Growing artificial societies. Cambridge: MIT Press.Google Scholar
  27. 27.
    Fogel, D. (1995). Evolutionary computation: toward a new philosophy of machine intelligence. Piscataway: IEEE Press.Google Scholar
  28. 28.
    Fogel, D. (2000). Real-valued vectors. In T. Bäck, D. Fogel, & T. Michalewicz (Eds.), Evolutionary computation 1: basic algorithms and operators. Bristol: Institute of Physics Publishing.Google Scholar
  29. 29.
    Fogel, L., Owens, A., & Walsh, M. (1966). Artificial intelligence through simulated evolution. New York: Wiley.Google Scholar
  30. 30.
    Galan, J. M., & Izquierdo, L. R. (2005). Appearances can be deceiving: lessons learned re-implementing Axelrod’s ‘Evolutionary Approach to Norms’. Journal of Artificial Societies and Social Simulation, 8(3), 2.Google Scholar
  31. 31.
    Gilbert, N., & Troitzsch, K. G. (2005). Simulation for the social scientist (2nd ed.). Maidenhead: Open University Press.Google Scholar
  32. 32.
    Goldberg, D. (1989). Genetic algorithms in search, optimization, and machine learning. New York: Addison-Wesley.Google Scholar
  33. 33.
    Gould, S. J., & Lewontin, C. R. (1971). The spandrels of San Marco and the Panglossian pradigm: a critique of the adaptationist programme. Proceedings of the Royal Society of London. Series B, Biological Sciences, 205(1161), 581–598.CrossRefGoogle Scholar
  34. 34.
    Grimm, V., Berger, U., Bastiansen, F., Eliassen, S., Ginot, V., Giske, J., et al. (2006). A standard protocol for describing individual-based and agent-based models. Ecological Modeling, 198(1), 115–126.CrossRefGoogle Scholar
  35. 35.
    Hales, D. (2000). Cooperation without space or memory: tags, groups and the prisoner’s dilemma. In S. Moss & P. Davidsson (Eds.), Multi-agent-based simulation, lecture notes in artificial intelligence. Berlin: Springer.Google Scholar
  36. 36.
    Hall, B. K., & Hallgrimsson, B. (2008). Strickberger’s evolution. New York: Jones and Bartlett Publishers.Google Scholar
  37. 37.
    Hammond, R. A., & Axelrod, R. (2006). The evolution of ethnocentrism. Journal of Conflict Resolution, 50, 926–936.CrossRefGoogle Scholar
  38. 38.
    Hancock, P. J. B. (2000). A comparison of selection mechanisms. In T. Bäck, D. Fogel, & T. Michalewicz (Eds.), Evolutionary computation 1: basic algorithms and operators. Bristol: Institute of Physics Publishing.Google Scholar
  39. 39.
    Henrich, J. (2004). Demography and cultural evolution: how adaptive cultural processes can produce maladaptive lossesthe Tasmanian case. American Antiquity, 69(2), 197–214.CrossRefGoogle Scholar
  40. 40.
    Hobbes, T. (1651). Leviathan. Folkestone: Renaissance Books.Google Scholar
  41. 41.
    Holland, J. (1962). Outline for a logical theory of adaptive systems. JACM, 9, 297–314.Google Scholar
  42. 42.
    Holland, J. (1975). Adaptation in natural and artificial systems. Ann Arbor: University of Michigan Press.Google Scholar
  43. 43.
    Holland, J. (1992). Adaptation in natural and artificial systems (2nd ed.). Cambridge: MIT Press.Google Scholar
  44. 44.
    Janssen, M. (2005). Evolution of cooperation when feedback to reputation scores is voluntary. Journal of Artificial Societies and Social Simulation, 9(1), 17.Google Scholar
  45. 45.
    Kachel, A. F., Premo, L. S., & Hublin, J. J. (2010). Grandmothering and natural selection. Proceedings of the Royal Society B Biological Sciences, vol. 278 (pp. 384–391).Google Scholar
  46. 46.
    Kertzer, D. I. (1989). Ritual, politics, and power. New Haven: Yale University Press.Google Scholar
  47. 47.
    Klos, T. B. (1999). Decentralized interaction and co-adaptation in the repeated prisoners dilemma. Computational and Mathematical Organization Theory, 5(2), 147165.CrossRefGoogle Scholar
  48. 48.
    Kohler, T. A., Cockburn, D., Hooper, P. L., Bocinsky, R. K., & Kobti, Z. (2012). The coevolution of group size and leadership: an agent-based public goods model for prehispanic Pueblo societies. Advances in Complex Systems, 15(1–2), 1–29.Google Scholar
  49. 49.
    Koza, J. (1992). Genetic programming. Cambridge: MIT Press.Google Scholar
  50. 50.
    Lake, M. W. (2001). The use of pedestrian modelling in archaeology, with an example from the study of cultural learning. Environment and Planning B: Planning and Design, 28, 385–403.CrossRefGoogle Scholar
  51. 51.
    Lake, M. W. (2014). Trends in archaeological simulation. Journal of Archaeological Method and Theory, 21(2), 2258287.CrossRefGoogle Scholar
  52. 52.
    Lake, M. W., & Crema, E. R. (2012). The cultural evolution of adaptive-trait diversity when resources are uncertain and finite. Advances in Complex Systems, 15(1–2), 1–19.Google Scholar
  53. 53.
    Macy, M., & Skvoretz, J. (1998). The evolution of trust and cooperation between strangers: a computational model. American Sociological Review, 63, 638–660.CrossRefGoogle Scholar
  54. 54.
    Mattias, K.E., Whitley, L.D.: Transforming the search space with gray coding. In Proceedings of the first IEEE conference on evolutionary computation. IEEE world congress on computational intelligence, vol. 1, pp. 513–518 (1994)Google Scholar
  55. 55.
    Miller, J. H. (1996). The co-evolution of automata in the repeated prisoner’s dilemma. Journal of Economic Behavior and Organization, 29(1), 87–112.CrossRefGoogle Scholar
  56. 56.
    Mitchell, M. (1998). An introduction to genetic algorithms. Cambridge: MIT Press.Google Scholar
  57. 57.
    Moravec, H. (1989). Human culture: a genetic takeover underway. In: C. G. Langton (Ed.), Artificial Life: The Proceedings of an Interdisciplinary Workshop on the Synthesis and Simulation of Living Systems, Los Alamos, New Mexico, September 1987. SFI Studies in the Sciences of Complexity, vol. VI (pp. 167–199). Redwood City, CA: Addison-Wesley.Google Scholar
  58. 58.
    Pepper, J.W., Smuts, B.: The Evolution of Cooperation in an Ecological Context: An Agent-based Model. In: G.G.J. Kohler T. A. (ed.) Dynamics in Human and Primate Societies: Agent-Based Modelling of Social and Spatial Processes, pp. 45–76. Oxford University Press, New York, New York (2000)Google Scholar
  59. 59.
    Perez-Losada, J., & Fort, J. (2011). Spatial dimensions increase the effect of cultural drift. Journal of Archaeological Science, 38(6), 1294–1299.CrossRefGoogle Scholar
  60. 60.
    Poundstone, W. (1992). Prisoner’s dilemma: John von Neumann, game theory, and the puzzle of the bomb. Oxford, United Kingdom: Oxford University Press.Google Scholar
  61. 61.
    Powell, A., Shennan, S. J., & Thomas, M. G. (2009). Late pleistocene demography and the appearance of modern human behavior. Science, 324, 12981301.CrossRefGoogle Scholar
  62. 62.
    Premo, L. S., & Hublin, J. J. (2009). Culture, population structure, and low genetic diversity in pleistocene hominins. Proceedings of the National Academy of Science, 106(1), 33–37.CrossRefGoogle Scholar
  63. 63.
    Rappaport, R. A. (1967). Ritual regulation of environmental relations among a New Guinea people. Ethnology, 6(1), 17–30.CrossRefGoogle Scholar
  64. 64.
    Rechenberg, I. (1965). Cybernetic solution path of an experimental problem. Royal Aircraft Establishment, Library Translation 1122, Farnborough.Google Scholar
  65. 65.
    Riolo, R.: The Effects of Tag-Mediated Selection of Partners in Evolving Populations playing the iterated prisoner’s dilemma. Working Paper 97-02-016, Santa Fe Institute, Santa Fe, NM (1997). http://www.cfpm.org/papers/mbruet.pdfGoogle Scholar
  66. 66.
    Rousseau, J. J. (1762). The Social Contract. Penguin Classics. New York: Penguin Classics.Google Scholar
  67. 67.
    Saam, N. J., & Harrer, A. (1999). Simulating norms, social inequality, and functional change in artificial societies. Journal of Artificial Societies and Social Simulation, 2(1), 2.Google Scholar
  68. 68.
    Santos, F. C., Pacheco, J., & Lenaerts, T. (2006). Cooperation prevails when individuals adjust their social ties. PLoS Computational Biology, 2, e140.CrossRefGoogle Scholar
  69. 69.
    Sarma, J., & De Jong, K. (2000). Generation gap methods. In T. Bäck, D. Fogel, & T. Michalewicz (Eds.), Evolutionary computation 1: basic algorithms and operators. Bristol: Institute of Physics Publishing.Google Scholar
  70. 70.
    Saussure, F. (1986). Course in general linguistics. Chicago: Open Court.Google Scholar
  71. 71.
    Schelling, T. C. (1971). Dynamic models of segregation. Journal of Mathematical Sociology, 1, 143–186.CrossRefGoogle Scholar
  72. 72.
    Simon, H. A. (1996). Sciences of the artificial (3rd ed.). Cambridge: MIT Press.Google Scholar
  73. 73.
    Takahashi, N. (2000). The emergence of generalized exchange. The American Journal of Sociology, 105(4), 1105–1134.CrossRefGoogle Scholar
  74. 74.
    Vaesen, K. (2012). Cumulative cultural evolution and demography. PLoS ONE, 7(7), e40989.CrossRefGoogle Scholar
  75. 75.
    de Vos, H., Smaniotto, R., & Elsas, D. A. (2001). Reciprocal altruism under conditions of partner selection. Rationality and Society, 13(2), 139–183.CrossRefGoogle Scholar
  76. 76.
    Xue, J. Z., Costopoulos, A., & Guichard, F. (2011). Choosing fitness-enhancing innovations can be detrimental under fluctuating environments. PLoS ONE, 6(11), e26770.CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2017

Authors and Affiliations

  1. 1.Center for Social Complexity, Krasnow Institute for Advanced StudyGeorge Mason UniversityFairfaxUSA

Personalised recommendations