Adaptive Agents in Changing Environments, the Role of Modularity

Abstract

We explored the role of modularity as a means to improve evolvability in populations of adaptive agents. We performed two sets of artificial life experiments. In the first, the adaptive agents were neural networks controlling the behavior of simulated garbage collecting robots, where modularity referred to the networks architectural organization and evolvability to the capacity of the population to adapt to environmental changes measured by the agents performance. In the second, the agents were programs that control the changes in network’s synaptic weights (learning algorithms), the modules were emerged clusters of symbols with a well defined function and evolvability was measured through the level of symbol diversity across programs. We found that the presence of modularity (either imposed by construction or as an emergent property in a favorable environment) is strongly correlated to the presence of very fit agents adapting effectively to environmental changes. In the case of learning algorithms we also observed that character diversity and modularity are also strongly correlated quantities.

Appendix: A Algorithm construction by Genetic Programming (GP)

In this section we describe briefly our implementation of GP for the problem at hand. Conventional genetic algorithm (GA) works by manipulating fixed-length character strings that represent candidate solutions of a given problem. There are problems though, for which fixed-length character-structures are not flexible enough for representing their candidate solutions. In those cases a program representing the solution is a better alternative.

The simulation starts with a population of randomly created programs. All these programs have been constructed using pre-determined sets of variables and operators. The construction process respects some rules in order to avoid the creation of programs that can not be evaluated. The GP operations are used to create the population of the next generation. The programs are ranked by their fitness and then the GP operations are applied again. These two steps are then iterated.

Probably the most didactic way to illustrate the functioning of GP is by using the computing language LISP. We call a Faithful symbolic expression (FSE) a LISP program that do not return an error message when evaluated. Program components can be either functional operators or variables. Let $\mathcal{F}$ be the set of all the operators and $\mathcal{V}$ the set of all the variables used to write down the FSEs. The choice of these sets depends upon the nature of the problem being solved. For instance, if the solution of a problem can be represented by quotients of polynomials, a suitable choice of operators and variables is $\mathcal{F}$ = {+ - * /} and $\mathcal{V}$ = {x 1}. For example, a FSE is (+ (+ x x) (* x (- x (- x x)))), which is a (non unique) LISP representation of the function $2x+x^{2}$ . The simplest FSE is a variable. The next simplest FSE is an operator followed by the appropriate number of variables (two in the example above). All FSEs are either a variable or a list with an operator followed by an appropriate number of FSEs. Unfaithful symbolic expressions are for instance: (x x), (+ x *) and (x - x).

The GP operations considered in the present work are asexual reproduction, mutation and cross-over. During asexual reproduction a certain fraction of the top ranked individuals, the Top set, are copied without any modification into the new generation, ensuring the preservation of structures that made them successful. Mutation is implemented on an individual by changing an atom at a random position. The new and old atoms are different but of the same kind to ensure faithfulness. Finally the modified tree is copied into the new generation. In order to accelerate the dynamics different mutation rates can be used for different atom types.

There are no sexes associated to the programs and cross-over is a hermaphrodite sexual GP operation. Cross-over parents are chosen by tournament, which is done as follows. First, consider a set of the population, then select an age $a$ such that $1\le a\le P$ ($P$ is the maximum age) with a probability proportional to $a$. From the chosen set, a certain number (e.g. ten) individuals are selected at random. The program with smaller generalization error at age $a$ is selected for cross-over. In our experiments, the first parent is chosen from the Top set by tournament [16]. The second parent is chosen by tournament among the entire population. From each parent, an atom of the same type is selected at random. The FSEs with roots in the selected atoms are interchanged to generate two offspring. In order to avoid uncontrolled growth if the depth of any of the offspring is above a given threshold, the program is deleted. The mutation and cross-over operations are represented in Figs. 11 and 12.

Fig. 11

LISP programs as parsing trees before and after a GP mutation. A randomly selected atom in the parse tree is changed to another randomly selected atom of the same type. In this example a multiplication operation is replaced by an addition

Fig. 12

GP cross-over. Two parents are selected from the population. A random point in each tree is selected. The branches that grow from the point are interchanged in order to generate two offspring

The GP parameters used in the simulation are shown in Table 1. A short discussion about the parameter values will be presented in the conclusions. At generation zero a population of 500 FSEs is created at random. The programs have (in agreement with Table 1) a maximum depth of $7$ nested parentheses. The variable set is as presented in (5) and the operator set is

$$\begin{aligned} \mathcal{F}=\left\{ \mathsf{Psqr\; Pexp\; Plog\; abs\;+\;-\;*\;\%\; p.\; pN.\; ev\!*\; vv\!+\; vv\!-}\right\} \,, \end{aligned}$$

(11)

where Psqr, Pexp, Plog, and % are the protected square root, exponential, logarithm and division; abs, +, -, and * are the usual absolute value, addition, subtraction and multiplication; and p., pN., ev $\mathsf *$, vv $\mathsf +$, and vv $\mathsf -$ are the inner product (e.g. $\mathbf{x}\cdot \mathbf{y}$ for $\mathbf {x},\mathbf {y}\in \mathbb {R}^{N}$) , normalized inner product ($\mathbf{x}\cdot \mathbf{y}/N$ ), the product of a scalar times a vector ($a\mathbf{x}$ ), the addition of two vectors and the subtraction of two vectors ($\mathbf{x}\pm \mathbf{y}$ ) respectively. Protected functions are functions whose definition domains have been extended in order to accept a larger set of arguments. The definitions of these functions appear in Table 2.

Table 1 Control parameters for the GP simulation in our experiments.

Table 2 Definition of the protected function as FSEs

If either one of the offspring has a depth bigger than 17, it is deleted. With a mutation rate of 0.01 % (one every 20 generations) a mutation is performed to the offspring. The process is repeated until the new population reaches the full size fixed here at 500.

After a new population is created, the fitness of each individual is measured and so a new ranking is built.