Second-order agent-based models of emergent behaviour of Dictyostelium discoideum and their inspiration for swarm robotics

By studying and modelling the behaviour of Dictyostelium discoideum, we aim at deriving mechanisms useful for engineering collective artificial intelligence systems. This paper discusses a selection of agent-based models reproducing second-order behaviour of Dictyostelium discoideum, occurring during the migration phase; their corresponding biological illustrations; and how we used them as an inspiration for transposing this behaviour into swarms of Kilobots. For the models, we focus on: (1) the transition phase from first- to second-order emergent behaviour; (2) slugs’ uniform distribution around a light source; and (3) the relationship between slugs’ speed and length occurring during the migration phase of the life cycle of D. discoideum. Results show the impact of the length of the slug on its speed and the effect of ammonia on the distribution of slugs. Our computational results show similar behaviour to our biological experiments, using Ax2(ka) strain. For swarm robotics experiments, we focus on the transition phase, slugs’ chaining, merging and moving away from each other.


Introduction
We are interested in agent-based modelling of biological organisms. Our goal is to derive decentralised, self-organising mechanisms useful to engineer collective artificial intelligence systems, such as swarm robotics.
Swarm behaviour-such as the one we can observe in ants, bees, fish, as well as during the aggregation phase (e.g. streaming) of D. discoideum lifecycle-displays first-order emergence: the emergent properties arise from the collective behaviour of individual entities. In this paper we are particularly interested by modelling swarm behaviour displaying second-order emergence: individual entities, organise themselves into "super-organisms", which themselves display a swarm-like behaviour. We observe this behaviour during the migration phase of D. discoideum lifecycle. Slugs behave like swarms and, as slugs, exhibit first-order behaviour. Slugs are composed of cells each retaining their individual behaviour. By modelling the individual cells (within the slugs), so that collectively they behave like slugs, we provide second-order emergence.
This work was presented in part at the 3rd International Symposium on Swarm Behavior and Bio-Inspired Robotics (Okinawa, Japan, November 20-22, 2019) In previous works, we provided various agent-based models of both first-and second-order behaviour: (1) streams formation and centres establishment based on six chemical signals [20] and stream-breaking during the aggregation phase [23]; (2) a second model focusing on the slug migration phase of D. discoideum, highlighting slugs locomotion, phototaxis and slugs' merging [21]. This paper discusses a new selection of models, their biological illustration and swarm robotics experiments related to second-order emergence, namely: (1) the transition phase from aggregation to migration (transition from first-to second-order model); (2) for the migration phase (second-order emergent behaviour): (a) slugs' uniform distribution around a light source under the effect of ammonia, (b) the relationship between slugs' speed and length. Similarly, swarm robotics experiments relate to the transition phase, slugs' locomotion, slugs' merging and moving away from each other. The complete sets of models, results, biological validation and swarm robotics experiments can be found in [19].
We first briefly review the life cycle of D. discoideum in Sect. 2, and summarise related works in Sect. 3. We discuss the transition phase from first-to second-order in Sect. 4. For the migration phase, we first discuss our model for slug's speed and length in Sect. 5, followed by results in Sect. 6, and biological illustration in Sect. 7. We then show how to transpose some of this behaviour into swarm robotics in Sect. 8. Finally, Sect. 9 concludes the paper. Figure 1 illustrates the different phases of the D. discoideum life cycle. In the vegetative phase, individual amoeba move around feeding themselves on available food. Once the food is scarce, cells gather in response to cAMP signals, which emanates from specific cells, called centres. During the aggregation phase cells converge towards centres.

D. discoideum life cycle
Cells migrate forming patterns similar to streams (firstorder behaviour). Afterwards, streams aggregate and form a hemispherical mass, called mound. During this transition phase, cells start to differentiate into several different cell types, e.g., prestalk cells which will make the stalk and the basal disk, upper and lower part of the fruiting body; and prespore cells, which will transform into spores. After aggregation and mound formation, the cells make a coherent and cohesive organisation, which is enclosed by a slime sheathsimilar to a slug without organs. During the migration phase, the slug displays phototaxis and thermotaxis behaviour (second-order behaviour). After about 24 h, the cell mass transforms into a new organisation called a fruiting body consisting of a globe of spores on top of a cellular stalk [6]. Eventually dispersing, spores germinate, each releasing a new amoeba. At this point, the cells resume their individual behaviour. This behaviour is particularly appealing to inspire the engineering of artificial collective systems, where a large number of simple, self-organised and homogeneous agents (cells) display emergent behaviour of various levels (firstand second-order emergence).

Migration phase
Some mathematical models study the slug migration phase of D. discoideum [5,31] by underlining the significance of the mechanical interactions between cells. These works examine how wave generation and cell migration together result in slug migration and slug form variations. They present a mixture of the hydrodynamic phenomenon of cell flows and a fixed pattern based on the concentration of cAMP, which is propagated from the tip of the slug.
In most recent works using cellular automata or cellular Potts approach a cell is described by several smaller automatons [11,13]. The authors consider the cell boundaries continuous, and study the cell's size and cell's shape. These models do not consider the physical forces of cells and neighbours' interactions. For instance, Mare et al. [13,15] analyse the slug tactic behaviour in a 2D lattice, in the absence and in the presence of a temperature gradient. They succeed to reproduce the same behaviour using cellular automata for modeling deformable individual cells and partial differential equation for cAMP simulation. The motive force plays an important role in the combination of cAMP Palsson et al. [17,18] introduce a biologically realistic model for slug's movement that considers: cell's shape as deformable viscoelastic ellipsoids, the cells active forces. In this off-lattice model, cells communicate via surface molecules, and they measure and react to different chemotactic signals. The model considers both internal and external forces. A 3D model expresses the association of cell-cell adhesion and cell-signalling with collective cell movements. The movement of the cell depends on the cell's internal state and external parameters, such as the neighbouring cells and chemical signals.
Research in modelling the migration phase of D. discoideum vary from mathematical models, lattice-based models, to off-lattice models involving motive force, adhesion, and possibly a 3D view. We are specifically interested in the understanding of the process of first-to second-order (transition phase) and to investigate off-lattice agent-based models (cells) displaying second-order emergent behaviour.

Swarm robotics
Kilobots [25,26], with a diameter of 3 cm, were developed at Harvard University's Self-organising Systems Research Lab. They serve the need for actual experimentation of collective algorithms in self-organising systems.
The Swarmanoid [8] project developed a swarm robotic system made of many robots of different types. The system uses different coordination mechanisms to provide complicated assignments in three-dimensional environments. Examples of such assignments include finding and retrieving a book on a shelf. Such a system possesses different robotic components with diverse abilities and different actuating modalities. The system is made from eye-bots, equipped with vision and autonomous flight capabilities, foot-bots, moving around on the floor, and hand-bots able to seize or grip objects.
Mathews et al. [16] introduce a new concept of robots with mergeable nervous systems (MNS robots). When robots physically dock, they share a joint architecture for sensing the environment and for the decision-making process, called the robot's nervous systems. In such an order, one robot will act as a brain unit to make the decisions.
Valentini et al. [29,30] propose an iterative strategy to tackle decision problems in a swarm of robots. Each robot examines all available choices, measures the quality of each option. Then, the robot makes an autonomous decision and broadcasts the decision to its neighbours. The proposed strategy is implemented through a swarm of 100 Kilobots, involving the motors, the light sensor, and the infrared system of the Kilobots.
Research in swarm-robotics investigates various organisations, from centralised or hierarchic, to decentralised ones, with homogeneous or heterogeneous robots, aiming at various activities (solving specific tasks or decision problems). We are interested in the full process from identifying firstand second-order emergent behaviour in natural systems and how to translate them into artificial ones, e.g. swarm robotics.

Transition from first-to second-order emergence: slug formation
The transition phase corresponds to the moment, when individual cells, identify through some local sensing, an emergent property, which triggers a behaviour change. In our case, cells identify that the system is moving from the aggregation phase to the migration phase. Figure 2 shows different elements: (a) the transition phase from first-to second-order emergence (green part); and (b) slugs behaviour: (1) slugs merging (blue part) and (2) phototaxis orientation and movement toward the light source (red part). Regarding the transition phase, the PST cells notice when they become part of a slug (identification of emergent property), i.e. when they realise that they are surrounded by the same type of cells (i.e. they are surrounded by other PST cells). They change behaviour and become also sensitive to light. This is   (2) phototaxis orientation and movement toward light (the red part). Both abilities are made possible by measuring the concentration of chemical signals and light using two-dimensional diffusion functions. In our model we used two "Major Voting" processes to determine the collective movement direction (number 1 and number 2 in small circles). We have different migrating velocities for slugs, based on their length a case of reification and identification of properties allowing agents to identify an emergent phenomena. Additionally, we use a diffusion based approach based on a chemical signal (DIF 1 ) that helps PST cells inside the anterior part to measure the population size and decide whether or not to merge with another slug. PST cells also release cAMP to lead the population inside the slug, initially for slug formation and afterwards for phototaxis. This is the emergence of the new phenomenon whereby the entire slug can be led through interactions among individual agents. The model also integrates elements linking speed and size of the slugs.

Second-order collective behaviour: agent-based model of slugs' speed and length
Our previous work covered various slugs' behaviour modelling, such as slugs' merging, slugs' phototaxis both reported in [23]. In this paper we focus on the effect of ammonia on the uniform distribution of slugs around a light source and the relationship between slug's speed and size. We first itemize the slug's key characteristics.

Slug's key characteristics
Phototaxis [4,9,14]: Moving toward the light sourcesphototactic behaviour. In our desired model, individual amoebae are not able to measure the direction from which the light comes, and differences in light intensity do not lead to differentiation in motion velocity. Nevertheless, the whole slug orientates itself toward the light. The tip of the slug, formed by pre-stalk cells, leads the slug toward light. Thermotaxis [4,9]: Slugs move toward the heat as a result of positive thermotaxis. D. discoideum slugs exhibit negative and positive thermotaxis, both of which promote migration of the slug to the soil surface for optimal spore dispersal [9]. Basically, in daylight time, slugs move upwards to the surface, towards higher temperatures. Similarly to phototaxis, thermotaxis is a capability of the slug, but not of the individual amoeba.
Slug's locomotion, interactions, and coordination [9]: In the absence of external thermal or chemical gradients, slugs migrate uniformly in random directions to find an appropriate place for culmination. Additionally, depending on their size, they can merge or split.
Uniform distribution by ammonia as a gas gradient effect [4]: In the wild, ammonia (as a gas gradient) is implicated in different phases of D. discoideum life cycle [4], in particular, new centre prevention in the aggregation phase and increased slug migration in slug formation phase. Slugs orient themselves away from high concentrations of ammonia to bypass the starved cell crowd or other slugs [7]. This negative chemotaxis to ammonia is considerable when two slugs are very close to each other. They migrate away from each other even though they still orient themselves to the light source gradient. Thus, they follow the resultant of two gradients. It can be found using the simple parallelogram method.
Slug's speed and size [3]: Each slug consists of an anterior (tip) and posterior part. The one which is shorter and fatter will move slower than the one which is thinner and longer.

Phototaxis
During the transition phase, Fig. 2 (green part), a PST cell moves toward the centre of the slug if it finds itself surrounded by PSP cells. The aggregation centre (pacemaker) takes on a PST cell behaviour, changes behaviour, becomes sensitive to light (red part of Fig. 2) and leads the slug. The model of the slug follows a leader-follower pattern, where the cells at the tip of the slug act as leaders, the following cells follow the cells in front of them. In our model, a slug consists of five blocks; one red block (itself composed of several cells) act as supreme-leader and four blue blocks act as the followers, and also temporary leaders. The supremeleader cells release cAMP to guide the follower cells (red part). Thus, the cAMP concentration will be displayed as a gradient inside the slug; its lowest concentration at the posterior part and highest at the anterior part.

Merging slugs
The slug size plays a central role in the development process of D. discoideum. What happens when the slug is too small or the anterior part cannot lead the rest of the slug properly? Under these circumstances, small slugs need to merge to produce larger slugs or unite with bigger slugs. To that end, the cells in the anterior part monitor the total number of cells composing the slug, which is possible by the ability of PST cells to perceive and also respond to their micro-environment. To achieve this purpose, the cells in the anterior part use a chemical signal (released into the extracellular space) to estimate the whole population by measuring only PST cell number (since they are 20% of all population [12]). If the concentration is above the threshold, they will activate the merging process (blue part of Fig. 2).
All cells receive information from their neighbouring cells, their environment, and other slugs' cells. We modeled a local signal "merge-flag" which serves specifically for small slugs to signal themselves in the environment. Individual PSP cells follow some simple rules to achieve a complex and coordinated movement. They will follow the internal signal provided by PST cells (phototaxis) unless they receive a "merge-flag" from another slug (which has priority over the PST cells signalling). In that case, they follow that signal and merge with the other slug. To achieve such a collective movement, the cells need to choose one direction over the other all together as a group.
Cells use the majority voting process to have an influential decision-making process of the direction. Our method is single-choice voting (simple plurality) to accomplish the collective directional change. Each voter (a cell inside a block) is allowed to vote for only one direction. Hence, the direction which pulls the most among their counterparts is chosen.

Uniform distribution and the ammonia effect
To reach a uniform distribution of the slugs, we use the ammonia effect, modeled as a repulsive gradient generated by all cells of a given slugs. The ammonia increases the slug migration and helps the slugs move away from each other using a negative chemotaxis [7]. In our model, we assume that every cell inside a slug can produce ammonia, but only PST cells can check its concentration.

Relationship between slug's length and speed
Most of the slug properties are controlled by the tip. Regardless of the slug's age, each cell contributes to the movement, with the tip giving the direction and the signal for movement. To model the slug movement, we used the formulated equation of the slug movement of Innouye's work [10] and the results of Smith's work [27]. There are some assumptions of slug movement in Innouye's work [10]: 1. A slug does not change its speed during the movement. 2. Migrating velocity is essential when the slug moves forward to the light and heat. 3. All cells move in the same direction with the same speed. 4. Each cell moves actively with the constant motive force. 5. Each cell meets with an intrinsic resistance related to its speed 6. The cells in the tip of the slug give the direction and the sheath keeps all cells constrained together. 7. There is no resistance from the sheath on the sides nor on the back of the slug.
Also, in Innouye's work [10], we can find an equation between the slug's length, width, and velocity. They have extracted Eq. (1) by making a multiple regression analysis on the data of 27 different slugs.
where, v denotes the migrating velocity and L, for length and width of the slug respectively. In this equation, the coefficient for the width of the slug is so small so that it can be regarded as zero. Thus, we can conclude the Eq. (2) based on the length of the slug: As shown in Fig. 2 (green part), in the beginning, each cell can be in one of the three cell types: PST (pre-stalk), PSP (pre-spore), or Pacemaker. If it is a PSP cell, it will just follow the tip cAMP signal. However, if it is a PST cell, it will follow the whole flowchart. The PST cells notice when they become a slug (identification of emergent property), i.e., when they realise that they are surrounded by the same type of cells (they are surrounded by other PST cells). Additionally, if they are pacemaker cells, they act as PST cells, with the ability to release cAMP signals spontaneously. In this model, we have two quorum sensing phenomena, which are indicated by red circles (one and two), used for the PST cells to collectively decide on which direction to follow. Additionally, we use a diffusion-based approach based on a chemical signal (DIF) that helps PST cells inside the anterior part to measure the population size, informing in this manner the cell about the slug length, its motive force and the velocity to adopt (Eq. 2). This is used both for slugs' merging and during phototaxis.

Second-order collective behaviour: simulation results
We concentrate here on slugs' merging, the effect of ammonia, and the slug's size and speed. Phototaxis can be observed through all these results. We comment on results related to these aspects, corresponding to an implementation of the model in Python. Slugs merging: Small slugs identify their size and start signalling themselves for merging. In our simulations, each slug has a "merge-flag", which should be the same for all of the PST cells of the slug. If the merge-flag is set to "TRUE", consequently the slug will be available to merge with other slugs which, has also merge-flag = "TRUE". We observe the two slugs merging (see Fig. 3a, b) before moving as a larger slug toward the light (see Fig. 3c, d).
Cells proportion after merging: After consolidation of the two smaller slugs, we observe the creation of a larger slug with the same proportion of PST and PSP cells ( 20% PST and 80% PSP cells [12,28]) and the same behaviours regardless of environment or circumstance (see Figs. 3c, d, and 2 (blue part)). The effect of ammonia: As we saw in previous sections, ammonia acts as a repulsive gradient and helps the slugs move away from each other when PST cells sense that particular gradient. In Fig. 4, the two slugs orient themselves away from each other once they realise they are close. They are capable of this by measuring the concentration of ammonia around themselves.
Slugs' uniform distribution: Since ammonia helps the slugs be away from each other, we see a uniform distribution of the slugs around the light source Fig. 6h.
Slug's size and speed: As we saw in the previous section, the shorter slug migrates slower. Figure 5 shows the case of    The slugs with label orange and green are ready to merge. The slugs with label red and brown are going away from each other (a-d). In this experiment, our ambition is to reach the uniform distribution around the light source. We see that, when the slugs (labeled as red and brown in c) (blue and the merged green and orange in g) are close, they start to move away from each other simulation we ran four different times in the same specific time period with various slugs sizes in each experiment. The objective here is to see what the best size is for the slug to reach the light source in the given time. Each experiment consists of two different independent slugs, very close to each other, and attracted to the light. We see in Experiment 4 that slugs, with a longer size reach the light source in the dedicated time.

Second-order collective behaviour: biological illustration
Dictyostelium discoideum, strain Ax2(ka) [2], was cultured in Petri dishes (Corning, 100×20 mm) at 22 • C in H5Lc medium. The developmental cycle was initiated by exchanging the culture medium with the SorMC (15 mM KH 2 PO 4 , 2 mM Na 2 HPO 4 , 50 M MgCl 2 , 50 M CaCl 2 , nominal pH 6.0) non-nutrient buffer, concomitantly with the plating of two million cells on a thin layer of 3 mm of BactoAgar (2% in SorMC) in an ibidi glass-bottom dish (ibiTreat -dish 35 m high, 35× 12 mm). The cells were imaged using differential interference contrast microscopy with a Leica DMI8 widefield microscope (10× objective, tile scans of 10× 10 field of views, one image every 5 min) for up to 18 h.
In Fig. 7, the behaviour of the slugs moving away from each other at 180 angle is reminiscent from the movement of slugs in our simulation (Fig. 4) as a function of the ammonia gas concentration. We also observed slugs merging (Figs. 8  and 9), visible also in our simulations (Fig. 3).

Transposing models to Kilobots: second-order emergent behaviour
The main objective of this section is to reproduce some of the behaviour of motile Dictyostelium discoideum slugs (secondorder emergent behaviour), to an artificial system made of Kilobots [25,26], namely: transition phase, leader-follower behaviour and chain behaviour, slugs merging, phototaxis and ammonia effect. Figure 10 shows the transition phase, i.e. after the aggregation but before slugs are formed. Figure 10a shows five groups of four Kilobots, representing five aggregation of cells, leading to the corresponding five slugs. Figure 10b shows the Kilobots progressively organising themselves as a chain with a robot in front standing for the tip and the others following it. Progressively, the various slugs are formed and start moving (Fig. 10c,  d). To achieve this behaviour, the Kilobot acting as the tip follows the light, while the following robot reproduces the behaviour of the Kilobot just in front of it. Each one of them acts as a leader for the next one in line, and sends its movement to its follower to reproduce.

Single slug behaviour: chain formation
A robot is able to operate in two states, move and pause. Imagining the form of a chain, a robot in its pause state, it waits until both Kilobots in front and behind have settled and transferred to the pause state. Next, the Kilobot changes to the moving state. The Kilobots, in the moving state, proceed toward the direction given by the acquired strength message from their leaders [1]. Each Kilobot transfers again to the pause state when the distance to its relative leader is less than one centimeter. The leader-follower approach is inspired by Rubenstein et al. work [24] and we provided it as a design pattern for decentralised systems [22]. In real D. discoideum, cells have the capability of pulling and pushing each other, which is not possible with Kilobots. We define the Kilobot gradient, which has the same effect of cAMP molecules among the original cell. We choose a robot as the head of the slug. This robot is responsible for finding a good direction and for leading the whole slug. This robot acts as the root of the gradient. We call the head Kilobot as the "source" or "supreme-leader". The gradient for the source is zero. The nearest robot after the source has the gradient one and the second nearest robot is two, and so on. In each time-step, the robots send the signals and share the value of their gradients with each other as well. Therefore, the robots can replace their positions in the chain. In this way, the Kilobots recognise their leader and follower at each time-step. Figure 11, shows the case of a slug made of one robot as the leader and three robots as the followers.

Multiple slugs: merging and staying away
In this section, we show a more complex behaviour involving two slugs and implement it with Kilobots, in two different scenarios Fig. 12): 1. The two slugs act individually, without merging. 2. The two slugs are attracted to each other and can merge.
Two slugs merge if they are small and close to each other.
The left column is dedicated to the first experiment, and the right column is dedicated to the second experiment. Slugs are labeled by green and red color. Each slug has one leader with ID zero and three followers without any given ID (initially). The leaders always move straight. We do not consider yet robot merging. Figure 12 ( a 1 ), ( b 1 ) show the beginning and the end of the experiment, respectively. In Fig. 12 ( a 1 ), the leaders start to move, and followers follow them one by one in each time step, with a dynamic, new ID. So, at the end of this experiment, Fig. 12 ( b 1 ), we see the green slug making a turn at some point, and the two slugs are far from each other. Figure 12 ( c 1 ) shows the trace of each robot. In the second experiment on the right side of Fig. 12, we consider the slug merging process. Again, Fig. 12 ( a 2 ) shows the first frame of the experiment before the slugs move. Figure 12 ( b 2 ) shows the last frame of the experiment when two slugs approach each other and merge. Two slugs consolidate and make one bigger slug with only one leader. The leader of the green slug became a normal entity with a random ID. A dead Kilobot in the experiment stopped for an unknown reason. Figure 12 ( c 2 ) illustrates the track of the individual Kilobots with different colors. Figure 13 shows the case of three slugs moving towards a light source positioned on the right and slightly at the back of the three slugs. For each slug, the robot at the front moves towards the light (Fig. 13a-c). The three other robots, forming the slug, follow the robot in front of them, according to the same schema as detailed above in Sect. 8.1. We see progressively how the three robots at the head of the slugs turn to the right in the direction of the light (Fig. 13d, e). The robots at the front use an environmental sensor to detect and move towards the light. Figure 14 shows a similar case of phototaxis, this time with an additional behaviour integrated in the robots at the front. Sensing their neighbours, the robots at the head move away from each other, leading the whole slug with them. This is particularly visible in third frame (Fig. 14c), where the slug at the back approaches the slug in the middle. In the next two frames (Fig. 14d, e), we see the robots at the head of these two slugs moving away from each other: one towards the left (slug at the back) and the other towards the right (slug in the middle), de facto avoiding each other and leading the two slugs away from each other. To achieve repulsion, leaders broadcast a gradient. When sensing the gradient of another leader, the robot at the head of the slug makes the opposite movement it just took (go to the left if it had made a move to the right).

Conclusion
In this study, we discuss slugs' uniform distribution, speed and length by taking into account the motive force, and by modelling and simulating key chemical signals propagation. We enriched our previous models [21] with slugs' speed and length during the migration phase. Results show the relationship between the length and the speed of the slug as well as the effect of ammonia on the uniform distribution of slugs around a light source.
We complement our simulation results with corresponding biological illustrations. We also show how some parts of the models translate into swarm robotics. We continue implementing our models into Kilobots to investigate how the whole model translates into swarm robotics.
Funding Open access funding provided by University of Geneva. This research is supported by the Swiss National Science Foundation (SNSF) [205321_179023].
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creat iveco mmons .org/licen ses/by/4.0/.