Self-organized multi-target trapping of swarm robots with density-based interaction

The task of multi-target trapping in swarm robots can often be solved by global shape planning and target assignment, but it still remains a challenge to achieve fully self-organized multi-target trapping behavior based on local information. In this paper, inspired by the concept of spatial density in physics and biology, we proposed a novel density-based method to enable the swarm robots to entrap multiple targets with either single-ring, multi-ring or multi-subgroup formation in a distributed and self-organized way while neither communication among robots nor encirclement function is required. Each robot’s local spatial density is considered as the main clue for the individual’s motion decision-making and the enclosed configurations emerge from such individual-level interactions rather than being explicitly designed. Numerical simulations and real robotic experiments are conducted to validate the effectiveness of the proposed method. The results show that the proposed self-organized trapping method allows a swarm of robots to entrap multiple moving targets in a stable, flexible, noise-tolerate and size-scalable fashion.


Introduction
In nature, a particular form of intelligence is implicit in complex system composed of thousands, millions, or trillions of individual elements at all levels of biological organization, from cell clusters, to social insects, to animal societies. This collective intelligence, also known as swarm intelligence, is fascinating to scientists across disciplines, as unpredicted global collective behaviors can emerge from local and limited interactions between individuals and with the environment [1,2].
The study of swarm robotics that takes inspiration from swarm intelligence observed in nature aims at creating artificial systems with capabilities of accomplishing collective tasks that either cannot be accomplished by a single individual, or are carried out more efficiently by a swarm of robots. In previous studies, the swarm robotic system has been extensively studied to accomplish some complex tasks due to its advantage of distributed control, object clustering and sorting, navigation, pattern formation, and collaborative manipulation. It is suggested to refer to [3] for an extensive review of swarm robots.
In this paper, we take inspiration from the concept of density in physics and biology to address multi-target trapping task in swarm robotics, the solutions to which usually depends on ad hoc formation control strategies, pairwise attractive-repulsive interactions, global shape planning or optimization-based methods. The motivation of this work comes from the fact that density-based or concentrationbased interaction is a primary trigger condition in natural collective behaviors, in which the comprehensive characteristics and properties from a coherent or intermittent whole can influence the decision-making of an individual. More importantly, such density-like information (e.g., light intensity, sound intensity, odor source concentration, thermal field, electromagnetism field intensity and crowd density) in real swarm robotic applications can be collected by onboard sensors and involved in more intuitive data-processing or decision-making functions. As will be discussed in this paper, a novel density-based interaction mechanism is introduced for both inter-robot interaction and robot-target interaction, using which the robot system has the capabilities of transformation of trapping shapes, splitting and merging, tolerance to noise and scalability. Such great characteristics enable the swarm robotic systems to solve the problem of multi-target trapping in a distributed and self-organized way. The validity of the proposed method is confirmed by numerical simulations and real robotic experiments (up to 40 robots and 8 targets).
This paper is organized as follows. We discuss some related work and motivation in the next section. The problem formulation is presented in the subsequent section followed by which our proposed method is presented. Then simulation and experimental results are presented, respectively. Finally, the paper is closed with conclusions and directions for future work.

Related work
Swarm intelligence principles observed in nature have been widely studied and applied to a number of collective tasks where a group of autonomous robots are used to replace a single intelligent robot for the advantage of robustness, flexibility and scalability [4,5]. These collective tasks include aggregation [6,7], pattern formation and self-assembly [8][9][10][11], pursuit and evasion [12,13], defense and intrusion [14,15], coordinately navigation [16], collective construction [17], task allocation [18], etc. Research works have been also carried out to the emergence of simple cognitive abilities, such as leader election [19], autonomous task sequencing [20] and collective decision-making [21,22].
Multi-target trapping is one of challenging research areas in swarm robotics. It takes advantage of cooperation of simple robots in large numbers to entrap multiple moving targets that cannot be stopped by a single one. Such phenomenon is very common in animal world and has been extensively studied in prey-predator interactions. For predator, the hunting behavior usually relies on circle-like formation so that a group of weaker predators are capable of catching a stronger prey, as reported in ant colony predation [23] and wolf pack predation [24]. For prey, the defense behavior (antiattack behavior) also shows entrapment dynamics. In [25], the authors reported that spotted hyenas, which always lives in packs, display anti-attack behavior when the pack is under attack by lions. This defense behavior places the individual survival at great risk but protects the puppies or the younger members. In swarm robotic systems, the multitarget trapping has been successfully applied in applications such as search and rescue [26], collective transportation and construction [27], convoy/escorting missions [28,29] and perimeter defense tasks [30].
Over the past decade, the artificial potential field method has been widely used in multi-target trapping, where the robot moves along the gradient of several potential field functions. Generally speaking, the robot swarm is mainly driven by the attractive-repulsive force, in which the generation of trapping shape is achieved by attractive force; the collision avoidance and the obstacle avoidance are achieved by repulsive force. In [31], the attractive force between hunters and prey, the repulsive force between a pair of hunters were only introduced to achieve entrapment task. There is no need to introduce an encirclement function in this method, but the robot swarm is only able to entrap a single target with a circle shape. Similarly, in [32] the authors considered the entrapment task based on the virtual structure. The robot swarm moves towards the virtual center of a circle, and due to the repulsive force among inter-robot interaction, regular polygon formations are realized. In [33], the inter-robot interactions were assumed to be globally repulsive and selectively attractive, so that trapping shapes can be generated by choosing different interactive topologies. However, the expected trapping shapes are limited to a few of formations, such as line, circle and ring.
By introducing an encirclement function, the diversity of trapping shape has been significantly improved. In [34], the authors introduced the elliptical encirclement function into the control policy. By combing normal and sigmoid limited functions, they can organize the robot swarm into elliptical trapping shapes. However, it is limited to entrap a single target or multiple targets with elliptical shapes. To generate an arbitrary shape [35], used radial basis implicit function to describe an arbitrary pattern and the robot swarm could be driven to those predefined shapes. Although the flexibility of shape formation gets great improvement, the authors only focused on pattern formation control of swarm robots, thus no further work considering the transformation of trapping shapes was conducted. This limitation is also seen in [36] where the authors replaced the radial basis implicit function with non-uniform rational B-spline function to describe the expected pattern. The gene regular network-based controller was adopted to drive the robot swarm into the predefined patterns with no transformation and multi-target trapping abilities. These limit the applications of multi-target trapping methods in real-world problem. In our previous work [10,11], we enhanced the flexibility of radial basis implicit function in describing the trapping shapes. The modified method is capable of driving the robot swarm into a dynamically changing shape according to the specific distribution of the moving targets. Meanwhile, the abilities of splitting and merging of robot swarm are also fulfilled. In addition, influence of disturbances, modeling errors and various uncertainties in the real systems limit the applications of many multi-target trapping methods, fault diagnosis methods and reinforcement learning methods could be good potential solutions [37][38][39].
In general, the previous works show the applicability of proposed methods for multi-target trapping tasks that can be accomplished by pairwise interactions. The pairwise interaction means one robot interacts with another robot in a moment with such as attractive-repulsive interaction or obtains the weighted sum of these interactions linearly. In contrast to pairwise interaction, the density-based or concentration-based interaction uses the property of group's spatial distribution to influence the actions of individuals. For example, odor concentration is largely behind the individual's decision-making of several species, such as larvae of cockroaches prefer to follow their own strain odor rather than that of others [40], and carpenter ants were reported to be capable of aversive learning and can confirm previous findings about the different resources solicited by differential odor-heat conditions [41]. The huddling behavior of emperor penguins was reported to be a thermoregulatory behavior which reduces exposure and preserves body heat. This temperature field guided behavior enables all breeders to get a regular and equal access to the warmth of the huddles during the Antarctic winter [42]. Light traps of various species especially for flying moths were observed as lightstrength-based interaction, in which the moths method the light source according to the light intensity distribution [43]. Another important example is embryogenesis where tissue differentiation and embryonic development are guided by morphogenesis which is adjusted by concentration of gene and its products [44]. Although such phenomena have been extensively studied by biologists, few robotics researchers have taken them as inspiration for control policy design in swarm robotic tasks.

Contribution
In this paper, we consider the case of a swarm of homogenous robots entrapping multiple moving targets with appropriate encirclements. However, contrary to what is usually used in the rest of the literature [31][32][33], our controller design does not depend on the pairwise attractive-repulsive interactions, but we rather leverage the density-based interactions in both inter-robot interaction and robot-target interaction, which allow the robots to consider the local spatial density as the main clue for the individual-level decision-making.
With respect to the solutions in the rest literature [10,11,[34][35][36], our method does not involve any use of ad hoc formation control strategies to force the robots surround the targets. The proposal allows the swarm robots to take either the single-ring formation, multi-ring formation or multisubgroup formation to entrap the targets in a distributed way. These enclosed patterns cannot be obtained by a single framework in previous work, thus significantly improving the flexibility and adaptability of the proposed trapping method. Generally speaking, it is density-based interactions that drive the swarm robots like a fluid to encircle the targets, thus the enclosed formations are obtained by the property of fluid rather than the explicit formation functions. The enclosed configurations emerge from the individual-level interactions rather than being explicitly designed; therefore, the proposal is also self-organized. The effectiveness, robustness and scalability of the proposal are evaluated in numerical simulations. The validity of the proposal is also confirmed in real robotic experiments.

Multi-target trapping problem
Consider a swarm of homogenous robots and a group of targets moving in 2-D obstacle-free space. The targets can be moving vehicles or other robots. Both robot and target have limited sensing range and cannot share information among neighbors. The task assigned to the swarm robots is to entrap the targets, i.e., move around a target in a circle formation or move around a group of targets in an irregular encirclement, which is also referred to as escort/entrapment problem [28], as illustrated in Fig. 1.
Let r i , v i and u i be the position, velocity and control input of robot (target) i where i = 1, 2, 3, . . . , N (M). Then the robot (target) can be described by the double integrator dynamics as follows: where v max and u max are the maximum velocity and acceleration for robot (target), respectively. The entrapment task requires the robots to overtake and box the target in, therefore it is not feasible to form a circle-like formation if the robots move slower than the target. For example, the target can always keep away from the robots by moving at its maximum speed in the opposite direction if they are far apart initially. To make it possible to entrap the moving targets, we assume the maximum speed of robots is greater than that of targets. It can be envisioned that a stable entrapment state will occur when a target is surrounded by the robot swarm symmetrically. To evaluate the phenomenological dynamics and group-level configurations of multi-target trapping process, we assess the quality of collective trapping by considering the following three different metrics.
(1) A morphological quantity is introduced to characterize whether the target is encircled by a group of robots. It is wherer ik is an unit vector pointed from robot i to target k, A k and a k are the set and the number of robots surrounding the target k, respectively. C k pat ∈ [0, 1]: as C k pat ≈ 0, robots surround the target evenly (a perfect trapping circle); as C k pat ≈ 1, robots are located at one side of the target, which usually displays the collective chasing behavior.
(2) The distance between the target and the robot is used to present the situations in which the target is being chased and/or surrounded by the group of robots. It is defined as where M i is the set of targets perceived by robot i (see Eq. (8)). (3) The maneuverability of target is evaluated to characterize whether the target is forced to stop its movement. It is regarded as a successful entrapment when all targets get stuck and unable to break out from the encirclement. Formally, it is given by where C k mov presents the movement state of target k. If all targets are perfectly entrapped by the robot swarm, we have C mov = 0.
With these three metrics, the multi-target trapping problem can be described as: Given N homogenous robots, M targets and any initial configuration in a limited space ⊂ R 2 , design a distributed control policy for the robots to entrap all targets without interrobot collision, i.e., C k pat (t) → 0, C mov (t) → 0 and d it (t) converges to a constant in a limited time.
Specially, our objective is to design a distributed controller that allows swarm robots to entrap multiple moving targets in a self-organized way while satisfying the requirements of adaptive splitting and merging of encirclement shapes. We now discuss the proposed methodology to solve this formulated problem.

Methodology
In this section, we first provide the dynamics of moving targets in the multi-target trapping process. We then present how to leverage the density-based interactions between inter-robot and robot-target for designing control policy in multi-target trapping problem.

Dynamics of targets
Prior to giving the solution of multi-target trapping problem, we first provide the dynamics of moving targets. Assume that the targets have an escape reaction to nearby robots and there is no interaction between targets. The maneuvering mode of the target is given as The first term u self k is the self-propulsion of target. It is used to regulate the speed autonomously, allowing the target to accelerate from standstill and decelerate from high speed to maintain a suitable speed. Formally, it is given by where a friction-like dissipative force ζ v k 2 v k is balanced by a self-accelerating force α k v k , such that the targets can move at a suitable speed v k = √ α k /ζ . Note that this selfpropelling term tends to maintain the speed of the target but not the direction.
For simplify, we assume that target has an escape reaction to its nearby robots and there is no interaction between targets. The escape reaction of target k from N k nearby robots is presented by a repulsive force where r ik = r i − r k is the position vector from target k to robot i ∈ N k andr ik is its unit vector. d 0 > 0 is a constant, determining the point that generates the maximum value of the repulsive force ( r ik = d 0 ). It is worth noting that the escape response described by Eq. (7) is a non-monotonic function, see Fig. 2A. When r ik > d 0 , the target's response gradually increases as the robot approaches  5) and (9), respectively the target; When r ik < d 0 , however, the target's response gradually decreases, representing that the target is going to lose maneuverability gradually if the robot is close enough to the target (e.g., a direct physical attack such as tearing occurs). As shown in Fig. 2B, target's escape response is strongest when d 0 = 0, and it decreases gradually as d 0 increases. As d 0 > 1, target's response is almost disabled and it remains stationary even after being attacked. In this paper, we let the self-propelled coefficient α k = 0 and d 0 = 0.5. It means that the target remains motionless unless the threat is perceived directly. Note that this strategy of avoiding danger is relatively conservative but energy-efficient.

Controller of robots
Due to physical constraints and limited sensing range of real robotic system, we assume that each robot interacts with nearby robots (targets) within a certain range R, see Fig. 1A. N i and M i are defined as the set of robots and targets in the neighborhood of robot i: The control input u i of robot i is composed of four parts, namely, the self-propulsion u self i , the collision avoidance u rep i , the inter-robot interaction u rob i and the robot-target interaction u tar i : The self-propulsion. The first self-propelling term is similar to Eq. (6) which makes robots move at a suitable speed Refer to the movement ability of the actual robot, and to avoid unrealistic acceleration of individuals and oscillation of group, we let the acceleration coefficient α i = 5 and the friction coefficient ζ = 10 in all simulations. The collision avoidance. To ensure collision avoidance, the inter-robot repulsive interaction is used and is given by where k rep > 0 is the repulsive interaction strength. Although the repulsive interaction is primarily responsible for collision avoidance, we allow this repulsive force to be implemented on the whole sensing area instead of a local collision avoidance area. The motivation behind this setting is that we put emphasis on more moving space of the whole robot swarm rather than a safe distance between a pair of robots. The repulsive interaction through the whole neighborhood area is also beneficial for the necessary segregation (splitting) behavior, which is a good characteristic in meeting the requirement of entrapment of multiple targets with multiple subgroups, see Fig. 3D. We have k rep = 50 in all simulations. In the following of this section, we are presenting how to estimate the density filed of a swarm robotic system and how to leverage the "robot density" into inter-robot and robottarget interactions. These novel density-based interactions allow the swarm robots to entrap the moving targets in a distributed way while no encirclement function is needed.

Estimation of the robot density
Based on the concept of density, the spatial distribution characteristics of a robot group can be quantitatively characterized as a form of spatial density field similar to magnetic or optical field. The robot density, strictly speaking the intensity of the density field at the robot's location, is defined as the weighted sum of distances to its nearby robots within the sensing range (in practice the density can be obtained by distance-relevant quantities instead of using relative distance directly, such as light intensity and sound intensity). Formally, the definition of density ρ i for robot i can be described as where W ( * , * ) is the kernel function that is used to estimate the spatial distribution density of surrounding neighbours; R

Fig. 3
Schematic diagram of how to estimate the density field and how to leverage the robot density into inter-robot and robot-target interactions. A Some kernel functions that can be used in density estimation. B Demonstration of the local density field generated by 10 robots. The focal robot is marked by the symbol of '•' and its neighbours are marked by the symbol of '×'. C Illustrations of the 1D schematic for densitybased inter-robot interaction. D Schematic diagram of the effects of inter-robot interaction, collision avoidance and robot-target interaction on group trapping behavior is the sensing range of robot; r i j = r i − r j is the position vector from robot j to robot i. It is reasonable to assume that the contribution of an individual to the spatial density field decays with distance. Thus, a variety of kernel functions can be employed to estimate the robot's local density, such as Gaussian-like function, cubic polynomial, and four-power polynomial. As shown in Fig. 3A, these kernel functions decrease monotonically with distance and they are smooth and derivable. In this work, the cubic spline function is used as the density function which is widely used in smoothed particle hydrodynamics [45] where r i j is the relative distance between robot i and j. As an integrated and comprehensive property of the local group, the robot density represents the neighborhood distribution characteristics of the robot swarm. Figure 3B shows a case of density field formed by 10 robots with random distribution.

Density-based interactions
There are various versions of inter-robot interaction mechanisms, where the basic idea of most works models the inter-robot interaction with pairwise interaction using the position and velocity information of nearby robots. Instead of using the pairwise interaction, we leverage the robot density and propose a new density-based interaction mechanism for robot swarm.
Density-based inter-robot interaction. Taking inspiration from simulating the dynamics of fluid, the smoothed particle hydrodynamics is well studied to be an appropriate framework for control of swarm robots, which is capable of driving the swarm robots to move in a fluid-like formation [45][46][47][48][49].
We follow this line of methodology and the inter-robot interaction force is defined as where ρ 0 > 0 is a reference density that controls the expected distribution of the robot swarm and k rob > 0 is the inter-robot interaction strength. ∇ i is the gradient operator used to calculate the gradient of the density field at the position of robot i. Figure 3C gives an illustration of how inter-robot interaction works by means of density-based interaction (A simple case in 1-D with 4 robots is presented). It is seen that the focal robot can be driven by either repulsive or attractive force in case of different desired densities ρ 0 . Actually, the robotic swarm will converge asymptotically to a predefined density ρ 0 from the gradient descent direction of the density field, i.e., all robots will reach a steady state with the same robot density, which corresponds to a ring robotic pattern (for 2-D space) or a sphere robotic pattern (for 3-D space). However, the inter-robot space is inconstant and the collision avoidance cannot be guaranteed, see Fig. 3D. This issue can be well addressed by leveraging the inter-robot repulsive interaction as provided by Eq. (11).
Density-based robot-target interaction. With density-based inter-robot interaction and inter-robot repulsive interaction, the robotic swarm can spontaneously form the regular singleor multi-ring-formation, as well as multiple sub-group structures, see Sect. 5.1 for details. The question is how to make these promising formations around the target. The idea is that we consider the target as a "condensation nucleus" so that the robots gravitate towards this "condensation nucleus" and an encirclement can be obtained ultimately, see Fig. 3D. Based on this idea, the robot-target interaction is introduced by modifying the inter-robot interaction as follows: where M i is the set of targets detected by robot i, k tar > 0 is the robot-target interaction strength. To generate trapping shapes, an encirclement function is usually needed for entrapment task in previous works. Here the densitybased interaction mechanism is further used in robot-target interaction and the trapping shapes can be self-generated without any encirclement function involved. The key idea is to allow each robot to exert density-based interactions to both neighbor robots and detected targets while with more weight on the robot-target interaction. Thus, the entrapment state can be obtained. Through parameter sensitivity analysis (see below Sect. "Single-target trapping" for details), we let k tar = 10k rob and k rob = 100 to ensure that the robot performs more weight on the robot-target interaction in all simulations.
In summary, the proposed density-based controller of swarm robots for multi-target trapping can be expressed as It is worth noting that the parameters included in Eq. (16) are divided into two categories, namely, the intrinsic parameters of robots and the tunable parameters of target trapping. The intrinsic parameters are used to characterize the motion, perception, and inter-robot interaction, including α i , ζ , R, k rep , and k rob , which we suggest to set according to the actual robot platform. The tunable parameters of the trapping algorithm corresponds to the robot-target interaction, including ρ 0 and k tar , which can be used to regulate the performance of multi-target trapping. Therefore, we only consider ρ 0 and k tar in our analysis of parameter sensitivity.

Remark 1
In terms of the computational cost of robots, our method essentially produces a trapping behavior that does not reply on information sharing, predefined shape functions, global shape planning or optimization operations. The main computational steps are density estimation and density gradient calculation, both of which are positively and linearly increased with the number of robots and targets within the individuals' sensing range. Therefore, the computational complexity of our method for robots is low compared to global shape planning or optimization-based trapping methods.

Simulation results
In this section, we first investigate the density-based controller for swarm robotic systems in the absence of target. The exposition of this part is to present the starting point and basic inspiration of using density-based controller to solve multitarget trapping problem. The single-target trapping problem is then investigated to present the basic trapping patterns and the parameter sensitivity, followed by the investigation of multi-target trapping. Finally, the robustness under noisy environment and the scalability to large groups are analyzed.
Unless otherwise stated, the values of parameters for our simulations are set as R = 10m, k rob = 100, k tar = 1000, k rep = 50, v max = 5m/s, u max = 20m/s 2 , α i = 5, ζ = 10 for robots; k esp = 20, v max = 3m/s, u max = 20m/s 2 , d 0 = 0.5m, α k = 0, ζ = 10 for targets. Initially, all robots and targets are randomly distributed inside of a circular area with the radius of R to ensure that the targets are not missed by robots. Animations corresponding to the simulation results can be found in https://youtu.be/Yc_P94UJCCU.

Pattern formation in the absence of target
We start by investigating the stable configurations of robots generated by the density-based controller in the absence of target. It is observed that the robots can form one or several stationary equilibrium clusters in which the robots are evenly distributed, see Fig. 4A. By altering the reference density ρ 0 from 1.0 to 0.06, the distribution of the symmetric pattern changes regularly, from multi-ring (phase I) to single-ring (phase II), to multi-subgroups (phase III). Two metrics: the distance to group center d c and the nearest neighbor distance d nn are used to characterize these three phases. With the two metrics, a more intuitive diagram that shows how reference density determines the patterns of robots is presented in Fig. 4B. It can be seen that the phase changes from I to II and III with the decrease of the reference density. The critical densities for transitions I to II and II to III are ∼ 0.25 and ∼ 0.19 respectively. Although the specific values of critical densities may vary with different simulation setups, the qualitative results we have obtained are well maintained. The results demonstrate that it is promising to use the density-based controller for multi-target trapping problem as the controller is very flexible in generating symmetric and closed formations in three different phases.
By changing the reference density or sensing range of the robots, the self-organized merging and splitting behaviors can be observed in the robot system, which implies that the density-based controller has a great potential for applications in multitasking and concurrency. A typical splitting-merging case is shown in Fig. 5. It is observed that the self-organized splitting and merging behaviors emerge at 100th and 400th iteration respectively where the reference density ρ 0 is changed from 0.5 to 0.1, and the sensing range R is enlarged from 10 to 20 m. Such merging and splitting behaviors can be regarded as a fission behavior where the robot swarm has a "critical capacity", beyond which the robot swarm is going to divide into multiple subgroups. It is still an open question what exactly determines the specific upper-bound, one can refer to our previous article [50] for more theoretical results about this issue. However, we can use this characteristic to qualitatively fulfill entrapment tasks that require splitting and merging behaviors.

Single-target trapping
Before moving to the multi-target trapping problem, we start our trapping investigation from the simplest scenario where only one target is involved. The first primary issue of our interest is what is the minimum number of robots required for successfully trapping a single target? We compose a series of single-target trapping scenarios in Fig. 6. It is seen that an interesting "encircling-following" pattern emerges in the cases of one-target-one-robot and one-target-two-robot, where the robot(s) follows and moves around the target in a circular formation, see Fig. 6a1, a2. Obviously, such periodic pattern demonstrate that a single target cannot be blocked and successfully entrapped by a single or two robots. In fact, at least three robots are required to entrap a single target stably, as shown in Fig. 6a3. The entrapment configuration can be also ensured if more than three robots are involved, see Fig. 6B. In addition, Fig. 6b1-b3 show that a single target can be well entrapped with either single-ring or multi-ring formation with appropriate reference densities. For the single-ring formation, the density of the robot swarm converges to a specified value (which has a trivial difference with the predefined reference density due to the fact of repulsive force among inter-robot). For the multi-ring formation, the density distribution in steady state agrees with the number of rings.
The second issue is what is the effect of different parameter settings on the final morphology of target trapping? That is, the parameter sensitivity analysis of the proposed method. As shown in Fig. 6C, we observe that as the reference density ρ 0 increases, the values of C pat and C mov decrease and increase slightly, respectively, but always remain near the minimum value of 0. It indicates that the change of ρ 0 does not affect the formation of the symmetric, ring-shaped encirclement. However, as ρ 0 increases, the trapping size gradually increases; after ρ 0 > 0.5, robots are more likely to form a single encirclement. In contrast, when ρ 0 < 0.05, the probability of trapping size less than 3 increases noticeably, which leads to an increase in the rate of trapping failure. For the parameter k tar , when k tar < 5k rob , both C pat and C mov are much larger than 0, which indicates that the robots gather on one side of the target and cannot form a ring-shaped encirclement around the target (see the insect of Fig. 6c4), i.e., the capture fails.    In contrast, when k tar ≥ 5k rob , the robots are able to form a ring-shaped formation to encircle the target. Meanwhile, as k tar increases, the number of robots surrounding the target also increases. Therefore, we let k tar > 5k rob and ρ 0 > 0.1 in the simulation.
The third issue is what kind of motion patterns and dynamics can emerge from the density-based interactions? As shown in Fig. 7, two similar but not identical motion patterns are observed during the trapping process, which are referred to as the encirclement pattern and the pursuit-encirclement pattern. The encirclement pattern emerges when the target is placed inside of the group of robots. In such case the target attracts the robots to generate a ring-like encirclement around itself. The pursuit-encirclement pattern emerges when the target is placed outside of the group of robots. In such case the robots will first move towards the target (pursuit behavior), then get around the target (overtaking behavior), and ultimately entrap the target (encirclement behavior).
Although only one target is involved in this subsection, the simulation results indicate that the proposed density-based method is flexible in entrapping the target with either a singlering formation or a multi-ring formation regardless of the target being initially placed insider or outside of the group of robots, which is of great significance in practice.

Multi-target trapping
Compared with the scenario of single-target trapping, the dynamic process of multi-target trapping is much more complex. A typical case of multi-target trapping is given in Fig. 8A, where 30 robots and 5 targets are involved. As we can see the robot group spontaneously splits into 5 clusters and ultimately entrap all fleeing targets in a steady state. Both encirclement motion pattern and pursuit-encirclement motion pattern are observed during the trapping process, demonstrating that they are two primary behaviors in multitarget trapping regardless of the number of robots and targets involved. Furthermore, Fig. 8B shows several scenarios with different number of robots and targets. It can be observed that most targets are stably encircled by more than three robots. Some unsuccessful cases are also observed where one (or two) robot fails to entrap the target. As the entrapment configurations emerge in a self-organized way, these failed cases occur stochastically. It is believed that the number of robots and targets mainly determines the final configurations.
To quantify the multi-target trapping performance of the proposed density-based method, three metrics are introduced to characterize the efficiency of trapping process: (i) the trapping size (n trap ), which denotes the group size of each trapping cluster after the target is encircled; (ii) the unsuccessful trapping probability (P fail ), which denotes the proportion of uncaught targets among all targets. A target being entrapped by 0, 1 or 2 robot(s) is regarded as an unsuccessful case; (iii) the trapping time (T trap ), which denotes the elapsed time from the initial configuration to the final encirclement configuration, i.e., where θ trap is a threshold that determines whether the trapping process is complete (we let θ trap = 0.1 in all simulations). For a multi-target trapping task, the ideal situation is that the unsuccessful trapping probability is zero, the trapping size is the same for each target, and the encirclement pattern of robots can be formed as quickly as possible. Figure 8C shows the statistical results in multi-target trapping, each simulation is implemented by 50 runs. It is seen that when 30 robots are used to entrap 2-4 targets, the robot group spontaneously splits into 2-4 clusters with similar size to entrap these targets. However, when the number of targets is more than five, the unsuccessful case is observed and this issue is about to get worse when increasing the number of targets. When the number of robots is increased to 50, the unsuccessful probability decreases dramatically, demonstrating that the group size of robots play a key role in avoiding unsuccessful trapping. An interesting point is that when trapping 2-5 targets with 30 robots or trapping 4-10 targets with 50 robots, the trapping size for each target is statistically homogeneous, i.e., n trap ≈ N /M. This highly expected property comes from the "critical capacity" of robot groups, i.e., the main-group can spontaneously split to multiple sub-clusters with approximately the same size. It emerges from the novel density-based interactions; See Sect. "Pattern formation in the absence of target" and our separate article [50] for more details. Moreover, as the number of targets increases, the elapsed time for robot groups to entrap multiple targets will be longer.
Above results indicate that the proposed density-based controller is effective in solving multi-target trapping problem. It is worth noting that there is no mechanism for negotiation and task allocation between individuals. Thus, the enclosure movement of robots is completely self-organizing.

Robustness and scalability
For real-world applications, noise or randomness is inevitable and may come from various sources, such as perturbations of the medium (winds, currents, uneven ground, etc.), sensor errors and actuation errors of motion control devices. In some cases, these uncertainties may cover up the real intentions of robotic system, leading to the loss of capability of performing even very simple missions. In this section, we investigate the performance of our proposed method in noise environment, i.e., the robustness to noise.
The stochastic noise is introduced into individuals' motion equation as where i ∼ U (−0.5, 0.5) is a unit uniform distribution and ξ > 0 is the noise intensity. Here, the noise term is added to the acceleration to characterize that the desired acceleration is perturbed by certain irresistible external forces, which can result in significant changes of the individual's velocity. Figure 9A shows several multi-target trapping cases with varying noise intensities. As the increase of noise intensity, it is observed that the motion of robots and targets has great randomness and strong uncertainty, especially when ξ ≥ 30. Although such randomness leads to zigzag-like trajectories and irregular encirclements, all targets are still successfully entrapped by the robots with multiple clusters. In the presence of noise, another interesting phenomenon is that mergingsplitting behavior occurs more frequently, see video in the supplementary information for details. This is because the sub-clusters are more likely to move close to one another due to random movement. As a result, the robots are going to reorganize the trapping shape according to the changing configurations. In addition, Fig. 9A shows the targets can be still well entrapped by the robot group when the number of targets and the group size of robots become larger, demonstrating that the proposed method has a good scalability.
To further evaluate the performance of our density-based trapping method in the noisy environment, we conducted a statistical analysis of multi-target trapping and the statistical results are shown in Fig. 9B, in which each simulation is implemented by 50 runs. It can be seen that when the noise intensity ξ < 50, the probability of unsuccessful trapping is extremely low and can be negligible. It indicates that weak or moderate noise does not affect the trapping performance of the robot groups. When ξ ≥ 50, we observe that both P fail and T trap increase significantly, which indicates that too strong noise leads to a high randomness of movement, such that the trapping efficiency of the robot groups will decrease obviously. Interestingly, however, the change of noise intensity has no significant effect on the trapping size, which stays near the statistical mean. This demonstrates that the proposed density-based trapping method is well tolerance to noise.
It is worth noting that the uncertainty of targets can be hardly predicted and thus making the multi-target trapping task more challenging. The proposed method has a promising performance in such case and can ensure the robots to catch up with the targets. Although the robots cannot maintain regular formations (regular circle formations), they are consistently located around the target during the trapping process, indicating that the encirclement effect of the robot swarm is still valid and the robot group does not lost its ability of trapping in the presence of noise.

Benchmark comparison
This subsection presents a comparison investigation with a greedy trapping method as reported in [31], in which the interactions of robot-robot repulsion and target-robot attraction are considered. For comparison, we replace the robot-target interaction as follows: where k tar is the strength of robot-target interaction (k tar = 100), m ∈ M i is the target of robot i. The closest target is selected if more than one target exists within the robot's sensory range. It means that the target of each robot depends entirely on the initial distribution of robots and targets, and the finial size of each subgroup is almost fully random. In this way, we can use the greedy multi-target trapping method as a benchmark (a feasible lower bound) for the performance comparison. The most significant advantage of our trapping method over the greedy trapping method is that the size of encir- clement, i.e., the number of robots encircling the target, is well controllable. As shown in Fig. 10a, the size of the encirclement decreases as the reference density ρ 0 is decreased from 0.5 to 0.2. It implies that the robotic system with the proposed trapping method is capable of fulfilling a specific target-trapping task in a self-adaptive way. However, this adaptive allocation is beyond the capability of the greedy trapping method, where all robots will encircle the target no matter how many robots are involved.
To further confirm this advantage, the statistical comparison results for multi-target trapping scenario with [N = 50] are presented in Fig. 10b, c. Each simulation is implemented 50 independent runs. It is seen that when using the greedy trapping method, the unsuccessful trapping probability and the difference of the trapping size for each target are significantly higher than that of using our proposed method. This result is independent of whether the noise is introduced or not. It indicates that the multi-target trapping behavior achieved by the greedy trapping method is completely random. In contrast, the statistical results show that the trapping size of each encirclement is very close when using our trapping method. Besides, the probability of unsuccessful trapping is also very low.

Real robotic experiments
To evaluate the feasibility of the proposed controller in real environment, we conducted experiments using SwarmBang robotic system. The SwarmBang robot developed by our team is a differential-wheeled driven, teacup-sized mobile robot specialized for the self-organized swarming behavior studies, see Fig. 11. The experiments were conducted within an area of 5140mm × 4700mm with the origin located at the center of the area. We employed a motion capture system manufactured by NOKOV to track the positions and headings of all robots. The global information is broadcasted to all robots via a customized communication protocol based on a 2.4 G RF module. Each working robot makes its own decision according to the local information extracted from the global message. That is, the setup of our experiment is centralized-perception and decentralized-decision-making. This pseudo-distributed structure is flexible in conducting a wide range of swarm robotic experiments.
The proposed density-based controller cannot be applied directly as the real robot is not a fully actuated agent. To address this issue, the robot is controlled by its linear and angular velocities v and ω. Assume that the current moving direction and the expected moving direction of the robot are θ and θ d , respectively. The orientation error then is dθ = θ d −θ , where dθ ∈ [−180, 180) deg. The angular speed is expressed as ω = dθ t with a maximum vale of ω max = 47.75 deg/s, where t = 0.5 sec is the update interval. The speed of robot is discretized into five settings based on the excepted Then, the linear and angular speeds are used as inputs for the differential drive model as where v L and v R are the speeds of left and right wheels, respectively. l L R is the distance between the left and right wheels (l L R = 48 mm for our SwarmBang Robot).
Using this simple open-loop control strategy for stepper motors, the robot can actually track the velocity with a sufficient accuracy. Unless otherwise stated, the parameter values for all real robotic experiments are set as R = 1000 mm, k rob = 100, k tar = 1000, k rep = 10, v max = 20 mm/s, u max = 100 mm/s 2 , ω max = 47.75 deg/s, α i = 2, ζ = 5, λ = 0.2 for robots; k esp = 20, v max = 10 mm/s, u max = 100 mm/s 2 , d 0 = 10 mm, α k = 0, ζ = 10 for targets. Note that, targets are assumed to move more slowly than the robot swarm. Video corresponding to experiments 2-3 can be found in the supplementary information.

Experiment 1: single-target trapping
We first experimentally validate the performance of our method in the case of one target. We use 10 robots to demonstrate the encirclement pattern and the pursuit-encirclement pattern. For the encirclement pattern, we deploy the robots with random orientations and positions, and the target is inside of the robot group. As we can see from Fig. (12)A, the robots are attracted to the target and drove onto a ring formation according to the density-based interaction. This ring formation will be further fine-tuned by collision avoidance force and ultimately a regular circle formation can be generated with the robots. Since the target is approximately For the pursuit-encirclement pattern, we deploy the target 1 m away from the robots initially. As we can see from Fig. 12B, the robot group splits into two groups first, with six robots that are closer to the target chasing the fleeing target and five robots forming a smaller circle formation near the initial place. During the chasing process, the two frontmost robots suddenly decelerate once the distance to the target reaches a certain value (it seems like they are waiting for the others), and then accelerate again when the followers catch up. The six robots ultimately overtake the target and form a stable encirclement. Such process is referred to as the pursuit-encirclement pattern in this work. Since the target keeps moving away from the robots during the pursuitevasion stage, it moves a relatively large distance.
Although only ten robots and one target are involved in the single-target trapping scenario, the experimental results already show that the proposed method can successfully implement both the encirclement and pursuit-encirclement patterns. Multi-ring formations can be also obtained if the number of robots or the reference density is increased, which is omitted here. Note that the robots' velocity is non-zero when the configuration is stable, which is a result of the physical constraint of our robots (they are not fully actuated). Similarly, the density of robots also fluctuates regularly near the steady state caused by the rotating motion of the robots. Despite these practical issues, our method is quite effective in real robotic environment.

Experiment 2: multi-target trapping
We implemented a series of experiments in multi-target trapping scenarios, all of which yielded similar outcomes. Here we present two specific cases: the small-size trapping task and the medium-size trapping task.
The small-size trapping experiment was conducted with 20 robots and 3 targets. The experimental results are shown in Fig. 13A, in which the snapshots of the experiment, the full trajectories of robots and targets, and the profiles of speed and density of robots are presented. It is observed that the robot group spontaneously splits into three clusters (with 7, 7 and 6 robots in each cluster) corresponding to the three moving targets, and ultimately entraps all three targets with ring formations. It is worth noting that the target is going to be enclosed by which robot cannot be predetermined or predicted. The final encirclement configuration is heavily influenced by the initial configuration and motion uncertainty during the trapping process. The video in the supplementary information shows that the robots may change the goal (the specific target) frequently, especially when the robot group is splitting from the original group to several new groups.
The medium-size multi-target trapping experiments were conducted with 40 robots and 5-8 targets, see Fig. 13B. It is observed that our trapping method still performs well even in the case of doubling the number of robots and quadrupling the number of targets. Although only up to 48 robots are used in the experiments, it is straightforward to use more robots and targets to fulfill the multi-target trapping tasks as the basic motion patterns remain unchanged. This indicates that our proposed density-based multi-target trapping method has good scalability. The behavioral characteristics and dynamics are similar to the aforementioned small-size case and will not be described again here. One new observation is that the group size of robots in each encirclement may be very different in real applications. For example, a target is entrapped by three robots while another one is entrapped by ten robots, see the case of 40 vs 7 in Fig. 13B. It seems inevitable as the multi-target trapping behavior is fulfilled by the robot swarm in a self-organized way.

Experiment 3: patrolling-trapping task
To demonstrate the application of the proposed method in real-world problems, a more realistic task is considered. It is called patrolling-trapping task, in which a team of robots patrol a given region with a tight formation (the patrolling mode), and when they encounter the targets, they switch into the trapping mode. Unlike the previous setups where the robot group and the targets were deployed together at the initial stage, in the new task the robot group and the targets were deployed far away from each other. The trapping behavior is only activated for those robots who detect the targets. Since the robots have limited sensing range, the trapping mode may be activated asynchronously. The experimental results are shown in Fig. 14, where 20 robots and 3 targets are involved and the Viseck-like behavioral rules are implemented for the robots in the patrolling mode.
It is observed that the robot swarm performs the regional patrolling task with a compact formation along with a 'U'shape trajectory, the targets move along with a 'S'-shape trajectory. The targets are detected at t = 267 sec by some robots, and then the self-organized trapping behavior of the robotic swarm is triggered. Subsequently, the robots disperse and move according to the density-based method, result- ing in a pursuit-encirclement motion pattern. Ultimately, the three targets are entrapped by three sub-groups of robots. The entire trapping process takes about 150 sec. This patrollingtrapping experiment suggests that in combination with other collective motions, our proposed density-based controller is very promising for a wide range of swarm robotic applications.

Conclusion
In this paper, we proposed a novel density-based controller for the robot swarm to address the multi-target trapping problem in a self-organized way. We take pattern formation in the absence of target as our starting point. A series of symmetric ring formations generated by the proposed method suggest that the multi-target trapping problem can be well solved using these enclosed formations. The selforganized splitting and merging behaviors observed from the pattern formation process ensure that the robot swarm can be leveraged in applications of multitasking and concurrency. We then demonstrated that the proposed density-based controller can be well extended into single-target trapping and multi-target trapping problems. The simulation and experimental results demonstrate that the proposed density-based controller has good flexibility, scalability and robustness in addressing multi-target trapping problem.
In spite of the positive results, there are still limitations that can be further improved. For example, our method does not consider the issue of obstacle avoidance. Another independent repulsive force is usually used for obstacle avoidance, which increases the difficulty of multi-target trapping. This is because the final stable configuration can be seriously affected by the obstacles and the robot swarm is likely to be failed to accomplish the entrapment task. On the other hand, some kind of obstacles can be beneficial for the entrapment task where part of or all of them can be build blocks of entrapment circle. Considering these facts, the obstacle avoidance is not taken into account in this paper and will be an interesting research topic of swarm robotics multi-target trapping in the near future.