Collective transport of arbitrarily shaped objects using robot swarms

Out-of-the-box swarm solutions powering industrial logistics will need to adapt to the tasks at hand, coordinating in a distributed manner to transport objects of different sizes. This work designs and evaluates a collective transport strategy to move large and arbitrarily shaped objects in warehouse environments. The strategy uses a decentralized recruitment and decision-making process, ensuring that sufficient robots are in place for a coordinated, safe lift and transport of the object. Results show robots having no prior knowledge about the object’s size and shape were successfully able to transport them in simulation.


Introduction
A recent study found that potential users of storage and retrieval systems perceive swarm robots to be useful when they facilitate efficient storage, automatic inventory check, and sorting abilities [1]. Despite the generally positive reaction to robot swarms, the interviewees expressed several concerns related to safety, predictability, and the system's trustworthiness. This study explores how warehouse swarm robots can safely transport large and arbitrarily shaped objects towards a target direction without any prior knowledge of the object and the number of agents required. A large and arbitrarily shaped object in this context refers to a closed 2-dimensional shape representing the payload's base area. This area is larger than the unit size a single robot can carry. This paper presents a collective transport strategy that consists of a coordinated lift of the object once sufficient agents are recruited.
Nature provides a useful paradigm for collective transport strategies in swarm robotics. Ant colonies are a prominent source of inspiration due to their ability to collectively transport large prey to their nest. This task exceeds the capabilities of a single ant and can be achieved only through reactive actions. Such collective behaviors in nature have attracted researchers' attention for several decades [2,3]. Research has shown that there is no centralized control. Such biological systems are robust, flexible, and scalable [4,5]. Discovered characteristics and models enable engineers to replicate such complex emergent behaviors artificially. Besides using swarm robots to retrieve large and arbitrarily shaped objects in warehouses, a robotic swarm system performing collective transport could also be used in other industries ranging from logistics, agriculture, and mining to disaster support [6]. Examples of such applications are moving oversized goods such as massive airplane parts, prefabricated construction items, heavy mining machinery, or rescue equipment.
In warehouses and distribution centers, centralized multirobot systems have proven effective in performing stock storage and retrieval in highly controlled environments [7]. As online purchases increase and include larger and bulkier items like appliances, furniture or gym equipment, orders often exceed the standard size and weight a single mobile robot can carry. Nevertheless, customers expect delivery times similar to those of smaller items such as books and electronic products [8]. Another challenge is handling oversized objects often requires some form of manual labor. Despite existing health and safety regulations, warehouse workers are at high risk of accidents [9]. Large and heavy object transport is, therefore, well suited for automation. For instance, in [10], mobile robots are proposed that use handcarts and outriggers. However, solutions adaptable for various large object sizes and shapes are rare. We propose that robot swarms could be a safe and adaptable load handling solution for warehouses. In addition, unlike existing centralized warehouse systems, a swarm system only requires minimal infrastructure and setup time. Such a system could be a scalable out-of-the-box solution, especially for small and medium-sized enterprises (SMEs), which often cannot invest in a sophisticated centralized system [1,11]. This article is organized as follows: Sect. 2 discusses the background and places the study in context. Section 3 describes the methods and outlines the proposed collective transport strategy. In Sect. 4, the simulation results and performance analysis are presented and discussed. Finally, in Sect. 5, conclusions are drawn, and future work is outlined.

Background and previous research
In swarm robotics, many robots are used to collectively perform a relatively complex task. Swarm robots are designed to be simple and inexpensive, having only limited sensors and communication ranges [12]. The collective behavior at the group level emerges from individual actions robots take based purely on local information from other robots and the environment [13,14]. One of the most challenging questions in swarm robotics is how to choose the behavior of individual agents that results in the desired emergent group behavior [15]. Swarm robotics research in warehouses and storage environments has been primarily focused on collective retrieval of unit sized objects, each carried by a single robot [16].
The task of retrieving large and arbitrarily shaped objects can be divided into three subtasks. First, the object to be transported must be found. Second, sufficient numbers of agents must be recruited for a safe lift and transport. Third, the object must be moved towards a target direction.

Object search
Random walks are fundamental search strategies to find an object, particularly when there are no environmental indications, and where walkers lack localization and mapping capabilities [17]. There are many variants of random walks.
In two recent independent studies, the ballistic motion variant was reported to result in the broadest area coverage of closed environments [18,19].

Collective transport
In many ant species, workers cooperate to retrieve large prey that a single ant cannot retrieve. Typically, the ant that finds a prey object first tries to move it. If it remains unsuccessful, it recruits nestmates [12] [p. 256-259]. Like the ants, robotic swarms could work together to retrieve large objects, recruiting sufficient robots. In decentralized collective transport, coordination can be achieved without direct communication, instead relying on coordination through the object being transported [12] [p. 260]. The effect of one agent engaged in collective transport modifies the stimuli perceived by other agents and in turn, produces a change by these agents. This mechanism is known as stigmergy [20].
Other approaches are based on leader-follower principles [21][22][23]. In those studies, a leader plans the trajectory and controls the motion, while the remaining robots support the leader's motion in a coordinated manner. In other approaches, an external supervisor observes the collective motion and sends suitable control actions to the robots to lead the object towards some goal direction [24]. However, these strategies have not been evaluated and generalized to arbitrarily shaped objects. More analytical strategies have also been proposed, but they rely on sophisticated robots that know the object's shape and position to be transported [25][26][27]. As opposed to centralized control approaches, decentralized control approaches are likely to result in suboptimal system performance. However, decentralized strategies have the advantage of being scalable and can be implemented on robots with limited onboard capabilities. The agents only require local information, which they acquire through sensing their environment [28]. Some studies aim to solve the problem of transporting larger objects using robots that can self-assemble into a stronger unit to pull or push an object. Suitable controllers can be synthesized using an evolutionary algorithm [29,30]. In the work of [6], it was shown that if a number of individual agents apply forces towards the target direction, the object's center of mass will move in a straight line to the direction of the goal. The versatility of this simple strategy was experimentally demonstrated with up to 100 Kilobot robots [31] collectively transporting numerous complex object shapes. Further, it was noted that the object's rotational velocity is insignificant compared to its translational velocity. This study provides a valuable example of independently acting robots transporting objects. In this work, we consider distributed robot systems that need to actively lift an object, rather than push it. Unlike with pushing, the risk of instability makes it necessary to reach consensus about when to lift, before attempting a lift. We address this by combining a passive recruitment strategy with a decentralized decision-making process to ensure a coordinated, safe lift and transport.

Methods
A lift and transport of payload in a warehouse environment must be executed in a safe and coordinated fashion for users to accept and trust the swarm robotic solution [1,32]. We present a strategy that recruits robots under an object upheld on stilts, then coordinates the lift of the object once sufficient numbers of robots are distributed underneath the object. Here, we outline the simulated scenario and control algorithm.

Simulated scenario
Experiments were performed in a custom-built agent-based simulator in Python. Robots modeled are omnidirectional, capable of movement in any direction at a speed of 0.2 m/s. The random walk behavior and obstacle avoidance was implemented using a subsumption architecture as introduced by Brooks in [33,34]. Robots can detect they are under an object and their distance to nearby robots. Further, it is assumed that the robots are equipped with wireless mesh communication technology to communicate with each other. The swarm's task is to transport an object to the retrieval area (2 m × 5 m) on the side of the area (7 m × 5 m) , as illustrated in Fig. 1 with an arbitrarily shaped object of size 2 m 2 . The robots are initialized with random orientations and at random positions within the warehouse environment, excluding the object's area. The performance of the collective transport strategy is measured in terms of retrieval time, and number of robots used to transport an arbitrarily shaped object. The retrieval time is defined as the time taken until the robots move the center of the object beyond the vertical dashed line at x = 500 cm and into the retrieval area. The maximum time given for the swarm to complete the task is 20 min. The independent variables of the outlined scenario are the swarm size (i.e., the number of deployed robots) and the object's size. Table 1 shows the simulated parameters for the experimental warehouse setup. These parameters were chosen based on the design of our new swarm for logistics.

Collective transport strategy
As illustrated in Fig. 2, a finite-state machine (FSM) is implemented on the robots to select which behavior(s) to utilize for any given situation. Figure 3 shows an example simulation of a swarm consisting of 30 agents that collectively retrieve an arrowshaped object with an area of 1.26 m 2 . Important steps are annotated, explaining the collective transport strategy. The three main phases are object search (1a-c), recruitment of sufficient agents for safe lift and transport (2a-c), and collective transport (3a-b).

Criteria for a safe lift and transport
Positioned agents monitor the state of nearby agents. As the recruitment of agents underneath the object progresses, agents start to fulfill one of the two criteria defined below. The two criteria provide a metric to assess the local arrangement of agents. With this local metric, the aim is to allow the swarm to make a collective decision as to whether there are sufficient agents in place and whether the positioned  agents are well-distributed throughout the object. Once all positioned agents have fulfilled either one of the criteria, the swarm collectively decides that it is now safe to lift and transport the object.
1. Agent positioned near the border of the object: Every positioned agent records the number of visits from randomly walking agents. These are agents that bump into the positioned agent ( ≤min. distance to other agents, specified in Table 1). Once the number of visited agents exceeds the defined threshold (i.e., min. number of visited randomly walking agents), the agent knows that it must be positioned close to the object's border. An agent situated close to the border of the object relies on this metric to assess its local readiness to contribute to a safe lift and transport. 2. Agent positioned within the object: To judge whether agents positioned within the object (i.e., not close to the border) are well-distributed, every positioned agent observes its local neighborhood. If there are more than a predefined min. number of positioned neighbor agents within a particular radius (i.e., radius to other positioned agents), the agent evaluates itself to be part of a locally well-distributed group of agents that is ready to lift and transport the object safely. Table 2 shows the chosen parameters. Note that a smaller number of visited, randomly walking agents would lead to a lift and transport sooner. However, there is a trade-off between ensuring safety and lifting the object in a timely fashion. Choosing the parameters depends on the priority given to safety. Through numerous simulation runs, the parameters were chosen to support a safe lift and transport while also considering the agents' time to fulfill the criteria.

Collective decision making
Once a positioned agent fulfills either of the previously outlined criteria, it changes into the locally safe to lift state, in which the robot initiates a collective lift request. An agent in the locally safe to lift state is ready to lift the object and checks whether all other agents underneath the object are also ready to lift. Each agent that switches into the locally safe to lift state sends out a lift request to all agents underneath this specific object. All agents situated underneath the object respond to the request with either a negative (i.e., locally not ready for a safe lift) or positive (i.e., locally ready for a safe lift) feedback message. This lift request-response communication model is illustrated in Fig. 4. If the requesting agent receives one or more negative feedback messages, no lift is initiated. Note that the communication cost increases the larger the object because the lift requesting robot has to communicate with all other robots positioned underneath the object to check their statuses. Finally, once the last agent underneath the object has changed to the locally safe to lift state and has not received any negative feedback, the lift is initiated by a lift command message sent to all involved agents. As a result, all involved agents lift the object in a coordinated fashion and transport it towards the goal direction.

Results and discussion
Simulations are conducted to assess and validate the proposed collective transport strategy's performance for various configurations of the swarm and object size. The simulated object sizes range from 1.5 to 3.5 m 2 in 0.5 m 2 steps. These are typical object sizes of home appliances, gym equipment,  The first agent detects the object and positions itself straight away (orange state), as this agent has not received a message from any other agent that had already found the object.
More agents find the object and continue their random movement within the object area unƟl they encounter the existing cluster of the positioned robot(s).
Positioned agents that fulfill one of the two criteria change to the locally safe to liŌ state (green), in which they initially request a lift. As other positioned agents are not yet locally ready to liŌ, the liŌ request is rejected.
1a Robots are initialized in the search space, performing random walks until they find themselves under the object. The last agent fulfills either of the two criteria and switches to the safe to lift state. This agent successfully requests a collective lift as all other agents are locally ready to liŌ as well.
Upon the successful collective lift request, the requesting agent broadcasts a lift command message that is passed on to all involved agents. Subsequently, all agents collectively transport the large and arbitrarily shaped object towards the goal direction.
Agents not involved in collecƟve transport conƟnue their random walk behavior, and thus continue to avoid other agents and the object that is being transported.
A terminating message from a network host in the retrieval area is sent to at least one transporting agent and then broadcast to all involved transporting swarm agents, making them stop, put down, and leave the object.
Agents that have left the object are ready to retrieve other objects in the environment.  and furniture. The swarm sizes range from 20 to 40 robots in 5 robot intervals. The simulations were run 10 times for each configuration of an object and swarm size. In each simulation run, the object was placed at x = 200 cm and y = 250 cm with a randomly selected orientation. New large and arbitrarily shaped objects were generated for each configuration. The object generation algorithm is based on a region growing procedure that adds pixels to the existing cluster in a random fashion until the object reaches the desired area.

Reliability
To assess whether sufficient agents lift and transport the object, the number of agents transporting the object was analyzed for each simulation run and summarized for all configurations in Fig. 5. The horizontal red lines show the minimum number of agents required to feasibly lift and transport an object of a given size. As specified in Table 1 on page 3, every robot can lift a maximum load of 2 kg, and the object is defined to have an equally distributed mass of 8 kg/m 2 . Therefore, for example, for a 1.5 m 2 sized object, at least six agents are required for a feasible lift and transport. All simulation runs above those red horizontal lines, also highlighted in green, show scenarios where the object was successfully retrieved with sufficient agents. In the simulation runs highlighted in blue, the object was not lifted and transported because the safe lift and transport criteria were not fulfilled. Respectively, the number of agents transporting the object in these cases was zero.
The safe lift and transport criteria are less likely to be met by all involved agents attempting to safely transport the larger objects because more agents are to be recruited and have to fulfil the criteria within the given time constraint. Also, the number of remaining available robots that assist in fulfilling the object border agents' safe lift and transport criteria (i.e., Criteria 1) decreases as more robots are positioned underneath the object. To improve the number of cases that result in a successful lift and transport of the larger objects, the proposed method may be enhanced by an active recruitment strategy where already positioned agents actively attract available agents towards the object. Moreover, robots may be equipped with more sophisticated sensing capabilities (e.g., vision system) to identify the object and estimate the number of required agents before moving underneath the object.
Overall, the linear relationship between the object size and the number of agents that decided to lift and transport the object shows the proposed collective transport strategy's adaptability, reliability and scalability. Given that sufficient agents are deployed and each object has a unique identifier, multiple objects could potentially be transported simultaneously in a fully decentralized fashion. Figure 6 shows a heatmap of the average time taken for swarms of different sizes to retrieve different sized arbitrarily shaped objects. For the calculation of the average retrieval time, only simulation runs in which the object was retrieved are considered. The grey heatmap entries show swarm and object size pairs that did not result in any completed retrievals within the given 20 min.

Average retrieval time
Larger swarms generally retrieve objects faster because the object is more likely to be encountered by randomly walking agents. In addition, more agents in the environment help to fulfil the positioned border agents' safety criteria, leading to an earlier lift and transport of the object. Note that the number of randomly walking agents assisting in fulfilling the positioned border agents' safety criteria decreases as more agents find and enter underneath the object. Thus, especially for larger objects, swarm size should be increased initially or dynamically on demand. Altogether, the results show that large and arbitrarily shaped objects can be retrieved both reliably and in a timely fashion, given a large enough swarm size.

Conclusions and future work
This study designed a decentralized collective transport strategy that allows a swarm of relatively simple warehouse robots to retrieve large and arbitrarily shaped objects without any prior knowledge of the object's size, shape and location. The introduced approach to recruiting sufficient agents to the object, and the collective decision-making process, allows for a safe lift and transport. Thereby, we present a system that adapts to the task at hand and has the potential to become a scalable out-of-the-box swarm robotic solution requiring minimal infrastructure and setup time. The proposed collective transport strategy, verified successfully in simulations, can now be tested experimentally on real warehouse robots present in our laboratory. simulation runs in which the object was not moved s im u la ti o n ru n s th a t re s u lt e d in a s u c c e s s fu l li ft a n d tr a n s p o r t simulation runs in which the obj c e oved ct was not mo o Fig. 5 Diagram showing the number of agents moving different object sizes. The scatter points highlighted in green summarize simulation runs that resulted in a successful lift and transport of the object. In the simulation runs highlighted in blue, the positioned agents did not fulfill the safe lift and transport criteria and thus did not move the object. No simulation run resulted in a lift and transport of the object with an insufficient number of robots Fig. 6 Average time taken to complete the retrieval task where the maximum time limit is 1200 s. The strategy works most effectively when sufficient agents are deployed Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/.