Urban Firefighting Drones: Precise Throwing from UAV

In recent years, the use of unmanned aerial vehicles has spread across different fields of the industry due to their ease of deployment and minimal operational risk. Firefighting is a dangerous task for the humans involved, in which the use of UAVs presents itself as a good first-action protocol for a rapid response to an incipient fire because of their safety and speed of action. Current research is mainly focused on wildland fires, but fires in urban environments are barely mentioned in the bibliography. To motivate the research on this topic, ICUAS’22 organized an international competition inspired by this mission, with the challenge of a UAV traversing an area populated by obstacles, finding a target, and precisely throwing a ball to it. For this competition, the Computer Vision and Aerial Robotics (CVAR-UPM) team developed a solution composed of multiple modules and structured by a mission planner. In this paper, we describe our approach and the developed architecture that led us to be awarded the first prize in the competition.


Introduction
In recent years, unmanned aerial vehicles (UAVs) are being used in more and more fields across the industry because they are easy to deploy and have minimal operational risk. Examples range from industrial inspection or search and rescue to volcano monitoring. One of the fields of interest for their use is firefighting, since the dangerousness of the task combined with the importance of a quick reaction. Fires can spread quickly, damaging not only people, but also the armature of buildings or similar structures, and consuming oxygen in enclosed spaces. Therefore, the use of UAVs could be a good first-action protocol as a quick response to a starting fire, due to their safety and speed of action. Its fast deployment and ability to reach places of difficult access permit them to reach the fire easier than a human team with any other kind of tool or vehicle.
Current research focuses mainly on wildland fires, but fires in urban environments are hardly mentioned in the literature. Urban firefighting has its own peculiarities. Buildings are much taller than trees, so the clearance height is much higher than in a natural environment. Consequently, approaching the fire zone may imply to plan a route between buildings while avoiding collision with them. In addition, the fire might emerged inside constructions, only accessed by their exterior windows. This, coupled with the limited payload of drones designed for city flying, suggests that launching fire suppression devices through windows from a UAV becomes a strategy to be taken into account.
Nowadays, the tasks developed for drones are mainly piloted by humans from the ground. But current research in aerial robotics is focused on increasing drones autonomy, making possible to complete the mission without human intervention.
In this situation, the 2022 International Conference on Unmanned Aircraft Systems (ICUAS'22) organized its first UAV competition, designed to simulate the mission of extinguishing a fire inside a building by precisely throwing a spherical extinguishing device through a window or similar opening. The competition consisted of two parts: a virtual phase for classification and a real final phase at the conference venue in Dubrovnik, Croatia.
To tackle the challenge, we developed a modular framework, where each module is designed to perform one task. Our solution also proposes a mission planner algorithm for activating and deactivating each module when needed. In the competition, we tested our approach in the virtual environment, obtaining first place in the classification phase. Moreover, we brought the system into the real environment with great results, winning the finals against other finalist teams from around the world. In this paper, we describe our approach and the solution developed for this challenge.
The main contribution of this work is the design of a framework that can be used in a simplified urban firefighting mission. Besides, we would like to highlight the mission planner that provides a structured behavior to the solution, a seek strategy for target detection, and a throwing maneuver using speed commands.
The rest of this paper is organized as follows: Section 2 provides related work relevant to this project. Section 3 describes the challenge and its constraints. Section 4 presents our solution, describing in detail each module that composes it. Section 5 describes the scenarios faced in both phases of the competition, and Section 6 presents the results and subsequent discussion. Finally, Section 7 explains conclusions and future work.

Related Work
In the last decade, many researchers and industries have suggested different drone applications to help firefighters do their job. In fact, drone technology has been applied to different missions such as fire detection, fire monitoring, and fire extinguishing. This work focuses on the last topic, fire extinguishing and, more specifically, ball-tossing fire extinguishing.
The use of drones in firefighting has already been extensively discussed in the literature: • In [1], authors explored how drones are integrated into firefighting teams. They propose a Human-Drone Interaction methodology based on gestures, where both can communicate directly with each other.
• Mauro S. Innocente and Paolo Grasso propose selforganizing swarms of firefighting drones, with a focus on the self-coordination mechanism to emerge the desired firefighting behavior [2]. They explore how self-organized drone swarms can reduce the propagation of wildfires autonomously and collaboratively, without risking human lives. • In [3], the steps taken to design and construct a semiautonomous fire-proof indoor UAV are explained. However, no additional firefighting behavior is included in the drone. • In [4], a novel approach to autonomous water spray extinguishing of indoor fires inside a building is presented. The authors also presented a constrained optimal control solution that exploits the variable time mass of the drone [5]. However, the use of water spray as firefighting behavior is very limited to the amount of water that the drone can carry (payload capacity), strongly limiting the extinguishing power of the drone. • A similar approach based on water spray is followed by [6] under outdoor conditions. The novelty of this research is based on its lightweight and low power consumption water collecting/releasing system. • Wang et al. [7] introduces a novel fire drone extinguishing system for high-rise buildings. The system contains a twin-rotor drone among others high-pressure fire extinguishing equipment. They present different firefighting tactics based on drone and fire truck cooperation and multi-drone cooperation. • In [8], the authors proposed an approach to the extinguishing of ground fires with fire blankets deployed by autonomous multirotor UAVs. However, this firefighting method may not be effective on fires inside buildings where the drone can not penetrate the building and where a fire extinguishing ball can be applied. • In the same direction, Martinez et al. presented a team of ground and aerial extinguishing robots [9]. In the work, they proposed two extinguishing behaviors, water spray, and fire blankets. The authors have not mentioned fire extinguishing by ball throwing methods. • In [10], the authors examined the potential use of fire extinguishing balls as part of a proposed system, where drone and remote-sensing technologies are utilized cooperatively. Their research demonstrates the efficiency of fire extinguishing balls. However, their work is limited to wildfires and they have not developed the drone system. • Abdel Ilah N. Alshbatat proposes a fire extinguishing system mounted on a quadrotor [11]. The quadrotor carries a specific payload that is capable of throwing extinguishing balls. The system has been tested on real scenarios with feasible results. However, the throwing methodology consists of hovering the fire point and opening a mechanical claw, which can not be applied in multiple scenarios. As a future work, the author suggests the use of an automatic electric-spring operated gun with a fuzzy-based stabilizer controller to improve the throwing solution. • Barua et al. propose the design and implementation of a prototype of a ball-thrown fire extinguisher quadcopter [12]. Their approach is similar to the previous one, but their drone can carry up to eight fire extinguishing balls instead of one.
Drone firefighting is an emerging topic in drone application research. This field is very wide and includes many related topics that are not covered in this paper. For a general survey addressing unmanned aerial vehicles in firefighting, see [13]. For a recent survey that analyzes potential technologies that could be applied in the future in firefighting missions, see [14].

Challenge description
The main goal of the challenge was to simulate the performance of a UAV intended for firefighting in an urban environment. In these missions, the vehicle must be able to navigate safely in these environments, where it needs to detect possible obstacles, such as buildings, and generate collision-free paths until reaching the affected area. Once there, the UAV looks for the fire in the building and performs a maneuver to throw the extinguishing device, for example, through a window.
The challenge aimed at the competition is a simplified problem of the topic. To successfully extinguish a fire, a UAV must navigate through a dense urban environment, locate the fire, and deploy a fire extinguishing ball precisely in the target area. The challenge then is divided into three sub-challenges that will test the perception capabilities, speed, and agility of the UAVs. The scenario is also divided into three zones, as shown in Fig. 1: Zone 1 is a clear area for taking off, where the drone can be placed anywhere in this area at the beginning of the challenge; Zone 2 is an area populated by obstacles; and Zone 3 is a larger clear area where the target is located. In addition, a difficult aspect such as the localization of the drone is provided by the organization with an external motion capture system. The full description of the challenge can be found in the competition rule book [15].
In the scenarios, the three main tasks must be performed sequentially and completely autonomously, with performance on some tasks contributing to the final score. The scoring equation for each task was provided by the organi- Fig. 1 Virtual scenario divided into three zones: Zone 1, for taking off, Zone 2, for the exploration task, and Zone 3, for the precision delivery task zation, with coefficients calculated to weight each task over a maximum score of 100. The tasks to be performed were: 1. Exploration task. This task began after the takeoff of the drone. The UAV had to start autonomous navigation from the first zone to the third zone, crossing the second zone.
In this second zone, there were an undetermined number of static obstacles in unknown positions, which could lead to dead-end routes, simulating buildings and other urban objects. 2. Target detection task. The detecting fire problem was replaced by the detection of a known AR tag, measuring 19 cm square. It was placed in an unknown position in the third zone, free of obstacles. For this reason, the UAV had to explore the area until it found the tag. For this task, the metric used (Eq. 1) was the distance measured, in meters, between the calculated position and the real AR tag position.

Precision delivery task.
Once the target's position was located, the UAV had to carry out the launch maneuver of the fire extinguishing device, simulated by a ball with a magnetic drive. After that, the UAV had to stabilize, change the control to manual, and proceed to land. The metric applied in this task (Eq. 2) was the minimal distance, in meters, between the ball contact point and the AR tag.
The last scoring metric was the time invested, in seconds, in carrying out the complete mission (see Eq. 3).

Hardware description
The hardware used in the real challenge was a Kopterworx Eagle platform with Ardupilot firmware using a Pixhawk flight controller. Mounts an onboard Intel NUC computer, which used the Mavlink protocol to communicate with the flight controller, and an Intel Realsense D435 camera, with RGB and depth information. Onboard the computer, a Docker with ROS [16] supplied by the organizers was executed, which provided information on the motion capture system and the camera, and applied control signals for the movement of the UAV and the ball launch actuator. The hardware was provided by the organization, so there was no access to it until the competition was held. However, a simulated environment was provided in Gazebo [17], for both the area and the hardware, with its sensors.

Competition Constraints
From the description of the challenge, the constraints for the design of our solution were as follows: • An external localization system provides the state of the UAV, which makes it not necessary to develop a selflocalization algorithm. • Only an RGB-D camera is available to detect and locate obstacles. • The target is an AR tag that should be accurately detected and localized. • There is no launching system for ball deployment, which requires the drone to perform a maneuver to launch the ball onto the target. • Not being able to access the low-level controllers of the aircraft, such as the attitude or speed controller, just position or trajectory controller access was provided. • In the real environment, the computation has to be done on-board. • There was no access to the real drone prior to competition on-site, making the gap between simulation and real-world behavior unknown.

Solution description
The solution developed for this competition is divided into many different modules, each of which contributes to achieving one or more of the three main tasks of the challenge. We renamed them Go Across, Search Target, and Throw Ball, respectively. During the whole mission, only two modules will work continuously: the mission planner and the controller. The rest of the modules will be called as they are needed to complete each different stage. Also, in the context of the competition, we use many parameters in order to adjust the behavior of the system according to the previous knowledge that we have of the environment. Some examples of these parameters are the resolution of the occupancy map or the minimum safety distance to the obstacles for the path planner module; the height and distance to the walls for the goal seeker; and the launch maneuver height and speed or the estimated delay time of the magnet system for the ball thrower. A more detailed explanation of each of these modules will follow.

Mission Planner
The mission planner is in charge of starting the modules needed for each mission task, checking the completion condition of each task, terminating the corresponding modules, and starting the ones needed for the next stage. A flow diagram of the mission planner behavior is shown in Fig. 2.
Several assumptions were taken when designing this mission planner, that simplified the development: • Error handling: The mission was designed in a sequential way, in which each task begins whenever the previous one ends. If the system gets stuck in any of the tasks, like in the search target phase if it is not able to find the target, then the robot will keep trying it until the end. No errors in the execution of the tasks were taken into account. • Success Monitoring: Depending on the task, the progress of it can be monitored precisely or in a more heuristic way, e.g., in the Go across task, the position of the aircraft is known precisely during the entire mission, so it is easy to know when the aircraft reaches the Zone 3. However, the Throw ball task is really hard to track performance, since there is not enough sensing, and no information about if the ball is mounted or if it is released correctly can be retrieved. In this case, this task is just performed in an open-loop fashion, without any feedback on the actual performance. • Smooth transitions: Each task initializing has been tailored to ensure smooth transitions between the different tasks, and no big jumps in the actuation commands are sent to the aircraft.

Controller
As stated in Section 3.3, giving low-level commands to the aircraft, such as attitude or speed commands, was not allowed in this competition. The UAV provided by the organization could only be controlled by the teams using Position or Trajectory commands. We could use these command references for the first two phases of the challenge (Go-Across and Search-Target), but for the launch maneuver, we wanted to have easy control over the vehicle speed. In order to send speed references to the aircraft, we built a separate controller that runs on top of the aircraft position controller. 1 Speed Controller We decided to develop our own speed controller on top of the position controller provided by the organization to obtain more control over the platform to achieve precise and smooth maneuvers. The input of this controller was speed references and the output was the desired references (position and yaw orientation) to the position controller".
The internal position controller of the aircraft consisted of a cascade PID. The principal feature of this controller, on which our controller is based, is the fact that the internal position controller provides a higher speed when the reference is further from the UAV and a lower speed when it is closer. So we can control the speed of the aircraft by modulating the relative distance between the position reference and the current aircraft position.
Considering that in time t, the UAV has position X(t) and velocity V(t) and takes a speed reference V re f (t).
Being C mod (t) the "Carrot" separation that increases or decreases the distance between P(t) and P re f (t). The P re f (t) is the computed position reference that will be sent to the position controller.

Path Planning
This module uses the depth image provided by the RGBD sensor to cross through a zone populated with obstacles. It creates an occupancy map and then an A* algorithm finds the best path to reach the goal point. For a better explanation, it can be split into two different submodules: occupancy map generator and path planner.

Occupancy Map Generator
The obstacles in the competition had a prismatic or cylindrical shape. Since the obstacles are as high as the surrounding walls and maintain a consistent shape along the z-axis, planning can be simplified to a 2D problem.
Laser map The camera's depth image is converted into a 2D laser scan on the horizontal camera plane. Then, knowing the position of the camera relative to the drone, and the position of the drone in the world, the measurements of this single-layer laser are transformed into the fixed world frame. However, the camera plane moves in accordance with the drone's pitch and roll, which can result in measurements of the ground being included as obstacles in the occupancy map. To prevent this issue, any measurements located below a minimum height of 1 meter are disregarded.
Occupancy map Due to the onboard computation, and to increase the search speed of the path planning algorithm, the occupancy map is made at low resolution, being this a parametrized value with typical values of 0.3 or 0.5 meters per pixel, depending on the clearance between obstacles. This reduces the number of cells in the path planner algorithm, making it faster to recalculate the path when new obstacles are found.
The generation of the occupancy map follows the four steps shown in Fig. 3. First, a higher-resolution map is created from the size of the space, with a resolution of 0.1 meters per pixel. This map is called a laser map because it contains the laser scan measurements and is a representation of the surfaces in the environment. In the beginning, both the laser map and the occupancy map are clean, meaning that the space is empty of obstacles. With the measurements of the laser scan, filtering out outliers, e.g., produced by noise in the real-world sensors, each point detected is established as an obstacle surface in the laser map.
The laser map is then converted into a distance field, where the intensity of each pixel depends on its distance from each laser detection. Subsequently, this map is saturated at a security distance of 1 meter, to prevent the drone from approaching obstacles closer to this distance.
Finally, we resize this distance-saturated map to generate the occupancy map using a convolution in which the kernel sums the values inside and checks whether they represent an occupied cell. If the proportion of pixels marked as occupied within the grid is greater than a certain value, such as 0.3, the occupancy map cell is set as occupied.
Filtering To prevent false sensor measurements from disturbing the occupancy map creation, a filtering process has been applied. Before a new cell is occupied on the occupancy map, the value of this cell increases for each new measurement in this cell. Although, every time the cell is considered as not occupied, the value of the cell is reduced by the same amount, with a minimum of 0. Therefore, in order to consider the cell as finally occupied, this cell has to be marked as occupied in the generation of the occupancy map at least 5 consecutive times. This value was determined experimentally, allowing false positives to be filtered out without adding relevant delays in the map creation process.

A* Path Planner
Since we have an occupancy grid, we decided to convert it into a graph in order to use graph-based search algorithms to find the "best" way to navigate the cluttered area (zone 2). Because the map will be generated as the drone flies through the scenario, we need to recompute the path every time the occupancy map has modifications.
In the end, we decided to use an A* path planning algorithm [18] because of its simplicity and fast execution.
Each graph node n(i, j) represents the equivalent cell in the occupancy grid at position i, j. Each node can have up to 8 connections (one per neighbor cell) and a cost function f (n).
In the A * algorithm, the cost function f (n) is compounded by two parts, an accumulative one c(n) that tracks the cumulative cost until reaching the node n and a heuristic one h(n) that "predicts" how much will cost reaching the goal from the node n.
Considering a node n and a goal node G, the cost function we used was: h(n) = (n, G) 2 = (x n − x G ) 2 , (y n − y G ) 2 (8) Fig. 3 Occupancy map creation from laser measurements. The process can be resumed as: raw additive laser measurements (a), distance field (b), distance umbralization (c), and downsampling (d) being α = 1 a constant factor that takes into account the cost of moving to another node, and n par the parent node of n in the graph. To avoid taking into account the full occupancy grid in the search, the graph is created dynamically as long as it is required for the A* algorithm. The initial node I corresponds to the position of the drone within the occupancy map, and the goal node is the center of the beginning of Zone 3.
Whenever the best path is found, control commands must be sent to the UAV. For this, we consider 2 options: speedbased and position-based control. In both cases, the yaw of the drone is set so that the camera always faces in the direction of movement, to avoid colliding with an unmapped object.
Position references Once the path is generated, there is an "optimization" process to reduce the number of waypoints sent to the controller. The position sent to the controller is the position of the next node to be visited in the path, when the UAV enters the new node, the position reference changes to the next node, and so on. When there are more than two points in the same direction, vertical or horizontal, the intermediate points are not sent to the controller. This makes the movement of the drone more fluent, preventing the drone from making small steps along a straight line. Figure 4 shows how the waypoints sent to the drone on a large straight path are reduced to the start and end point of the straight path.
Speed references In order to smooth the control of the drone, we can use the speed controller. In this case, we compute the vector v that points to the next node in the path. Subsequently, we normalize it, v norm , and scale it to the desired cruise speed, V nav , with the result that v re f = V nav v norm is the reference sent to the speed controller.

Target Localizer
During the competition, the target was represented by an AR tag, as shown in Fig. 5. Fiducial markers such as AR tags are designed to be easy to detect and identify. Also, knowing the size of the marker and the intrinsic parameters of the camera, it is quite simple to estimate the position of the marker relative to the camera. Again, like with the position of the obstacle, the location of the target is transformed into the world reference frame.
For the competition, as the boundaries of the area were known, and the AR tag position couldn't be out of them, we filtered the estimation of the target position to not exceed these limits.

Target Seeker
This module considers two different phases: search and inspection. During the search, this module sends a set of positions to the UAV, touring the search area at a sufficient distance d search from the walls to sweep the entire surface  The walls of the scenario were decorated with different colors, which made detection challenging with the camera on each pass and be able to locate the AR tag. This theoretical distance is calculated as follows: where h wall is the height of the wall, F OV v the vertical field of view of the camera. With 42.5º of vertical field of view of the Intel Realsense D435 camera, the theoretical search distance in the simulation was 5.80 meters. Although, at this distance, the AR tag detector has problems finding the tag. Therefore, we decided to reduce this distance to 3 meters. Due to the fact that the drone bends in the roll while facing the wall, and the diagonal FOV of the D435 is 77º, with a roll degree close to 45º the color camera could cover a vertical wall of 4.80 meters. Even if the roll degree is not 45°, at this distance, we can expect to be close to covering the full height of the wall and, more importantly, still be able to detect the label with confidence. In the real environment, since the target height was already known (1 meter), this parameter was not as critical.
Depending on which side of the arena the UAV reached zone 3, it can start the search in one order (left to right) or inverse. The search continues in one direction and the reverse direction until the target is identified. The target seeker strategy is explained in Fig. 6.
Once the target is detected, the inspection phase begins, with the objective of obtaining a precise location of the target. At this moment, the module sends the UAV to the inspection waypoint. This waypoint is located at a certain distance, d inspection = 4m, in front of the target localization detected, Fig. 6 Target seeker scheme with waypoints (green) and drone trajectory (blue) and a certain height above it, h inspection = 2m. The drone waits at this point, looking to the tag until the difference between consecutive measurements of the target position is below a threshold, th inspection = 5mm. At that time, the module sends the target position to the mission planner.

Ball Thrower
One of the main difficulties of throwing the ball was the inability to give low-level commands to the aircraft, which discouraged us from computing complex throwing maneuvers. Taking advantage of our speed controller, we decided to use constant-speed maneuvers to throw the ball.
Since we have some fixed constraints, such as the ball and target size, we designed a simple maneuver that allows us to modify these constraints with ease. The maneuver can be divided into 3 stages that will be explained below.
Initial Positioning After localizing the target, the drone is positioned in front of it at a certain distance from the target d launch = 6m and a certain height z launch = 1.5m above it. This allows the drone to reach the desired constant velocity before the aircraft reaches the launch point.
Constant Speed approach From this point, we consider that the full maneuver takes place in a 2D plane X-Z with the X axis aligned with the direction that heads the target from the UAV. With this constraint, the objective is to fly at a certain height z launch from the objective at a constant speed v launch = 3.5m towards the X axis. During this trajectory, the dynamics of the falling object will be constantly evaluated to check if it is the right time to throw the ball to reach the target, see Fig. 7.
Considering the simplified horizontal launch, the predicted coordinates of the ball along time after the release will be described as: Fig. 7 Ball throwing maneuver scheme being g = 9.81 m/s 2 the gravity constant, [x 0 , z 0 ] the position of the ball when the ball is released. We decided to opt for a constant-speed maneuver to assume that the ball speed [ẋ 0 ,ż 0 ] is the same as that of the UAV at the instant of the throw.
Our objective is to release the ball at the precise moment when the predicted coordinates of the ball [x(t cont ), z(t cont )] match the target coordinates [x target , z target ] at a concrete time t cont .
From Eqs. 10 and 11 we can deduce that so when x(t cont ) = x target the ball should be released.
In order to make this approach more realistic, we should relax this constraint a little by modifying the last trigger condition by adding a precision coefficient δ acc and considering a possible delay in the throwing physical mechanism t delay . When these are joined, the final trigger condition ends as follows: When this condition is satisfied, the ball is released. Since we do not control the UAV directly, this speed could not be as stable as it is desirable, so we could not just trust that the UAV speed or position will be planned. Hence, we recompute continuously to handle these uncertainties if the trigger condition is satisfied.
Back to safety Finally, the UAV performs a return maneuver by temporarily flying upward and returning to a safe point away from the target to avoid colliding with the wall. This safe point is defined by a parameter, and within the competition, it was a point near the center of Zone 3.

Experiments
As explained in Section 3, the competition was divided into two different phases: the classification phase, in a virtual environment, and the final phase, in a real environment.

Virtual Environment
The virtual scenario used during the first phase was developed on Gazebo [17] by the organization. There were several possible scenario configurations. In each configuration, the number and distribution of obstacles are different. An example of one of these configurations is shown in Fig. 8.
The Gazebo simulation area was 25 meters long and 15 meters wide, divided into three zones of 4.5, 9, and 11.5 meters deep, respectively, surrounded by a wall of 4.5 meters high, as shown in Fig. 1. The diameter of the ball used was 0.4m.
In each experiment, the UAV spawned at an unknown point in Zone 1, and the tag was placed in an unknown position on one of the three walls surrounding Zone 3. The obstacles to avoid were prismatic, square-based, and 4.5 meters high. Colliding with these obstacles meant the end of the trial.

Real Environment
Real environment experiments were carried out in a smaller area, 8 meters long and 5 meters wide, dividing the three zones of 1.5, 0.5, and 8 meters, respectively, and with a maximum height of 3.5 meters, as can be seen in Fig. 10.
For safety reasons, the obstacles in the real experiments were smaller than in the simulation. Although the UAV was prohibited from flying within 1 meter of any obstacle, it was monitored by the external positioning system capable of stopping the drone if this restriction was violated, which meant the end of the trial. Thus, although the distance between obstacles seemed wide, the clearance to pass was narrow.
Moreover, to avoid complex maneuvers that could end in a drone collision, the AR tag was placed at 45º to the ground  Regarding the differences between the UAV and the simulation model, it behaved quite similarly. However, there were some differences. The real electromagnet in the UAV was not instantaneous as in the simulation, but there was a time delay between the release order and the real ball release. Furthermore, the diameter of the ball used was 65cm, attached to a steel plate that allowed it to be attached to the drone's magnet, as shown in Fig. 9.
The finals consisted of two different scenarios with two different obstacle setups, Figs. 10 and 11. In each scenario, the target platform was situated in different unknown positions.

Classification: Simulation environment
For this phase, we show the metrics obtained in the evaluation performed by the organizers. Since the first evaluations were done to check that our solution worked correctly in the evaluation test, we have chosen the last 10 evaluations. The results are shown in Tables 1 and 2.  The metrics show that the localization of the target was pretty accurate, with an average error of 0.08 meters, less than the half size of the AR tag. Also, this error is similarly distributed in axis y (horizontal) and z (vertical). There was no error in the x-axis (depth) because of our previous knowledge that the AR tag will be on a wall. Regarding throwing accuracy, the average error is 0.17 meters, almost the size of the AR tag. Considering the diameter of the ball is 0.4 meters, that means the ball usually hits close to the target. Figure 12 shows the trajectory of both the drone and the ball during one trial in simulation, and the distance between the tag position, the target localization, and the ball contact point.
With these results, we were ranked first out of the 15 international teams that participated in this phase. 2

Finals: Real environment
The 5 best teams from the previous phase qualified for the finals. The goal was an AR tag whose size was 190 x 190 mm, and it was printed on A3 size paper (297 x 420 mm), as represented in Fig. 13. Finally, this paper is situated in a structure with a 45-degree face bend with respect to the vertical. This face is about twice the size of the paper. So, to measure the accuracy of the drone throwing the ball, we assume that the maximum distance to the target is half of the diagonal of the surface: • Hit the AR-Tag (Goal): 0.13 m • Hit the paper (White surface): 0.26 m • Hit the structure (Black surface): 0.51 m As it is not possible for us to determine where the ball landed when it did not hit any surface, these cases have not been taken into account to calculate the accuracy of the ball throwing.
Differences with the simulation environment made us modify multiple parameters. The map size was adapted to the competition area and the limits of the search path were reduced. Due to the reduced space, the parameters related to the drone maneuvers were also decreased: the inspection of the target is performed closer, d inspection = 2m and h inspection = 1m, and also the ball launch maneuver, d launch = 3m, z launch = 1.5m and v launch = 2m. Since the detection of the tag in the real world is less accurate than in the simulation, the threshold for calculating the location of the tag was increased, th inspection = 0, 1m. For the obstacle avoidance task, we limited the max speed of the drone to 1m/s. This was faster than the rest of the teams, with an average mission completion time of 20 seconds, almost half the time of the second-fastest team. As there are no metrics available for the rest of the teams, it is not possible to make an accurate comparison.
As mentioned in Section 5, two different scenarios were set up. Table 3 shows time metrics obtained in both scenarios, while distance errors are shown in Table 4.
During the second scenario, as a result of being the last team scheduled, we had some issues with magnet overheating. This means that sometimes the ball was released later than planned, which caused the ball to not follow the calculated trajectory. To compensate for this effect, we had to adjust the behavior of the drone the modifying the launcher parameters to launch the ball closer, d launch = 1m and z launch = 1m, and tuning the t delay parameter depending of the response of the magnet. In a few trials, we achieved a large increase in launch accuracy, hitting the target surface on most attempts. After hitting the target for the third time, the organization decided to stop our shift since no other team had succeeded three or more times and we were the last team scheduled.

Conclusions and Future Work
We developed an architecture that performed well in a simulated urban environment. It has been tested in ICUAS'22 first international UAV competition, qualifying for the finals with the highest score in the classification phase and winning the real environment challenge with 3 goals, more than any  other team, and also being the fastest solution, completing the mission with an average time of 20 seconds. 3 Our solution achieves great precision in the throwing maneuver, with a mean minimum distance between the ball and the target of 0.17 meters in simulation, and being able to consistently throw the ball with a real drone, achieving hitting a target of 19x19 mm repeatedly.
The solution designed is composed of a set of modules in charge of different tasks, and above all of them, a mission planner that manages them to complete the mission. Furthermore, to avoid restrictions on the control modes of the platform provided for the competition, a speed controller has been created that translates the desired velocity into position references.
Due to the modularity of this architecture, it is designed to be able to upgrade, change, or add the necessary modules to be used in a more realistic mission than the one faced in the competition. This could make this framework a good starting point for addressing the application of UAVs to urban firefighting, or similar.
The development of the trajectory generation algorithm for ball throwing has been conditioned by the lack of access to low-level control of UAV movement. Future work should develop more accurate algorithms to perform this task with full UAV control capabilities.
To bring the problem closer to real applications, it is necessary to replace the external positioning device with the development of a self-localization system, adapted to urban environments in outdoor flights. In the same line of research, the simplification of the AR tag must be replaced by visual detection of real fires, as well as determining the most effective point to release the fire extinguishing device. In addition, adding an exception handler to the mission planner, e.g., if the route planner fails or if the ball is not thrown, will make the system more robust.
Finally, another research path would be to compare the algorithms developed with the performance of a Neural Network trained through the use of Imitation Learning or Reinforcement Learning techniques. This is interesting because of the great efficiency of Neural Networks in this type of application and their great capacity for adaptation to uncontrolled environments, such as in emergencies.

Declarations
Ethics approval All of the authors confirm that there are no potential acts of misconduct in this work, and approve of the journal upholding the integrity of the scientific record.

Consent to participate
The authors consent to participate.

Consent for publication
The authors consent to publish.

Competing Interests
The authors have no relevant financial or nonfinancial interests to disclose.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecomm ons.org/licenses/by/4.0/.