# Drone delivery systems: job assignment and dimensioning

**Part of the following topical collections:**

## Abstract

This article studies how to dimension and control at the system level a fleet of autonomous aerial vehicles delivering goods from depots to customers. Customer requests (jobs) arrive according to a space-time stochastic process. We compute a lower bound for the infrastructure expenditure required to achieve a certain expected delivery time. It is shown that job assignment policies can exhibit a tipping point behavior: One vehicle makes the difference between almost optimal delivery time and instability. This phenomenon calls for careful dimensioning of the system. We thus demonstrate the trade-off between financial costs and service quality. We propose a policy that assigns each incoming job to the vehicle that will do the job faster than other ones, seeking to minimize the overall workload in the system in the long term. This policy is scalable with the number of depots and vehicles, performs optimal in low load, and works well up to high loads. Simulations suggest that it stabilizes the system for any load if the number of vehicles per depot is sufficient.

## Keywords

Unmanned aerial vehicles (UAVs) Queuing theory Task allocation Decision making System dimensioning Level of service Investment## 1 Introduction

Small drones, also called unmanned aerial vehicles (UAVs), have successfully made their way to civil applications. They fly routes in an autonomous manner, carry cameras for aerial photography, and transport goods from one place to another. The range of applications is broad, including aerial monitoring of plants and agricultural fields as well as support for first time responders in case of disasters (Andre et al. 2014; Kovacina et al. 2002; Lima et al. 2014; Quaritsch et al. 2008). Delivering goods with a fleet of drones (Grippa 2016; Grippa et al. 2017) becomes an option if classical means of transportation—such as trucks, trains, and planes—are inappropriate. This comes about if roads, railway tracks, or landing facilities do not exist, if weather conditions make it impossible to use them, or if their use is too dangerous or time consuming. A compelling service in this context is the delivery of medicine, vaccinations, or laboratory samples for patients in remote areas and crisis regions. Such a service is also worthwhile in densely populated metropolitan areas when congestion makes roads nearly impassable.

This article investigates drone-based delivery at a system level. The entities of the system are goods, customers, vehicles, and depots. Customers request goods that are stored in depots and delivered by vehicles. Service requests, also denoted as jobs or customer requests, are not known in advance and arrive over time at certain locations according to a space-time stochastic process. We introduce an approach to dimension the system (How many vehicles and depots are needed for a certain area?) and propose and analyze policies for job assignment (How to assign customer requests to vehicles to minimize the expected delivery time?).

If the vehicles are capable of consecutively serving several customers before they return to a depot, e.g., if goods are lightweight and total distances are small, the problem of selecting customer requests to be served falls into the domain of dynamic vehicle routing with stochastic demands, dating back to Bertsimas and Ryzin (1991, 1993a, b). In contrast to this, vehicles in our system serve no more than one customer per trip for capacity reasons, and we use the term *job assignment* to emphasize the difference to routing. Focusing on depot-to-customer delivery, we complement research on routing policies for wide-area surveillance (Bullo et al. 2011; Enright et al. 2009; Frazzoli and Bullo 2004; Pavone et al. 2011; Savla et al. 2008).

We model the system as an *M* / *G* / *K* queue with \({K>1}\). Our performance measure is delivery time, i.e., the time it takes for a customer from requesting to obtaining a good. The stability of this service is linked to the queuing of jobs, i.e., customers may have to wait until other customers have been served. The system becomes unstable if the average number of waiting customers persistently increases over time.

Our contributions are as follows: First, we give a lower bound for the expenditure needed to set up a stable system as a function of the targeted average delivery time. Second, we analyze two simple job assignment policies: *Nearest job first to random vehicle* (NJR) and *first job first to nearest vehicle* (FJN). These policies are suboptimal but their performance yields insight on the system dynamics which helps us to design advanced policies. Third, we analyze a policy known from queuing theory, named *first job to vehicle with the smallest workload* (\({\text {FJW}\pi }\)) in this paper, and modify it according to the lessons learned from NJR and FJN. We propose the *first job first to vehicle with smallest additional workload* (\({\text {FJW}\delta }\)) policy. \({\text {FJW}\pi }\) is not optimal in light load and stabilizes the system only for a fraction of the possible arrival rates. The novel \({\text {FJW}\delta }\) is optimal in light load and stabilizes the system for almost all arrival rates. Fourth, we show that FJN and FJW exhibit a tipping point behavior: One vehicle makes the difference between almost optimal performance and instability. Therefore, careful dimensioning is required. In contrast to this, \({\text {FJW}\delta }\) can stabilize the system for almost all arrival rates, as long as the number of vehicles per depot is sufficient. This finding yields a connection between dimensioning and control of the system. Fifth, this last finding enables proper dimensioning of the delivery system controlled with \({\text {FJW}\delta }\). In particular, the infrastructure of depots is subject of a long-term decision, the number of vehicles can be modified in short-term. The setup of the system (number of depots and vehicles) shapes its financial costs and, in conjunction with the job assignment policy, the service quality provided to the customers for which they are willing to pay. Hence, which minimum expenditure is required to provide a certain service quality is an interesting question with respect to investing in such a delivery system. The lower bound mentioned in the first point provides the answer. We illustrate the application of this service-possibility-frontier with parameters reported by the company Matternet. Parts of this article related to the first, the second, and the fifth points are published in Grippa et al. (2017). The bound in the first point has been improved. All results and discussions related to FJW policies are novel.

## 2 Related work

### 2.1 Types of vehicle routing problems (VRPs)

The framework of the used model dates back to Dantzig and Ramser (1959), who introduced their formulation of the vehicle routing problem (VRP) as a generalization of the traveling salesman problem (Flood 1956). Since then, the operations research community has intensively studied how a central planner determines optimal sets of routes for fleets of homogeneous vehicles, supplying given sets of geographically dispersed customers with goods (Golden and Assad 1998). In the context of a “classical VRP”, such an optimal set of routes accomplishes that (i) all customers are supplied with the requested products, (ii) none of the vehicles exceeds its capacity traveling along its route, (iii) no customer is visited more than once, (iv) all routes start and end at one central depot, and (v) the overall routing cost is minimized. In practical applications, VRPs have a broad diversity of additional requirements and operational constraints affecting the construction of the optimal set of routes. Among these are periodic VRPs (Angelelli and Speranza 2002), VRP with pickup and delivery (Desaulniers et al. 2002), VRP with split deliveries (Dror et al. 1994), and VRP with time windows to serve customers within (Cordeau et al. 2002). For reviews of exact and approximate methods for solving the classical VRP, we refer to Baldacci et al. (2007), Cordeau et al. (2007), Laporte (1992, 2007, 2009), Toth and Vigo (2002a, b) and, for an exhaustive bibliography on vehicle routing, to Laporte and Osman (1995).

VRPs are classified according to the nature of system input information. If all system input is known before the vehicles leave the depot(s) and does not change during mission execution, the matter of concern is described in the paragraph above, and said to be both static and deterministic. For many real-world applications at least some input information like customer arrivals behaves according to a probability distribution rather than is known a priori. These VRPs are denoted as stochastic. If some input information appears or changes during mission execution and has to be integrated immediately into decision-making, the VRP is called dynamic (Psaraftis 1998, 1995, or, for a recent review, Pillac et al. 2013). Then, designing sets of routes has to be replaced by designing routing policies which describe the evolution of motion paths as a function of newly arriving input.

### 2.2 Stochastic and dynamic vehicle routing in robotics and aeronautics

Bertsimas and Ryzin (1991, 1993a, b) were the first to comprehensively analyze a stochastic and dynamic VRP. The problem is also named dynamic traveling repair-person problem (DTRP): Requests are distributed according to a space-temporal stochastic process. Every request is associated with one location. Vehicles travel from request to request and spend time in each location providing on-site service. This problem received considerable attention for applications in robotics and aeronautics. Particular attention was devoted to the motion coordination of mobile robots, which includes spatially-distributed surveillance policies for UAVs that are adaptive to network changes (Bullo et al. 2011; Enright et al. 2009; Frazzoli and Bullo 2004; Pavone et al. 2011; Savla et al. 2008). In some of these policies, named partitioning policies, the service area is partitioned in sub-areas, one for each vehicle, and every vehicle serves only the requests in its sub-area according to certain rules (Pavone et al. 2011). Strategies were developed to ensure that a certain fraction of requests is served before the jobs expire (Pavone et al. 2009), that account for service priorities (Smith et al. 2010) and translating requests (Bopardikar et al. 2010), and that accomplish an effective system management without explicit communication (Arsie et al. 2009).

Considerably less work was dedicated to another vehicle routing problem called dynamic pickup and delivery problem (DPDP) (Swihart and Papastavrou 1999; Waisanen et al. 2008). In this problem, every request is associated with two locations (pickup and delivery) and, at each service, a vehicle transports a good from pickup to delivery location. This problem is different from the DTRP because the service of the request requires the vehicle to change location. Both pickup and delivery locations are drawn according to a continuous spatial distribution, i.e., goods are not stored in fixed depots.

In our problem, goods are available in depots with fixed locations. This problem can be seen either as a DTRP with a non-euclidean distance, or as a DPDP with pickup locations chosen from the depot locations. For these reasons, the approaches developed in related work are not directly applicable to this problem. Furthermore, we explicitly take battery charging into account.

## 3 System model

### 3.1 Entities of a delivery system

The system is composed of \({K\in \mathbb {N}}\) vehicles moving in a bounded and convex service area \({\mathcal {A}\subset \mathbb {R}^2}\) of size \(A{:}{=}\Vert \mathcal {A}\Vert \), where \(\Vert \cdot \Vert \) is the Euclidean norm. A vehicle is denoted by \(v_{k}\) with identifier \({k\in \{1,\ldots ,K\}}\). The current position of \(v_{k}\) at time *t* is \(\mathbf{v _{k}(t)\in \mathcal {A}}\) with \(t \ge 0\). All vehicles travel at the same constant velocity \({\nu \in \mathbb {R}^+}\) and are equipped with a battery, whose charging level at time *t* is represented by \({b_k(t) \in [\,0,1]}\). The fact that batteries have to be recharged or exchanged is quantified by the parameter \(\alpha \in (0,1]\), which is the air-time ratio, i.e., \({\alpha = \text {air time}/(\text {air time + charge time})}\).

The arrival of delivery requests for goods within \(\mathcal {A}\) is generated by a Poisson process with finite intensity \({\lambda \in \mathbb {R}^+ }\), where \(\lambda \) is the arrival rate. The requests, also called jobs, are indexed by the job identifier \({n \in \mathbb {N}}\), which indicates the order of request arrivals. The corresponding customer is called \(c_{n}\); his or her position is denoted by \(\mathbf c _n \in \mathcal {A}\) and assumed to be independently and uniformly distributed in \(\mathcal {A}\). The information about the waiting customers is centrally stored. In distributed policies (NJR), vehicles periodically access the information and select the jobs. In centralized policies (FJN and FJW), a central unit periodically asks the vehicles for information and assigns the jobs to them. We assume that communication happens just before selection/assignment. However, in FJW policies, communication and assignment can be postponed without any effect on the policy outcome (see Sect. 6.1). Customer requests do not only differ with respect to timing and locations but also with respect to the goods requested to be delivered.

### 3.2 Service operations and delivery time

The delivery time for customer \(c_{n}\) (\(n \in \mathbb {N}\)) is denoted by the stochastic variable \({T_n = W_n + R_{n} + S_n}\), where \(W_{n}\) is the waiting time, \(S_{n}\) is the service time, and \(R_{n}\) is the return time. Figure 1 illustrates all operations involved in the service of \(c_{n}\) from arrival at \(t=\tau _0\) to service completion at \(t=\tau _{10}\). The waiting time is \({W_{n}=\tau _5-\tau _0}\). The return time is \({R_{n}=\tau _7-\tau _5}\) and depends on the position of the customer \(c_{n^-}\) having been served before \(c_n\) and on the system’s load. \(R_{n}\) is included in \([0,R'_{n}]\): It is null if the vehicle is ready at the depot, and maximum if the service of \(c_{n^-}\) is not completed at the arrival of \(c_n\). The service time is \({S_{n}=\tau _{10}-\tau _7}\).

In the context of “classic” queuing theory, the term “service time” indicates the total time to process one request \(B_n\), which corresponds to \({S_n + R_n}\) in this paper. Bertsimas and Ryzin (1991, 1993a, b) use service time to indicate the time spent on-site at the request location which corresponds to the term unloading time in this paper.

### 3.3 Queuing phenomena and stability

*N*(

*t*) denotes the number of pending jobs at time

*t*. We assume that \({N(0)=0}\), i.e., no customer is waiting at \(t=0\).

*M*/

*G*/

*K*queue with interdependent service times \(B_n\).

*M*indicates Poisson distributed customer arrivals,

*G*indicates service times distributed according to a generic distribution, and

*K*is the number of servers. For

*M*/

*G*/

*K*queues with independent service times (Kleinrock 1975), the load factor is defined as

## 4 Expenditure for minimum infrastructure

*necessary*for building a stable system that meets a targeted performance \(\bar{T}\). Note that (13) is

*sufficient*to guarantee system stability if the job allocation policy can stabilize the system for all \(\rho <1\).

We show in the following sections that one vehicle can make the difference between almost optimal performance and an unstable system (see Fig. 3). Therefore, choosing a sufficient number of vehicles and depots is crucial for a reliable system. We call this operation system dimensioning. In Sect. 7 we use (13) in a numerical example and discuss the result.

## 5 Simple job assignment policies

### 5.1 Description of policies

Features of job assignment policies

Feature | Feature details | NJR soon | NJR late | FJN soon | FJN late |
---|---|---|---|---|---|

Order of selection | Jobs are selected in FCFS order from a shared queue | – | – | \(\checkmark \) | \(\checkmark \) |

There is no specific assignment order of jobs | \(\checkmark \) | \(\checkmark \) | – | – | |

Decision location | Decisions at the customer, just after completing a service | \(\checkmark \) | – | \(\checkmark \) | – |

Decisions at the depot, just before loading goods | – | \(\checkmark \) | – | \(\checkmark \) | |

Coordination mechanisms | If many vehicles seek to select a job, a random one does | \(\checkmark \) | \(\checkmark \) | – | – |

If many vehicles seek to select a job, the nearest one does | – | – | \(\checkmark \) | \(\checkmark \) | |

If there are no jobs a vehicle goes to the nearest depot | \(\checkmark \) | \(\checkmark \) | \(\checkmark \) | \(\checkmark \) | |

Battery management | If a vehicle’s battery level is below 30% the vehicle approaches the nearest depot. If a vehicle’s battery reaches a charging level higher than 80% the vehicle is ready for service. If there is no customer the vehicle fully recharges its battery | \(\checkmark \) | \(\checkmark \) | \(\checkmark \) | \(\checkmark \) |

Job assignment goes beyond merely picking the next request. It has to specify all decisions needed to operate a delivery system including which customer request to serve first, which vehicle to let serve the next customer request, at which depot to let vehicles load up goods, when to recharge the battery, and where to let vehicles return to if no customers are waiting. We first analyze two simple classes of job assignment policies: nearest job first to random vehicle (NJR) and first job first to nearest vehicle (FJN). NJR policies select jobs based on the location of the customer; FJN policies select jobs based on the arrival time of the customer requests. By comparing these two policies, we seek to gain insights into the following question: Is it worth to delay the decision on job assignment to obtain more information about new customer requests in order to reduce the delivery time? For both classes we evaluate two extreme cases: when assignment is made as soon as possible (just after the previous service, NJR-soon and FJN-soon), and when assignment is made as late as possible (just before loading the good, NJR-late and FJN-late).

A first issue is *the timing of job assignments*. It seems plausible to assume that decisions should be made as late as possible to utilize the most recent information about the system status, basically postponing the individual job assignments until loading goods in the depot. Such delayed decisions can also have a negative effect. This relates to the fact that, for \(L\ge 2\), one of the most important decisions to make is where a vehicle should return to after satisfying a customer request. If a non-postponed decision about the next job includes a “clever” assignment of the depot to travel to, while a postponed decision leads to being at a suboptimal place at the moment of job assignment, the advantage gained by processing more recent information is counteracted by the disadvantage of the extra distance traveled to the customer to be served next. We will investigate this effect in Sect. 5.4.

A second issue is the *coordination mechanism* to avoid that more than one vehicle is assigned to the same job. Such a scenario, where a particular customer, say \(c_{\hat{n}}\), is selected by \(K'\) vehicles with \({2 \le K' \le K}\), can easily happen if the system is not fully utilized, i.e., for \({\rho \rightarrow 0}\). FJN policies are centralized: A central entity selects the next job and assigns it to the nearest vehicle. Hence, the assignment is based on arrival time and localization of waiting customers, positions of depots and vehicles. NJR policies are distributed: Every vehicle selects jobs based on its position with respect to the position of customers and depots. Whenever a vehicle selects a job, it is removed from the list of waiting jobs. Every vehicle knows its own position, the positions of depots and of waiting customers. For these policies, the coordination mechanism is randomized: If more than one vehicle is ready to take a new job at a given time, a random priority order is assigned to the vehicles involved. The randomize mechanism corresponds to a real case where vehicles are not coordinated at all and the connection delay to the list of waiting customers is not predictable because it depends on the network conditions.

Table 1 shows all simple policies and classifies them according to three features: the order of jobs to be done, the timing of assignment, and the coordination of jobs in case of ambiguous assignments. A feature included in all policies is the effect of a vehicle \(v_{k}\)’s battery level, \({b_k(t)}\), on the selection of new jobs. In particular, if at the time of job selection/assignment the battery level is below \(30\%\), the vehicle approaches the nearest depot to recharge and does not select a job until its battery level has reached \(80\%\). If no service request arrives, \(v_{k}\) fully recharges.

#### 5.1.1 NJR policies

#### 5.1.2 FJN policies

*n*with \({c_{n} \in \mathcal {N} (t)}\), where

*t*indicates the instance of the job assignment. Let this request be called \(\underline{n}(t)\). In particular, policy FJN-soon implies that whenever at least one vehicle completes a service at time \({t=\tau }\) a central controller selects the next customer in line, i.e., request \(\underline{n}(\tau )\), and chooses the vehicle \(v_{k^* }\) and the path through the depot, \(d_{l^*}\), that minimizes the total distance between its current position \(\mathbf v _{k}(\tau )\), a depot’s position \(\mathbf d _l\), and the customer’s position \(\mathbf c _{\underline{n}(\tau )}\):

### 5.2 Simulation setup

For each of the simple policies, we simulate the movement of \({K\in \{1,\ldots ,24\}}\) vehicles in a square area of \({A=16 \text{ km }^2}\). The \({L\in \{1,4,9,16\}}\) depots are located such that the average distance of any potential request in \(\mathcal {A}\) from the nearest depot is minimized. Customer requests are assumed to randomly arrive over time at a constant rate \({\lambda =0.65\, \text{ requests/min }}\). All vehicles travel at a constant velocity of \(\nu =30 \text{ km/h }\), neglecting acceleration and deceleration phases and neglecting the extra time for starting and landing. The vehicles’ air time is limited due to their finite battery capacity, which is assumed to be full at \(t=0\). We choose the ratio between air time and charge time as being 1 / 3, i.e., \({\alpha =0.25}\). Moreover, every \(v_{k}\) is placed at one of the depots at \(t=0\). Finally, we neglect the loading time of goods.

In the simulations, the system presented in Sect. 3 is observed every \(\delta \) time units. Interarrival times are generated according to a geometric distribution so that the number of customer arrivals per time unit follows a binomial distribution with parameters \(h=1/\delta \) and \(p=\lambda \delta \). The Poisson distribution is a sufficient approximation of the binomial distribution if \(p \le 0.08\) and \({h\ge 1500 p}\) (Bronshtein et al. 2007). Therefore, for all of our simulation runs, we choose \(\delta \) such that \({\delta \le \min \{0.08/\lambda ,1/\sqrt{1500\lambda }\}}\).

The data collected from the simulations are used to compute the average delivery time. Specifically, we firstly estimate the length of the warm-up phase using Welch’s method (1983). Secondly, we compute the average delivery time with the replication/deletion method (Law 2007), which is equivalent to the independent replications method (Welch 1983). These methods are used for statistical analysis of data and enable us to estimate expected values and confidence intervals.

For every parameter setup, we perform 10 simulations with a minimum of 4000 jobs for every simulation. If it is impossible to evaluate whether the system reaches the steady state within this period, we simulate 10, 000 customers requests which is more than 10 days of continuous operation.

### 5.3 Warm-up phase

Moreover, Fig. 2 introduces a result addressed when discussing Fig. 3d below: FJN-soon can yield a substantially better system performance than NJR-soon. However, FJN-soon cannot stabilize the system for \(K=11\).

### 5.4 Performance evaluation and lesson learned

Figure 3 shows the average delivery time \(\bar{T}\) for different job assignment policies as a function of the number of vehicles \(K\) and the number of depots \({L}\). The shaded areas indicate “impossible regions”, i.e., all ordered pairs \((K,\bar{T})\) for which either \(\rho > 1\) or \(\bar{T}<\bar{T}_\mathrm {min}\) or both. The following basic phenomena can be observed.

#### 5.4.1 Coordination mechanism and low load

FJN policies are optimal in light load, i.e., the best vehicle to serve a request (in order to minimize the delivery time) is the vehicle nearest to that request. If the system is in light load, increasing the number of vehicles \(K\) has no effect on the performance. Additional vehicles are not used because, on average, there is a vehicle ready to serve every new request from the nearest depot. As the load factor increases, FJN policies become unstable for arrival rates \(\lambda \) rather far from the theoretical limit. This is probably due to the non-optimal battery management. In high load, vehicles spend a significant amount of time recharging the battery while the customers are waiting. Therefore, a good policy has to include the battery level in the job allocation decision.

The randomized coordination mechanism of NJR policies performs poorly in light load. Even if there are vehicles available a the depot nearest to a new request, the serving vehicle might come from another depot. This increases the minimum average service time and dramatically degrades performance.

#### 5.4.2 Tipping point behavior

As the number of vehicles decreases, FJN policies show a tipping point behavior: One vehicle makes the difference between almost optimal performance and instability. In FJN policies, as the number of vehicles decreases beyond a certain threshold, the vehicles often have to change depot. Such depot switching increases the minimum service time making the system unstable. The phenomenon is made worse by the non-optimal battery management. In terms of queuing theory, this extreme behavior is explained by the fact that the delivery time grows much faster with the parameters defining the load factor (\(\lambda \), \(\bar{B}\), \(\alpha \), \(K\)) than in “classic” queues. Bertsimas and van Ryzin report the same for waiting time and arrival rate (Bertsimas and Ryzin 1991). In a “classic” queue, a decrement in the number of vehicles \(K\) would increase the load factor according to \({\lambda \bar{B}/(\alpha (K-\Delta K))}\). In our problem, the average time to process a request \(\bar{B}\) increases as the number of vehicles decreases beyond a certain threshold. Therefore, the load factor increases according to \({\lambda (\bar{B}+\Delta B)/(\alpha (K-\Delta K))}\), which determines higher increment on the delivery time. Given this relationship between delivery time and number of vehicles and the fact that the number of vehicles is discrete, one vehicle can make the difference between being in mid/light load and exceeding the stability region. The difference in stability of FJN and NJR policies derives from the differing ability to keep \(\bar{B}\) small as the load factor increases.

Despite the random coordination, NJR-soon performs better than the FJN policies in high load. In high load, there are fewer conflicts, and assignment of the nearest job limits the transitions between depot sub-areas (with high probability there are requests in the same sub-area). However, this capacity to stabilize the system for higher arrival rates comes at the price of lack of fairness. Requests are treated with priorities depending on the distance from the depot. Therefore, there is a set of demands that is served with long delivery time.

#### 5.4.3 Timing matters

Using FJN policies, timing has no effect on delivery time and system stability (see Fig. 3). In contrast, using NJR policies, an early decision is beneficial for stability, i.e., NJR-soon is more robust than NJR-late. The positive effect of more recent information outweighs the negative effect of returning to a depot located sub-optimally (on average) for the next job. The gap between the two NJR policies widens with more depots, i.e., the effect of “being at a suboptimal place” is more pronounced in systems with many depots. This happens because in heavy load conditions randomized job coordination is rarely necessary (if at all), and NJR behaves similar to a nearest neighbor policy (as mentioned above) which serves the nearest demand after every service completion.

## 6 Workload-based job assignment

### 6.1 Description of policies

We now study two policies that take into account the workload of vehicles. The first is known from queuing theory (Asmussen 2003); the second is an extension based on the lessons learned from the analysis of simple policies. The workload at time *t* is the remaining time that a vehicle needs to complete all assigned jobs.

The first policy is *first job to the vehicle with the smallest workload* (\(\text{ FJW }\pi \)). It serves jobs in an FCFS order and assigns them to the vehicle with the smallest workload. At every job arrival, a central unit asks each vehicle to compute its current workload. All vehicles send back their computed values and the central unit assigns the job to the vehicle with the minimum value. A vehicle with no workload heads toward the nearest depot to charge its battery. For a standard *G* / *G* / *K* queue, this policy minimizes all moments of the waiting time, number of jobs in the system (queue length), and workloads for all vehicles (Asmussen 2003). In our delivery system, however, the policy performs poorly because it does neither consider the positions of the vehicles relative to the customer and depots nor the energy needed to do the job.

The second policy is *first job to the vehicle with the smallest additional workload* (\(\text{ FJW }\delta \)). Instead of using the current workload of a vehicle as an assignment criterion, this policy employs the amount of workload that the new job adds to the current workload. The key idea behind this policy is to minimize the overall workload in the entire system in the long run. \(\text{ FJW }\delta \) considerably increases stability and reduces delivery time. The additional workload takes into account the way from the previous customer to the depot in order to pick up the new good (or the current location if the vehicle is not serving), battery charging in the depot (to reach the new customer and come back to the nearest depot), and transport the good from the depot to the customer. If a vehicle is currently not serving a customer, the current location is taken instead of the customer location.

It should be emphasized that the policy is scalable with the number of vehicles and depots: Each vehicle computes the minimum of \(L\) additional workload values to choose the depot for every assignment, and the central unit computes the minimum of \(K\) additional workload values to choose the vehicle. The policy can thus be applied in large systems. Alternatively, it can be applied locally by dividing the multi-depot system in many single-depot systems.

Some design choices of \(\text{ FJW }\delta \) can be traced back to the lessons learned from the simple policies: First, we focus on FCFS to have fairness among requests with varying distances from the nearest depot. Second, we explicitly take into account the battery level in job assignments. Third, assignments can be postponed without impact on performance until the workload of the chosen vehicle is zero. This is important from an implementation point of view as the system can be well operated in areas with poor network coverage. Vehicles can exchange information with the central unit whenever they are intermittently connected. Note that good assignments are especially important in systems with high load; since vehicles in high load have on average a long time to reach a workload of zero, the communication can in fact be postponed over a longer period. In this way, \(\text{ FJW }\delta \) combines the best of two NJR policies: NJR-soon has good performance but strict connectivity requirements; NJR-late has relaxed connectivity requirements but bad performance.

### 6.2 Performance evaluation

Figure 4 shows the expected delivery time over the job arrival rate for FJW policies. \(\text{ FJW }\pi \) is suboptimal in low load (similar to NJR) and cannot stabilize the system for high load. This is because the policy assigns the job to the vehicle that will be ready for service first but which might be far away from the customer. In contrast, \(\text{ FJW }\delta \) is optimal in low load and performs very well in high load. It is optimal in low load because the chosen vehicle typically comes directly from the depot nearest to the customer it will serve.

A necessary condition to have an optimal policy is that it performs at least as good on a global level as on the local level. This is true in our case: \(\text{ FJW }\delta \) applied at local level performs worse than at global level for mid load. The reason is as follows: In mid load, there is a benefit in exchanging vehicles between different depots, so that vehicles from depots with few jobs can help in depots with many jobs. As the load increases, all depots have many jobs, thus vehicles tend to stay at their depot.

Results of Fig. 5 are obtained as follows: For every given combination of \((L,K)\), we find the stability threshold \(\rho _\mathrm{stable}\) by visual inspection of \(\bar{T}(\lambda )\)-plots. Each point on the \(\bar{T}(\lambda )\)-plot is obtained by simulating the system ten times over 100,000 jobs. In all scenarios, depots are located to minimize the \(L\)-median function.

A comparison of \(\text{ FJW }\delta \) with the simple policies NJR and FJN confirms these findings (see Fig. 3). \(\text{ FJW }\delta \) outperforms both in all cases except in the configuration \((K,L)=(7,16)\), where NJR-soon seems to have a lower average delivery time. \(\text{ FJW }\delta \) is still preferable because of fairness issues with NJR-soon. Even though \(\text{ FJW }\delta \) works very well, when looking at the system from the dimensioning point of view, the tipping point behavior cannot be eliminated. Therefore, a reliable system inevitably calls for careful dimensioning.

## 7 Dimensioning

*I*for depots and vehicles to average delivery time \(\bar{T}\). The purpose of such a plot is to provide decision making support for companies that set up an airborne delivery system equipped with small UAVs.

To give an example, Fig. 6 is produced on the assumption that the cost of a UAV suitable to deliver two-kilogram packages is 1000 US$ plus a maintenance cost of 100 US$ per annum, and the cost of a depot is 15, 000 US$ plus a maintenance cost of 500 US$ per annum. Operating the system over ten years, the costs per vehicle and depots are \(C_v=2000\) US$ and \(C_d=20{,}000\) US$, respectively. These parameter values are those assumed by the company Matternet (Raptopoulos 2012).

A lower bound on the expenditure required to build a stable system, denoted by \(I_\text {min}\), is derived in Sect. 4 and is given by (13). Figure 6 plots this bound for the given parameters, where all values below the bound are shown as a shaded area. The bound has a “staircase” shape with tread levels being the minimum average delivery time achievable with a particular number of depots. No operable system exists for parameters in this area, while every combination of *I* and targeted \(\bar{T}\) located above fulfills \({\rho <1}\) and \({\bar{T}> \bar{T}_\mathrm {min}}\). The bound corresponds to a *service possibility frontier*; it gives a necessary but not sufficient condition for infrastructure expenditure. The actual performance is policy dependent, and more financial resources than \(I_\text {min}\) may be needed to operate the system in a stable manner and to meet the targeted performance. The performance of the FJW\(\delta \) policy is plotted for scenarios with a varying number of depots.

A company that wants to operate a delivery service and serve a customer within a certain average delivery time can employ such a diagram as follows: If the average delivery time should be no more than \(\tau \), the company has to look for feasible combinations of infrastructure and stabilizing policies, i.e., squares and triangles, that are located as close as possible to the origin and below the \((\bar{T}=\tau )\)-line. If the customers’ willingness to pay for several levels of service quality is given, it is possible to quantify the company’s marginal revenues of increasing performance. From Fig. 6 we know the marginal cost of decreasing delivery time. Then we are able to determine the infrastructure that maximizes the company’s profit. For the parameters of Matternet we find that a system with \(L=1\) depot can serve a customer in about 3 min on average in an area of \(\text{16 } \text{ km }^2\), which comes at a cost of 60,000 US$. This time can be reduced to less than 1.5 min if the company spends more than 100,000 US$ for infrastructure (associated with \(L=4\) depots). If the company’s marginal revenue of reducing delivery time by 1.5 min (50 %) is larger than approximately 40,000 US$, the four-depot configuration is better than the one-depot configuration. In other words, Figure 6 informs about the financial resources required for achieving a certain quality of service with a certain policy and about the volume of additional financial resources obligatory to “buy” a shorter delivery time.

## 8 Conclusions

This article addresses the high-level control and dimensioning of a drone-based delivery system using simulations and queuing theory. It was found that job assignment policies can experience a tipping point behavior: A stable system could immediately become unstable if one vehicle fails. An advanced job assignment policy is proposed that uses the increment in workload as assignment metric: The job is assigned to the vehicle that will do the job faster than other vehicles. This policy, called *first job first to vehicle with the smallest additional workload*, leads to an optimal average delivery time for low loads and works very well up to high loads. It is scalable with the number of depots and vehicles. Simulation results indicate that the policy stabilizes the system for all loads if the number of vehicles per depot is sufficient. To account for the tipping point behavior we show how to dimension a stable delivery system for resolving the trade-off between expenditure and service quality. The dimensioning considers two time horizons: long-term decisions on the number of depots to deploy in the service area and short-term decisions on the number of vehicles to use. Future work will analyze systems with inhomogeneous customers and real-world data.

## Notes

### Acknowledgements

Open access funding provided by University of Klagenfurt. The authors would like to thank Kornelia Lienbacher for proofreading.

## References

- Andre, T., Hummel, K. A., Schoellig, A. P., Yanmaz, E., Asadpour, M., Bettstetter, C., et al. (2014). Application-driven design of aerial communication networks.
*IEEE Communications Magazine*,*52*(5), 129–137.Google Scholar - Angelelli, E., & Speranza, M. G. (2002). A periodic vehicle routing problem with intermediate facilities.
*European Journal of Operational Research*,*137*, 233–247.zbMATHGoogle Scholar - Arsie, A., Savla, K., & Frazzoli, E. (2009). Efficient routing algorithms for multiple vehicles with no explicit communication.
*IEEE Transactions on Automatic Control*,*54*(10), 2302–2317.MathSciNetzbMATHGoogle Scholar - Asmussen, S. (2003).
*Applied probability and queues*(2nd ed.). Berlin: Springer.zbMATHGoogle Scholar - Baldacci, R., Toth, P., & Vigo, D. (2007). Recent advances in vehicle routing exact algorithms.
*4OR*,*5*(4), 269–298.MathSciNetzbMATHGoogle Scholar - Bertsimas, D. J., & van Ryzin, G. J. (1991). A stochastic and dynamic vehicle routing problem in the Euclidean plane.
*Operations Research*,*39*(4), 601–615.zbMATHGoogle Scholar - Bertsimas, D. J., & van Ryzin, G. J. (1993a). Stochastic and dynamic vehicle routing in the Euclidean plane with multiple capacitated vehicles.
*Operations Research*,*41*(1), 60–76.MathSciNetzbMATHGoogle Scholar - Bertsimas, D. J., & van Ryzin, G. J. (1993b). Stochastic and dynamic vehicle routing with general demand and interarrival time distributions.
*Advances in Applied Probability*,*25*, 947–978.MathSciNetzbMATHGoogle Scholar - Bopardikar, S. D., Smith, S. L., Bullo, F., & JaP, Hespanha. (2010). Dynamic vehicle routing for translating demands: Stability analysis and receding-horizon policies.
*IEEE Transactions on Automatic Control*,*55*(11), 2554–2569.MathSciNetzbMATHGoogle Scholar - Bronshtein, I. N., Semendyayev, K. A., Musiol, G., & Mühlig, H. (2007).
*Handbook of mathematics*(5th ed.). Berlin, DE: Springer.zbMATHGoogle Scholar - Bullo, F., Frazzoli, E., Pavone, M., Savla, K., & Smith, S. L. (2011). Dynamic vehicle routing for robotic systems.
*Proceedings of the IEEE*,*99*(9), 1482–1504.Google Scholar - Cordeau, J. F., Desaulniers, G., Desrosiers, J., Solomon, M. M., & Soumis, F. (2002). The VRP with time windows. In
*The vehicle routing problem, society for industrial and applied mathematics*, Philadelphia, PA, Chapter 7, pp. 157–172.Google Scholar - Cordeau, J. F., Laporte, G., Savelsbergh, M. W., & Vigo, D. (2007). Vehicle routing. In C. Barnhart & G. Laporte (Eds.),
*Handbooks in operations research and management science*(1st ed., Vol. 14, pp. 367–428). Amsterdam, NL: Elsevier.Google Scholar - Dantzig, G. B., & Ramser, J. H. (1959). The truck dispatching problem.
*Management Science*,*6*(1), 81–90.MathSciNetzbMATHGoogle Scholar - Desaulniers, G., Desrosiers, J., Erdmann, A., Solomon, M. M., & Soumis, F. (2002). The VRP with pickup and delivery. In
*The vehicle routing problem, society for industrial and applied mathematics*, Philadelphia, PA, Chapter 9, pp. 225–242.Google Scholar - Dror, M., Laporte, G., & Trudeau, P. (1994). Vehicle routing with split deliveries.
*Discrete Applied Mathematics*,*50*(3), 239–254.MathSciNetzbMATHGoogle Scholar - Enright, J. J., Savla, K., Frazzoli, E., & Bullo, F. (2009). Stochastic and dynamic routing problems for multiple uninhabitated aerial vehicles.
*Journal of Guidance, Control, and Dynamics*,*32*(4), 1152–1168.Google Scholar - Flood, M. M. (1956). The traveling-salesman problem.
*Operations Research*,*4*(1), 61–75.MathSciNetGoogle Scholar - Frazzoli, E., & Bullo, F. (2004). Decentralized algorithms for vehicle routing in a stochastic time-varying environment. In
*Proceedings of IEEE conference on decision and control*, Paradise Island, pp. 3357–3363.Google Scholar - Golden, B. L., & Assad, A. A. (Eds.). (1988).
*Vehicle routing: Methods and studies*. Amsterdam: Elsevier.zbMATHGoogle Scholar - Grippa, P. (2016). Decision making in a UAV-based delivery system with impatient customers. In
*Proceedings IEEE/RSJ international conference on intelligent robots and systems (IROS)*, Daejeon, Korea, pp. 5034–5039.Google Scholar - Grippa, P., Behrens, D., Bettstetter, C., & Wall, F. (2017) Job selection in a network of autonomous UAVs for delivery of goods. In
*Proceedings of robotics: Science and systems*, Cambridge, MA.Google Scholar - Kleinrock, L. (1975).
*Queueing systems: Theory*(Vol. 1). New York: Wiley Interscience.zbMATHGoogle Scholar - Kovacina, M. A., Palmer, D., Yang, G., & Vaidyanathan, R. (2002). Multi-agent control algorithms for chemical cloud detection and mapping using unmanned air vehicles. In
*Proceedings of IEEE/RSJ international conference on intelligent robots and systems (IROS)*, Lausanne, CH, pp. 2782–2788.Google Scholar - Laporte, G. (1992). The vehicle routing problem: An overview of exact and approximate algorithms.
*European Journal of Operational Research*,*52*(3), 345–358.MathSciNetzbMATHGoogle Scholar - Laporte, G. (2007). What you should know about the vehicle routing problem.
*Naval Research Logistics*,*54*(8), 811–819.MathSciNetzbMATHGoogle Scholar - Laporte, G. (2009). Fifty years of vehicle routing.
*Transport Science*,*43*(4), 408–416.Google Scholar - Laporte, G., & Osman, I. H. (1995). Routing problems: A bibliography.
*Annals of Operations Research*,*61*(1), 227–262.zbMATHGoogle Scholar - Law, A. M. (2007).
*Simulation modeling and analysis*(4th ed.). Boston, MA: McGraw-Hill.Google Scholar - Lima, P., Floreano, D., Schill, F.S., & Basiri, M. (2014) Audio-based localization system for swarms of micro air vehicles. In
*Proceedings of IEEE international conference on robotics and automation (ICRA)*, Hong Kong, CHN.Google Scholar - Papadimitriou, C. H. (1981). Worst-case and probabilistic analysis of a geometric location problem.
*SIAM Journal on Computing*,*10*(3), 542–557.MathSciNetzbMATHGoogle Scholar - Pavone, M., Bisnik, N., Frazzoli, E., & Isler, V. (2009). A stochastic and dynamic vehicle routing problem with time windows and customer impatience.
*Mobile Networks and Applications*,*14*(3), 350–364.Google Scholar - Pavone, M., Frazzoli, E., & Bullo, F. (2011). Adaptive and distributed algorithms for vehicle routing in a stochastic and dynamic environment.
*IEEE Transactions on Automatic Control*,*56*(6), 1259–1274.MathSciNetzbMATHGoogle Scholar - Pillac, V., Gendreau, M., Guéret, C., & Medaglia, A. L. (2013). A review of dynamic vehicle routing problems.
*European Journal of Operational Research*,*225*(1), 1–11.MathSciNetzbMATHGoogle Scholar - Psaraftis, H. N. (1988). Dynamic vehicle routing problems. In
*Vehicle routing: Methods and studies*, Elsevier, Amsterdam, pp. 223–248.Google Scholar - Psaraftis, H. N. (1995). Dynamic vehicle routing: Status and prospects.
*Annals of Operations Research*,*61*(1), 143–164.zbMATHGoogle Scholar - Quaritsch, M., Stojanovski, E., Bettstetter, C., Friedrich, G., Hellwagner, H., Rinner, B., Hofbaur, M., & Shah, M. (2008). Collaborative microdrones: Applications and research challenges. In
*Proceedings of international conference on autonomic computer and communication systems*, Turin, IT, pp. 38:1–38:7.Google Scholar - Raptopoulos, A. (2012). Physical transport. Talk at Solve for X. https://youtu.be/7B3E4OOuGDk.
- Savla, K., Frazzoli, E., & Bullo, F. (2008). Traveling salesperson problems for the dubins vehicle.
*IEEE Transactions on Automatic Control*,*53*(6), 1378–1391.MathSciNetzbMATHGoogle Scholar - Smith, S. L., Pavone, M., Bullo, F., & Frazzoli, E. (2010). Dynamic vehicle routing with priority classes of stochastic demands.
*SIAM Journal on Control and Optimization*,*48*(5), 3224–3245.MathSciNetzbMATHGoogle Scholar - Swihart, M. R., & Papastavrou, J. D. (1999). A stochastic and dynamic model for the single-vehicle pick-up and delivery problem.
*European Journal of Operational Research*,*114*(3), 447–464.zbMATHGoogle Scholar - Toth, P., & Vigo, D. (2002a). Models, relaxations and exact approaches for the capacitated vehicle routing problem.
*Discrete Applied Mathematics*,*123*(1–3), 487–512.MathSciNetzbMATHGoogle Scholar - Toth, P., & Vigo, D. (Eds.). (2002b).
*The vehicle routing problem*. Philadelphia, PA: Society for Industrial and Applied Mathematics.zbMATHGoogle Scholar - Waisanen, H. A., Shah, D., & Dahleh, M. A. (2008). A dynamic pickup and delivery problem in mobile networks under information constraints.
*IEEE Transactions on Automatic Control*,*53*(6), 1419–1433.MathSciNetzbMATHGoogle Scholar - Welch, P. D. (1983). The statistical analysis of simulation results. In S. S. Lavenberg (Ed.),
*Computer performance modeling handbook*. New York, NY: Academic Press Inc. chap 6.Google Scholar - Zemel, E. (1984). Probabilistic analysis of geometric location problems.
*Annals of Operations Research*,*1*(3), 215–238.zbMATHGoogle Scholar

## Copyright information

**Open Access**This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.