Abstract
With the invention of the COVID19 vaccine, shipping and distributing are crucial in controlling the pandemic. In this paper, we build a meanfield variational problem in a spatial domain, which controls the propagation of pandemics by the optimal transportation strategy of vaccine distribution. Here, we integrate the vaccine distribution into the meanfield SIR model designed in Lee W, Liu S, Tembine H, Li W, Osher S (2020) Controlling propagation of epidemics via meanfield games. arXiv preprint arXiv:2006.01249. Numerical examples demonstrate that the proposed model provides practical strategies for vaccine distribution in a spatial domain.
Introduction
The COVID19 pandemic has affected society significantly. Various actions are taken to mitigate the spread of the infections, such as the travel ban, social distancing, and maskwearing. The recent invention of the vaccine yields breakthroughs in fighting against this infectious disease. According to the recent effectiveness study [17], vaccines including Pfizer, Moderna, and Janssen (J &J) show approximately 66–\(95\%\) efficacy at preventing both mild and severe symptoms of COVID19. Therefore, the deployment of COVID19 vaccines is an urgent and timely task. Many countries have implemented phased distribution plans that prioritize the elderly and healthcare workers getting vaccinated. Meanwhile, the shipping of vaccines is expensive due to the cold chain transportation [30]. An effective distribution strategy is necessary to eliminate infectious diseases and prevent more death.
In this work, we propose a novel meanfield control model based on [28]. We consider two approaches (controls) to control the pandemic: relocation of populations and distribution of vaccines. The first one has been discussed thoroughly in [28], where we address the spatial effect in pandemic modeling by introducing a meanfield control problem into the spatial SIR model. By applying spatial velocity to the classical disease model, the model finds the most optimal strategy to relocate the different populations (susceptible, infected, and recovered), controlling the epidemic’s propagation. We considered several aspects of the vaccine in our model for vaccine distribution, including manufacturing, delivery, and consumption. Our goal is to find an optimal strategy to move the population and distribute vaccines to minimize the total number of infectious, the amount of movement of the people, and the transportation cost of the vaccine with limited vaccine supply. To tackle this question, we ensemble these two controls and propose the following constrained optimization problem:
subject to
and
In our model, different populations are described using \(\rho _i\) (\(i\in \{S, I, R\}\)), representing the susceptible, infectious, and recovered. The term \(\rho _V(x,t)\) describes the density distribution of the vaccine over the spatial domain at location x and time t. The control variables \(v_i\) (\(i\in \{S, I ,R \}\)) create velocity fields over timespace domain that move the corresponding populations. As for vaccines, the control variable \(v_V\) represents the vaccine’s transportation strategy, and the control variable f(t, x) describes how many vaccines are produced at a specific time and location. The optimization objective function G is the sum of terminal costs \({\mathcal {E}}_{final}\) and running costs \({\mathcal {E}}_{running}\). The terminal costs \({\mathcal {E}}_{final}\) represent the goal of our control to achieve at the terminal time, such as minimizing the total number of infectious individuals and maximizing the total number of recovered (immune) persons. The running costs \({\mathcal {E}}_{running}\) include the costs of transportation of vaccines and different classes of the populations, etc. We will discuss more details of cost functionals in Sect. 2.3. As for constraints of our optimization problem, the five partial differential equations of \(\rho _i\), \(v_i\) (\(i\in \{S, I, R, V\}\)) describe the dynamics of the different classes of population and vaccines in terms of densities and velocities. The inequalities of f(t, x) model the limitation of vaccine manufacturing. Vaccines are produced at particular factory locations \(\Omega _{factory}\) with a daily maximal production rate \(f_{max}\). The dynamics of the vaccine density \(\rho _V\) share some similar aspects to the unnormalized optimal transport [27]. Specifically, they both study mass transportation with a source term that creates masses.
We solve the main problem using the algorithm based on the firstorder PrimalDual Hybrid Gradient (PDHG) method [6, 7]. Due to the multiplicative interaction terms, \(\rho _S K*\rho _I\), \(\rho _I K*\rho _S, \rho _V\rho _S\), the optimization problem is based on nonlinear PDE constraints, whereas the PDHG only considers linear constraints. We use the extension of the PDHG [10] that solves nonsmooth optimization problems with nonlinear operators between function spaces. We extend the method utilizing the preconditioning operator from [21] which provides a suitable choice of variable norms to achieve a convergence rate independent of the nonlinear operator. As a result, the algorithm converges to the saddle point locally with step length parameters independent of the finitedifference mesh size; see Sect. 3.1 for details.
Lots of mathematical models have been invented to predict the future of COVID19 epidemics. Recently proposed models take more realworld situations into consideration and tend to be more effective in quantitative forecasting. Specifically, there have been studies on the impact of actions such as lockdown, social distancing, wearing a mask [12, 13, 16]. Datadriven approach and machine learning techniques are also integrated to estimate the parameters for the epidemic better and boost the prediction of the trend of the pandemic model [32, 35]. Meanwhile, optimal control serves as an important tool in pandemic control. They seek the optimal strategy to minimize the total number of infected people while keeping certain costs at a minimum. There are work focused on mitigating the epidemic with limited medical supply, such as ICU capacity [8], face masks [31], and vaccines [19, 22, 24, 29, 37]. In [22], an optimal vaccine distribution strategy is proposed with a limited total amount of vaccines and maximal daily supply. [29] first uses an inverse problem to determine the parameters of the SIR model. Then, it formulates two optimal control problems, with mono and multiobjective, and solves for the optimal strategy of vaccine administration. Other nonpharmaceutical interventions are also considered in the scope of optimal control of epidemics, including social distancing, closing schools, and lockdown [18, 23, 36]. [23] computes the optimal nonpharmaceutical intervention strategy based on an extended SEIR model with the absence of the vaccine. The meanfield control problem can be viewed as a particular type of optimal control applied to an individual in terms of population density.
Meanfield game (control), introduced by [20, 26], describes the deterministic (stochastic) differential games as the number of players tends to infinity, where a given player interacts through the distribution of all players in the statespace. It is a thriving research direction with applications in economics, crowd motion, industrial engineering, and more [3, 14, 25]. Numerical methods are invented to obtain quantitative information of such meanfield game (control) models, especially when the statespace is in high dimensions [1, 2, 5, 34]. Multipopulation meanfield game (control) problems have also drawn lots of attention [4, 9, 15]. This type of problem studies the interactions on two levels: between agents of the same population and between populations. Our model is a multipopulation meanfield control problem with population dynamics described using reactiondiffusion equations adopted from the epidemic model and the controls over the vaccine production and distribution. Therefore, we obtain a novel meanfield control problem.
The rest of the paper is organized as follows. Section 2 proposes a novel multipopulation meanfield control model and explains how population movement and vaccine distribution are integrated into a constrained optimization problem. Section 3 discusses the challenges in numerically solving this meanfield control model, proposes a firstorder primaldual algorithm to solve it, and shows the local convergence of the algorithm. Lastly, in Sect. 4, we present numerical experiments with different model parameter choices and discuss their implications on meanfield controls.
Models
In this section, we review the classical SIR model. Based on it, we formulate the spatial SIR dynamics with vaccine distribution, namely SIRV dynamics. We then introduce a variational problem to control the SIRV dynamics.
Classical SIR model
The SIR epidemic model describes an infectious disease epidemic via an ordinary differential equation system
The population is divided into three classes: susceptible, infected and recovered. While assuming a closed population without births or deaths, the model uses S(t), I(t), and R(t) to represent the number in each compartment at time t. The SIR model has two parameters: \(\beta \) is the effective contact rate of the susceptible individual being infected and \(\gamma \) is the recovery rate of the infected individual. The simplicity of this model allows people to predict an infectious disease epidemic by only estimating a few parameters. However, it has limitations by assuming the population is homogeneousmixing, which means that every individual has an equal probability of diseasecausing contact. As a result, the predictions will lack spatial information and may not help the (local) governments make policies or relocate medical resources. Therefore, we are motivated to study the spatial SIR model. On the other hand, the SIR model does not consider the latent period between when a person is exposed to a disease and when they become infected. This leads to the extension of the SIR model, such as the SEIR model. Our proposed model has a flexible structure and can naturally be generalized to such epidemiological models.
Spatial SIR variational problem with vaccine distribution
In [28], we add the spatial dimension to the S, I, R functions. Let \(\Omega \subset {\mathbb {R}}^d\) be a bounded domain. Consider the following density functions
Here, \(\rho _S\), \(\rho _I\), and \(\rho _R\) represent susceptible, infected, and recovered populations distribution, respectively. We assume \(\rho _i\) for each \(i\in \{S,I,R\}\) moves over a spatial domain \(\Omega \) with a velocity \(v_i\). Here \(v_i, i\in \{S,I,R\}\) are our controls variables. With change of variables \(m_i=\rho _i v_i\), we define the momentum
that govern the corresponding density flows. In the following, instead of using control variables \(v_i\), we replace them with \(\frac{m_i}{\rho _i}\) and regard \(m_i\) as the control variables.
We can describe the flows of the densities by the following continuity equations.
This system of continuity equations describes the flows of three groups of densities while satisfying the SIR model. The nonnegative constants \(\eta _i\) (\(i\in \{S,I,R\}\)) are the coefficients for viscosity terms. These terms can also be understood as noise terms generated by the data. \(K=K(x,y)\) is a symmetric positive definite kernel with \((K*\rho )(x,t) = \int _{\Omega }K(x,y)\rho (y,t)\,dy\). In this model, we consider the Gaussian kernel
The kernel convolution describes the spreading rate of infectious disease over the spatial domain. In addition, we assume the Neumann boundary conditions on \(\partial \Omega \). Since we don’t consider birth or death in our model, the total population is conserved for all time \(t\in [0, T]\), which leads to the following equality
In this paper, we consider the optimization problem for the distribution of vaccines. We add an extra function \(\rho _V:[0,T]\times \Omega \rightarrow [0,\infty )\) which represents the vaccine density in \(\Omega \) at each time \(t\in [0,T]\). The vaccine distribution will be described as the following PDE:
where \(m_V:[T',T)\times \Omega \rightarrow {\mathbb {R}}^d\) is a momentum, \(\theta _2\) represents the utilization rate of vaccines, and \(f:(0,T')\times \Omega \rightarrow [0,\infty )\) represents the production rate of vaccines in \(x \in \Omega \) at \(0<t<T'\). During \(0<t<T'\), the vaccines are produced with a production rate f and used at a rate \(\theta _2 \rho _V\rho _S\). During \(T'\le t < T\), the vaccines are delivered to the area where the susceptible population is located, and they are used at a rate of \(\theta _2 \rho _V \rho _S\). In summary, the first part of the PDE describes vaccines’ production, and the second part describes the delivery of vaccines. For all time \(0<t<T\), the susceptible population is vaccinated if the vaccines are available in the same area. Now, we are ready to introduce the new system of equations for the SIRV model.
In the first and third equations, we add the terms \(  \theta _1 \rho _V \rho _S\) and \( + \theta _1 \rho _V \rho _S\), respectively. The constant \(\theta _1\) represents the vaccine efficiency and \(\theta _1 \rho _V(t,x) \rho _S(t,x)\) represents the vaccinated population at \((t,x) \in (0,T)\times \Omega \). We denote a set \({\mathbb {S}} := \{S,I,R,V\}\) and define a nonlinear operator A as follows
where \({\mathcal {X}}_C:[0,T]\rightarrow {\mathbb {R}}\) is a step function that equals 1 on C and 0 otherwise.
The cost functional
The cost functional we propose in this paper is the extension of [28]. We design the cost functional so that the solution \((\rho _i, m_i)\), \(i\in {\mathbb {S}}\) satisfies the following criteria:

(i)
minimize the transportation cost for moving each population;

(ii)
minimize the total number of infected people and the total number of susceptible people by maximizing the usage of the vaccines at time T;

(iii)
maximize the total number of recovered people at time T;

(iv)
avoid high concentration of population and vaccines at each time \(t\in (0,T)\);

(v)
minimize the amount of vaccines produced during \(t \in (0,T')\);

(vi)
minimize the transportation cost for delivering vaccines during \(t \in (T',T)\).
Item (i) can be described by
for \(i\in \{S,I,R\}\) where
which is convex, lower semicontinuous, and 1homogeneous with respect to \((\rho _i,m_i)\). The parameter \(\alpha _i\) characterizes the cost of moving \(\rho _i\) with velocity \(\frac{m_i}{\rho _i}\). Larger \(\alpha _i\) means it is more expensive to move \(\rho _i\). Note that this function comes from the quadratic kinetic energy. To see this, we use the definition \(m_i = \rho _i v_i\) and plug into formula (2.5):
Item (ii) and (iii) can be described by the terminal costs of the cost functional
where functions \(e:[0, \infty )\rightarrow [0, \infty )\) are convex and lower semicontinuous functions. We also minimize the terminal cost for \(\rho _V\) because maximizing the usage of vaccines is equivalent to minimizing the number of vaccines left at the terminal time T. The total number of the recovered can be maximized by penalizing the density at the terminal time if the value of \(\rho _R(T,x)\) is far away from 1 for \(x\in \Omega \). In this paper, we use a quadratic cost function
where \(a_i\) is some constant.
For Item (iv), the cost functional for the concentration of the total population and vaccines can be represented by
where
for \(u:\Omega \rightarrow [0,\infty )\) and convex and lower semicontinuous functions \(g_P, g_V:[0,\infty )\rightarrow [0,\infty )\). Similar to \(e_i\) (2.6) from Item (ii), we use quadratic functions for \(g_P\) and \(g_V\).
Items (v) and (vi) are criteria specific to the vaccine distribution. From PDE (2.2), the vaccines are produced during \(0<t<T'\) by a function f. We use the similar functional (2.7) to minimize the amount of vaccines produced by f. Thus, we set the functional
where \(g_0:[0,\infty )\rightarrow [0,\infty )\) is a convex and lower semicontinuous function.
The vaccines are delivered during \(T'< t< T\). Similar to the Item (i), we set
where \(F_V\) has the same definition as (2.5).
The total cost functional we consider is then
In the perspective of a control problem, the first term at the righthand side in (2.8) is the terminal cost, while the rest of the terms accounts for the running costs. The quadratic terms in the last line is a \(\lambda \)strongly convex functional. The functional F is \(\lambda \)strongly convex if for any \(u=((\rho _i,m_i)_{i\in {\mathbb {S}}},f)\), F satisfies
where \(\Vert {{\tilde{u}}}  u\Vert _{L^2}^2\) is defined as
and \(\partial F\) denotes the convex subdifferential of F. Since \({\mathcal {E}}_i\), \(F_i\), \({\mathcal {G}}_i\) are convex and lowersemicontinuous, G is \(\lambda \)strongly convex as the sum of convex and \(\lambda \)strongly convex functionals. The strong convexity of G is important as the algorithm of the paper requires the objective cost functional to be strongly convex (Theorem 3.3).
Constraints for vaccine production
In addition to the constraint from (2.3), we adapt the following constraints to reflect the limited vaccination coverage:
where \(\Omega _{factory} \subset \Omega \) indicates the factory area where vaccines are produced and \(f_{max}\) is a nonnegative constant representing the maximum vaccine production rate. In the third inequality, a nonnegative constant \(C_{factory}\) limits the total number of vaccines produced during \(0< T < T'\).
Constraints (2.9) can be imposed by having the following functionals for \({\mathcal {G}}_V\) and \({\mathcal {G}}_0\).
where \(\Omega _{factory} \subset \Omega \) indicates the factory area where vaccines are produced. The functionals \(i_{[\infty , C_{factory}]}\) and \(i_{[\infty , f_{max}]}\) are defined as
where a, b are constants and \(u:\Omega \rightarrow {\mathbb {R}}\) is a function. The function \(i_{\Omega _{factory}}(x)\) is defined as
This function forces \(f(t,x)=0\) if \((t,x)\in (0,T')\times (\Omega \backslash \Omega _{factory})\), thus vaccines are produced only in \(\Omega _{factory}\).
Remark 2.1
The formulation is not limited to SIR epidemic model. For example, we can describe the SIRD (SusceptibleInfectedRecoveredDeceased) epidemic model by adding an extra population \(\rho _D\) for the deceased population with a mortality rate \(\mu \).
Properties
From the definition of the cost functional and constraint (2.3), we have the following minimization problem:
We first define the inner product of vectors of functions in \(L^2\). Given vectors of functions \(u=(u_1(t,x),u_2(t,x),\cdots ,u_k(t,x))\) and \(v=(v_1(t,x),v_2(t,x),\cdots ,v_k(t,x))\) with \(u_i,v_i: [0,T]\times \Omega \rightarrow {\mathbb {R}}\), the \(L^2\) inner product of vectors u and v and \(L^2\) norm of u are defined by
where \((\cdot ,\cdot )_{L^2([0,T]\times \Omega )}\) is a \(L^2\) inner product such that
We introduce dual variables \((\phi _i)_{i\in {\mathbb {S}}}\) for each continuity equation from (2.4). Using the dual variables and the definitions of the inner products, we convert the minimization problem into a saddle point problem.
where \({\mathcal {L}}\) is the Lagrangian functional defined as
For brevity, we denote
We can rewrite the Lagrangian as
where the nonlinear operator A(u) is defined as
As noted in [28], the dual gap, the difference between the primal solution and dual solution, may not be zero because the nonconvex functions \((\rho _S,\rho _I) \mapsto \rho _S K * \rho _I\) and \((\rho _S,\rho _V) \mapsto \rho _S \rho _V\) make the feasible set nonconvex. We circumvent the problem by linearizing the nonlinear operator at a base point \({\bar{u}}\)
In our formulation, the linearized operator \({{\bar{A}}}_{{{\bar{u}}}}(u)\) can be written as follows.
where \({{\bar{u}}} = u = (({{\bar{\rho }}}_i,{{\bar{m}}}_i)_{i\in {\mathbb {S}}},\bar{f})\). We define a linearized Lagrangian as
In the paper [10], the author developed a primaldual algorithm using the linearized Lagrangian (Algorithm (3.5)) and proves that the sequence \((u^{(k)},p^{(k)})^\infty _{k=1}\) from the algorithm converges to the saddle point \((u_*,p_*)\) (in Sect. 3.1, we prove the local convergence to the saddle point given \((u^{(0)},p^{(0)})\) is sufficiently close to the saddle point). By the firstorder optimality conditions (also known as Karush–Kuhn–Tucker (KKT) conditions), the saddle point satisfies
In the next proposition, we present the equations derived from the KKT conditions (2.17).
Proposition 2.2
By KKT conditions, the saddle point \(((\rho _i, m_i, \phi _i)_{i\in {\mathbb {S}}},f)\) of (2.13) satisfies the following equations.
The terms \(\frac{\delta {\mathcal {G}}_P}{\delta \rho }\), \(\frac{\delta {\mathcal {G}}_V}{\delta \rho }\), \(\frac{\delta {\mathcal {G}}_P}{\delta \rho }\), \(\frac{\delta {\mathcal {G}}_0}{\delta f}\), and \(\frac{\delta {\mathcal {E}}_i}{\delta \rho (T,\cdot )}\) are the functional derivatives. In other words, given \(F:{\mathcal {H}}\rightarrow {\mathbb {R}}\) be a smooth functional where \({\mathcal {H}}\) is a separable Hilbert space and \(\rho \in {\mathcal {H}}\), we say a map \(\frac{\delta F}{\delta \rho }\) is the functional derivative of F with respect to \(\rho \) if it satisfies
for any arbitrary function \(h:\Omega \rightarrow {\mathbb {R}}\).
The dynamical system models the optimal vector field strategies for S, I, R populations and the vaccine distribution. It combines both strategies from mean field controls and SIRV models. For this reason, we call it Meanfield control SIRV system. The proof of Proposition 2.2 can be found in the Appendix.
Algorithms
In this section, we propose an algorithm to solve the proposed SIRV variational problem. We use the primaldual hybrid gradient (PDHG) algorithm [6, 7]. The PDHG can solve the following convex optimization problem.
where f and g are convex functions and A is a continuous linear operator. The algorithm solves the problem by converting the problem into a saddle point problem by introducing a dual variable p.
with \(L^2\) inner product is defined in (2.12) and
is the Legendre transform of f. The method solves the saddle point problem by iterating
The scheme converges if the step sizes \(\tau \) and \(\sigma \) satisfy
where \(\Vert \cdot \Vert \) is an operator norm in \(L^2\). However, the SIRV variational problem has a nonlinear function A for the constraint. Thus, we use the extension of the algorithm from [10] which solves the nonlinear constrained optimization problem.
where A is a nonlinear function. The scheme iterates algorithm (3.1) with a linear approximation of A at a base point \({{\bar{u}}}\)
Denote \(A_{u} := \nabla A(u)\). We have a linearized saddle point problem
and the scheme iterates
The paper [10] proves that the sequence \(\{u^{(k)},p^{(k)}\}^\infty _{k=0}\) of the algorithm converges to some saddle point \((u_*,p_*)\) that satisfies (2.17). However, the scheme converges if the step sizes satisfy
Suppose we use an unbounded operator that depends on the grid size, for example, \(A = \nabla \). The discrete approximation of the operator norm of A increases as the grid size increases (Fig. 1 illustrates the relationship between the norm of an unbounded operator and grid sizes). Thus, the scheme can result in a very slow convergence if we use a fine grid resolution. To circumvent the problem, we use the Generalproximal PrimalDual Hybrid Gradient (Gprox PDHG) method from [21] which is another variation of the PDHG algorithm. This variant provides an appropriate choice of norms for the algorithm, and the authors prove that choosing the proper norms allows the algorithm to have larger step sizes than the vanilla PDHG algorithm. The Gprox PDHG iterates
where the norm \(\Vert \cdot \Vert _{{\mathcal {H}}^{(k)}}\) is defined as
By choosing the proper norms, the step sizes only need to satisfy
which are clearly independent of the grid size.
Local convergence of the algorithm
In this section, we show the iterations from algorithm (3.6) locally converges to the saddle point. The local convergence theorem in this paper is mainly based on Theorem 2.11 from [10]. However, we add a preconditioning operator from the Gprox PDHG method. We show that the method converges locally to the saddle point with the step sizes independent of the nonlinear operator A.
From algorithm (3.6), \((u^{(k+1)},p^{(k+1)})\) satisfies the following firstorder optimality conditions
which can be rewritten as
with \(q=(u,p)\). Here, the monotone operator \(H_{{{\bar{u}}}}\) is defined as
and
where Id is an identity operator.
Recall that from (2.17), the saddle point \(q_*=(u_*,p_*)\) has to satisfy
Throughout, we assume that
Lemma 3.1
There exists constants \(0<c<C\) and \(R>0\) such that
where \(\Vert \cdot \Vert \) is an operator norm.
Proof
This follows immediately from (3.9) and the fact that the derivative \(\nabla A(u)\) is continuous with respect to u. \(\square \)
Lemma 3.2
Suppose (3.9) holds and let \(\tau ^{(k)} \sigma ^{(k)} <1\). Then there exist constants \(0<\theta < \Theta \) such that
where
A proof of Lemma 3.2 is provided in the appendix.
With the above Lemmas, we can use Theorem 2.11 from [10] to show the local convergence of the algorithm.
Theorem 3.3
Let \((u_*, p_*) \in L^2 \times {\mathcal {H}}^{(*)}\) be a solution to (2.17) where \(\Vert p\Vert ^2_{{\mathcal {H}}^{(*)}} = \Vert A_{u_*}^T p\Vert ^2_{L^2}\). Let the step sizes \(\tau ^{(k)}\) and \(\sigma ^{(k)}\) satisfy \(\tau ^{(k)}\sigma ^{(k)}<1\) for all k. Then there exists \(\delta >0\) such that for any initial point \((u^{(0)},p^{(0)})\in L^2 \times {\mathcal {H}}^{(0)}\) satisfying
the iterates \((u^{(k)}, p^{(k)})\) from (3.6) converges to the saddle point \((u_*, p_*)\).
Proof
By Lemma 3.1, Lemma 3.2, and strong convexity of the functional G from (2.8), we can use [10, Theorem 2.11], which proves the theorem. \(\square \)
Remark 3.4
[10, Theorem 2.11] requires \(H_{u_*}\) to satisfy the condition called metric regularity. In our formulation, the constraint \(A(u) = 0\) makes \(H_{u_*}\) metrically regular by [11, Section 5.3]. We refer readers to [10, 11, 33] for further details about metric regularity.
Implementation of the algorithm
To implement the algorithm to the minimization problem (2.8), we set
We use (2.15) for the definition of the operator A. Define the Lagrangian functional as
where \(\langle \cdot , \cdot \rangle _{L^2}\) is defined in (2.12). We summarize the algorithm as follows.
Here, \(L^2\) and \(H^{(k)}_i\) norms are defined as
for any \(u : [0,T] \times \Omega \rightarrow [0,\infty )\). Moreover, the relative error is defined as
In sect. 4, We use quadratic functions for \({\mathcal {E}}_i\) \((i\in \{S,I,V\})\), \({\mathcal {G}}_P\), \({\mathcal {G}}_V\), \({\mathcal {G}}_0\). With definition (2.10), we use
Thus, we can write the cost functional as follows
where \(a_i\), \(d_P\), \(d_V\), \(d_0\) are nonnegative constants. With this cost functional, we find explicit formula for each variable \(\rho _i^{(k+1)},m_i^{(k+1)},\phi _i^{(k+1)}\) \((i\in {\mathbb {S}}), f^{(k+1)}\).
Proposition 3.5
The variables \(\rho _i^{(k+1)},m_i^{(k+1)},\phi _i^{(k+1)}\) (\(i\in {\mathbb {S}}\)), and \(f^{(k+1)}\) from Algorithm 3.1 satisfy the following explicit formulas:
where \(root_+(a,b,c)\) is a positive root of a cubic polynomial \(x^3 + a x^2 + b x +c = 0\) and we approximate the \(A_iA_i^*\) as follows
We use FFTW library to compute \((A_i A_i^T)^{1}\) (\(i\in {\mathbb {S}}\)) and convolution terms by Fast Fourier Transform (FFT), which is \(O(n\log n)\) operations per iteration with n being the number of points. Thus, the algorithm takes just \(O(n\log n)\) operations per iteration.
Experiments
In this section, we present several sets of numerical experiments using Algorithm 3.1 with various parameters. We wrote C++ codes to run the numerical experiments. Let \(\Omega = [0,1]^2\) be a unit square in \({\mathbb {R}}^2\) and the terminal time \(T=1\). The domain \([0,1]\times \Omega \) is discretized with the regular Cartesian grid below.
where \(N_{x_1}\), \(N_{x_2}\) are the number of discretized points in space and \(N_t\) is the number of discretized points in time. For all the experiments, we use the same set of parameters,
By setting a higher value for \(\alpha _I\), we penalize the infected population’s movement more than other populations. Considering the immobility of the infected individuals, this is a reasonable choice in terms of realworld applications. By setting \(T'=1/2\), the solution will produce the vaccines during \(0\le t < 1/2\) and deliver them during \(1/2 \le t \le 1\). Furthermore, we fix the parameters for the infection rate and recovery rate
The paper [28] describes how the parameters \(\beta \) and \(\gamma \) affect the propagation of the populations. In this paper, we focus on the vaccine productions and distributions. Recall that from formulation (3.10), we have terminal functionals
Thus, the solution to the problem has to minimize the total number of susceptible, infected, and vaccines at the terminal time T. The solution reduces the total number of infected by recovering them with a rate \(\gamma \) and decreases the total number of susceptible by transforming the susceptible to the infected with a rate \(\beta \) or to the recovered with a rate \(\theta _1\) (Fig. 2). If the \(\beta \) is large and \(\gamma \) is small, the number of infected will grow since there are more inflows from susceptible than the outflows to the recovered. To minimize the total number of the infected, the solution has to vaccinate the susceptible as much as possible to avoid the susceptible becoming infected. Thus, the vaccines need to be produced and delivered to the susceptible efficiently while satisfying constraint conditions (2.9).
We present two experiments that demonstrate how the various factors in the formulation affect the production and the distribution of vaccines.
Experiment 1
In this experiment, we show that Algorithm 3.1 converges independent of grid sizes when we use the preconditioning operator defined in Proposition 3.5. Consider the initial densities for the \(\rho _i\) (\(i\in {\mathbb {S}}\)) and the factory location \(\Omega _{factory}\) as
where \((x)_+ = \max (x,0)\) and \(B_r(x)\) is a ball of a radius r centered at x. Figure 3 shows the images of initial conditions (4.1).
We compute the solution of the SIRV variational problem (2.11) with the above initial conditions using Algorithm 3.1. For simplicity, we assume recovered population density \(\rho _R\) does not move. Thus, we use an arbitrary large number for a parameter \(\alpha _R = 10^4\) to penalize when \(m_R > 0\). The rest of the parameters are identical to the parameters defined in the preceding section. We ran four simulations with same initial conditions and same step sizes (\(\tau =0.05\), \(\sigma =0.2\)) with four different grid sizes:
\(N_{x_1}\)  \(N_{x_2}\)  \(N_t\) 

32  32  32 
64  64  32 
128  128  32 
256  256  32 
The result of the experiment is depicted in Fig. 4. The figure shows the convergence plot of the algorithm with respect to the number of iteration for each grid size. The xaxis indicates the iteration number and the yaxis indicates the value of the following Lagrangian functional:
Note that this Lagrangian functional \(\tilde{{\mathcal {L}}}\) is different from (2.8) and (2.13). The terms with indicator functions \(i_{\Omega _{factory}}\), \(i_{[\infty ,t]}\) are removed to avoid representing \(+\infty \) numerically. The absence of the terms may explain that the value \(\tilde{{\mathcal {L}}}\) increases in the first 500 iterations and then decreases afterward. Figure 5 shows the computed solutions at iteration 3000 from four different spatial grid sizes (\(32\times 32\), \(64\times 64\), \(128\times 128\), \(256\times 256\)). Each row of the figure shows the evolution of a vaccine density \(\rho _V\) from time \(t=0\) to \(t=1\) computed from each grid size. These figures clearly show that the algorithm converges to the same saddle point independent of the grid sizes.
Experiment 2
In this experiment, we show how the parameters related to the vaccine density variable \(\rho _V\) (\(\theta _1, \theta _2, f_{max}, C_{factory}\)) affect the solution. We use the same initial densities for \(\rho _i\) (\(i\in {\mathbb {S}}\)) and f as in Experiment 1. With the initial densities (4.1), we run two simulations with different values for \(\theta _1\), \(\theta _2\), and \(f_{max}\).
Parameters  Sim 1  Sim 2  Description 

\(\theta _1\)  0.5  0.9  Vaccine efficiency 
\(f_{max}\)  0.5  10  Maximum production rate of vaccines 
\(C_{factory}\)  0.5  2  Maximum amount of vaccines 
that can be produced at \(x\in \Omega \) during \(0\le t \le \frac{1}{2}\) 
Figure 6 shows the comparison between the results from the simulation 1 and the simulation 2. The first three plots (Fig. 6a) show the total mass of \(\rho _i\) (\(i=S,I,R\)), i.e.,
and the last plot (Fig. 6b) shows the total mass of \(\rho _V\) during \(0\le t \le \frac{1}{2}\)
The total number of vaccines produced from the simulation 1 is smaller than that from the simulation 2 because the solution cannot produce a large amount of vaccines due to the low production rate \(f_{max}\). Furthermore, the solution from the simulation 1 cannot vaccinate a large number of susceptible due to a small \(\theta _1\). Thus, there are more susceptible and less recovered at the terminal time in the simulation 1.
Experiment 3
This experiment includes the spatial obstacles and shows how the algorithm effectively finds the solution that utilizes the vaccine production and distribution given spatial barriers. Denote a set \(\Omega _{obs} \subset \Omega \) as obstacles. We use the following functionals in the experiment.
The densities \(\rho _i\) (\(i\in {\mathbb {S}}\)) cannot be positive on \(\Omega _{obs}\) due to \(i_{\Omega _{obs}}\). Thus, the densities transport while avoiding the obstacle \(\Omega _{obs}\). We show two sets of experiments based on this setup.
Single factory
We set the initial densities and \(\Omega _{factory}\) as follows
and fix the parameters
The initial densities are shown in Fig. 7.
Figures 8 and 9 show the evolution of densities with and without obstacles, respectively. In both simulations, the density of vaccines \(\rho _V\) (the fourth row) transports to the areas where the susceptible people are present. In Fig. 9, \(\rho _V\) transports while avoiding the obstacle at the right. Figure 10 shows the comparison between these two solutions and how the presence of the obstacle affects the production and delivery of vaccines quantitatively. Figure 10a shows the total mass of the vaccines in the factory area \(\Omega _{factory}\) during the production time
Figure 10b shows the total mass of the vaccines during the delivery time at the left side and the right side of the domain
during \(t\in [0.5,1]\). When there is no obstacle, the vaccines are delivered more to the right than to the left (Fig. 10b). The number of susceptible people at the left decreases very fast because there are infected people with a high infection rate. When \(\rho _V\) starts to transport at time \(t=0.5\), the number of susceptible is lower at the left. Thus, the solution distributes fewer vaccines to the left with less susceptible people. When there is an obstacle, \(\rho _V\) has to bypass the obstacle to reach the susceptible areas. Thus, the kinetic energy cost during the delivery time \(t\in [0.5,1]\) increases at the right. The solution cannot deliver the vaccines as much as the case without the obstacle. It results in a fewer number of vaccines produced during \(t\in [0,0.5)\) (Fig. 10a) and delivered to the right during \(t\in [0.5,1]\) when there is an obstacle (Fig. 10b).
Multiple factories
Similar to the previous experiment, we show how the obstacles in the spatial domain affect the production and distribution of the vaccines. We use more complex initial densities, an obstacle set \(\Omega _{obs}\), and three factory locations in this experiment. We set the initial densities and \(\Omega _{factory}\) as follows
and fix the parameters
The initial densities are shown in Fig. 11.
Figures 12 and 13 show the evolution of densities with and without obstacles, respectively. The experiment demonstrates that even with the complex initial densities, the algorithm successfully converges to the reasonable solution that coincides with the previous experiments. The density of vaccines \(\rho _V\) (the fourth row) transports to the areas where the susceptible people are present while avoiding the obstacles.
Figure 14a shows the total mass of the vaccines produced during the production time at each factory location. Without the obstacles, the total mass of \(\rho _V\) at the middle is the lowest at time 0.5 because the factory at the middle is the farthest away from the susceptible people. It is more efficient to produce the vaccines at the factories closer to the susceptible (the top and the bottom) to reduce the kinetic energy cost during the delivery time \(t\in [0.5,1]\). However, the vaccines are produced the most at the middle factory with the obstacles. Since the obstacles block the paths between the top and the bottom factories and the susceptible people, \(\rho _V\) has to bypass them to reach the target area. The pathways from the middle factory to the susceptible people are not blocked as much as from the top and the bottom factories. Thus, producing more vaccines at the middle factory is more efficient.
Figure 14b shows the total mass of the vaccines during the delivery time at different locations. The lines in the plot represent the following quantities:
over \(t\in [0.5,1]\). With the obstacles, the kinetic energy cost increases since \(\rho _V\) has to bypass to reach to the targets when it transports from the top and the bottom factories. As a result, the vaccines are not produced as much as the simulation without the obstacles, and there are less vaccines reached to the targets.
Experiment 4
This experiment compares the vaccine production strategy generated by the algorithm and the strategy with the fixed rates of production without using the algorithm. The initial densities and \(\Omega _{factory}\) are set as follows
We fix the parameters
The initial densities and locations of factories are shown in Fig. 15.
To fairly compare the effect of the optimal vaccine production strategy, we remove the momentum of S, I, R groups; thus, removing the spatial movements defined by \(m_S\), \(m_I\), \(m_R\). We consider the following PDEs:
Furthermore, by taking out the momentum terms from S, I, R groups, the cost functional for this experiment is
With the PDEs and the cost functionals above, we compare two results. The first result is using the optimal vaccine production and distribution strategy generated by Algorithm 3.1. The second result is using the fixed vaccine production rate and the algorithm’s distribution strategy. In the second result, the factory variable f is fixed as
Figure 16 shows the comparison between these two results. The result from the fixed production rate is “without control,” and the result from the optimal vaccine production strategy is “with control.” The labels “left,” “middle,” and “right” are the locations of the factories in Fig. 15. The solid lines, the result with the same fixed rates of production, show that all three factories produce identical amounts of vaccines. The dotted lines show the least amount of vaccines in the middle factory and much more in the left and right factories. When vaccines produce at the middle factory, one needs to pay more transportation costs because they bypass the obstacles. The obstacle does not block the paths from the left and right factories to the susceptible. Thus, it’s an optimal choice to utilize the left and right more than the middle to minimize the transportation costs.
The table below is the quantitative comparison between the two results.
Quantity  Description  Algorithm 3.1  Fixed rates 

\(\int _\Omega \rho _V(\frac{1}{2},x)\,dx\)  The total amount of vaccines produced.  \(7.997\times 10^{3}\)  \(8.411\times 10^{3}\) 
\(\int _\Omega \rho _S(1,x)\,dx\)  The number of susceptible people at the terminal time.  \(1.520\times 10^{2}\)  \(1.525\times 10^{2}\) 
\(\int _\Omega \rho _I(1,x)\,dx\)  The number of infected people at the terminal time.  \(5.133\times 10^{3}\)  \(5.134\times 10^{3}\) 
\(\int ^1_{\frac{1}{2}}\int _\Omega \frac{m_V^2}{2\rho _V}\,dx\,dt\)  The transportation cost of vaccines.  \(7.339\times 10^{3}\)  \(7.544\times 10^{3}\) 
The first row of the table shows that more vaccines are produced with a fixed rate of production. However, the result of the fixedrate vaccinizes fewer susceptible people; as a result, more infected people at the terminal time. Furthermore, the result from the fixed rate shows higher transportation costs. The algorithm finds the more efficient strategy with fewer vaccines produced.
Data Availability Statement
All data generated or analyzed during this study are included in this published article.
References
Achdou, Y., CapuzzoDolcetta, I.: Mean field games: numerical methods. SIAM J Numer Anal 48(3), 1136–1162 (2010)
Achdou, Y., Laurière, M.,: Mean field games and applications: numerical aspects. arXiv preprint arXiv:2003.04444, (2020)
Aurell, A., Carmona, R., Dayanikli, G., Lauriere, M.: Optimal incentives to mitigate epidemics: a stackelberg mean field game approach. arXiv preprint arXiv:2011.03105, (2020)
Bensoussan, A., Huang, T., Laurière, M.: Mean field control and mean field game models with several populations. arXiv preprint arXiv:1810.00783, (2018)
BriceñoArias, L., Kalise, D., Kobeissi, Z., Laurière, M.A., González, M., Silva, F.J.: On the implementation of a primaldual algorithm for second order timedependent mean field games with local couplings. SAIM: Proceed Surv 65, 330–348 (2019)
Chambolle, A., Pock, T.: A firstorder primaldual algorithm for convex problems with applications to imaging. J. Math. Imaging Vision 40(1), 120–145 (2011)
Chambolle, A., Pock, T.: On the ergodic convergence rates of a firstorder primaldual algorithm. Math. Program. 159(1–2, Ser. A), 253–287 (2016)
Charpentier, A., Elie, R., Laurière, M., Viet CT.: Covid19 pandemic control: balancing detection policy and lockdown intervention under icu sustainability. Mathematical Modelling of Natural Phenomena, 15:57, (2020)
Cirant, M.: Multipopulation mean field games systems with neumann boundary conditions. J. de Mathématiques Pures et Appliquées 103(5), 1294–1315 (2015)
Clason, C., Valkonen, T.: Primaldual extragradient methods for nonlinear nonsmooth pdeconstrained optimization. SIAM J Optimiz 27(3), 1314–1339 (2017)
Clason, C., Valkonen, T.: Stability of saddle points via explicit coderivatives of pointwise subdifferentials. SetVal. Variat. Anal. 25(1), 69–112 (2017)
Di Domenico, L., Pullano, G., Sabbatini, P.Y.B., Chiara, E., Colizza, V.: Impact of lockdown on covid19 epidemic in îledefrance and possible exit strategies. BMC Med 18(1), 1–13 (2020)
Dimarco, G., Perthame, B., Toscani, G., Zanella, M.: Social contacts and the spread of infectious diseases. arXiv preprint arXiv:2009.01140, (2020)
Doncel, J., Gast, N., Gaujal, B.: A mean field game analysis of sir dynamics with vaccination. Probability in the Engineering and Informational Sciences, pages 1–18, (2020)
Feleqi, E.: The derivation of ergodic mean field game equations for several populations of players. Dyn Gam Appl 3(4), 523–536 (2013)
Flaxman, S., Mishra, S., Gandy, A., Unwin, H.J., Mellan, T.A., Coupland, H., Whittaker, C., Zhu, H., Berah, T., Eaton, J.W., et al.: Estimating the effects of nonpharmaceutical interventions on covid19 in Europe. Nature 584(7820), 257–261 (2020)
Francis, A.I., Ghany, S., Gilkes, T., Umakanthan, S.: Review of covid19 vaccine subtypes, efficacy and geographical distributions. Postgrad Med J 98(1159), 389–394 (2021)
Godara, P., Herminghaus, S., Heidemann, K.M.: A control theory approach to optimal pandemic mitigation. PloS one 16(2), e0247445 (2021)
Hansen, E., Day, T.: Optimal control of epidemics with limited resources. J Math Bio 62(3), 423–451 (2011)
Huang, M., Malhamé, R.P., Caines, P.E.: Large population stochastic dynamic games: closedloop mckeanvlasov systems and the nash certainty equivalence principle. Commun. Infor & Sys. 6(3), 221–252 (2006)
Jacobs, M., Léger, F., Li, W., Osher, S.: Solving largescale optimization problems with a convergence rate independent of grid size. arXiv:1805.09453 [math], (2018)
Jang, J., Kwon, H.D., Lee, J.: Optimal control problem of an sir reactiondiffusion model with inequality constraints. Math Comp Simul 171, 136–151 (2020)
Kantner, M., Koprucki, T.: Beyond just “flattening the curve’’: optimal control of epidemics with purely nonpharmaceutical interventions. J Math Ind 10(1), 1–23 (2020)
Kim, J., Kwon, H.D., Lee, J.: Constrained optimal control applied to vaccination for influenza. Comput & Math Appl 71(11), 2313–2329 (2016)
Laguzet, L., Turinici, G.: Individual vaccination as nash equilibrium in a sir model with application to the 2009–2010 influenza a (h1n1) epidemic in france. Bullet Math Bio 77(10), 1955–1984 (2015)
Lasry, J.M., Lions, P.L.: Mean field games. Jap J Math 2(1), 229–260 (2007)
Lee, W., Lai, R., Li, W., Osher, S.: Generalized unnormalized optimal transport and its fast algorithms. J Comput Phys 436, 110041 (2021)
Lee, W., Liu, S., Tembine, H., Li, W., Osher, S.: Controlling propagation of epidemics via meanfield games. arXiv preprint arXiv:2006.01249, (2020)
Libotte, G.B., Lobato, F., Platt, G.M., Neto, A.J.: Determination of an optimal control strategy for vaccine administration in covid19 pandemic treatment. Comp Meth Prog Biomed 196, 105664 (2020)
Lin, Q., Zhao, Q., Lev, B.: Cold chain transportation decision in the vaccine supply chain. Eur J Operat Resear 283(1), 182–195 (2020)
Liu, J., Wang, XS.: Optimal allocation of face masks during the covid19 pandemic: a case study of the first epidemic wave in the united states. arXiv preprint arXiv:2101.03023, (2020)
Ndiaye, BM, Tendeng, L., Seck, D.: Analysis of the covid19 pandemic by sir model and machine learning technics for forecasting. arXiv preprint arXiv:2004.01574, (2020)
Rockafellar, R.T., Wets, R.J.B.: Variational analysis. Springer, Berlin (2009)
Ruthotto, L., Osher, S.J., Li, W., Nurbekyan, L., Fung, S.W.: A machine learning framework for solving highdimensional mean field game and mean field control problems. Proceed Nat Acad Sci 117(17), 9183–9193 (2020)
Sesterhenn, J.: Adjointbased data assimilation of an epidemiology model for the Covid19 pandemic in 2020. arXiv preprint arXiv:2003.13071, (2020)
Silva, C.J., Cruz, C., Torres, D.F.M., Muñuzuri, A.P., Carballosa, A., Area, I., Nieto, J.J., FonsecaPinto, R., Passadouro, R., Santos, E.S.D., et al.: Optimal control of the covid19 pandemic: controlled sanitary deconfinement in portugal. Scient Rep 11(1), 1–15 (2021)
Zaman, G., Kang, Y.H., Jung, I.H.: Stability analysis and optimal vaccination of an sir epidemic model. BioSystems 93(3), 240–249 (2008)
Funding
This work is supported by AFOSR MURI FA95501810502
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
Proof of Proposition 2.2
From the saddle point problem (2.13), we can rewrite the problem as
where a function \({\mathcal {Q}}:(0,T)\times \Omega \rightarrow {\mathbb {R}}\) is defined as
If \(((\rho _i, m_i, \phi _i)_{i\in {\mathbb {S}}},f)\) is the saddle point of the problem, the differential of Lagrangian with respect to \(\rho _i\), \(m_i\), \(\phi _i\) (\(i\in {\mathbb {S}}\)), f and \(\rho _i(T,\cdot )\) (\(i\in \{S,I,V\}\)) equal to zero. Thus, from \(\frac{\delta {\mathcal {L}}}{\delta \phi _i}=0\) we have
Using integration by parts, we reformulate the Lagrangian function (4.3) as follows.
From \(\frac{\delta {\mathcal {L}}}{\delta \rho _i}=0\) (\(i\in \{S,I,R\}\)),
From \(\frac{\delta {\mathcal {L}}}{\delta \rho _V}=0\),
From \(\frac{\delta {\mathcal {L}}}{\delta \rho _i(T,\cdot )}=0\) (\(i\in {\mathbb {S}}\)),
From \(\frac{\delta {\mathcal {L}}}{\delta f}=0\),
From \(\frac{\delta {\mathcal {L}}}{\delta m_i}=0\) (\(i\in {\mathbb {S}}\)),
By replacing \(\frac{\alpha _i m_i}{\rho _i} =  \nabla \rho _i\) in \(\frac{\delta {\mathcal {L}}}{\delta \rho _i}=0\) and \(\frac{\delta {\mathcal {L}}}{\delta \phi _i}=0\), we derive the result. \(\square \)
Proof (Proof of Lemma 3.2)
Let \(q=(u,p)\). By the definition of \(M^{(k)}\), we have
Using Young’s inequality and Lemma 3.1,
We are left to show the lower bound. Let \(\epsilon >0\) be such that \(\tau ^{(k)} \sigma ^{(k)} = (1\epsilon )^2\). Then using Hölder’s inequality,
Again, using Young’s inequality and Lemma 3.1,
This proves the claim. \(\square \)
Rights and permissions
About this article
Cite this article
Lee, W., Liu, S., Li, W. et al. Mean field control problems for vaccine distribution. Res Math Sci 9, 51 (2022). https://doi.org/10.1007/s40687022003502
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s40687022003502
Keywords
 Mean field games
 Primal dual hybrid algorithm
 Vaccine distribution