A Stochastic Nash Equilibrium Problem for Medical Supply Competition

In this paper, we study the competition of healthcare institutions for medical supplies in emergencies caused by natural disasters. In particular, we develop a two-stage procurement planning model in a random environment. We consider a pre-event policy, in which each healthcare institution seeks to minimize the purchasing cost of medical items and the transportation time from the first stage, and a recourse decision process to optimize the expected overall costs and the penalty for the prior plan, in response to each disaster scenario. Thus, each institution deals with a two-stage stochastic programming model that takes into account the unmet demand at the first stage, and the consequent penalty. Then, the institutions simultaneously solve their own stochastic optimization problems and reach a stable state governed by the stochastic Nash equilibrium concept. Moreover, we formulate the problem as a variational inequality; both the discrete and the general probability distribution cases are described. We also present an alternative formulation using infinite-dimensional duality tools. Finally, we discuss some numerical illustrations applying the progressive hedging algorithm.


Introduction
Emergencies resulting from man-made or natural events strongly affect our social and economic life. Depending on the type of emergency different hazards may occur in the emergency locations. Thus, emergency management has raised increasing interest. In particular, businesses require special measures to protect their activities from any potential dangerous effect of an emergency. Therefore, it is important to establish a plan before the occurrence of these events to be prepared in case an emergency happens. A business continuity plan defines how a company will continue operating, even in the case of a natural disaster, IT failure or a cyber attack. The end goal is to preserve profitability and market position [12].
In this paper, we focus on a plan for the storage and distribution of medical supplies among healthcare institutions in emergencies caused by natural disasters. In particular, we model the competition among hospitals as a Nash equilibrium problem and introduce a stochastic programming model to design and evaluate the behavior of each demand location. Inspired by [26,35], we provide a two-stage stochastic programming model based on disaster scenarios that introduces the unmet demand at the first stage and the consequent penalty at the second stage, see also [13]. Thus, we consider a preevent policy, in which each healthcare institution minimize both the purchasing cost of medical items and the transportation time from the first stage. Then, we present a post-event policy through a recourse decision process to optimize the expected overall costs, and the penalty for the preassigned plan, in response to each possible disaster scenario of the second stage. Institutions simultaneously solve their own two-stage stochastic optimization problems and reach a stable state governed by the stochastic Nash equilibrium concept, that is formulated as a large-scale variational inequality. In addition, in the case of a general probability distribution, we define the stochastic Nash equilibrium as a random variational inequality in a Hilbert space setting. Then, we give the first order optimality conditions for the second-stage problem in terms of Lagrange multipliers, using a separation assumption, called Assumption S, as a constraint qualification [4][5][6]9]. This condition results to be a necessary and sufficient condition for strong duality to hold. In infinite-dimensional spaces, the classical theorems, which prove strong duality and existence of multipliers, require that the interior of the ordering cone be nonempty [21]. However, in most infinite-dimensional cases, where the functional space is L 2 or a Sobolev space, the ordering cone has the empty interior. Therefore, we aim at proving that the second-stage problem verifies the Assumption S. As a result, we ensure the existence of Lagrange multipliers and give an alternative formulation of the two-stage problem. Moreover, we show that the dual variables regulate the medical item procurement. In fact, they represent the control variables on the first-stage demand, on the second-stage demand, and on the unfulfilled demand.
The importance of an efficient approach to emergency management and medical supply planning has been investigated in several papers. As an example, in [26], the authors presented a stochastic programming model, in which they selected the storage locations of medical supplies and use inventory levels for medical items. In the model, they captured the information updating during disaster scenarios. In [28], Nagurney et al. developed a stochastic generalized Nash equilibrium model consisting of multiple purchase locations for the disaster relief items, multiple humanitarian organizations, multiple freight service provision options and multiple hubs for storage to multiple points of demand. In [29], the authors presented a generalized Nash equilibrium model with stochastic demand to analyze competition among organizations at demand points for medical supplies. In [30], Nagurney and Salarpour introduced a two-stage stochastic game theory model in order to examine the behavior of national governments during Covid-19 pandemic, and their competition for essential medical supplies in both the preparation and response phases. All the problems presented in [28][29][30] were solved as variational inequalities, using the concept of variational equilibrium. We remark that our model differs from the treatment in [26] as we develop a variational inequality approach. In addition, although the problems introduced in [28][29][30] have similarities, they are all restricted to the case of discrete probability distribution, whereas our model is valid also for general probability distribution. This poses challenges for both theory and computations.
Recently, two-stage stochastic variational inequalities have been introduced to model cases where one looks for a decision vector before the real situation is known, and a new one after the scenario has been realized. In [10], the authors formulated the two-stage stochastic variational inequality as a two-stage stochastic programming problem with recourse. In [23], Li and Zhang studied the transformation of a general two-stage stochastic programming problem to a two-stage stochastic variational inequality. In [14], the authors presented an evacuation model where a population had to be evacuated from crisis areas to shelters, and, due to the uncertainty in the size of the population to be evacuated, a two-stage stochastic variational inequality model was given. In [32], Rockafellar and Wets discussed the multistage stochastic variational inequality. In [33], the authors developed progressive hedging methods for solving multistage convex stochastic programming, see also [34].
In [16], Gwinner and Raciti studied the random variational inequality and general random equilibrium problems. In particular, they worked on a class of linear random variational inequalities on random sets, with results on measurability, existence and uniqueness in a Hilbert space. Furthermore, they provided an approximation procedure in a special case. Then, in [17], the same authors carried out the theory of random variational inequalities to study a class of random equilibrium problems on networks in the linear case, and in [18] they studied the application to nonlinear random traffic equilibrium problem. A valuable additional contribution of the same authors is the book in [19]. In [11], the authors formulated the multicriteria spatial price network equilibrium problem as a random variational inequality, in which the consumers weight, using random fluctuations, transportation cost and the transportation time associated with the shipment of a given item. In [7,8], the authors applied a general random traffic equilibrium problem, featuring the random Wardrop equilibrium distribution using random variational inequality. In [22], Jadamba and Raciti explored stochastic Nash equilibrium problems using monotone variational inequalities in probabilistic Lebesgue spaces. Their results are applied to a class of oligopolistic market equilibrium problems.
Inspired by the above works, in this paper, we provide a variational inequality formulation of the two-stage stochastic optimization problem describing the competition of healthcare institutions in case an emergency happens. The main contributions of our work are: • Modeling a medical supply network that involves warehouses and hospitals with multiple medical items and multiple transportation modes. • Providing a two-stage stochastic programming model based on disaster scenarios that considers the unmet demand at the first stage and the consequent penalty at the second stage. • Deriving a variational inequality formulation in both the discrete and general probability cases. • Characterizing the second-stage equilibrium, in the case of general probability distribution, by means of infinite-dimensional Lagrange duality. • Testing the equilibrium model with numerical illustrations with realistic data.
An analysis of the Lagrange multipliers is also performed and, hence, this paper adds to the literature on the study of marginal utilities in the more challenging setting of stochastic programming problems. This paper is organized as follows. In Sect. 2, we introduce the two-stage stochastic model for the medical supply competition. In Sect. 3, we present the stochastic Nash equilibrium concept underlying our model and the equivalent variational inequality formulation. The cases of discrete and general probability distribution are discussed. In Sect. 4, we recall some infinite-dimensional duality tools, and, in Sect. 5, we present an alternative formulation of the second-stage problem. The progressive hedging algorithm is then applied to some numerical examples in Sect. 6. We summarize our results and draw our conclusions in Sect. 7. of transportation. We note that the suitable transportation mode is often connected to the distance between supply and demand locations. Thus, for long distances airplanes are preferred to transportation by truck or train. The choice of the transportation mode may also depend on the type of medical item or on the severity of the emergency.
Let x k wh be the amount of medical item of type k from warehouse w to hospital h, and let ρ k w be the unitary price of medical item k at warehouse w. Let x wh denote the total amount delivered from warehouse w to hospital h, where We further group the x wh into the W H−dimensional column vector x.
Moreover, we introduce the transportation time t m wh from warehouse w to hospital h with mode m and assume that it depends on the total amount x, namely, t m wh = t m wh (x). We consider two stages, where one corresponds to the preparedness phase and the other represents the response phase. In the first phase, each demand location, namely the hospital, looks for minimizing the purchasing cost of medical items and the transportation time from the first stage; in the second one, a recourse decision process is developed to optimize the transportation costs from the second stage, in response to each disaster scenario. Let ( , F, P) be a probability space, where the random parameter ω ∈ represents the typical disaster scenario. For each ω ∈ , we denote by ξ : → R W H K +H K a finite-dimensional random vector and by E ξ the mathematical expectation with respect to ξ . In order to formulate the twostage stochastic model, we introduce two types of decision variables. The first-stage decision variable x k wh is used to represent the quantity of medical supplies of type k from warehouse w to hospital h in stage one. The second-stage decision variables are y k wh (ω) and z k h (ω). The variable y k wh (ω) represents the quantity of medical supplies of type k to be delivered from warehouse w to hospital h under scenario ω. The variable z k h (ω) is the unfulfilled demand at hospital h of medical supply item k under scenario ω. The amount of unfulfilled demand z k h (ω) is penalized by the penalty function π k h = π k h (ω, z k h (ω)). We note that x k wh is chosen before a realization of ξ is revealed and later y k wh (ω) and z k h (ω) are selected with known realization. We set y wh (ω) = k∈K y k wh (ω). We further group the y wh (ω) into the W H−dimensional column vector y(ω). Finally, we introduce the transportation cost c m wh from warehouse w to hospital h with mode m and assume that it depends on the total amount y(ω), namely, c m wh = c m wh (ω, y(ω)). Table 1 summarizes the relevant notations used in the model formulation.
We aim at obtaining an efficient plan of medical item procurement of each demand location in the first stage by the evaluation of adaptive plans in the second stage. Thus, for each hospital, we minimize the purchasing cost and the transportation time of the first stage with the expected overall costs and the penalty for the prior plan. For each hospital, a two-stage procurement planning model in a random environment is formulated. We first present the hospital's problem as a two-stage stochastic programming problem and then define the stochastic Nash equilibrium describing the competition of all hospitals. Table 1 The notation for the two-stage stochastic model The objective function (1) minimizes the sum of the purchasing cost for early supply plan, the transportation time, and the expected value of the second stage solution, with respect to disaster scenario, h (x, ξ(ω)). Constraint (2) states that hospital h receives at least the needed amount of medical items; constraint (3) is the maximum availability constraint of each medical supply type k; constraint (4) is the non-negativity requirement on variables. In order to ensure that the constraint set is nonempty, we require that d k h ≤ e k , ∀h, k. For a given realization ω ∈ , h (x, ξ(ω)) is given by Thus, h (x, ξ(ω)) is the optimal value of the second-stage problem (5)-(8) associated with hospital h, where the constraints hold almost surely (P-a.s.). We remark that h (x, ξ(ω)) depends on x via constraint (7). The objective function (5) minimizes the total cost and the penalty for unmet demand at the second stage. Constraint (6) states that the supply at the second stage plus the unmet demand should be at least as much as the demand at the second stage. Constraint (7) is the maximum availability constraint of each medical supply of type k. We emphasize that the connection between stage-wise decision variables x and y is captured by constraint (7). It is the linking factor between the first and second stage and communicates the first-stage decisions to the second one. Constraint (8) is the non-negativity requirement on variables. We also assume that z k h (ω) ≤ α d k h , α ∈]0, 1], P-a.s., namely, the unmet demand cannot exceed a fixed percentage of the first-stage demand.
In order to ensure that the constraint set of the second stage is nonempty, it suffices to require that We assume that: , a.e. in , are continuously differentiable and convex for all

Stochastic Nash Equilibrium Problem
In this section, we present the equilibrium concept underlying our model and the equivalent variational inequality formulation. Both the cases of discrete and general probability distribution are discussed. Each hospital minimizes the deterministic costs and the expected costs for all the scenarios. Then, the hospitals simultaneously solve their own optimization problems and reach a stable state governed by the Nash equilibrium concept.
We define the sets: (14) hold , We will refer to the objective function (9) as the function . According to the above definition, a Nash equilibrium is established if no hospital can unilaterally improve upon his profit by choosing an alternative medical item flow pattern, given other hospitals' decision strategies.

Discrete Probability Distribution
If the random parameter ω ∈ follows a discrete distribution with finite support = {ω 1 , . . . , ω r } and probabilities p(ω r ) associated with each realization ω r , r ∈ R = {1, . . . , R}, then the objective function (9) for h ∈ H becomes and the cost minimization problem for h ∈ H is given by It is well known that a Nash equilibrium can be characterized as a solution to a variational inequality problem (see [20,27] for theory and applications on variational inequalities). Thus, the competition among hospitals under the Nash criterion is described by the following variational inequality: We note that the set S, and the set T , ∀r ∈ R, are nonempty, compact and convex, and the operator entering (28) is continuous. Therefore, a solution to the above problem exists from the standard theory of variational inequalities [20].

General Probability Distribution
In the case of a general probability space ( , F, P), studying the optimality conditions can be very hard, as one should state the order of the decision process explicitly. For this reason, we choose as our functional setting a Hilbert space and assume that y ∈ L 2 ( , P, R W H ), z ∈ L 2 ( , P, R H K ), d ∈ L 2 ( , P, R H K ). L 2 ( , P, R W H ) denotes the class of R W H -valued functions defined in , that are square integrable with respect to the probability measure P.
Proof The proof procceds as in [3].
To ensure the existence of solutions, we may apply the results in [25]. We first recall some definitions.
Let E be a reflexive Banach space with dual space E * and K ⊂ E a closed convex set. • for each sequence u n weakly converging to u in K and such that lim sup n Au n , u n − u ≤ 0 it results lim inf n Au n , u n − v ≥ Au, u − v , ∀v ∈ K ;

Definition 3.2 A mapping
• for each v ∈ K the function u → Au, u − v is lower bounded on the bounded subsets of K .

Definition 3.3 A mapping
Theorem 3.2 Let us assume that the map A : K → E * be B-pseudomonotone or F-hemicontinuous and there exist u 0 ∈ K and R > u 0 such that Then, the variational inequality Au, v − u , ∀v ∈ K admits solutions.

Theorem 3.3 Let A : K → E * be a K-pseudomonotone map which is lower hemicontinuous along line segments. Let us assume that condition (32) holds true. Then, variational inequality Au, v − u , ∀v ∈ K admits solutions.
We recall that condition (32) is satisfied if the coercivity condition is verified: We can apply Theorem 3.2 and Theorem 3.3, assuming that the operator of the variational inequality is B-pseudomonotone or F-hemicontinuous and (32) or (33) holds true, or assuming that it is K-pseudomonotone, conditions (29)-(30) are verified, and (32) or (33) holds true. We also recall that condition (30) is sufficient to guarantee that the operator is lower hemicontinuous along line segments (see [15]).
X denote a real normed space and X * the topological dual of all continuous linear functionals on X . Given C, a nonempty subset of X , and an element x ∈ X , the set x n ∈ C ∀n ∈ N, lim n→∞ x n = x is called the contingent cone to C at x. Of course, if T C (x) = ∅, then x belongs to the closure of C, denoted by cl C. If C is convex, then [21] We now present the statement of Theorem 3.2 in [24]. Let X be a real normed space real and S be a nonempty subset of X ; let (Y , · ) be a real normed space, partially ordered by a convex cone C. Let f : S → R and g : S → Y be two convex functions. Let us consider the primal problem and the dual problem where C * is the dual cone of C. We say that Assumption S is fulfilled at a point x 0 ∈ K if and only if it results: Then, in [24] the following theorem is proved. (35), it results that u, g(x 0 ) = 0.

Theorem 4.1 Under the above assumptions, if problem (34) is solvable and Assumption S is fulfilled at the extremal solution x 0 ∈ K , then also problem (35) is solvable, the extreme values of both problems are equal and, denoted by u the optimal solution of
The following result entitles us to characterize a solution of problem (34) as a saddle point of the Lagrange function [4].

Theorem 4.3 The vector (y * h , z * h ) ∈ T h , for all h ∈ H, is an optimal solution of the second-stage problem (5)-(8) if and only if (y
Now, we give two preliminary results.

is a solution to (36) and prove that Assumption
with γ n ≥ 0, λ n > 0, u i n ≥ 0, i = 1, . . . , 5, and then l must be nonnegative. We prove that every term in (40)-(41) tends to zero. We first considers only the terms in y: Taking into account that w∈W y * k wh (ω) = − w∈W x k wh + e k (ω), all the terms tends to zero. Analogously, we can prove that the other terms tends to zero.
Therefore, the two-stage problem can be reformulated as follows: We now describe some relevant consequences that gives an insights into the market behavior with respect to the product shipment. Dual variables λ 1k w regulate the medical item procurement. In particular, λ 1k h is a control variable on the first-stage demand; λ 2k h is a control variable on the item availability level; μ 1k w is a control variable on the second-stage demand; μ 2k w and μ 3k w are control variables on the unfulfilled demand. We discuss some cases, considering active and non-active constraints. We have: If y * k wh (ω) > 0, then μ 1k w (ω) = 0, P-a.s., and m∈M ∂c m wh (ω, y * (ω)) ∂ y k namely, the marginal cost is equal to the difference of the control variables on demand and market item availability. Moreover, if λ * 1k h (ω) = 0, λ * 2k h (ω) > 0, P-a.s., we find m∈M ∂c m wh (ω,y * (ω)) ∂ y k wh = −λ * 2k h (ω), P − a.s., and the marginal cost , P-a.s., and the marginal cost increases. From we note that the marginal penalty is equal to the difference between the control variables on the unfulfilled demand.
s. This is the case of an effective emergency plan, in which hospital does not incur in any unmet demand. If μ * 3k (ω) > 0, then μ * 2k (ω) = 0, P − a.s., and namely, the marginal penalty decreases.

Numerical Example
In this section, we present two small numerical examples for illustrative purposes. We consider two warehouses (w = 2), two hospitals (h = 2), three different items (k = 3), one transportation mode (m = 1) and five scenarios. The economic data mainly come from [1]. For the calculation of transportation costs, we apply the Product & Distance-based calculation rule, which computes the transportation costs based on the coefficients for transportation costs (that include all the different terms, e.g., fuel price, tolls, etc.) for national shipments assumed to be 0.19e/km * m 3 . For transportation time from warehouses to hospitals, we consider hourly cost set, that includes the time spending for the loading process, the route to go and the unloading process. Penalty costs, concerning the unfulfilled demand, depends on the number of items that are not delivered in one day.
The numerical simulations are solved applying the Progressive Hedging Method (PHM) [31]. This is a well-known algorithm that has been recently extended to multistage SVI and multistage stochastic Lagrangian variational inequalities [33,34]. In [2], the authors presented a new framework that shows how PHM can be utilized, while guaranteeing convergence, to globally optimal solutions of mixed-integer stochastic Numerical Example 1: In this numerical example, the items are collected in multiple packages, and the coefficients of the cost functions are related to a single package. We have considered high penalty functions, since not satisfying the demand for a particular item would cause severe discomfort. It can be noted that, given the danger of the penalty, at equilibrium we get zero penalties as it is likely that hospitals pay more attention to some indispensable items. The matrix of the cost functions is given by ⎛ ⎝ 2, 6 2, 9 3, 5 7, 2 0, 00019 0, 00029 0, 00038 0, 00023 1800 1600 2, 7 2, 5 1, 7 2, 2 0, 00025 0, 00031 0, 00024 0, 00018 1500 1400 1, 6 1, 9 1, 02 2, 03 0, 00021 0, 00036 0, 00032 0, 00028 1300 1500 First of all, we focus our attention on the flows x k wh of the first stage. We find From the numerical result of the first example [see (48), Table 2], we notice that in a condition without emergency, each hospital prefers to choose his trusted warehouse. In particular, • All hospitals decide to buy the medical item one (k = 1) from warehouse one (w = 1); • All hospitals decide not to buy the medical item two (k = 2) from warehouse one (w = 1) or two (w = 2); • For medical item three (k = 3), hospital two (h = 2) decides to rely on warehouse one (w = 1) and hospital one (h = 1) on warehouse two (w = 2).
In the second stage, namely, in an emergency situation, the usual choice is no longer the optimal one, but demand must always be satisfied by minimizing costs. Furthermore, the penalties are fortunately null for each hospital and for each item. The results are shown in Table 2. Numerical Example 2: In this numerical example, the items are treated as boxes and the coefficients of the cost functions are related to boxes which contain thousand packages. We have considered low penalty functions, since not satisfying the demand for a particular item would not cause severe discomfort. In this case, the amount of unfulfilled demand at hospital h of medical supply item k under scenario ω, for all h, k are not null. This is a consequence of the fact that for these items it is not necessary to satisfy fully the daily demand. Another difference with the first numerical example is that all hospitals use all warehouses, without choosing the trusted warehouses.
The coefficient matrix of the cost functions is represented by (49).

Conclusions
In this paper, we constructed a stochastic Nash equilibrium model for a medical supply network that consists of warehouses and hospitals with multiple medical items and multiple transportation modes. Each hospital solves a two-stage stochastic optimization problem, where, in the first stage, seeks to minimize the purchasing cost of medical items and the transportation time. Then, we introduced a recourse decision process to optimize the expected overall costs and the penalty for the prior plan, in response to each possible disaster scenario of the second stage. The hospitals simultaneously solve their own stochastic optimization problems and reach a stable state given by the stochastic Nash equilibrium concept. Specific features of the model include: the uncertainty of the scenarios, the supply availability of medical items, the penalty for unmet demand and the fluctuating costs. The model is formulated as a variational inequality. In the case of general probability distribution, we characterized the Nash equilibrium of the problem as a solution to an infinite-dimensional variational inequality in the Hilbert space L 2 . The associated Lagrange function was studied, and a strong duality result was provided. Finally, we presented some numerical illustrations solved applying the progressive hedging algorithm. The results reveal that hospitals are able to re-arrange timely their requests in order to satisfy the need for medical items. In emergencies, uncertainty plays a fundamental role in the success of disaster management; hence, health institutions must be ready to adjust the request of medical items. Our contributions to the literature lie in advancing the state-of-the-art of stochastic programming for disaster management as well as applications of variational inequalities and strong duality. We also emphasize that to date there has been limited work on stochastic programming problems under general probability distribution.
This model could be extended in future research. For example, we could incorporate additional details to the model and solve examples using data from real situations. The extension to a multi-stage problem where we consider different stages of information is another future research opportunity.
Funding Open access funding provided by Università degli Studi di Catania within the CRUI-CARE Agreement.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.