De Finetti’s Control Problem with Competition

We investigate the effects of competition in a problem of resource extraction from a common source with diffusive dynamics. In the symmetric version with identical extraction rates we provide conditions for the existence of a Nash equilibrium where the strategies are of threshold type, and we characterize the equilibrium threshold. Moreover, we show that increased competition leads to lower extraction thresholds and smaller equilibrium values. For the asymmetric version, where each agent has an individual extraction rate, we provide the existence of an equilibrium in threshold strategies, and we show that the corresponding thresholds are ordered in the same way as the extraction rates.


Introduction
With roots in the classical Faustmann problem of forest rotation (for modern references, see [1,2,19]), the research on resource extraction problems in random environments is vast.In this literature, a tradeoff between profitable resource extraction and sustainability is an underlying theme: a business wants to extract resources at a reasonable rate to gain profit, while a too aggressive extraction strategy may lead to extinction.Many articles focus on the single-player problem, where a single agent extracts resources from a stochastically fluctuating population (see [7,9,13,14]).The problem is closely connected with the dividend problem in finance, also known as the De Finetti problem (see [3,20]).However, there are also application specific differences in the model set-ups, such as different dynamics used for the underlying process, different boundary conditions at extinction/default, differences regarding whether exerting control is costly or not, and whether the extraction rate is controlled directly or indirectly via an effort control.
In many applications, however, several agents are present and thus strategic considerations need to be taken into account.In [10,18], the specific case of competitive resource extraction from a common pool of fish is considered in particular diffusion models, and with specific choices of market prices and extraction costs.In such a framework, [10] obtains an explicit feedback Nash equilibrium in which extraction rates are proportional to the current stock of biomass.In an extended model allowing for a two-species fish population with interaction, [18] shows that the corresponding results also hold in higher dimensions.Related literature includes [8,11] where cooperative equilibria are discussed, and [16] in which a myopic setting is used.On the financial side, the reference that is closest in spirit that we have found is [21], in which a non-zero-sum game between insurance companies is studied where the control of each company is a dividend flow as in the De Finetti problem.

Mathematical setup
On a family of complete filtered probability spaces {(Ω, F, (F t ) t≥0 , P x ), x ≥ 0} satisfying the usual conditions we consider a one-dimensional process X = (X t ) t≥0 -corresponding to the total value of a scarce resource -given under P x by where µ and σ are given functions (satisfying certain conditions, see Assumption 2.2 below), B = (B t ) t≥0 is a one-dimensional Wiener process, and λ i = (λ it ) t≥0 , i = 1, ..., n are non-negative controls that satisfy λ it ≤ K at all times, where K > 0 is a given maximal extraction rate.In view of the given Markovian structure, we focus on Markovian controls of the type λ it = λ i (X t ), where λ i : [0, ∞) → [0, K] is measurable.We refer to such a Markovian control as an admissible (resource extraction) strategy, and an n-tuple λ = (λ 1 , ..., λ n ) of admissible strategies is said to be an admissible strategy profile.We note that the conditions on the coefficients µ and σ specified in Assumption 2.2 below guarantee that (2.1) has a unique strong solution for any given admissible strategy profile, cf.[12,Proposition 5.5.17].
For a given admissible strategy profile λ, the reward of each agent i is given by where r > 0 is a constant discount rate, τ := inf{t ≥ 0 : X t ≤ 0} is the extinction time and In this setting we allow each agent i to select the strategy λ i .Naturally, each agent i seeks to make this choice in order to maximize the reward (2.2), and we define a Nash equilibrium accordingly.Note that while our main emphasis is on the strictly competitive case n ≥ 2, our results also hold for the one-player game with n = 1.In that case, the strategic element is eliminated, and the set-up reduces to a classical problem of stochastic control.
Assumptions and preliminaries: Throughout the paper we rely on the following standing assumption (the only exception is Lemma 2.3, where we only assume (A.1)-(A.3)).
For any fixed constant A ≥ 0 we define the downwards shifted drift function and we denote by ψ A (ϕ A ) a positive and increasing (decreasing) solution to Then, ψ A and ϕ A are unique up to multiplication with positive constants, cf.[5, p. 18-19], and C 3 ([0, ∞)).We use the normalization given by ψ ′ A (0) = 1 and ϕ A (0) = 1.If A = 0 so that µ A = µ, then we for notational convenience write ψ and ϕ instead of ψ 0 and ϕ 0 .
To rule out b * = ∞, note that in that case ψ is concave on [0, ∞), so > 0 so that b * = 0 also in this case.
Remark 2.4.It is well-known that the inflection point b * is the level at which X should be reflected in the single-agent version (n = 1) of the game described above when allowing for singular extraction rates.Specifically, consider the problem an agent would face if she were alone (n = 1) and could instead of t 0 λ 1t ds, cf.(2.1), select any non-decreasing adapted RCLL process Λ (satisfying admissibility in the sense that Λ 0− = 0 and X τ ≥ 0) in order to maximize the reward E x τ 0 e −rt dΛ t , (cf.(2.2)); see [17] for the original formulation.In the present setting, this problem has the value function

A threshold Nash equilibrium
The main objective of the present paper is to find and study Nash equilibria (Definition 2.1) of threshold type, i.e.Nash equilibria where each extraction rate λ i has the form λ it = KI {Xt≥b i } for some constant b i ≥ 0 and all t ≥ 0.
With a slight abuse of notation we write a strategy profile comprising only threshold strategies as λ = (b 1 , ..., b n ).

Deriving a candidate Nash equilibrium
In this section we derive a candidate Nash equilibrium of threshold type.We remark that this section is mainly of motivational value and that a verification result is reported in Section 3.2.
Since the agents are identical it is natural to search for a symmetric Nash equilibrium, i.e. an equilibrium of the kind λ = ( λ, ..., λ) for some admissible strategy λ = ( λt ) t≥0 .Clearly, if λ is such a symmetric equilibrium, then the corresponding equilibrium values are all identical, and we will denote this shared equilibrium value function by V = {V (x), x ≥ 0}.Now, if agent n (who is singled out without loss of generality) deviates from such an equilibrium by using an alternative strategy λ, then the corresponding state process is given by For λ to be a Nash equilibrium it should be the case that setting λ = λ in (3.1) maximizes the corresponding reward for agent n and that this strategy gives the equilibrium value V .Hence, from standard martingale arguments it follows that V should satisfy for all λ ∈ [0, K] and x > 0, and that equality in (3.2) should be obtained when λ = λ, i.e.
i.e. (i) each agent extracts resources according to the maximal extraction rate K when V ′ (x) < 1, (ii) no resources are extracted when V ′ (x) > 1.Since we seek a Nash equilibrium of threshold type we now make the Ansatz λ(x) = KI {x≥ b} for some threshold b ≥ 0. In this case it should hold, cf.(3.3), that as well as V ′ ( b) = 1 in case b > 0. We treat the cases b = 0 and b > 0 separately.Suppose first that b = 0. Solving (3.6) then yields V (x) = C 1 ψ nK (x) + C 2 ϕ nK (x) + K r for constants C 1 and C 2 .Imposing V (∞) = K/r (corresponding to maximal extraction for an infinite time) and Moreover, in view of (3.4), we find that V ′ (x) ≤ 1 must hold for all x ≥ 0. In view of the convexity of ϕ nK , we conclude that the case b = 0 requires that then we expect that b > 0 and a corresponding equilibrium value function of the form for some constants D i , i = 1, ..., 4 and b to be determined.Imposing the boundary conditions V (0) = 0 and V (∞) = K/r gives D 2 = 0 and D 3 = 0. We also make the Ansatz that V (x) is differentiable at b, which determines D 1 and D 4 .We thus obtain where and Finally, in line with the arguments above, we determine b by imposing the condition Differentiating (3.10) thus gives the equation which can be rewritten as

Proof. Define the function
and note that f (0) = −1/ϕ ′ nK (0) < K/r by (3.9).Thus any solution to the equation where we have used the ODEs that ϕ nK and ψ satisfy and, in the last inequality, that ϕ nK is convex so that It follows that f is increasing as long as f ≤ nK/r.Consequently, the equation f ( b) = K/r has at most one positive solution.Moreover, by similar calculations as in (3.16), so at the inflection point b * of ψ we have Comparing with (3.16) we find that In both cases with D 1 and D 4 specified in (3.11)-(3.12).Note that ).The concavity of V thus follows from convexity of ϕ nK .If instead (3.9) holds, then the claim is verified using (3.17),Lemma 3.1 and Lemma 2.3.b) is a Nash equilibrium and the corresponding equilibrium value functions are identical and given by V defined in (3.17).

A verification theorem for the threshold Nash equilibrium
Proof.Suppose an arbitrary agent i deviates from λ = ( b, ..., b).More precisely, consider the strategy profile (λ, λ−i ) (cf.Definition 2.1), where λ is an arbitrary admissible strategy.Then X is given by Using Itô's formula we obtain where θ t := inf{s ≥ 0 : X s ≥ t} ∧ t (with the standard convention that the infimum of the empty set is infinite).It is directly verified, cf.(3.5)-(3.6),that for all x = b, which implies that In case (3.8) holds, we have . By concavity of V (cf.Lemma 3.2) we thus obtain Similarly, if (3.9) holds, then it follows by concavity of V and V ′ ( b) = 1 that (3.18) holds.Hence Taking expectation yields where we used that V ′ (•) and σ(•) are bounded on the stochastic interval [0, θ t ] so that the expectation of the stochastic integral vanishes.Since V ≥ 0, using monotone convergence we find that Furthermore, using the threshold strategy KI {Xt≥ b} instead of λ t in the calculations above yields so V (0) = 0 and dominated and monotone convergence (using concavity of V for the domination) yield Consequently, λ = ( b, ..., b) is a Nash equilibrium.

Properties of the threshold Nash equilibrium
To understand the effect of competition, we provide a short study of the dependence on the number n of agents for the equilibrium threshold and for the corresponding equilibrium value.Our first result in this direction shows that increased competition lowers the equilibrium threshold and decreases each agent's equilibrium value.Proof.First note that ϕ ′ nK (0) is increasing in n, so there exists n such that condition (3.8) holds if n ≥ n (so that b = 0 for such n), and (3.9) holds if n < n.Moreover, To show that V is decreasing in n, let m ≤ n and denote by bm , bn and V m , V n the corresponding thresholds and equilibrium values, respectively.By the above, bm ≥ bn .
As a consequence of (3.19), we also have with boundary conditions u( bm ) ≥ 0 and u(∞) = 0.The maximum principle then yields u ≥ 0 on [ bm , ∞).Consequently, V m ≥ V n everywhere, which completes the proof.
Intuitively, the effect in Theorem 3.4 of adding more agents of the same type is negative as it both adds competition and also increases the maximal total push rate nK (while the individual push rate K is constant).In the following result we isolate the effect of increased competition by studying the case with a fixed maximal total push rate K. Proof.The function f in (3.15) is in this case given by f , which is independent of n, and the threshold b = b(n) is given as the unique solution of f ( b) = K/r = K/(rn) provided ϕ ′ K (0) < −rn/K, and b(n) = 0 otherwise.Since f is increasing as long as f < K/r (cf. the proof of Lemma 3.1), the solution of f ( b(n)) = K/(nr) is decreasing in n.It remains to show that nV (x) decreasing.Recall that if (3.9) holds, and b(n) = 0 otherwise.Moreover, .
In order to show that nV (x; n) is decreasing in n we now show that nD 1 (n) and nD 4 (n) are decreasing in n.We begin with nD 1 (n).Since b(n) is decreasing in n (Theorem 3.4), it suffices to show that However, for x ∈ (0, b( 1)) (cf. the proof of Lemma 3.1), so nD 1 (n) is decreasing in n.For nD 4 (n) we want to show for x ∈ (0, b(1)), that Hence, also in this case (3.20) gives us what is needed.Now, since nD 1 (n) and nD 4 (n) are decreasing in n, it follows that mV (x; m) ≥ nV principle gives H ≥ 0, so mV (x; m) ≥ nV (x; n) everywhere.
Remark 3.6.In the single-agent game, the optimal threshold (given by b = b(K) determined with n = 1) can be shown to be increasing as a function of the maximal extraction rate K. Indeed, implicit differentiation of the relation where the inequality is a consequence of ψ ′′ ( b) ≤ 0. In the competitive setting of the current paper (where we may have n ≥ 2), however, the effect of an increase in the extraction rate is ambiguous as it facilitates fast extraction not only for you but also for your competitors.We therefore do not expect any monotone relationship between b and K; see Figure 3 for an illustration of this relationship in the case of constant coefficients.

The case of constant coefficients
Consider the case of constant drift and diffusion coefficients µ > 0 and σ > 0. In this case + 2r σ 2 < 0. Let us use Theorem 3.3 to find the Nash equilibrium for this model.First note that condition (3.9) is equivalent to

Individual extraction rates
In this section we consider the asymmetric case when maximal extraction rates are not necessarily identical.In particular, a Markovian control λ it = λ i (X t ) is in this section an admissible strategy for agent i if it takes values in [0, K i ], where K 1 , ..., K n are given positive constants.Again we will search for Nash equilibria of threshold type, which in this section means that for all i = 1, ..., n we have for some constant b i ≥ 0 and all t ≥ 0.
In Section 4.1 we report a verification type result for the asymmetric game.In Section 4.2 we establish the existence of a Nash equilibrium of threshold type using a fixed-point argument.The corresponding thresholds are shown to be ordered in the same way as the maximal extraction rates.

A single-agent problem and a verification result
By definition a threshold strategy ( b1 , ..., bn ) is Nash equilibrium if the threshold strategy corresponding to bi is optimal (in the usual sense) for any agent i in case the other agents use the fixed threshold strategy ( b1 , ..., bi−1 , bi+1 , ..., bn ) (we use this notation to mean the threshold strategy obtained when removing the threshold of agent i also in case i = 1 or i = n).Thus, all we need in order to verify that a proposed threshold equilibrium is indeed an equilibrium is to solve a single-agent version of the problem investigated in the previous section, but with the complication of a drift function with negative jumps; in particular, the problem of selecting an admissible strategy λ i = (λ it ) t≥0 for agent i in in order to maximize the reward In line with Section 3.1, we thus denote by and by with the boundary specifications Note that these functions are analogous to ψ and ϕ nK of the previous section, but due to the discontinuities of the drift function we now merely have that However, since the jumps of the drift are negative, we may use approximation by drift coefficients for which (A.1) and (A.3) are valid.More specifically, consider drift coefficients {µ k } ∞ k=1 satisfying (A.1) and (A.3), and with µ k ↑ µ − j =i K j I {•≥b j } pointwise.We then see that the conclusions of Lemma 2.3 still hold.In fact, the corresponding fundamental solutions converge pointwise from above to ψ (i) and ϕ (i) , as k → ∞.Using that the pointwise limit of concave/convex functions is again concave/convex, Lemma 2.3 extends as follows.(i) Each function In practice, each ψ (i) can be found by (i) identifying the fundamental increasing and decreasing solutions to (4.3) separately on the intervals on which the drift function does not jump, (ii) on each interval forming linear combinations of the increasing and decreasing fundamental solutions, (iii) pasting the linear combinations together to form a function R + → R, (iv) choosing the constants of the linear combinations so that the resulting R + → R function is continuously differentiable and the boundary conditions are satisfied; this function is then ψ (i) .We can similarly obtain ϕ (i) .Remark 4.3.As in Remark 2.4, the inflection point b * * i corresponds to the optimal barrier policy in case the maximal push rate of agent i is K i = ∞.
Relying on Lemma 4.1, the reasoning in Section 3.1 extends to the present case.In particular, by relying on a single-agent version of Theorem 3.2 for the present setting, we can verify if it is optimal for any agent i to comply with a proposed threshold equilibrium.We thus obtain the following verification type result.Theorem 4.4.Suppose a threshold strategy ( b1 , ..., bn ) satisfies and .
We will now report a boundedness and continuity result that is essential for the fixed-point argument in Section 4.
for i = 1, ..., n.The interpretation is that b Z i is the optimal threshold strategy for agent i under the assumption that the other agents use a fixed threshold strategy (b 1 , ..., b i−1 , b i+1 , ..., b n ).We remark that a more explicit interpretation of the mappings b Z i can be extracted from the proof below.
is zero for y = 0 and increasing on [0, x], so ψ (n) (x) ≥ xψ (n) ′ (x) for such x.This implies that By Assumption 2.2(A.3), the last expression above is positive whenever x ≥ c; thus, in order for ψ (where E 1 and E 4 are defined so that Z is C 1 at b Z n , i.e. analogously to the constants in Theorem 4.
Note that this is a continuous mapping from K to K and with Brouwer's fixed-point theorem we conclude that it has a fixed-point ( b1 , ..., bn ).Clearly, the threshold strategy profile ( b1 , ..., bn ) is then a Nash equilibrium and the corresponding equilibrium value functions are given by (4.5).
For the ordering result, it suffices to consider the case n = 2 since the strategies of the remaining agents can be considered already locked in.Assume that K 1 ≤ K 2 and consider a threshold type equilibrium ( b1 , b2 ) as obtained above.Now assume (to reach a contradiction) that b1 > b2 .If b2 > 0, then Using strict concavity of ψ (on the interval (0, b2 )) and A similar reasoning shows that (4.9) also holds in the case 0 = b2 < b1 .Define It follows that b1 ≤ b2 .Now that b1 ≤ b2 , a similar reasoning as above shows that V 1 ≤ V 2 on (0, b2 ).Since

The case of constant coefficients
We consider two agents i = 1, 2 with maximal extraction rates K 2 ≥ K 1 > 0 and constant drift and diffusion coefficients µ > 0 and σ > 0. Then existence of a threshold equilibrium ( b1 , b2 ) satisfying b2 ≥ b1 ≥ 0 follows from Theorem 4.6.Hence, an equilibrium of either of the following kinds can be found (i) ( b1 , b2 ) = (0, 0), (ii) (0, b2 ) with b2 > 0, and (iii) ( b1 , b2 ) with b2 ≥ b1 > 0. We will now describe how to determine if an equilibrium of the last kind exists and how it can be found.Equilibria of the other kinds can be investigated similarly.
Using the program in Remark 4.2 we find that for some constant C > 0 implying that the boundary condition ϕ (2) (0) = 1 is satisfied (there is no need to determine these functions outside the specified intervals and the constant C will cancel in the relevant calculations); here α + 2r σ 2 , and the constants F i (which depend on b 1 and b 2 ) are determined so that the functions are continuously differentiable, i.e. according to the linear equation systems  From Theorem 4.4 we know that if the system V ′ i ( b1 ) = E 1,i ψ (i) ′ ( bi ) = 1, i = 1, 2, (with E 1,i as in (4.6)) has a solution ( b1 , b2 ) then it is an equilibrium.Using the definition of E 1,i in (4.6), we rewrite the equation system as which, in the present case with constant coefficients, is equivalent to  Determining expressions for F 1 ( b1 ) and F 2 ( b1 ) (using (4.12)), using these in the second equation of The problem of finding a Nash equilibrium ( b1 , b2 ) is thus effectively transformed to a one-dimensional problem.Indeed, relying on (4.13) we find e α 1 b2 (α 1 − β 2 ) − e β 1 b2 (β 1 − β 2 ) .
This expression together with (4.15) induces an explicit mapping b1 → F 3 .Viewing F 3 in the first line of (4.14) as this mapping gives us an explicit equation for b1 (which for brevity is not included), which can be easily studied numerically; if it has a solution b1 , then ( b1 , b2 ), with b2 = b2 ( b1 ) determined according to (4.15), is an equilibrium; if no solution exists than an equilibrium of either of the other kinds mentioned above can be found using methods similar to that described above.
The corresponding equilibrium value functions are illustrated in Figure 4.In Figure 5, the equilibrium thresholds are illustrated when varying one of the maximal extraction rates.At least for the chosen parameter values, the effect of a change in K 2 on b1 is relatively small.Right: The equilibrium threshold b2 (K 2 ) when K 2 is varied and K 1 = 0.1 (solid).For comparison we have also included the optimal thresholds in the corresponding one-player game (dashed), which are determined according to Section 3.4.The straight line is the optimal barrier in the corresponding one-player singular stochastic control problem, cf.Remark 2.4.
and the optimal policy is to reflect the state process X at a barrier consisting of the inflection point b * , with an immediate dividend of x − b * in case x > b * ; see[17, Theorem 4.3]  and[6, Proposition 2.6].
which implies that f (b * ) ≥ nK/r.Thus, by continuity there exists a point b ∈ (0, b * ] such that f ( b) = K/r, which completes the proof.Note that if b = 0, then D 4 = −K/r and the representation of V in (3.10) yields (3.7).Let us summarize the above discussion by specifying the candidate equilibrium threshold b and the corresponding equilibrium value function V .If (3.8) holds, then we define b = 0.If instead (3.9) holds, then we define b as the unique positive solution of (3.14).

Theorem 3 . 4 .
(The effect of increased competition.)With all other parameters being held fixed, the threshold b = b(n) in the symmetric threshold strategy λ = ( b, ..., b) is decreasing in n.In fact, there exists a number n such that b is decreasing for n < n and b = 0 for n ≥ n.Moreover, the equilibrium value V (x) = V (x; n) is decreasing in n.

Theorem 3 . 5 .
(The effect of increased competition with constant total extraction rate.)Let the maximal individual extraction rate be K = K/n (so that the total maximal extraction rate K is independent of n).Then the equilibrium threshold b = b(n) and the total equilibrium value nV (x) are both decreasing in n.

. 21 )Figure 1 Figure 2 Figure 3
Figure 1 Left: The individual equilibrium value function V (x) (same for each agent) when the number of competitors n is varied and the individual maximal extraction rate K is fixed.Right: The total equilibrium value function nV (x) when n is varied and the total maximal extraction rate K is fixed.(Larger n correspond to lower equilibrium values.)

Proposition 4 . 5 .
For each i = 1, ..., n, the mapping (4.7) is bounded by c and continuous.Proof.Suppose without loss of generality that i = n.Let us first show that the function (b 1 , ..., b n−1 ) → b * * n is bounded.Consider any x > 0 with x = b i for all i < n and suppose ψ (n) ′′ (x) < 0. The function (n) ′′ (x) < 0 to hold we must have x < c.Consequently, b * * n ≤ c, and since c does not depend on b 1 , ..., b n−1 , it follows that (b 1 , ..., b n−1 ) → b * * is bounded.It can now be seen (cf. the single-agent, n = 1, version of Theorem 3.2), that the optimal value function corresponding to b Z n = b Z n (b 1 , ..., b n−1 ) is 4) and bZ n ∈ (0, b * * n ] is the unique threshold satisfying Z ′ (b Z n ) = 1 in case ϕ (n) ′ (0) < − r K n and b * * n > 0 hold,and b Z n = 0 otherwise.We thus find (i) since b Z n ≤ b * * n , it holds that b Z n is bounded, and (ii) since the solutions ψ (n) and ϕ (n) to the ODEs (4.3) and (4.4) (with i = n) depend continuously on the parameters b 1 , ..., b n−1 , it holds that the continuity of b Z n can be obtained from the explicit defining relations above, see e.g.(4.8).

Proof.
For any fixed i, the mapping b Z i defined in (4.7) produces the optimal threshold strategy for agent i under the assumption that the other agents use a fixed threshold strategy (b 1 , ..., b i−1 , b i+1 , ..., b n ).Recall that each b Z i is a bounded and continuous mapping by Proposition 4.5; in fact, b Z i ≤ c for all i and (b 1 , ..., b n ).Define the hyper-cube K = [0, c] n and the composite mapping