Nonzero-Sum Games of Optimal Stopping for Markov Processes

Two players are observing a right-continuous and quasi-left-continuous strong Markov process X. We study the optimal stopping problem Vσ1(x)=supτMx1(τ,σ)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$V^{1}_{\sigma }(x)=\sup _{\tau } \mathsf {M}_{x}^{1}(\tau ,\sigma )$$\end{document} for a given stopping time σ\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma $$\end{document} (resp. Vτ2(x)=supσMx2(τ,σ)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$V^{2}_{\tau }(x)=\sup _{\sigma } \mathsf {M}_{x}^{2}(\tau ,\sigma )$$\end{document} for given τ\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\tau $$\end{document}) where Mx1(τ,σ)=Ex[G1(Xτ)I(τ≤σ)+H1(Xσ)I(σ<τ)]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathsf {M}_{x}^{1}(\tau ,\sigma ) = \mathsf {E}_{x} [G_{1}(X_{\tau })I(\tau \le \sigma ) + H_{1}(X_{\sigma })I(\sigma < \tau )]$$\end{document} with G1,H1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$G_1,H_1$$\end{document} being continuous functions satisfying some mild integrability conditions (resp. Mx2(τ,σ)=Ex[G2(Xσ)I(σ<τ)+H2(Xτ)I(τ≤σ)]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathsf {M}_{x}^{2}(\tau ,\sigma ) = \mathsf {E}_{x} [G_{2}(X_{\sigma })I(\sigma < \tau ) + H_{2}(X_{\tau })I(\tau \le \sigma )]$$\end{document} with G2,H2\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$G_2,H_2$$\end{document} being continuous functions satisfying some mild integrability conditions). We show that if σ=σD2=inf{t≥0:Xt∈D2}\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma = \sigma _{D_{2}} = \inf \{t \ge 0: X_t \in D_2\}$$\end{document} (resp. τ=τD1=inf{t≥0:Xt∈D1}\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\tau = \tau _{D_{1}} = \inf \{t \ge 0: X_t \in D_1\}$$\end{document}) where D2\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$D_{2}$$\end{document} (resp. D1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$D_1$$\end{document}) has a regular boundary, then VσD21\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$V^{1}_{\sigma _{D_{2}}}$$\end{document} (resp. VτD12\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$V^{2}_{\tau _{D_{1}}}$$\end{document}) is finely continuous. If D2\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$D_{2}$$\end{document} (resp. D1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$D_1$$\end{document}) is also (finely) closed then τ∗σD2=inf{t≥0:Xt∈D1σD2}\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\tau _*^{\sigma _{D_2}} = \inf \{t \ge 0: X_{t} \in D_{1}^{\sigma _{D_{2}}}\}$$\end{document} (resp. σ∗τD1=inf{t≥0:Xt∈D2τD1}\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{*}^{\tau _{D_1}} = \inf \{t \ge 0: X_{t} \in D_{2}^{\tau _{D_{1}}}\}$$\end{document}) where D1σD2={VσD21=G1}\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$D_{1}^{\sigma _{D_{2}}} = \{V^{1}_{\sigma _{D_{2}}} = G_{1}\}$$\end{document} (resp. D2τD1={VτD12=G2}\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$D_{2}^{\tau _{D_{1}}} = \{V^{2}_{\tau _{D_{1}}} = G_{2}\}$$\end{document}) is optimal for player one (resp. player two). We then derive a partial superharmonic characterisation for VσD21\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$V^{1}_{\sigma _{D_2}}$$\end{document} (resp. VτD12\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$V^{2}_{\tau _{D_1}}$$\end{document}) which can be exploited in examples to construct a pair of first entry times that is a Nash equilibrium.


Introduction
Optimal stopping games, often referred to as Dynkin games, are extensions of optimal stopping problems. Since the seminal paper of Dynkin [14], optimal stopping games have been studied extensively. Martingale methods for zero-sum games were studied by Kifer [32], Neveu [44], Stettner [55], Lepeltier and Maingueneau [41] and Ohtsubo [45]. The Markovian framework was initially studied by Frid [22], Gusein-Zade [26], Elbakidze [18] and Bismut [5]. Bensoussan and Friedman [2] and Friedman [20,21] considered zero-sum optimal stopping games for diffusions and developed an analytic approach by relying on variational and quasi-variational inequalities. Ekström and Peskir [16] proved the existence of a value in two-player zero-sum optimal stopping games for right-continuous strong Markov processes and construct a Nash equilibrium point under the additional assumption that the underlying process is quasi-left continuous. Peskir in [51] and [52] extended these results further by deriving a semiharmonic characterisation of the value of the game without assuming that a Nash equilibrium exists a priori. In particular, a necessary and sufficient condition for the existence of a Nash equilibrium is that the value function coincides with the smallest superharmonic and the largest subharmonic function lying between the gain and the loss function which, in the case of absorbed Brownian motion in [0,1], is equivalent to 'pulling a rope' between 'two obstacles' (that is finding the shortest path between the graphs of two functions). Connections between zero-sum optimal stopping games and singular stochastic control problems were studied in [23,31] and [4]. Cvitanic and Karatzas [11] showed that backward stochastic differential equations are connected with the value function of a zero-sum Dynkin game. Advances in this direction can be found in [27]. Various authors have also studied zero-sum optimal stopping games with randomised strategies. For further details one can refer to [39] and the references therein. Zero-sum optimal stopping games have been used extensively in the pricing of game contingent claims both in complete and incomplete markets (see for example [15,17,24,25,30,33,35,36,38] and [19]).
Literature on nonzero-sum optimal stopping games is mainly concerned with the existence of a Nash equilibrium. Initial studies in discrete time date back to Morimoto [42] wherein a fixed point theorem for monotone mappings is used to derive sufficient conditions for the existence of a Nash equilibrium point. Ohtsubo [46] derived equilibrium values via backward induction and in [47] the same author considers nonzero-sum games in which the lower gain process has a monotone structure, and gives sufficient conditions for a Nash equilibrium point to exist. Shmaya and Solan in [54] proved that every two player nonzero-sum game in discrete time admits an ε-equilibrium in randomised stopping times. In continuous time Bensoussan and Friedman [3] showed that, for diffusions, a Nash equilibrium exists if there exists a solution to a system of quasi-variational inequalities. However, the regularity and uniqueness of the solution remain open problems. Nagai [43] studies a nonzero-sum stopping game of symmetric Markov processes. A system of quasi-variational inequalities is introduced in terms of Dirichlet forms and the existence of extremal solutions of a system of quasivariational inequalities is discussed. Nash equilibrium points of the stopping game are then obtained from these extremal solutions. Cattiaux and Lepeltier [8] study special right-processes, namely Hunt processes in the Ray topology, and they prove existence of a quasi-Markov Nash Equilbrium. The authors follow Nagai's idea but use probabilistic tools rather than the theory of Dirichlet forms. Huang and Li in [29] prove the existence of a Nash equilibrium point for a class of nonzero-sum noncyclic stopping games using the martingale approach. Laraki and Solan [40] proved that every two-player nonzero-sum Dynkin game in continuous time admits an ε−equilibrium in randomised stopping times. Hamadène and Zhang in [28] prove existence of a Nash equilibrium using the martingale approach, for processes with positive jumps. One application of nonzero-sum optimal stopping games is seen in the study of game options in incomplete markets, via the consideration of utility-based arguments (see [34]). Nonzero-sum optimal stopping games have also been used to model the interaction between bondholders and shareholders in the study of convertible bonds, when corporate taxes are included and when the company is allowed to claim default (see [9]).
In this work we consider two player nonzero-sum games of optimal stopping for a general strong Markov process. The aim is to use probabilistic tools to study the optimal stopping problem of player one (resp. player two) when the stopping time of player two (resp. player one) is externally given. Although this work does not deal with the question of existence of mutually best responses (that is the existence of a Nash equilibrium) the results obtained can be exploited further in various examples, to show the existence of a pair of first entry times that will be a Nash equilibrium. Indeed, the results derived here will be used in a separate work (see [1]) to construct Nash equilibrium points for one dimensional regular diffusions and for a certain class of payoff functions. This paper is organised as follows: In Sect. 2 we introduce the underlying setup and formulate the nonzero-sum optimal stopping game. In Sect. 3 we show that if the strategy chosen by player two (resp. player one) is σ D 2 (resp. τ D 1 ), the first entry time into a regular Borel subset D 2 (resp. D 1 ) of the state space, then the value function of player one associated with σ D 2 (resp. the value function of player two associated with τ D 1 ), which we shall denote by V 1 ), is finely continuous. In Section 4 we shall use this regularity property of V 1 ) to construct an optimal stopping time for player one (resp. player two). In Sect. 5 we shall use the results obtained in Sects. 3 and 4 to provide a partial superharmonic characterisation for V 1 ). More precisely if D 2 (resp. D 1 ) is also a closed or finely closed subset of the ) can be identified with the smallest finely continuous function that is superharmonic in D c 2 (resp. in D c 1 ) and that majorises the lower payoff function. In Section 6 we shall consider stationary one-dimensional Markov processes and we shall assume that there exists a pair of stopping times (τ A * , σ B * ) of the form that is a Nash equilibrium point. We first show that V 1 σ B * (resp. V 1 τ A * ) is continuous at A * (resp. at B * ). Then for the special case of one dimensional regular diffusions we shall use the results obtained in Sect. 5 to show that V 1 σ B * (resp. V 2 τ A * ) is also smooth at A * (resp. B * ) provided that the payoff functions are smooth. This is in line with the principle of smooth fit observed in standard optimal stopping problems (see for example [49] for further details).

Formulation of the Problem
In this section we shall formulate rigorously the nonzero-sum optimal stopping game. For this we shall first set up the underlying framework. This will be similar to the one presented by Ekström and Peskir (cf. [16, p. 3]). On a given filtered probability space , F, (F t ) t≥0 , P x we define a strong Markov process X = (X t ) t≥0 with values in a measurable space (E, B), with E being a locally compact Hausdorff space with a countable base (note that since E has a countable base then it is a Polish space) and B the Borel σ -algebra on E. We shall assume that P x (X 0 = x) = 1, that the sample paths of X are right-continuous and that X is quasi-left-continuous (that is X ρ n → X ρ P x -a.s. whenever ρ n and ρ are stopping times such that ρ n ↑ ρ P x -a.s.). All stopping times mentioned throughout this text are relative to the filtration (F t ) t≥0 introduced above, which is also assumed to be right-continuous. This means that entry times in open and closed sets are stopping times. Moreover F 0 is assumed to contain all P xnull sets from F X ∞ = σ (X t : t ≥ 0), which further implies that the first entry times to Borel sets are stopping times. We shall also assume that the mapping Finally we shall assume that is the canonical space E [0,∞) with X t (ω) = ω(t) for ω ∈ . In this case the shift operator θ t : → is well defined by θ t (ω)(s) = ω(t+s) for ω ∈ and t, s ≥ 0.
The Markovian version of a nonzero-sum optimal stopping game may now be formally described as follows. Let G 1 , G 2 , H 1 , H 2 : E → R be continuous functions satisfying G i ≤ H i and the following integrability conditions; for i = 1, 2. Suppose that two players are observing X . Player one wants to choose a stopping time τ and player two a stopping time σ in such a way as to maximise their total average gains, which are respectively given by For a given strategy σ chosen by player two, we let and for a given strategy τ chosen by player one, we let We shall refer to V 1 σ (resp. V 2 τ ) as the value function of player one (resp. player two) associated with the given stopping time σ (resp. τ ) of player two (resp. player one). We shall assume that the stopping times in (2.4) and (2.5) are finite valued and if the terminal time T is finite we shall further assume that for i = 1, 2. In this case one can think of X as being a two dimensional process ((t, Y t )) t≥0 so that G i and H i will be functions on [0, T ] × E (cf. [49, p. 36]).
(We note that if the terminal time T is infinite and the stopping times τ and σ are allowed to be infinite our results will still be valid provided that lim sup The game is said to have a solution if there exists a pair of stopping times (τ * , σ * ) which is a Nash equilibrium point, that is M 1 x (τ * , σ * ) for all stopping times τ, σ . This means that none of the players will perform better if they change their strategy independent of each other. In this case is the payoff function of player one and V 2 τ * (x) = M 2 x (τ * , σ * ) the payoff function of player two in this equilibrium. So V 1 σ * and V 2 τ * can be called the value functions of the game (corresponding to (τ * , σ * )). In general, as we shall see in Sect. 6, there might be other pairs of stopping times that form a Nash equilibrium point, which can lead to different value functions.

Fine continuity property
In this section we show that if the strategy chosen by player two (resp. player one) corresponds to the first entry time into a subset D 2 (resp. D 1 ) of E, whose boundary ∂ D 2 (resp. ∂ D 1 ) is regular, then V 1 ) is continuous in the fine topology (i.e. finely continuous). For literature on the fine topology one can refer to [6,10] and [13]. We first define the concept of a finely open set and a regular boundary of a Borel subset of E.  We now introduce preliminary results which are needed to prove the main theorem of this section.

Lemma 3.3 For any given stopping time
Proof To prove measurability of the mapping x → V 2 τ (x) one can follow the proof in [16, p. 5, pt. 3] by replacing G 1 and G 2 with G 2 and H 2 respectively (note that our payoff functions are assumed to be continuous, hence finely-continuous). So we shall only prove the result for V 1 σ . Ekström and Peskir [16, p. 5, pt. 3] proved that for any given stopping time σ , the functionṼ 1 σ of the optimal stopping problem sup τM is measurable. The same method of proof can be applied in this setting with the following slight modification: Let G σ,1 Note that the mapping t → G σ,1 t is not right-continuous. Now for any stopping time τ in the optimal stopping problem (2.4) we let τ n = k 2 n on { k−1 2 n < τ ≤ k 2 n } for each n ≥ 1. It is well known that τ n , for each n is a stopping time on the dyadic rationals Q n of the form k 2 n and that τ n ↓ τ as n → ∞. Since G 1 (X ) is right-continuous we have that Since G 1 ≤ H 1 we get, upon using (3.1) and Fatou's lemma (the required integrability condition for using Fatou's lemma can be derived from the integrability assumption (2.1)), that Taking the supremum over all τ it follows that V 1 σ (x) follows from the measurability property of sup n≥1 V σ,1 n (x) as in [16].

Lemma 3.4 Let D be a Borel subset of E and let x ∈ ∂ D,
where ∂ D is a regular boundary for D. Suppose that (ρ n ) ∞ n=1 is a sequence of stopping times such that ρ n ↓ 0 P x -a.s. as n → ∞. Set σ ρ n = inf{t ≥ ρ n : X t ∈ D}. Then σ ρ n ↓ 0 P x -a.s. as n → ∞.
Proof Let x ∈ ∂ D. By regularity of ∂ D for any ε > 0 there exists t ∈ (0, ε) such that X t ∈ D P x -a.s. Since σ ρ n is a sequence of decreasing stopping times then σ ρ n ↓ β P x -a.s. for some stopping time β. So suppose for contradiction that β > 0. Now ρ n ↓ 0 P x -a.s. and for each n we have σ ρ n ≥ β P x -a.s. So for any given ω ∈ \N where P x (N ) = 0 we have that X t (ω) / ∈ D for all t ∈ (0, β(ω)) and this contradicts the fact that ∂ D is regular for D.
The next lemma and theorem, which we shall exploit in this study, provide conditions for fine continuity. The proofs of these results can be found in [13].

Lemma 3.5 A measurable function F : E → R is finely continuous if and only if
for every x ∈ E. This is further equivalent to the fact that the mapping Theorem 3.6 Let F : E → R be a measurable function and suppose that then F is finely continuous.
We next state and prove the main result of this section, that is the fine continuity property of V 1 n=1 be any sequence of stopping times such that ρ n ↓ 0 P x -a.s. as n → ∞. Suppose that D 1 , D 2 are Borel subsets of E having regular boundaries ∂ D 1 and ∂ D 2 respectively. Then Proof We will only prove (3.5) as (3.6) follows by symmetry.
is a random variable. By the strong Markov property of X we have that where we set τ ρ n = ρ n + τ • θ ρ n and σ ρ n = ρ n + σ D • θ ρ n . It is well known that τ ρ n and σ ρ n are stopping times (see for example [10, Section 1.3, Thoerem 11]). Let us set for given stopping times τ, σ and ρ. Then from (3.7) and (3.8) we get that The last equality follows from the fact that for every stopping time τ ≥ ρ n , there exists a function τ ρ n : × → [0, ∞] such that for all ω ∈ . Note that the latter assertion can be derived from Galmarino's test. In particular if τ is the first entry time of X into a set, then τ = σ + τ • θ ρ n and τ ρ n can be identified with τ in the sense that τ ρ n (ω, ϑ) = τ (ϑ) for all ω and ϑ. Taking expectations on both sides in (3.10) we get For this we show that for any two stopping times So let τ 1 , τ 2 ≥ ρ n be any two stopping times given and fixed and define the set This follows from the fact that the sets A and A c belong to F ρ n and that {τ i ≤ t} ⊆ {ρ n ≤ t} for i = 1, 2. So τ 3 is a stopping time and hence, For this we first show that Since σ ρ n ≥ ρ n we have that from which (3.17) follows. By considering separately the sets {τ > 0}, and by using Lemma 3.4 we get for n sufficiently large. The last equality in (3.19) can be seen as follows: If x ∈ intD 2 the interior of D 2 then by the right-continuity property of the sample paths it follows that σ ρ n = 0 P x -a.s. for n sufficiently large. If on the other hand x ∈ ∂ D 2 then by Lemma 3.4, we have that σ ρ n ↓ 0 P x -a.s. Note that in the case x / ∈ D 2 ∪ ∂ D 2 then σ D 2 > 0 P x -a.s. and so the terms in the right-hand side of (3.19) vanish. Combining (3.17) and (3.19) we get for n sufficiently large. So The first inequality follows from (3.20). Indeed, since G 1 and H 1 are continuous, the composed processes G 1 (X ) and H 1 (X ) are right-continuous and so both terms on the right hand side of (3.20) tend to zero as n → ∞ (note that if x ∈ ∂ D 2 this follows from Lemma 3.4 whereas if x ∈ intD 2 or x / ∈ D ∪ ∂ D the result follows as explained in the text before (3.20)). The last equality in (3.21) follows from the fact that the family {M x (τ, σ ρ n |F ρ n ) : τ ≥ ρ n } is upwards directed (see step 2 • ), and so we can interchange the expectation and the essential supremum in (3.14).
for any stopping time τ . From this, together with Lebesgue dominated convergence theorem (upon recalling assumption (2.1)) and the continuity property of G 1 and H 1 we conclude that We next present an example to show that if ∂ D is not regular for D then V 1 σ D may not be finely continuous.
Example 3.8 Let E = R and B the Borel σ -algebra on R. Suppose that X is the deterministic motion to the right, that is the process starts at x ∈ R and X t = x + t P xa.s. for each t ≥ 0. In this case the fine topology coincides with the right-topology (cf. [53]), so a function is finely continuous if it is right-continuous. Define the functions s. We show that for any given ε > 0 the stopping time τ ε = (τ [1,∞) [1,∞) = σ D }, is optimal for player one given the strategy σ D chosen by player two. For each ε > 0 we have that . On the other hand, for any stopping time τ one can see that where the last inequality follows from the fact that G 1 is decreasing in [1, ∞). Taking the supremum over all τ we get that for all x and so we must have that τ ε is optimal for player one, provided that player two selects strategy σ D . The value function is thus given by

Towards a Nash equilibrium
The main result of this section is to show that if σ D 2 (resp. τ D 1 ) is externally given as the first entry time in D 2 (resp. D 1 ), a set that is either closed or finely closed, and has a regular boundary, then the first entry time τ The proof of this result will be divided into several lemmas and propositions.
Proof We shall only prove (4.1) as for (4.2) the result follows by symmetry. The proof will be carried out in several steps. 1 • Consider the optimal stopping problem Recall that the mapping x →Ṽ 1 [16, p. 5]) and soṼ 1 is a random variable for any stopping time ρ. By the strong Markov property of X it follows that for any stopping time ρ given and fixed and so we have that is right-continuous, adapted and satisfies the integrability condition where the last inequality follows from assumption (2.1). So the martingale approach in the theory of optimal stopping (cf. [49, Theorem 2.2]) can be applied in this setting to deduce that there exists a right-continuous modification of the supermartingalẽ known as the Snell envelope (for simplicity of exposition we shall still denote the right-continuous modification byS ), such that the stopping timeτ t :=τ for every stopping time ρ ≤τ ε whereτ ε := τ Using the fact that σ ρ D 2 = σ D 2 for any stopping time ρ ≤ σ D 2 , that P x -a.s., the essential supremum and its right-continuous modification are equivalent at stopping times and that the essential supremum is attained at hitting times (cf. [49]) it follows, from (4.6), thatṼ for all stopping times τ and for all x ∈ E. Taking the supremum over all τ we get that To prove the reverse inequality we will show thatM 1 for all stopping times τ . Now take any stopping time τ and set τ ε = (τ + ε) If the time horizon T is finite, then we shall replace τ + ε in the definition of τ ε with (τ + ε) ∧ T ). Then we have that The first and third expressions in the second equality follow from assumption (2.6). The second expression also follows from assumption (2.6) together with the fact that I (τ ε < σ D 2 ) = I (τ < σ D 2 ) on the set {τ < T }. The last expression follows from the fact that I (τ ε = σ D 2 ) = 0 and I (σ D 2 < τ ε ) = 1 A + I (σ D 2 < τ) on the set {τ < T } ∩ {σ D 2 < T }. So for any given stopping time τ we have V 1 where the first equality follows from the fact that I (σ D 2 ≤ t) = 0. Since σ D 2 ∧τ ε ≤ σ D 2 , by (4.9) it follows that Using (4.9) again we getṼ So by (4.12) we can conclude that By using the definition ofτ ε together with (4.15) one can see that σ D 2 ∧τ ε ≤τ ε . By using (4.8), (4.9) and that (S From (4.16) together with the fine continuity property of V 1 σ D 2 (upon using Lemma 3.5), the right-continuity property of the composite process G(X ) and the fact that V 1 for any ε > 0, where the third equality follows from the fact that Lemma 4.2 Let {ρ n } ∞ n=1 be a sequence of stopping times such that ρ n ↑ ρ P x -a.s. For a given Borel set D ⊆ E define the entry times σ ρ n = inf{t ≥ ρ n : X t ∈ D} and σ ρ = inf{t ≥ ρ : X t ∈ D}. If either D is closed, or (4.19) D is finely closed with regular boundary ∂ D, (4.20) then σ ρ n ↑ σ ρ P x -a.s.

Proposition 4.3 Let D 1 , D 2 be either closed or finely closed subsets of E.
Suppose also that their respective boundaries ∂ D 1 and ∂ D 2 are regular. Let τ D 1 and σ D 2 be the first entry times into D 1 and D 2 respectively. Set τ Proof We shall only prove that τ s. as the other assertion follows by symmetry. Recall the definition ofṼ 1 σ D 2 from (4.3)-(4.4). From step 2 • in the proof of Proposition 4.1 it is sufficient to prove thatτ ε ↑τ * , where we recall that For stopping times σ β = inf{t ≥ β : X t ∈ D 2 } and στ ε = inf{t ≥τ ε : where the first inequality follows from the fact that G 1 ≤ H 1 . By Lemma 4.2 we have that στ ε ↑ σ β as ε ↓ 0 and so the first term on the right hand side of the above expression tends to zero uniformly over all τ . Since H 1 (X ) is quasi-left-continuous, the second expression also tends to zero. By the strong Markov property of X (recall that the expectation and the essential supremum in (3.14) can be interchanged) it Using again the fact thatṼ 1 is finely continuous, together with the fact that G 1 (X ) is left-continuous over stopping times, we get, from Lemma 3.5, thatṼ 1 (4.27) Combining (4.27) with the fact thatṼ 1 We now state and prove the main result of this section.

Theorem 4.4 Given the setting in Proposition 4.3, we have
Proof We shall only prove (4.29) as the proof of (4.30) follows by symmetry.
Recall, from Proposition 4.1, that V 1 . For simplicity of exposition let us set τ ε := τ σ D 2 ε and τ * := τ σ D 2 * . Now The first inequality follows from the assumption −G 1 ≥ −H 1 whereas the second equality follows from the fact that The penultimate inequality follows from the fact that whereas the last inequality follows from the fact that Letting ε ↓ 0 in (4.31) we get, from Proposition 4.3, that I (τ ε < σ D 2 < τ * ) converges to zero uniformly over σ D 2 . Moreover, by using the quasi-left-continuity property of and this completes the proof.

Partial superharmonic characterisation
The purpose of the current section is to utilise the results derived in Sects. 3 and 4 to provide a partial superharmonic characterisation of V 1 ) when the stopping time σ D 2 (resp. τ D 1 ) of player two (resp. player one) is externally given. This characterisation attempts to extend the semiharmonic characterisation of the value function in zero-sum games (see [51] and [52]) and can informally be described as follows: Suppose that G 2 ≡ −∞ in (2.3). Then the second player has no incentive of stopping the process and so (2.4) reduces to the optimal stopping problem V 1 By results in optimal stopping theory V 1 ∞ admits a superharmonic characterisation. More precisely V 1 ∞ can be identified with the smallest superharmonic function that dominates G 1 (see [49,p. 37 Theorem 2.4]). However, if G 2 is finite valued then there might be an incentive for the second player to stop the process. This raises two questions: (i) is the superharmonic characterisation of V 1 σ still valid before the second player stops the process, (ii) does V 1 σ coincide with H 1 at the time the second player stops the process? If the second player selects the stopping time σ := σ D 2 = inf{t ≥ 0 : X t ∈ D 2 } where D 2 is a closed or finely closed subset of the state space E having a regular boundary ∂ D 2 then the above questions can be answered affirmatively and we will say that the value function of player one associated with the stopping time σ D 2 admits a partial superharmonic characterisation. To be more precise let us consider the set where K 1 is the smallest superharmonic function that dominates H 1 and [G 1 , Then the value function of player one can be identified with the smallest finely continuous function from K 1 ). Likewise, suppose that player one selects the stopping time τ D 1 = inf{t ≥ 0 : X t ∈ D 1 } where D 1 is a closed or finely closed set having a regulary boundary ∂ D 1 , and consider the set where K 2 is the smallest superharmonic function that dominates H 2 and [G 2 , Then the value function of player two associated to τ D 1 can be identified with the smallest finely continuous function from Sup 2 D 1 (G 2 , K 2 ). Fig. 1 The double partial superharmonic characterisation of the value functions in a nonzero-sum optimal stopping game for absorbed Brownian motion The above characterisation of V 1 can be used to study the existence of a Nash equilibrium. Indeed suppose that one can show the existence of finely continuous functions u and v such that: (i.) u lies between G 1 and K 1 , u is identified with H 1 in the region D 2 = {v = G 2 } and u is the smallest superharmonic function that dominates G 1 in the region {v > G 2 } (ii.) v lies between G 2 and K 2 , v is identified with H 2 in the region D 1 = {u = G 1 } and v is the smallest superharmonic function that dominates G 2 in the region Then under the assumption that D 1 and D 2 have regular boundaries and are either closed or finely closed, u and v coincide with V 1 respectively. In this case we shall say that together, V 1 admit a double partial superharmonic characterisation (see Fig. 1) and can be called the value functions of the game (2.4)-(2.5). Moreover, the pair (τ D 1 , σ D 2 ) will form a Nash equilibrium point.
To prove the partial superharmonic characterisation of V 1 show that for any stopping time σ (resp. τ ), V 1 σ (resp. V 2 τ ) is bounded above by K 1 (resp. K 2 ). For this we define the concept of superharmonic functions.
Proof We shall only prove (5.3) as (5.4) can be proved in exactly the same way. Take any stopping time σ and any F ∈ Sup(H 1 ). Then for all stopping times τ . The first two inequalities follow from the fact that for all stopping times τ and for all x ∈ E. Taking the infimum over all F in Sup(H 1 ) on the right hand side and the supremum over all τ on the left hand side of (5.6) we get the required result.
ii. Similarly suppose that K 2 be the smallest superharmonic function that dominates H 2 . Let u ≥ G 1 be a finely continuous function, with Proof We shall only prove (i.) as (ii.) follows by symmetry. We first show that u ≥ V 1 for every stopping time τ and for all x ∈ E, where the last inequality follows from the fact that F ≥ G 1 . Since v is finely continuous and G 2 is continuous (hence finely continuous), D 2 is finely closed and thus by the definition of Sup 1 D 2 (G 1 , K 1 ), upon using (3.4) (since F is finely continuous and H 1 is continuous hence finely continuous) it can be seen that F(X σ D 2 ) = H 1 (X σ D 2 ). Thus for any F ∈ Sup 1 10) for every stopping time τ and for all x ∈ E. Taking the infimum over all F and the supremum over all τ we get that u(x) ≥ V 1 . For this it is sufficient to prove that V 1 as the result will follow by definition of u. Recall, from Theorem 3.7, that V 1 σ D 2 is finely continuous since the boundary ∂ D 2 is assumed to be regular for D 2 . The fact that V 1 Then by selecting any stopping time τ > 0 we get that From this we conclude that V 1 0 (x) = H 1 (x). It remains to prove that V 1 is superharmonic in D c 2 By the strong Markov property of X we have for any stopping time ρ where we recall that M 1 for stopping times τ, σ and ρ. and B * satisfying −∞ < A * < B * < ∞ such that (i.) for given D 2 of the form [B * , ∞), the first entry time τ A * = inf{t ≥ 0 : X t ≤ A * } (as obtained from Theorem 4.4) is optimal for player one and (ii.) for given D 1 of the form (−∞, A * ], the first entry time σ B * = inf{t ≥ 0 : X t ≥ B * } (as obtained from Theorem 4.4) is optimal for player two.
So in this section we will assume the existence of a pair (τ A * , σ B * ) that is a Nash equilibrium.

The principle of double continuous fit
We prove that V 1 σ B * (resp. V 2 τ A * ) is continuous at A * (resp. B * ). We shall refer to this result as the principle of double continuous fit. For this we shall further assume that the following time-space conditions hold: as ε, h ↓ 0 and whenever ρ n is a sequence of stopping times such that ρ n ↑ ρ. Conditions (6.1)-(6.4) imply that the mapping x → X x (stochastic flow) is continuous at A * and B * . Stochastic differential equations driven by Lévy processes, for example, satisfy this property under regularity assumptions on the drift and diffusion coefficients (see for example ([37, p. 340])). We shall first state and prove the following lemma.
Lemma 6.1 Let σ A * B * = inf{t ≥ 0 : X t ≥ B * } be the optimal stopping time for player two under P A * and let σ A * +ε B * = inf{t ≥ 0 : X t ≥ B * } be the optimal stopping time of player two under P A * +ε , for given ε > 0. Then, if condition (6.3) is satisfied we have that σ A * +ε is the optimal stopping time for player two under P B * and τ B * −ε A * = inf{t ≥ 0 : X t ≤ A * } is the optimal stopping time of player one under P B * −ε , for given ε > 0, then if condition (6.4 Proof We shall only prove that σ A * +ε as ε ↓ 0 can be proved in the same way. Since Law( Proof We shall only prove the result for V 1 σ B * as for V 2 τ A * the result will follow by symmetry. To prove this it is sufficient to show that ) ≤ 0 so that we get the required result. Given ε > 0, let τ A * +ε A * = inf{t ≥ 0 : X t ≤ A * } be the optimal stopping time for player one under P A * +ε . Then we have that τ A * +ε The first expectation in the last expression on the right hand side of (6.6) can be written as The last expression in (6.7) follows from the fact that P(τ A * +ε A * =σ A * B * ) = 0 for all ε > 0 sufficiently small (upon assuming that the hitting times considered are finite). and from the fact thatσ A * +ε B * ↑σ A * B * as ε ↓ 0 (see Lemma 6.1). In a similar way one can show that the second expectation in the last expression on the right hand side of (6.6) can be written as Since G 1 and H 1 are continuous and bounded then by the time space conditions (6.1) and (6.3) together with Lemma 6.1 and Fatou's lemma we get the required result.

Remark 6.3
The assumption of boundedness on G 1 and H 1 in Proposition 6.2 can be relaxed. For example, the result will also hold provided that G 1 (X A * +ε and ) are bounded above by some integrable random variables Z 1 andZ 2 respectively. Similarly the boundedness assumption on G 2 and H 2 can be relaxed.

The principle of double smooth fit
In this section we will consider the special case when X is a one dimensional regular diffusion process and we shall assume that V 1 σ B * and V 2 τ A * are obtained from the double partial superharmonic characterisation as explained in Sect. 5. More precisely we shall assume that the functions u, v introduced in Theorem 5.3 (i.) coincide with those from Theorem 5.3 (ii.) so that a mutual response is assumed to exist. The aim is to use this characterisation to derive the so-called principle of double smooth fit. This principle is an extension of the principle of smooth fit observed in standard optimal stopping problems (see [49]). We note that in the case of more general strong Markov processes in R this principle may break down. As observed in standard optimal stopping problems this may happen for example when the scale function of X is not differentiable (see [50]) or in the case of Poisson process (see [48]). Carr et. al in [7], for example, also showed that this principle breaks down in a CGMY model.

Remark 6.4
Examples of nonzero-sum optimal stopping games for one dimensional regular diffusion processes, for which the optimal stopping regions are of the threshold type are given in [1] and [12] 1 . In particular the authors therein provide sufficient conditions for existence and uniqueness of Nash equilibria.
So suppose that X is a regular diffusion process with values in R. We shall also assume that the fine topology coincides with Euclidean topology so that fine continuity is equivalent to continuity in the classical sense. In this context we can define the scale function S of the process X , that is the mapping S : R → R which is a strictly increasing continuous function satisfying for any c < x < d where τ y = inf{t ≥ 0 : X t = y} for y ∈ R. Since we are assuming that D 2 = [B * , ∞) then for any given a, b ∈ (−∞, B * ) such that a < b we have The first inequality follows from the fact that u is superharmonic in D c 2 (recall Definition 5.1). This means that u is S-concave in every interval in (−∞, B * ) and as for concave functions this implies that the mapping is decreasing provided that y = x. By symmetry we have that v is S-concave in every interval in (A * , ∞) and that the mapping y → v(y)−v(x) S(y)−S(x) is decreasing provided y = x.
From the results for the Dirichlet problem (see for example [49, (7.1.2)-(7.1. 3)]) one can show that L X u = 0 and L X v = 0 in C 1 ∩ C 2 u ∂C 1 = G 1 and u ∂C 2 = H 1 v ∂C 1 = H 2 and v ∂C 2 = G 2 where C 1 = D c 1 and C 2 = D c 2 . The aim is to show that u A * = G 1 A * and v B * = G 2 B * These two conditions will be referred to as the principle of double smooth fit. Informally this principle states that the optimal stopping boundary points A * and B * must be selected in such a way that u and v are respectively smooth at these points. The proof of this result follows in a similar way as the proof of Theorem 2.3 in [50]. We shall first state the following lemma, the proof of which can be found in [50]. Lemma 6.5 Suppose that f, g : R + → R are two continuous functions such that f (0) = g(0) = 0, f (ε) > 0 whenever ε > 0, and g(δ) > 0 whenever δ > 0. Then for every ε n ↓ 0 as n → ∞, there exists ε n k ↓ 0 and δ k ↓ 0 as k → ∞ such that lim k→∞ f (ε n k ) g(δ k ) = 1. Proposition 6.6 Suppose that D 1 is of the form (−∞, A * ] and D 2 of the form [B * , ∞) for some points A * , B * such that A * < B * . Suppose that G 1 is differentiable at A * and G 2 is differentiable at B * . If the scale function S of X is differentiable at A * and B * , then u (A * ) = G 1 (A * ) and v (B * ) = G 2 (B * ).
Proof We shall first consider the case when S (A * ) = 0. Since u is superharmonic in (−∞, B * ) we have that E x [u(X ρ∧σ B * )] ≤ u(x) for all stopping times ρ and for all x ∈ R. Define the exit time τ ε = inf {t ≥ 0 : X t / ∈ (A * − ε, A * + ε)} for ε > 0 such that A * + ε < B * . Then ≤ u(A * ) = u(A * )P A * (X τ ε = A * + ε) + G 1 (A * )P A * (X τ ε = A * − ε) (6.12) where the last equality follows from the fact that u(A * ) = G 1 (A * ). On the other hand we have E A * [u(X τ ε )] = u(A * + ε)P A * (X τ ε = A * + ε) + G 1 (A * − ε)P A * (X τ ε = A * − ε). (6.13) By combining (6.12) and (6.13) it follows that (6.14) Since we are assuming that S and G 1 are differentiable at A * and that S (A * ) = 0 we get, upon using the facts P A * (X τ ε = A * − ε) = S Acknowledgements The author is grateful to Professor Goran Peskir for introducing the topic of optimal stopping games, for the many fruitful discussions on the subtleties of Markov processes and the principles of smooth and continuous fit in zero-sum games, and for providing insight into the variational approach as a way of observing and understanding the principles of 'double smooth fit' and 'double continuous fit' in nonzero-sum games.
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.