Parameterized games of perfect information

Abstract

Considered are perfect information games with a Borel measurable payoff function that is parameterized by points of a Polish space. The existence domain of such a parameterized game is the set of parameters for which the game admits a subgame perfect equilibrium. We show that the existence domain of a parameterized stopping game is a Borel set. In general, however, the existence domain of a parameterized game need not be Borel, or even an analytic or co-analytic set. We show that the family of existence domains coincides with the family of game projections of Borel sets. Consequently, we obtain an upper bound on the set-theoretic complexity of the existence domains, and show that the bound is tight.

1 Introduction

This paper is motivated by the expanding literature on the existence of subgame perfect (epsilon-)equilibrium in perfect information games. Much of this literature focusses on identifying sufficient conditions for the existence of subgame perfect (epsilon-)equilibrium. In particular, subgame perfect equilibrium is known to exist in many classes of continuous perfect information games [we refer to section 3 of the survey by Jaśkiewicz and Nowak (2016) for references], and in games where each player’s payoff function is characterized by a Borel winning set (Grädel and Ummels 2008). All zero-sum games with bounded Borel payoffs [Mertens and Neyman, see Mertens (1987)], games with bounded lower semicontinuous payoffs (Flesch et al. 2010), games with bounded upper semicontinuous payoffs (Purves and Sudderth 2011), and games with common preferences at the limit (Flesch and Predtetchinski 2017) have a subgame perfect epsilon-equilibrium for each positive epsilon. If in addition the payoff function has a finite range, then the games of all these classes also have a subgame perfect equilibrium. Le Roux and Pauly (2014) provide transfer results, i.e. starting with the existence of some type of equilibria for a small class of games, they allow one to conclude the existence of some type of equilibria for a larger class.

There exist classes of perfect information games that do not always admit a pure subgame perfect epsilon-equilibrium, but do have a subgame perfect epsilon-equilibrium in mixed strategies: for example stopping games (Mashiah-Yaakovi 2014) and recursive games where each player controls only one state (Kuipers et al. 2016).

The issue of existence of subgame perfect equilibrium has been a major topic of study in the context of discounted stochastic games with simultaneous moves, one of the most prominent results being Mertens and Parthasarathy (2003). More references on this branch of the literature can be found in the survey by Jaśkiewicz and Nowak (2016).

In contrast, few necessary and sufficient conditions for the existence of subgame perfect epsilon-equilibria are known. T. Brihaye, V. Bruyère, N. Meunier, J.-F. Raskin (2016) [as reported in Meunier (2016)] provide a characterization of games that admit a subgame-perfect equilibrium. With each perfect information game, they associate a two-player zero-sum game, called the prover game. They show that the original perfect information game has a subgame-perfect equilibrium if and only if player I has a winning strategy in the corresponding prover game. Le Roux (2016) provides an intriguing analysis of games with $\varvec{\Delta }^{0}_{2}$-payoff functions. Essentially, he shows that, in the two-player case, the only payoff pattern that could rule out the existence of subgame-perfect equilibrium is that found in the example by Solan and Vieille (2003). Alós-Ferrer and Ritzberger (2016a, b) derive a condition on the topology of the set of plays that is both necessary and sufficient for the existence of a subgame-perfect equilibrium.

In this paper we take a somewhat different perspective on the question of existence of subgame perfect equilibrium: we study the set of games that admit a subgame perfect equilibrium. In a nutshell, our approach is as follows: we consider perfect information games where the payoff functions are parameterized by elements of a Polish space. We are interested in the set of parameters for which the game has a subgame perfect equilibrium. Our language and methods are borrowed from descriptive set theory.

Perfect information games are specified as follows: There is a set of actions that is assumed to be countable. Finite sequences of actions are called histories. A single active player is assigned to each history. The game begins at the empty history. Once a given history is reached, the corresponding active player chooses an action, leading to one of the successor histories. This process generates an infinite sequence of actions, called a play. The play determines the payoffs.

In a parameterized game the payoffs depend not only on the play, but also on an additional variable, called the parameter. The set of parameters is a given Polish space, and the payoff function of a parameterized game is assumed to be Borel measurable jointly in the plays and the parameters. For some of our results we furthermore assume that the payoff function only takes finitely many values.

A particularly simple example of our setup is a parameterized perfect information stopping game. In such a game each player has two actions, stop and continue. Once a player chooses to stop, the game effectively terminates, and the payoffs are received. Thus the game is specified by the sequence of payoff vectors, one vector for each period of time in which the game could be terminated. Such a sequence of payoff vectors serves as a parameter of the stopping game.

To each parameterized perfect information game we associate the set of parameters for which the game has a subgame perfect equilibrium. This set is called the existence domain of the parameterized game. The two groups of main results are as follows.

The first result relates to stopping games. We show that the existence domain of a parameterized stopping game is a Borel set. Put equivalently, the set of payoffs for which the stopping game has an SPE is a Borel set. In general, however, the existence domain of a parameterized game need not be a Borel, or even an analytic or a co-analytic set.

Our second result provides an upper bound on the set-theoretic complexity of the existence domains, and the third result shows that the bound is tight. This is achieved by relating the existence domains to the so-called game projection operator $\Game$. The latter has been intensively studied by set theorists (see e.g. Moschovakis 1980; Kanovei 1988).

More precisely, we show that the family of existence domains coincides with the family of game projections of Borel sets. This implies that the family of existence domains is much richer than the class of Borel sets. It includes all analytic and coanalytic sets, the sigma-algebra generated by analytic sets, $\mathbf {C}$-sets, Borel programmable sets, and $\mathbf {R}$-sets. In its turn, it is contained in the class of absolutely $\varvec{\Delta }_{1}^{2}$-sets, which in turn is contained in $\varvec{\Delta }_{1}^{2}$-sets.

A closely related paper is that by Prikry and Sudderth (2016). The authors consider a class of zero-sum simultaneous move games where the payoff function is parameterized by points of a Polish space. The authors study measurability properties of the (upper) value, as a function of the parameter. Levy (2013) uses parameterized games with simultaneous moves to obtain counterexamples to the existence of equilibria in models with overlapping generations and in games with a continuum of players.

An important branch of the literature deals with the issues of computational complexity in infinite sequential games. Considered there is the complexity of finding the winning strategies, determining the winner, finding Nash and subgame perfect equilibria (see e.g. Le Roux and Pauly 2015; Brattka et al. 2015). In particular, Le Roux and Pauly (2015)’s work, while addressing questions similar to those in this paper, use a different formalism and different mathematics. Their formalism is that of so-called represented games, whereby games of some class are coded by elements of the Baire space. The natural coding is essentially derived from Borel codes as used in effective descriptive set theory. The authors use Weihrauch reducibility and degree, and the Hausdorff difference hierarchy. In contrast, the pointclasses considered in this paper are more closely related to the Borel and the projective hierarchies.

The works on minimax selection theorems are somewhat related to our study in that there one considers parameterized payoff functions in zero-sum games [see Nowak (2010) and the references therein].

The paper is organized as follows: In Sects. 2 and 3 we detail our setup, and in Sect. 4 we state our main results. Proofs are carried out in Sects. 5, 6 and 7. Section 8 contains an extension of the main results that deals with the concept of subgame perfect $\epsilon$-equilibrium.

2 Perfect information games

Perfect information games Let $\mathbb {N} = \{0,1,2\dots \}$ be the set of natural numbers. Let I be a non-empty finite set of players, and A be a non-empty countable set of actions. Let $H = \cup _{t \in \mathbb {N}}A^{t}$ be the set of all finite sequences of elements of A, including the empty sequence $\emptyset$. As usual we let $A^{\mathbb {N}}$ denote the set of all infinite sequences of elements of A. Throughout the paper the set $A^{\mathbb {N}}$ is endowed with the product topology. Elements of H are called histories and elements of $A^{\mathbb {N}}$ are called plays. Let $\iota : H \rightarrow I$ be a function which assigns an active player to each history. Finally, each player $i \in I$ is given the payoff function $u_{i} : A^{\mathbb {N}} \rightarrow \mathbb {R}$. For now we assume that $u_{i}$ takes a finite number of integer values and is Borel measurable. In Sect. 8 of the paper we will consider more general games where the the assumption of finite range is replaced by a weaker assumption of bounded range. We write $u(p) = (u_{i}(p))_{i \in I}$. The sets I, A, and the functions $\iota$ and u determine the game G.

The game G is known to the players. The game is played as follows: let $h_{0} = \emptyset$. Suppose a history $h_{t} \in H$ has been determined for some $t \in \mathbb {N}$. Player $\iota (h_{t})$ then chooses an action $a_{t} \in A$, leading to the history $h_{t+1} = (h_{t},a_{t})$. The players thus generate a play $p = (a_{0}, a_{1}, \dots )$, and each player $i \in I$ receives payoff $u_{i}(p)$.

In this paper, we only consider pure strategies. A (pure) strategy for player i is a function $\sigma _{i}$ assigning to each history $h \in H$ with $\iota (h) = i$ an action $\sigma _{i}(h)$ in A. A strategy profile is a tuple $\sigma =(\sigma _{i})_{i\in I}$ where $\sigma _{i}$ is a strategy for every player $i \in I$. Given a strategy profile $\sigma$ and a strategy $\eta _{i}$ for player i, we write $\sigma /\eta _{i}$ to denote the strategy profile obtained from $\sigma$ by replacing $\sigma _{i}$ with $\eta _{i}$.

At every history $h\in H$, every strategy profile $\sigma$ induces a unique play $\pi _{h}(\sigma )$. We write $\pi (\sigma ) = \pi _{\emptyset }(\sigma )$ to denote the play induced from the root.

A strategy profile $\sigma$ is called an equilibrium of the game G if for each player $i \in I$ and for each strategy $\sigma _{i}^{\prime }$ of player i it holds that

$$\begin{aligned} u_{i}(\pi (\sigma )) \ge u_{i}(\pi (\sigma /\sigma _{i}^{\prime })). \end{aligned}$$

The result of Mertens and Neyman (reported in Mertens 1987) implies that G has an equilibrium. A strategy profile $\sigma$ is called a subgame-perfect equilibrium (SPE) of the game G, if it induces an equilibrium in each subgame of G, or equivalently, if for each history $h \in H$, each player $i \in I$, and each strategy $\sigma _{i}^{\prime }$ of player i it holds that

$$\begin{aligned} u_{i}(\pi _{h}(\sigma )) \ge u_{i}(\pi _{h}(\sigma /\sigma _{i}^{\prime })). \end{aligned}$$

Unlike equilibrium, an SPE need not exist in a perfect information game under the maintained assumptions. Examples of games having no SPE have been considered in Solan and Vieille (2003) and in Flesch et al. (2014).

We proceed to define two classes of perfect information games that play an important role in our analysis.

Martin games A perfect information game is said to be a Martin game if the set of players is $I = \{\mathrm{I}, \mathrm{II}\}$, the action set A is¹$\mathbb {N}$, and there exists a Borel set $W \subseteq \mathbb {N}^{\mathbb {N}}$ such that the payoff function $u_{I}$ is the indicator function of W, and the payoff function $u_{II}$ is the indicator function of the complement of W. The set W is called the winning set of player I. With a slight abuse of notation we also use the symbol W to denote the Martin game itself.

A strategy $\sigma _{I}$ for player I is said to be winning in W if $\pi (\sigma _{I},\sigma _{II}) \in W$ for every player II’s strategy $\sigma _{II}$. One analogously defines a winning strategy for player II. The Borel determinacy theorem (Martin 1975) states that either player I has a winning strategy in W, or player II has a winning strategy in W.

Stopping games Stopping games are perfect information games where the active player can either stop the game or continue. As soon as the active player stops, the game essentially terminates in the sense that subsequent actions do not influence the payoffs. If no player ever stops, the payoffs are zero to every player.

Formally, a perfect information game J is called a stopping game if the action set is $A = \{s, c\}$ where s stands for stopping and c for continuing, and for each $t \in \mathbb {N}$ and each player $i \in I$, the payoff function $u_{i}$ is constant, say equal to $x_{i}(t)$, on the set of plays that extend the history $(c^{t}, s)$. For convenience we also assume that the payoff on the play $(c,c,c,\dots )$ is zero to each player. A stopping game is completely specified by the sequence $x = (x(0),x(1),\dots )$ where $x(t) = (x_{i}(t))_{i \in I}$.

3 Parameterized games

In this section we introduce families of perfect information games where the payoff functions depend on a parameter.

Parameterized (perfect information) games A parameterized (perfect information) game $G_{X}$ consists of a non-empty finite set of players I, a non-empty countable set of actions A, the assignment $\iota : H \rightarrow I$ of active players to histories, a non-empty Polish space X, and for each player i a payoff function $u_{i} : X \times A^{\mathbb {N}} \rightarrow \mathbb {R}$. Element of X are parameters. We assume that $u_{i}$ takes a finite number of integer values and is Borel measurable. Given a parameterized game $G_{X}$ and an $x \in X$, we write $G_{x}$ to refer to the particular game with the payoff functions $u_{i}(x,\cdot )$ for $i \in I$. The parameter x is known to the players.

Parameterized Martin games A parameterized perfect information game is said to be a parameterized Martin game if the set of players is $I = \{\mathrm{I}, \mathrm{II}\}$, the action set A is $\mathbb {N}$, and there exists a Borel set $W_{X} \subseteq X \times \mathbb {N}^{\mathbb {N}}$ such that the payoff function $u_{I}$ is the indicator function of $W_{X}$, and the payoff function $u_{II}$ is the indicator function of the complement of $W_{X}$. We use the symbol $W_{X}$ to denote the parameterized Martin game itself. For each $x \in X$ we write $W_{x}$ to refer to the Martin game with the winning set $\{p \in \mathbb {N}^{\mathbb {N}}: (x,p) \in W_{X}\}$.

To a parameterized Martin game $W_{X}$ we associate the set

$$\begin{aligned} \Game W_{X} = \{x \in X:\text { player I has a winning strategy in the Martin game }W_{x}\}. \end{aligned}$$

The operator $\Game$ is called the game projection.

We let $\mathbf {W}_{X}$ denote the set of all Martin games with the parameter space X, that is, the family of Borel subsets of $X \times \mathbb {N}^{\mathbb {N}}$, and we let

$$\begin{aligned} \Game \mathbf {W}_{X} = \{\Game W_{X}:W_{X} \in \mathbf {W}_{X}\}. \end{aligned}$$

The family $\Game \mathbf {W}_{X}$ and its relation to other classes of sets have been studied in descriptive set theory. Below we mention some of the most important properties of $\Game \mathbf {W}_{X}$. More details can be found in Burgess (1983a, b), Burgess and Lockhart (1983), Moschovakis (1980) and Kanovei (1988).

The family $\Game \mathbf {W}_{X}$ is a sigma-algebra. It is closed under the Souslin operation. The family $\Game \mathbf {W}_{X}$ is much richer than the Borel sigma-algebra of X. Some of the prominent classes of sets, listed in the increasing order, are: Borel sets $\subseteq$ analytic sets and coanalytic sets $\subseteq$ the sigma-algebra generated by the analytic sets $\subseteq$C-sets $\subseteq$ Borel-programmable sets $\subseteq$R-sets $\subseteq$$\Game \mathbf {W}_{X}$$\subseteq$ absolutely $\varvec{\Delta }_{2}^{1}$-sets $\subseteq$$\varvec{\Delta }_{2}^{1}$-sets. In particular, every set in $\Game \mathbf {W}_{X}$ is a projection of a coanalytic set.

Parameterized stopping games Recall that stopping games are specified by the sequence x of stopping payoffs. Formally we define a parameterized perfect information game $J_{X}$ to be a parameterized stoping game if

• [A] The action set is $A = \{s, c\}$,

• [B] The space of parameters is the product space $X = \{-D,\dots ,D\}^{I \times \mathbb {N}}$, where D is a natural number.

A typical element of X is denoted $x = (x(0),x(1),\dots )$ where $x(t) = (x_{i}(t))_{i \in I}$.

• [C] For each player $i \in I$ the payoff function is defined as follows: $u_{i}(x,p) = x_{i}(t)$ for every play p extending the history $(c^{t},s)$, and $u_{i}(x,(c,c,c,\dots )) = 0$.

4 The main results

In the previous section we have introduced perfect information games where the payoff functions are parameterized by elements of a Polish space. In this section we summarize the main results on such parameterized games. We are interested in the so-called SPE existence domain of a parameterized game, that is, in the the set of parameters for which the game has an SPE.

Our first result deals with parameterized stopping games. We show that the set of payoff sequences for which the stopping game admits an SPE is a Borel set. As it turns out, for more general parameterized games the SPE existence domain is not necessarily a Borel set. However, we are able to put an upper bound on its set-theoretic complexity. This is our second result. Our third result shows that the bound is sharp.

We now turn to a formal statement of the results. Fix a set of parameters X, a non-empty Polish space. The main object of our study is the collection of sets defined as follows: Given a parameterized game $G_{X}$ as in the Sect. 3 we define the SPE existence domain of $G_{X}$ as the following set of parameters:

$$\begin{aligned} E(G_{X}) = \{x \in X: \text {the game }G_{x}\text { has an }{{\text {SPE}}}\}. \end{aligned}$$

In the introduction, we have enlisted several known classes of perfect information parameterized games where $E(G_{X}) = X$. In this paper we are interested to know just how complex the set $E(G_{X})$ generally could be.

Theorem 4.1

Let $J_{X}$ be a parameterized stopping game. Then the set $E(J_{X})$ is a Borel subset of X.

As follows from the results below, in general the set $E(G_{X})$ need not be Borel. However, we are able to place an upper bound on the set-theoretic complexity of $E(G_{X})$ by linking it to the game projection operator. More precisely, our findings are as follows.

Theorem 4.2

Let $G_{X}$ be a parameterized game. Then $E(G_{X})$ belongs to the family $\Game \mathbf {W}_{X}$.

The next result provides the converse to the previous one.

Theorem 4.3

Consider any member of the family $\Game \mathbf {W}_{X}$, say $\Game W_{X}$. There exists a parameterized game $G_{X}$ such that $E(G_{X}) = \Game W_{X}$. One can choose the game $G_{X}$ to have 2 players, with a payoff function u only taking three distinct values: $(1,-2)$, $(2,-1)$, and (0, 0).

Theorems 4.2 and 4.3 could be stated more concisely as

$$\begin{aligned} \{E(G_{X}):G_{X}\text { is a parameterized game}\} = \Game \mathbf {W}_{X}. \end{aligned}$$

To prove Theorem 4.2 we associate to the parameterized game $G_{X}$ a parameterized Martin game $W_{X}$ such that player I has a winning strategy in $W_{x}$ exactly when $G_{x}$ has an SPE. The intuition behind the game $W_{x}$ is as follows: At each move of the game $W_{x}$ player I produces a “recommended” action as well as a payoff vector. The latter is interpreted as the payoff vector player I promises to the players of $G_{x}$, under the condition that his recommendations are being followed. Player II produces the actual action, and is free to follow the recommendations of player I or to deviate at any point. Player II’s objective is to beat the utility payoff vector announced by player I. Similar constructions appeared in N. Meunier’s doctoral Thesis [Chapter 9 co-authored by T. Brihaye, V. Bruyère, N. Meunier, J.-F. Raskin (2016)]. A somewhat related game has also been used in Flesch and Predtetchinski (2017).

To prove Theorem 4.3 we explicitly construct a parameterized game $G_{X}$ with the desired property. The game involved could be seen as an extension of the example in Solan and Vieille (2003).

Theorem 4.1 is the most challenging of the three results. We argue that the set $X - E(J_{X})$ is Borel. We first consider the subset of $E'(J_{X})$ of $E(J_{X})$ consisting of the parameters $x \in X$ for which the game $J_{x}$ admits an SPE with the property that at most one player stops infinitely often. It is straightforward to show that $X - E'(J_{X})$ is Borel. The bulk of the proof is devoted to showing that the set $X - E(J_{X})$ is Borel as a subset of $X - E'(J_{X})$.

To do so we construct a continuous function f mapping each point x of $X - E'(J_{X})$ to an irreflexive and transitive binary relation f(x) on natural numbers such that

$$\begin{aligned} X - E(J_{X}) = f^{-1}(\mathrm{TW}(\omega ^{2})), \end{aligned}$$

where $\mathrm{TW}(\omega ^{2})$ denotes the set of well-founded relations on natural numbers of rank smaller than $\omega ^{2}$. That is, the game $J_{x}$ has no SPE if and only if the relation f(x) is well-founded. And whenever f(x) is well-founded, it has a rank smaller than $\omega ^{2}$. Since $\mathrm{TW}(\omega ^{2})$ is a Borel set, the set $X - E(J_{X})$ is a Borel subset of $X - E'(J_{X})$, as desired.

5 The proof of theorem 4.2

The informal description of the proof We associate to the parameterized game $G_{X}$ a parameterized Martin game $W_{X}$ such that player I has a winning strategy in $W_{x}$ exactly when $G_{x}$ has an SPE, that is $E(G_{X}) = \Game W_{X}$. For an $x \in X$ the game $W_{x}$ proceeds thus:

$$\begin{aligned} \begin{array}{cccccc} \hbox {I}&{}r_{0},v_{0}&{}&{}r_{1},v_{1}&{}\cdots &{}\\ \hbox {II}&{}&{}a_{0}&{}&{}a_{1}&{}\cdots \end{array} \end{aligned}$$

Player I’s moves are pairs $(r_{t},v_{t})$ where $r_{t}$ is an action in $G_{x}$ and $v_{t}$ a vector of payoffs. We think of $r_{t}$ as an action recommended to player II and of $v_{t}$ as a payoff vector promised by player I, under the condition that his recommendations are being followed. Player II’s moves $a_{t}$ are actions in $G_{x}$, interpreted as the actual actions that are being carried out. If $a_{t} = r_{t}$, we say that player II agrees to player I’s recommendation. Otherwise we say that player II deviates from the recommended action. Two consistency conditions are imposed on player I’s choice of the vector $v_{t}$ of payoffs: Firstly, player I is allowed to update the vector of payoffs only right after a deviation by player II. Secondly, if player II agrees to the recommended action in every period starting with t, then $v_{t}$ is to be equal to the actual payoffs on the play that player II has generated. Any strategy for player I in the game $W_{x}$ satisfying these consistency conditions gives rise to a strategy profile in the original game $G_{x}$, and vice versa.

Player I wins $W_{x}$ if there exists no period t and no player i of the game $G_{x}$ such that, firstly, after period t player II only deviates at histories controlled player i, and secondly, the play player II produces is preferred by player i to the payoff $v_{t}$ announced by player I at period t.

A similar construction (called a prover game) appeared in N. Meunier’s doctoral Thesis [Chapter 9 co-authored by T. Brihaye, V. Bruyère, N. Meunier, J.-F. Raskin 2016)]. A somewhat related game has also been used in Flesch and Predtetchinski (2017).

The proof Formally the parameterized Martin game $W_{X}$ is described as follows: player I’s action set is $A \times V$ where V is the range of the function $u = (u_{i})_{i \in I}$, while player II’s action set is A. To define player I’s winning set in the game $W_{X}$, consider a play $p^{*} = ((r_{0}, v_{0}), a_{0}, (r_{1}, v_{1}), a_{1}, \dots )$ of $W_{X}$. Let $p = (a_{0},a_{1},\dots )$ be the sequence of moves of player II, and let $h_{0} = \emptyset$ and $h_{t} = (a_{0},\dots ,a_{t-1})$ for each $t \ge 1$. Let $I_{t} = \{\iota (h_{k}): k \ge t\text { and }a_{k} \ne r_{k}\}$ be the set of players who deviate after period t. Player I’s winning set $W_{X}$ consists of pairs $(x,p^{*})$ satisfying the following three conditions for every $t \in \mathbb {N}$:

• [1] If $a_{t} = r_{t}$, then $v_{t+1} = v_{t}$.

• [2] If $I_{t} = \emptyset$, then $v_{t} = u(x,p)$.

• [3] If $I_{t} = \{i\}$ for some $i \in I$, then $v_{t}^{i} \ge u_{i}(x,p)$.

Conditions [1] and [2] could be thought of as consistency conditions on the moves of player I, while condition [3] encapsulates the idea that player II wins in $W_{X}$ if player i can deviate profitably in $G_{X}$. The set $W_{X}$ is Borel. Indeed, for every $t \in \mathbb {N}$ each of the conditions [1], [2], and [3] defines a Borel set of parameter-play pairs $(x,p^{*})$, and $W_{X}$ is the intersection of all these sets.

We proceed to showing that player I has a winning strategy in $W_{x}$ exactly when $G_{x}$ has an SPE. Let $x \in X$.

Suppose first that $\sigma$ is an SPE of $G_{x}$. Define the strategy $\sigma _{I}^{*}$ for player I in $W_{x}$ as follows: let $h^{*}$ be a history for player I in $W_{x}$, and let h be the corresponding sequence of moves of player II. Define

$$\begin{aligned} \sigma _{I}^{*}(h^{*}) = (\sigma (h), u(x,\pi _{h}(\sigma ))). \end{aligned}$$

We argue that $\sigma _{I}^{*}$ is a winning strategy for player I in $W_{x}$. Let $p^{*}$ denote any play of $W_{x}$ consistent with the strategy $\sigma _{I}^{*}$. Then $p^{*}$ satisfies conditions [1] and [2] above. We argue that it also satisfies condition [3]. Suppose to the contrary. Then there exist $t \in \mathbb {N}$ and $i \in I$ such that $I_{t} = \{i\}$ and $v_{t}^{i} < u_{i}(x,p)$. Note that $v_{t}^{i} = u_{i}(x,\pi _{h_{t}}(\sigma ))$. But then, since player i can induce the play p in the game $G_{x}$ starting with the history $h_{t}$, he has a profitable deviation from $\sigma$, leading to a contradiction.

Conversely, suppose that $\sigma _{I}^{*}$ is a winning strategy for player I in $W_{x}$. Define the strategy profile $\sigma$ for $G_{x}$ as follows: Take a history $h \in A^{t}$ in $G_{x}$ and let $h^{*}$ be player I’s history in $W_{x}$ of length 2t where player II’s moves correspond to h and player I’s moves are obtained using $\sigma _{I}^{*}$. More precisely if $h = \emptyset$ let $h^{*} = \emptyset$, and if $h = (a_{0},\dots ,a_{t-1})$ let $h^{*} = ((r_{0}, v_{0}), a_{0}, \cdots , (r_{t-1}, v_{t-1}), a_{t-1})$ where $(r_{k}, v_{k}) = \sigma _{I}^{*}((r_{0}, v_{0}), a_{0}, \cdots , (r_{k-1}, v_{k-1}), a_{k-1})$ for $k \le t-1$. Let $(r_{t}, v_{t})$ denote $\sigma _{I}^{*}(h^{*})$. We define $\sigma (h)$ to be $r_{t}$.

We first claim that $v_{t} = u(x,\pi _{h}(\sigma ))$. Indeed if player I plays according to $\sigma _{I}^{*}$ and player II agrees to player I’s recommendations as of period t, then player II produces a sequence of moves $p = \pi _{h}(\sigma )$. The claim now follows since $\sigma _{I}^{*}$ is winning and by [2].

We show that $\sigma$ is an SPE of $G_{x}$. Suppose to the contrary. Then there is a history $h \in A^{t}$ of the game $G_{x}$ such that player $i = \iota (h)$ has a profitable deviation from $\sigma$, say the strategy $\eta _{i}$. Let $p' = \pi _{h}(\sigma /\eta _{i})$. If player II plays according to $p'$ in the game $W_{x}$, then he wins against $\sigma _{I}^{*}$, since then condition [3] is violated. This is a contradiction to the choice of $\sigma _{I}^{*}$, and hence $\sigma$ is an SPE as claimed.

6 The Proof of Theorem 4.3

Let B be any member of the family $\Game \mathbf {W}_{X}$. Our goal is to construct a parameterized perfect information game $G_{X}$ such that $E(G_{X}) = B$.

Lemma 6.1

There exists a parameterized Martin game $V_{X}$ such that

$$\begin{aligned} B = \{x \in X: \text {player I has a strategy that is winning in each subgame of }V_{x}\}. \end{aligned}$$

Proof

First we notice that, for any parameterized Martin game $V_{X}$ the following conditions are equivalent for each $x \in X$: [1] player I has a strategy that is winning in each subgame of $V_{x}$ and [2] for each subgame of $V_{x}$ player I has a winning strategy. Condition [1] obviously implies condition [2]. And if [2] holds, we define a strategy for player I as follows, using an idea of Mertens and Neyman as reported in Mertens (1987) and in Mertens et al. (2015): player I begins following a strategy that is winning in $V_{x}$. If at any point player I makes a mistake and takes an action that is incompatible with the chosen strategy, player I adopts a strategy that is winning in the subgame that has thus been reached, and keeps following it. And so on. The strategy thus constructed is winning in each subgame of $V_{x}$.

Since $B \in \Game \mathbf {W}_{X}$, we know that there exists a parameterized Martin game $W_{X}$ such that $B = \Game W_{X}$.

The idea for constructing the parameterized game $V_{X}$ is as follows: We augment the game $W_{X}$ by allowing player I the possibility to “restart” the game after any sequence of moves, by marking his action with a $*$. Once player I plays a marked action, we enter a part of the game that is identical to $W_{X}$. However, if I restarts the game infinitely many times, player II wins.

Formally let the set of actions be $A = \mathbb {N}^{*} \cup \mathbb {N}$ where $\mathbb {N}^{*}$ is a copy of $\mathbb {N}$. Let player I’s winning set in $V_{X}$ consist of the pairs $(x, p) \in X \times A^{\mathbb {N}}$, where $p = (a_{0},a_{1},\dots )$ such that [A] $a_{2t} \in \mathbb {N}^{*}$ for at most finitely many $t \in \mathbb {N}$, and [B] $(x,a_{2t},a_{2t+1},\dots )$ is in the winning set of the game $W_{X}$, where t is the largest number such that $a_{2t} \in \mathbb {N}^{*}$, or $t = 0$ if $a_{2k} \in \mathbb {N}$ for all $k \in \mathbb {N}$.

Suppose $x \in B$. Then player I has a winning strategy in $W_{x}$, say $\sigma _{I}$. We argue player I has a winning strategy in each subgame of $V_{x}$. It suffices to show that player I has a winning strategy in each subgame of $V_{x}$ starting with a move by player I. Thus let h be player I’s history. A winning strategy of player I in the subgame of $V_{x}$ starting at h is as follows: At h, play the marked action $\sigma _{I}(\emptyset )^{*}$ (the element of $\mathbb {N}^{*}$ identical to $\sigma _{I}(\emptyset )$), and subsequently follow $\sigma _{I}$, placing the actions in $\mathbb {N}$.

Conversely, suppose that $x \in X \setminus B$. Then player II has a winning strategy in $W_{x}$, say $\sigma _{II}$. Define $\sigma _{II}'$ be player II’s strategy in $V_{x}$ as follows: Let player II follow $\sigma _{II}$ for as long as player I does not restart the game. As soon as player I restarts the game, let player II erase the memory and begin playing according to $\sigma _{II}$. And so on. Formally, $\sigma _{II}'(a_{0},\dots ,a_{2n}) = \sigma _{II}(a_{2k},\dots , a_{2n})$ where $k \le n$ is the largest number such that $a_{2k} \in \mathbb {N}^{*}$, or $k = 0$ if $a_{0},a_{2},\dots ,a_{2n}$ are all elements of $\mathbb {N}$. We argue that $\sigma _{II}$ is winning strategy in $V_{x}$. Consider any play $(a_{0},a_{1},\dots )$ consistent with $\sigma _{II}'$. If infinitely many of the actions $a_{0},a_{2},\dots$ are in $\mathbb {N}^{*}$, player II wins the play. Otherwise, let t be the largest number such that $a_{2t} \in \mathbb {N}^{*}$, and $t = 0$ if $a_{0},a_{2},\dots$ are all elements of $\mathbb {N}$. By definition of $\sigma _{II}'$ the play $(a_{2t},a_{2t+1},\dots )$ is not in $W_{x}$. Thus player II wins. $\square$

Let $V_{X}$ be a parameterized Martin game as in the preceding lemma. As usual, player I’s winning set is also denoted by $V_{X} \subseteq X \times \mathbb {N}^{\mathbb {N}}$.

In the Proof of Theorem 4.3 we construct a parameterized game $G_{X}$ that is reminiscent of an example in Solan and Vieille (2003). In $G_{X}$ there are two players, I and II, who move alternately, starting with player I. Intuitively, either player can stop the game using the action s, or play a natural number. If player I stops the game, the payoffs are $(1,-2)$, if player II stops the game the payoffs are $(2,-1)$. If neither player stops the game, the payoffs depend on x as follows: if the generated play is an element of $V_{x}$, the payoffs are $(2, -1)$. If it is not, the payoffs are (0, 0).

Let $A = \{s\} \cup \mathbb {N}$ and $H = A^{\mathbb {N}}$. We say that a history $h \in H$ is terminal if one of the actions in h is s. Otherwise h is said to be non-terminal. The payoff functions are as follows: For a play p such that the action s occurs for the first time at an even period, we have $u(x,p) = (1,-2)$. For a play p such that the action s occurs for the first time at an odd period, $u(x,p) = (2,-1)$. For a play p that is an element of $V_{x}$ we have $u(x,p) = (2,-1)$. For a play p that is an element of $\mathbb {N}^{\mathbb {N}}\setminus V_{x}$ we have $u(x,p) = (0,0)$.

The game $G_{X}$ is a parameterized game. It remains to show that $E(G_{X}) = B$.

We show that $E(G_{X}) \supseteq B$. Take an $x \in B$. By the above lemma, there is a strategy $\sigma _{I}$ for player I that is winning in each subgame of $V_{x}$. Clearly $\sigma _{I}$ could be seen as player I’s strategy in $G_{x}$ that never assigns the action s. Let $\sigma _{II}$ be the strategy for player II that requires player II to stop at each of his histories. We argue that $\sigma = (\sigma _{I},\sigma _{II})$ is an SPE of $G_{x}$. Indeed, under $\sigma$ player I receives the maximal payoff of 2 at each non-terminal history h. Due to the choice of $\sigma _{I}$, player II has no profitable deviation at any of his histories because he is unable to induce a play in $\mathbb {N}^{\mathbb {N}}\setminus V_{x}$.

Conversely, we show that $E(G_{X}) \subseteq B$. Take an $x \in X \setminus B$. We argue that the game $G_{x}$ has no SPE. Suppose on the contrary and let $\sigma$ be an SPE of $G_{x}$.

We first argue that $u(x,\pi _{h}(\sigma )) = (2,-1)$ for every non-terminal $h \in H$ where player II is active. Since player II can secure a payoff of $-1$ by stopping, $u(x,\pi _{h}(\sigma ))$ equals either (0, 0) or $(2,-1)$. If $u(x,\pi _{h}(\sigma )) = (0,0)$ then $\sigma (h) \ne s$. At history $(h,\sigma (h))$ player I can secure a payoff of 1 by stopping, and hence $u(x,\pi _{h}(\sigma ))$ is either $(1,-2)$ or $(2, -1)$, leading to a contradiction.

It follows that $\sigma (g) \ne s$ for each non-terminal history g where player I is active. For in view of the result of the previous paragraph, player I can get a payoff of 2 by playing any natural number n. Thus $\sigma _{I}$ returns a natural number after every non-terminal history in $G_{x}$, and can be viewed as a strategy in the game $V_{x}$.

Now recall that x is not in B, so that player I has no strategy that is winning in every subgame of $V_{x}$. In particular, $\sigma _{I}$ is not a winning strategy in some subgame of $V_{x}$. Hence, $\sigma _{I}$ is not a winning strategy in a subgame of $V_{x}$ that starts with player II’s history h. Consequently, at h, player II can deviate from $\sigma$ and induce a play that is an element of $\mathbb {N}^{\mathbb {N}}\setminus V_{x}$ and obtain the payoff 0. Since $u(x,\pi _{h}(\sigma )) = (2, -1)$, such a deviation is profitable, contradicting that $\sigma$ is an SPE.

7 The Proof of Theorem 4.1

The Proof of Theorem 4.1 relies on a number of concepts originating in descriptive set theory, including that of a well-founded relation and its rank. We split this section into two subsections. In the first subsection we review the definitions related to well-founded relations and their rank. In the second subsection we proceed to the proof of the theorem.

7.1 Well-founded relations and ranks

In this subsection we briefly review the definitions of well-founded relations and ranks. For more details we refer to Kechris (1995).

Let $\mathbb {N} = \{0,1,\dots \}$ be the set of natural numbers. A binary relation $\triangleright$ on the set of natural numbers is a subset of $\mathbb {N} \times \mathbb {N}$. For $t, k \in \mathbb {N}$ we write $t\triangleright k$ if $(t,k) \in \triangleright$, and we say that k is $\triangleright$-smaller than t. The relation $\triangleright$ is called irreflexive if there is no $n \in \mathbb {N}$ such that $n\triangleright n$. It is said to be transitive if whenever $t\triangleright m$ and $m\triangleright n$ for some $t,m,n \in \mathbb {N}$, it holds that $t\triangleright n$. The relation $\triangleright$ is said to be well-founded if each non-empty set $S \subseteq \mathbb {N}$ has an $\triangleright$-minimal element, that is, an element $s \in S$ such that there is no $t \in S$ with $s\triangleright t$. Equivalently, $\triangleright$ is well-founded if it has no infinite descending chain, i.e. no sequence $t_{0},t_{1},\dots$ of natural numbers such that $t_{n}\triangleright t_{n+1}$ for each $n \in \ \mathbb {N}$. Notice that a well-founded relation is necessarily irreflexive.

Let $\triangleright$ be a transitive well-founded relation. We define the rank function $\rho _{\triangleright }$ assigning to each natural number a countable ordinal, by recursion as follows: $\rho _{\triangleright }(t) = 0$ if t is a $\triangleright$-minimal element of $\mathbb {N}$, and otherwise

$$\begin{aligned} \rho _{\triangleright }(t) = \sup \{\rho _{\triangleright }(n)+1:n\in \mathbb {N},t\triangleright n\}. \end{aligned}$$

The rank of $\triangleright$ is defined to be the ordinal number

$$\begin{aligned} \rho (\triangleright ) = \sup \{\rho _{\triangleright }(n) + 1:n \in \mathbb {N}\}. \end{aligned}$$

We let $\mathrm{TI}$ denote the set of transitive and irreflexive binary relations. We let $\mathrm{TW}$ denote set of transitive well-founded relations, and, for a countable ordinal $\alpha$, we define $\mathrm{TW}(\alpha ) = \{\triangleright \in \mathrm{TW}:\rho (\triangleright ) < \alpha \}$.

We identify a binary relation $\triangleright$ on $\mathbb {N}$ with the indicator function of $\triangleright$, an element of the set $\{0,1\}^{\mathbb {N}\times \mathbb {N}}$. Then the set $\mathrm{TI}$ is a closed subset of $\{0,1\}^{\mathbb {N}\times \mathbb {N}}$, and hence a Polish space. For each countable ordinal $\alpha$, the set $\mathrm{TW}(\alpha )$ is a Borel subset² of $\mathrm{TI}$.

7.2 The proof

Let $\sigma$ be a strategy profile. With a slight abuse of notation we shall often write t to mean the history $c^{t}$. We say that player istops infinitely (resp., finitely) often under $\sigma$ if the set $\{t \in \mathbb {N}:\iota (t) = i\text { and }\sigma (t) = s\}$ is infinite (resp., finite). Let $E'(J_{X})$ denote the set of parameters $x \in X$ such that the game $J_{x}$ admits an SPE $\sigma$ with the property that at most one player stops infinitely often under $\sigma$. We first show that $E'(J_{X})$ is Borel and then proceed to the proof that $E(J_{X}) - E'(J_{X})$ is Borel.

Lemma 7.1

The set $E'(J_{X})$ is Borel.

Proof

Without loss of generality assume that for each player i the set $\{t\in \mathbb {N}:\iota (t)=i\}$ is infinite. Define

$$\begin{aligned} \alpha _{i}(x) = \limsup _{\begin{array}{c} t\rightarrow \infty \\ \iota (t)=i \end{array}}x_{i}(t). \end{aligned}$$

Thus $\alpha _{i}(x)$ is the highest payoff player i can get by stopping the game in very deep subgames. It is a Borel measurable function.

Since the payoff functions have finite range, a game $J_{x}$ has an SPE such that all players stop finitely often if and only if $\alpha _{i}(x) \le 0$ for each $i \in I$. Clearly the set $\{x \in X:\alpha _{i}(x) \le 0\text { for each }i \in I\}$ is Borel.

A game $J_{x}$ has an SPE such that player i is the only player to stop infinitely often if and only if it satisfies the following two conditions:

• [1] $0 \le \alpha _{i}(x)$ and

• [2] there exists $n' \in \mathbb {N}$ such that for every $n \in \mathbb {N}$ with $n \ge n'$ there is a $t \in \mathbb {N}$ with $t > n$ such that

  • [2a] $\iota (t) = i$, $\alpha _{i}(x) \le x_{i}(t)$, and

  • [2b] for every k with $n \le k < t$ it holds that $x_{\iota (k)}(k) \le x_{\iota (k)}(t)$.

For suppose that $\sigma$ is an SPE of $J_{x}$ where player i is the only player to stop infinitely often. Then [1] is clearly true. To see that property [2] let $n'$ be the period of time such that no player other than i stops after $n'$. Take an $n > n'$ and let $t > n$ be the earliest period of time such that $\iota (t) = i$ and $\sigma (t) = s$. Conversely, suppose that conditions [1] and [2] hold. Choose $n'$ as in [2]. Let $t_{0}$ be the smallest t which satisfies [2a] and [2b] for $n = n'$. Suppose that $t_{m}$ has been defined for some $m \in \mathbb {N}$. Let $t_{m+1}$ be the smallest t which satisfies [2a] and [2b] for $n = t_{m}$. Define $\sigma$ by letting $\sigma (t) = s$ if and only if $t = t_{m}$ for some m.

The set of $x \in X$ satisfying [1] and [2] is visibly Borel. $\square$

By the above lemma the set $X - E'(J_{X})$ is a Borel subset of X. To prove the theorem it is thus sufficient to show that the set $X - E(J_{X})$ is Borel as a subset of $X - E'(J_{X})$. To do so we construct a continuous function

$$\begin{aligned} f : X - E'(J_{X}) \rightarrow \mathrm{TI}\end{aligned}$$

such that

$$\begin{aligned} X - E(J_{X}) = f^{-1}(\mathrm{TW}(\omega ^{2})). \end{aligned}$$

(7.1)

The result is then implied by the fact that $\mathrm{TW}(\omega ^{2})$ is a Borel set.

For $x \in X$ and $t \in \mathbb {N}$ let $J_{x}(t)$ be a version of the game $J_{x}$ truncated at period t: the game $J_{x}(t)$ proceeds exactly like the game $J_{x}$ until period t. In period t the active player has only one action, s, resulting in the payoff vector x(t).

We presently construct a function $f : X - E'(J_{X}) \rightarrow \mathrm{TI}$. Take an $x \in X - E'(J_{X})$. Let f(x) be the following binary relation $\triangleright$ on the natural numbers. For $t_{0},t_{1} \in \mathbb {N}$ let $t_{0}\triangleright t_{1}$ if the following conditions hold:

• [1] $t_{0} < t_{1}$ and

• [2] the game $J_{x}(t_{1})$ has an SPE $\sigma$ such that

  • [2a] $\sigma (t_{0}) = s$ and

  • [2b] there exists a period t such that $t_{0} \le t < t_{1}$ with $\iota (t) \ne \iota (t_{1})$ such that $\sigma (t) = s$.

Notice that if $\iota (t_{0}) \ne \iota (t_{1})$ then condition [2b] follows from [2a] since we can set $t = t_{0}$. Condition [2b] only has a bite if $\iota (t_{0}) = \iota (t_{1})$. It implies that if $t_{0}\triangleright t_{1}$ and $\iota (t_{0}) = \iota (t_{1})$, then there exists a t such that $t_{0}\triangleright t$ and $t\triangleright t_{1}$ and $\iota (t) \ne \iota (t_{0}) = \iota (t_{1})$. Thus [2b] guarantees that $\triangleright$ has an infinite descending chain if and only if it has an infinite descending chain in which the periods are controlled by at least two distinct players. It is this property that allows us to connect an infinite descending chain of $\triangleright$ to SPE of $J_{x}$.

Lemma 7.2

The function f is continuous.

Proof

It suffices to observe that, for the natural numbers $t_{0} < t_{1}$, whether $t_{0}f(x)t_{1}$ or not depends only on $x(0),\dots ,x(t_{1})$. $\square$

Lemma 7.3

Let $x \in X - E'(J_{X})$ and $\triangleright = f(x)$. Then [1] the relation $\triangleright$ is an element of $\mathrm{TI}$, and [2] $x \in X - E(J_{X})$ if and only if $\triangleright$ is an element of $\mathrm{TW}$.

Proof

[1] The fact that $\triangleright$ is irreflexive is obvious. We prove that $\triangleright$ is transitive.

Take $t_{0},t_{1},t_{2} \in \mathbb {N}$ such that $t_{0}\triangleright t_{1}$ and $t_{1}\triangleright t_{2}$. Then $t_{0}< t_{1} < t_{2}$, and hence $t_{0} < t_{2}$. Let $\sigma ^{\prime }$ be an SPE of $J_{x}(t_{1})$ that witnesses the relation $t_{0}\triangleright t_{1}$, and let $\sigma ^{\prime \prime }$ be an SPE of $J_{x}(t_{2})$ that witnesses the relation $t_{1}\triangleright t_{2}$. Notice that $\sigma ^{\prime }(t_{1}) = \sigma ^{\prime \prime }(t_{1}) = s$. Define

$$\begin{aligned} \sigma = (\sigma ^{\prime }(0),\dots ,\sigma ^{\prime }(t_{1}),\sigma ^{\prime \prime }(t_{1}+1)\dots ,\sigma ^{\prime \prime }(t_{2})). \end{aligned}$$

It is easy to see that $\sigma$ is an SPE of $J_{x}(t_{2})$. We show that $\sigma$ witnesses the relation $t_{0}\triangleright t_{2}$. We have $\sigma (t_{0}) = \sigma ^{\prime }(t_{0}) = s$, establishing [2a]. To check [2b] suppose that $\iota (t_{0}) = \iota (t_{2})$, for otherwise the condition is satisfied. If $\iota (t_{1}) \ne \iota (t_{2})$, then we are done since $\sigma (t_{1}) = \sigma ^{\prime }(t_{1}) = s$. If $\iota (t_{1}) = \iota (t_{2})$, then there is a t such that $t_{1} \le t < t_{2}$ with $\iota (t) \ne \iota (t_{2})$ and $\sigma (t) = \sigma ^{\prime \prime }(t) = s$. Thus $\sigma$ satisfies [2a] and [2b] for $t_{0}$ and $t_{2}$. This proves that $t_{0}\triangleright t_{2}$.

[2] Suppose that $x \in E(J_{X})$. Then $J_{x}$ has an SPE $\sigma$. Since x is not an element of $E'(J_{X})$, at least two distinct players stop infinitely often under $\sigma$. Consequently we can extract a sequence $t_{0}< t_{1} < \dots$ such that $\sigma (t_{n}) = s$ and $\iota (t_{n}) \ne \iota (t_{n+1})$ for each $n \in \mathbb {N}$. For each $n \in \mathbb {N}$ the strategy profile $(\sigma (0),\dots ,\sigma (t_{n+1}))$ is an SPE of the truncated game $J_{x}(t_{n+1})$. Thus $t_{n}\triangleright t_{n+1}$. Hence $\triangleright$ is not well-founded, so that $\triangleright$ is not in $\mathrm{TW}$.

Conversely, suppose that $\triangleright$ is not an element of $\mathrm{TW}$. Thus $\triangleright$ has an infinite descending chain, say $t_{0},t_{1},\dots$. Let $\sigma _{n}$ be an SPE of $J_{x}(t_{n+1})$ that witnesses the relation $t_{n}\triangleright t_{n+1}$. For each $t \in \mathbb {N}$, define $\sigma (t) = \sigma _{n}(t)$ for the unique $n \in \mathbb {N}$ such that $t_{n} \le t < t_{n + 1}$. Then $\sigma$ is an SPE of $J_{x}$, because by [2b] under $\sigma$ at least two distinct players stop infinitely often. Thus $x \in E(J_{X})$. $\square$

The following lemma is the key step of the proof.

Lemma 7.4

Let $x \in X - E(J_{X})$ and $\triangleright = f(x)$. Then $\rho (\triangleright ) < \omega ^{2}$.

Proof

Suppose that $\omega ^{2} \le \rho (\triangleright )$. We work towards a contradiction. The idea of the proof is as follows. Let N be the number of distinct payoff vectors in the game plus 1, a number which is finite since all payoffs are integers bounded by $-D$ and D.

Claim

There exist sequences $t_{0},\dots ,t_{N}$, and $m_{1},\dots ,m_{N}$ such that

• [A] $t_{0}< \dots < t_{N}$ and $t_{N} < m_{n}$ for each $n \in \{1,\dots ,N\}$,

• [B] $\iota (t_{0}) = \cdots = \iota (t_{N}) \ne \iota (m_{1}) = \cdots = \iota (m_{N})$.

For $n \in \{0, \dots , N\}$ let $S_{n} = \{t \in \mathbb {N}:t_{n}\triangleright t\}$ and $M_{n} = S_{n} \setminus (S_{0}\cup \cdots \cup S_{n-1})$.

• [C] For $n \in \{1, \dots , N\}$, we have $m_{n} \in M_{n}$.

We first complete the proof of the lemma assuming the claim, and then come back to prove the claim. For each $n \in \{1,\dots ,N\}$ let $\sigma _{n}$ be an SPE of $J_{x}(m_{n})$ that witnesses the relation $t_{n}\triangleright m_{n}$. Let $v_{n}$ denote the payoff vector that $\sigma _{n}$ induces at period $t_{N}+1$, i.e. $v_{n}^{i} = u_{i}(\pi _{t_{N}+1}(\sigma _{n}))$. We argue that the vectors $v_{1},\dots ,v_{N}$ are all distinct. Suppose to the contrary. Let the numbers $1 \le k < n \le N$ be such that $v_{k} = v_{n}$. Then the strategy profile $\eta = (\sigma _{k}(0),\dots ,\sigma _{k}(t_{N}),\sigma _{n}(t_{N}+1),\dots \sigma _{n}(m_{n}))$ is an SPE of $J_{x}(m_{n})$. Moreover, $\eta (t_{k}) = \sigma _{k}(t_{k}) = s$. Since $\iota (t_{k}) \ne \iota (m_{n})$, we conclude that $t_{k}\triangleright m_{n}$, so that $m_{n} \in S_{k}$, contradicting condition [C].

Proof of the claim Take a sequence $t_{0}^{\prime },t_{1}^{\prime },\dots$ of natural numbers such that $\rho _{\triangleright }(t_{k}^{\prime }) = \omega \cdot k$. The existence of such a sequence follows from the fact that $\rho _{\triangleright }$ maps $\mathbb {N}$ onto the ordinal $\rho (\triangleright )$, which is at least $\omega ^{2}$ by our supposition. Extract an <-increasing subsequence $t_{0},\dots ,t_{N}$ of the sequence $t_{0}^{\prime },t_{1}^{\prime },\dots$ such that $\iota (t_{0}) = \cdots = \iota (t_{N})$. For concreteness assume that $\iota (t_{0}) = \cdots = \iota (t_{N})$ is equal to 1.

Let us say that the number $t \in \mathbb {N}$ is white if $\iota (t) = 1$, and say that it is black otherwise. Let $M_{n}^{\prime }$ be the set of black elements of $M_{n}$.

We argue that for each $t \in M_{n} - M_{n}^{\prime }$ there exists an $m \in M_{n}^{\prime }$ such that $m\triangleright t$. To see this take a $t \in M_{n} - M_{n}^{\prime }$. Then both t and $t_{n}$ are white, and $t_{n}\triangleright t$. It follows by the definition of $\triangleright$ that there exists some black m such that $t_{n}\triangleright m$ and $m\triangleright t$. Thus $m \in S_{n}$. Moreover if m were an element of $S_{i}$ for some $i < n$, relations $t_{i}\triangleright m$ and $m\triangleright t$ would imply that $t_{i}\triangleright t$, contradicting the choice of t in $M_{n}$. Thus $m \in M_{n}^{\prime }$, as desired.

We then have

$$\begin{aligned} \rho _{\triangleright }(t_{n}) = \sup _{t \in S_{n}}(\rho _{\triangleright }(t)+1) = \sup _{t \in M_{n}}(\rho _{\triangleright }(t)+1) = \sup _{m \in M_{n}^{\prime }}(\rho _{\triangleright }(m)+1). \end{aligned}$$

Here the first equation is by the definition of the rank function. The second equation holds because $\rho _{\triangleright }(t_{i}) < \rho _{\triangleright }(t_{n})$ for each $i < n$. The third equation holds by the result of the previous paragraph. It follows in particular that $M_{n}^{\prime }$ is an infinite set, for otherwise $\rho _{\triangleright }(t_{n})$ would be a successor ordinal.

Finally, choose an element $m_{n} \in M_{n}^{\prime }$ such that $m_{n} > t_{N}$. Such a an element exists since $M_{n}'$ is infinite. The numbers $t_{0},\dots ,t_{N}$ are chosen to be white, while $m_{1},\dots ,m_{N}$ are chosen to be black, so that [B] is satisfied. This completes the proof of the claim and of the lemma. $\square$

This establishes the Eq. (7.1) and completes the Proof of Theorem 4.1.

We conclude with the conjecture that one could give an alternative Proof of Theorem 4.1 by using the approach of Sect. 5 and the fact (see Kanovei 1988) that the game projection of clopen sets of $X \times \mathbb {N}^{\mathbb {N}}$ are exactly the Borel sets of X.

8 Parameterized games with bounded payoffs

We have assumed that in parameterized games the payoff function takes only finitely many values. In this subsection we drop the assumption of finite range and require only that the payoff function be Borel measurable and bounded. Since even very simple one-player games may not have a (subgame perfect) equilibrium under these assumptions, it is more natural to consider the question of whether a game has a subgame perfect $\epsilon$-equilibrium, for each positive $\epsilon$.

Consider a perfect information game G as in Sect. 2, but now assuming only that the payoff functions are bounded and Borel measurable. Let $\epsilon > 0$ be an error term. A strategy profile $\sigma$ is a subgame-perfect $\epsilon$-equilibrium ($\epsilon -{\text {SPE}}$) of G if for each history $h \in H$, each player $i \in I$, and each strategy $\sigma _{i}^{\prime }$ of player i it holds that

$$\begin{aligned} u_{i}(\pi _{h}(\sigma )) \ge u_{i}(\pi _{h}(\sigma /\sigma _{i}^{\prime })) - \epsilon . \end{aligned}$$

(8.1)

Consider a parameterized game $G_{X}$ as in Sect. 3, with a bounded Borel measurable payoff function $u_{i}: X \times A^{\mathbb {N}} \rightarrow \mathbb {R}$ for each player $i \in I$. Let

$$\begin{aligned} E^{*}(G_{X}) = \{x \in X:\text{ the } \text{ game } G_{x}\text { has an }\epsilon -{\text {SPE}}\text { for each }\epsilon > 0\}. \end{aligned}$$

We obtain the following analogue of Theorem 4.2.

Theorem 8.1

Let $G_{X}$ be a parameterized game with a bounded Borel measurable payoff function. Then $E^{*}(G_{X})$ is a member of $\Game \mathbf {W}_{X}$.

Proof of theorem 8.1

Let $\mathbb {Q}$ denote the set of rational numbers. Given $\epsilon > 0$ consider the parameterized Martin game $W_{X}^{\epsilon }$, where player I’s moves are pairs $(r_{t},v_{t}) \in A \times \mathbb {Q}^{n}$ and player II’s moves are $a_{t} \in A$. Notice that the actions sets of both players are countable so that we can identify them with $\mathbb {N}$. Consider a play $p^{*} = ((r_{0}, v_{0}), a_{0}, (r_{1}, v_{1}), a_{1}, \dots )$ of the game $W_{X}^{\epsilon }$. With the notation as in the Proof of Theorem 4.2, we define the winning set in $W_{X}^{\epsilon }$ to consist of pairs $(x, p^{*})$ satisfying the following three conditions: For each $t \in \mathbb {N}$

• [1] If $a_{t} = r_{t}$, then $v_{t+1} = v_{t}$.

• [2] If $I_{t} = \emptyset$, then $v_{t}^{i} \le u_{i}(x,p)$ for each $i \in I$.

• [3] If $I_{t} = \{i\}$ for some $i \in I$, then $v_{t}^{i} + \epsilon \ge u_{i}(x,p)$.

The winning set in $W_{X}^{\epsilon }$ is Borel. Let

$$\begin{aligned} E^{\epsilon }(G_{X}) = \{x \in X:\text{ the } \text{ game } G_{x}\text { has an }\epsilon -{\text {SPE}}\}. \end{aligned}$$

We show that for each rational $\epsilon > 0$ we have

$$\begin{aligned} E^{\epsilon /2}(G_{X}) \subseteq \Game W^{\epsilon } \subseteq E^{\epsilon }(G_{X}). \end{aligned}$$

To prove the first inclusion, suppose that $G_{x}$ has an $\frac{\epsilon }{2}$-SPE, say $\sigma$. Define player I’s strategy $\sigma _{I}^{*}$ in $W_{x}^{\epsilon }$ as follows. Let $h^{*}$ be player I’s history in $W_{x}^{\epsilon }$ of length 2t, and let h be the corresponding sequence of moves of player II of length t. Define $\sigma _{I}^{*}(h^{*}) = (r_{t},v_{t})$ where $r_{t} = \sigma (h)$ and

$$\begin{aligned} v_{t}^{i} = \max \{\tfrac{\epsilon }{2}n:n \in \mathbb {Z},\tfrac{\epsilon }{2}n\le u_{i}(x,\pi _{h}(\sigma ))\} \end{aligned}$$

for each $i \in I$. One then shows that $\sigma _{I}^{*}$ is player I’s winning strategy in $W_{x}^{\epsilon }$.

To prove the second inclusion suppose that $\sigma _{I}^{*}$ is player I’s winning strategy in $W_{x}^{\epsilon }$. Define the strategy profile $\sigma$ for $G_{x}$ as follows: Take a history h of length t in $G_{x}$ and let $h^{*}$ be player I’s history in $W_{x}^{\epsilon }$ of length 2t where player II’s moves correspond to h and player I’s moves are obtained using $\sigma _{I}^{*}$. Now suppose that $\sigma _{I}^{*}(h^{*}) = (r_{t}, v_{t})$. We define $\sigma (h)$ to be $r_{t}$. One then shows that $\sigma$ is an $\epsilon -{\text {SPE}}$ of $G_{x}$.

We conclude that

$$\begin{aligned} E^{*}(G_{X}) = \bigcap _{n \in \mathbb {N}}E^{2^{-n}}(G_{X}) = \bigcap _{n \in \mathbb {N}}\Game W^{2^{-n}}. \end{aligned}$$

Since $\Game \mathbf {W}_{X}$ is closed under countable intersections, the result follows. $\square$

We do not know whether theorem 4.1 could be extended to stopping games with a bounded payoff function. The proof of the theorem, especially that of the crucial Lemma 7.4, seem to rely heavily on the assumption that there are only finitely many payoff vectors in the game. However, we are able to sharpen the conclusion of 8.1 somewhat for stopping games. As we argue below the existence domain of a parameterized stopping game with a bounded payoff function is analytic.

Let $J_{Z}$ be a parameterized stopping as in Sect. 3, but with condition [B] replaced by

[B$'$]:

The space of parameters is the product space $Z = [-D,D]^{I \times \mathbb {N}}$.

Theorem 8.2

The set $E^{*}(J_{Z})$ is analytic.

Proof

As before we write t to denote the history $c^{t}$ in the game $J_{Z}$. We identify a strategy $\sigma$ of the game $J_{Z}$ with an element $(\sigma (0),\sigma (1), \dots )$ of $A^{\mathbb {N}}$.

Consider the set

$$\begin{aligned} B^{\epsilon } = \{(z,\sigma ) \in Z \times A^{\mathbb {N}}:\sigma \text { is an }\epsilon -{\text {SPE}}\text { of }J_{z}\}. \end{aligned}$$

We claim that the set $B^{\epsilon }$ is a Borel set. To see this, let us call player i’s strategy $\eta _{i}$ to be a threshold strategy if $\eta _{i}$ recommends that player i never stops, or if there is a time $t \in \mathbb {N}$ such that $\eta _{i}$ recommends player i to stop at period $k \in \mathbb {N}$ with $\iota (k) = i$, if and only if $k \ge t$. Then the set $B^{\epsilon }$ consists precisely of the pairs $(z,\sigma ) \in Z \times A^{\mathbb {N}}$ such that

$$\begin{aligned} u_{i}(z,\pi _{t}(\sigma )) \ge u_{i}(z,\pi _{t}(\sigma /\sigma _{i}^{\prime })) - \epsilon . \end{aligned}$$

for each $i \in I$, each $t \in \mathbb {N}$, and each threshold strategy $\sigma _{i}^{\prime }$ of player i. The claim now follows since the number of threshold strategies is countable.

Let $E^{\epsilon }(J_{Z})$ be the projection of the set $B^{\epsilon }$ on the first coordinate. It is an analytic subset of Z. It follows that $E^{*}(J_{Z}) = \cap _{n \in \mathbb {N}}E^{2^{-n}}(J_{Z})$ is also analytic. $\square$