Fall back proper equilibrium

Proper equilibrium plays a prominent role in the literature on noncooperative games. The underlying thought experiment in which the players play a passive role is, however, unsatisfying, as it gives no justification for its fundamental idea that severe mistakes are made with a significantly smaller probability than innocuous ones. In this paper, we introduce a more active role for the players, leading to the refinement of fall back proper equilibrium.


Introduction: proper equilibrium and its thought experiment
In this paper, we reconsider the concept of proper equilibrium (Myerson 1978) in mixed extensions of finite strategic games, from now on just abbreviated to games. To adequately state our purposes and ideas, we first recall the underlying framework and basic notation and definitions. A game is given by G = (N , . . , n} the player set, M i the mixed strategy space of player i ∈ N , with M i = {1, . . . , m i } the set of pure strategies, and π i : j∈N M j → R the von Neumann Morgenstern expected payoff function of player i. A pure strategy k ∈ M i of player i is alternatively denoted by e i k , a typical element of M i by x i . We denote the probability which x i assigns to pure strategy k by x i k . The set of all strategy profiles is given by = i∈N M i , a typical element of by x. The most fundamental concept in games is that of Nash equilibrium (Nash 1951). A strategy profilex is a Nash equilibrium of G, denoted byx ∈ NE(G), if π i (x) ≥ π i (x i ,x −i ) for all x i ∈ M i and all i ∈ N . Here, (x i ,x −i ) is the frequently used shorthand notation for the strategy profile (x 1 , . . . ,x i−1 , x i ,x i+1 , . . . ,x n ).
The set of Nash equilibria may be very large and can contain counterintuitive outcomes. Selten (1965) introduced the concept of perfect equilibrium as a refinement of the set of Nash equilibria. The essential idea in the thought experiment underlying perfect equilibrium is that no pure strategy should ever be given zero probability, since there is always a small chance that any pure strategy might be chosen, if only by mistake. To further refine the set of (perfect) equilibria, Myerson (1978) introduced the concept of proper equilibrium. Definition 1.1 (Myerson 1978 be an n-player game. A strategy profile x ∈ is a proper equilibrium of G if there exists a sequence {ε t } t∈N of positive real numbers converging to zero, and a sequence {x t } t∈N of completely mixed strategy profiles converging to x such that x t is ε t -proper for all t ∈ N, i.e., The properness concept plays an important role in the game theoretic literature and is widely studied in various directions, see, e.g., van Damme (1984), García-Jurado and Prada-Sánchez (1990), Blume et al. (1991), Yamamoto (1993 and Schuhmacher (1999). In the equilibrium refinement literature, it is featured most prominently in the work on stable sets (Kohlberg and Mertens 1986;Mertens 1989;Hillas 1990 andMertens (1991)), as each stable set contains a proper equilibrium. The attractiveness of the properness concept is mainly based on the fact that this concept selects the intuitively appealing strategy combinations in many (well-known) games (see, e.g., Myerson 1978;van Damme 1991). Moreover, properness captures the extensive form notion of sequential rationality (van Damme 1984). In that sense, we recognize the selective power of proper equilibrium. In our opinion, however, the definition and underlying thought experiment of proper equilibrium are somewhat unsatisfying. In the thought experiment underlying properness the idea is that, just as in the thought experiment underlying perfectness, players make mistakes. Contrary to the concept of perfectness, however, these mistakes are not made randomly; the trembles are somehow sensible, meaning that innocuous mistakes are made with a significantly higher probability than mistakes that have a substantial negative impact on the payoff of the players. However, in the thought experiment players have a passive role in the sense that they do not consciously decide on (an ordering of the) alternatives to their preferred strategies. More precisely, in the thought experiment underlying properness the alternatives are exogenously ordered based on the corresponding payoffs (given the opponent's strategies). Hence, what is missing is an appropriate justification for obtaining this specific ordering. This problem is also addressed in van Damme (1991) who shows that the use of control costs does not provide such a justification. We provide a justification for the fundamental idea underlying properness by starting out from a different, active, thought experiment.
In this alternative approach, each player in the thought experiment is conscious of the fact that both his intended strategy and the intended strategies of his opponents might not be executed. In this approach, we then explicitly model how each player actively anticipates on the occurrence of such events. More specifically, in this thought experiment all the actions of each player are blocked exogenously with a small but positive probability. Since each player wants to play a best reply, each player has to strategically decide beforehand on a back-up action in case his first choice is blocked. However, since this back-up action might be blocked as well, he also has to decide on a second back-up action in case the first back-up action turns out to be unavailable, and so forth and so on. Hence, each player must decide on a complete ordering of his actions beforehand. The probability with which a player is unable to play a certain action is assumed to be independent of the particular choice he makes. This probability may, however, vary between players. We stress that players are not given the opportunity to block each other's actions, but rather that actions can be blocked exogenously. The common ground with proper is that some exogenous event occurs (mistake/blocking), but the difference lies in what happens after that event. In proper, players somehow accidentally coordinate their mistakes, whereas in our setting the players have to consciously make a contingency plan in case their first choice turns out to be impossible for whatever reason (which we call "blocked").
The described thought experiment results in the concept of fall back proper equilibrium, which alternatively can be seen as a hierarchical extension to the concept of fall back equilibrium, introduced by Kleppe et al. (2012) and further discussed in Kleppe et al. (2013). In the original fall back equilibrium concept, a player's pure strategy in the fall back game is an intended pure strategy in the original game, backup up by a single fall back choice rather than a complete hierarchy. Although fall back proper equilibrium is an extension of the idea behind fall back equilibrium, the two solutions are logically unrelated (cf. Kleppe 2010).
Another refinement concept that is also based on a thought experiment with a more active anticipation concept is the notion of informationally robust equilibrium as introduced by Robson (1994) and further elaborated upon by Reijnierse et al. (2007).
To formalize the concept of fall back proper equilibrium, we introduce some additional notation. The action set in the fall back proper game for player i ∈ N within the thought experiment described above equals the set of all orderings of the action set M i , and is denoted by i . Hence, the total number of actions in the fall back proper game for player i equalsm i = m i !. A typical element of i is denoted by σ , where the action on position s of σ is given by σ (s) ∈ M i . A pure strategy σ ∈ i will alternatively be denoted by e i σ .
The mixed strategy space of player i is given by i . We assume that each action of player i is blocked with the same probability, denoted by ε i , but we allow for different probabilities among the players. Hence, let ε = (ε 1 , . . . , ε n ) be an n-tuple of (small) non-negative probabilities. If player i plays action σ ∈ i in the fall back proper game, he plays with probability (1 − ε i )(ε i ) s−1 action σ (s) of the game G for s ∈ {1, . . . , |m i |}. With probability (ε i ) m i , all actions of player i are blocked, the game is not played and the payoff to all players is defined to be zero.
The fall back proper gameG(ε) = (N , is the mixed extension of the corresponding finite game withm i pure strategies for each player i ∈ N . The payoff functions {π i ε } i∈N on mixed strategy combinations in i∈N i are derived in the standard way using expected payoffs from the payoff functions on pure strategy combinations in i∈N i , as described by The residual probability in which at least one player is unable to play any of his actions is implicitly incorporated in this payoff function, as in that case the payoff to every player is zero. Note that the zero payoff is arbitrary and will not influence the equilibria of the game, because it does not depend on the players' strategy choices.
A typical element of i is denoted by ρ i , and the probability which ρ i assigns to pure strategy σ is given by ρ i σ . The set of all strategy profiles is given by˜ = i∈N i , an element of˜ by ρ.
be an n-player game. A strategy profile x ∈ is a fall back proper equilibrium of G if there exists a sequence {ε t } t∈N of n-tuples of positive real numbers converging to zero, and a sequence {ρ t } t∈N such that ρ t ∈ NE(G(ε t )) for all t ∈ N, converging to ρ ∈˜ , with x i k = σ ∈ i k ρ i σ for all k ∈ M i and all i ∈ N . The set of fall back proper equilibria of a game G is denoted by FBPR(G).
In the thought experiment underlying fall back proper equilibrium, all the actions of each player are blocked with a small but positive probability. Therefore, players decide beforehand on a complete ordering of their actions. This is modeled by letting players play the fall back proper game in which each action consists of a full ordering of the actions of the original game such that the first action is played with a probability close to one and each following action with a smaller probability of a fixed factor. A fall back proper equilibrium of the original game is then deduced from the limit point of a sequence of Nash equilibria of the corresponding fall back proper games when the blocking probabilities converge to zero.
Fall back proper equilibrium can be seen as a hierarchical extension of fall back equilibrium 1 (Kleppe et al. 2012), in which players are only allowed to use a single backup action. As a result, one might think that the set of fall back proper equilibria refines the set of fall back equilibria. We refer to Kleppe (2010) for an example which shows that this is not the case. This paper shows that fall back proper equilibrium is a refinement of proper equilibrium. So, interestingly, the more active thought experiment underlying fall back proper equilibrium sheds some new light on the original, more passive thought experiment underlying proper equilibrium. We show that the two concepts coincide for two-player games. In general, fall back proper constitutes a refinement of proper, where designing an example where the concepts differ requires an intricate construction.
The outline of the remainder of the paper is as follows. In Sect. 2, we provide an alternative characterization of fall back proper equilibrium based only on limitations of the strategy spaces. Using that characterization, we show in Sect. 3 that the set of fall back proper equilibria is a (possibly strict) non-empty and closed subset of the set of proper equilibria, and in Sect. 4 that for two-player games the sets of proper and fall back proper equilibria coincide. Section 5 concludes the paper.

A characterization of fall back proper equilibrium
In this section, we provide an alternative characterization of fall back proper equilibrium in which the perturbations of the thought experiment are fully captured by limitations of the strategy spaces. This allows for a perturbed game of the same dimensions as the original one. For a (sufficiently small) blocking vector δ ∈ R N + , the blocking game in the sense that the strategy spaces are restricted to for all i ∈ N , with the domains of the payoff functions restricted accordingly. We define the set of all strategy profiles of the blocking game by (δ) = j∈N M j (δ j ). Note that this blocking game gives the maximum probability by which each number of actions can be played, e.g., if player i puts the maximum allowed probability on the actions in a set T i , then any other strategy k / ∈ T i can be played with a probability of at most blocking vector, and letG(δ) be the corresponding fall back proper and blocking game, respectively. Then, there exists an onto map f δ :˜ → (δ) such that π i δ (ρ) = π i ( f δ (ρ)) · j∈N (1 − (δ j ) m j ) for all ρ ∈˜ and all i ∈ N .
Proof We explicitly construct a map f δ satisfying the conditions of the lemma. Let ρ ∈˜ . We define f δ (ρ) = x, with for all k ∈ M i and all i ∈ N . By considering the most extreme case in which ρ i σ is a pure strategy in the fall back proper game, it is readily checked that k∈T i . Furthermore, the probabilities put by strategy profile x on all the action profiles in the game G are equal to the probabilities put by ρ on these action profiles multiplied by 1 is readily checked that f δ is onto.
As a consequence of Lemma 2.1, a fall back proper equilibrium can also be defined in terms of a sequence of Nash equilibria of blocking games.

an n-player game. Then, a strategy profile x ∈ is a fall back proper equilibrium of G if and only if there exists a sequence {δ t } t∈N of blocking vectors of positive real numbers converging to zero and a sequence {x t } t∈N converging to x such that x t ∈ N E(G(δ t )) for all t ∈ N.
Proof We just prove the "only if" part, the reverse statement can be shown analogously. Assumex ∈ FBPR(G). Then by definition, there exists a sequence {δ t } t∈N of n-tuples of positive real numbers converging to zero, and a sequence {ρ t } t∈N converging tô ρ ∈˜ , withx i k = σ ∈ i kρ i σ for all k ∈ M i and all i ∈ N , such thatρ t ∈ NE(G(δ t )) for all t ∈ N. By Lemma 2.1, there exists a sequence {x t } t∈N converging tox ∈ , and all t ∈ N, which proves thatx t ∈ NE(G(δ t )) for all t ∈ N and, therefore, completes the proof. Let t ∈ N and let (x i t ,x −i t ) ∈ (δ t ). Then by Lemma 2.1, we can take a strategy Note that it immediately follows from Theorem 2.1 that each completely mixed Nash equilibrium is a fall back proper equilibrium.

General results
In this section, we show that the set of fall back proper equilibria is a (possibly strict) non-empty and closed subset of the set of proper equilibria.
Theorem 3.1 Let G be an n-player game. Then, each fall back proper equilibrium of G is a proper equilibrium of G.
be an n-player game and let x ∈ FBPR(G). Then by Theorem 2.1, there exists a sequence {δ t } t∈N of blocking vectors converging to zero, and a sequence {x t } t∈N such that x t ∈ NE(G(δ t )) for all t ∈ N, converging to x ∈ . Let the sequence {ε t } t∈N be given by ε t = max i∈N δ i t for all t ∈ N. Let i ∈ N and let is a sequence of positive real numbers converging to zero and {x t } t∈N is a sequence of completely mixed strategy profiles converging to x such that for all t ∈ N for all k, ∈ M i and all i ∈ N . Hence, x is a proper equilibrium.
Hence, the set of fall back proper equilibria is a subset of the set of proper equilibria.
The following theorem states that this subset is non-empty and closed.
Theorem 3.2 Let G be an n-player game. Then, the set of fall back proper equilibria of G is non-empty and closed.
Proof We first show non-emptiness. Let {δ t } t∈N be a sequence of blocking vectors converging to zero. Take a sequence {x t } t∈N such that x t ∈ NE(G(δ t )) for all t ∈ N.
Since the strategy spaces are compact, there exists a subsequence of {x t } t∈N converging to, say, x ∈ . By Theorem 2.1, x ∈ FBPR(G). Secondly, we show that FBPR(G) is closed. Take a converging sequence {x t } t∈N with x t ∈ FBPR(G) for all t ∈ N, with limit x. For all t ∈ N there exists a sequence {δ tr } r ∈N of blocking vectors converging to zero and a sequence {x tr } r ∈N converging to x t such that x tr ∈ NE(G(δ tr )) for all r ∈ N. Considering the sequences {δ tt } t∈N and {x tt } t∈N , one readily establishes that x ∈ FBPR(G).
Using a quite specific construction, the following example is designed to show that the set of fall back proper equilibria can be a strict subset of the set of proper equilibria.
Example 3.1 Consider the following three-player game G in which the third player chooses one of the four matrices, e 3 1 , e 3 2 , e 3 3 or e 3 4 , respectively. In this example, it is possible to coordinate the probabilities on the lower-level actions in such a way that x = (e 1 1 , e 2 1 , e 3 and z are equally important for player 3 since twice a fourth choice (e 1 4 , e 2 4 ) has the same number of factors δ as one fifth and one third choice (e 1 5 , e 2 3 ). 2 As a result, the 4 at z trumps the 1 at y and consequently, x is not fall back proper. Nevertheless, we can still construct a sequence of x t supporting x as a proper equilibrium. We only have to take care that player 1's fifth choice (his worst pure strategy) is played with a sufficiently small probability to neutralise the 4 in z. In fall back proper equilibrium, it is not allowed to coordinate small probabilities on specific deviations in the game.
Note that to make the construction in Example 3.1 work, we need the three payoffs of 10 (one per player) outside the proper equilibrium. For the remaining payoffs in the trimatrix, we actually have some slack, as can be seen from the final inequality, δ 1 t ≥ 4δ 1 t . If you replace the 4 by any constant larger than 1, the same contradiction follows. As a result, the class of games for which the concepts of proper equilibrium and fall back proper equilibrium do not coincide does not have measure zero.

Results for two-player games
In the previous section, we showed that in general the set of fall back proper equilibria is a (possibly strict) subset of the set of proper equilibria. Interestingly, for two-player games the sets of proper and fall back proper equilibria coincide.

Theorem 4.1 Let G be a two-player game. Then, the sets of proper and fall back proper equilibria of G coincide.
Hence,x i