Costless Delay in Negotiations

We study strategic negotiation models featuring costless delay, general recognition procedures, endogenous voting orders, and finite sets of alternatives. Two examples show: 1. non-existence of stationary subgame-perfect equilibrium (SSPE). 2. the recursive equations and optimality conditions are necessary for SSPE but insufficient because these equations can be singular. Strategy profiles excluding perpetual disagreement guarantee non-singularity. The necessary and sufficient conditions for existence of stationary best responses additionally require either an equalizing condition or a minimality condition. Quasi SSPE only satisfy the recursive equations and optimality conditions. These always exist and are SSPE if either all equalizing conditions or all minimality conditions hold.


Introduction
Strategic negotiation theory has contributed signi…cantly to the understanding of negotiation processes. 1 Many in ‡uential contributions analyze limits of vanishing costly delay and some consider costless delay. 2 Costly delay is often modeled as a risk of breakdown or discounting. Negotiation models with costly delay have the property of continuity at in…nity, a su¢ cient condition under which the one-stage-deviation property characterizes subgame-perfect equilibrium (SPE) and SPE in stationary strategies (SSPE). This property states that all one-stage deviations are unpro…table. Negotiation models with costless delay, however, lack continuity at in…nity, see e.g. Bloch (1996). Also, the players' expected utilities fail lower semi-continuity in stationary strategies, a condition under which Alós-Ferrer and Ritzberger (2012) establish the equivalence between SPE and the onestage-deviation property in discrete extensive forms with perfect information. This raises the obvious question whether the one-stage-deviation property still characterizes equilibria under costless delay, which we consider to be stationary for explanatory reasons.
We show the necessity to address these issues by means of two motivating examples. The most intriguing example is the symmetric hedonic game of coalition formation proposed in Bloch (1996) and Bloch and Diamantoudi (2011) that has no SSPEs in pure strategies under costless delay. We derive the symmetric SSPE in mixed strategies under costly delay, and derive its limit under vanishing costly delay. Even though the SSPE's limit strategy pro…le is well-de…ned, we show that it fails as an SSPE under costless delay.
Moreover, we show non-existence of symmetric SSPE in mixed strategies under costless delay. Technically speaking, we show that the correspondence of symmetric SSPE strategies lacks upper semi-continuity, may fail to be closed and may even be empty valued. A puzzling phenomenon is that the symmetric SSPE converges to a strategy pro…le that induces Pareto ine¢ cient perpetual disagreement, whereas the corresponding SSPE utilities converge to Pareto e¢ cient utilities. Also puzzling is that the limit SSPE utilities are a solution to the system of recursive equations, but fail to represent the correct expected utilities. We provide an explanation for these phenomena and derive the necessary and su¢ cient conditions that do characterize SSPEs in mixed strategies.
We address these issues in a general negotiation model in discrete time with an arbitrary number of players, stochastic recognition of the proposing player, public and sequential en-1 Rubinstein (1982) boosted the literature on strategic negotiations. For surveys we refer to e.g. Osborne and Rubinstein (1990), Muthoo (1999), Houba and Bolt (2002), Banks and Duggan (2006), and Ray (2007). 2 Costless delay is analyzed in e.g. Binmore (1986), Perry and Reny (1994), Moldovanu and Winter (1995), Bloch (1996), Banks and Duggan (2000), Dávila and Eeckhout (2008), Horniaµ cek (2008), and Herings and Houba (2010). dogenous voting orders, and discrete sets of feasible alternatives. We explicitly include costly and costless delay to enhance studying limits of vanishing costly delay. Also, this enables us to pin down the di¤erences between costly and costless delay. At any negotiation round, one player is recognized to make a proposal. Proposals specify an alternative, a set of players who have the right to approve this alternative, and an order in which the players in this set sequentially and publicly vote. The …rst vote against the proposed alternative ends the current round of voting. After that, nature decides whether the negotiations permanently break down, or who will be next round's recognized player. Our model's recognition rules represent more general institutions than analyzed in the literature so far and allow many special cases: Fixed rotating orders of recognized players including alternating-o¤ers procedures; Markov recognition probabilities including stationary random recognition rules; and coalitional negotiation procedures including endogenous protocols such as the rejector-becomes-proposer protocol in e.g. Selten (1981). 3 The players'preferences are represented by expected utility functions.
The main reasons to assume public and sequential voting in our model are that i) it captures such voting rules of several negotiation models in the literature and ii) that, in SSPE, such voting rules are equivalent to the stage-undominated voting strategies under simultaneous voting rules as in Baron and Kalai (1993). So, our model implicitly obtains the most appealing voting strategies under simultaneous voting. The endogenous voting orders in our model extend upon the exogenous voting rules in the literature.
This class of negotiation models belongs to the class of recursive games with perfect information, which is a subclass of stochastic games. Under costly delay, existence of an SSPE is not an issue, since it follows from standard results on equilibrium existence in stochastic games, see e.g. Fink (1964), Takahashi (1964), and Sobel (1971). For the class of stochastic games, Haller and Laguno¤ (2000) show that generically the set of SSPEs is …nite, and Herings and Peeters (2004) show that generically there is an odd number of SSPEs. For stochastic games under costless delay, using the average reward criterion to evaluate payo¤ streams, non-existence of Nash equilibrium in mixed strategies has been noted by Blackwell and Ferguson (1968) and has spurred an extensive literature seeking existence of weaker notions of Nash equilibrium in special classes of stochastic games and conditions under which SPE exist. For "-Nash equilibria, existence has been shown by Mertens and Neyman (1981) for two-person zero-sum stochastic games and by Vieille (2000aVieille ( , 2000bVieille ( , and 2000c, for general two-player stochastic games. A general result for stochastic games with three or more players is lacking thus far. For the subclass of recursive games with non-negative utilities, Flesch et al. (2010) have demonstrated the existence of a subgame-perfect "-equilibrium for every " > 0. For a class of coalitional bargaining models that belong to the class of negotiation models we consider, Bloch and Diamantoudi (2011) derive a necessary and su¢ cient condition for the existence of pure SSPEs under costless delay. General conditions for existence of mixed SSPE in stochastic recursive games is an open issue.
Under costly delay, the one-stage-deviation property is equivalent to dynamic programming, which can be separated into the optimality conditions and the recursive equations. 4 Both characterize SSPE strategy pro…les and their conditional expected utilities. For costless delay, we denote strategy pro…les that satisfy the optimality conditions and the recursive equations as quasi SSPE. Quasi SSPE always exist, but as our motivating examples show these may fail to be SSPE.
We will now summarize our main results. Our …rst main result states exclusion of perpetual disagreement is the necessary and su¢ cient condition such that the expected utilities induced by a stationary strategy pro…le constitute the unique solution to the recursive equations. The explanation is as follows: Stationary strategy pro…les induce a Markov process and the corresponding expected utilities can be expressed in terms of this process. These utilities always satisfy the recursive equations, but it is not the case that any solution to the recursive equations corresponds to the expected utilities. The Markov process has absorbing states that either represent which agreement has been reached or represent permanent breakdown. This process might cycle forever on the other states, which represent who is recognized. Such cycling is excluded by the necessary and su¢ cient condition. Of course, under costly delay, the positive risk of breakdown excludes forever cycling a priori, and then the recursive equations admit a unique solution. Only under costless delay it may occur that the optimality conditions return stationary strategy pro…les with forever cycling, called perpetual disagreement from here on, and then the recursive equations have the entire Null space as its solution, which is of dimension one or higher.
Indeed, this is the case in one of our examples: the limit symmetric SSPE induces perpetual disagreement, which explains one puzzle for the limit SSPE.
Our examples illustrate that, in case of singularity of the recursive equations under costless delay, dynamic programming is no longer su¢ cient to characterize the players' best responses and, consequently, for SSPE. Our second main result provides a characterization of SSPE that is also valid when the recursive equations are singular. Given the other players'stationary strategies, each player's best response is the optimal solution of a stationary Markov decision process with expected total rewards that are bounded. We show that such decision processes are well-de…ned for our class of negotiation models and that stationary best responses exist. The necessary and su¢ cient conditions for stationary best responses require that, in addition to the conditions of optimality and recursive equations, either an equalizing condition or a minimality condition has to be satis…ed.
The equalizing condition is a necessary condition for a solution to the recursive equations to coincide with the expected utilities induced by stationary strategies. The minimality condition is needed to select among several expected utilities in case the optimality conditions and the recursive equations return several candidates for the optimal solution with di¤erent induced expected utilities. What is relevant for practical purposes, whenever the recursive equations admit a unique solution, then both the minimality condition and the equalizing conditions automatically hold. This is equivalent to a player's stationary best response against the other players'stationary strategies that, combined to form a strategy pro…le, exclude perpetual disagreement.
In essence, SSPE requires that each player's stationary strategy is a best response given the stationary strategies of the other players. This means that all the necessary and suf-…cient conditions of all the stationary Markov decision processes of the individual players together form the equilibrium conditions. A su¢ cient condition such that all SSPE strategy pro…les exclude perpetual disagreement is the following: Every player is able to propose some alternative and some coalition whose members all prefer this alternative to the status quo. This is a mild assumption that relaxes the popular assumption of an essential bargaining problem. It is also easy to check. Under this assumption, SSPE strategy pro…les are equivalent to the optimality conditions and the recursive equations on the domain of strategy pro…les that exclude perpetual disagreement. The latter automatically holds for costly delay. Under costless delay, all strategy pro…les with perpetual disagreement have to be discarded a priori and the resulting subdomain of strategy pro…les is no longer closed, which is a technical problem in establishing existence of equilibria. Our existence result for quasi SSPE is an inferior substitute for existence of SSPE. Although SSPEs are popular in negotiation theory and are axiomatized in Bhaskar et al. (2013), these are also criticized as being too speci…c. Our focus on SSPE obscures that many of our results are more general than might appear. At the end of our paper, we show that, by enlarging the state space, more general results can be immediately obtained.
Our results extend to SPE on the class of non-stationary strategy pro…les that can be represented as …nite automata, which covers most of the relevant strategy space in many negotiation models, see e.g. the discussion in Section 3.5 in Osborne and Rubinstein (1990).
This class of automata includes the automata needed to apply the method proposed in Shaked and Sutton (1984) to establish lower and upper bounds on the set of SPE payo¤s.
The enlarged state space can also capture multilateral contracting as in e.g. Gomes (2005) and stochastically ‡uctuating sets of feasible utilities as in e.g. Merlo and Wilson (1995) and Duggan (2011). This paper is organized as follows. After the introduction of the negotiation model and discussing preliminaries such as dynamic programming, the one-stage-deviation principle and quasi SSPE in Section 2, two motivating examples are discussed in Section 3. The necessary and su¢ cient conditions such that the recursive equations admit a unique solution are established in Section 4. The necessary and su¢ cient conditions for stationary best responses are derived in Section 5. In Section 6 the SSPE conditions are stated, a su¢ cient condition that excludes perpetual disagreement in SSPE is proposed, and vanishing costly delay is investigated. Section 7 discusses how our negotiation model incorporates several in ‡uential negotiation models and presents several extensions of our results. Section 8 concludes.

The Model and Preliminaries
In this section, we introduce our model before we discuss some preliminaries that identify the technical issues to be dealt with and that are also necessary for the motivating examples.
The description of the model and the preliminaries are separated into two subsections.

The Model
Consider n 2 players who negotiate the selection of an alternative from m 1 alternatives in the shadow of a status quo under sequential and public voting. Players are indexed by i and belong to the …nite set N = f1; : : : ; ng : The status quo is the outcome under breakdown and it is denoted by q: Proposals consist of an alternative and a voting order of a decisive coalition, i.e., a group of players who have the right to approve in order for a proposal to be accepted. Voting orders are permutations of groups of players that form decisive coalitions. The set A = fa 1 ; : : : ; a m g denotes the …nite set of feasible alternatives. We assume without loss of generality that q = 2 A. As in many in ‡uential models, the recognized player is assumed to cast a vote in favor of his proposed alternative and is excluded in the proposed voting order.
Formally, for C a proper subset of N , the collection (C) consists of all permutations of the players in C, and the set O [ C22 N :jCj<n (C) consists of all feasible voting orders.
The non-empty set X i A O denotes the set of feasible proposals of a recognized player i. So, a proposal (a; o) 2 X i consists of an alternative a 2 A and a voting order o 2 O of players that does not include player i. Our formulation allows for the possibility that the set of decisive coalitions depends upon the proposed alternative.
Negotiations proceed in discrete time, where t 2 N denotes round t. At round t, recognized player i t 2 N …rst proposes x t = (a t ; o t ) 2 X i t , after which all players in the range C(o t ) of o t sequentially and publicly vote in the order described by o t . Given Otherwise, the …rst voter in C(o t ) against ends the voting in round t and alternative a t is rejected. The identity of the …rst voter in C(o t ) against is denoted by r t : If all players in o t approve, we de…ne r t = 0: If r t 2 N , round t is concluded with a draw by nature that is modeled as a compound lottery: First, nature decides with probability 2 [0; 1] whether the negotiations proceed to round t+1. With complementary probability 1 the negotiations break down, leading to the implementation of the status quo q: Note that it is standard to identify costly delay with < 1 and costless delay with = 1. Second, in case negotiations proceed to round t + 1, nature recognizes player i at round t + 1 with probability Prior to the …rst round, nature recognizes player i with probability i 2 [0; 1], where P i2N i = 1. The negotiation procedure …ts the framework of multi-stage games with perfect information, see e.g. Fudenberg and Tirole (1991). It has n + 1 stages per round t 2 N under the understanding that i) at most one player is active per stage and all other players choose the trivial action "do nothing" ii) all players do nothing either after all players in C (o t ) have voted in favor or after the …rst vote against. In terms of Maskin and Tirole (2001), the multi-stage game is cyclical with cycle length n + 1. Stages are indexed (t; k), t 2 N and k = 1; : : : ; n + 1. The recognized player proposes at stage k = 1, all other players sequentially vote or do nothing at stages k = 2; : : : ; n and nature moves at stage k = n + 1. As soon as a proposal is accepted, the negotiations end and the draw by nature becomes trivial.
To keep track of the voting behavior in the various stages k = 1; : : : ; n + 1, we de…ne r t;k 2 C(o t ) [ f0g as follows. Since there is no voting in stage (t; 1); we set r t;1 = 0. For k = 2; : : : ; n + 1, we de…ne r t;k = 0 if no rejection has occurred in stages (t; 2); : : : ; (t; k 1). Otherwise, r t;k is equal to the …rst player in C(o t ) who rejected the proposal. Notice that r t;n+1 = r t : Finally, r t;n+1 = r t = 0 implies that the proposed x t at round t is accepted after which the bargaining ends and all players do nothing forever.
Histories are de…ned recursively for all t 2 N and k = 1 : : : ; n + 1. The history up to stage (t; k) is denoted h t;k . The initial history h 0;n+1 = ;: For t 2 N, de…ne the history at the …rst stage of round t as h t;1 = (h t 1;n+1 ; i t ), the history at the second stage as h t;2 = (h t;1 ; x t ) ; and the history at stages k = 3; : : : ; n + 1 as h t;k = h t;k 1 ; r t;k . The non-empty and …nite set of all histories up to stage (t; k) is denoted H t;k and the set of all histories is H = [ (t;k)2N f1;:::;n+1g H t;k . Since the negotiation procedure has perfect information, histories de…ne subgames and vice versa.
Mixed behavioral strategies and strategy pro…les are de…ned in the usual way: i is a function from the set of histories at which player i has to act into a probability distribution over the history-dependent set of feasible actions and i denotes the set of all such strategies. A strategy pro…le is 2 i2N i . Sometimes we write = ( i ; i ). Any strategy pro…le 2 induces cumulative probabilities that some agreement is accepted prior to or at round t. For 2 , (a; t; ; ) 2 [0; 1] denotes the cumulative probability of reaching agreement on a 2 A at a round t. For all 2 , these cumulative probabilities are well-de…ned, non-decreasing in t, and bounded due to P a2A (a; t; ; ) 1 for all t 2 N. Hence, for all a 2 A, (a; t; ; ) converges as t goes to in…nity and we de-…ne (a; ; ) as this limit cumulative probability: (a; ; ) = lim t!1 (a; t; ; ). Note that (q; ; ) = 1 P a2A (a; ; ) is the probability of perpetual disagreement plus the probability of breakdown. In particular, (q; ; ) = 1 implies (a; ; ) = 0 for all a 2 A.
Players have expected utility functions. Player i derives utility from agreed upon alternatives and the status quo alternative denoted by the numbers u i (a), a 2 A, respectively, u i (q). Because expected utility functions are unique up to a¢ ne transformations, we use the normalization u i (q) = 0 and u i = max a2A[fqg u i (a) 0. Also, we de…ne u i = min a2A[fqg u i (a) 0. Expected utilities are de…ned in the usual way. Finally, we de…ne the set of all feasible utility pro…les as U = convf(u 1 (a); : : : ; u n (a)) 2 R n j a 2 A [ fqgg; where conv denotes the convex hull of a set.
In terms of cumulative probabilities, player i's expected utility of 2 is given by Note that (U 1 ( ; ) ; : : : ; U n ( ; )) 2 U : In case (q; ; ) = 1, it holds that the expected utility U i ( ; ) = 0. We assume non-negative utilities: For all i 2 N , u i = 0. As will be made clear later, under costless delay this assumption ensures that each player's best response against arbitrary stationary strategies will be the optimum of a positive bounded model in terms of Markov Decision Theory.
Since we have a multi-stage game with perfect information, the concept of subgameperfect equilibrium (SPE) is appropriate. A strategy pro…le is an SPE if no player has a pro…table deviation at any history. Our main analysis deals with strategy pro…les that we call stationary strategy pro…les.
We …rst de…ne an exogenous partition of the set of all histories, and then de…ne strategy pro…les on this partition. 5 The partition for non-trivial rounds is as follows: At stage (t; 1) only the identity of the recognized player matters. At voting stages (t; 2); : : : ; (t; jo t j + 1), the identity of the recognized player and the proposal x t matter. Formally, for i 2 N , x 2 X i , and k 2 f2; : : : ; ng, we de…ne i; x; 0; : : : ; 0) for some t 2 N : A stationary strategy S;i for player i speci…es S;i h t;k = S;i ( h t 0 ;k ) whenever either k = 1 and h t;k ; h t 0 ;k 2 H(i) or it holds that k 2 f2; : : : ; ng, h t;k ; h t 0 ;k 2 H(j; x; k) for j 6 = i; and x 2 X j . Therefore, player i's stationary strategy re ‡ects that bygones are bygones.
When player i 2 N is chosen as the recognized player, he chooses a history-independent probability distribution over X i : When player i 2 N is chosen as a responder at stage (t; k) ; he conditions his behavior only on the recognized player and the proposal made. We denote (X i ) as the space of probability distributions on X i and we de…ne recognized player i's randomized proposal as i 2 (X i ) with i (x) as the probability that x 2 X i is proposed. Similarly, we de…ne i (j;

Preliminary Results
The main motivation for our analysis is that SPE and SSPE are well understood under costly delay and that open issues appear under costless delay. For costless delay, matters are less straightforward and it will pay o¤ to focus on SSPE before considering SPE in the class of …nite automata. For 2 [0; 1), existence of an SSPE is not an issue, since it follows from standard results on equilibrium existence in stochastic games, see e.g. Fink (1964), Takahashi (1964), and Sobel (1971). For the class of stochastic games, Haller and Laguno¤ (2000) show that the set of SSPEs is generically …nite, and Herings and Peeters (2004) show that generically there is an odd number of SSPEs and they provide an algorithm to compute the equilibrium that would be selected by a generalization of the tracing procedure. Also for 2 [0; 1), the one-stage-deviation principle, see e.g. Blackwell (1965) and Fudenberg and Tirole (1991), applies to characterize SPEs in our negotiation model. It states that, for any strategy pro…le, SPE is equivalent to the one-stage-deviation property. Furthermore, SSPEs can be characterized using dynamic programming techniques as …rst pioneered by Bellman (1953) and Shapley (1953). To formalize these dynamic programming techniques and the one-stage-deviation property, we need to de…ne the state space …rst, where we con…ne this space to states where either players take non-trivial actions or states are absorbing.
Stationary strategy pro…le S induces a stationary Markov process on the state space where A [ fqg is the set of absorbing states associated with having reached either agreement a 2 A, or the status quo outcome q under breakdown. The matrices denoted P S S ; , P A S ; and P q S ; assign transition probabilities, respectively, from S to S, S to A, and S to fqg. We state the following result without proof.
Moreover, all probabilities in S ; are continuous in ( ; ) and .
The initial distribution and these Markov transition probabilities determine the probabilities of reaching states in S [ A [ fqg at the start of round t. These probabilities in turn determine the cumulative probabilities a; t; S ; . In order to state these probabilities, denote S = S 1 ; : : : ; S n 2 R jSj + as the initial distribution over states in S, where S i = ( i ; 0; : : : ; 0) 2 R jS i j + , i 2 N . 6 Notice that S puts probability zero on the voting states V and that the probability on a state i 2 N is equal to i as speci…ed in the previous subsection. We state the following result without proof.
Lemma 2 For S = ( ; ), the probability that player i 2 N is recognized at round t 2 N is the element of S P S S ; n(t 1) associated with s = i, i 2 N , the probability that alternative a`2 A is approved in round t 2 N is the`-th element of S P S S ; n(t 1) P A S ; and the probability of breakdown q at round t 2 N is equal to S P S S ; n(t 1) P q S ; .
Furthermore, the cumulative probability a`; t; S ; that alternative a`2 A is approved on or before round t is the`-th element of Moreover, all these probabilities are continuous in ( ; ) and .
For …nite t 2 N, all cumulative probabilities encountered thus far are continuous in the stationary strategy pro…le S and 2 [0; 1]. Recall that, for all S and 2 [0; 1], the limit probability a; S ; = lim t!1 a; t; S ; exists. Given stationary strategy pro…le S and 2 [0; 1], player i's conditional expected payo¤ in state s 2 S is denoted v i (s; S ; ), which in vector notation is written as v i S ; 2 R jSj .
The following result is also given without proof.
Lemma 3 For stationary strategy pro…le S = ( ; ) and 2 [0; 1]: where u i = (u i (a 1 ) ; : : : ; u i (a m )) > , and ; s 2 N , through induction. This latter insight conforms with the common practice to analyze negotiation models on the subset of states that can be reached at the beginning of

Quasi SSPE and the One-Stage-Deviation Property
Given S; i , player i's set of best responses consists of the set of all optimal strategies of a stationary Markov Decision Process (MDP) in which the subset S i represents all states where player i takes a non-trivial decision. For 2 [0; 1), each player's MDP is a well-de…ned MDP with discounting. Such an MDP always returns at least one stationary strategy as one of the possibly many optimal strategies that can be either characterized by dynamic programming techniques or by the one-stage deviation (OSD) property, which says that a player cannot get strictly higher expected payo¤s by a one-shot deviation in a single state, conditional on being in that state. For = 1, each player's objective function corresponds to the expected total-reward criterion and matters become di¤erent.
Without going into details at this moment, each player's MDP always returns at least one stationary strategy as one of the possibly many optimal strategies that can be characterized by dynamic programming techniques if some additional condition is invoked. Without the additional condition, applying the dynamic programming techniques may lead to solutions that do not satisfy the OSD property and should be regarded as quasi solutions. Whatever 2 [0; 1], combining the MDPs of all the players together determines SSPE. In order to be clear, we will make precise what we mean.
We de…ne solutions when applying the dynamic programming techniques without the additional condition as quasi SSPE. Formally, an arbitrary conditional expected payo¤ for player i in state s 2 S is denoted w i (s), which in vector notation is written as w i 2 R jSj . We introduce the following de…nition.
De…nition 4 For 2 [0; 1], the strategy pro…le S = ( ; ) is a quasi SSPE if, for each player i 2 N , there exist values w = (w i ) i2N such that 1. The optimality conditions hold: 2. The recursive equations hold: w 2 U jSj and, for every i 2 N; The restriction w 2 U jSj is very natural and super ‡uous for 2 [0; 1). However, for = 1 the restriction rules out solutions to (4) and (5) with w = 2 U jSj that lack any interpretation in the motivating Example 8 in the next section. The following result states that quasi SSPEs exist. We defer all proofs to the appendix.
The following result summarizes the discussion for costly delay and is stated without proof.
Theorem 6 For 2 [0; 1), the one-stage-deviation principle applies and the sets of SSPEs and quasi SSPEs coincide.
As mentioned in the introduction, our motivating examples reveal that some quasi SSPE may fail to be SSPE. In those cases, we also observe that, for the quasi SSPE strategy pro-…le S , the values w i (s) of De…nition 4 for states s 2 S di¤er from the conditional expected payo¤s v i s; S ; in (3) for those states. The application of dynamic programming underlying quasi SSPE wrongly suggests that we also satisfy robustness against one-stage deviations. However, the OSD property for stationary strategy pro…les is the absence of one-stage deviations that are pro…table with respect to the conditional expected payo¤s v i s; S ; . The following de…nition makes this formal.
De…nition 7 For 2 [0; 1], the OSD property holds for the strategy pro…le S = ( ; ) if, for each player i 2 N , the optimality conditions (4) hold with w i equal to the conditional expected utilities v i ( S ; ) given by (3).
When we compare the de…nitions of quasi SSPE and the OSD property we have the same optimality conditions and that the main di¤erence is whether (5) or (3) is imposed. We will derive conditions under which a quasi SSPE also satis…es the OSD property.

Motivating examples
We discuss two important examples in this section, where we restrict attention to symmetric strategy pro…les for explanatory reasons. The …rst example, which is deliberately oversimpli…ed, illustrates some of the technical issues that arise in applying the conditions of optimality and the recursive equations under costless delay. The second example illustrates that the symmetric SSPE under costly delay converges to a quasi SSPE as the costs of delay vanish, but that its limit fails the OSD property under costless delay and, hence, fails as an SSPE under costless delay. Moreover, this example does not have any symmetric SSPEs under costless delay. 7

Example 8 Common-interest alternating-o¤ers bargaining
Consider bilateral alternating-o¤ers bargaining with two players, so N = f1; 2g, the set of alternatives A = fâg, the set of voting orders O = f(1) ; (2)g and, for i; j = 1; 2 and j 6 = i, the set of feasible proposals X i = f(â; (j))g. For each player, the utility of acceptingâ is 1 and the utility of the status quo q is 0. To obtain the alternating-o¤ers bargaining procedure, we specify recognition probabilities i (j; x i ; 1) = 1 for x i 2 X i . This example is a special case of Muthoo (1991).
We consider symmetric stationary strategies: Recognized player i proposes (â; (j)) with probability 1. When it comes to a vote, player i approvesâ with probability 2 [0; 1]. Conditional on being recognized, v p v i (i; ; ) denotes the expected utility for the proposing player. The conditional expected utility of the responding player in the role of a voter is denoted v r v j (i; x j ; 2; ; ). We adopt similar notation for w p and w r .
Under costly delay, the one-stage-deviation property is necessary and su¢ cient for SSPE and the de…nition of SSPE coincides with the de…nition of quasi SSPE. We …rst derive all quasi SSPEs for all 2 [0; 1] by solving plus the restriction w p ; w r 2 [0; 1]. Since w p ; w r 2 [0; 1] ; solving under 2 [0; 1) gives the unique solution w p = w r = 1 and = 1, which means a unique SSPE with immediate agreement in every subgame. For = 1, we obtain w p = w r = 1 and 2 [0; 1]. Furthermore, the restrictions on w p and w r exclude the class of bizarre solutions (w p ; w r ) > (1; 1) and = 0. Combining these results for all 2 [0; 1], these conditions result in a correspondence of solutions in w p ; w r ; -space that is non-empty and compact valued and upper semi-continuous in 2 [0; 1]. Taking the limit as goes to 1 is well de…ned. The boundary solution = 0 under costless delay is counter-intuitive, because it induces zero probability of agreement in each round, i.e. perpetual disagreement, and according to (3) each player has a conditional expected utility of 0. However, in the above solution w p = w r = 1 6 = 0. Moreover, = 0 allows for a pro…table one-stage deviation and fails to satisfy the OSD property. Clearly, the equivalence between the set of quasi SSPEs and the set of outcomes derived from applying the OSD property breaks down under costless delay.
We provide the following insight why this occurs. Given an arbitrary stationary strategy pro…le , agreement onâ in round t is reached with conditional probability 2 [0; 1] per round and, under costless delay, the negotiations proceed to round t + 1 with probability (1 ) t . Hence, for costless delay, the conditional expected utilities v p and v r are given by which are discontinuous in . Consequently, recursive equations (5) and the conditional expected utilities (3) are no longer equivalent. Furthermore, conditional expected utilities (3) always satisfy the recursive equations given by The matrix is non-singular and admits the unique solution v r = v p = 1 if and only if 2 (0; 1], i.e., no perpetual disagreement. Under perpetual disagreement, we obtain the entire Null space ( ; ), 2 R, as solutions, which contains the conditional expected utilities v r = v p = 0 associated with = 0. So, the possible singularity of the recursive equations causes a breakdown in the equivalence between quasi SSPE and the OSD property. A major question is what conditions on the set of stationary strategies are necessary and su¢ cient for non-singularity of the recursive equations. In general, this is the set of stationary strategy pro…les that exclude perpetual disagreement.
In this example, even if = 1; SSPE is equivalent to the OSD property. To see this, yields v r = v p = 1 and the set of symmetric SSPE strategy pro…les given by 2 (0; 1].
Several observations follow. First of all, the set of strategy pro…les that satisfy the OSD property and the set of SSPEs are no longer closed sets and, consequently, the correspondence of SSPEs on the domain 2 [0; 1] in the v p ; v r ; -space is not compact valued and fails upper semi-continuity. As goes to 1, the unique SSPE converges to an SSPE for = 1: In general, this needs not be the case. Second, characterizing each player's stationary best responses against the other player's stationary strategy solves a stationary Markov decision problem with expected total rewards when delay is costless.
The formation of a coalition is determined by the rejector-becomes-proposer protocol introduced in Selten (1981). Some player, say i 2 N , is selected randomly at the …rst round. This player proposes to one of the other players to form a coalition, either x + = (a i;i+1 ; (i + 1)) or x = (a i 1;i ; (i 1)). 8 It holds that i prefers the coalition with i + 1 to the coalition with i 1. If the player who is proposed to, say j 2 N n fig, approves, the negotiations end with i and j forming a coalition. Otherwise, j is next in turn to make a proposal, unless breakdown occurs. Thus, j (i; x; j) = 1 and j proposes next with probability .
The stationary partition of the state space S of relevant histories can be characterized as follows: i proposes, i is proposed to by i 1, and i is proposed to by i + 1. In this example, we consider symmetric stationary strategies. Such strategies are summarized by three probabilities, , , and + , where denotes a player's probability of proposing x + , his most preferred coalition, is the probability by which a player approves his less preferred coalition and + is the probability by which a player approves his most preferred coalition. It follows that a player proposes x with probability 1 . A symmetric SSPE is therefore denoted by ( ; ; + ). Conditional on being recognized, v v i (i; ( ; ; + ); ) denotes the expected utility for the recognized player, v + v j (i; ( ; ; + ); ) denotes the expected utility of his most preferred partner j; and v v j 0 (i; ( ; ; + ); ) that of his least preferred partner j 0 : We adopt similar notation for w; w + ; and w : Clearly, it holds that 0 v + v + + v 3. In particular, perpetual disagreement implies v + v + + v = 0, whereas for < 1, v + v + + v = 3 holds if and only if agreement is immediate.
The following result is shown in the Appendix. The …rst two cases cover 2 [0; 1) and are derived by applying standard conditions of optimality and the recursive equations. For = 1, we …rst apply these conditions to characterize a unique quasi SSPE, and afterwards, for reasons similar to Example 8, we verify the OSD property. The unique quasi SSPE fails the OSD property and, hence no SSPE exists.
2. For 2 (1=2; 1), the unique symmetric SSPE is given by We note that the non-existence of SSPE under costless delay arises because for each stationary strategy pro…le some player has a pro…table one-stage deviation and the OSD property fails.
For 2 [0; 1), the recognized player always proposes his most preferred coalition, i.e. x + . A player always approves his most preferred coalition, i.e. + = 1. For 1 2 , a player also approves his least preferred alternative, i.e.
= 1, and consequently, there is immediate agreement with probability one on the recognized player's most preferred coalition.
However, for > 1 2 , a player randomizes when voting on his least preferred coalition, i.e. 0 < < 1: Nevertheless, the recognized player forgoes the immediate agreement on x for sure and strictly prefers the risky proposal x + : In such an SSPE, negotiations end with probability t 1 (1 ) t 1 > 0 in round t and before termination we observe the following cycling behavior on the equilibrium path: …rst proposer i proposes to i + 1, who in turn proposes to i 1, who in turn proposes to i; and so on. Perpetual disagreement occurs with probability zero. Since < 1 means that delay is costly, the SSPE is Pareto ine¢ cient without relying on features like asymmetric information or increasing cake sizes over time as in Merlo and Wilson (1995). In Figure 1 we plot v + v + + v , and we observe an U shape on the domain ( 1 2 ; 1), with minimum around 2:60 at 0:81. When goes to 1; v + v + + v goes to 3 with a slope converging to +1. Although this limit exists and coincides with the unique symmetric quasi SSPE, it fails as an SSPE at = 1. The intuitive reason is that the limit SSPE speci…es = 1; = 0 and + = 1; so the players end up in perpetual disagreement with probability one and each gets a conditional expected utility of 0 while recognized player i can secure a utility of 1 by proposing x to form a coalition with his least preferred partner i 1. In general, for costless delay, the set of SSPEs forms a subset of strategy pro…les that satisfy the OSD property, which in turn are a subset of the set of quasi SSPEs.
Since the unique symmetric quasi SSPE fails the OSD property, the set of strategy pro…les for which the OSD property holds is empty and, consequently, no symmetric SSPE exists. 9 So, the correspondence of symmetric SSPEs may be empty valued and, therefore, lacks upper semi-continuity at = 1.
For completeness, we mention that the limit value of the SSPE utilities is given by v = v + = v = 1; which does not capture the limit situation of perpetual disagreement with v = v + = v = 0: Similar as in the previous examples, the standard recursive equations that determine v, v + ; and v in the symmetric SSPE are singular for = 1 9 Proposition 10 extends the non-existence result for pure strategies in Bloch (1996) at = 1 to mixed strategies. It also extends the non-existence result for pure strategies in Livshits (2002) At = 1 and perpetual disagreement at = 1; = 0 and + = 1; this system is singular and any (w; w + ; w ) satisfying w = w + = w = , 2 R, is a solution.
Example 8 might be criticized as being too trivial and of no practical relevance because the limit SSPE quali…es as an SSPE under costless delay. Example 9 shows that these issues should be taken seriously and dealt with. This is the purpose of our study.

Expected utilities and recursive equations
The motivating examples show that the recursive equations might be singular under costless delay and that this is an issue. And as will become clear later, non-singularity will turn out to be crucial in our further analysis. In this section, we derive the necessary and su¢ cient conditions such that the recursive equations are non-singular.
The recursive equations are well known in dynamic programming. These state that each player's expected present value in the current state is equal to the instantaneous expected value in the current state plus the weighted sum over all future states of the state-dependent present values in these future states times their probabilities. In our negotiation model, the recursive equations on state space S are given by where the …rst term expresses the instantaneous expected utility and the second term the weighted sum over all future states. Obviously, the right-hand side is a continuous function in S , , and w i . The following result establishes that the conditional expected utilities of (3) satisfy the recursive equations.
Proposition 11 For i 2 N , v i S ; in (3) is a solution to the recursive equations (7).
This last result implies that the recursive equations are necessary. The discussion in Example 8 and 9 indicates that they are not su¢ cient. Su¢ cient conditions are derived next. Before we do so, we report a mathematical result for later reference. Recursive substitution of (7) implies Then, w i = v i S ; if and only if the second term of the right-hand side of (8) converges to 0: The next proposition follows.
Proposition 12 For i 2 N , if w i is a solution to (8) such that The recursive equations can be rewritten as I P S S ; w i = P A S ; u i and these equations admit a unique solution if and only if the matrix I P S S ; is nonsingular. Then, w i = I P S S ; 1 P A S ; u i and, by Proposition 11, w i = v i S ; .
The key insights underlying our main result is to consider the necessary and su¢ cient conditions of the opposite case, i.e., I P S S ; is singular. Singularity is equivalent to a determinant equal to zero, and this implies that P S S ; has at least one eigenvalue equal to 1. So, the matrix I P S S ; is non-singular if and only if all eigenvalues of P S S ; are unequal to 1: Solow (1952) derives a simple condition that is applicable to our negotiation model. This condition is related to irreducible matrices and their …nest decomposition. An jSj jSj matrix M 0 is irreducible if there does not exist an jSj jSj permutation matrix such that whereM 11 andM 22 are square and non-trivial, see Solow (1952). 10 If a matrix is reducible, then an appropriate permutation matrix exists from which the upper-triangular block form can be obtained. In many cases it is possible to further decomposeM 11 orM 22 . A …nest decomposition of M consists of an upper-triangular block form whose diagonal blocks are irreducible. A …nest decomposition exists, see Solow (1952 whereP de S ; 0, d; e = 1; : : : ; f and d e, denotes the (d; e)-th block or matrix in this decomposition and all diagonal blocksP ee are irreducible square matrices. Notice that a 1 1 matrix is trivially irreducible, also if its unique element is equal to zero. Denote all states associated withP ee S ; as S e S ; S: The following result characterizes the necessary and su¢ cient conditions such that the recursive equations are non-singular.
Proposition 13 The recursive system of equations has a unique solution, equal to v i S ; ; if and only if each irreducible blockP ee S ; , e = 1; : : : ; f , has at least one row sum less than 1.
Proposition 13 states the necessary and su¢ cient condition such that the system of recursive equations has a unique solution. This condition has the following interpretation.
If the initial state i 1 2 S e S ; , thenP ee S ; describes the transition probabilities on the subset S e S ; . BecauseP ee S ; is irreducible, each state in S e S ; has a positive probability of being reached within …nite time. In case a state is reached with a row sum less than 1, then there is a positive transition probability to some state outside S e S ; , being either an absorbing state in A [ fqg or a state in SnS e S ; . In both cases, the Markov process will never return to states in S e S ; : Therefore, within …nite expected time, the Markov process leaves the subset of states S e S ; . In case it transits to a state in SnS e S ; , then the …nest decomposition implies this state must be in In the proof of Proposition 13, we establish the equivalence between I P S S ; is non-singular and the largest absolute value of eigenvalues of P S S ; is smaller than 1. Then, the inverse matrix of I P S S ; is given by P 1 =1 P S S ; 1 . Moreover, these results imply that lim T !1 P S S ; T = 0 and condition (9) holds. We therefore have the following result.
Corollary 14 If I P S S ; is non-singular, then v i S ; in (3) is the unique solution to (8) and lim T !1 P S S ; T = 0.
Note that if for all i 2 N the row of P S S ; associated with s = i sums to 1, then the Markov process cannot reach any of the absorbing states, including q, and it must cycle on the states in S forever, which necessarily can only occur when = 1. The reason is that each player i randomizes over proposals that will be rejected with probability equal to 1 and no breakdown occurs. The following result reformulates this insight into the necessary and su¢ cient condition under which the row of P S S ; associated with s = i, i 2 N , sums to less than 1; which we state without further proof. Before stating this result, we denote the support of i by Supp ( i ) X i : Proposition 15 It holds that P s 0 2S P S ss 0 (( ; ); ) < 1 for s = i, i 2 N , if and only if < 1 or for at least one x = (a; o) 2 Supp ( i ) it holds that j (i; x) > 0 for all j 2 C(o), i.e., there is a positive probability that recognized player i proposes x = (a; o) and all players j 2 C(o) approve x with positive probability.
For < 1, the unique solution to the system of recursive equations is equal to v i S ; : For = 1, we must place additional restrictions on the stationary strategy pro…les S in order to apply these equations. This condition trivially holds for stationary strategy pro…les S that induce immediate agreement in every state s = i, i 2 N .
We note that many of the in ‡uential bargaining procedures in the literature can be captured by a matrix of recognition probabilities that is irreducible. For example, in case all j (i; x; r) = i;j > 0, i; j 2 N , x = (a; o) 2 X i and r 2 C (o), correspond to timeinvariant recognition probabilities, which includes the special cases of …xed rotating orders among all players and random recognized players. In general for such irreducible matrices of recognition probabilities we will have that if the strategy pro…le S induces agreement with positive probability in at least one state s = i, i 2 N , then there is a sequence of states connecting every other state in N to state i that each have positive probability of being realized. If that is the case, the set of absorbing states will be reached within …nite expected time independent of the initial state. In our general setting, the recognition probabilities j (i; x; r) may also depend upon the proposal x 2 X i and the identity of the rejector r 2 N n fig. Therefore, there may be multiple matrices of recognition probabilities that depend upon proposals and rejectors. Formally, for i 2 N; we choose x i = (a i ; o i ) 2 X i and r i 2 C(o i ) and de…ne the matrix of recognition probabilities R x 1 ; : : : ; x n ; r 1 ; : : : ; r n = 2 6 4 1 (1; x 1 ; r 1 ) 1 (n; x n ; r n ) . . . . . . n (1; x 1 ; r 1 ) n (n; x n ; r n ) 3 7 5 : We have the following result.
Corollary 16 Let all matrices of recognition probabilities R be irreducible and = 1. Consider some strategy pro…le S : The system of recursive equations has a unique solution, given by v i S ; 1 ; if and only if there is a state s = i, i 2 N , such that some proposal x 2 X i is proposed and accepted with positive probability.
The matrix on the right-hand side is irreducible. If we denote this matrix as M , then lim !1 M does not exist. The matrix I M has a one-dimensional Null space spanned by (1; 1; 1). This explains our …nding that w = w + = w holds in this example. Moreover, condition (9) holds if and only if w i = 0 and this condition rules out any (w; w + ; w ) = ( ; ; ) with 6 = 0:

Best responses
In Example 8 we argue that the derivation of best responses against stationary strategies by the other players is more involved under costless delay than under costly delay. Given the other players' stationary strategies, characterizing a player's set of best responses is equivalent to solving a stationary Markov decision problem (MDP) and, under costless delay, the expected total-reward criterion is appropriate. In this section, we characterize each player's set of best responses under costly delay and costless delay by applying the theory of MDPs to our negotiation model.
The following result states the necessary and su¢ cient conditions for a best response in stationary strategies against S; i . We also include the existence of such a best response.
Before we do so, we de…ne the set of all expected values that correspond to solutions of the optimality conditions and the recursive equations as (4) and (5) Or the minimality condition holds: w i is a minimal element of W i S; i ; : By de…nition of W i ( S; i ; ); the values w i are non-negative. Therefore, (11) guarantees that (9) holds. The third condition then also implies that a stationary best response for player i is equivalent to the conditions imposed by the OSD property for this player.
This result is important because it extends the standard approach for costly delay to costless delay. Before discussing it, we derive another result that, similar to Corollary 14, states that if the recursive equations are non-singular then the third condition also holds.
Corollary 18 Let S; i and 2 [0; 1] be given. If S;i and w i form a solution to conditions 1. and 2. of Proposition 17 and I P S S;i ; S; i ; is non-singular, then S;i is a best response to S; i : For 2 [0; 1), the recursive equations are non-singular on the domain of stationary strategies S;i , and by the last result, player i's best responses are fully characterized by the conditions of optimality and the recursive equations, which con…rms standard theory. For = 1, one of two additional conditions is required, but as a consequence of Corollary 18, only in case the optimality conditions return stationary strategies that induce singular recursive equations. Without repeating the arguments of Section 4, non-singularity requires that all diagonal blocks of the …nest decomposition of P S S;i ; S; i ; have at least one row sum smaller than 1. Then, P A S;i ; S; i ; 6 = 0 and the system cannot cycle forever on the transient states in S. The need for an additional condition follows from Examples 8 and 9. Puterman (1994) notes that equalizing stationary strategies drive the system to states in which there is no opportunity for positive future rewards, which is exactly the case in our absorbing states in A [ fqg or when the process cycles forever on the transient states S. In particular, for 2 [0; 1) cycling forever is impossible because of the positive probability of a permanent breakdown. Puterman (1994) also notes that taking the limit as goes to 1 is well de…ned. In fact, this is how existence of an optimal value under = 1 in MDPs is shown. What is di¤erent in a game theoretic context, as Example 9 illustrates, is that the entire SSPE strategy pro…le S depends upon and, hence player i's MDP changes as goes to 1.
Proposition 17 does not rely on our assumption of non-negative utilities. By the theory of MDPs in e.g. Puterman (1994), it extends to the class of positive bounded models. Then, the stationary strategies of the other players, i.e., S; i , have to be such that player i can reach at least one agreement associated with non-negative utilities, which is a condition that we have avoided for explanatory reasons. In case S; i restricts player i to reach only agreements with negative utilities, player i's MDP has become a negative bounded model for which similar results hold that need to be speci…ed somewhat di¤erently. For explanatory reasons, we have assumed non-negative utilities.
The linearity of the objective function in the optimality condition for i con…rms the standard wisdom that whenever the recognized player opts for a randomized i over X i , then all x; x 0 2 Supp ( i ) should have equal expected conditional utilities that are also maximal among all feasible proposals. Formally, for x; x 0 2 Supp ( i ) and x 00 2 X i ; Similar, the optimality condition for i states that every voter has to consider the trade-o¤ between his conditional expected utility w i (j; x; k i (o) + 1) from approving and the conditional expected utility P i 0 2S i 0 (j; x; i) w i (i 0 ) from forcing disagreement in the current round. If w i (j; x; k i (o) + 1) is strictly largest, then it is optimal for player i to approve with probability i = 1. If w i (j; x; k i (o) + 1) is strictly smallest, then player i will disapprove with probability one, i.e., i = 0. In case w i (j; x; k i (o) + 1) = P i 0 2N i 0 (j; x; i) w i (i 0 ), any randomization over approving and disapproving is optimal. In particular, whenever player i is the last voter for a proposal x; he approves for sure if . Whatever voting stage k i (o) player i is in, the expected continuation utility P i 0 2N i 0 (j; x; i) w i (i 0 ) of disapproving acts as a threshold.
6 Stationary subgame perfect equilibrium The previous two sections revealed several issues that can be traced back to the possible singularity of the recursive systems and the players'MDPs characterizing best responses against stationary strategies. We have investigated both these issues in isolation. In this section, we …rst address the implications for the game theoretic analysis in the negotiation model, then we derive several useful results for applications and …nally we investigate vanishing cost of delay and quasi SSPE.
The de…nition of SSPE states that each player's stationary strategy is a best response given the stationary strategies of the other players. This means that we have an MDP for each player and all MDPs together form the equilibrium conditions. The following proposition follows trivially from the stationary MDPs that characterize the players'stationary best responses.
Proposition 19 For 2 [0; 1], the strategy pro…le S is an SSPE if and only if, for each player i 2 N , there exists values w = (w i ) i2N such that 1. The optimality conditions (4) hold.

Either equalizing condition (11) holds,
Or the minimality condition holds: w i is a minimal element of W i S; i ; : This result extends the standard approach to characterize SSPE under costly delay to costless delay. Similar as in Section 5, we …rst state an additional result in case the recursive equations are non-singular.
Corollary 20 If S is a quasi SSPE and I P S S ; is non-singular, then S is SSPE.
For 2 [0; 1), the recursive equations are non-singular on the entire domain of stationary strategies S : We obtain that SSPE is equivalent to the conditions of optimality and the recursive equations. For = 1, the same two conditions remain equivalent to SSPE on the subdomain of stationary strategies S for which the recursive equations are non-singular, i.e., the subdomain of strategy pro…les that exclude perpetual disagreement.
Only in case the …rst two conditions return stationary strategy pro…les that induce perpetual disagreement, one of two additional conditions is required. The need of additional conditions follows from our motivating examples and we forego repeating the arguments in Section 5.
In many negotiation models in the literature there exists an alternative that is strictly preferred to the status quo outcome by all players, i.e., the bargaining problem is essential, and every recognized player can propose this alternative. Then, perpetual disagreement in SSPE is impossible. Assuming an essential bargaining problem is too restrictive and does not capture the utilities of Example 9. We formulate a weaker condition that is easy to check.
Assumption 21 For each i 2 N; there exists a proposal (a; o) 2 X i such that u i (a) > 0 and u j (a) > 0 for all j 2 C (o) : This assumption includes all essential bargaining problems in case each player is allowed to propose an alternative with positive utility for all players together with a decisive coalition to approve this alternative, say N: Formally, for all i 2 N , the essential bargaining problem speci…es some alternative a 2 A for which u j (a) > 0 for all j 2 N , and for some order o with C (o) = N n fig we have that (a; o) 2 X i . In Example 9, every recognized player has two feasible proposals with positive utilities for some pair of players (including the recognized player) that forms a decision coalition and zero for the third player. Therefore, Example 9 also satis…es Assumption 21. We obtain the following result.
Proposition 22 Let Assumption 21 hold. Any SSPE strategy pro…le S 2 S induces a probability of perpetual disagreement equal to 0. Moreover, strategy pro…le S 2 S is an SSPE if and only if S 2 S is a quasi SSPE that induces a probability of perpetual disagreement equal to 0.
This result states that Assumption 21 rules out that SSPEs induce perpetual disagreement. It implies that we may consider the subdomain of strategy pro…les that exclude perpetual disagreement. Under costless delay, this subdomain is no longer closed and this is technically speaking unfortunate in deriving results about existence of equilibria. As Example 9 shows, the set of SSPE may be empty. In this example, the optimality conditions and recursive equations identify a unique strategy pro…le that induces perpetual disagreement as a candidate for SSPE, but given that this example also satis…es Assumption 21, this candidate fails as an SSPE by our last result.
Our results extend the equivalence result between SSPE and the one-stage-deviation property in Alós-Ferrer and Ritzberger (2012), who assume no moves by nature, to our negotiation model. For 2 [0; 1) and for = 1 and the subdomain of S that consists of stationary strategy pro…les without perpetual disagreement, the expected utility function U i S ; is lower semi-continuous in S . In Alós-Ferrer and Ritzberger (2012) this is exactly the condition under which the equivalence between SSPE and the one-stage-deviation property is established.
Proposition 22 is of practical relevance. Under Assumption 21, the standard approach of computing SSPEs from the conditions of optimality and the recursive equations, i.e., computing quasi SSPE, is always available. Either these return strategy pro…les that exclude perpetual disagreement and that automatically qualify as SSPE. Or, the conditions return strategy pro…les that induce perpetual disagreement and that obviously fail as SSPE.
An application of these insights is already conducted in Herings and Houba (2010). They study negotiations among three players who are randomly recognized and the entire class of non-negative utilities that give rise to a Condorcet paradox. Their results are derived from the OSD property of De…nition 7. They report two robust classes of probabilities and utilities, one for which a generically unique SSPE exists and one where non-existence prevails. In the former class, SSPE strategy pro…les exclude perpetual disagreement and, hence, the optimality conditions and the recursive equations su¢ ce for SSPE according to Proposition 22. In the latter class, the OSD property returns strategy pro…les that induce perpetual disagreement and that fail as SSPE.
It is common practice in bargaining theory to attach special meaning to the limit of vanishing costly delay, see e.g. Binmore et al. (1986), or establish existence of limit pure SSPEs, as in e.g. Muthoo (1991). We refer to the limit SSPE strategy pro…le after taking the limit goes to 1 as a limit SSPE. The following results are immediate.
Corollary 23 If there exists a limit SSPE strategy pro…le that induces a probability of perpetual disagreement equal to 0, then it is an SSPE under costless delay.
Corollary 24 If there do not exist SSPEs under costless delay, then all limit SSPEs strategy pro…les induce a probability of perpetual disagreement equal to 0.
The …rst result provides a condition that is relatively easy to verify, but it requires computation of the set of SSPEs under costly delay …rst, which is impractical. As Example 9 shows, imposing Assumption 21 that excludes perpetual disagreement in any SSPE is not strong enough in obtaining a limit SSPE without perpetual disagreement. Also, this example has non-negative utilities, the key assumption in Flesch et al. (2010) for obtaining existence of subgame-perfect "-equilibrium for every " > 0 for = 1. Recall that Herings and Houba (2010) report two robust classes of probabilities and utilities, one for which a generically unique SSPE under costless delay exists and one where non-existence prevails.
The second corollary implies that, for the latter class, introducing costly delay and studying limit SSPE is futile if one hopes to establish existence of SSPE under costless delay.
There are several general existence results under costly delay that establish non-emptiness and upper-semi continuity of the set of SSPE strategies, see e.g. Duggan (2000, 2006) and Duggan (2011). In these references, the set of alternatives is continuous and so are the utility functions, but the negotiation procedure is less general than in our model. Our results are complementary. As in Duggan (2011), we do not impose any restriction on the shape of the utility functions to obtain existence of SSPEs under costly delay. However, Example 9 also indicates that upper semi-continuity of quasi SSPE is not su¢ cient for limit SSPEs to be SSPE under costless delay. This insight is novel.

Discussion
In this section, we will demonstrate how to generalize our results. It turns out that our assumptions and analysis capture the essence of very general models.

Related models
In this subsection, we illustrate how institutional aspects of several in ‡uential bargaining models can be seen as special cases of the model speci…ed in Section 2. We …rst discuss the set of feasible proposals. In bilateral unanimity bargaining over a discrete set of alternatives A, 11 X i = f(a; ( i)) ja 2 Ag expresses that approval by the responding player i 6 = i is required. Unanimity bargaining can be modeled as X i A fo 2 OjC (o) = N n figg : The case of an exogenous voting order, say in ascending order, is captured by: if (a; o) 2 X i , then o = (1; : : : ; i 1; i + 1; : : : ; n). Majority approval implies where voting orders are feasible if its voters plus player i form a majority. In case player j is a veto-player, then x = (a; o) 2 X i with i 6 = j implies j 2 C(o): Or, in case player j is a dictator, then x = (a; o) 2 X i with C (o) = fjg for all i 6 = j re ‡ecting that other players may propose when being recognized but that only the dictator is decisive, and x = (a; o) 2 X j implies C(o) = ;; re ‡ecting that j does not require approval from the other players. These special cases can be easily extended to general collections of decisive coalitions as in e.g. Baron and Ferejohn (1989) and Duggan (2000, 2006).
In many coalitional negotiation models, the recognized player proposes a coalition and an alternative from a set of feasible alternatives that may be coalition dependent. In our 11 For example, in case of a smallest money unit in van Damme et al. (1990) and Muthoo (1991). framework, voting orders induce coalitions, and coalitions induce feasible sets of alter-

natives. Let A (C)
A denote the set of feasible alternatives coalition C is allowed to propose. Then, X i f(a; o) 2 A Oja 2 A (C (o) [ fig)g captures such coalitional negotiation models. Although we maintain the assumption that recognized players automatically vote in favor, voting by recognized players requires only a minor modi…cation of an additional voting stage in the multi-stage game, which is conceptually similar. For instance, revoking a proposed alternative as in the bilateral bargaining game of Muthoo (1990) can be modeled as if the recognized player casts the last vote and X i = A f( i; i)g : We now turn to popular recognition rules in the literature. These are captured as follows: The bilateral alternating-o¤ers procedure imposes 2 (1; x; r) = 1 (2; x; r) = 1.
Fixed rotating orders of recognized players can be modeled similarly, for example the in…nitely-repeated order 1; : : : ; n is captured by setting i+1 (i; x; r) = 1; where we write i + 1 instead of i + 1 mod n. Markov recognition probabilities are studied in Kalandrakis (2004b), Britz et al. (2010) and Predtetchinski (2010, 2012). In this case, the probability of player j being recognized in round t + 1 conditional on player i being the proposer in round t is given by i;j : We obtain Markov recognition probabilities by setting j (i; x; r) = i;j . Time-invariant recognition probabilities as in Binmore (1987) are obtained as a special case in which all i;j are independent of the recognized player i. Nohn  Duggan (2011) assumes such probabilities to depend upon the proposed alternative a 2 A; i.e., j (i; a; o; r) = i;j (a). Our framework integrates both formulations by allowing that such Markov recognition probabilities depend upon the alternative, the proposed coalition and even the voting order through the proposal x 2 X i , i.e., j (i; x; r) = i;j (x) : Finally, we consider coalitional negotiation models. The rejectorbecomes-proposer protocol in e.g. Selten (1981), Chatterjee et al. (1993) andKawamori (2013) is that player r becomes next round's recognized player, i.e., r (i; x; r) = 1:

Extensions
Our negotiation model can also be extended by enlarging the state space S. Formally, let Z, jZj jSj, be a …nite set of state variables and denote z 2 Z. In each round t, the state variable z 2 Z is publicly observed and speci…es player i 2 N who is recognized given this state, denoted by the function : Z ! N . The set of feasible proposals becomes X i (z) for i = (z), the utility of proposal x 2 X i (z) becomes u j (x; z) for j 2 N and the transition probabilities become i (z; x; r). Such an expanded state space would lead to a stationary Markov process. Stationary strategy pro…les can be de…ned in the obvious way on this state space and induce modi…ed stationary Markov processes similar to those in Section 2.1. Once more, the crucial part of our analysis remains valid. For instance, the conditions that ensure that the largest absolute value of eigenvalues of the matrix with transition probabilities is less than 1 still requires that we do not have perpetual disagreement.
Maintaining A is …nite, the enlarged state space o¤ers an amazing number of important extensions.
1. Stochastic utilities, as in e.g. Merlo and Wilson (1995) and Duggan (2011). Then, our analysis extends upon assumptions as costly delay, exogenously given public and sequential voting, and recognition probabilities.
2. General coalitional bargaining procedures, as in Chatterjee et al. (1993), Bloch (1996) and Bloch and Diamantoudi (2011) in which coalitions leave after they form. The state indicates which coalitions have formed (and their members left the game) and what they have agreed upon. The players who did not join any coalition continue the negotiations among them. These procedures are especially important in market situations such as two-sided matching models and roommate problems.
3. Multilateral contracting processes that allow coalitions of agents to renegotiate or rewrite earlier contracts and to merge with other coalitions, as e.g. Gomes (2005).
The state indicates which coalitions have formed and what they agreed upon.
4. SPEs on the class of non-stationary strategy pro…les that can be represented as …nite automata. As argued in e.g. Section 3.5 of Osborne and Rubinstein (1990), …nite automata cover most of the relevant strategy space in many negotiation models. The state space Z then represents the virtual state space of …nite automata. Shaked and Sutton (1984). This method is widely applied to establish lower and upper bounds on each player's set of SPE utilities. If these two bounds coincide for all players, we would have uniqueness of SPE utilities, otherwise multiplicity. It corresponds to a class of automata with state space Z = S N .

The method proposed in
Then, the state z = (s; i) 2 Z indicates which player i 2 N is kept to his lowest SPE payo¤. Under costly delay, we obtain as an immediate result that these bounds always exist. As argued in Houba and Wen (2014), this method might also be applied to establish existence of SPE under costless delay in the model of Herings and Houba (2010), especially for parameter values for which no SSPE exists.
Our main results imply that all these extensions can be accommodated for as long as the extended stationary strategy pro…le excludes perpetual disagreement under costless delay.
We do note, however, that the models under 2. and 3. involve at most a …nite number of state transitions involving non-zero utilities associated with the formation of smaller coalitions during the negotiation process. This causes no technical problems, because the expected total rewards remain bounded.

Concluding remarks
The main results show that the equilibrium analysis of strategic negotiation models un- Alós-Ferrer and Ritzberger (2012) show that lower semi-continuity of all utility functions in strategy pro…les is necessary in order to have equivalence between the one-stagedeviation property and SSPE. This identi…es lower semi-continuity as a necessity for well-de…ned problems. Such an approach might suggest to discard problems that lack this property. However, restricting attention to classes of economic models that are well-de…ned in this way, discards important applications such as e.g. the negotiation model under costless delay. Our study o¤ers an alternative approach in which the lack of lower semi-continuity is acknowledged as a fact. Before conducting any equilibrium analysis, this lack is dealt with by deriving necessary and su¢ cient conditions that restore the equivalence between the set of strategy pro…les that satisfy the one-stage-deviation property and the set of subgame perfect equilibria on a particular class of strategy pro…les, stationary and …nite automata in our case. This will not automatically resolve the non-existence of subgame perfect equilibrium, but it provides a systematic and sound approach that includes taking care of neglected aspects. Also, our study emphasizes the importance of properly de…ning the one-stage-deviation property. Duggan (2000, 2006) and Duggan (2011) provide general existence results for a class of negotiation models with sets of feasible alternatives that consist of continuous variables and discounting. They also establish non-emptiness and upper-semi continuity of the set of SSPE strategies in the discount factor on the closed interval that includes costless delay. So, the set of limit SSPEs as the discount factor goes to one is well de…ned. Such properties also hold in our Example 9, the unique SSPE converges to some well-de…ned limit, yet this limit fails to be SSPE under costless delay. The issues discussed in our study are de…nitely not an artefact of assuming a …nite set of alternatives. These issues also show up in the multilateral contracting model of Gomes (2005) that has sets of feasible alternatives that consist of continuous variables. In Section 4.2 of this reference, an example is presented with a unique SSPE under costly delay. In the limit SSPE, the recognized player passes the initiative with probability one and perpetual disagreement results. The limit of the conditional expected SSPE utilities, however, exceeds that of the conditional expected utility of perpetual disagreement, which is similar as in our coalition formation example. Our results pin down what is going on: the limit SSPE should be seen as a quasi SSPE that fails to be SSPE due to the lack of su¢ ciency of the recursive equations condition. Our study therefore provides important foundations for further research on sets of feasible alternatives that consist of continuous variables.
Proof of Theorem 5. Throughout this proof 2 [0; 1] is …xed and we suppress in our notation. The proof consists of applying Kakutani's …xed-point theorem to a mapping that maps any S 2 S and w = (w 1 ; : : : ; w n ) 2 U jSj into S U jSj : For i 2 N , de…ne the function f i : S U jSj ! [0; u i ] jSj as f i ( S ; w) = P A S u i + P S S w i . Since f i ( S ; w) = P A ( S )u i + P S ( S )w i + P q ( S )0 and, for every s 2 S; the sum of all the components of row s of P A ( S ); P S ( S ); and P q ( S ) together is 1, it follows that (f 1 ( S ; w); : : : ; f n ( S ; w)) 2 U jSj : By Lemma 1, the function f i is continuous.
For given S 2 S , in any state s 2 S i active player i's conditional best-response corre- Finally, by construction of the correspondence , each …xed point S ; w satis…es the optimality conditions and the recursive equations and w 2 U jSj : So, each …xed point is a quasi SSPE.
Claim: 2 (0; 1]: Suppose to the contrary that = 0: Now (14) implies that w = 2; so by (15) we obtain 2 = 2 + (1 )w 1; a contradiction. We divide the remaining cases in six classes, and show that each one leads to a contradiction, with the exception of class four.
6. Suppose = 1 and = 1: By (20) it follows that w 1; but by (12) it holds that w = 2 = 2; a contradiction. Hence, we conclude that perpetual disagreement given by = 1, + = 1 and = 0 together with w = w + = w = 1 is the unique quasi SSPE. Note that the restriction (w; w + ; w ) 2 U in De…nition 4 excludes, in case 4, = 1, + = 1 and = 0 with w = w + = w > 1, which is discussed in the main text. The derivation of strategy pro…les for which the OSD property holds is similar, except in case 4. There, an agreement is never reached, so v = v + = v = 0: Since = 1; we …nd by (17) that 0 = 2 + (1 )v 1; a contradiction. So, there are no strategy pro…les for which the OSD property holds. Hence, there are no symmetric SSPEs at = 1.
Proof of Proposition 10 part 1 and part 2 For 2 [0; 1), it holds that ( ; ; + ) forms an SSPE if and only if there is (w; w + ; w ) such that the conditions for quasi SSPE in Proposition 10 part 3 are satis…ed when all w, w + and w are replaced by w, w + and w , except of course those on the left-hand side of the recursive equations (12)-(14). Then, part 1 and 2 of Proposition 10 can be veri…ed.
Proof of Proposition 11. By (3), we have that By rewriting this equation, we obtain the stated result.
Proof of Proposition 13. By de…nition, P S S ; 0 is a non-negative matrix with row sums of at most 1. So, many standard results as in e.g. Solow (1952) (or the references therein) directly apply.
Let (M ) denote the largest absolute value of eigenvalues of a square matrix M 0 whose row sums are at most 1. Then, we have 1. P S S ; 2 [0; 1] is real, 2. the sum P 1 t=0 P S S ; t exists if and only if P S S ; < 1, 3. if P S S ; < 1, then the inverse I P S S ; 1 = P 1 t=0 P S S ; t exists and is non-negative. For our negotiation model, 3 implies v i S ; = 1 X =1 P S S ; 1 P A S ; u i = I P S S ; 1 P A S ; u i ; and then the following result is immediate: A. If P S S ; < 1, then the unique solution of the recursive equations coincides with v i S ; .
The results in Solow (1952) relate conditions on P S S ; to row sums associated with the irreducible diagonal matricesP ee S ; of the …nest decomposition of P S S ; .
Then, the following results, which are translated from Solow (1952), hold: 4. If all rows ofP ee S ; sum to 1, then P ee S ; = 1.
5. If at least one row sum ofP ee S ; is strictly less than 1, then P ee S ; < 1. For our purposes, we slightly extend these results. The converse of 5 states: 5'. If P ee S ; = 1, then all row sums ofP ee S ; are at least 1.
Because all row sums ofP ee S ; are bounded by 1, combining 4 and 5'imply the following two results: B. P ee S ; = 1 if and only if all row sums ofP ee S ; equal 1.
C. P ee S ; < 1 if and only if at least one row sum ofP ee S ; is less than 1.
With respect to the …nest decomposition of P S S ; : Since the set of eigenvalues of P S S ; is equal to the union over all e = 1; : : : ; f of the set of eigenvalues ofP ee S ; , we have that P S ( S ; ) = max e=1;:::;f f (P ee S ; )g: Hence, P S S ; < 1 if and only if (P ee S ; ) < 1 for all e = 1; : : : ; f; if and only if at least one row sum ofP ee S ; is strictly less than 1 for all e = 1; : : : ; f .

Proof of Proposition 17.
Given that < 1 is the standard case, we only provide a proof for = 1. Given stationary strategies S; i , all of agent i's best responses have to be optimal in a MDP with the expected total-reward criterion, where such MDPs are de…ned in e.g. Chapter 7 of Puterman (1994). In this reference, the class of positive bounded models is de…ned as those MDPs that have at least non-negative reward per round and a sum of expected rewards that is bounded. In our negotiation model, the total rewards satisfy U i S ; 2 [0; u i ] because player i only receives a non-negative utility at most once when the state moves from the transient states S to an the absorbing states A [ fqg and otherwise receives utility of 0 per round.
By Proposition 7.1.1 in Puterman (1994), for each s 2 S i , the vector of optimal values w i 2 R jSj can be supported by a stationary strategy and we may restrict attention to such strategies. By Theorem 7.1.3 in Puterman (1994), the value w i 2 R jSj has to satisfy w i (s) = sup i 2 (X i ) P which can be separated into the optimality conditions (4) and the recursive equations (5).
Both existence and the necessary and su¢ cient conditions then follow from combining Proposition 7.2.1, Theorem 7.2.3 and Theorem 7.2.4 in this reference. These results establish that the optimal w i 0 additionally has to satisfy either the minimal vector of values in W i S; i ; , or lim sup T !1 P S S;i ; S; i ; T w i = 0.

Proof of Proposition 22.
Suppose not: S is SSPE and induces perpetual disagreement. Then, v j i; S ; = 0 for all i; j 2 N . Consider the voting stages associated with the proposal (a i ; o i ) 2 X i such that u i (a i ) > 0 and u j (a i ) > 0 for all j 2 C (o i ) : The last voter of o i has a threshold of 0: The optimality condition implies he approves a i with probability 1. The second-last voter of o i also has threshold 0 and knows that his approval of a i followed by the last voter's approval will implement alternative a i , from which he derives positive utility. The optimality condition implies the second-last voter in o i approves a i with probability 1.
By backward induction, in this SSPE with perpetual disagreement all voters in o i have a threshold of 0 and will approve alternative a i with probability 1. So, player i as the recognized player has a pro…table one-stage deviation, namely propose (a i ; o i ) from which he obtains u i (a i ) > 0 with probability 1. This contradicts that S is SSPE and induces perpetual disagreement.