Games and cost of change

Most microeconomic and game theoretic models of individual choice overlook adjustment costs. Rather often, the modeler’s concern is just with improvement of objectives. This optic doesn’t quite fit agents somewhat tied to status quo. If rational, any such agent reasons whether moving to another state be worth his while. For that, the realized gains must outweigh the inconveniences of the move. Choosing games as chief setting, this paper offers some observations as to the fact that strategy change usually entails cost.


Introduction
Game theory and microeconomics-henceforth just called theory-abounds in agent-based models of decision problems. However, even among those models concerned with just a single agent, many display three objectionable features. First, the concerned agent should, with little or no hesitation, leap directly to a best choice. Second, his behavior ought be totally goal-oriented-and never affected by any cost of change. Third, he is often depicted as fully detached from history, precedence or status quo.
Such modelling of individual behavior begs immediate objections. Indeed, an agent's choice may emerge step-wise; his cost of change can be considerable; and each arrival stems from some point of departure.
It's comforting therefore, that algorithms geared toward best or better choice, have-at least since Cauchy (1847)-been coached as iterative processes. These require more than just one step. It also comforts that recent decades have brought forward procedures that expressly account for adjustment costs. 1 If several agents take part, an important fourth feature adds extra complexity, namely: How can participants foresee or observe actions taken by others-and respond to these?
Regrettably, much theory sidesteps all four features. In fact, large parts move straight to steady states or terminal outcomes, if any, called equilibria (Osborne and Rubinstein 1994;Vega-Redondo 2003). Thereby, many studies shun queries as to equilibrium emergence and strategic learning (Fudenberg and Levine 1994;Peyton Young 2004). Serious issues about perfection, selection and stability of equilibrium then escape attention ( van Damme 1991;Harsanyi and Selten 1988;Samuelson 1997;Selten 1975).
In social sciences, various concepts of steady-state solutions exert considerable attractionfor good reasons. Each instance describes how parties behave, communicate or fare in equilibrium. However, off such special states, the underlying concept can, by itself, hardly explain attainment of a solution. 2 To emphasize or justify some "focal" state as particularly plausible, at least one stable process ought eventually approximate or reach that distinguished outcome (Fisher 1983).
Two hypotheses have often been invoked to this end. Either is tempting but neither quite attractive. One posits that agents, even out of equilibrium, behave as in it. The other presumes that each party acts throughout as though fully foresighted, marvelously competent, and perfectly rational.
More realistic approaches ought tolerate imperfections in agents' capacities to choose, foresee or know. 3 Accordingly, here below, local perceptions replace global views, and improvements substitute for full optimization. While seeking own betterment, agents adapt, but usually in somewhat moderate or myopic manner (Miller and Page 2007) . If so, might they eventually come to a halt? And then, where?
These questions motivate the paper. For preparation, Sect. 2 considers just one agent, isolated from others. By contrast, Sect. 3 lets him play normal-form games among noncooperative strategists. Then, convexity, in one form or another, plays important roles. Section 4 dispenses with convexity, letting play unfold in metric spaces. Provided cost of change exceeds the distance, the metric setting may frame proximal procedures and Nash equilibria in new manners; see Theorem 4.1 and its corollary. Section 5 concludes by briefly considering extensive-form games of Stackelberg variety.
The two main novelties come late in the paper: Theorem 4.2 leans on Caristi's metric fixed point theorem to consider existence of strong Nash equilibria. Theorem 5.1 contends with topological assumptions for convergence to Stackelberg solutions of the principal-agent sort. The paper addresses various readers. Included are operation researchers and computer scientists concerned with multi-agent interaction, optimizers using proximal procedures, and theorists who consider fixed points or games in metric spaces.

Preliminaries concerning the single agent
This section introduces notations and preliminaries. To begin with, and to simplify, it considers just one agent. 4 Actually, he holds a "position" x. If deviating from x tox, that transition gives him net benefit b(x |x ). His improvement or betterment ∈ X × X for some non-empty viability set X in the ambient space X of alternatives. The "probabilistic" notation b(x |x ) emphasizes that the agent, while conditioned by his point of departure x, seeks a suitable point of arrivalx. In particular, given x ∈ X , he might maximize b(x |x ) subject tox ∈ X . (1) Many formalized decision problems mention no point of departure -or implicitly, regard the latter to be of no or negligible importance. Moreover, upon leaping directly to a very best choice, the agent seemingly incurs no cost for "dislodging" himself. 5 To wit, classical and customary instances let for some gross benefit function β : X → R∪ {−∞}, having effective domain X := β −1 (R). Since x is sunk already, this model incorporates no adjustment costs. The agent in question appears fully goal-driven-and never troubled by friction or inertia. More realistically, proximal point methods (Rockafellar 1976;Teboulle 1997 for some (adjustment) cost function c : X × X → R + ∪ {+∞} which vanishes on the diagonal: c(x |x ) = 0 ∀x ∈ X . 6 No symmetry is presumed; it may well happen that c(x |x ) = c(x x ); the forward fare can differ from the backward one. It often appears natural though, that c satisfies the triangle inequality: Then, ( 2) makes a direct move x →x preferable to any indirect one x →x →x. 7 (2) supports the standing interpretation that b(x |x ) denotes additional benefit in arrival statex, net of cost c(x |x ) incurred upon departing directly from x.
Ifx = x, the agent stays put; otherwise, he moves . A move from x tox is declared (strictly) improving iff b(x |x ) > 0. Naturally, suppose that staying put entails no improvement; that is, b(x |x ) ≤ 0 for all x ∈ X . In many cases, b(x |x ) = 0.
Stationary states stand out by allowing no improvements. They solve problem (1) by bringing up contingent fixed points: Definition 2.1 (stationary states). x ∈ X is declared stationary for the bivariate mapping Each stationary point x makes b(· |x ) ≤ 0 on X , and in most cases b(x |x ) = 0. This framing of the agent's decision problem begs the question: Is there some stationary state? The following positive (albeit particular) answer is just a restatement of Ky Fan's inequality (Aubin and Ekeland 1984;Fan 1972): Theorem 2.1 (on existence of stationary states). Suppose X is a non-empty compact convex subset of a topological vector space X. Also suppose b(x |x ) be quasi-concave inx ∈ X , lower semicontinuous in x ∈ X , and b(x |x ) ≤ 0 ∀x. Then there exists at least one stationary state.
Theorem 2.1 points to topological vector spaces as tractable settings. It also emphasizes the important role of closed convex preferences. 8 Granted existence of at least one stationary state, how might the agent eventually reach one of those-and come to rest there? As in Polak (1997), Zangwill (1969) it's convenient to model his step-wise adjustments in terms of a point-to-set correspondence A : X ⇒ X . It appears natural to have From some accidental or historical point x 0 ∈ X , there emanates an iterative process Proposition 2.1 (on appropriate cluster points). Let X be a closed subset of a topological space X. Suppose (4) generates a summable sequence k → b(x k+1 x k ) ≥ 0. Further suppose that each non-stationary point x ∈ X has some neighborhood N and number Then, either the sequence (x k ) is finite with a stationary last point-or, every cluster point of the infinite sequence must be stationary.

Proof.
In the viable set X , let x = lim k∈K x k for some infinite subsequence K of natural numbers. Suppose x isn't stationary. With no loss of generality, take x k ∈ N for all k ∈ K . Then, it obtains the contradiction Granted (3), if process (4) reaches a stationary point x (Definition 2.1), and then For these reasons, instance (2) and (3), always with c ≥ 0, will be central and henceforth called the workhorse model.
The results of this section apply directly to interaction among several agents i ∈ I , each just choosing his component x i of an overall profile x = (x i ). Such settings are commonly framed as games-considered next.

Non-cooperative games
Henceforth I always denotes a finite ensemble of "players", at least two of them. Then, by a strategy profile x = (x i ) is meant a mapping i ∈ I → x i ∈ X i where X i is a non-empty "viability set" in some ambient space X i of alternatives. Let X := i∈I X i . Further, posit X := i∈I X i , except under other notice.
Given a strategy-profile x = (x i ) ∈ X , suppose member i ∈ I anticipates net benefit b i (x i |x ) ∈ R upon deviating unilaterally, within his viability set X i , from actual strategy x i to another onex i . Then, by assumption, b i (x i |x ) already incorporates eventual cost of change. In terms of the residual profile x −i := (x j ) j =i , actually implemented by his rivals, player i acts as though the updated version equals (x i , x −i ). That belief is justified iff he alone deviates.
Definition 3.1 (Non-cooperative stationary states). A strategy profile x = (x i ) ∈ X is declared multi-agent stationary (or simply stationary) iff In much studied, but special instances, multi-agent stationarity adds nothing to the concept of Nash equilibrium (Osborne and Rubinstein 1994): Proposition 3.1 (on multi-agent stationary states as Nash equilibria). Suppose player i ∈ I worships gross benefit β i : Then, a state x ∈ X is multi-agent stationary ( 8) iff it's a Nash equilibrium of the noncooperative game G := (β i , X i ) i∈I , meaning It's objectionable that Proposition 3.1 mentions no adjustment costs. Each player agent appears fully goal-driven. Nobody is ever troubled by friction or inertia. More realistically, instead of (9), following the lead of proximal point methods and workhorse model (2) for some cost function c i : That function could be asymmetric in the agent's own arguments (x i , x i ). The dependence of c i (x i |x ) on the entire profile x fits games featuring congestion (Rosenthal 1973) or use of common resources (Flåm 2017). That feature of (10) differs from the two-player games in Attouch et al. (2007) where If a Nash solution isn't unique, (10) also bears on equilibrium refinement, selection and stability (van Damme (1991); Harsanyi and Selten (1988)) ; see the remark below on strong equilibria, motivating Theorem 4.2. Whilex i → β i (x i , x −i ) is the customary Nash maximand, (10) subtracts a perturbation c i (x i |x ) ≥ 0 which disappears in equilibrium but is apt to affect behavior elsewhere; see Geanakoplos (2000). 10 Indeed, out of equilibrium, considerable cost of change may steady a player's trembling hand and focus his mind.
In whatever form, b i (x i |x ) is meant to measure cardinal betterment for player i. Contending with ordinal comparisons, there is a noteworthy link to characterization and existence of stationary points: Proposition 3.2 (on concave ordinal improvements). For each i ∈ I , suppose X i is a nonempty compact convex subset of some topological vector space X i . Further suppose that with gross benefit function β i : X → R concave inx i ∈ X i and jointly continuous in x ∈ X . Then, there exists at least one Nash equilibrium in the game G = (β i , X i ) i∈I . Each such equilibrium x is stationary (Definition 2.1) for When moreover, b i (x i |x ) = 0 ∀i, that equilibrium is multi-agent stationary (8).
is a Nash equilibrium and stationary (Definition 2.1) for (12). Provided b i (x i |x ) = 0 ∀i, condition (8) is also satisfied.
Maintaining the conditions on β i , Proposition 3.2 fits the instance (10) when c i (x i |x ) ≥ 0 is convex inx i and continuous in x. Then, it also enters within the frames of Theorem 3.1. Monderer and Shapley (1996) for some player-independent generalized ordinal potential P : X → R∪ {−∞} with P finite on X . Then, P may replace β i in (11). In many games, strategic interaction works via objectives and constraints; see (Flåm and Ruszczyński 2008;Flåm 2017) and references therein. In fact, besides the individual restrictions that x i ∈ X i ∀i, choice could also be subject to coupling constraints in that each strategy profile x = (x i ) must belong to a non-empty, "non-rectangular" subset X i∈I X i . Then, still with cost of strategy change embedded in a profile x ∈ X is stationary-and declared a generalized Nash equilibrium -iff (Definition 2.1) holds. Theorem 2.1 immediately entails (13) is quasi-concave inx, lower semicontinuous in x, and b(x |x ) ≤ 0, then there exists a generalized Nash equilibrium.

Theorem 3.1 (on stationary states and generalized Nash equilibria). Suppose X is a nonempty compact convex subset of a topological vector space
Remark (on strong Nash equilibria). Present several players, expression (13) hides, or glosses over, the important fact that, in general, individual benefits need neither be comparable nor transferable-whence do not easily add to form a meaningful overall criterion. So, while format (13) may shed light on computation or existence of stationary points, it doesn't necessarily suit models of disequilibrium dynamics.

Cost of displacement in metric spaces
This section specializes one feature but generalizes another. The space X is metric now but convexity plays no role any longer. A main motivation is to loosen the grips convexity conditions hold on game theory.
Here, to bring different problems under a common umbrella and save space, x ∈ X denotes either the position of an isolated optimizer or the strategy profile x = (x i ) of several interacting agents i ∈ I . For greater flexibility in modelling step-wise adaptations, one may replace the timeinvariant A in (4) with stage-dependent correspondences A k : X ⇒ X to have Definition 4.1 (On asymptotic closure and regularity) In a metric space (X, d), a limiting correspondence A : The rest of this section only considers workhorse model (2) For illustration of (14), replace functions β, c with stage-dependent versions β k : Proposition 4.1 (on convergence). Let (X, d) be complete metric, X ⊆ X non-empty closed , and the sequence A k (16) closed by the workhorse correspondence Suppose any sequence x k+1 ∈ A k (x k ), k = 0, 1.. , has lim sup β k (x k ) finite and for some number δ > 0. Then, (A k ) is asymptotically regular, and any sequence (x k ) generated by (14) & (16) converges to a stationary point (Definition 2.1).
Proof By telescoping, Since the metric space is complete, (17) implies that x k → x for some unique limit x. Also by (17), there is asymptotic regularity: d(x k+1 , x k ) → 0. Hence, by closure (15), x ∈ A(x), and stationarity obtains.
Contending without convexity here, existence of a stationary point (Definition 2.1) requires other arguments. For such, a minor addition to Caristi's theorem (Caristi 1976) will do: Theorem 4.1 (on singleton fixed points of a correspondence). Let the space (X , d) be complete metric, A : X ⇒ X a point-to-set correspondence with non-empty values, and β : X → R bounded above, upper semicontinuous. If, for each x ∈ X andx ∈ A(x), then, A has a fixed point x at which A(x) reduces to a singleton (7).
Proof following (Kirk 1976), is included for completeness. The binary relation defines a partial order on X . Consider totally ordered subsets of X , all containing some fixed reference point. By Zorn's lemma, there exists a totally ordered X ⊆ X which is maximal under set inclusion ⊆. Let X = {x n : n ∈ N } for some index set or "net" N , totally ordered in the same manner: xn x n ⇐⇒n n. The net n ∈ N → β(x n ) ∈ R is increasing. Being bounded above, lim n↑ β(x i ) =: r ∈ R is well defined (Kelley 1955). Consequently, given any ε > 0, there exists n(ε) ∈ N such that n n(ε) ⇒ r ≥ β(x n ) ≥ r − ε.
Therefore, ifn n n(ε), then This proves that (x n ) n∈N is a Cauchy net (Kelley 1955), hence has a limit x. Now, in the last string, letn ↑, and invoke the upper semicontinuity of β, to get and thereby, x x n . Since X is maximal under inclusion, x ∈ X . Also, for anyx ∈ A(x), it follows from (18) thatx x. So, again since X is maximal under inclusion,x ∈ X . But thereby, x x. The upshot is that {x} = A(x).
Specializing A to be the workhorse model (2) and (3), with c ≥ 0, it follows forthwith: Corollary 4.1 (on singleton stationary point).Let (X , d) be a complete metric space, and β : X → R bounded above, upper semicontinuous. For each x ∈ X , suppose the set is non-empty, with c(· |x ) ≥ d(·, x) upper semicontinuous. 12 Then, there exists a stationary point x (Definition 2.1)at which Concluding this section is a metric existence theorem on strong Nash equilibrium. As customary, a player may impact the objectives of others but, less common, also their constraints. Moreover, the selected equilibrium becomes strong by withstanding joint deviations. From Theorem & Corollary 4.1 follows directly: Theorem 4.2 (on strong Nash equilibria). Let agent i ∈ I choose his strategy x i in a subset X i of a complete metric space (X i , d i ). Suppose non-cooperative play be constrained to a non-empty closed set X ⊆ i∈I X i in the product space X = i∈I X i , endowed with compatible metric d. Let agent i worship maximization of own benefit (  (12), there exist a Nash equilibrium x ∈ X which is strong in that

Stackelberg games
Stationarity (8) captures non-cooperative equilibrium in strategic-form games where players commit their moves once and simultaneously. That form hardly illuminates extensive-form instances where players act step by step. 13 In such games, some "protocol" governs who moves next-on the basis of which information.
To illustrate some of the difficulties that emerge, this section concludes by considering, in some generality, the most "simple" yet utterly important setting of Stackelberg sort (Lignola and Morgan 2017). Merely two players take part: I = {±1}. Each moves just once but in specific order.
The leading player 1 first chooses some x 1 ∈ X 1 . Observing that choice, the other player −1 responds, choosing some x −1 ∈ X −1 . Thereafter, each collects his benefit β i (x i , x −i ). Every set X i is a compact topological space. As before, let X := i∈I X i .
In essence, the follower reduces to a strategic dummy. He is just an "agency", a robot who selects some best response By contrast, up front, the leader ought maximize β 1 (x 1 , R(x 1 )) subject to x 1 ∈ X 1 .
His task is often rather demanding. He had better foresee or guess-or outright be told-the entire response correspondence R(·) (19). Moreover, if some R(x 1 ) isn't a singleton, which selection therein appears appropriate? 14 Suppose the two agents play this stage game iteratively, each remembering his most recent choice. Can they eventually learn-and implement-an equilibrium of the underlying Nash-Stackelberg variety? 15 For a positive answer, suppose players, upon entering stage k = 0, 1, .., with most recent choices x k i ∈ X i , already sunk, use conditional benefit functions Inequality (20) reflects two features. First, agent i incurs some non-negative cost by deviating from his last choice x k i . Second, each conditional benefit β k i underestimates the true version β i . Suppose that Similarly, suppose the condition Assumptions (21) and (22) capture that ultimately, as play settles, adjustment costs disappear.
For (21), provided the leader's choice x k 1 ∈ X 1 converges to some x 1 , asymptotically his cost of change doesn't affect his benefit. For (22), provided x k −1 → X −1 x −1 and the strategy pair χ k ∈ X converges to some χ, asymptotically the follower's cost of change has no impact on his benefit. 16 At stage k the leader believes or expects that the follower will apply a single-valued response function r k : X 1 → X −1 -not necessarily a selection of R (19). His belief or expectation must, however, be approximately rational in so far as, for any most recent χ 1 , χ −1 , On these premises, at stage k, the leader chooses an update After observing x k+1 1 , the follower comes up with a best response The resulting process is not construed as realizing equilibrium play in an infinitely repeated stage game. More simply, the limit should just qualify as Nash outcome in a single interaction over two stages. Note that because of the sequential mode of play, the coupled updates (24 ), (25) do not fit (16). Nonetheless, it holds: Theorem 5.1 (convergence in Stackelberg games). Let hypotheses (20)-(23) be in vigor. Suppose each function x ∈ X → β i (x) is upper semicontinuous, and that the leader's objective β 1 (x 1 , x −1 ) is lower semicontinuous in x −1 ∈ X −1 . Also suppose that for any point χ = (χ 1 , χ −1 ) ∈ X and sequence χ k If r k converges continuously to some r : X 1 → X −1 , meaning then it holds for each limit x 1 = lim x k 1 ∈ X 1 of the leader's play (24) that x 1 ∈ arg max β 1 (·, r (·)) and x −1 := r (x 1 ) ∈ arg max β −1 (·, x 1 ) = R(x 1 ). 16 (20) holds if (21) and (22) hold when each c k i → 0 + .
Proof Player 1 chooses x k 1 at stage k. Suppose x 1 = lim x k 1 . Then, by continuous convergence of the response functions (27), r (χ 1 )).
Invoking numerous assumptions, Theorem 5.1 proves existence and learning of Nash-Stackelberg equilibrium in an extensive game, featuring just two players, each moving just once, a leader ahead of the follower. It appears that several followers could be accommodated.
Admittedly, the modelling leaves several open ends. In particular, what sort of approximations β k i might be expedient? What learning scheme, if any, could justify which response functions r k ? And, when will these functions converge continuously? These questions go beyond the scope of this paper. Suffice it to say that, for finite-action games, fictitious play may offer insights (Brown 1951;Fudenberg and Levine 1994;Robinson 1951;Shapley 1964). For games with continuous actions spaces, see the proximal point procedures in Caruso et al. (2018).
Much criticism of economics and operations research centers on the paradigm of optimizing behavior (Scitovsky 1992). Both fields of inquiry depict agents who know what they are doing and do the best they can. This paper rather emphasizes that agents, within their circumstances, just contend with-but invariably seek-eventual improvements, if any. On that simpler premise, some equilibrium may eventually obtain. article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.