Risk Aversion, Reservation Utility and Bargaining Power: An Evolutionary Algorithm Approximation of Incentive Contracts

Abstract

In this paper we analyze the links between the agent’s reservation utility, bargaining power, and risk aversion in terms of their simultaneous effects on the structure of optimal static contracts. We compare the following principal-agent models in the symmetric and asymmetric information environments: the standard approach, which includes a participation constraint, and a multi-objective (MO) optimization approach in which the objective function is a convex combination of the expected utilities of the principal and the agent. The MO model does not include a participation constraint, but it includes a parameter for the agent’s bargaining power. We also study an Evolutionary Algorithm implementation of the static principal-agent model to support and extend our analytical results. We show that the numerical solution approximated by our implementation of an evolutionary algorithm is in line with the analytical solutions mentioned before. That is, for every admissible value of the agent’s reservation utility, there is a corresponding admissible value of the agent’s bargaining parameter, both in the MO approach and the EA implementation.

Appendices

Appendices

A Computing the Optimal Contract using an Evolutionary Algorithm

Multi-objective (MO) functions are used to optimize functions that conflict with each other. Then, there is no unique solution, but a set of solutions found through the use of the Pareto Optimality criterion. This type of problem is usually of a very high computational complexity, i.e., they are categorized as NP-Complete, see Coello et al. (2007). For this reason, many heuristics inspired by evolutionary algorithms have been used to solve these problems.

An evolutionary algorithm (EA) applies the principles of evolution encountered in nature to the problem of finding an optimal solution to a given problem. An EA manages a population of strings P(t) at each stage t. A chromosome characterizes each individual, each chromosome is a candidate solution to the problem domain and thus has a value related to his performance; this evaluation is called fitness value. Then, genetic material from selected parents is used to give birth to a new generation of offspring. Mating and mutation are prevalent methods to achieve genetic material variation, and the parameters that encapsulate these concepts are crossover and mutation. Though these mechanisms, the algorithm ensures the diversity of the solutions and a rapid search process.

B NSGA-II in the Principal-Agent Problem

NSGA-II (Non-dominated Sorting Genetic Algorithm) is one of the most efficient algorithms for MO optimization (Ishibuchi et al., 2008). The main features that distinguish this algorithm are fitness assignment, elitism, and mechanism. The critical idea in NSGA-II is the use of non-dominated sorting for fitness assignment. In this algorithm, individuals or agents are grouped in fronts, where the ones in the front i dominate all other fronts j with $j>i$. Tournaments between two agents are used in the selection process, where the agent that belongs to the lowest number front wins. Higher fitness is assigned to individuals located in the sparsely populated part of the front to enhance diversity. In every iteration, n new offsprings are generated from n parents; both parents and children reunite and compete with each other to be chosen for the next iteration. The details of the implementation of this algorithm can be found in Deb et al. (2002). The algorithm’s complexity is $O(mn^2)$, where m is the number of generations (iterations), and n is the population size.

Now, we explain the pseudocode of NSGA-II adapted to the principal-agent problem (Algorithm 1 of section 3). First, an initial population of wages is randomly generated following the constraints of the principal-agent problem, and the value of this population is evaluated for the first time through the principal-agent MO function (lines 2 to 3). The population is sorted based on non-domination, and the parent population is ranked (line 4). Then, an offspring population is created through binary tournament-selection, recombination, and mutation (line 5). The kth iteration of the while loop is as follows (lines 6 to 14). Ranked fronts are created from the combined populations of children and parents. Subsequently, sets are accommodated in the kth-population according to the crowding distance⁸ until all population slots are filled. Then, this population is used for selection, crossover, and mutation to create a new population. This procedure will continue until the stop condition is met. We use as stop criteria a number of generations big enough that ensures convergence; details about the quality of approximations are presented in the next sections.

1.1 Model Configuration

In this subsection, we describe the configuration of the MO approximation of the Pareto Frontier in the static principal-agent context with asymmetric information, for which the EA approximation is imperative. The principal is assumed to be risk neutral and has an expected utility function $\mathbb {E}U=\Pr (\pi _H|a)*(\pi _H-w_H) + \Pr (\pi _L|a)*(\pi _L-w_H)$, where w is compensation, e is the agent’s effort and $\pi _H$ and $\pi _L$ are high and low outcome respectively. The agent’s expected utility function is $\mathbb {E}V = f(\pi _H|a)(u(w_H)-a)+ f(\pi _L|a)(u(w_L)-a)$. The utility the agent gains from w is $u(w)=\frac{w^{1-h}}{1-h}$, where h is the relative risk averse coefficient and efforts are defined in $\mathbb {R}_+$. There are two feasible production levels $\pi _L=2$ and $\pi _H =4$. The probability function of output $\pi$ given effort e is defined by $f(\pi _L|a) = exp(-a)$ and $f(\pi _H|a) = 1-exp(-a)$.

The MO maximization problem is

$$\begin{aligned} \max _{ \begin{array}{c} {0 \le w_L \le \pi _L} \\ {0 \le w_H \le \pi _H} \end{array} } \left\{ \mathbb {E}U ,\mathbb {E}V \right\} \end{aligned}$$

s.t.

$$\begin{aligned} e\in \text {argmax }_{a'\ge 0} { \quad \Pr (\pi _H|a')*(v(w_H)-a' )+ \Pr (\pi |a')*(v(w_L)-a')} \end{aligned}$$

(26)

In the case of the perfect information solution, the same objectives functions are taken, but without constraint (26) and making $w_H=w_L$.To obtain the bounded solutions, we add the restrictions $\mathbb {E}U \ge 0$ and $\mathbb {E}V \ge 0$.

1.2 MOEA Configuration

In this work, we adapt the NSGA-II to find the Pareto Frontier of the principal-agent problem. The chromosomes of the individuals in the population are going to represent a contract. In the case of symmetric information, the phenotype of the individual is the pair $\{w, a\}$, that is the contract that both parties agreed. In the asymmetric information context, this entails that the phenotype of an individual is a pair $\{w_L,w_H\}$, where the first element indicates the compensation if the low output is observed and the second argument represents the compensation if the high output is observed. Fitness is the utility pair for the agent and the principal $\{U(w, a),$ $V(w, a)\}$, where w is the vector of salaries. Notice that the optimal effort induced for the wage pair is computed to get the fitness value associated with each individual; this is because agents chose their effort to maximize their utility.

To achieve the convergence of the frontier, we calibrate the four parameters of the genetic algorithm: the number of generations, initial population, crossover rate, and mutation rate. The number of generations indicates the number of iterations or loops; in our case, this parameter is also the stop criterion. The initial population refers to the number of individuals in the first generation, and choosing this parameter entails selecting both the size of the population and the particular individuals that comprise it. The crossover rate indicates the probability that a new offspring is born because his parents mated; for example, 40 percent means that 2 out of 5 new individuals are clones (or versions if there is mutation of chromosomes) of an older individual. The mutation rate indicates the probability that an offspring is conceived by variation of chromosomes. Both of these latter parameters are set on the [0, 1] interval.

Fig. 14

The genetic algorithm convergence for the case of symmetric information

Fig. 15

The genetic algorithm convergence path of the pf for the case of asymmetric information

Figures 14 and 15 depict the convergence of the first-best (symmetric information) and second-best (asymmetric information) solution, respectively. The setup in both cases is 10000 generations, we show selected cuts of the procedure, including the final iteration. For the case of symmetric information (Fig. 14) we choose 10 predetermined seeds (candidates) spread along the Pareto Frontier. On the other hand, for the asymmetric information example, the initial population size configuration is 50, and there are chosen randomly within the space of candidates.

In both models, the number of generations chosen for our simulations ranges between 10000 and 50000, and the initial population size ranged from 10 to 100. The crossover and mutation rates that we choose in our final simulations were 1.0 and 0.2, respectively. In the following sections, we show how these parameters were chosen in each case. The computational library used to support the implementation of the algorithm was Garrett (2012).

C Approximation Quality Measures

Given that NSGA-II is a stochastic algorithm, every time an approximation of the PF is found, it contains a different subset of the actual PF. Because of the difficulty of evaluating these solutions pose, enormous efforts have been carried out to find metrics that measure the quality of a Pareto set (Zitzler et al., 2003; Paquete and Stützle, 2018; Knowles et al., 2006). Accomplished measures should consider the distance between the front and the real Pareto Frontier, the extension of the front, and the diversification of the solutions.

In this work, we choose the Hypervolume Indicator (HV) to evaluate the quality of the approximations and parameters. This metric, also known as Lebesgue Measure or S-measure, was proposed by Zitzler (1999). The HV is the n-dimensional space contained by a solution set, i.e., the n-dimensional volume of the set relative to some reference point. Due to properties of the indicator, a set with a larger HV is likely to represent a better approximation than sets with lower HVs.

The Slicing Objectives procedure of While et al. (2005) implemented by the library was used to calculate the HV of a non-dominated set, see Garrett (2012). The reference point we select was (0, 0); our value functions are utilities, which are always positive in the bounded version. Usually, the HV measure is normalized in [0, 1] to compare it with other algorithms⁹.

Fig. 16

Hypervolume indicator (HV) across generations; risk aver coefficient is 0.5, crossover rate is 1.0 and mutation rate is 0.2

In Fig. 16, we depict the HV indicator progression of two simulations: in the left, the symmetric case, and, in the right, the asymmetric case. These simulations lasted for 10000 generations with an initial population of 10 and a risk averse coefficient of 0.5. Notice that the HV metric does not change significantly for the last 1000 iterations, which serves as evidence of the robustness of the convergence of the approximations of the PF.

Lastly, we examine the relationship between the risk aversion coefficient and the number of generations. Because the frontiers are considerably more extensive when the coefficient increases, we need more iterations to obtain the full frontier. Our experiments point that while for $h=0.5$, a 10000-generation approximation achieves a diversified outcome with full extension of the frontier, for $h=0.9$, at least 30000 iterations are needed with a random initial population of 50.

Parameters

Determining the values of the parameters has a considerable impact on the quality of the approximation; that is why, in this section, we focus on finding the best configuration for the crossover and mutation rate. To compare the different Pareto Frontiers, once again, we rely on the HV index, and we use the population size of the final iteration as a complementary measure. To carry these experiments, we set the risk averse coefficient on 0.5 because most of our analysis and examples have this setup. The number of generations, another critical parameter of the algorithm, is chosen in a range that ensures an adequate convergence according to the HV; see the previous subsection. Finally, we discuss the strategies employed to select the initial population setup.

Fig. 17

Convergence metrics by crossover and mutation for the case of symmetric information

Figure 17 presents a color map in which we vary the values of crossover and mutation, within the sets $\{0.4,0.6,0.8,1.0\}$ and $\{0.01,0.05,0.1,0.05,0.2\}$, respectively, as a function of two convergence metrics. In the first box, we depict the parameters as a function of the final population size, while in the second box, as a function of the HV index. The darker tones of red are associated with a higher level of the convergence metric. We obtain the simulation of the pairs of crossover and mutation by using the same set of initial population candidates and a number of generations of 10,000. The population size ranged from 4229 to 12147, and the HV index from 5.000 to 5.0276. In the second box, we show how the algorithm attains its best results with a 100% crossover rate and 20% mutation rate. Notice that for the case of symmetric information, the configuration that achieves the most populated approximation (crossover of 0.6 and mutation of 0.01) is not the one that gives us the highest HV index, and thus it is not the most diversified and extensive frontier (Fig. 18).

Fig. 18

Convergence metrics by crossover and mutation for the case of asymmetric information

Similarly, Fig. 17 depicts color maps that illustrate the relationship between crossover and mutation percentage for the same convergence metrics and employing the same setting, but for the case of asymmetric information. In this instance, the population size ranged from 2065 to 8712, and the HV index from 3.846 to 4.464. The second box reveals that the parameters for which the NSGA-II accomplishes the highest measures are a 100% of crossover rate and 20% of mutation rate; this time for both metrics. Therefore, the same configuration gives us the best outcomes for the symmetric and asymmetric information; for this reason, it is the setting we use to get the Pareto Frontiers in our numerical approach. One possible concern, as pointed out by DeJong (1975), is the large mutation rate (higher than 5%), which could potentially lead to random search. However, a large mutation rate is not unique to our setting, since many other papers also find high optimal mutation rates, see Hassanat et al. (2019) for an analysis on optimal mutation and crossover parameters. Moreover, we argue that the particularities of our optimization program and our parameter tuning are unlikely to lead to random search in our simulations. First, because we use the adjustments in archive population size, and control mutation rate parameters according to the archive population, a higher mutation rate is more convenient. When we have a small population size, we obtain different parameters, and, in particular, a smaller mutation rate. Furthermore, a large maximum number of iterations is needed for convergence, and thus initial iterations benefit from a larger mutation rate. Lastly, the nature of our problem helps explain why we prefer high mutation rates; our objective is to uncover the complete Pareto Frontier, so we benefit from greater exploration.

Finally, we debate our approach with the initial population parameter. The initial population size determines the search space; but also affects the computational burden. Accordingly, the balance between a small and big initial population must be considered. We find that a population size ranging from 10 to 70 individuals randomly selected, as in the example of Fig. 15, ensures that, in most cases, the solution is not biased to some portion of the Pareto Frontier and that the computational burden does not get unmanageable. We also play with candidates that constitute the initial population; we find that this strategy does not significantly improve the HV when holding the population size constant. On the contrary, when choosing, for example, the initial population across a previously obtained approximation, we get a slower convergence in terms of HV of 10000-generation approximations. We believe that this is because by serving the algorithm with a relatively small initial population with high MO values, the offspring tends to have lower fitness than their parents, hampering the evolution of the population. However, if the initial points are distributed across the frontier but are not close to the theoretical frontier, then the convergence occurs faster than with a random initial population, as seen in Fig. 14.