Investment and operational decisions for start-up companies: a game theory and Markov decision process approach

This paper analyses the contract between an entrepreneur and an investor, using a non-zero sum game in which the entrepreneur is interested in company survival and the investor in maximizing expected net present value. Theoretical results are given and the model’s usefulness is exemplified using simulations. We have observed that both the entrepreneur and the investor are better off under a contract which involves repayments and a share of the start-up company. We also have observed that the entrepreneur will choose riskier actions as the repayments become harder to meet up to a level where the company is no longer able to survive.


Introduction
A start-up company is created when an entrepreneur with an idea for a new product or service is supported by investors who can provide sufficient capital to allow the new venture to become established. During this initial phase, the objectives of the entrepreneur and investors are likely to differ. Investors are seeking a good return on their investment in the company, while some entrepreneurs might be more interested in establishing the company as a viable business to provide sufficient income and a stimulating working environment in the future. The investors' return and the entrepreneur's management of the company are largely determined by the contract or agreement specifying how much is to be invested initially and the timing and form of repayments made by the entrepreneur to investors. Investors wish to design a contract that incentivises the entrepreneur to manage the company in a way that provides an attractive return on their investments. If the repayment terms are too harsh, so that the company is almost certain to fail regardless of the actions of the entrepreneur, the entrepreneur may devote little effort to managing the company and returns for investors will be low. Similarly if the repayment terms are too benevolent, so that the company is almost certain to survive regardless of the actions of the entrepreneur, the entrepreneur may divert effort into activities to enhance the product or service without generating additional value. Therefore, we might expect that the contract should ensure that there is a good chance that the company survives provided the entrepreneur exerts effort to deliver value and, therefore, also provide a good return for investors. In this paper, we investigate how the design of the contract between the entrepreneur and investors influences the entrepreneur's actions and the outcomes for both parties.
Theoretical models of start-up finance contracts in the academic literature generally assume that the objectives of the entrepreneur and investors can be expressed in monetary terms (for example Elitzur and Gavious 2003b;de Bettignies and Brander 2007;Aouni et al. 2013;Lukas et al. 2016;Tavares-Gärtner et al. 2018). However, as Burchardt et al. (2016, p. 34) observe, investors and entrepreneurs often have conflicting objectives due in part to non-monetary benefits that entrepreneurs associate with "their role in the company and with the existence of the company as a whole". Our research assumes that investors seek to maximize the expected return on their investments, but that the entrepreneur seeks to maximize the probability of survival of the company. We argue that this assumption is more appropriate for "lifestyle" entrepreneurs who seek a good living from the start-up company and consequently do not prioritise high profit and growth. It has been reported that 90% of all start-ups are lifestyle ventures (de Bettignies 2008), so this type of entrepreneur represents an important segment. Moreover this segment has largely been overlooked in the literature on start-up finance contracts which focuses on high-growth start-ups financed by venture capitalists. Schwienbacher (2007) observes that the stigma of failure can make securing finance difficult in the future and so offers another reason for an entrepreneur to focus on the survival of the venture. Wennberg et al. (2016) propose a framework in which entrepreneurs make decisions based on survival and aspiration reference points. Empirical evidence supports the view that an entrepreneur's emphasis on survival changes with factors such as the age, size and performance of the venture. Our model considers the early development of a start-up company when we argue the entrepreneur is most concerned with survival. If the company is able to survive for a sufficiently long period, the entrepreneur's goals may shift to become more aspirational.
Most of the academic literature on start-up finance contracts focuses on the involvement of venture capitalists. See Burchardt et al. (2016) for a review of theoretical and empirical research in this area. Models generally assume that the success of the venture depends on effort from the entrepreneur and the venture capitalist. Entrepreneurs contribute creativity and technical skills while venture capitalists contribute business expertise (Casamatta 2003). As each party's contribution to the venture is not observable by the other, this creates a double moral hazard problem. The aim of contract design is to provide incentives which mitigate the impact of the moral hazard (Vergara et al. 2016). Fairchild (2011a) introduces an interesting element to this strand of the literature by considering how the behaviour of both parties is affected by their concept of social fairness. The literature emphasises the active role that the venture capitalist plays in managing the start-up company which might conflict with the entrepreneur's desire for control and could even stifle innovation (Cestone 2014).
In contrast to the literature on venture capital contracts, our model considers a start-up company that is financed by a passive investor. Studies that have analyzed different schemes and policies for debt financing include Cerqueti (2012), Tavares-Gärtner et al. (2018) and Cole and Sokolyk (2018). Schwienbacher (2007) highlights the reluctance of venture capitalists to invest in early stage ventures due to the small sums involved and the relatively high cost of monitoring the venture. The gap is filled by other investors such as business angels and banks who generally have a less active role in the management of the venture. It has been pointed out that venture capital funding does not have the largest share of entrepreneurial finance and is concentrated in high technology industries (de Bettignies and Brander 2007). Despite this, there is limited research on contracts for the financing of entrepreneurial ventures involving other types of investors. Elitzur and Gavious (2003a) examine a model in which a business angel invests in a start-up company prior to additional investment from a venture capitalist. Fairchild (2011b) models the choice of start-up finance under the assumption that business angels have more empathy towards the entrepreneur than venture capitalists. De Bettignies and Brander (2007) and de Bettignies and Duchene (2015) examine the entrepreneur's choice between venture capital finance and bank finance. Zhong et al. (2018) propose a model to match venture capitalists investment profiles/preferences with those of the entrepreneur, while Aouni et al. (2013) propose a multi-criteria model to maximize the return of the investment as well at the survival rate while trying to minimize investment risk. Schwienbacher (2013) considers investors with different degrees of specialism. Generalist investors offer continuity of finance through the stages of the start-up company's development, but may not provide the best support for each stage.
Our model assumes that the investor is not involved directly in the management of the startup company. We argue that this is appropriate for an early-stage start-up company created by a lifestyle entrepreneur who places a high value on control of the venture. In common with much of the literature, we consider contracts that provide finance in the form of debt and equity. We examine how the form of the contract affects the outcomes for the investor and the entrepreneur. This analysis provides insight on the design of contracts that give incentives for the entrepreneur to exert effort that results in an attractive return for the investor.
Most models of start-up finance contracts assume that the success or failure of the venture is revealed at a fixed point in time (for example Vergara et al. 2016;Fairchild 2011b;Elitzur and Gavious 2003a;Zhong et al. 2018). This does not reflect the way in which an entrepreneur may draw on capital reserves to endure a run of poor outcomes or accumulate resources before pursuing a high growth strategy. Coad et al. (2013) propose a classical random walk as an approximation to the growth path for the newest and smallest start-up companies. We build on this research by assuming that the growth of an early-stage start-up company follows a random walk that is influenced by the effort exerted by the entrepreneur. Multi-period models of start-up finance in the literature generally relate to the staging of venture capital finance (see for example Admati and Pfleiderer 1994;Dahiya and Ray 2012). The period represents a stage in the development of the venture. In each period, investors have the option to invest further or to exit. Depending on the type of contract, continued investment may be assumed provided pre-determined goals are met (Elitzur and Gavious 2003b) or may be a decision taken in the period (Lukas et al. 2016). The period in our model plays a completely different role. The success of the venture depends on the outcomes of a series of trials (one per period) rather than a single outcome at the end of the stage. In common with Elitzur and Gavious (2003b), we consider a multi-period model in which the finance contract is agreed initially and each period in the planning horizon is either a success or a failure. However, while Elitzur and Gavious (2003b) assume that failure is terminal, we assume that failure in a period is only terminal if it exhausts the resources available to the company. Hence, we combine a multi-period model of start-up finance with the concept of a growth path based on a random walk. Elitzur and Gavious (2003b) assume that for as long as the company remains successful, the venture capitalist invests in the next stage, pays the entrepreneur an agreed amount in each period and accumulates any surplus value. We argue that this implies a high level of investor control which is not the scenario we wish to investigate. In contrast, we consider investors who make an upfront investment in return for a share of the venture and payments from the entrepreneur in each period. We assume that investors exit at the end of the agreement leaving the entrepreneur in control of the venture.
The paper is organized as follows. Section 2 provides a detailed description of the model analyzed. Section 3 discusses the optimal contract design from the viewpoints of the entrepreneur and the investors. Section 4 presents the results of a numerical investigation into the impact of the contract design on the outcomes for the entrepreneur and investors. Section 5 concludes.

Basic setup
The model considers an entrepreneur (E) and a single investor (I ) and describes the decisions that have to be made as part of the initial agreement between the two parties. These decisions concern how much I invests initially, the timing of payments from E to I and the form that these payments take. Subsequently, these decisions influence the way the company is run in terms of the deployment of effort by E and the risk associated with E's decision making. Ultimately, the terms of the agreement and the behaviour of E determine the payoffs for both parties. We model the situation as a non-zero sum game between the two parties.
We assume that E is concerned with the way the company prospers over a time interval consisting of N periods, but that I will only wish to be involved in the venture for a limited time of T periods (T < N ). The initial agreement between the parties, determines how much I invests initially, namely C 0 , the time for which I is involved in the venture, T , and the dividends that I will receive from E in each of the subsequent T periods, C j for j = 1, . . . , T . It is assumed that C j ≥ 0 for 0 ≤ j ≤ T . However we note that, by allowing C j < 0 for some j satisfying 1 ≤ j ≤ T , the model could easily be adapted to cover the staging of finance. The initial agreement also determines α (satisfying 0 ≤ α < 0.5) such that 100α is the percentage of the company that is owned by I . Note that 100α is constrained to be less than 50% to ensure that E always retains a controlling interest in the company. Under the terms of the agreement, I will exit at time T by withdrawing 100α% of the value of the company at that time.
With the agreement in place, E then manages the company for N periods by choosing the effort to devote to value creation in each period. These decisions are dependent on the success of the company which is measured by its accumulated capital. In each period there are M possible effort levels ranging from level 1, with least effort, least return and least risk, to level M, with most effort, most return and most risk. We assume that in each period, the company is either a success or a failure. When E chooses effort level k in any period, then: • S(k) = profit to the company when its activities during the period are a success; • F(k) = loss to the company when its activities during the period are a failure; • p(k) = probability the company's activities during the period are a success. Hence, when effort level k is chosen in a period, the expected profit is given by and the variance in profit is given by We assume that the expected profit and the variance in profit, which we consider as an indicator of risk, are non-decreasing as the effort level k increases. We note that the model could be generalised to allow the profit, loss and success probability to depend on the period. For example, these values could be chosen to reflect a projected pattern of growth for the company.
We assume the following sequence of events in each period, see Fig. 1. First E chooses the effort level for the period. Then the outcome of whether or not the company is successful in the period is observed. Next I receives the agreed dividend for the period or, if the company has insufficient capital to meet this payment, the maximum amount possible. Finally, the capital of the company is calculated and, if positive, the company is allowed to operate for (at least) another period. If the company runs out of capital and cannot trade anymore, we assume k = 0 (with S(0) = F(0) = 0) to represent E's action in this "coffin" state.

Entrepreneur's objective
We assume that the objective of E is to maximize the probability of survival of the company. The planning horizon for E consists of the first T periods, during which I is involved with the company, followed by a further N − T periods without I 's involvement. We have the following two objective functions of interest for the entrepreneur: • Q j (x) = the maximum probability that the company survives until the end of period N given the company has accumulated capital x by the beginning of period j where 1 ≤ j ≤ T ; • q j (x) = the maximum probability that the company survives until the end of period N given the company has accumulated capital x by the beginning of period j where T + 1 ≤ j ≤ N .
We define k j (x) to be the optimal action for E in period j, satisfying 1 ≤ j ≤ T , when the capital available to the company is x.
At the end of period T , I withdraws 100α% of the value of the company. This is reflected in the boundary condition The company survives if and only if the accumulated capital at the beginning and end of every period is positive. This leads to further boundary conditions: For x > 0 and T +1 ≤ j ≤ N , the objective function q j (x) is determined by the following optimality equation: While for x > 0 and 1 ≤ j ≤ T , the objective function Q j (x) and the corresponding optimal action k j (x) are determined by the following optimality equation: Note that Eq. (1) calculates the effort level to maximize the probability of survival after I is no longer involved, while Eq.
(2) calculates the effort level to maximize the probability of survival when I is still involved and so has to account for the dividend paid. The objective of E is to maximize Q 1 (C 0 ).

Investor's objective
We assume the objective of I is to maximize the expected net present value (NPV) of the investment in the company. Let r be the effective interest rate per period, so that future payments are discounted at a factor of 1/(1 + r ) per period. Define V j (x) to be the expected NPV of future payments to I at the beginning of period j (1 ≤ j ≤ T ) when the capital available to the company is x. We assume that I is aware of E's objective to maximize the probability of the venture surviving for N periods. Therefore, I can anticipate the actions of E for any given contract design. For example, when the capital available to the company at the beginning of period j is x, I can anticipate that E would choose the action k j (x) determined from Eq. (2). Hence, for j satisfying 1 ≤ j ≤ T and for all x > 0, V j (x) is determined as follows: where V T +1 (x) = αx if x > 0 and V j (x) = 0 if x ≤ 0 and 1 ≤ j ≤ T + 1. We assume that the initial investment is made at the beginning of the first period and that dividends are paid just before the end of a period. The objective of I is then to maximize:

Negotiating a contract
The two parties, I and E, are involved in a non-zero sum game to determine α, T , C 0 , C 1 , . . ., C T in which I is interested in maximizing V 0 (C 0 ) and E is interested in maximizing Q 1 (C 0 ). It is easy to see that E prefers the investment to be as large as possible, to pay I as little as possible, as late as possible and to retain as much of the company's value at period T as possible. These observations are formalised in the following proposition.

Proposition 1 (i) Q 1 (x) is non-decreasing in x.
(ii) LetQ 1 (x) denote the probability that the company survives until the end of period N given the company has capital x initially and pays I a single dividend not exceeding Proof In any situation in which the company survives with initial capital x, the company would also have survived if it had chosen the same actions with initial capital greater than x. Hence (i).
Note that C j ≥ 0 for 1 ≤ j ≤ T , so the payment of the dividend in any period will never increase the capital available to the company in that period. Hence, in any situation in which the company survives, the company would also have survived if it had chosen the same actions with the same initial capital but reduced the value of one or more of the dividends and/or delayed the payment of dividends until period T . The result (ii) follows.
Finally for (iii), in any situation in which the company survives, the company would also have survived if it had chosen the same actions with the same initial capital but I had withdrawn less of the company's value at time T .
In contrast, it is not obvious what form of agreement should be favoured by I . While increasing the dividends paid to I and increasing the proportion of the company owned by I will increase the potential payoff for I , the benefit might not be realised because of a reduction in the probability the company survives until the payments to I are due or because E chooses to devote less effort to value creation. Due to the complexity of dealing with the most general form of contract, we restrict our attention to cases where E agrees to pay I a fixed dividend of C per period from period t until period T . There are then 5 variables that define a contract: C 0 , the initial investment; C, the fixed dividend; t, the period in which E starts paying the dividend; T , the period in which I 's involvement in the venture ends; and α, the proportion of the company's value paid to I on exit. In the following section we investigate computationally how the form of contract affects the return to I , the probability of survival of the company and the actions of E.

Numerical investigation
We investigate the impact of the design of the contract between E and I on the outcomes for both parties using variants of a base case problem with the following parameters. The planning horizon for E consists of N = 36 periods and the planning horizon for I consists of a maximum of 24 periods. Notionally, we consider a period to be an interval of around 3 months,  Fig. 2 Survival probability of company versus total repayment for four contract designs so that the planning horizon for I is at most 6 years. This is consistent with the findings of Manigart et al. (2002) who report that early-stage ventures mature after approximately 6 years on average. Initially I makes an investment of C 0 = 40 units. While the capital available to the company remains positive, E chooses one of M = 2 levels of effort. With effort level 1, the company makes a profit of S(1) = 10 during a period with probability p(1) = 0.6 and makes a loss of F(1) = 10 otherwise. With effort level 2, the company makes a profit of S(2) = 12 during a period with probability p(2) = 0.9 and makes a loss of F(2) = 50 otherwise. Table 1 compares the characteristics of the two effort levels. The effective interest rate per period is r = 0.01. The parameters to be determined by the design of the contract are t, T , α and C. We constrain α to be at most 0.45 so that E always owns at least 55% of the company. When comparing contracts, we consider the total repaid to I which is equal to (T − t + 1)C. For any given total repayment, E's optimal contract provides an upper bound on the survival probability of the company (by definition) and a lower bound on I 's expected NPV (as E would readily accept the terms of this contract). Hence, E's optimal contract is a useful benchmark when evaluating contract designs.
We initially consider how the total repaid to I affects the objectives of the two parties under E's optimal contract. We know from Proposition 1 that the optimal design of contract for E will always set α = 0 and t = T = 24. The solid line in Fig. 2 shows that, as expected from Proposition 1, the probability of survival of the company decreases as the total repaid increases. The graph shows that the fall in the survival probability is generally steady with occasional steep drops after 200, 260 and 320. This pattern can be explained by examining the actions chosen by E. Using simulation, the frequency with which E chooses the riskier action during the first T periods of the planning horizon was estimated. Figure 3 shows that, in general, the frequency with which E chooses the riskier action increases as the total repayment increases. This trend is to be expected because the company needs to earn more revenue to meet higher repayments. However, the frequency with which the riskier action is chosen drops sharply after 200, 260 and 320 as it becomes impossible for the company to survive in more scenarios. This has an impact on the expected NPV of I , as can be seen from the solid line in Fig. 4. Under the optimal contract for E the expected NPV generally increases with the total repayment, but it falls sharply after 260 and 320 (and slightly after 200) due to the shift in the policy chosen by E away from the riskier, more profitable, action. Eventually, the repayment is so great (> 320) that the chance of the company being able to meet it in full is negligible. In such cases regardless of the contract design, E always chooses to exert the least effort (k = 1), the probability of survival is 0 and the expected NPV is constant at a relatively low level. This confirms the intuition that when the terms of the contract are too onerous, E will not devote much effort to the management of the company and both the survival probability and expected NPV will be low. The crosses on the vertical axes of Figs. 2 and 4 show the survival probability and expected NPV, respectively, under the optimal contract for I involving a share of the company, but no payments. While the survival probability is high, the expected NPV is low (actually negative) and this contract would not be attractive to I . Figure 5 shows that when the contract includes a share of the company, I would generally prefer as large a share as possible. (Recall that in this problem the maximum share is 45%.) The only deviation from this occurs for relatively high total repayments when the company has little chance of survival and, consequently, the contract will not be attractive to E and the contribution of the share of the company to I 's expected NPV will be relatively small.
The dashed lines with the circle and diamond markers in Fig. 4 show the expected NPV under the optimal contracts for I with payments and a share of the company and with payments the survival of the company and so the objectives of both parties are more closely aligned. This is likely to have substantial impact on the outcome of negotiations. In this example, by including a 45% share of the company, E can offer I a contract with an expected NPV of more than 50 units while ensuring a survival probability of 54% rather than 40%. Further insight on the tradeoff between I 's expected NPV and the survival probability of the company is provided in Fig. 8. Points on this figure represent pareto optimal contract designs for the problem. Contracts involving payments with no share of the company are only pareto optimal when survival probability is relatively low, while contracts involving a share of the company with no payments (including the optimal contract for E) are only pareto optimal when expected NPV is relatively low. The vast majority (95%) of pareto optimal contract designs involve contracts with both payments and a share of the company. Most importantly, all the pareto optimal designs in the region where a compromise is likely to be struck, correspond to contracts involving payments and a share of the company. There are 38 pareto optimal designs that provide an expected NPV of at least 50 and a survival probability of at least 0.5. Within these designs the total repayment ranges from 120 to 190, the share of the company ranges from 0.15 to 0.45, the period in which payments start ranges from 8 to 15 and the period in which I exits is either 23 or 24. Contract designs with higher survival probability generally involve lower repayments that start later in the planning horizon. Most of the contract designs involve a 40% or 45% (the two highest values considered) share of the company. We calculated the internal rate of return (IRR) of the investment in the company for the 38 pareto optimal contract designs. We found the IRR to be in the range 5-9% per period which, assuming a period to be 3 months, corresponds to 21-41% per year. Manigart et al. (2002) report that, on average, venture capitalists require an IRR of between 36 and 45% for early stage investments. This suggests that the IRR for the problem considered here might be too low to interest venture capitalists. However, we argue that such a project would need to seek funding from other types of investors and so the IRR is appropriate given our focus.
The analysis provides evidence that both parties should favour contracts that provide I with regular payments and a share of the company. It also provides a rough characterisation of a contract that should be fair to both parties. Of course this discussion is based on only one combination of the model parameters. However, the details presented above are typical of the many different parameter combinations we have considered.
We have performed sensitivity analysis on the parameters of the base case problem within the ranges shown in Table 2. The parameter combinations were chosen to ensure that the mean return per period was unchanged for both actions and that the variance of return remained lower for action 1 than for action 2. In all cases the initial investment was C 0 = 40 units, E's planning horizon was N = 36 periods, I 's planning horizon was at most 24 periods and the effective interest rate per period was r = 0.01. We investigated the impact of varying the interest rate per period and increasing E's planning horizon, but the results proved to be particularly insensitive to these changes.
For each combination of parameters, we calculated all pareto optimal contract designs with an expected NPV of at least 50 and a survival probability of at least 0.5. For ease of reference, in the following discussion we will call the pareto optimal contract designs satisfying these criteria "candidate designs" to reflect the fact that such contract designs might be said to satisfy both parties' minimum requirements. All of the candidate designs provide I with payments and over 99% of them provide I with both payments and a share of the company. The modal share of the company in the candidate designs was 45% (the highest value considered) and over 70% of these designs allocated at least 35% of the company to I . The period in which repayments started typically fell around the middle of the planning horizon with repayments starting between periods 8 and 17 more than in 52% of the candidate designs. However, in approximately 27% of the candidate designs repayments started within the first 3 periods of the planning horizon. We observed that this happened in cases for which action 2 involved a higher level of risk. The period in which I exits was never earlier than period 20 for any of the candidate designs and I exits in period 24 (the maximum value considered) in 85% of the cases. In more than 90% of candidate designs, the total repayment was in the range 120-200.

Conclusion
We have presented and analyzed a Markov decision process model to study optimal contracts between an entrepreneur and an investor, focusing on a non-zero sum game where the entrepreneur is interested in company survival and the investor in maximizing expected NPV. We have presented some theoretical results about the model and demonstrated the usefulness of the model using simulations with numerical examples. We found the model useful in explaining behaviours about the entrepreneur's effort level, and the general characteristics of a contract under different scenarios. An entrepreneur will choose riskier actions as the repayments become harder to meet which will impact the expected NPV up to a level where the company is unable to survive. Both the entrepreneur and investor will be better off under a contract with payments and a share of the company. Contracts involving a share of the company give the investor an interest in the survival of the company and bring the objectives of the two parties closer together. The investor might then agree to a contract in which payments start later in the planning horizon and are spread over a longer interval. Contracts with these characteristics tend to give the company a higher chance of survival.
Our analysis suggests that the investor and the entrepreneur should agree a contract that provides the investor with regular repayments and a share of the company. Under the terms of this contract, the total repayment will typically be 3-5 times the value of the initial investment and repayments will typically start around the middle of the investor's involvement with the venture. The contract agreed will typically allow the entrepreneur to retain between 55 and 65% of the company. These findings provide insight on the most desirable outcomes of the negotiations between a passive investor and a lifestyle entrepreneur.