Research and Development Cooperatives and Market Collusion: A Global Dynamic Approach

Abstract

We present a continuous-time generalization of the seminal research and development model of d’Aspremont and Jacquemin (Am Econ Rev 78(5):1133–1137, 1988) to examine the trade-off between the benefits of allowing firms to cooperate in research and the corresponding increased potential for product market collusion. We show the existence of a solution to the optimal investment problem using a combination of results from viscosity theory and the theory of planar dynamical systems. In particular, we show that there is a critical level of marginal cost at which firms are indifferent between doing nothing and starting to develop the technology. We find that colluding firms develop further a wider range of initial technologies, pursue innovations more quickly, and are less likely to abandon a technology. Product market collusion could thus yield higher total surplus.

1 Introduction

An important reason for allowing firms to set up research and development (R&D) cooperatives is that these “organizations, jointly controlled by at least two participating entities, whose primary purpose is to engage in cooperative R&D” [1] internalize technological spillovers—the free flow of knowledge from the knowledge creator to its competitors. Indeed, Bloom et al. [2] estimate that a $10\%$ increase in a competitor’s R&D is associated with up to a $3.8\%$ increase in a firm’s own market value. The exemption for R&D cooperatives in anti-cartel legislation is thus perceived to diminish the failure of the market for R&D. However, as Scherer [3] observes: “the most egregious price fixing schemes in American history were brought about by R&D cooperatives”, an observation that constitutes the classic counterargument to a permissive antitrust treatment of R&D markets [4,5,6]. For instance, Goeree and Helland [7] find that in the US the probability that firms join an R&D cooperative has dropped due to a revision of antitrust leniency policy in 1993. This revision is perceived as making collusion less attractive. They conclude that “Our results are consistent with RJVs [research joint ventures] serving, at least in part, a collusive function.” The laboratory experiments of Suetens [8] also show that members of an RJV are more likely to collude on price. At the same time, it is quite well established that the prospect of future market power enhances a firm’s incentives to invest in R&D [9]. As Greenspan [10] puts it:

No one will ever know what new products, processes, machines, and cost-saving mergers failed to come into existence, killed by the Sherman Act before they were born. No one can ever compute the price that all of us have paid for that Act which, by inducing less effective use of capital, has kept our standard of living lower than would otherwise have been possible.

In this paper, we develop a dynamic model of R&D that considers explicitly the cost of “new ... processes” that “failed to come into existence ... before they were born” because of the ban on price-fixing agreements.

The channels through which cooperation in R&D facilitates product market collusion have been examined in a number of theoretical studies [11,12,13,14,15]. According to Fisher [16, p. 194]:

... [firms] cooperating in R&D will tend to talk about other forms of cooperation. Furthermore, in learning how other firms react and adjust in living with each other, each cooperating firm will get better at coordination. Hence, competition in the product market is likely to be harmed.

In the short run, the reduced intensity of product market competition is likely to hurt consumers. At the same time, it could enhance the functioning of an R&D cooperative. For instance, Geroski [17] argues that it is the feedback from product markets that directs research toward profitable tracks and that, therefore, for an innovation to be commercially successful, there must be strong ties between marketing and development of new products. And Jacquemin [18] puts forward that R&D cooperatives are fragile and unstable. He reasons that when there is no cooperation in the product market, there exists a continuous fear that one partner in the R&D cooperative may be strengthened in such a way that it will become too strong a competitor in the product market. Preventing firms from collaborating in the product market may therefore destabilize R&D cooperatives or prevent their formation in the first place. Our focus is on private incentives to develop cost-saving technologies over time. In particular, we show that if firms collude in the product market, a wider range of technologies is fully developed. We also show that firms competing in the product market realize an inferior productive efficiency. We thus identify situations where product market collusion increases total surplus.

Dynamic models of R&D were first introduced to study patent races whereby successful innovators capture the entire market. This literature starts with Loury [19] and Lee and Wilde [20] ([21] surveys the early contributions). Patent race models examine, in essence, the time it takes for a cost-saving innovation to be completed. R&D investments reduce this completion period. Because in these models the R&D process itself cannot fail, the R&D-investment decision is transformed into a static one. Meanwhile, a large literature has developed on the relation between intellectual property rights and antitrust policies. For instance, Quirmbach [22] finds that there is an optimal level of collusion that is in-between perfect competition and full collusion. And Green and Scotchmer [23] show that it is optimal to allow for collusion through sequential licensing in case the next innovation is a truly new application of existing patents. More recently, another strand of dynamic R&D models has developed: continuous-time generalizations of strategic R&D models. These models allow for “smoothing the investment efforts over a long time” [24], a type of investment behavior that is observed in practice and that constitutes a key feature of continuous-time models. Cellini and Lambertini [24] is the first continuous-time generalization of the seminal analysis of d’Aspremont and Jacquemin [25]. In the duopoly game of d’Aspremont and Jacquemin [25], firms first invest in cost-reducing R&D and then play a Cournot game in the product market. In the continuous-time version of Cellini and Lambertini [24], both firms start from an initial technology (that is, a level of marginal cost) and invest continuously in R&D. This gradually reduces the initial level of marginal cost toward the steady-state level. In contrast to the static generalization of d’Aspremont and Jacquemin [25] by Hinloopen [26], Cellini and Lambertini [24] find that the aggregate level of R&D is monotonically increasing in the number of independent competitors.

We also consider a continuous-time generalization of d’Aspremont and Jacquemin [25]. There are two distinguishing features of our analysis. First, we consider all possible initial marginal cost levels, including those exceeding the choke price (the lowest price for which there is no demand). Especially in the early stages of development, it is quite likely that the cost of a new technology (the cost, say, to develop a prototype) exceeds the highest willingness to pay in the market. We characterize situations where such initial technologies are only developed if firms collude in the product market. Indeed, excluding initial marginal costs that are above the choke price ignores “ ... new ... processes ... [that] failed to come into existence, [as they are] killed by the Sherman Act before they were born.” These instances constitute a direct welfare gain of product market collusion.

Second, in addition to near-equilibrium paths, we consider all trajectories that are candidates for an optimal solution. This global analysis yields a bifurcation diagram that indicates for every possible parameter combination the qualitative features of any market equilibrium as well as of the transient dynamics toward it. We thus identify critical parameter values: points in parameter space at which the optimal investment function changes qualitatively. In particular, we determine the value of marginal costs for which R&D investments are terminated, and for which they are not initiated at all. We prove that these critical cost levels are affected by firm conduct. Therefore, extending the R&D cooperative to product market collusion can lead to qualitatively different long-run solutions, in spite of starting from an identical initial technology.

The related literature [24, 27,28,29,30] has not considered initial marginal cost levels that exceed the choke price nor has it carried out a global analysis. In all these papers, any of the initial (permissible) technologies will be developed to full materialization; technologies that are only developed under specific regimes (i.e., product market collusion) remain hidden. The only exception is Hinloopen et al. [31], who characterize the equilibria of a continuous-time dynamic monopoly with R&D investments. We expand their analysis in three directions. First, we consider a duopoly rather than a monopoly. Second, we examine two different scenarios: one in which firms cooperate in R&D and compete in the product market (labeled “partial collusion”), and one in which firms cooperate both in R&D and in setting price (labeled “full collusion”). Indeed, comparing the two scenarios allows us to examine the effects of extending cooperation in R&D toward collusion in the product market. And third, rather than relying on numerical simulations, we prove a set of propositions that characterize the dynamics of the model throughout the entire parameter space.

Our framework yields four possible outcomes for any initial draw of a new technology (cf. [31]). First of all, a “Promising Technology” arrives, whereby the initial technology is developed through continuous R&D investments. This can occur for initial cost levels both below and above the choke price. In the latter case, production starts only after some time, because early R&D efforts have to bring down marginal cost below the choke price. Second, a “Strained Market” arises: initial marginal cost is below the choke price and firms invest in R&D, but the technology is not likely to be developed to full materialization. In case of an “Uncertain Future,” the third situation, it is not immediately clear whether the long-run steady state will be reached, or that it is optimal to gradually leave the market. Only time will tell. Fourth, an “Obsolete Technology” can emerge: whatever the initial marginal cost, the technology is either not developed or developed only to be taken off the market in due time. The long-run steady state will not be reached in either case.

All four technologies can emerge under both partial collusion and full collusion. Comparing the two scenarios throughout the entire parameter space, we find that if firms collude in the product market (i) it is more likely that an initial technology qualifies as a “Promising Technology,” and if so, that it is more likely to be developed further, (ii) it is less likely that an initial technology qualifies as an “Obsolete Technology,” and if so, it is more likely that firms invest in R&D, albeit temporarily, and (iii) if an initial technology causes a “Strained Market” or if it induces an “Uncertain Future,” it is less likely that it will be taken off the market in due time. Put differently, due to product market collusion it is more likely that firms invest in R&D, and that these investments eventually lead to a steady state with positive production.

Our analysis qualifies the per se prohibition of collusion in product markets for high-tech industries. A higher total surplus obtains if colluding firms develop an initial technology and arrive at the saddle-point steady state while firms that compete in the product market would not develop the technology at all. We show that this is more likely to happen if new technologies arrive in circumstances that offer a high profit potential (that is, large markets and efficient R&D processes). Under these circumstances, product market collusion can also yield higher total surplus if competing firms would develop the new technology as well, be it to take it off the market in due time, or to arrive at the saddle-point steady state. And in so far, higher R&D investments as such are desirable (as suggested in the endogenous growth literature; see, e.g., [32, 33]) the case for prohibiting collusion per se is further weakened. On the other hand, colluding firms tend to hold on longer to technologies that are destined to leave the market. This is not desirable from a social welfare point of view if that prevents the development of new, superior technologies.

A particularly difficult situation arises when the initial technology is above the choke price and if it will be developed only if firms collude in the product market. The welfare cost of prohibiting firms to collude then remains hidden because no production is affected by this prohibition. There is no production yet, and because collusion is prohibited, there will be no production in the future. Put differently, no production will be taken off the market if firms are prohibited to collude in the product market, leaving the welfare cost unnoticed. Our analysis thus offers a first glance at “new ... processes ... [that] failed to come into existence, killed by the Sherman Act before they were born.”

The remainder of the paper is organized as follows. The basics of the model are introduced in Sect. 2. In Sect. 3, the necessary conditions for optimal production and investment schedules are derived under partial collusion and full collusion. The corresponding bifurcation diagrams are derived in Sect. 4 and the two scenarios are compared in Sect. 5. Section 6 concludes. Appendices contain the proofs of all propositions.

2 The Model

Our present model is an extension of the global monopoly framework of Hinloopen et al. [31] to two firms, and it builds on Cellini and Lambertini [27]. Do note that Smrkolj and Wagener [34] show that the equilibrium considered in [27] is not subgame perfect. Time t is continuous: $t\in [0,\infty [$. There are two a priori fully symmetric firms that both produce a homogeneous good at constant marginal costs $c_i(t)$. At every instant, the market price p(t) is given as

$$\begin{aligned} p(t)=\bar{A}-Q(t), \end{aligned}$$

(1)

where $Q(t)=q_1(t)+q_2(t)$, with $q_i(t)$ the quantity produced by firm i at time t, and where $\bar{A}$ is the choke price.

Each firm i can reduce its marginal cost $c_i(t)$ by investing in R&D. In particular, when firm i exerts R&D effort $k_{i}(t)$, its marginal cost evolves as

$$\begin{aligned} \frac{{\hbox {d}}c_{i}}{{\hbox {d}}t}(t) \equiv \dot{c}_{i}(t) = c_{i}(t)\left( -k_{i}(t)-\bar{\beta } k_{j}(t)+\bar{\delta } \right) , \end{aligned}$$

(2)

where $k_{j}(t)$ is the R&D effort exerted by its rival and where $\bar{\beta } \in [0,1]$ measures the degree of spillover.

Note that efficiency of production is assumed to decrease at a constant rate, as captured by $\bar{\delta } >0$. This depreciation is due to (exogenous) aging of technology and organizational forgetting [28, 35]. As Benkard [36, p. 590] observes: “... an aircraft producer’s stock of production experience is constantly being eroded by turnover, lay offs and simple losses of proficiency at seldom repeated tasks. When producers cut back output, this erosion can even outpace learning, causing the stock of experience to decrease.” In our model, R&D investment yields know-how gains but the logic of the argument is the same. For instance, complementary inputs that are typically purchased also constitute a fraction of production cost. Incorporating these inputs becomes ever more costly due to their inherent evolution over time, especially for firms that are relatively sluggish in R&D, as R&D efforts also determine any firm’s “absorptive capacity” [37].

A non-positive depreciation rate yields trivial equilibria. Every initial technology will be developed in case $\bar{\delta }$ is negative, as there is an exogenous reduction in marginal cost over time. For $\bar{\delta }=0$, consider $\bar{\delta }$ to be marginally positive. In that case, the value of initial marginal cost that would make it optimal not to invest in R&D is far above the choke price because only an infinitesimally small investment in R&D is then needed to reduce marginal cost over time.

Both firms are endowed with a given identical initial technology $c_i(0)=c_j(0)=c_0$, which represents the state of the technology at the moment of the invention of the product. Per unit of time, the costs of R&D efforts are

$$\begin{aligned} \varGamma _{i}(k_i)=\bar{b} k_i^2, \end{aligned}$$

(3)

where $\bar{b}>0$ is inversely related to the cost-efficiency of the R&D process. The R&D process is thus assumed to exhibit decreasing returns to scale ([38]; see also the discussion in [31]). Both firms discount the future with the same constant rate $\bar{\rho } >0$. Either firm’s instantaneous profit therefore equals

$$\begin{aligned} \pi _i(q_i,Q,k_i,c_i)=(\bar{A}-Q-c_i)q_i-\bar{b} k_i^2, \end{aligned}$$

(4)

with corresponding total discounted profit

$$\begin{aligned} \varPi _i(q_i,Q,k_i,c_i)=\int _0^\infty \pi _i(q_i,Q,k_i,c_i) {\hbox {e}}^{-\bar{\rho }t}{\hbox {d}}t. \end{aligned}$$

(5)

The model has five parameters: $\bar{A}$, $\bar{\beta }$, $\bar{b}$, $\bar{\delta }$, and $\bar{\rho }$. To simplify the analysis, we rescale the model such that it has only three parameters. Rescaling is done by choosing “natural units” for the problem; it does not involve making special parameter choices. Rather, each choice of parameters in the original model corresponds to a choice of parameters in the rescaled model. The complexity reduction obtained by the scaling is a consequence of the fact that in the original parameters, many choices give rise to mathematically equivalent models. In mathematical terms, we embed the given five-parameter family of models in a six-parameter family. We then show that the scaling transformations we consider allow us to choose three parameter values to be equal to 1, effectively reducing the number of parameters to three.

Lemma 2.1

The following equations define new variables

$$\begin{aligned} t= & {} \frac{\tilde{t}}{\bar{\delta }}, \qquad c_i = \bar{A}\tilde{c}_i, \qquad q_i = \bar{A}\tilde{q}_i, \quad Q = \bar{A}\tilde{Q},\\ k_i= & {} \frac{\bar{A}}{\sqrt{\bar{b}}} \tilde{k}_i,\quad \pi _i = \bar{A}^2\tilde{\pi }_i,\quad \varPi _i = \frac{\bar{A}^2}{\bar{\delta }} \tilde{\varPi }_i, \end{aligned}$$

and new parameters $\phi = \bar{A}/(\bar{\delta }\sqrt{\bar{b}})$ and $\bar{\rho } = \bar{\delta } \tilde{\rho }$. In the new variables, the model takes the form:

$$\begin{aligned}&\tilde{\pi }_i(\tilde{q}_i,\tilde{Q},\tilde{k}_i,\tilde{c}_i) = (1-\tilde{Q}-\tilde{c}_i)\tilde{q}_i-\tilde{k}_i^2, \end{aligned}$$

(6)

$$\begin{aligned}&\tilde{\varPi }_i(\tilde{q}_i,\tilde{Q},\tilde{k}_i,\tilde{c}_i) = \int _0^\infty \tilde{\pi }_i(\tilde{q}_i,\tilde{Q},\tilde{k}_i,\tilde{c}_i) {\hbox {e}}^{-\tilde{\rho } \tilde{t}}d\tilde{t}, \end{aligned}$$

(7)

$$\begin{aligned}&\dot{\tilde{c}}_i = \tilde{c}_i\left( 1-\phi \left( \tilde{k}_i+\bar{\beta }\tilde{k}_j\right) \right) , \quad {\tilde{c}_{i}(0)}={\tilde{c}_{0}}, \end{aligned}$$

(8)

where $\tilde{c}_i\ge 0$, and with the control restrictions $\tilde{q}_i\ge 0$ and $\tilde{k}_i\ge 0$.

The proof of the lemma is given in “Appendix A.”

Remark 2.1

Rescaling the model as in Lemma 2.1 introduces a new parameter: $\phi$. It is one-to-one related to the profit potential of a technology. Higher potential revenues come with a higher $\bar{A}$, and each unit of R&D effort costs more if $\bar{b}$ increases, while it reduces marginal cost by less the higher is $\bar{\delta }$. In sum, a lower (higher) $\phi$ corresponds to a lower (higher) profit potential.

Remark 2.2

In mathematical terms, the original model is a specimen of the six-parameter model given by

$$\begin{aligned} \dot{c}_i = c_i\left( \delta - \phi (k_i + \beta k_j)\right) , \quad \varPi _i = \int _0^\infty \left( (A-Q-c_i)q_i-bk_i^2\right) {\hbox {e}}^{-\rho t}{\text {d}}{t}, \end{aligned}$$

with parameters values

$$\begin{aligned} (A,b, \delta ,\beta ,\phi ,\rho )=(\bar{A},\bar{b},\bar{\delta }, \bar{\beta },1,\bar{\rho }). \end{aligned}$$

The—equivalent—model in new variables is an instance of the same six-parameter model, but with parameters values

$$\begin{aligned} (A,b,\delta ,\beta ,\phi ,\rho ) = \left( 1,1,1,\bar{\beta }, \frac{\bar{A}}{\bar{\delta }\sqrt{\bar{b}}}, \frac{\bar{\rho }}{\bar{\delta }}\right) . \end{aligned}$$

We can, and will, without loss of generality drop the tildes from the “new” variables, the bars from the parameters, and take $A=1$, $b=1$ and $\delta =1$.

To illustrate the usefulness of Lemma 2.1, consider two models with different original parameterizations: (i) $\bar{A}=10$, $\bar{b}=1$, $\bar{\delta }=0.2$, $\bar{\rho }=0.1$, $\bar{\beta }=0.5$, (ii) $\bar{A}=20$, $\bar{b}=4$, $\bar{\delta }=0.2$, $\bar{\rho }=0.1$, $\bar{\beta }=0.5$. Both models correspond to the same rescaled model with $\phi =\bar{A}/\bar{\delta }\sqrt{\bar{b}}=50$, $\tilde{\rho }=\bar{\rho }/\bar{\delta }=0.5$, $\bar{\beta }=0.5$ and are therefore mathematically equivalent in the sense that they have the same solution in rescaled variables.

3 Partial Collusion and Full Collusion

In this section, we derive the necessary conditions for optimal production and investment schedules in case firms cooperate in R&D but compete in the product market (Sect. 3.1), and in case firms cooperate in R&D and collude in the product market (Sect. 3.2).

3.1 Partial Collusion

Both firms operate their own R&D laboratory and production facility. They select their output levels non-cooperatively and adopt a strictly cooperative behavior in determining their R&D efforts so as to maximize joint profits. These assumptions amount to imposing a priori the symmetry condition $k_i(t)=k_j(t)=k(t)$. $c_i(0)=c_j(0)=c_0$ implies that $c_i(t)=c_j(t)=c(t)$. Equation (8) thus reads as

$$\begin{aligned} \dot{c}=c(1-(1+\beta )\phi k). \end{aligned}$$

(9)

It may seem reasonable to assume that when firms cooperate in R&D, they also fully share information, that is, to assume the level of spillover to be at its maximum $(\beta =1$; see [39]). For the sake of generality, we do not a priori fix the value of $\beta$ at its maximal value. There are also intuitive arguments for not doing so as there might still be some ex post duplication and/or substitutability in R&D outputs if firms operate separate laboratories (see the discussion in [40]).

The instantaneous profit of firm i is

$$\begin{aligned} \pi _i(q_i,Q,k,c)= (1-Q-c)q_i-k^2, \end{aligned}$$

(10)

with $Q=q_1+q_2$, yielding its total discounted profit over time

$$\begin{aligned} \varPi _i(q_i,Q,k,c)=\int _0^\infty \pi _i(q_i,Q,k,c){\hbox {e}}^{-\rho t}{\hbox {d}}t. \end{aligned}$$

(11)

As firms jointly decide on their R&D efforts, the only independent decisions are those of production. However, as quantity variables do not appear in the equation for the state variable (9), production feedback strategies of a dynamic game are simply static Cournot–Nash strategies of each corresponding instantaneous game.

Maximizing $\pi _i$ over $q_i\ge 0$ gives us standard Cournot best-response functions for the product market

$$\begin{aligned} q_i(q_j) = {\left\{ \begin{array}{ll} \frac{1}{2}(1-c-q_j),\quad &{} \text {if } q_j<1-c, \\ 0, &{} \text {if } q_j\ge 1-c. \end{array}\right. } \end{aligned}$$

(12)

Note that the constraint $q_i\ge 0$ is binding when $q_j\ge 1-c$. Solving for Cournot–Nash production levels, we obtain

$$\begin{aligned} q^N={\left\{ \begin{array}{ll} \frac{1}{3}(1-c),\quad &{} \text {if } c<1, \\ 0, &{} \text {if } c\ge 1. \end{array}\right. } \end{aligned}$$

(13)

Consequently, the instantaneous profit of each firm is

$$\begin{aligned} \pi (c,k)={\left\{ \begin{array}{ll} \frac{1}{9}(1-c)^2-k^2, &{} \text {if } c<1,\\ -k^2, &{} \text {if } c\ge 1. \end{array}\right. } \end{aligned}$$

(14)

We assume that firms face no financial constraints; they can invest in R&D prior to production. Indeed, credit rationing would impose an upper limit on the value of an indifference point; qualitatively it would not change our conclusions, however. Also, for a sample of Italian manufacturing firms, Piga and Atzeni [41] find that credit constraints are negligible for R&D intensive firms. And Bond et al. [42] find no significant relationship between the level of R&D investments and cash flow for German and UK firms, while Harhoff [43] finds a weak but statistically significant relationship for both small and large German firms. The sensitivity of R&D investments to cash flow fluctuations seems to be stronger for US firms (e.g., [44, 45]), but by and large, the literature on the importance of financial constraints for R&D investment is inconclusive (see [46] for an overview).

The dynamic optimization problem of the R&D cooperative boils down to finding an R&D effort schedule $k^*$ for either firm that maximizes the total discounted joint profit, taking into account the state Eq. (9), the initial condition $c(0)=c_0$, and the control constraint $k(t)\ge 0$ which must hold at all times. Note that according to (9), if $c_0>0$, then $c(t)>0$ for all t. The natural state space of this problem would be the interval $]0,\infty [$ of positive marginal cost levels, but for mathematical convenience, we extend this to ${\mathbb {R}}$ by specifying that $\pi _i(q_i,Q,k,c)=0$ if $c<0$.

In order to close the model, we have to specify the set of admissible effort schedules k(t).

Definition 3.1

An R&D effort schedule is admissible if it is a bounded nonnegative measurable function.

To solve this problem, we introduce the current-value Pontryagin function (also called the un-maximized Hamilton or pre-Hamilton function), whereby we omit a factor 2 for joint profits to obtain the solution expressed in per-firm values (due to symmetry, maximizing per-firm total profit corresponds to maximizing joint total profit)

$$\begin{aligned} P(c, k, \lambda )={\left\{ \begin{array}{ll} \frac{1}{9}(1-c)^2-k^2 +\lambda c(1-(1+\beta )\phi k),\quad &{} \text {if } c<1, \\ -k^2 +\lambda c(1-(1+\beta )\phi k), &{} \text {if } c\ge 1, \end{array}\right. } \end{aligned}$$

(15)

where $\lambda$ is the current-value costate variable of a firm in the R&D cooperative. The costate (or shadow value) measures the marginal worth of the increment in the state c for each firm at time t when moving along the optimal path. As we expect an increase of the marginal costs to entail lower profits for the firm, we expect the shadow value to be non-positive—that is $\lambda (t)\le 0$—along optimal trajectories.

We use Pontryagin’s maximum principle to obtain the solution to our optimization problem. Maximizing over the control $k\ge 0$ yields

$$\begin{aligned} k = \max \left\{ 0,- \frac{1}{2}\lambda c (1+\beta )\phi \right\} . \end{aligned}$$

(16)

The maximum principle states further that the optimizing trajectory necessarily corresponds to the trajectory of the state–costate system

$$\begin{aligned} \dot{c} = \frac{\partial P }{\partial \lambda },\quad \dot{\lambda } = \rho \lambda - \frac{\partial P }{\partial c}, \end{aligned}$$

(17)

where k is replaced by its maximizing value. For $\lambda \le 0$, relation (16) gives a one-to-one correspondence between the costate $\lambda$ and the control k. We use this relation to transform the state-costate system into a state-control system which an optimizing trajectory has to satisfy necessarily as well. This system consists of two regimes (following the two part composition of the Pontryagin function). The first one corresponds to $c<1$ and positive production ($q=(1-c)/3)$. The second one corresponds to $c\ge 1$ and zero production. Note that in the non-rescaled model, the analogous conditions for positive and zero production are $c(t)<\bar{A}$ and $c(t)\ge \bar{A}$, respectively. The state-control system with positive production consists of the following two differential equations:

$$\begin{aligned} \dot{c}=c\left( 1-(1+\beta )\phi k\right) ,\quad \dot{k}=\rho k -\frac{(1+\beta )\phi }{9}c(1-c). \end{aligned}$$

(18)

The state-control system with zero production is given by

$$\begin{aligned} \dot{c}=c\left( 1-(1+\beta )\phi k\right) ,\quad \dot{k}=\rho k. \end{aligned}$$

(19)

3.2 Full Collusion

Under full collusion, firms determine jointly their R&D efforts and their output levels. This amounts to imposing a priori the symmetry conditions $k_i(t)=k_j(t)=k(t)$ and $q_i(t)=q_j(t)=q(t)$. Equation (8) reads again as Eq. (9). The profit of each firm at every instant is

$$\begin{aligned} \pi (q,k,c)= (1-2q-c)q-k^2, \end{aligned}$$

(20)

with corresponding total discounted profit

$$\begin{aligned} \varPi (q,k,c)=\int _0^\infty \pi (q,k,c){\hbox {e}}^{-\rho t}{\hbox {d}}t. \end{aligned}$$

(21)

The optimal control problem of the two colluding firms is to find controls $q^*$ and $k^*$ that maximize the profit functional $\varPi$ subject to the state Eq. (9), the initial condition ${c(0)}=c_0$, and two control constraints that must hold at all times: $q\ge 0$ and $k\ge 0$. Notice, again, that according to (9), if $c_0>0$, then $c(t)>0$ for all t.

The current-value Pontryagin function in case of full collusion reads as:

$$\begin{aligned} P(c, q, k, \lambda ) = \left( 1-2q-c\right) q-k^2+\lambda c\left( 1-(1+\beta )\phi k\right) , \end{aligned}$$

(22)

where $\lambda$ is the current-value costate variable. It now measures the marginal worth at time t of an increment in the state c for a colluding firm when moving along the optimal path.

The necessary conditions for the solution to the dynamic optimization problem consist again of a state-control system which has two regimes. As in the partial collusion case, the first regime corresponds to $c<1$ and positive production ($q=(1-c)/4$), while the second regime corresponds to $c\ge 1$ and zero production.

The state-control system in the region with positive production reads as

$$\begin{aligned} \dot{c}=c\left( 1-(1+\beta )\phi k\right) ,\quad \dot{k}=\rho k -\frac{(1+\beta )\phi }{8}c(1-c), \end{aligned}$$

(23)

whereas the state-control system with zero production is

$$\begin{aligned} \dot{c}=c\left( 1-(1+\beta )\phi k\right) ,\quad \dot{k}=\rho k. \end{aligned}$$

(24)

4 Analysis

Consider the system

$$\begin{aligned} \dot{c} = c\left( 1-(1+\beta )\phi k\right) ,\quad \dot{k} = \rho k - \alpha \phi (1+\beta ) c(1-c) \chi (c), \end{aligned}$$

(25)

where $\chi (c)=1$ if $0<c<1$ and $\chi (c)=0$ if $c \ge 1$ (or $c\le 0$). Systems (18)–(19) and (23)–(24) are instances of system (25), with $\alpha =1/9$ for the partial collusion scenario and $\alpha =1/8$ for the full collusion scenario. Indeed, the monopoly system in Hinloopen et al. [31] is also a special case of system (25), with $\alpha =1/4$.

The first result gives the properties of the steady states of the state-control system (see “Appendix B” for the proof).

Proposition 4.1

Let

$$\begin{aligned} D=\frac{1}{4} - \frac{\rho }{\alpha (1+\beta )^2\phi ^2}. \end{aligned}$$

Depending on the value of D, there are three different situations.

• A. If $D>0$, the state-control system with positive production (23) has three steady states:

  • i. $(c^{e},k^{e})=(0,0)$ is an unstable node,

  • ii. $(c^{e},k^{e}) =\left( \frac{1}{2}+\sqrt{D},\frac{1}{(1+\beta )\phi }\right)$ is either an unstable node or an unstable focus, and

  • iii. $(c^{e},k^{e})=\left( \frac{1}{2}-\sqrt{D},\frac{1}{(1+\beta )\phi }\right)$ is a saddle-point steady state.

• B. At $D=0$, there are two steady states:

  • i. $(c^e,k^e)=(0,0)$, which is an unstable node, and

  • ii. $(c^e,k^e)=\left( \frac{1}{2},\frac{1}{(1+\beta )\phi }\right)$, which is a semi-stable steady state.

• C. If $D<0$, the origin $(c^e,k^e)=(0,0)$ is the unique steady state of the state-control system with positive production, which is unstable.

The system consequently exhibits a saddle-node bifurcation at $D=0$.

Remark 4.1

The stable manifold of the saddle-point steady state is one of the candidates for an optimal solution. However, as neither the Mangasarian nor the Arrow concavity conditions are satisfied, the stable manifold is not necessarily optimal. Proposition 4.1 already implies that there should be other candidates for optimality as there is a parameter region for which there is no saddle point and hence no stable manifold to it.

The following result clarifies (“Appendix C” contains the proof).

Proposition 4.2

The set of candidates for an optimal solution consists of the stable paths $W^s_-$ of the saddle-point steady state and the trajectory E through the point $(c, k)=(1,0)$.

Proposition 4.2 is illustrated in Fig. 1. The thick black lines $W^s_{-}$ and E indicate optimal solutions. The dotted vertical line $c=1$ separates the region with zero production from the region with positive production. We label the trajectory E the “exit trajectory”, as following this trajectory implies that firms eventually leave the region with positive production.

Fig. 1

Candidate maximizing trajectories $W^s_-$ and E in the state-control space

Proposition 4.2 only reduces the set of trajectories by applying necessary conditions for optimality, but there is no guarantee that an optimal solution exists. The next proposition summarizes when an optimal solution exists.

Proposition 4.3

For all admissible values of the parameters, and all initial points, the optimal control problem has at least one solution, which is among the candidates specified in Proposition 4.2. Moreover, there is at most one initial state $\hat{c}$ such that there are two optimizing trajectories starting at $\hat{c}$.

The proof is in “Appendix D.”

To assess the dependence of the solution structure on the model parameters, we carry out a bifurcation analysis. This consists of identifying those parameter values for which the qualitative structure of the optimal dynamics changes. These “bifurcating” values bound open parameter regions such that the optimal dynamics are qualitatively identical for all parameter values in a region (see [47, 48]). Put differently, for all points in a region, a sufficiently small change in parameter values will not lead to a qualitative change of the optimal dynamics; regions characterize stable types of dynamics.

System (25) has four distinct stable dynamics types (cf. [31]). These are illustrated in Fig. 2 in case of partial collusion. Note that the same types emerge under full collusion (the stable dynamics types are compared across scenarios in Sect. 5). The first type is a “Promising Technology.” In this case, there exists an initial technology $\hat{c} >1$ that is an indifference threshold: a point in state space where the decision maker is indifferent between two optimal trajectories that have distinct long-term limit behavior. In particular, for $0<c_0\le \hat{c}$ it is optimal to start developing the initial technology, ending up in the saddle-point steady state in the region with positive production. If $1<c_0\le \hat{c}$, initially firms invest only in R&D; production begins whenever $c(t)<1$. If $c_0\ge \hat{c}$, it is optimal not to initiate R&D efforts; potential future profits do not suffice to compensate for losses that would be incurred in the initial periods during which firms would invest in R&D but would not produce yet (note that for $c_0=\hat{c}$, there are two distinct R&D investment trajectories, which are, nevertheless, both optimal; see also Proposition 4.3).

Fig. 2

R&D investment trajectories for the four stable dynamics types of system (25) with $\alpha = 1/9$ (partial collusion). Parameters: a $(\beta , \rho , \phi ) = (1, 0.5, 4)$, b $(\beta , \rho , \phi ) = (1, 1, 3.5)$, c $(\beta , \rho , \phi ) = (1, 4, 6.1)$, d $(\beta , \rho , \phi ) = (1, 1, 2.5)$. a Promising Technology, b Strained Market, c Uncertain Future, d Obsolete Technology

The second type corresponds to a “Strained Market,” where there is an indifference threshold below the choke price (that is, in the region with positive production): $0<\hat{c} < 1$. In this case, if $0< c_0 < \hat{c}$, the initial technology will be developed toward the saddle-point steady state. If $\hat{c}< c_0<1$, the exit trajectory applies; R&D investments only serve to slow down the technological decay.

In a small part of the parameter space, the third type arises: an “Uncertain Future.” Initial technologies (states) are now divided by a repelling steady state (rather than an indifference point). If the system starts exactly at the repelling point $c_R$, it stays there indefinitely; when it starts close to it, it stays there for a long period of time, after which it converges to one of the two attractors: the steady state or the exit trajectory.

The fourth type typifies the dynamics of an “Obsolete Technology.” Whatever the initial technology, (eventually) the firms leave the market; R&D investments are only used to slow down the technical decay.

The four different dynamics types are grouped conveniently in a bifurcation diagram (see Fig. 3): the graph that indicates for every possible parameter combination the qualitative features of any market equilibrium as well as the transient dynamics toward them. In Fig. 3, the uppermost curve represents parameter values for which the indifference point is exactly at $c=1$. At the saddle-node curve (SN), an optimal repeller and an optimal attractor collide and disappear. The curve SN’ corresponds to saddle-node bifurcations in the state-control system that do not correspond to optimal dynamics. At the indifference-attractor bifurcations (IA), an indifference point collides with an optimal attractor and both disappear. Finally, at an indifference-repeller bifurcation (IR), an indifference point turns into an optimal repeller. The central indifference-saddle-node (ISN) bifurcation point at $(\rho ,\phi (1+\beta ))\approx (2.14,8.78)$ organizes the bifurcation diagram. The curve representing indifference points at $c=1$ obtains a value of $\phi (1+\beta )\approx 2.998$ for $\rho =1\times 10^{-5}$.

Fig. 3

Bifurcation diagram (partial collusion)

5 Collusion and the Incentives to Innovate

In this section, we compare the global optimum of the two scenarios. For a welfare comparison, we introduce total discounted values of profits ($\varPi$), consumer surplus (CS), and total surplus (TS):

$$\begin{aligned} \varPi= & {} \int _{0}^{\infty }\pi (t){\hbox {e}}^{-\rho t}{\hbox {d}}t, \end{aligned}$$

(26)

$$\begin{aligned} {\hbox {CS}}= & {} \int _{0}^{\infty }\frac{1}{2}(1-p(t))Q(t){\hbox {e}}^{-\rho t}{\hbox {d}}t=\int _{0}^{\infty }2q(t)^2{\hbox {e}}^{-\rho t}{\hbox {d}}t, \end{aligned}$$

(27)

$$\begin{aligned} {\hbox {TS}}= & {} 2\varPi +{\hbox {CS}}, \end{aligned}$$

(28)

where at time $t=0$ firms start with $c_0$ and then invest along the optimal trajectory $\gamma (t)=(c(t),k(t))$ as $t\rightarrow \infty$.

We first formally establish that the two scenarios yield different (optimal) trajectories. In Fig. 4, the bifurcation diagrams of both scenarios are superimposed. There are significant quantitative differences between the two diagrams, as reflected by the different locations of the curves that divide the parameter space. Let $I_i, II_i,\ldots , i=1,2$ denote regions I, $II, \ldots$ under scenario i, with $i=1$ (2) corresponding to partial (full) collusion. The following then holds (see “Appendix E” for the proof).

Fig. 4

Superimposed bifurcation diagrams. Partial collusion curves are gray, full collusion ones are black

Proposition 5.1

The following inclusions hold:

$$\begin{aligned} I_1 \subset I_2,\quad I_1 \cup II_1 \subset I_2 \cup II_2, \quad I_1 \cup II_1 \cup III_1 \subset I_2 \cup II_2 \cup III_2. \end{aligned}$$

The first inclusion of Proposition 5.1 implies that the “Promising Technology” region is larger if firms collude in the product market; due to collusion, the situation where firms first invest in R&D, and only after some initial development period start producing, is more likely to occur. From the third inclusion follows that the “Obsolete Technology” region is smaller if firms collude; firms that collude are less likely either not to develop an initial technology or to invest in R&D only to abandon the technology in time.

5.1 R&D Investment Incentives

In line with much of the related literature [9], Proposition 5.1 suggests that colluding firms have in general a stronger incentive to invest in R&D. This turns out to be the case, as the next proposition formally shows (see “Appendix F.1” for the proof).

Proposition 5.2

Investment in R&D in the full collusion scenario is always at least as high as in the corresponding partial collusion scenario.

Proposition 5.2 implies the following. First of all, whenever both scenarios lead to the saddle-point steady state, marginal costs in the full collusion scenario are lower than in case of partial collusion, because fully colluding firms have invested more in cost-reducing R&D to arrive at the long-run equilibrium. Put differently, product market collusion yields a higher production efficiency.

Second, if the initial technology leads to production after some initial development period only, colluding firms will enter this production phase more quickly because at every instant of the pre-production phase they invest more in R&D in order to bring the level of marginal costs below the choke price.

Third, firms that collude in the product market abandon obsolete technologies at a lower pace. This implication has a similar vein as the argument of Arrow [49], that a monopolist has less incentive to invest in R&D than an otherwise identical but perfectly competitive market, because by doing so the monopolist replaces current monopoly profits by future (higher) monopoly profits. Here, the alternative for colluding firms is to exit the market more quickly (rather than staying in the market as a monopolist, as in Arrow [49]), an alternative that for them is not optimal (see Fig. 5).

Fig. 5

State-control space (a), total discounted profit (b), consumer surplus (c), and total surplus (d), when the exit trajectory is an optimal solution. Parameters: $(\beta , \rho , \phi )=(1, 1, 2)$. Curves of the partial (full) collusion scenario are gray (black)

The difference in R&D intensity across the two scenarios is also reflected in the type of trajectories that firms select. To characterize this difference for all possible situations, it is convenient to have defined the threshold level of initial marginal cost $\hat{c}$ between “eventual exit” and “eventual positive production.” Formally, set $\hat{c} = 0$ in the “Obsolete Technology” region and let $\hat{c}_1$ and $\hat{c}_2$ denote the threshold level for the partial collusion and the full collusion scenarios, respectively. We can then state the following (see “Appendix F.2” for the proof).

Proposition 5.3

For all parameter values, either $\hat{c}_1 < \hat{c}_2$ or $\hat{c}_1=\hat{c}_2=0$.

The implications of Proposition 5.3 are twofold. First, if firms collude in the product market, the set of initial technologies that are developed toward the saddle-point steady state is larger (see Fig. 6). In particular, if the initial technology $c_0$ falls in the non-empty interval $]\hat{c}_1,\hat{c}_2 [$, it will only be brought to full materialization if firms collude in the product market.

Fig. 6

State-control space (a), total discounted profit (b), consumer surplus (c), and total surplus (d), when the indifference point is in the region with zero production. Parameters: $(\beta , \rho , \phi )=(1, 0.1, 2.25)$. Curves of the partial (full) collusion scenario are gray (black)

Second, the set of initial technologies that triggers no investment in R&D at all or that induces firms to select the exit trajectory is smaller if firms collude in the product market. Figure 7 illustrates this for a “Strained Market.” The strained investment circumstances induce partially colluding firms to exit the market in due time for all $c_0>\hat{c}_1$. In contrast, fully colluding firms exit the market only for $c_0>\hat{c}_2$. Initial technologies $c_0$ in the interval $]\hat{c}_1, \hat{c}_2 [$ are therefore only brought to full maturation by firms that collude in the product market.

Fig. 7

State-control space (a), total discounted profit (b), consumer surplus (c), and total surplus (d), when the indifference point is within the region with positive production. Parameters: $(\beta , \rho , \phi )=(1, 0.1, 2)$. Curves of the partial (full) collusion scenario are gray (black); curves of the stable path (exit trajectory) are solid (dotted). Dots indicate the saddle-point steady state

We can conclude that due to collusion in the product market (i) it is more likely that an initial technology qualifies as a “Promising Technology,” and if so, that it is more likely to be developed further, (ii) it is less likely that an initial technology qualifies as an “Obsolete Technology,” and if so, it is more likely that firms invest in R&D, albeit temporarily, and (iii) if an initial technology causes a “Strained Market” or if it induces an “Uncertain Future,” it is less likely that it will be taken off the market in due time. Put differently, due to product market collusion it is more likely that firms invest in R&D, and that these investments eventually lead to a steady state with positive production.

5.2 Total Surplus

We next consider the effect of product market collusion on total surplus. Obviously, collusion in the product market yields higher total surplus if colluding firms develop an initial technology and arrive at the saddle-point steady state while firms that compete in the product market would not develop the technology at all. Formally:

Proposition 5.4

Whenever both scenarios have an indifference point above the choke price, the full collusion scenario yields higher consumer surplus and total surplus than the partial collusion scenario for all initial technologies in-between the two indifference points.

Proof

The proof follows trivially from the fact that i) for all values of c above the indifference point in the region where $c\ge 1$, both $q=0$ and $k=0$ for all $t\in [0,\infty [$, and ii) for all values of c below the indifference point, $\varPi >0$ and for some finite time T also $q>0$ for all $t > T$ as $t\rightarrow \infty$. $\square$

Figure 6 illustrates Proposition 5.4: for all $c_0$ $\in$ $]\hat{c}_1, \hat{c}_2[$, collusion in the product market yields a higher total surplus. Figure 8 illustrates some comparative statics of the indifference points in this case. Indeed, these points are positively related to market size and R&D efficiency. Note, however, that also $\Delta \hat{c}=\hat{c}_2-\hat{c}_1$ increases if the R&D process becomes more efficient and/or if the market size becomes larger, the more so the lower is the discount rate (in Fig. 8, a lower discount rate corresponds to a larger slope of the convex curves). Because future mark-ups are positively related to both market size and R&D efficiency, an increase in either of these two has a larger (positive) effect on future profits if firms collude in the product market. And these future benefits feature more prominently in total discounted profits if the discount rate is lower. Therefore, indifference points correspond to lower marginal cost values if the discount rate goes up, all else equal (cf. the relative location of $C_1$ and $C_2$ in Fig. 8).

Fig. 8

Dependence of the indifference point $\hat{c}$ on model parameters. Curves of the partial (full) collusion scenario are dotted (solid)

Product market collusion can also yield higher total surplus if colluding firms arrive at the saddle-point steady state while firms that compete in the product market would select the exit trajectory. In these cases, firms that compete in the product market temporarily produce more. This is offset by the added benefits of sustained R&D investments under full collusion if the discount rate is sufficiently small (see Fig. 7).

Finally, collusion in the product market can also yield a higher total surplus if under both scenarios firms would select the trajectory toward the saddle-point steady state: in Fig. 9, for all $c_0 \in ]c^\star , \hat{c}_2 [$, total surplus is higher if firms collude in the product market. In this example, the discount rate is high: $\rho =10$, which corresponds, for instance, to the non-rescaled variables $\bar{\delta }=0.01$ and $\bar{\rho }=0.1$. Also, the initial marginal costs are sufficiently high. In such an environment, the higher R&D investments and the reduced importance that is attached to future surplus work in favor of product market collusion as under this scenario firms will reach the production stage more quickly, a benefit that more than offsets the welfare loss of future increased mark-ups. That is, a higher (rescaled) discount rate $\tilde{\rho }=\bar{\rho }/\bar{\delta }$ implies either a higher discount rate $\bar{\rho }$ or a lower $\bar{\delta }$. With a lower $\bar{\delta }$, cost reductions take longer, such that the time difference in reaching the production stage between the two scenarios becomes more pronounced.

Fig. 9

Total surplus when the indifference point is in the region with zero production. Parameters: $(\beta , \rho , \phi )=(1, 10, 50)$. Gray curves correspond to partial collusion, whereas the black ones correspond to full collusion. $c^\star \approx 3.6$, $\hat{c}_1\approx 4.01$, $\hat{c}_2\approx 4.74$. For all $c_0 \in ]c^\star , \hat{c}_2 [$, total surplus is higher if firms collude in the product market

6 Conclusions

Schumpeter’s famous observation continues to challenge the design of optimal competition policies for high-tech sectors. The classic rationale for competition policies is rooted in their effect on total surplus. Typically, product market collusion transfers consumer surplus to firm profits, resulting in a net loss of total surplus. To date, the literature considers this result to be robust to the increased incentive to invest in R&D that comes with collusion in the product market. Our analysis shows that it actually fails this robustness check if the phase of development prior to production is taken into account and/or if all possible R&D investment trajectories are considered.

According to our analysis, extending an R&D cooperative agreement to collusion in the product market is welfare enhancing if the market size is large and/or the R&D process is efficient, given a relatively modest discount rate. The profit potential of a new technology is then relatively large. As a result, firms that collude in the product market bring more initial technologies to full materialization.

A particularly disturbing situation arises when the initial draw $c_0$ out of $]\hat{c}_1, \hat{c}_2[$ is above the choke price ($c_0>1$). The welfare cost of prohibiting firms to collude in the product market then remains hidden because no production is affected by this prohibition. There is no production yet, and because collusion is prohibited, there will be no production in the future. Put differently, no production will be taken off the market if firms are prohibited to collude in the product market.

Our analysis thus signals a potential problem for antitrust policy as it shows that prohibiting collusion in the product market per se is not univocally welfare enhancing. It also shows that the associated welfare costs might not surface. Further research is needed to substantiate our qualification of prohibiting collusion per se, including the development of richer models that allow for learning by doing, stochastic R&D, and asymmetries between firms.

Appendices

Appendix A: Proof of Lemma 2.1

We shall refer to the original variables t, $q_i$, $\ldots$ as the “old” variables, and to the variables $\tilde{t}$, $\tilde{q}_i$, $\ldots$ as the “new” variables.

In the new variables, the left- and right-hand sides of equation (2) take the form

$$\begin{aligned} \frac{{\text {d}}c_i}{{\text {d}}t}&= \bar{A}\bar{\delta } \frac{{\text {d}}\tilde{c}_i}{{\text {d}}\tilde{t}}, \quad c_i(-k_i - \bar{\beta } k_j + \bar{\delta }) = \bar{A} \tilde{c}_i\left( - \frac{\bar{A}}{\sqrt{\bar{b}}}\tilde{k}_i - \bar{\beta } \frac{\bar{A}}{\sqrt{\bar{b}}}\tilde{k}_j + \bar{\delta }\right) . \end{aligned}$$

Equation (2) then simplifies to

$$\begin{aligned} \frac{{\text {d}}\tilde{c}_i}{{\text {d}}\tilde{t}}&= \tilde{c}_i \left( 1 - \frac{\bar{A}}{\bar{\delta }\sqrt{\bar{b}}}\left( \tilde{k}_i + \bar{\beta }\tilde{k}_j\right) \right) = \tilde{c}_i \left( 1 - \phi \left( \tilde{k}_i + \bar{\beta }\tilde{k}_j\right) \right) , \end{aligned}$$

as claimed in the lemma.

Writing the total discounted profit in the new variables yields

$$\begin{aligned} \varPi _i&= \int _0^\infty \left( (\bar{A} - Q - c_i)q_i - \bar{b} k_i^2\right) {\hbox {e}}^{-\bar{\rho } t}{\text {d}}t \\&= \int _0^\infty \left( (\bar{A} - \bar{A}\tilde{Q} - \bar{A}\tilde{c}_i)\bar{A} \tilde{q}_i - \bar{b} \left( \frac{\bar{A}}{\sqrt{\bar{b}}}\right) ^2 \tilde{k}_i^2\right) {\hbox {e}}^{-\bar{\delta } \tilde{\rho } (\tilde{t}/\bar{\delta })} \frac{1}{\bar{\delta }}{\text {d}}\tilde{t} \\&= \frac{\bar{A}^2}{\bar{\delta }} \int _0^\infty \left( (1 - \tilde{Q} - \tilde{c}_i)\tilde{q}_i - \tilde{k}_i^2\right) {\hbox {e}}^{-\tilde{\rho } \tilde{t}} {\text {d}}\tilde{t} = \frac{\bar{A}^2}{\bar{\delta }} \tilde{\varPi }_i. \end{aligned}$$

$\square$

Appendix B: Proof of Proposition 4.1

Second rescaling of the problem Recall the dynamic optimization problem: to maximize

$$\begin{aligned} \varPi = \int _0^\infty \left( \alpha (1-c)^2 \chi (c) - k^2 \right) {\hbox {e}}^{-\rho t} {\text {d}}t, \end{aligned}$$

subject to the dynamic restriction $\dot{c} = (1 - \phi (1+\beta ) k) c.$ This problem is rewritten by introducing constants

$$\begin{aligned} K = \frac{1}{\phi (1+\beta )} \quad \text {and}\quad \mu = \frac{\alpha \phi ^2(1+\beta )^2}{4\rho }, \end{aligned}$$

(29)

and the variable $u = k/K$. It is then seen to be equivalent to the problem to maximize

$$\begin{aligned} V = \frac{\varPi }{K^2} = \int _0^\infty \left( 4\rho \mu (1-c)^2 \chi (c) - u^2 \right) {\hbox {e}}^{-\rho t} {\text {d}}t, \end{aligned}$$

(30)

subject to the dynamic restriction

$$\begin{aligned} \dot{c} = \left( 1 - u\right) c \end{aligned}$$

(31)

and the control restriction $u \ge 0.$ The Pontryagin function of this problem is

$$\begin{aligned} P = 4 \rho \mu (1-c)^2 \chi (c) - u^2 + \lambda c(1-u), \end{aligned}$$

which is maximized at

$$\begin{aligned} u = \max \left\{ 0,- \frac{c}{2}\lambda \right\} . \end{aligned}$$

(32)

This yields the Hamilton function

$$\begin{aligned} H = 4 \rho \mu (1-c)^2 \chi (c) + \lambda c + {\left\{ \begin{array}{ll} \frac{(\lambda c)^2}{4}, \quad &{}\text {if }\lambda \le 0, \\ 0, &{} \text {if }\lambda > 0. \end{array}\right. } \end{aligned}$$

(33)

If $\lambda \le 0$, the associated state-costate equations read as

$$\begin{aligned} \dot{c}&= H_\lambda = \frac{\lambda c^2}{2} + c, \quad \dot{\lambda } = \rho \lambda - H_c = \rho \lambda + 8 \rho \mu (1-c)\chi (c) - \frac{\lambda ^2}{2} c -\lambda , \end{aligned}$$

(34)

whereas if $\lambda >0$, they simplify to

$$\begin{aligned} \dot{c} = c, \quad \dot{\lambda } = (\rho - 1)\lambda + 8 \rho \mu (1-c)\chi (c). \end{aligned}$$

(35)

For $\lambda \le 0$, relation (32) defines a variable transformation that puts the system into state-control form

$$\begin{aligned} \dot{c}&= F_1(c,u) = c\left( 1-u\right) , \quad \dot{u} = F_2(c,u) = \rho \left( u - 4 \mu c(1-c) \chi (c)\right) . \end{aligned}$$

(36)

Note that this system is only valid for $u \ge 0$, as for $\lambda >0$, the relation between u and $\lambda$ fails to be one-to-one. For later use, we note that in (c, u) variables, the Hamilton function takes the form

$$\begin{aligned} H_\text {control}(c,u)&= 4\rho \mu (1-c)^2\chi (c) + u^2 - 2u. \end{aligned}$$

(37)

1.1 B.1 Steady States

To determine the steady states of the state-control system (36), we solve the equations $\dot{c} = 0$, $\dot{u} =0$. It is immediate that this system has no solutions if $c>1$.

If $0\le c\le 1$, the equation $\dot{c}=0$ is satisfied if $c=0$ or $u=1$. Substitution into $\dot{u}=0$ of the former yields the steady state $(c,u)=(0,0)$. Substitution of the latter leads to the quadratic equation $c^2 - c + 1/(4\mu ) = 0,$ which can be written as $\left( c-\frac{1}{2}\right) ^2 - D = 0$ with

$$\begin{aligned} D = \frac{1}{4}\left( 1-\frac{1}{\mu }\right) . \end{aligned}$$

(38)

Note that $D<\frac{1}{4}$, as all parameters are assumed to have positive values. For $D>0$, the quadratic equation has two real solutions

$$\begin{aligned} c_\pm = \frac{1}{2} \pm \sqrt{D} = \frac{1\pm \sqrt{1-1/\mu }}{2}, \end{aligned}$$

both satisfying $0<c_\pm <1$; for $D=0$, there is a single real solution $c=1/2$, while for $D<0$, there is no real solution.

Summarizing, in the region $0\le c \le 1$, there is always the steady state $(c,u)= e_0 = (0,0).$ If $D=0$, there is the additional steady state

$$\begin{aligned} (c,u) = e_b = \left( \frac{1}{2}, 1\right) , \end{aligned}$$

(39)

and if $D\ge 0$, there are

$$\begin{aligned} (c,u) = e_\pm = (c_\pm ,u_\pm ) = \left( \frac{1}{2}\pm \sqrt{D}, 1\right) . \end{aligned}$$

(40)

These are all the steady states of the state-control system (36).

1.2 B.2 Stability

To analyze stability, we have to determine the eigenvalues of

$$\begin{aligned} DF =\begin{pmatrix} 1 - u &{} -c \\ 4\rho \mu (2c-1) &{} \rho \end{pmatrix} \end{aligned}$$

at the steady states $e_0$, $e_+$ and $e_-$. As

$$\begin{aligned} DF(e_0) = \begin{pmatrix} 1 &{} 0 \\ -4\rho \mu &{} \rho \end{pmatrix}, \end{aligned}$$

which has eigenvalues $\rho$ and 1, the point $e_0$ is always an unstable node.

Denote the eigenvalues of the matrix

$$\begin{aligned} DF(e_\pm )=\begin{pmatrix} 0 &{} - c_\pm \\ \pm 8\rho \mu \sqrt{D} &{} \rho \end{pmatrix} \end{aligned}$$

(41)

by $\lambda ^i_\pm$, $i=1,2$. They satisfy $\lambda ^1_\pm +\lambda ^2_\pm = \text{ trace }\, DF(e_\pm ) = \rho$ and $\lambda ^1_\pm \lambda ^2_\pm = \det DF(e_\pm ) = \pm 8\rho \mu c_\pm \sqrt{D}.$ We have seen before that $c_\pm >0$ whenever it is real. If $D>0$, it follows that the eigenvalues $\lambda ^1_-,\lambda ^2_-$ have opposite sign, and $e_-$ is a saddle, whereas $\lambda ^1_+$ and $\lambda ^2_+$ have the same sign and positive sum, implying that $e_+$ is an unstable node.

Expressing these results in the original variables, we obtain the results announced in the proposition.

1.3 B.3 Bifurcation Analysis

It remains to prove the occurrence of a saddle-node bifurcation. If $\mu =\mu _b=1$, then $D=0$ and the point $e_b=(c_b,u_b) = (1/2,1)$ is a steady state with eigenvalues 0 and $\rho$, respectively.

We use a result from Sotomayor [50] (quoted as Theorem 3.4.1 in Guckenheimer and Holmes [51]), which for planar dynamical systems states that if the family

$$\begin{aligned} \dot{x} = F(x;\mu ) \end{aligned}$$

parametrized by $\mu$ satisfies the following three conditions

• A. $D_xF(x_0;\mu _0)$ has a simple eigenvalue 0 with right eigenvector v and left eigenvector w;

• B. $w D_\mu F(x_0;\mu _0) \ne 0$;

• C. $w \left[ D_{x}^2 F(x_0;\mu _0)(v,v)\right] \ne 0$;

then it features a non-degenerate saddle-node bifurcation at $x=x_0$ for $\mu =\mu _0$.

As $DF(e_b;\mu _b)=\begin{pmatrix} 0 &{} -1/2 \\ 0 &{} \rho \end{pmatrix}$, it follows that $v=(1,0)$ and $w^\mathrm{T}=(2,1)$ are respectively left and right eigenvectors associated with the eigenvalue 0. Furthermore,

$$\begin{aligned} w D_\mu F(e_b;\mu _b) = w \begin{pmatrix} 0 \\ -\rho \end{pmatrix} = -\rho \ne 0 \end{aligned}$$

and

$$\begin{aligned} w \left[ D_x^2 F(e_b;\mu _b)(v,v)\right] = w \frac{\partial {}^2 \phantom {F}}{\partial c^2}F = w \begin{pmatrix} 0 \\ 8\rho \end{pmatrix} = 8\rho \ne 0. \end{aligned}$$

We conclude that a non-degenerate saddle-node bifurcation occurs in the system at $\mu =1$. This completes the proof of Proposition 4.1. $\square$

Appendix C: Proof of Proposition 4.2

As in the proof of Proposition 4.1, introduce the constants

$$\begin{aligned} K = \frac{1}{\phi (1+\beta )} \quad \text {and}\quad \mu = \frac{\alpha \phi (1+\beta )}{4\rho K} = \frac{\alpha \phi ^2(1+\beta )^2}{4\rho }, \end{aligned}$$

as well as the rescaled control variable $u=k/K$. The state-control system then takes the form

$$\begin{aligned} \dot{c} = c \left( 1-u\right) ,\quad \dot{u} = \rho \Bigl (u - 4 \mu c (1-c) \chi (c)\Bigr ). \end{aligned}$$

(42)

Recall also the notations

$$\begin{aligned} e_0 = (0,0), \quad e_- = \left( \frac{1-\sqrt{1-1/\mu }}{2}, 1\right) , \quad e_+ = \left( \frac{1+\sqrt{1-1/\mu }}{2}, 1\right) \end{aligned}$$

for the three steady states of the system, and introduce $e_1 = (1,0).$

To prove the proposition, the state-control space is partitioned into four subsets, $R_1$, $R_2$, $R_3$ and E. Of these, the sets $R_3$ and E are independent of the values of the system parameters. They are given as $R_3 = \{(c,u)\,:\, 0< c < 1,\, u=0\}$ and $E = \{(c,u)\,:\, c\ge 1,\, u=0\}.$ Let $U=\{(c,u)\,:\, u>0\}$ be the upper half plane. Given the set $R_1$, the set $R_2$ is equal to $R_2 = U \backslash R_1.$

It remains to specify $R_1$, which is the first step in the proof. Then it is shown that no trajectory in either $R_2$ or $R_3$ can be optimal. The next step is to demonstrate that of the trajectories in $R_1$, only those can be optimal which converge either to a steady state in $R_1$, necessarily a saddle, or which end up in the “exit trajectory” E. Then it has to be shown that the trajectories that are not excluded up to this point, the candidate trajectories, “cover” the state space; that is, for every initial state $c_0$, there is at least one candidate trajectory passing through the line $c=c_0$. Using parts of the remaining candidate trajectories, we construct a viscosity solution of the Hamilton–Jacobi equation, which is then necessarily the value function. This shows the optimality of the remaining trajectories.

1.1 C.1 Definition of $R_1$

Set $u_0 = \max \{1,\mu \},$ and consider the trajectory $\gamma (t)=(c(t),u(t))$ of the system (42) that passes at $t=0$ through the point $(1,u_0)$.

If $\mu \le 1$, then $u_0=1$ and $R_1$ is specified as $R_1 = \left\{ (c,u)\,:\, 0\le c \le 1,\, 0 < u \le 1 \right\} .$

If the other possibility $\mu >1$ obtains, then $u_0=\mu >1$ and $\dot{c}(0)<0$. In this situation, let $\tau$ be the least upper bound of those negative values of t that satisfy $c(t)\le 1$; that is, let $\tau = \sup \{t<0\,:\, c(t)\le 1\}.$

We claim that $\tau$ is finite. Arguing by contradiction, assume that $\tau = -\infty$. Then for all $t<0$ we have $c(t)>1$, and Eq. (42) implies that for all $t<0$ we have $u(t) = u_0 {\hbox {e}}^{\rho t}.$ In particular, there is a $t_1<0$ such that $u(t)< u_0 {\hbox {e}}^{\rho t_1} =: K_1 < 1$ for all $t<t_1$. But for those values of t, it follows that $\dot{c} = \left( 1 - u\right) c > \left( 1 - K_1\right) c =: K_2 c,$ where $K_2>0$. Gronwall’s lemma implies then that $c(t) < {\hbox {e}}^{K_2 (t-t_1)} c(t_1)$ if $t<t_1$. But for t sufficiently small, this is smaller than 1, contradicting the hypothesis that $\tau =-\infty$. Hence $\tau$ is finite.

Introduce $u_\tau$ by the equation $\gamma (\tau ) = (1,u_\tau )$. The set $R_1$ is defined as follows: it is the open region bounded by the concatenation of the curve $\gamma$ taken between $t=0$ and $t=\tau$, connecting $(1,u_0)$ and $(1,u_\tau )$, the vertical line segment connecting $(1,u_\tau )$ to $e_1$, the horizontal segment connecting $e_1$ to $e_0$, the vertical segment connecting $e_0$ to $(0,u_0)$, and the horizontal segment connecting $(0,u_0)$ to $(1,u_0)$. See Fig. 10 for the possible shapes of $R_1$.

Fig. 10

Definition of the set $R_1$. Solid curves denote the boundary of the set, dashed curves the isoclines of the system (36). a $\mu \le 1$, b $\mu > 1$

1.2 C.2 Trajectories in $R_2$ Cannot Be Optimal

In the second step of the proof, the transversality condition is used to show that any trajectory that passes through points in $R_2$ cannot be optimal.

Beginning with $R_2$, we note that the subset

$$\begin{aligned} R^{(1)}_2 = \left\{ (c,u)\,:\, 0\le c \le 1\right\} \cap R_2 \end{aligned}$$

of $R_2$ is a forward trapping region: once a trajectory of (42) is inside $R^{(1)}_2$, it remains inside for all subsequent times. This fact is established by demonstrating that the vector field defined by (42) is inward pointing on the boundary of $R^{(1)}_2$. For, if $u=u_0=\max \{1,\mu \}$ and $0\le c\le 1$, then

$$\begin{aligned} \dot{u} \ge \rho (\mu - 4 \mu c(1-c)) = 0. \end{aligned}$$

If $c=0$, then $\dot{c} = 0$, and if finally $c=1$ and $u\ge u_0\ge 1$, then

$$\begin{aligned} \dot{c} \le c\left( 1 - 1\right) = 0. \end{aligned}$$

Actually, we can make the sharper statement that if $u>u_0$, then

$$\begin{aligned} \dot{u} > 0. \end{aligned}$$

(43)

To show that no trajectory that enters $R^{(1)}_2$ can be maximizing, pick an arbitrary trajectory $\gamma$ such that $\gamma (t_0)\in R^{(1)}_2$ at a given time $t_0$. By the Poincaré-Bendixon theorem, $\gamma (t)$ is either unbounded, or its $\omega$-limit set is a steady state, or a limit cycle. The latter possibility is excluded, as the state-costate system, which is in one-to-one relation with the state-control system, has constant positive divergence everywhere (see [47]). There are no steady states in $R^{(1)}_2$. Hence there is a sequence $t_0, t_1, \ldots$ such that $|\gamma (t_i)|\rightarrow \infty$. In particular, there is $\bar{t}>t_0$ such that $u(\bar{t})>2u_0$. But then u(t) is monotonely increasing toward infinity as $t>\bar{t}$, as a consequence of (43).

Consequently, if $t\ge \bar{t}$, then $\dot{c} \le \left( 1 - 2 u_0\right) c \le - c.$ By Gronwall’s lemma it follows that

$$\begin{aligned} c(t) \le c(\bar{t}) {\hbox {e}}^{-(t-\bar{t})}. \end{aligned}$$

(44)

Likewise, if $t\ge \bar{t}$, then $u(t)>2u_0$ and $\dot{u} \ge \rho (u - \mu ).$ Gronwall’s lemma implies then that

$$\begin{aligned} u(t) \ge \mu + (2u_0-\mu ){\hbox {e}}^{\rho (t-\bar{t})}. \end{aligned}$$

(45)

If the trajectory $\gamma (t)=(c(t),u(t))$ is optimal, then by the Hamilton–Jacobi equation (see, e.g., Wagener [47]), the total profit $\varPi$ takes the value

$$\begin{aligned} \varPi (c(0)) = \frac{1}{\rho } H(c(0),\lambda (0)) = \frac{1}{\rho } H_\text {control}(c(0),u(0)). \end{aligned}$$

(46)

Michel’s transversality condition (Michel [52]) states that along a maximizing trajectory the relation

$$\begin{aligned} \lim _{t\rightarrow \infty } \varPi (c(t)){\hbox {e}}^{-\rho t} = 0 \end{aligned}$$

holds. Combining (46) and (37) yields

$$\begin{aligned} \varPi (c(t)){\hbox {e}}^{-\rho t}&\ge \left( 4\rho \mu (1-c(t))^2\chi (c(t)) + u(t) (u(t)-2) \right) \frac{{\hbox {e}}^{-\rho t}}{\rho }. \end{aligned}$$

Using that the first term between brackets is always nonnegative, and taking into account (45) yields that

$$\begin{aligned} \varPi {\hbox {e}}^{-\rho t}&\ge (2u_0 - \mu ) {\hbox {e}}^{\rho (t-\bar{t})} \bigl (\mu - 2 + (2u_0-\mu ){\hbox {e}}^{\rho (t-\bar{t})}\bigr ) \frac{{\hbox {e}}^{-\rho t}}{\rho }. \end{aligned}$$

As $2u_0-\mu \ge \mu >0$, it follows that the right hand side of this inequality tends to infinity as $t\rightarrow \infty$. But then

$$\begin{aligned} \lim _{t\rightarrow \infty }\varPi (c(t)){\hbox {e}}^{-\rho t} = \infty , \end{aligned}$$

and $\gamma$ cannot be a maximizing trajectory.

It remains to show that no trajectory passing through $R^{(2)}_2 = R_2 \backslash R^{(1)}_2,$ the complement of $R^{(1)}_2$ in $R_2$, can be optimal. Consider therefore a trajectory $\gamma$ such that $\gamma (t_0)\in R^{(2)}_2$ for some $t_0$. As in the definition of the region $R_1$, using Gronwall’s lemma it can be shown that there is some $t_1>t_0$ such that $u(t_1)>1$, and some $t_2>t_1$ such that $u(t_2)>1$ and $c(t_2)=1$. But then $\gamma$ enters the trapping region $R^{(1)}_2$, and we have already seen that such trajectories cannot be optimal.

1.3 C.3 Trajectories Intersecting $R_3$ Cannot Be Optimal

If a trajectory intersects $R_3$, the state-control representation breaks down, and we have to switch to the state-costate representation.

Pick an arbitrary state-costate trajectory $\gamma (t)=(c(t),\lambda (t))$, with associated control $u(t)=\max \{0,- c(t) \lambda (t)/2\},$ such that $\bigl (c(t_0),u(t_0)\bigr )\in R_3$ for some $t_0\ge 0$ and $\bigl (c(t),u(t)\bigr )\in R_1$ for all $t<t_0$ that are sufficiently close to $t_0$. The costate $\lambda$ satisfies $\lambda (t_0)=0$. Equation (34) then implies that $\dot{\lambda }(t_0) > 0$. The region $\tilde{R}_3=\{(c,\lambda )\,:\,\lambda >0\}$ is a trapping region for the state-costate flow, as $\dot{\lambda } \ge 0$ whenever $\lambda =0$.

Using Gronwall’s lemma, we show first that

$$\begin{aligned} c(t) \ge c(t_0) {\hbox {e}}^{(t-t_0)}, \end{aligned}$$

for $t>t_0$, since $\dot{c} = c \ge c$ in $\tilde{R}_3$ (Eq. 35). It follows that there is $t_1>t_0$ such that $c(t)>1$ for all $t>t_1$. Consequently $\chi (c(t))=0$ for these values of t, and the state-costate equations reduce to

$$\begin{aligned} \dot{c} = c,\quad \dot{\lambda } = (\rho - 1)\lambda . \end{aligned}$$

(47)

Let $h(t) = H(c(t),\lambda (t))$. For all $t>t_1$ we have $c(t)>1$ and $\lambda (t)>0$ and consequently $h(t)=\lambda (t)c(t) > 0$. Compute:

$$\begin{aligned} \dot{h}&= \dot{\lambda } c + \lambda \dot{c} = \rho \lambda c = \rho h. \end{aligned}$$

Hence $h(t) = h(t_1) {\hbox {e}}^{\rho (t-t_1)}$ for all $t>t_1$. But then $\lim _{t\rightarrow \infty } h(t){\hbox {e}}^{-\rho t} = h(t_1){\hbox {e}}^{-\rho t_1}>0.$ If $\gamma$ is optimal, Michel’s transversality condition implies that

$$\begin{aligned} \lim _{t\rightarrow \infty }\varPi (c(t)){\hbox {e}}^{-\rho t} = \lim _{t\rightarrow \infty }\frac{1}{\rho } H(c(t),\lambda (t)){\hbox {e}}^{-\rho t} = \lim _{t\rightarrow \infty }\frac{h(t)}{\rho }{\hbox {e}}^{-\rho t} = 0. \end{aligned}$$

As this is a contradiction, the trajectory $\gamma$ cannot be optimal.

1.4 C.4 Trajectories in $R_1$ with Wrong Limit Behavior Cannot Be Optimal

As the set $R_1$ is bounded, by the Poincaré-Bendixon theorem trajectories in $R_1$ can either converge to a steady state or leave $R_1$ (cf. the argument in Sect. C.2). Those entering either $R_2$ or $R_3$ have already been shown to be suboptimal. The remaining possibility is to leave $R_1$ through the point $e_1$ and enter the line segment E; these trajectories remain candidates for optimality.

Trajectories remaining in $R_1$ have to converge to a steady state. From proposition 4.1, we learn that $e_0$ and $e_+$ are unstable nodes, to which no trajectory can converge as $t\rightarrow \infty$. The only remaining candidate is then the saddle $e_-$, if $\mu >1$, or the bifurcating point $e_b$ if $\mu =1$.

This completes the proof of Proposition 4.2. $\square$

Appendix D: Proof of Proposition 4.3

1.1 D.1 Construction of Policy Functions

The first step in the proof is to construct those (parts of) trajectories of the system (42) that will turn out to optimize the profit functional. In particular, we shall construct a, possibly multivalued, policy function $u_\mathrm{f}$ such that the following holds. If $(c_0,u_0)$ is such that $u_0=u_\mathrm{f}(c_0)$, then the trajectory (c(t), u(t)) of (42) starting at this point satisfies, for all $t\ge 0$, that $\dot{c}(t)\ne 0$ and $u(t)=u_\mathrm{f}(c(t))$.

Again we have to distinguish between the situations $\mu <1$ and $\mu \ge 1$.

1.1.1 D.1.1 No Steady States in $R_1$

If $\mu <1$, the only steady state of (42) is the origin $e_0$, which is an unstable node. Therefore, the only candidate optimizer is the trajectory $\gamma (t)$ passing through the point $\gamma (0)=e_1$; see Fig. 12a. Note that a corollary of the analysis performed above is that the set $R_1$ is a backward trapping region: if a trajectory is in $R_1$ for some time, it is in $R_1$ for all previous times, and it necessarily converges to the origin as $t\rightarrow -\infty$.

Write $\gamma (t)=(c_\gamma (t),u_\gamma (t))$. The fact that $\gamma (t)\in R_1$ for all $t<0$ implies that $\dot{c}_\gamma >0$ for all $t<0$—recall that $R_1$ is open. Moreover, as $u(t)=0$ for all $t\ge 0$, it follows that $\dot{c}_\gamma >0$ for all t as well, and that the map $c_\gamma :{\mathbb {R}}\rightarrow ]0,\infty [$ is invertible, with inverse $t=t_\gamma (c)$. Define $u_\mathrm{f}:\, ]0,\infty [\rightarrow {\mathbb {R}}$ by

$$\begin{aligned} u_\mathrm{f}(c) = u_\gamma \left( t_\gamma (c)\right) . \end{aligned}$$

Then the image of the curve $\gamma :{\mathbb {R}}\rightarrow {\mathbb {R}}^2$ is equal to the graph of the function $u_\mathrm{f}:\,]0,\infty [\rightarrow {\mathbb {R}}$, as

$$\begin{aligned} u_\gamma (t) = u_\mathrm{f}(c_\gamma (t)) \end{aligned}$$

for all t.

1.1.2 D.1.2 $\mu > 1$: Construction of the Region $S_1$

If $\mu >1$, though $R_1$ is still a backward trapping region, there are at least two steady states in $R_1$: apart from the origin $e_0$, which is in the boundary of $R_1$, we have $e_-$ and $e_+$ in the interior of $R_1$. As seen before, if $D>0$, the first is a saddle and the second a repeller; if $D=0$, and hence $\mu =1$, these two points coincide in $e_b$.

Introduce the curve segments $\delta _i$, $i=1,\ldots ,4$, as follows: $\delta _1$ is the part of the parabola $u=4\mu c(1-c)$ connecting $e_0$ to $e_-$, $\delta _2$ the segment of the line $u=1$ connecting $e_-$ to $e_+$, $\delta _3$ that part of the same parabola connecting $e_+$ to $e_1$, and $\delta _4$ the segment of the line $u=0$ connecting $e_1$ to $e_0$. All curves $\delta _i$ are taken without their endpoints. Let finally $S_1\subset R_1$ be the open subregion of $R_1$ that is bounded by the curves $\delta _i$, $i=1,\ldots ,4$. See Fig. 11.

Fig. 11

Subdivision of region $R_1$. The vertices $e_0$, $e_1$, $e_-$ and $e_+$, the edges $\delta _i$, $i=1,\ldots ,4$, and the faces $S_i$, $i=1,\ldots ,4$ are defined in the text

Let, as before, $\gamma (t)=(c(t),u(t))$ be the trajectory of (42) satisfying $\gamma (0)=e_1$. As the open set $S_1$ is bounded, the trajectory $\gamma$ either converges to a steady state on the boundary of $S_1$, or it enters $S_1$ for the last time by crossing one of the curves $\delta _i$. We analyze the possibilities one by one.

1.1.3 D.1.3 Invoking the Poincaré–Bendixon Theorem

We classify the possible limit behavior of the trajectory $\gamma (t)$ that satisfies $\gamma (0)=e_1$ as $t\rightarrow -\infty$. The region $R_1$ being a bounded backward trapping region, $\gamma (t)\in R_1$ for all $t<0$.

The Poincaré–Bendixon theorem (cf. [53], p. 29) asserts that, asymptotically, $\gamma (t)$ converges either to a steady state, a limit cycle, or a heteroclinic cycle. Since the state–control system (36) is diffeomorphic to the state–costate system (34), and since the latter has positive divergence everywhere, the existence of limit cycles or heteroclinic cycles is ruled out (cf. [47]). Therefore $\gamma (t)$ can converge either to $e_0$, or $e_-$, or $e_+$, as $t\rightarrow -\infty$.

Looking more precisely at the behavior of $\gamma (t)$ in $S_1$, we claim that either of the following possibilities holds:

• A. $\gamma (t) \in S_1$ for all t and $\gamma (t)\rightarrow e_-$ as $t\rightarrow -\infty$;

• B. $\gamma (t) \in S_1$ for all t and $\gamma (t)\rightarrow e_+$ as $t\rightarrow -\infty$;

• C. there is a largest value $t_1$ of t such that $\gamma (t)\in S_1$ and $\gamma (t_1)\in \delta _1$;

• D. there is a largest value $t_1$ of t such that $\gamma (t)\in S_1$ and $\gamma (t_1)\in \delta _2$.

This is equivalent to stating that if $\gamma (t)\in S_1$ for all t, it cannot tend to $e_0$ as $t\rightarrow -\infty$.

The claim follows from the fact that $\dot{u}(t)<0$ in $S_1$ and that $e_0=(0,0)$, which implies that $\lim _{t\rightarrow -\infty } u(t)=0$, $u(0)=0$ and $\dot{u}(t)<0$ for $t<0$, which is a contradiction.

We proceed by analyzing these situations one by one.

1.1.4 D.1.4 The Trajectory $\gamma$ Remains in $S_1$ and Tends to $e_-$

Reasoning as in the situation $D<0$, we obtain a policy function $u^{(1)}_\mathrm{f}:\,]c_-,\infty [\rightarrow {\mathbb {R}}$ with $\lim _{c\downarrow c_-}u^{(1)}_\mathrm{f}(c) = u_- = 1.$ This function is however not defined for all $c>0$. To construct a policy function for $0<c<c_-$, we take a trajectory $\gamma ^s$ on the left half of the stable manifold of the saddle $e_-$.

We claim that this part of the stable manifold is contained in its entirety in the region $S_2$ that is bounded by $\delta _1$, the segment of $u=1$ connecting $e_-$ to the point (0, 1), and the segment of the line $c=0$ connecting the point (0, 1) to $e_0$. It is straightforward to show that $S_2$ is a backward trapping region; consequently, every trajectory in $S_2$ converges to the unstable node $e_0$ as $t\rightarrow -\infty$.

The stable manifold of $e_-$ is tangent to the stable eigenspace of

$$\begin{aligned} DF(e_-) = \begin{pmatrix} 0 &{} - c_- \\ - 8 \rho \mu \sqrt{D} &{} \rho \end{pmatrix}, \end{aligned}$$

cf. Eq. (41), at $e_-$. Note that the vector $v=(0,1)$ cannot be an eigenvector of this matrix, as $c_-\ne 0$. Therefore, any eigenvector $v=(v_1,v_2)$ satisfies $v_1\ne 0$; it may therefore be assumed that $v_1=1$.

Let $v^s=(1,v^s_2)$ be the stable eigenvector, with eigenvalue $\lambda ^s<0$. The eigenvalue equation $DF(e_-) v^s = \lambda ^s v^s$ then yields $v^s_2 = -\lambda ^s/c_->0.$ Locally around the saddle, the stable manifold coincides with the graph of a function $w^s$, defined on a neighborhood of $c_-$, which is of the form

$$\begin{aligned} w^s(c) = c_- + v^s_2 (c-c_-) + O((c-c_-)^2). \end{aligned}$$

In particular, if $c_0<c_-$ is sufficiently close to $c_-$, then ${\text {d}}w^s/{\text {d}}c > 0$ for all $c\in [c_0,c_-]$. The trajectory $\gamma (t)$ of (42) such that $\gamma (0)=(c_0,w^s(c_0))$ consequently satisfies $c_0\le c(t) < c_-$, as well as $\dot{c}(t)>0$ and $\dot{u}(t)>0$ for all $t\ge 0$. We infer that necessarily

$$\begin{aligned} 4\mu c(t)(1-c(t))<u(t)<1 \end{aligned}$$

for all $t\ge 0$, and hence $(c(t),u(t))\in S_2$ for all $t\ge 0$. But as $S_2$ is a backward trapping region, the trajectory $\gamma$ is contained in $S_2$ for all t, hence satisfying

$$\begin{aligned} \gamma (t)\rightarrow e_0 \quad \text {as}\quad t\rightarrow -\infty , \qquad \text {and}\qquad \gamma (t)\rightarrow e_- \quad \text {as}\quad t\rightarrow \infty . \end{aligned}$$

As in $S_2$, we have $\dot{c}>0$ everywhere, and we construct as above a policy function $u^{(2)}_\mathrm{f}: \, ]0,c_- [\rightarrow {\mathbb {R}},$ with $\lim _{c\uparrow c_-} u^{(2)}_\mathrm{f}(c) = u_- = 1.$ It follows that the function

$$\begin{aligned} u_\mathrm{f}(c) = {\left\{ \begin{array}{ll} u^{(1)}_\mathrm{f}(c),\quad &{} \text {if } c>c_-, \\ u_- , &{} \text {if } c=c_-, \\ u^{(2)}_\mathrm{f}(c), &{} \text {if } 0<c<c_-, \end{array}\right. } \end{aligned}$$

is a continuous policy function that is defined for all $c>0$.

1.1.5 D.1.5 The Trajectory $\gamma$ Remains in $S_1$ and Tends to $e_+$

As before, we can construct a policy function $u^{(1)}_\mathrm{f}: \, ]c_+,\infty [\,\rightarrow {\mathbb {R}}, \text {with } \lim _{c\downarrow c_+} u^{(1)}_\mathrm{f}(c)= u_+ = 1.$ The remaining part of the policy function has to be furnished by the stable manifold of $e_-$.

As above, the left half of this stable manifold furnishes a policy function $u^{(2)}_\mathrm{f}: \, ]0,c_-[\,\rightarrow {\mathbb {R}}, \text { with } \lim _{c\uparrow c_-} u^{(2)}_\mathrm{f}(c)= u_-=1.$ We turn to the right half of the stable manifold. For values of $c_0$ larger than but close to $c_-$, the point $(c_0,u_0)=(c_0,w^s(c_0))$ on the stable manifold is contained in the bounded open region $S_3$ that is bounded by the line $u=1$ and the parabola $u=4\mu c(1-c)$. In this region, $\dot{c}<0$ and $\dot{k}<0$. Fix $(c_0,u_0)$ and consider the trajectory $\gamma$ of (42) such that $\gamma (0)=(c_0,u_0)$. This trajectory enters $S_3$ through the part of the parabola connecting its vertex $(1/2,\mu )$ with the point $e_+$. It enters from the region $S_4$ that is bounded by that same part of the parabola, the line $u=u_+$ and the boundary of $R_1$. In that region, $\dot{c}<0$, but $\dot{k}>0$. It follows that the trajectory has to enter $S_4$ through the line segment of $c=c_+$ connecting $e_+$ and $(c_+,\mu )$, or through one of the endpoints.

If $\gamma (t)\rightarrow e_+$ as $t\rightarrow -\infty$, then its graph defines a policy function $u^{(3)}_\mathrm{f}: \,]c_-,c_+[\,\rightarrow {\mathbb {R}}\text { with } \lim _{c\downarrow c_-} u^{(3)}_\mathrm{f}(c)= u_-=1 \text { and } \lim _{c\uparrow c_+} u^{(3)}_\mathrm{f}(c) = u_+=1.$ A continuous policy function is then given by

$$\begin{aligned} u_\mathrm{f}(c) = {\left\{ \begin{array}{ll} u^{(1)}_\mathrm{f}(c),\quad &{} \text {if } c>c_+, \\ u_+, &{} \text {if } c=c_+, \\ u^{(2)}_\mathrm{f}(c), &{} \text {if } 0<c<c_-, \\ u_- ,&{} \text {if } c=c_-, \\ u^{(3)}_\mathrm{f}(c), &{} \text {if } c_-<c<c_+. \end{array}\right. } \end{aligned}$$

Otherwise, there is a time $t_1<0$ such that $c(t_1)=c_+$ and $u(t_1)>u_+$. As in this case $\gamma (t)$ does not tend to a steady state in the boundary of $S_4$, it has to enter $S_4$ for some $t_2<t_1$; the only possibility for this is through the line $u=1$. We therefore have

$$\begin{aligned} \gamma (t_2) = (c_M,1), \end{aligned}$$

where $c_M$ is defined by this equation. In this situation, the graph $\gamma ([t_2,\infty [)$ defines a policy function $u^{(3)}_\mathrm{f}: \,]c_-,c_M[\,\rightarrow {\mathbb {R}}\text { with } \lim _{c\downarrow c_-} u^{(3)}_\mathrm{f}(c) = u_-=1 \text { and } \lim _{c\uparrow c_M} u^{(3)}_\mathrm{f}(c) = 1.$ On the interval $]c_+,c_M[$, there are now two policy functions defined. Introduce the associated profits

$$\begin{aligned} \varPi ^{(i)}(c)=\rho ^{-1}H_\text {control}(c,u_\mathrm{f}^{(i)}(c)), \quad i=1,3. \end{aligned}$$

For fixed values of c, the function $H_\text {control}(c,u)$ is minimal at $u=1$. Hence the policy $u^{(3)}_\mathrm{f}$ is superior to $u^{(1)}_\mathrm{f}$ at $c=c_+$, in the sense that $\varPi ^{(3)}(c_+)> \varPi ^{(1)}(c_+),$ since $u_\mathrm{f}^{(1)}(c_+)=1$. In the same manner, it follows that $\varPi ^{(3)}(c_M)<\varPi ^{(1)}(c_M)$. As both functions are continuous, there is a value $c=\hat{c}$ such that $\varPi ^{(3)}(\hat{c})=\varPi ^{(1)}(\hat{c})$. This is an indifference point, as the manager is indifferent between two policies at this state. A policy function, which is at one point two valued, is then given by

$$\begin{aligned} u_\mathrm{f}(c) = {\left\{ \begin{array}{ll} u^{(1)}_\mathrm{f}(c),\quad &{} \text {if } c>\hat{c}, \\ u^{(1)}_\mathrm{f}(\hat{c})\text { or } u^{(3)}_\mathrm{f}(\hat{c}), &{} \text {if } c=\hat{c}, \\ u^{(2)}_\mathrm{f}(c), &{} \text {if } 0<c<c_-, \\ u_- ,&{} \text {if }c=c_-, \\ u^{(3)}_\mathrm{f}(c), &{} \text {if } c_-<c<\hat{c}. \end{array}\right. } \end{aligned}$$

The induced total profit $\varPi (c) = H_\text {control}(c,u_\mathrm{f}(c))/\rho$ is locally Lipschitz continuous for all $c\ne 0$. This follows from the coercivity of the Hamilton function H if c is bounded away from 0 as in Bardi and Capuzzo-Dolcetta [54], Proposition II.4.1.

As the instantaneous profit $\pi =\alpha (1-c)^2 - k^2$ is bounded, it is not hard to show that if the initial condition is larger than some value $\bar{c}$, the optimal action is to set $k=0$, which results in $\varPi (c)=0$ for $c>\bar{c}$. We conclude then that $\varPi (c)$ is Lipschitz continuous everywhere.

1.1.6 D.1.6 The Trajectory $\gamma$ Enters $S_1$ for the Last Time Through $\delta _1$

The next situation to be investigated is that the trajectory $\gamma$ satisfying $\gamma (0)=e_1$ enters $S_1$ through $\delta _1$ at some time $t_1<0$, and remains in $S_1$ for all $t_1<t<0$: see Fig. 12c.

Fig. 12

Illustration of various possibilities for the trajectory $\gamma$ through the point $e_1=(1,0)$. a $\mu <1$, b $\mu >1$, $\gamma$ in $S_1$ for all $t<0$, c $\mu >1$, $\gamma$ enters $S_1$ through $\delta _1$, d $\mu >1$, $\gamma$ enters $S_1$ through $\delta _2$

Since $\gamma (t_1)\in \delta _1$, it follows that $\gamma (t)\in S_2$ for all t close to, but smaller than $t_1$. As $S_2$ is a backward trapping region, $\gamma (t)\in S_2$ for all $t<t_1$, and necessarily $\gamma (t)$ converges to $e_0$ as $t\rightarrow -\infty$. Moreover, $\dot{c}>0$ in both $S_1$ and $S_2$, and we can construct a policy function that is differentiable for all $c>0$ exactly as in the situation that the trajectory remains in $S_1$ for $t<0$ and converges to $e_0$.

1.1.7 D.1.7 The Trajectory $\gamma$ Enters $S_1$ for the Last Time Through $\delta _2$

Finally, consider the situation that the trajectory $\gamma$ that passes through $e_1$ at $t=0$ enters $S_1$ through $\delta _2$ for some $t_1<0$, and remains in $S_1$ for all $t_1<t<0$: see Fig. 12d.

Introduce $c_m$ by setting $\gamma (t_1)=(c_m,1)$. Since $\gamma (t_1)\in \delta _2$, we have $c_-<c_m<c_+$. As $\dot{c}(t)>0$ for $t_1<t<0$ as well as for $t\ge 0$, we can construct a continuous policy function $u^{(1)}_\mathrm{f}:[c_m,\infty [\, \rightarrow {\mathbb {R}}\text { with } u^{(1)}_\mathrm{f}(c_m) = 1$ in the usual manner.

The left branch of the stable manifold of the saddle $e_-$ furnishes a continuous policy function $u^{(2)}_\mathrm{f}:\, ]0,c_-[\,\rightarrow {\mathbb {R}}, \text { with } \lim _{c\uparrow c_-} u^{(2)}_\mathrm{f}(c)= u_-=1,$ and the right branch of that manifold furnishes a continuous policy function $u^{(3)}_\mathrm{f}:\,]c_-,c_M[\,\rightarrow {\mathbb {R}}, \text { with } \lim _{c\downarrow c_-} u^{(3)}_\mathrm{f}(c)= u_-=1 \text { and } u^{(3)}_\mathrm{f}(c_M)= 1,$ where $c_+\le c_M$. Invoking the same arguments as above, we show that $u^{(3)}_\mathrm{f}$ is superior to $u^{(1)}_\mathrm{f}$ at $c=c_m$ and inferior to it at $c=c_M$. By the intermediate value theorem, there is an indifference point $\hat{c}$ such that $c_m<\hat{c}<c_M$, and such that the manager is indifferent between the two policies at $c=\hat{c}$. A policy function defined on all points of state space is then

$$\begin{aligned} u_\mathrm{f}(c) = {\left\{ \begin{array}{ll} u^{(1)}_\mathrm{f}(c),\quad &{} \text {if } c>\hat{c}, \\ u^{(1)}_\mathrm{f}(\hat{c})\text { or } u^{(3)}_\mathrm{f}(\hat{c}), &{} \text {if } c=\hat{c}, \\ u^{(2)}_\mathrm{f}(c), &{} \text {if } 0<c<c_-, \\ u_- ,&{} \text {if }c=c_-, \\ u^{(3)}_\mathrm{f}(c), &{} \text {if } c_-<c<\hat{c}. \end{array}\right. } \end{aligned}$$

1.1.8 D.1.8 At Bifurcation: $\mu = 1$

Finally, we have to discuss the case $\mu =1$, where there are only two steady states in the boundary of $R_1$: the repeller $e_0$ and the steady state $e_b=(1/2,1)$. As

$$\begin{aligned} DF(e_b) = \begin{pmatrix} 0 &{} -\frac{1}{2}\\ 0 &{} \rho \end{pmatrix}, \end{aligned}$$

$e_b$ has an unstable eigenspace, which is the span of the vector $v^u=(1,-2\rho )$, and a neutral eigenspace spanned by $v^c=(1,0)$. That is, $e_b$ is a stable node.

The trajectory $\gamma (t)$ for which $\gamma (0)=e_1$ can either cross the parabola $u=4c(1-c)$ for a value $0<c<\frac{1}{2}$ or it can remain in $R_1$ for all $t<0$ and tend to $e_b$. In the first situation, a policy function can be constructed exactly as in the situation that $\mu >1$ and $\gamma$ intersects $\delta _1$; see Sect. D.1.6.

In the second situation, $\gamma$ coincides with the unstable manifold of $e_b$. There is a unique trajectory $\tilde{\gamma }$ connecting $e_0$ with $e_b$, which is the limit of similar trajectories that connect $e_0$ with the stable manifold of $e_-$ for $\mu >1$. As $\tilde{\gamma }$ lies entirely in $S_2$, we have $\dot{c}(t)>0$ there. Proceeding as usual, we can find a continuous policy function $u_\mathrm{f}(c)$ such that its graph coincides for $0<c<\frac{1}{2}$ with the curve $\tilde{\gamma }$, while for $c>\frac{1}{2}$, it coincides with $\gamma$. Note that $u_\mathrm{f}(c)$ is continuously differentiable everywhere except at $c=0$ and $c=\frac{1}{2}$.

1.1.9 D.1.9 Summary

For all parameters, we have constructed a policy function $u_\mathrm{f}:\,]0,\infty [\,\rightarrow {\mathbb {R}},$ which is single valued except at most at one point $\hat{c}$, the indifference point. Moreover, the values of the two trajectories originating at an indifference point are the same, and $u_\mathrm{f}(c)=0$ if c is sufficiently large.

1.2 D.2 Policy Functions Generate Viscosity Solutions of the Hamilton–Jacobi Equation

Using relation (46), we have that

$$\begin{aligned} V(c) = \frac{1}{\rho }H_\text {control}(c,u_\mathrm{f}(c)) \end{aligned}$$

is well defined at $c=\hat{c}$, continuous and continuously differentiable at all points $c>0$ except $\hat{c}$. Moreover, the value of the total profit (30) along a trajectory $\gamma$ of the state-control system (36) such that $\gamma (0) = (c,u_\mathrm{f}(c))$ is equal to V(c).

We shall argue that V(c) is a viscosity solution—a term which we shall define shortly—of the Hamilton–Jacobi equation of our optimization problem, that the value function is another viscosity solution and that viscosity solutions of our problem are unique. From this argument, it follows that V(c) is the value function of our problem and that $u(t) = u_\mathrm{f}(c(t))$ is the optimizing control.

1.2.1 D.2.1 Viscosity Solutions

We quote the definition of viscosity sub- and supersolutions from Bardi and Capuzzo-Dolcetta [54, Sec. II.1, p. 26].

Definition D.1

Let $\varOmega$ be an open domain of ${\mathbb {R}}^n$, and let the function $H=H(x,p)$ be a continuous real valued function on $\varOmega \times {\mathbb {R}}^n$, defining the Hamilton–Jacobi equation

$$\begin{aligned} \rho W(x) - H(x,DW(x)) = 0, \end{aligned}$$

(48)

where DW(x) is the gradient of W in x.

$1^\circ$ :

A function W is a viscosity subsolution of (48) at x, if for every continuously differentiable function w such that the difference $W-w$ takes a local maximum at x, we have

$$\begin{aligned} \rho W(x) - H(x,Dw(x)) \le 0. \end{aligned}$$

(49)

$2^\circ$ :

A function W is a viscosity supersolution of (48) at x, if for every continuously differentiable function w such that the difference $W-w$ takes a local minimum at x, we have

$$\begin{aligned} \rho W(x) - H(x,Dw(x)) \ge 0. \end{aligned}$$

(50)

$3^\circ$ :

A function W is a viscosity solution of (48) at x, if it is both a viscosity subsolution and a viscosity supersolution at x.

1.2.2 D.2.2 V(c) is a Viscosity Solution

Recall that in Sect. 3.1, we have extended the mathematical definition of the problem to the real line, such that $V(c)=0$ for all $c\le 0$. Return to the state-costate representation (34), and introduce the feedback costate function $\lambda _\mathrm{f}(c) = - \frac{2}{c} u_\mathrm{f}(c)$ if $c>0$ and $\lambda _\mathrm{f}(c)=0$ if $c\le 0$. Note that then

$$\begin{aligned} V(c) = \frac{1}{\rho } H(c,\lambda _\mathrm{f}(c)). \end{aligned}$$

(51)

By construction, if $\gamma (t)=(c(t),\lambda (t))$ is a solution of the state-costate system such that $\lambda (0) = \lambda _\mathrm{f}(c(0))$, then $\lambda (t) = \lambda _\mathrm{f}(c(t))\text { for all } t.$ If $t>0$, then $c(t)\ne \hat{c}$ and $\lambda _\mathrm{f}$ is differentiable at c(t); by the chain rule

$$\begin{aligned} \dot{\lambda } = \lambda _\mathrm{f}'(c) \dot{c}. \end{aligned}$$

(52)

We claim that $\lambda _\mathrm{f}(c) = V'(c)$ for all $c\ne \hat{c}$. For, differentiating (51) with respect to c yields

$$\begin{aligned} V'(c) = \frac{1}{\rho }\left( H_c + H_\lambda \lambda _\mathrm{f}'(c)\right) . \end{aligned}$$

Evaluating this equation at $c=c(t)$, using first (52) and then (34) gives

$$\begin{aligned} V'\bigl (c(t)\bigr )&= \frac{1}{\rho } \left( H_c + H_\lambda \frac{\dot{\lambda }}{\dot{c}} \right) = \frac{1}{\rho } \left( H_c + H_\lambda \frac{\rho \lambda - H_c}{H_\lambda } \right) = \lambda (t) = \lambda _\mathrm{f}\bigl (c(t)\bigr ), \end{aligned}$$

which proves the claim.

It follows that the function V defined by (51) is a regular solution of the Hamilton–Jacobi equation

$$\begin{aligned} \rho V(c) = H(c,V'(c)) \end{aligned}$$

(53)

for all $c\ne \hat{c}$.

As V is continuously differentiable in the neighborhood of every point $\bar{c}\ne \hat{c}$, V is a viscosity solution of the Hamilton–Jacobi equation (53) at $\bar{c}$ if and only if it is a regular solution there.

It remains to show that V is a viscosity solution at an indifference point $\hat{c}$. Note that the left and right limits of $V'(c)$ exist at $\hat{c}$; we write

$$\begin{aligned} \hat{\lambda }^- = \lim _{c\uparrow \hat{c}} V'(c), \quad \hat{\lambda }^+ = \lim _{c\downarrow \hat{c}} V'(c). \end{aligned}$$

From the analysis done above, we infer that $\hat{\lambda }^- < \hat{\lambda }^+.$ Let v be a continuously differentiable function such that $V-v$ takes a local minimum at $c=\hat{c}$. Then necessarily

$$\begin{aligned} \lim _{c\uparrow \hat{c}} V'(c) - v'(c) \le 0, \text { } \lim _{c\downarrow \hat{c}} V'(c) - v'(c) \ge 0, \text { and hence } \hat{\lambda }^- \le v'(\hat{c}) \le \hat{\lambda }^+. \end{aligned}$$

(54)

As $\hat{c}$ is an indifference point, we have that $H(\hat{c},\hat{\lambda }^-) = H(\hat{c},\hat{\lambda }^+) = \rho V(\hat{c}).$ Moreover, the Hamilton function $H(c,\lambda )$ is convex in $\lambda$. Together with (54) this implies that

$$\begin{aligned} \rho V(\hat{c}) - H(\hat{c}, v'(\hat{c})) \ge 0. \end{aligned}$$

Hence V is a viscosity supersolution.

Consider now the situation that v is a continuously differentiable function such that $V-v$ takes a local maximum at $\hat{c}$. Then $\lim _{c\uparrow \hat{c}} V'(c) - v'(c) \ge 0, \text { } \lim _{c\downarrow \hat{c}} V'(c) - v'(c) \le 0, \text { and hence } v'(\hat{c}) \le \hat{\lambda }_- < \hat{\lambda }_+ \le v'(\hat{c}),$ which is a contradiction. There is no differentiable function such that $V-v$ takes a local minimum; but then for all such functions, the inequality (49) holds at $\hat{c}$, and V is a viscosity subsolution, and therefore a viscosity solution.

1.2.3 D.2.3 Uniqueness of Viscosity Solutions

We shall state a theorem that is a direct corollary of Theorem III.2.12 from Bardi and Capuzzo-Dolcetta to show that the value function V is the unique viscosity solution of the Hamilton–Jacobi equation (53).

Recall the notation $B_R(0)=\{x\in {\mathbb {R}}^n\,:\, |x|<R\}$, and recall that a function $\omega :[0,\infty [\,\rightarrow [0,\infty [$ is a modulus if it is continuous, non-decreasing and $\omega (0)=0$.

The theorem depends on four assumptions. The first three concern the controlled dynamics $\dot{y} = f(y,a)$.

Assumption D.1

A is a topological space, and $f:{\mathbb {R}}^n\times A \rightarrow {\mathbb {R}}^N$ is continuous.

Assumption D.2

f is bounded on $B_R(0)\times A$ for all $R>0$.

Assumption D.3

There is a constant $L>0$ such that $|f(x,a)-f(y,a)|\le L|x-y|$ for all $x,y\in B_R(0)$, $a\in A$.

The final assumption concerns the objective functional

$$\begin{aligned} J(x,\alpha ) = \int _0^\infty \ell (y_x(t),\alpha (t)) {\hbox {e}}^{-\rho t} {\text {d}}t, \end{aligned}$$

which is to be maximized; here the control function $\alpha (t)$ is a member of $\mathcal {A}$, which is the set of measurable functions $\alpha :[0,\infty [\,\rightarrow A$, and $y_x(t)$ is the solution of $\dot{y}(t) = f(y(t),\alpha (t))$ with $y(0)=x$.

Assumption D.4

$\rho >0$; $\ell$ is continuous; and there are a constant $B>0$ and a modulus $\omega$ such that $|\ell (x,a)|\le B$ and $|\ell (x,a)-\ell (y,a)|\le \omega (|x-y|)$ for all $x\in {\mathbb {R}}^n$ and all $a\in A$.

The value function of this problem is

$$\begin{aligned} v(x) = \sup _{\alpha \in \mathcal {A}} J(x,\alpha ). \end{aligned}$$

Theorem D.1

Assume D.1–D.4. Let the Hamilton function H be given as

$$\begin{aligned} H(x,\lambda ) = \sup _{a\in A}\left\{ \ell (x,a)+\lambda f(x,a)\right\} . \end{aligned}$$

Then the value function v is the unique viscosity solution of the Hamilton–Jacobi equation $\rho v = H(x,Dv).$

Proof

The assumptions imply assumptions A$_0$–A$_4$ in Chapter III of Bardi and Capuzzo-Dolcetta [54]. When formulating the problem as a minimization problem, $\ell$ is replaced by $-\ell$, v by $-v$ and $\lambda$ by $-\lambda$; this shows that the definition of Hamiltonian in statement of the theorem is equivalent to that in equation (2.9) in Chapter III of Bardi and Capuzzo-Dolcetta. The result follows then from their Theorem III.2.12. $\square$

1.2.4 D.2.4 Finishing the Proof

To apply the theorem, introduce the set $U=[0,M]$, where $M>0$ is a positive constant, and the functions $f:{\mathbb {R}}\times U\rightarrow {\mathbb {R}}$ and $\ell :{\mathbb {R}}\times U \rightarrow {\mathbb {R}}$ by

$$\begin{aligned} f(c,u) = {\left\{ \begin{array}{ll} c(1-u), \quad &{} c\ge 0, \\ 0, &{} c<0, \end{array}\right. } \end{aligned}$$

and

$$\begin{aligned} \ell (c,u) = {\left\{ \begin{array}{ll} 4\rho \mu (1-c)^2\chi (c) - u^2, \quad &{} c\ge 0, \\ 4\rho \mu - u^2, &{} c<0. \end{array}\right. } \end{aligned}$$

It is easy to see that assumptions D.1–D.4 are satisfied. The theorem then implies for every $M>0$ that the value function of the optimization problem is the unique viscosity solution of the Hamilton–Jacobi equation (53). Choose M such that $u_\mathrm{f}(c)<M$ for all c: this is possible since $u_\mathrm{f}(c)=0$ if c is sufficiently large. We extend the policy function to ${\mathbb {R}}$ by setting $u_\mathrm{f}(c)=0$ if $c<0$. If c(t) is a trajectory of the system dynamics

$$\begin{aligned} \dot{c} = f(c,u_\mathrm{f}(c)),\quad c(0)=c_0, \end{aligned}$$

then, by construction, $u(t)=u_\mathrm{f}(c(t))$ is a control such that

$$\begin{aligned} V(c_0) = \int _0^\infty \ell (c(t),u(t)) {\hbox {e}}^{-\rho t}{\text {d}}t. \end{aligned}$$

As $u_\mathrm{f}(c)<M$, this control is admissible. We have already shown that the function V(c) is a viscosity solution of the Hamilton–Jacobi equation; it therefore coincides with the value function, which by the theorem is the unique viscosity solution. But that implies that the controls generated by the policy function are optimal. $\square$

Appendix E: Proof of Proposition 5.1

This can be read off the optimization problem (30)–(31) obtained after the second scaling (29).

Let $\alpha _1=1/9$ and $\alpha _2=1/8$ be the respective partial and full collusion values of $\alpha$. Assume that there is a bifurcation in the partial collusion system at $\rho =\rho ^*$ and $K^{-1}=\phi (1+\beta )=\phi _1^*(1+\beta _1^*)=(K^*_1)^{-1}$. The equivalent scaled system (30)–(31), which only depends on the parameters $\rho$ and $\mu$, will then exhibit a bifurcation at $\mu ^* = \alpha _1/(4\rho ^* (K_1^*)^2).$ Consequently, if $K_2$ takes a value such that $\mu ^* = \alpha _2/(4\rho ^* (K_2^*)^2),$ then the full collusion system will exhibit a bifurcation as well.

This implies

$$\begin{aligned} (K_1^*)^2 = \frac{\alpha _1}{4\rho ^*\mu ^*} < \frac{\alpha _2}{4\rho ^*\mu ^*} = (K_2^*)^2, \end{aligned}$$

and consequently

$$\begin{aligned} \phi ^*_1(1+\beta ^*_1)>\phi ^*_2(1+\beta ^*_2). \end{aligned}$$

(55)

As a consequence, all full collusion bifurcation curves are lower than their partial collusion counterparts. This implies all inclusions stated in the theorem. This proves the proposition. $\square$

Appendix F: Proof of Propositions 5.2 and 5.3

We want to compare, for a given parameter combination, the full collusion situation $\alpha =\frac{1}{8}$, and the partial collusion situation $\alpha =\frac{1}{9}$. Performing the scaling to (c, u) variables and $(\mu ,\rho )$ parameters, this reduces to comparing the partial collusion situation $(\mu _1,\rho )$ with the full collusion situation $(\mu _2,\rho )$, where the $\mu _i$ are related as $\mu _2 = (9/8)\mu _1.$

Denote by $u^i_\mathrm{f}(c)$, $i=1,2$ the corresponding policy functions, and recall that their graphs are locally equal to a portion of a trajectory of (36), with u replaced by $u_1$ or $u_2$, depending on whether $\mu =\mu _1$ or $\mu =\mu _2$. Invoking the chain rule as in (52), we can derive a differential equation for $u_i=u^i_\mathrm{f}$ as follows:

$$\begin{aligned} \frac{{\text {d}}u_i}{{\text {d}}c}&= \frac{\dot{u}_i}{\dot{c}} = \frac{\rho \left( u_i - 4 \mu c(1-c)\chi \right) }{c(1-u_i)}; \end{aligned}$$

here, we have written $\chi =\chi (c)$ for brevity. This is a first-order non-autonomous differential equation, with singularities at $c=0$ and $u_i=1$.

Writing $\Delta \mu = \mu _2 - \mu _1$ and $\Delta u = u_2 - u_1$, the difference $\Delta u$ satisfies the following differential relation:

$$\begin{aligned} \frac{{\text {d}}\Delta u}{{\text {d}}c} =&\frac{\rho \left( u_2 - 4 \mu _2 c(1-c)\chi \right) }{c(1-u_2)} - \frac{\rho \left( u_1 - 4 \mu _1 c(1-c)\chi \right) }{c(1-u_1)} \\ =&\frac{\rho (1-u_1)\left( u_2 - 4 \mu _2 c(1-c)\chi \right) }{c(1-u_1)(1-u_2)} - \frac{\rho (1-u_2)\left( u_1 - 4 \mu _1 c(1-c)\chi \right) }{c(1-u_1)(1-u_2)} \\ =&\frac{\rho \left( u_2 - u_1 u_2 - 4 c(1-c)\chi (\mu _2 - u_1 \mu _2)\right) }{c(1-u_1)(1-u_2)} \\&- \frac{\rho \left( u_1 - u_1 u_2 - 4 c(1-c)\chi (\mu _1 - u_2 \mu _1) \right) }{c(1-u_1)(1-u_2)} \\ =&\frac{\rho \left( \Delta u - 4 c(1-c)\chi (\Delta \mu + u_2 \mu _2 - u_1 \mu _2 - u_2 \mu _2 + u_2 \mu _1)\right) }{c(1-u_1)(1-u_2)} \\ =&\frac{\rho \left( \Delta u - 4 c(1-c)\chi (\Delta \mu + \mu _2\Delta u - u_2 \Delta \mu )\right) }{c(1-u_1)(1-u_2)} \\ =&\frac{\rho \left( 1 - 4 \mu _2 c(1-c)\chi \right) }{c(1-u_1)(1-u_2)} \Delta u - \frac{4 \rho (1-c)\chi }{1-u_1} \Delta \mu \end{aligned}$$

Replace now $u_i$ by $u^i_\mathrm{f}$ for $i=1,2$ and introduce

$$\begin{aligned} a(c) = \frac{\rho \left( 1 - 4 \mu _2 c(1-c)\chi (c)\right) }{c(1-u^1_\mathrm{f}(c))(1-u^2_\mathrm{f}(c))}, \quad b(c) = -\frac{4 \rho (1-c)\chi (c)}{1-u^1_\mathrm{f}(c)}\Delta \mu . \end{aligned}$$

The differential equation for $\Delta u(c)$ is of the form

$$\begin{aligned} \frac{{\text {d}}\Delta u}{{\text {d}}c} = a(c) \Delta u + b(c), \end{aligned}$$

where a and b are known functions. The variations of constants formula for the solution reads as

$$\begin{aligned} \Delta u(c) = \Delta u(c_0) {\hbox {e}}^{\int _{c_0}^c a(x){\text {d}}x} + {\hbox {e}}^{\int _{c_0}^c a(x){\text {d}}x} \int _{c_0}^c b(x) {\hbox {e}}^{-\int _{c_0}^x a(y){\text {d}}y}{\text {d}}x. \end{aligned}$$

(56)

1.1 F.1 Proof of Proposition 5.2

Consider first the situation that there is a value $0\le \bar{c} \le 1$ such that for all $c\in \,]\bar{c},1]$ the optimal trajectories for both the partial and the full collusion case leave the production region through $e_1$. We know that trajectories through $e_1$ can be optimal only if they have not crossed the line $u=1$ yet: this is a consequence of the argument given in Sect. D.1.7. The term b of (56) therefore satisfies

$$\begin{aligned} b(c) = - \frac{4 \rho (1-c)\chi (c)}{1-u^1_\mathrm{f}(c)} \Delta \mu < 0 \end{aligned}$$

for $\bar{c}<c < 1$. If $c_0=1$, then $\Delta u(c_0)=0$, and the first term in (56) vanishes. Then $b(c)\le 0$ implies that $\Delta u(c)$ is decreasing. This implies for values of c smaller than 1 that $\Delta u(c)$ is positive, in particular $\Delta u(c) > 0$ for all $\bar{c}<c\le 1$. Hence, R&D effort under full collusion is always larger than R&D effort under partial collusion if both lead to eventually leaving the market.

Next, we consider the situation that there is some $\bar{c}>0$, such that for all $c\in \,]0,\bar{c}[$, the optimal trajectories for both the partial and the full collusion case converge to their respective steady states ${{e}}^1_-=(c_-^1,1)$ and ${{e}}^2_-=(c_-^2,1)$. As $\mu _2>\mu _1$, it follows that $0<c_-^2<c_-^1\le 1/2$. The stable manifold tending to ${{e}}^2_-$ can only leave the region bounded by the parabola $u=\mu _2 c(1-c)$ and the lines $u=1$ and $c=1/2$ through the line segment connecting the points (1 / 2, 1) with $(1/2,\mu _2)$. It follows that necessarily $u_2(c_-^1)> u_1(c_-^1), \text { or, equivalently, that } \Delta u(c_-^1)>0.$

We have already established that trajectories tending to either ${{e}}^1_-$ or ${{e}}^2_-$ can only be optimal if they do not cross the line $u=1$. Therefore

$$\begin{aligned} b(c) = \frac{4 \rho (1-c)\chi }{u_1-1} \Delta \mu > 0, \end{aligned}$$

if $0<c<\bar{c}$, and the variations of constants formula implies $\Delta u(c) > 0 \text { for all } c^1_- \le c < \bar{c}.$ Moreover $u_1(c)<1$ if $0<c<c_-^1$, implying that $b(c) < 0$ there. Again using the variations of constants formula, we obtain $\Delta u(c) > 0\text { for all } 0 < c \le c^1_-$ as well.

Finally, if the optimal trajectory of the full collusion case converges to ${{e}}^2_-$, whereas the optimal trajectory of the partial collusion case exits the production region through $e_1$, we have that the former satisfies $u\ge 1$ and the latter $u\le 1$.

This proves Proposition 5.2. $\square$

1.2 F.2 Proof of Proposition 5.3

To prove Proposition 5.3, we again use the fact that the value of the integral $\varPi$ over a trajectory starting at a point (c, u) equals

$$\begin{aligned} \varPi (c,u)&= \frac{1}{\rho } H_\text {control}(c,u) = \frac{1}{\rho } \left( 4\rho \mu (1-c)^2\chi - 1 {+} (u-1)^2\right) = h(c) {+} C (u{-}1)^2. \end{aligned}$$

(57)

If $c=\hat{c}$ is an indifference point, there are values $\hat{u}^{(1)}<\hat{u}^{(2)}$ such that the trajectories starting at $(\hat{c},\hat{u}^{(i)})$, for $i=1,2$, are both optimal and have both the same value. Note that the trajectory through $(\hat{c},\hat{u}^{(1)})$ goes to the right, and that through $(\hat{c},\hat{u}^{(2)})$ goes to the left. As

$$\begin{aligned} \varPi \left( \hat{c},\hat{u}^{(1)}\right) = \varPi \left( \hat{c},\hat{u}^{(2)}\right) , \end{aligned}$$

it follows that

$$\begin{aligned} \left| \hat{u}^{(1)} - 1\right| = \left| \hat{u}^{(2)} - 1\right| . \end{aligned}$$

Consider a fixed value of $\rho$ and two values $\mu _1$, $\mu _2$ of $\mu$ such that $\mu _2=(9/8)\mu _1$; that is, $(\mu _1,\rho )$ describes a partial collusion situation, and $(\mu _2,\rho )$ is the corresponding full collusion situation.

Assume first that there is an indifference point in the partial collusion problem; denote this point as $\hat{c}_1$, and the corresponding values of u as $\hat{u}_1^{(i)}$, $i=1,2$. Then

$$\begin{aligned} \hat{u}_{1}^{(1)} < \hat{u}_{1}^{(2)} \quad \text {and}\quad \left| \hat{u}_{1}^{(1)} - 1\right| = \left| \hat{u}_{1}^{(2)} - 1\right| . \end{aligned}$$

(58)

We have seen in the proof of Proposition 5.2 that if a partial collusion and a full collusion trajectory either go both to $e_1$ or to ${{e}}^1_-$ and ${{e}}^2_-$, respectively, the full collusion trajectory intersects a line $c= constant$ at a larger u-value than the partial collusion one. Denote the intersection of the full collusion trajectory going to $e_-^2$ with the line $c=\hat{c}_1$ by $(\hat{c}_1,\hat{u}^{(2)}_{2})$. We have that $\hat{u}^{(2)}_2>\hat{u}^{(2)}_1$, and therefore also

$$\begin{aligned} \left| \hat{u}^{(2)}_{2} - 1\right| > \left| \hat{u}^{(2)}_{1} - 1\right| . \end{aligned}$$

(59)

We argue by contradiction. Assume that the threshold $\hat{c}_2$ in the full collusion case exists and is below the threshold in the partial collusion case, then the full collusion trajectory going right, that is, to $e_1$, has to intersect the line $c=\hat{c}_1$ in a point $(\hat{c}_1,\hat{u}^{(1)}_2)$. Moreover, this trajectory has to be optimal at $\hat{c}_1$. Using (57), this implies that

$$\begin{aligned} \left| \hat{u}^{(1)}_{2} - 1\right| > \left| \hat{u}^{(2)}_2 - 1\right| . \end{aligned}$$

(60)

Finally, the full collusion trajectory has to be above the partial collusion trajectory going to $e_1$, implying

$$\begin{aligned} \left| \hat{u}^{(1)}_2 - 1\right| < \left| \hat{u}^{(1)}_{1} - 1\right| . \end{aligned}$$

(61)

Combining inequalities (58)–(61) implies

$$\begin{aligned} \left| \hat{u}^{(2)}_{2} - 1\right|>\left| \hat{u}^{(2)}_{1} - 1\right| = \left| \hat{u}^{(1)}_{1} - 1\right|>\left| \hat{u}^{(1)}_2 - 1\right| >\left| \hat{u}^{(2)}_{2} - 1\right| . \end{aligned}$$

But this is a contradiction. The proof in situation that the threshold is a repeller is similar and will be omitted. $\square$