Study of a One-Dimensional Optimal Control Problem with a Purely State-Dependent Cost

Abstract

A one-dimensional optimal control problem with a state-dependent cost and a unimodular integrand is considered. It is shown that, under some standard assumptions, this problem can be solved without using the Pontryagin maximum principle, by simple methods of the classical analysis, basing on the Tchyaplygin comparison theorem. However, in some modifications of the problem, the usage of Pontryagin’s maximum principle is preferable. The optimal synthesis for the problem and for its modifications is obtained.

Appendix

Here we consider some examples from the literature and show how they can be reduced to the above problems. For convenience, we will use our notation which may differ from the originals.

The Solow-Shell Model of Optimal Consumption–Accumulation Ratio

We begin with the famous problem of finding the optimal consumption–accumulation ratio in a one-sector production planning model (see e.g. [8, 13, 14, 18]). The problem is to maximize the total consumption

$$\begin{aligned} J=\int \limits ^{T}_{0}{e^{-rt}(1-u(t))\,g(x(t))\, dt}\longrightarrow \max \end{aligned}$$

(51)

subject to

$$\begin{aligned} {\dot{x}}= -\mu x + ug(x),\qquad x(0)=x_{0},\qquad x(T)\ge x_{T},\qquad u\in [0,1], \end{aligned}$$

(52)

where $g'(x)>0,\; g''(x) <0,$ and the parameters $r,x_{0},x_{T}\ge 0.$ Here x is the capital stock, g(x) the productivity function, u the rate of accumulation, and $1-u$ the consumption rate. (The case with a fixed $x(T)=x_{T}$ is more simple;  see “The standard case”).

Substituting $u\,g(x)= {\dot{x}}+\mu x$ into the cost and integrating the term $-e^{-r t}\dot{x}$ by parts, we rewrite the cost in the form

$$\begin{aligned} J=\int \limits ^{T}_{0}{e^{-rt}(g(x)-(r+\mu )x)\, dt}- e^{-rT}x(T)\rightarrow \max , \end{aligned}$$

(53)

which is of type (18) with $\Phi (x)=f(x)-(r+\mu )x$ and $S(x)= -e^{-rT}x.$ Obviously, $\Phi$ is unimodular with the maximum value at the point $x^*$ determined by the relation $f^{\prime }(x^*)= r+\mu ,$ which is exactly the classical golden rule of capital accumulation. Assume also that $u^* =\mu / g(x^*) \in (0,1).$ According to Theorem 3, here the optimal control is

$$\begin{aligned} {\hat{u}}=(sign^{+} (x_{0}-x^*),\,u^{*},\,sign^{+} (x_{T}-x^{*})) \end{aligned}$$

with two switchings at $t_{1}$ and $t_{2}.$ (Here $sign^+(z) = \partial (z^+)$ equals 1  if $z>0,$ and 0  if $z<0.$ If $x_{0}=x^*$ or $x_T=x^*,$ the corresponding interval disappears.) The first switching moment $t_{1}$ is determined by (14). The moment $t_{2}$ depends on the given value of $x_{T}$:

(1):

If $x_{T}\ge x^{*},$ then obviously $x(T)=x_{T}$ and $t_{2}$ is determined by (15).

(2):

If $x_{T}<x^{*},$ then, as was shown in “Other modifications of the problem”, we set $\psi = e^{-rT}\eta$ and come to the following boundary value problem with respect to x and $\eta$:

$$\begin{aligned} {\dot{x}}= & {} -\mu x-g(x),\qquad x(t_{2})=x^{*},\qquad x(T) \ge x_T,\\ {\dot{\eta }}= & {} (\mu +r)\eta -\Phi '(x),\quad \; \eta (t_{2})=0,\quad \eta (T)=\alpha -1, \end{aligned}$$

where $\alpha \ge 0$ satisfies the complementary slackness condition $\alpha (x(T)-x_{T})=0.$

Since these equations are time-independent, our task is to find the value $L=T-t_{2} >0$ such that the solution of the Cauchy problem with $x(0)=x^{*}$ and $\eta (0)=0$ has terminal values $x(L)\ge x_{T}$ and $\eta (L)=\alpha -1.$ It is clear that L is the first moment where either $x(L)=x_{T}$ or $\eta (L)=-1.$

Note that the advantage of representation of the cost in the form (53) is that the problem with the cost in its initial form (51) can hardly be solved in a simple way, even with the help of PMP, because the costate equation looks here rather nontrivial and the usual analysis of PMP involves essential difficulties (see e.g., [8]).

The Nerlove-Arrow Advertising Model

The problem is (see [14]) to maximize the present value of net revenue streams discounted at a fixed rate r : 

$$\begin{aligned} J=\int \limits ^{T}_{0}{e^{-rt}(\pi (x)-u)\, dt}\longrightarrow \max \end{aligned}$$

(54)

subject to

$$\begin{aligned} {\dot{x}}=u-\delta x,\qquad x(0)=x_{0},\qquad u\in [0,N], \end{aligned}$$

(55)

where x is the stock goodwill, u is the advertising effort and $\pi (x)$ is the total revenue net of production costs, which satisfies $\pi '(x)> 0$ and $\pi ''(x)<0$ for all $x \ge 0.$ The parameters $x_0,\, N,\,\delta >0$ and $\pi '(+\infty )< r < \pi '(0).$ Like before, substituting $u(t)= {\dot{x}}+ \delta x$ in (54), we represent the cost in the form (18):

$$\begin{aligned} J=\int \limits ^{T}_{0}{e^{-rt}(\pi (x)-rx)\,dt}\,-\,e^{-rT}x(T) \longrightarrow \max , \end{aligned}$$

with $\Phi (x)= \pi (x)-rx$ and $S(x)= -e^{-rT}x.$ Obviously, $\Phi$ is unimodular with maximum at the point $x^* >0$ determined by the golden rule: $\pi '(x^*) =r.$ Assuming that $u^*= \delta x^* <N,$ we have a problem of type (18)–(19). Thus, the optimal control is described in Theorem 3. Since $\psi (T) = -e^{-rT}<0,$ we have $u=0$ on $[t_{2},T],$ which implies that $t_{2}$ does not depend on N and can be found by applying the same method as in the previous example. The first switching moment $t_{1}$ is determined by (14). If $x_{0}\ge x^{*},$ then $u=0$ on $(0,t_{1}),$ where $t_{1}$ does not depend on N. However, if $x_{0}<x^*,$ then $u=N$ on $(0,t_{1}),$ where $t_{1}$ now depends on N. If $N\rightarrow \infty ,$ then obviously $t_{1}\rightarrow 0+,$ so that for the case of unbounded $u\ge 0$ we obtain an impulse control generating the jump of the state variable from $x(0)=x_{0}$ to $x(t)=x^{*}$ for $t\in (0,t_{2}].$

The Gordon–Schaefer–Goh Fishing Problem

The problem is to maximize the total catch

$$\begin{aligned} J=\int \limits ^{T}_{0}{u(t)\,dt}\longrightarrow \max , \end{aligned}$$

(56)

subject to

$$\begin{aligned} {\dot{x}}=\, x-\frac{x^{2}}{2b}-u,\qquad x(0)=x_{0},\qquad u\in [0,M], \end{aligned}$$

(57)

where the parameters $b,\, x_{0},\, M>0$ are such that $b<2M.$

Here x is the amount of fish in the ocean, and u is the catching rate of the fishing fleet. The original problem in [7, 15] contains the state constraint $x(t)\ge 0,$ which, as is well known, may cause serious difficulties in the solution. Therefore, let us replace it by the terminal constraint $x(T)\ge 0$ with the hope that then the optimal trajectory ${\hat{x}}$ would automatically satisfy ${\hat{x}}(t)\ge 0$ for all t. Expressing again u through $\dot{x}$ and x from (57), we rewrite the cost in the form

$$\begin{aligned} J=\int \limits ^{T}_{0}{\left( x-\frac{x^{2}}{2b}\right) \,dt}\, -\, x(T)\; \longrightarrow \max . \end{aligned}$$

(58)

Here $x^*= b$ and $u^* = b/2< M,$ so $u^*$ is an interior point of the admissible control set. Thus, we have a problem of type (18)–(19). Note however that the function $f(x)= x- x^{2}/{2b}\,$ is not monotone, so, strictly speaking, we cannot directly use the results of “Other modifications of the problem”. Nevertheless, here we can argue as follows.

Note first that according to the Filippov theorem, this problem has a solution. Let (x, u) be an optimal pair. Then is remains optimal in the problem with the fixed endpoint x(T). Since this reduced problem is of type (1)–(2), we have no need to require the function f to be monotone, and may conclude that the optimal control has the form (5) under the corresponding rescaling, with switching points $t_{1}\le t_{2},$ where $t_{1}$ is defined by analogy with (14): 

$$\begin{aligned} t_1\; = \int _{x_0}^{x^*} \frac{dx}{f(x)-u}, \end{aligned}$$

where $u=0$ if $x_0 < x^*,$ and $u=M$ if $x_0 > x^*.$ (If $x_0 = x^*,$ then simply $t_1 =0.)$ So, our task is just to find $t_{2}.$ To this end, we apply PMP for the problem with a variable x(T).  Here

$$\begin{aligned} H= & {} (\psi +1)\left( x-\frac{x^{2}}{2b}\right) -\psi u,\\ {\dot{\psi }}= & {} (\psi +1) \left( x/{b}\,-1 \right) ,\qquad \psi (T)=\alpha -1, \end{aligned}$$

where $\alpha \ge 0\,$ and $\,\alpha \, x(T)=0.$

Setting $\eta =\psi +1,$ we obtain a homogeneous equation

$$\begin{aligned} {\dot{\eta }}= \eta \left( \frac{x}{b}-1\right) , \qquad \eta (T)=\alpha . \end{aligned}$$

(59)

Since $u = u^*$ on $[t_1, t_2],$ we have $\psi =0$ there;  in particular $\psi (t_{2})=0,$ hence $\eta (t_{2})=1.$ If $\alpha =0,$ then $\eta (t)\equiv 0$ by (59), a contradiction. Therefore, $\alpha >0,$ which implies $x(T)=0.$ Then $u=M$ on $[t_{2},T],$ and so, $t_2$ is uniquely determined by the conditions $\, x(t_{2})=x^*\,$ and $\,x(T)=0.$

The Cliff–Vincent Model of Optimal Fish Harvest

The problem is (see [6, 7]) to minimize capital investments in the fishery business:

$$\begin{aligned} J=\int \limits ^{T}_{0}{(ax+bu-xu)\,dt}\longrightarrow \min \end{aligned}$$

(60)

subject to

$$\begin{aligned} {\dot{x}}=x(\gamma -x)-ux,\;\quad x(0)= x_{0}, \;\quad u\in [0,M], \end{aligned}$$

(61)

with the parameters $a,b,\gamma >0$ and $x_{0}\in (0,\gamma ).$

Here x is the density of fish in the pond, ax the cost of supporting the fish, bu the cost of its catching, and xu the amount of money which can be earned by selling fish on the market. Using a similar reformulation as in the previous example, we rewrite the cost in the form

$$\begin{aligned} J=\int \limits ^{T}_{0}{\left( -x^{2} +( b+\gamma -a)\,x\right) \, dt}\, +\, (b\, \ln x(T)-x(T))\longrightarrow \max , \end{aligned}$$

and again have a problem of type (18)–(19), now with $\Phi (x) = -x^{2}+ (b+\gamma -a)\,x$ and $S(x) = b\, \ln x(t)- x.$ Here the function S is concave but non-monotone, with the maximum at $x^{**} =b.$ One can consider two separate problems with the terminal constraint $x(T)\ge b$ and $x(T)\le b.$ Each of these problems falls into the above studied class. Comparing the optimal values of the cost in each of them, one would finally obtain the solution of the initial problem.

A similar approach can be as well applied to the Vidale–Wolf advertising model [12, 14], to the model of exploitation of a renewable resource from [17, p. 126], and to the Sethi model of epidemic control [14].