Optimal production management when there is regime switching and production constraints

Abstract

We study a production problem in which the cumulative consumer demand for an item follows a Brownian motion with drift, with both the drift and the variance parameters modulated by a continuous-time Markov chain that represents the regime. The company wants to maintain the inventory level as close as possible to a target inventory level, but there is a linear cost of production. We assume that the production rate is nonnegative. The company is penalized for deviations from the inventory target level and the cost of production, and the objective is to minimize the total discounted penalty costs. We consider two models. In the first model, there is no upper bound for the production rate, and in the second model there is an upper bound for the production rate. We solve both problems analytically and obtain the optimal production policy and the minimal total expected discounted cost. Our solutions allow us to obtain interesting managerial insights.

1 Introduction

We assume that the cumulative demand for an item follows a Brownian motion with drift modulated by a continuous-time Markov chain that alternates among regimes. We allow both the drift and the volatility to be affected by this continuous-time Markov chain, which represents the regime of the economy or the state of the world. In an application, one regime may represent a recessionary period with a low demand rate and the other may represent an expansionary period with a high demand rate. In another application, one regime may represent a high demand rate for a medical item because of an epidemic, and the other regime may represent a low demand rate for that medical item when there is not an epidemic.

The cost of having too much inventory above a preferred target level is due to inventory costs, that is, the cost of the actual maintenance of the inventory. The cost of having too little inventory is due to the perceived likelihood of depleting the inventory, which could lead to a loss of sales as well as a loss of goodwill. The perceived likelihood of such an event increases with a lower level of inventory. Thus, the management’s objective is to maintain the inventory level as close as possible to a fixed target level, so there is a running cost associated with the difference between the inventory level and its target. In addition to that cost, there is a linear cost of production, and the production rate is constrained to be nonnegative. Thus, there is a running cost of production. We consider two models. In the first model, there is no upper bound for the production rate, whereas in the second model there is an upper bound for the production rate. For instance, the second model occurs when a pharmaceutical company produces a vaccine during a pandemic. Although there is a high demand for the vaccine produced by this pharmaceutical company, the production capacity is bounded. That bound for the production capacity was evident in the recent covid crisis. The first model generates a singular stochastic control problem with regime switching, while the second model generates a classical stochastic control problem with regime switching. We determine for both models the optimal production policies by applying the dynamic programming method with regime switching. We obtain an analytical solution for both problems. This is of barrier-type and prescribes to increase the production only when the inventory level is sufficiently small, in particular when the inventory level is smaller than or equal to a regime-dependent barrier. On one hand, in the singular stochastic control model, the resulting optimal production policy combines jumps to singularly continuous displacements and it is mathematically described through the solution to a so-called Skorokhod reflection problem. On the other hand, in the classical control model, the optimal policy is of the “bang-bang” type and switches between no-production and production at the maximal rate.

The regime-dependent boundaries that trigger the optimal production plan are endogenously determined in both models through the so-called smooth-fit condition, which results in a system of highly nonlinear algebraic equations. The dependency of those trigger levels on the models’ parameters allows us to draw interesting economic conclusions (see Sects. 3.4 and 4.4, and 5). For example, while the optimal production boundaries in the unbounded from above model decrease in both regimes when the volatility increases, this is not always the case in the bounded from above model. Indeed, our numerical analysis of Sect. 4.4 reveals that while the production threshold in each regime is decreasing with respect to the volatility in regime 1 (low-demand regime), an increase in the volatility in regime 2 (large-demand regime) leads to an increase of the production threshold in both regimes and therefore to earlier production. Given that the rate at which production is performed is capped, the manager of the company decides to produce more frequently in both regimes even when the volatility of regime 2 becomes larger, somewhat to hedge against a change to regime 1 characterized by low demand. Such a result differs from the classical message related to the “value of waiting” in problems of investment under uncertainty and represents a novel managerial result of our analysis.

There is extensive literature on the theory of optimal production control in continuous-time when the demand is deterministic; see, for instance, Sect. 6 of Sethi (2019) for some references. S Sethi and Thompson (1981) and Chapter 12 of Sethi (2019) consider the case when the demand is stochastic but the demand rate is constant. Fleming et al. (1987) allow the demand rate to be stochastic and model the demand as a continuous-time Markov chain with a finite state space. While a continuous-time Markov chain model for the demand rate is more realistic than a deterministic model, it still is not ideal. A continuous-time Markov chain model assumes that the demand can take only a finite number of values and that it remains constant for long periods of time. It is not suitable for modeling the demand for items such as gold and oil. Bensoussan et al. (1984) allow the demand to be stochastic and model it as a Brownian motion with drift. They assume that there is no regime switching, or equivalently that there is only one regime. Cadenillas et al. (2013) allow the demand to be stochastic and model the cumulative demand as a Brownian motion with both the drift and the variance parameters modulated by a continuous-time Markov chain that represents the regime of the economy. They also study the case in which the cumulative demand is a geometric Brownian motion with both the drift and the variance parameters modulated by a continuous-time Markov chain that represents the regime of the economy. Covei et al. (2022a) and Covei (2023) also studied a production problem with regime switching, while Covei et al. (2022b) studied a random time-horizon not discounted optimal inventory problem with N produced goods (without regime switching). From all the above papers, only Bensoussan et al. (1984) make the realistic assumption that the production rate is nonnegative. This motivates us to study a production problem in which the demand process changes continuously over time and is allowed to take a continuum of values, and the production rate is nonnegative. In addition, we do allow the demand to depend on macroeconomic conditions or states of the world that remain in effect for extended periods of time. Besides the production rate to be bounded below by 0, we consider two cases: the production rate is unbounded from above and the production rate is bounded from above.

We obtain, for the first time in the literature, an analytical solution to an optimal production problem in which the production rate is nonnegative, and the cumulative demand is modeled by a Brownian motion with both the drift and the variance modulated by a continuous-time Markov chain.

This paper is organized as follows. We describe the production model and management’s objectives in Sect. 2. In Sect. 3 we study and solve the problem when the production rate is unbounded from above, and in Sect. 4 we study and solve the problem when the production rate is bounded from above. In Sect. 5, we compare the results for the case in which the production rate is unbounded from above with the case in which the production rate is bounded from above. We present our conclusions in Sect. 6.

2 The management model with nonnegative production and regime switching

We consider a company in which the uncertainty in the demand is modeled by a Brownian motion W and a finite-state continuous-time Markov chain $\epsilon$. The Brownian motion W models minuscule and continuous uncertain movements in the demand, while the finite-state continuous-time Markov chain $\epsilon$ models uncertain long-term conditions. Thus, $\epsilon$ models the regime, which can be the regime of the economy or the state of the world. At every point in time, the management of the company has information about these two sources of uncertainty, from the beginning up to the present. Formally, we consider a probability space $(\Omega ,{{\mathcal {F}}},{{\mathbb {P}}})$ together with a standard Brownian motion $W=\{W_t,\,t\ge 0\}$ and an observable finite-state continuous-time Markov chain $\varepsilon =\{\varepsilon _t,t\ge 0\}$. We assume that for every $t\ge 0$: $\varepsilon _t=\varepsilon (t)\in {\mathcal S}:=\{1,\cdots ,N\}$, $N\ge 2$. Here, $\varepsilon (t)$ represents the regime at time $t\ge 0$.

We also assume that $\varepsilon$ and W are independent, and that the Markov chain $\varepsilon$ has a strongly irreducible generator $Q=[\,q_{ij}\,]_{N\times N}$, where $q_{ii}=-\lambda _i<0$ and $\sum _{j\in {{\mathcal {S}}}}q_{ij}=0$ for every $i\in {{\mathcal {S}}}$. We denote by ${{\mathbb {F}}}=\{{{\mathcal {F}}}_t,t\ge 0\}$ the $\mathbb P$-augmentation of the filtration $\{{\mathcal F}_t^{({W,\varepsilon })},t\ge 0\}$ generated by the Brownian motion and the Markov chain, where ${\mathcal F}_t^{({W,\varepsilon })}:=\sigma \{W_s,\varepsilon _s:0\le s\le t\}$ for every $t\ge 0$.

We assume the cumulative demand process $D=\{D_t,t\ge 0\}$ satisfies the stochastic differential equation

$$\begin{aligned} dD_t\,=\,\mu _{\varepsilon (t)}\,dt+\sigma _{\varepsilon (t)}\,dW_t, \end{aligned}$$

where, for each $i \in {\mathcal {S}}$, the parameter $\mu _i$ represents the drift of the demand and $\sigma _i >0$ represents the volatility of the demand. We allow the change dD in the cumulative demand to take negative values, which represents the return of items

The manager of a firm wants to control the inventory of a given item. Thus, the manager selects a production strategy given by an adapted process $P=\{P_t,t\ge 0\}$, where $P_t$ represents the cumulative production up to time t. We assume that P is a nonnegative and nondecreasing stochastic process with sample paths that are left-continuous with right limits. Then, the inventory process $X=\{X_t,t\ge 0\}$ satisfies the dynamics

$$\begin{aligned} X_{t} \; = \; x + P_t -D_t, \end{aligned}$$

or equivalently

$$\begin{aligned} dX_t\,= dP_{t} - dD_{t} \,= - \mu _{\varepsilon (t)}\,dt- \sigma _{\varepsilon (t)}\,dW_t + dP_{t}, \end{aligned}$$

where the initial inventory level is $X_{0}=x$.

At every point in time t, the management of the company wants to keep the inventory level $X_t$ as close as possible to a target level ${{\mathcal {I}}}_{\varepsilon (t)}$, so there is a running cost associated with the difference between the inventory level and its target. Besides, there is a linear cost of production $k_{\varepsilon (t)}>0$. Zhou et al. (2007) also assume a linear cost of production. Hence, the management of the company wants to select a production policy that minimizes the expected total discounted costs.

Problem 1

Select a production control $P=\{P_t,t\ge 0\}$ that minimizes

$$\begin{aligned} J(x,i;P):=\,E_{x,i}\left[ \int _{0}^{\infty } e^{-\delta t} \alpha _{\varepsilon (t)} (X_{t} - {\mathcal I}_{\varepsilon (t)})^{2}dt + \int _0^{\infty }e^{-\delta t} k_{\varepsilon (t)} dP_t\,\right] , \end{aligned}$$

where $\delta > 0$ is a discount rate, $E_{x,i}$ represents the expectation conditioned to $X_0=x$ and $\varepsilon _0=\varepsilon (0)=i$, and $\alpha _{\varepsilon (t)}$ represents the importance that the inventory is far away from its target relative to the cost of production. Here, $\alpha _{\varepsilon (t)} >0$ for every $t \ge 0$.

We are assuming that the cumulative production is nondecreasing, or equivalently that 0 is a lower bound for the production rate. In this paper, we consider two different cases for production control. First, we consider the case that the production rate is unbounded from above, and then we consider the case in which the production rate is bounded from above. In both cases, we apply the dynamic programming method to obtain an analytical solution for the optimal production control and the value function. Moreover, we compare the solutions for these two cases (see Sect. 5 for details).

3 Optimal production control when production rates are unbounded from above

In this section, we study the case in which the production rate is unbounded from above. In this case, Problem 1 is modeled as a singular stochastic control problem with the following admissible strategies.

Definition 1

An ${\mathbb {F}}$-adapted, nonnegative, and nondecreasing control process $P:[\,0,\infty )\times \Omega \rightarrow [\,0,\infty )$, with sample paths that are left-continuous with right limits, is called an admissible control if

$$\begin{aligned} E_{x,i}\left[ \,\int _0^{\infty }\,e^{-\delta s} \alpha _{\varepsilon (s)} (X_{s} - {{\mathcal {I}}}_{\varepsilon (s)})^{2}ds + \int _0^{\infty }e^{-\delta s} k_{\varepsilon (s)} dP_s \right] < \infty , \end{aligned}$$

(1)

The set of all admissible controls is denoted by ${{\mathcal {A}}}_U$.

Under these conditions, Problem 1 takes the following form.

Problem 2

The management wants to obtain the optimal production policy ${\hat{P}} \in {{\mathcal {A}}}_U$ that solves the problem

$$\begin{aligned} V(x,i) \;:= & {} \; \inf _{P\in \,{{\mathcal {A}}}_U} J(x,i;P) \; \\= & {} \; \inf _{P \in \,{{\mathcal {A}}}_U} E_{x,i}\left[ \,\int _0^{\infty }\,e^{-\delta s} \alpha _{\varepsilon (s)} (X_{s} - {{\mathcal {I}}}_{\varepsilon (s)})^{2}ds + \int _0^{\infty }e^{-\delta s} k_{\varepsilon (s)} dP_s \right] . \end{aligned}$$

Notice that $0\le V(x,i) \le M(1+|x|^2)$, for some $M>0$, since $P\equiv 0$ is suboptimal a priori. We observe that Problem 2 is a problem of stochastic singular control with regime switching. We are not requiring (absolute) continuity for the control P. We define $\Lambda :=\{t\ge 0: P_{t+}\ne P_t\}$, the set of times where P has a discontinuity. The set $\Lambda$ is countable because P can jump only a countable number of times during the interval $[\,0,\infty )$. We will denote by $P^d$ the discontinuous part of P, that is $P_t^d:=\sum _{0\le s\le t,\,s\in \Lambda }(P_{s^+}-P_s)$; and by $P^c$ the continuous part of P, that is $P_t^c:=P_t-P_t^d$.

3.1 Verification Theorem

Let $\psi :(-\infty ,\infty )\times {{\mathcal {S}}}\rightarrow {\mathbb {R}}$ be a function such that $\psi (\cdot , i) \in C^2({\mathbb {R}})$ for every $i \in {\mathcal {S}}$ and define the operators ${\widetilde{L}}_i$, for each $i\in {{\mathcal {S}}}$, by

$$\begin{aligned} {\widetilde{L}}_i\,\psi \,:=\, \frac{1}{2}\,\sigma _i^2\,\psi '' - \mu _i\,\psi '-\delta \,\psi . \end{aligned}$$

Here, and in the sequel, $\psi '$, $\psi ''$, $\psi '''$, etc., denote the partial derivatives of $\psi (\cdot , i)$ for any $i\in {\mathcal {S}}$.

For a function $v:(-\infty ,\infty )\times {{\mathcal {S}}}\rightarrow {\mathbb {R}}$, consider the Hamilton–Jacobi–Bellman equation

$$\begin{aligned} \min \left\{ {\widetilde{L}}_iv(x,i)-\lambda _i\,v(x,i)+\sum _{j\ne i}q_{ij}\,v(x,j) + {\alpha }_{i} (x - {{\mathcal {I}}}_{i})^{2} ,\,\,\, v'(x,i) + k_{i} \right\} \,=\,0, \end{aligned}$$

(2)

where $x\in (-\infty ,\infty )$ and $i\in {{\mathcal {S}}}$. We observe that (2) defines, for each $i\in {{\mathcal {S}}}$, the continuation region

$$\begin{aligned} {{\mathcal {C}}}^{(v)}(i):= & {} \left\{ \,x\in (-\infty ,\infty ):\, {\widetilde{L}}_iv(x,i)-\lambda _i\,v(x,i)\nonumber \right. \\{} & {} \left. +\sum _{j\ne i}q_{ij}\,v(x,j) + {\alpha }_{i} (x - {{\mathcal {I}}}_{i})^{2} =0\,\,\,\, \text{ and } \,\, k_{i} + v'(x,i) > 0\,\right\} \end{aligned}$$

and the intervention region

$$\begin{aligned} {\Sigma \,}^{(v)}(i):= & {} \left\{ \,x\in (-\infty ,\infty ):\, {\widetilde{L}}_iv(x,i)-\lambda _i\,v(x,i)\right. \\{} & {} \left. +\sum _{j\ne i}q_{ij}\,v(x,j) + {\alpha }_{i} (x - {{\mathcal {I}}}_{i})^{2} > 0\,\,\,\, \text{ and } \,\, k_{i} + v'(x,i)=0\,\right\} . \end{aligned}$$

Thus, it is possible to construct a control process $P^v$ associated with v in the following sense.

Definition 2

An ${\mathbb {F}}$-adapted, nonnegative, and nondecreasing control process $P^v$ is associated with the function v above if it is such that

$$\begin{aligned}&(i)&X^{P^v}_t:=\, x - \int _0^t\mu _{\varepsilon (s)}\,ds - \int _0^t\sigma _{\varepsilon (s)}\,dW_s + P^v_t \nonumber \\&(ii)&X^{P^v}_t\in \,\,\overline{{\mathcal C}(\varepsilon _t)},\quad \text{ for } \text{ every } \quad t \in (0,\infty ),\quad {{\mathbb {P}}}-a.s.,\nonumber \\&(iii)&\int _0^{\infty }\!I_{\{X^{P^v}_s\in \,{\mathcal C}(\varepsilon _s)\}}\,dP^v_s\,\, =\,\,0,\quad {{\mathbb {P}}}-a.s. \end{aligned}$$

(3)

Notice that if a process $P^v$ as in Definition 2 exists, then the triple $(X^{P^v}, \epsilon , P^v)$ solves a so-called Skorokhod reflection problem at the topological boundary of the region ${\mathcal {C}}^v(i) \times \{1,2\}$. We shall see in the subsequent sections that for Problem 2, a process $P^v$ satisfying (3) can be indeed explicitly constructed.

Next, we present the Verification Theorem that gives sufficient conditions of optimality for the solution of Problem 2.

Theorem 1

Let $v(\cdot ,i)\in C^1((-\infty ,\infty )) \cap C^2( (-\infty ,\infty ) {\setminus } N_i)$, $i\in {{\mathcal {S}}}$, where $N_i$ are finite subsets of $(-\infty ,\infty )$. Let $v(\cdot ,i)$, $i\in {{\mathcal {S}}}$, be a convex function on $(-\infty ,\infty )$. Suppose that $v(\cdot ,i)$, $i\in {{\mathcal {S}}}$, have quadratic growth on $(-\infty ,\infty )$. Suppose that the function $v(\cdot ,i)$, $i\in {{\mathcal {S}}}$, satisfies the Hamilton–Jacobi–Bellman Eq. (2) for almost every $x \in (-\infty ,\infty )$, $i\in {{\mathcal {S}}}$, and consider the stochastic control $P^v$ associated with v. Then, the process ${\hat{P}}=\{{\hat{P}}_t,t\ge 0\}=:P^{v}$ is the optimal production control for Problem 2. Furthermore, v is the value function for Problem 2.

Proof

Consider an admissible control P and the corresponding semimartingale

$$\begin{aligned} X_t\,=\,x - \int _0^t\mu _{\varepsilon (s)}ds - \int _0^t\sigma _{\varepsilon (s)}dW_s + P_t^c + P_t^d. \end{aligned}$$

Consider also the function $f(\cdot ,\cdot ,i)$, $i\in {{\mathcal {S}}}$, defined by $f(t,x,i)=e^{-\delta t}v(x,i)$. We get

$$\begin{aligned} df(t,X_t,\varepsilon _t)= & {} \,\, \left( \frac{1}{2}\,\sigma _{\varepsilon (t)}^2\,f_{xx}(t,X_t,\varepsilon _t) - \mu _{\varepsilon (t)}\,f_x(t,X_t,\varepsilon _t)+f_t(t,X_t,\varepsilon _t)\right) dt\\{} & {} \,\,\,\,\,-\,\,\,f_x(t,X_t,\varepsilon _t)\,\sigma _{\varepsilon (t)}\,dW_t\,\,\, +\,\,\,f_x(t,X_t,\varepsilon _t)\,dP_t^c\\{} & {} \,\,\,\,\,+\,\,\left( \frac{}{}f(t,X_{t+},\varepsilon _t)-f(t,X_t,\varepsilon _t)\right) I_{\{t\,\in \,\Lambda \}}\\{} & {} \,\,\,\,\,+\,\,\left( -\lambda _{\varepsilon (t)}f(t,X_t,\varepsilon _t)+\sum _{j\ne \varepsilon (t)}q_{\varepsilon (t)j}\,f(t,X_t,j)\right) dt\,\,+\,\,dM_t^f\\= & {} \,\,\,\,e^{-\delta t}\left( {\widetilde{L}}_{\varepsilon (t)}v(X_t,\varepsilon _t)-\Delta _Qv(X_t,\varepsilon _t)\right) dt\,\, - \,\,\sigma _{\varepsilon (t)}\,e^{-\delta t}v'(X_t,\varepsilon _t)\,dW_t\\{} & {} \,\,\,\,+\,\,\,e^{-\delta t}v'(X_t,\varepsilon _t)\,dP_t^c\,\,+\,\,e^{-\delta t}\left( v(X_{t+},\varepsilon _t)-v(X_t,\varepsilon _t)\right) I_{\{t\,\in \,\Lambda \}}\,\, +\,\,dM_t^f, \end{aligned}$$

where $\Delta _{Q}v(X_t,\varepsilon _t):=\lambda _{\varepsilon (t)}v(X_t,\varepsilon _t)-\sum _{j\ne \varepsilon (t)}q_{\varepsilon (t)j}\,v(X_t,j)$ and $\Lambda$ has been defined below Problem 3.1. Here, the process $M^f=\{M^f_t,t\ge 0\}$ is a square-integrable martingale when $f(\cdot ,\cdot ,i)$, $i\in {{\mathcal {S}}}$, are bounded. We observe that $v(\cdot ,i)$ and $v'(\cdot ,i)$, $i\in {{\mathcal {S}}}$, are not necessarily bounded. To take care of that, we perform a localization procedure as follows. Let a and b be real numbers satisfying $-\infty< a<X_0=x<b<+\infty$, and define $\tau _a:=\inf \{t\ge 0:X_t=a\}$, $\tau _b:=\inf \{t\ge 0:X_t=b\}$ and $\tau :=\tau _a\wedge \tau _b$. Then, for every time $t\in [\,0,\infty )$, we have

$$\begin{aligned} e^{-\delta (t\wedge \tau )}v(X_{(t\wedge \tau )^+},\varepsilon _{t\wedge \tau })= & {} v(X_0,\varepsilon _0)+ \int _0^{t\wedge \tau }\!\!\!\!e^{-\delta s}\left( {\widetilde{L}}_{\varepsilon (s)}v(X_s,\varepsilon _s)-\Delta _Qv(X_s,\varepsilon _s)\right) ds \nonumber \\{} & {} -\,\,\quad \int _0^{t\wedge \tau }\!\!\!\!\sigma _{\varepsilon (s)}e^{-\delta s}v'(X_s,\varepsilon _s)\,dW_s + \int _0^{t\wedge \tau }\!\!\!\!e^{-\delta s}v'(X_s,\varepsilon _s)\,dP^c_s\nonumber \\{} & {} +\!\!\!\sum _{0\le s\le t\wedge \tau ,\,s\in \Lambda }\!\!\!\!\!\!e^{-\delta s}\left( v(X_{s^+},\varepsilon _s)-v(X_s,\varepsilon _s)\right) +M_{t\wedge \tau }^f-M_0^f. \end{aligned}$$

(4)

Taking conditional expectation, we have

$$\begin{aligned}{} & {} E_{x,i}\left[ e^{-\delta (t\wedge \tau )}v(X_{(t\wedge \tau )^+},\varepsilon _{t\wedge \tau })\right] \\{} & {} \quad =v(x,i)+ E_{x,i}\left[ \int _0^{t\wedge \tau }\!\!\!\!e^{-\delta s}\left( {\widetilde{L}}_{\varepsilon (s)}v(X_s,\varepsilon _s)-\Delta _Qv(X_s,\varepsilon _s)\right) ds\right] \\{} & {} \quad \quad - E_{x,i}\left[ \int _0^{t\wedge \tau }\!\!\!\!\sigma _{\varepsilon (s)}e^{-\delta s}v'(X_s,\varepsilon _s)dW_s\right] + E_{x,i}\left[ \int _0^{t\wedge \tau }\!\!\!\!e^{-\delta s}v'(X_s,\varepsilon _s)dP^c_s\right] \\{} & {} \quad \quad + E_{x,i}\left[ \sum _{0\le s\le t\wedge \tau ,\,s\in \Lambda }\!\!\!\!\!\!\!\!\!\!\!\!e^{-\delta s}\left( v(X_{s^+},\varepsilon _s)-v(X_s,\varepsilon _s)\right) \right] +E_{x,i}\left[ M_{t\wedge \tau }^f-M_0^f\right] \!. \end{aligned}$$

Equation (2) guarantees that ${\widetilde{L}}_{\varepsilon (t)}v(X_t,\varepsilon _t)-\Delta _Qv(X_t,\varepsilon _t) \ge - {\alpha }_{ \varepsilon (s) }( X_{s} - {{\mathcal {I}}}_{ \varepsilon (s) } )^{2}$. Moreover, $v'(x,i)\ge - k_{i}$ for $x\in (-\infty ,\infty )$ [recall Eq. (2)] and the Mean Value theorem imply that $v(y_1,i)-v(y_2,i)\ge - k_{i}(y_1-y_2)$ for every $y_1,y_2\in (-\infty ,\infty )$, $y_1>y_2$, and for every $i\in {{\mathcal {S}}}$. Hence, replacing $i=\varepsilon _t$, $y_1=X_t$ and $y_2=X_{t+}$, we obtain $v(X_{t+},\varepsilon _t)-v(X_t,\varepsilon _t)\ge - k_{ \varepsilon _t } (X_{t+}-X_t)$. We also note that $X_{s^+}-X_s= P_{s^+} - P_{s}$. Then, from the last displayed equation,

$$\begin{aligned}{} & {} E_{x,i}\left[ \,e^{-\delta (t\wedge \tau )}v(X_{(t\wedge \tau )^+},\varepsilon _{t\wedge \tau })\right] \\{} & {} \quad \ge v(x,i)\,-\,\,E_{x,i}\left[ \,\int _0^{t\wedge \tau }\!\!\sigma _{\varepsilon (s)}e^{-\delta s}v'(X_s,\varepsilon _s)dW_s\,\right] \, - \,E_{x,i}\left[ \,\int _0^{t\wedge \tau }\!\!e^{-\delta s}k_{ \varepsilon _s }dP^c_s\,\right] \\{} & {} \quad \quad - E_{x,i}\left[ \,\sum _{0\le s\le t\wedge \tau ,\,s\in \Lambda }\!\!\!\!\!\!e^{-\delta s} k_{ \varepsilon _s} \left( P_{s^+}-P_s\right) \,\right] \,+\,E_{x,i}\left[ \,M_{t\wedge \tau }^f-M_0^f\,\right] \\{} & {} \quad \quad -\,E_{x,i}\left[ \,\int _0^{t\wedge \tau }\!\!e^{-\delta s} {\alpha }_{ \varepsilon (s) }( X_{s} - {{\mathcal {I}}}_{ \varepsilon (s) } )^{2} ds\,\right] \\{} & {} \quad =v(x,i)\,-\,\,E_{x,i}\left[ \int _0^{t\wedge \tau }\!\!\!\!\!\!\sigma _{\varepsilon (s)}e^{-\delta s}v'(X_s,\varepsilon _s)\,dW_s\right] - E_{x,i}\left[ \int _0^{t\wedge \tau }\!\!\!\!\!\!e^{-\delta s} k_{ \varepsilon _s } dP_s\right] \\{} & {} \quad \quad +E_{x,i}\left[ M_{t\wedge \tau }^f\!\!-M_0^f\right] \! \\{} & {} \quad \quad -\,E_{x,i}\left[ \,\int _0^{t\wedge \tau }\!\!e^{-\delta s} {\alpha }_{ \varepsilon (s) }( X_{s} - {{\mathcal {I}}}_{ \varepsilon (s) } )^{2} ds\,\right] . \end{aligned}$$

We note that this inequality becomes equality for the stochastic control $P^v$ associated with v and, hence, for the admissible ${\hat{P}}$. Indeed, condition (ii) in (3) implies that $X^{P^v}_s\in \,\,{{\mathcal {C}}}(\varepsilon _s)$ Leb a.e. $s\in [\,0,t\wedge \tau )$ ${\mathbb {P}}$-a.e. and, hence,

$$\begin{aligned}{} & {} E_{x,i} \left[ \int _0^{t\wedge \tau }\!\!\!\!e^{-\delta s} \left( {\widetilde{L}}_{\varepsilon (s)}v(X^{P^v}_s\!\!,\varepsilon _s)-\lambda _{\varepsilon (s)}\,v(X^{P^v}_s\!\!,\varepsilon _s)\right. \right. \\{} & {} \left. \left. \quad \quad +\sum _{j\ne \varepsilon (s)}q_{\varepsilon (s)j}\,v(X^{P^v}_s\!\!,j) + {\alpha }_{ \varepsilon (s) }( X_{s} - {{\mathcal {I}}}_{ \varepsilon (s) } )^{2} \right) ds\right] =0. \end{aligned}$$

Furthermore, for the stochastic control $P^v$ associated with v,

$$\begin{aligned}{} & {} E_{x,i}\left[ \,\int _0^{t\wedge \tau }\!\!\!\!\!e^{-\delta s}\,v'(X^{P^v}_s\!\!,\varepsilon _s)\,d{(P^v})^c_s\,\right] + E_{x,i}\left[ \,\sum _{0\le s\le t\wedge \tau ,\,s\in \Lambda }\!\!\!\!\!\!\!\!\!\!\!e^{-\delta s}\left( v(X^{P^v}_{s^+}\!,\varepsilon _s)-v(X^{P^v}_s\!\!,\varepsilon _s)\right) \right] \\{} & {} \quad = \,\,\,\, - E_{x,i}\left[ \,\int _0^{t\wedge \tau }\!\!\!\!e^{-\delta s}\, k_{ \varepsilon _s } dP^v_s\,\right] . \end{aligned}$$

We note that $v(X_s,\varepsilon _s)$ and $v'(X_s,\varepsilon _s)$ are bounded when $s\in [\,0,t\wedge \tau ]$. Then, $\{M_{t\wedge \tau }^f,t\ge 0\}$ is a square-integrable martingale and, hence, $E_{x,i}\left[ M_{t\wedge \tau }^f-M_0^f\right] =0$, for every $t\ge 0$. Furthermore, $\sigma ^2_{\varepsilon (s)}\,e^{-2\delta s}(v'(X_s,\varepsilon _s))^2$ is bounded when $s\in [\,0,t\wedge \tau ]$, and hence

$$\begin{aligned} E_{x,i}\left[ \,\int _0^{t\wedge \tau }\sigma _{\varepsilon (s)}\,e^{-\delta s}\,v'(X_s,\varepsilon _s)\,dW_s\,\right] \,\,=\,\,0. \end{aligned}$$

Letting $a\downarrow - \infty$ and $b\uparrow +\infty$, we get $\tau _a\rightarrow +\infty$ and $\tau _b\rightarrow +\infty$. Then, $\tau \rightarrow \infty$. We notice that condition (1) and the quadratic growth of v imply that $\lim _{ t \uparrow \infty } E_{x,i}[e^{-\delta t} v(X_{t^+},\varepsilon _{t})] = 0.$ Taking $t\rightarrow \infty$, we get

$$\begin{aligned} v(x,i)\,\,\,&\le \,\,\,&E_{x,i}\left[ \,\int _0^{\infty } k_{ \varepsilon _s } e^{-\delta s}dP_s\,\right] + \,E_{x,i}\left[ \,\int _0^{\infty }\!\! e^{-\delta s} {\alpha }_{ \varepsilon (s) }( X_{s} - {{\mathcal {I}}}_{ \varepsilon (s) } )^{2} ds\,\right] \\&=\,\,\,&E_{x,i} \left[ \, \int _0^{\infty } e^{-\delta s} {\alpha }_{ \varepsilon (s) }( X_{s} - {{\mathcal {I}}}_{ \varepsilon (s) } )^{2} ds + \int _0^{\infty } e^{-\delta s} k_{ \varepsilon _s }dP_s\, \right] . \end{aligned}$$

In particular, for ${\hat{P}}$ we have $v(x,i){=} E_{x,i} \left[ \, \!\!\int _0^{\infty } e^{{-}\delta s} {\alpha }_{ \varepsilon (s) }( {\hat{X}}_{s}{-} {{\mathcal {I}}}_{ \varepsilon (s) } )^{2} ds {+} \!\int _0^{\infty } e^{{-}\delta s} k_{ \varepsilon _s }d{\hat{P}}_s\, \right]$. $\square$

3.2 Construction of the solution

We want to find a function v that satisfies the conditions of Theorem 1. In particular, we want Eq. (2) to be satisfied.

We conjecture that $v'(\cdot ,i)$ are continuous and nondecreasing functions. We denote

$$\begin{aligned} b_i:= \sup \{x \in (-\infty ,\infty ):\, v'(x,i)\le - k_{i}\} \end{aligned}$$

for each $i\in {{\mathcal {S}}}$. When $x\in (b_i, \infty )$, $v'(x,i)> - k_{i}$ because we conjecture above that $v'(\cdot ,i)$ are nondecreasing functions. Hence, in view of (2), for every $x \in [b_i, \infty )$:

$$\begin{aligned} \frac{1}{2}\,\sigma _i^2\,v''(x,i) - \mu _iv'(x,i)-\delta v(x,i) + {\alpha }_{i} (x - {{\mathcal {I}}}_{i})^{2} \,\,=\,\,\lambda _i\,v(x,i)-\sum _{j\ne i}q_{ij}\,v(x,j). \end{aligned}$$

(5)

When $x\in (-\infty ,b_i)$, $v'(x,i) \le - k_{i}$. The convexity of $v(\cdot ,i)$ and the fact that $v'(x,i)\ge -k_{i}$ [see Eq. (2)], imply that $v'(x,i)= -k_{i}$ for every $x\in (-\infty , b_i)$.

For simplicity, we assume in the remainder of this section that the economy shifts only between two regimes; that is, ${\mathcal S}=\{1,2\}$. Under this assumption, we consider only two thresholds: $b_1$ and $b_2$. The relation between $b_1$ and $b_2$ depends on the relations among the drift coefficients, the volatility parameters, and the rates $\lambda _1$ and $\lambda _2$. We will only consider the case $b_1<b_2$, because the case $b_1>b_2$ has a similar treatment. Thus, we will consider three possibilities for the initial level of the inventory: $x\in (-\infty ,b_1)$, $x\in [\,b_1,b_2)$, and $x\in [\,b_2,\infty )$.

We need the following lemma, which has been adapted from Remark 2.1 in Guo (2001)

Lemma 2

For $i\in {{\mathcal {S}}}$, consider the real function $\phi _i(z)=-\sigma _i^2z^2/2- {\tilde{\mu }}_i z+(\lambda _i+\delta )$ where ${\tilde{\mu }}_i$ is a function of $\mu _i$. Since $\sigma _1$, $\sigma _2$, $\lambda _1$ and $\lambda _2$ are positive, the equation $\phi _1(z)\,\phi _2(z)=\lambda _1\lambda _2$ has four real roots such that $z_1<z_2<0<z_3<z_4$.

When $x\in [b_2, \infty )$, (5) implies that v satisfies the system of differential equations

$$\begin{aligned} -(\lambda _1+\delta )\,v(x,1)-\mu _1\,v'(x,1)+\frac{1}{2}\,\sigma _1^2\,v''(x,1)+\lambda _1\,v(x,2) + {\alpha }_{1} (x - {{\mathcal {I}}}_{1})^{2}= & {} 0\nonumber \\ -(\lambda _2+\delta )\,v(x,2)-\mu _2\,v'(x,2)+\frac{1}{2}\,\sigma _2^2\,v''(x,2)+\lambda _2\,v(x,1) + {\alpha }_{2} (x - {{\mathcal {I}}}_{2})^{2}= & {} 0. \end{aligned}$$

(6)

A method to solve this kind of system or ordinary differential equations can be found, for instance, in Chapter 7 of Boyce and DiPrima (1997). According to Lemma 3.1, the solution is

$$\begin{aligned} v(x,1)\,\,= & {} \,\,A_1\,e^{\gamma _1(x-b_2)}\,+\,A_2\,e^{\gamma _2 (x-b_2)}\,+\,A_3\,e^{\gamma _3(x-b_2)}\,+\,A_4\,e^{\gamma _4 (x-b_2)} \\{} & {} + R_{1} (x-b_2)^{2} + S_{1} (x-b_2) + T_{1} \\ v(x,2)\,\ {}= & {} \,\,B_1\,e^{\gamma _1(x-b_2)}\,+\,B_2\,e^{\gamma _2 (x-b_2)}\,+\,B_3\,e^{\gamma _3(x-b_2)}\,+\,B_4\,e^{\gamma _4 (x-b_2)} \\{} & {} + R_{2} (x-b_2)^{2} + S_{2} (x-b_2) + T_{2}, \end{aligned}$$

where, for each $j=1,2,3,4$,

$$\begin{aligned} B_j\,\,=\,\,\frac{\phi ^1_1(\gamma _j)}{\lambda _1}\,A_j\,\,=\,\,\frac{\lambda _2}{\phi ^1_2(\gamma _j)}\,A_j. \end{aligned}$$

(7)

The real values $\gamma _1<\gamma _2<0<\gamma _3<\gamma _4$ above are the real roots of the characteristic equation $\phi ^1_1(\gamma )\,\,\phi ^1_2(\gamma )\,\,=\,\,\lambda _1\lambda _2,$ where

$$\begin{aligned} \phi ^1_i(\gamma )\,:=\,\,-\frac{1}{2}\,\sigma _i^2\gamma ^2 + \mu _i \,\gamma +(\lambda _i+\delta ),\qquad i=1,2. \end{aligned}$$

Furthermore, $R_{i}$, $S_{i}$ and $T_{i}$ are the solution of the system of 6 linear equations with 6 unknowns:

$$\begin{aligned} 0= & {} - (\lambda _{1} + \delta ) R_{1} + \lambda _{1} R_{2} + \alpha _{1} \nonumber \\ 0= & {} - (\lambda _{1} + \delta ) S_{1} -2 \mu _1 R_{1} + \lambda _{1} S_{2} + 2 \alpha _{1} (b_2 - {{\mathcal {I}}}_{1}) \nonumber \\ 0= & {} - (\lambda _{1} + \delta ) T_{1} - \mu _1 S_{1} + {\sigma }_{1}^{2} R_{1} + \lambda _{1} T_{2} + \alpha _{1} (b_2 - {{\mathcal {I}}}_{1})^{2} \nonumber \\ 0= & {} - (\lambda _{2} + \delta ) R_{2} + \lambda _{2} R_{1} + \alpha _{2} \nonumber \\ 0= & {} - (\lambda _{2} + \delta ) S_{2} -2 \mu _2 R_{2} + \lambda _{2} S_{1} + 2 \alpha _{2} (b_2 - {{\mathcal {I}}}_{2}) \nonumber \\ 0= & {} - (\lambda _{2} + \delta ) T_{2} - \mu _2 S_{2} + {\sigma }_{2}^{2} R_{2} + \lambda _{2} T_{1} + \alpha _{2} (b_2 - {{\mathcal {I}}}_{2})^{2}. \end{aligned}$$

(8)

We recall that we are conjecturing that $v(\cdot ,i)$ has quadratic growth. Thus, $B_{3}=B_{4}=A_{3}=A_{4}=0$. Hence, the solution of the system (6) is given by

$$\begin{aligned} v(x,1)\,\,= & {} \,\,A_1\,e^{\gamma _1(x-b_2)}\,+\,A_2\,e^{\gamma _2 (x-b_2)}\,+ R_{1} (x-b_2)^{2} + S_{1} (x-b_2) + T_{1} \end{aligned}$$

(9)

$$\begin{aligned} v(x,2)\,\ {}= & {} \,\,B_1\,e^{\gamma _1(x-b_2)}\,+\,B_2\,e^{\gamma _2 (x-b_2)}\,+ R_{2} (x-b_2)^{2} + S_{2} (x-b_2) + T_{2}. \end{aligned}$$

(10)

When $x\in [b_1,b_2)$, we define $v(x,2)\,:=\, -k_{2} (x- b_{2}) +D_2$, which satisfies $v'(x,2)= - k_{2}$. Furthermore, (5) and the assumption $b_1 < b_2$ imply that

$$\begin{aligned} -(\lambda _1+\delta )\,v(x,1)-\mu _1\,v'(x,1)+\frac{1}{2}\,\sigma _1^2\,v''(x,1) +\lambda _1\, v(x,2) + {\alpha }_{1} (x - {{\mathcal {I}}}_{1})^{2} =0. \end{aligned}$$

(11)

Solving this ordinary differential equation, we have

$$\begin{aligned} v(x,1)= & {} {\widetilde{A}}_1\,e^{{\tilde{\gamma }}_1 (x-b_2)}+{\widetilde{A}}_2\,e^{{\tilde{\gamma }}_2 (x-b_2)} + {\tilde{R}}_{1} (x-b_2)^{2} + {\tilde{S}}_{1} (x-b_2) + {\tilde{T}}_{1} \end{aligned}$$

(12)

$$\begin{aligned} v(x,2)= & {} -k_{2} (x- b_{2}) +D_2, \end{aligned}$$

(13)

where ${\tilde{\gamma }}_1<0<{\tilde{\gamma }}_2$ are the real roots of the equation

$$\begin{aligned} \phi _1^2({\tilde{\gamma }})\,:=\,\,-\frac{1}{2}\,\sigma _1^2\,{\tilde{\gamma }}^2 + \mu _1 \,{\tilde{\gamma }}+(\lambda _1+\delta )\,\,=\,\,0 \end{aligned}$$

and

$$\begin{aligned} 0= & {} - (\lambda _{1} + \delta ) {\tilde{R}}_{1} + \alpha _{1} \nonumber \\ 0= & {} - (\lambda _{1} + \delta ) {\tilde{S}}_{1} -2 \mu _1 {\tilde{R}}_{1} - \lambda _{1} k_{2} + 2 \alpha _{1} (b_2 - {{\mathcal {I}}}_{1}) \nonumber \\ 0= & {} - (\lambda _{1} + \delta ) {\tilde{T}}_{1} - \mu _1 {\tilde{S}}_{1} + {\sigma }_{1}^{2} {\tilde{R}}_{1} + \lambda _{1} D_{2} + \alpha _{1} (b_2 - {{\mathcal {I}}}_{1})^{2}. \end{aligned}$$

(14)

Finally, for every $x\in (- \infty , b_1)$, we define

$$\begin{aligned} v(x,1):= & {} -k_{1} (x - b_{1}) + D_1 \end{aligned}$$

(15)

$$\begin{aligned} v(x,2):= & {} -k_{2} (x- b_{2}) +D_2. \end{aligned}$$

(16)

To find the thresholds $b_1$ and $b_2$, and the coefficients and constants in Eqs. (9)–(10), (12)–(13), and (15)–(16), we conjecture that the smooth-fit condition holds. We also want $v'(b_i,i)= - k_{i}$ and $v''(b_i,i)=0$ for each $i=1,2$. Thus, we need to solve the system of equations

$$\begin{aligned} \begin{array}{lcll} v(b_2-,i)&{}=&{}v(b_2+,i)\qquad &{} \text{ for } \text{ both } i=1,2\\ v(b_1-,1)&{}=&{}v(b_1+,1)&{}\\ v'(b_1+,1)&{}=&{} - k_{1}&{}\\ v'(b_2+,2)&{}=&{} -k_{2}&{}\\ v'(b_2-,1)&{}=&{}v'(b_2+,1)&{}\\ v''(b_i+,i)&{}=&{}0&{} \text{ for } \text{ both } i=1,2. \end{array} \end{aligned}$$

(17)

The solution of (17) gives the values of $b_1$ and $b_2$, the values of the coefficients $A_j$, $j=1,2$, ${\widetilde{A}}_j$, $j=1,2$, and the constants $D_1$ and $D_2$. The values of $B_j$, $j=1,2$, can be found from (7).

Lemma 3

The following statements hold:

• (a) $\gamma _1< {\tilde{\gamma }}_1< \gamma _2< 0< \gamma _3< {\tilde{\gamma }}_2 < \gamma _4$, and

• (b) $\phi _1^1(\gamma _1)< 0, \quad \phi ^1_1(\gamma _2)>0,\quad \phi ^1_1(\gamma _3)> 0, \quad \phi ^1_1(\gamma _4)<0$.

Proof

For the proof of (a), we remember that $\gamma _i$ solves $\psi (\gamma _i):=\phi _1^1(\gamma _i) \phi _2^1(\gamma _i)-\lambda _1 \lambda _2=0$ and ${\tilde{\gamma }}_i$ solves $\phi _1^2({\tilde{\gamma }}_i)=0$. Moreover, we note that $\phi _1^1(\cdot )=\phi _1^2(\cdot )$. Since $\lim _{\gamma \rightarrow \pm \infty }\psi (\gamma )=\infty$, $\psi ({\tilde{\gamma }}_i)=-\lambda _1 \lambda _2<0$ and $\psi (0)=\lambda _1\delta +\lambda _2 \delta +\delta ^2>0$, the four roots $\gamma _i$ of $\psi$ are such that $\gamma _1< {\tilde{\gamma }}_1< \gamma _2< 0< \gamma _3< {\tilde{\gamma }}_2 < \gamma _4$.

The proof of (b) follows immediately from the proof of (a) together with the fact that $\lim _{\gamma \rightarrow \pm \infty } \phi _1^1=-\infty$, $\phi _1^1({\tilde{\gamma }}_i)=0$ and $\phi _1^1(0)=\lambda _i+\delta >0$. $\square$

3.3 Verification of the solution

In Sect. 3.2, we made conjectures to find a candidate v for value function. In this subsection, we will prove that v is indeed the value function of Problem 2.

Before proving this, we investigate the regularity of our candidate for value function.

Lemma 4

Let $b_i$, $i \in \{1,2\}$, $A_j$, $j\in \{1,2\}$, ${\widetilde{A}}_j$, $j \in \{1,2\}$, and $D_i$, $i \in \{1,2\}$, be the solution of the system of Eq. (17). Let $B_j$, $j \in \{1,2\}$, be defined by (7). Then, the function v defined by

$$\begin{aligned} v(x,1)\!=\! \left\{ \!\begin{array}{lll} -k_{1}(x-b_{1}) + D_{1} &{} \text{ if } &{} x\in (-\infty ,b_1),\\ {\widetilde{A}}_1\,e^{{\tilde{\gamma }}_1 (x-b_2)}\!+\!{\widetilde{A}}_2\,e^{{\tilde{\gamma }}_2 (x-b_2)} \!+\! {\tilde{R}}_{1} (x-b_2)^{2} \!+\! {\tilde{S}}_{1} (x-b_2) \!+\! {\tilde{T}}_{1} &{} \text{ if } &{} x\in [b_1, b_2),\\ A_1\,e^{\gamma _1(x-b_2)}\!+\! A_2\,e^{\gamma _2 (x-b_2)}\!+\! R_{1} (x-b_2)^{2} \!+\! S_{1} (x-b_2) \!+\! T_{1} &{} \text{ if } &{} x\in [\,b_2,\infty ), \end{array}\right. \end{aligned}$$

(18)

and

$$\begin{aligned} v(x,2)= \left\{ \begin{array}{lll} -k_{2}(x-b_{2}) + D_{2} &{} \text{ if } &{} x\in (-\infty ,b_2),\\ B_1\,e^{\gamma _1(x-b_2)}{+}B_2\,e^{\gamma _2 (x-b_2)}{+} R_{2} (x-b_2)^{2} {+} S_{2} (x-b_2) {+} T_{2} &{} \text{ if } &{} x\in [\,b_2,\infty ), \end{array}\right. \end{aligned}$$

(19)

is such that $v(\cdot ,1)\in C^{\infty }({\mathbb {R}} {\setminus } \{b_1,b_2\}) \cap C^4({\mathbb {R}} {\setminus } \{b_1\}) \cap C^2({\mathbb {R}})$ and $v(\cdot ,2)\in C^{\infty }({\mathbb {R}} {\setminus } \{b_2\}) \cap C^2({\mathbb {R}})$. Moreover, for fixed $i \in \{1,2\}$ and $x> b_i$, we have

$$\begin{aligned} -(\lambda _i + \delta ) v''(x,i) - \mu _i v'''(x,i)+ \frac{1}{2}\sigma _1^2 v''''(x,i) + \lambda _i v''(x,3-i)+ 2\alpha _i=0. \end{aligned}$$

(20)

Proof

First, we note that the system (17) implies that $v(\cdot ,2)\in C^{\infty }({\mathbb {R}} {\setminus } \{b_2\}) \cap C^2({\mathbb {R}})$. The regularity of $v(\cdot ,1)$ can be shown as follows. By construction, $v(\cdot ,1) \in C^{\infty }({\mathbb {R}} {\setminus } \{b_1,b_2\})$ and solves, for $x > b_1$,

$$\begin{aligned} -(\lambda _1+\delta )\,v(x,1)-\mu _1\,v'(x,1)+\frac{1}{2}\,\sigma _1^2\,v''(x,1)+\lambda _1\,v(x,2) + {\alpha }_{1} (x - {{\mathcal {I}}}_{1})^{2} =0. \end{aligned}$$

Hence, we can rewrite this equation and obtain

$$\begin{aligned} v''(x,1)= \frac{2}{\sigma _1^2} \left( (\lambda _1+\delta )\,v(x,1)+\mu _1\,v'(x,1)-\lambda _1\,v(x,2) \right) . \end{aligned}$$

Since $v(\cdot ,2) \in C^2({\mathbb {R}})$ by (17), we obtain that $v(\cdot ,1) \in C^4((b_1,\infty ))$ by differentiating the last equation. Moreover, by the system (17), we obtain that $v''(b_1-,1)=0=v''(b_1+,1)$. Hence, combining these two facts, we obtain that $v(\cdot ,1)\in C^{\infty }({\mathbb {R}} {\setminus } \{b_1,b_2\}) \cap C^4({\mathbb {R}} {\setminus } \{b_1\}) \cap C^2({\mathbb {R}})$. Furthermore, fixing $i \in \{1,2\}$ and $x>b_i$, we know that

$$\begin{aligned} -(\lambda _i + \delta ) v(x,i) - \mu _i v'(x,i)+ \frac{1}{2}\sigma _1^2 v''(x,i) + \lambda _i v(x,3-i) + \alpha _i (x-I_i)^2=0. \end{aligned}$$

Therefore, the proven regularity allows us to differentiate this equation two times with respect to x, and get

$$\begin{aligned} -(\lambda _i + \delta ) v''(x,i) - \mu _i v'''(x,i)+ \frac{1}{2}\sigma _1^2 v''''(x,i) + \lambda _i v''(x,3-i)+ 2\alpha _i=0. \end{aligned}$$

$\square$

The proven regularity of v allows us to apply Itô’s formula for Markov-modulated processes on $v''(\cdot ,i)$. This will allow us to prove convexity for the candidate for value function.

Theorem 5

Let $b_i$, $i \in \{1,2\}$, $A_j$, $j \in \{1,2\}$, ${\widetilde{A}}_j$, $j \in \{1,2\}$, and $D_i$, $i \in \{1,2\}$, be the solution of the system of Eq. (17). Let $B_j$, $j \in \{1,2\}$, be defined by (7). Then, the function v given by (18)–(19) is the value function V of Problem 2. The optimal production policy ${\hat{P}}$ is determined by Definition 3.2.

Proof

To prove that the function v defined above is the solution for Problem 2, it is enough to show that it satisfies the conditions of Theorem 1. We note that $v(\cdot ,i)$ is of quadratic growth and $v'(\cdot ,i)$ is of linear growth by construction.

Next, we show that v is convex. We already know that $v''(x,i)=0$ for every $x \le b_i$ and $i \in \{1,2\}$. By Lemma 4, we know that $v(\cdot ,1) \in C^4({\mathbb {R}} {\setminus } \{b_1\})$ and $v(\cdot ,2) \in C^4({\mathbb {R}} {\setminus } \{b_2\})$. Hence, for fixed $i \in \{1,2\}$ and $x > b_i$, we apply Itô’s formula for Markov modulated processes (using the fact that $v(\cdot ,i)$ is of quadratic growth) and obtain

$$\begin{aligned} E_{x,i}\left[ e^{-\delta \tau } v''(X_{\tau },\varepsilon _{\tau })\right] = v''(x,i) + E_{x,i}\left[ \int _0^{\tau } e^{-\delta s} A^i v''(X_s,\varepsilon _s) ~ds \right] , \end{aligned}$$

where $A^i \psi (x,i):=\frac{1}{2} \sigma _i \psi ''(x,i)-\mu _i \psi '(x,i)-(\lambda _i+\delta ) \psi (x,i)- \lambda _i \psi (x,3-i)$ and $\tau$ is a stopping time such that ${\mathbb {E}}\left[ \tau \right] <\infty$. From Eq. (20), taking $\tau :=\inf \{t\ge 0: (X_t,\varepsilon (t)) \in \{(x,i):x \le b_i \} \}$, we obtain, for $i \in \{1,2\}$ and $x> b_i$,

$$\begin{aligned} v''(x,i)=E_{x,i}\left[ 2 \int _0^{\tau } e^{-\delta s} \alpha _{\varepsilon (s)}~ds+e^{-\delta \tau } v''(X_{\tau },\varepsilon (\tau )) \right] . \end{aligned}$$

(21)

Since $v''(x,i)=0$ for all $x \le b_i$ by construction, (21) implies, together with the fact that $\alpha _i > 0$, that $v''(x,i)> 0$ for all $i \in \{1,2\}$ and $x> b_i$. Hence, it follows that $v(\cdot ,i)$ is convex for $i=1,2$.

Next, we show that the candidate v for value function satisfies the HJB Eq. (2). We define

$$\begin{aligned} ({\mathbb {L}}v)(x,i):=\frac{1}{2}\sigma _i^2 v''(x,i)-\mu _i v'(x,i)-\delta v(x,i)-\lambda _i v(x,i)+\lambda _i v(x,3-i)+\alpha _i (x-{\mathcal {I}}_i)^2. \end{aligned}$$

Obviously,

$$\begin{aligned} ({\mathbb {L}}v)(x,1)\in {\mathcal {C}}^0({\mathbb {R}})\cap {\mathcal {C}}^2({\mathbb {R}} \setminus \{b_1\} \;\;\; \text{ and } \;\;\; ({\mathbb {L}}v)(x,2)\in {\mathcal {C}}^0({\mathbb {R}}) \cap {\mathcal {C}}^{\infty }({\mathbb {R}} \setminus \{b_2\}). \end{aligned}$$

We will consider three cases: $x \in [b_2,\infty )$, $x \in [b_1,b_2)$, and $x \in (-\infty ,b_1)$.

Let $x \in [b_2,\infty )$. By construction of v, definition of $b_i$, the fact that $b_1 < b_2$, and the convexity of v, we get that v satisfies $({\mathbb {L}}v)(x,i)=0$ and $v'(x,i)>-k_i$. Hence, (2) is satisfied.

Let $x \in [b_1,b_2)$. By construction of v(x, 1), definition of $b_1$ and the convexity of v(x, 1), we have $({\mathbb {L}}v)(x,1)=0$ and $v'(x,1)>-k_1$. Let us now check v(x, 2). Since $v(x,2)=-k_2(x-b_2)+D_2$, we obtain

$$\begin{aligned} ({\mathbb {L}}v)(x,2)= & {} \mu _2k_2-(\delta +\lambda _2)[-k_2(x-b_2)+D_2]+\lambda _2 v(x,1)+\alpha _2(x-{\mathcal {I}}_2)^2 \\ ({\mathbb {L}}v)'(x,2)= & {} (\delta +\lambda _2)k_2+\lambda _2 v'(x,1)+2\alpha _2(x-{\mathcal {I}}_2), \end{aligned}$$

and, due to convexity,

$$\begin{aligned} ({\mathbb {L}}v)''(x,2)&=\lambda _2 v''(x,1)+2\alpha _2>0 . \end{aligned}$$

We want to show that $({\mathbb {L}}v)(x,2) \ge 0$. Since we know that $({\mathbb {L}}v)(b_2-,2)=({\mathbb {L}}v)(b_2+,2)= 0$, it is enough to show that $({\mathbb {L}}v)(x,2)$ is decreasing, or equivalently $({\mathbb {L}}v)'(x,2)\le 0$. Since $({\mathbb {L}}v)''(x,2)\ge 0$, we have that $({\mathbb {L}}v)'(x,2)$ is increasing, hence $({\mathbb {L}}v)'(x,2)\le ({\mathbb {L}}v)'(b_2-,2)$. Next, we show that $({\mathbb {L}}v)'(b_2-,2)\le 0$.

We have

$$\begin{aligned} ({\mathbb {L}}v)'(b_2-,2)=k_2(\delta +\lambda _2)+\lambda _2 v'(b_2-,1)+2\alpha _2(x-{\mathcal {I}}_2). \end{aligned}$$

We investigate $\lambda _2 v'(b_2,1)$.

$$\begin{aligned} \lambda _2 v'(b_2-,1)&=\lambda _2 v'(b_2+,1)=\lambda _2[S_1+\sum _{i=1}^{4}\gamma _i A_i] \\&=\lambda _2[S_1+\sum _{i=1}^{4}\gamma _i \frac{\phi _2^1(\gamma _i)}{\lambda _2}B_i]=\lambda _2S_1+\sum _{i=1}^{4}\gamma _i B_i [-\frac{1}{2}\sigma _2^2 \gamma _i^2+\mu _2 \gamma _i+(\lambda _2+\delta )]\\&=\lambda _2S_1-\frac{1}{2}\sigma _2^2\sum _{i=1}^{4}\gamma _i^3 B_i +\mu _2 \sum _{i=1}^{4}\gamma _i^2 B_i +(\lambda _2+\delta )\sum _{i=1}^{4}\gamma _i B_i\\&=\lambda _2 S_1 -\frac{1}{2}\sigma _2^2v'''(b_2+,2) +\mu _2 v''(b_2+,2)\\&\quad -2\mu _2 R_2 +(\lambda _2+\delta )v'(b_2+,2)-(\lambda _2+\delta )S_2\\&\le \lambda _2 S_1-2 \mu _2 R_2-k_2(\lambda _2+\delta )-(\lambda _2+\delta )S_2, \end{aligned}$$

where the last step follows from the fact that $v''(b_2+,2)=0$ by the smooth-fit conditions and $v'''(b_2+,2)\ge 0$. The fact that $v'''(b_2+,2)\ge 0$ follows by convexity. If $v'''(b_2+,2)< 0$, this would imply, since $v''(b_2+,2)= 0$, that $v''(b_2+ \delta ,2) < 0$ for $\delta$ small enough, which contradicts convexity. Hence, using (8), we obtain

$$\begin{aligned} ({\mathbb {L}}v)'(b_2-,2)&\le \lambda _2 S_1 - 2 \mu _2 R_2 +2 \alpha _2 (b_2-{\mathcal {I}}_2)-(\lambda _2+\delta )S_2=0. \end{aligned}$$

To summarize, (2) is satisfied in the interval $[b_1, b_2)$.

Next, let $x<b_1$. We calculate

$$\begin{aligned} ({\mathbb {L}}v)(x,i)= & {} \mu _i k_i-\delta v(x,i)-\lambda _i v(x,i)+\lambda _i v(x,3-i)+\alpha _i (x- {\mathcal {I}})^2 \\ ({\mathbb {L}}v)'(x,i)= & {} \delta k_i+\lambda _i k_i-\lambda _i k_{3-i}+2 \alpha _i (x- {\mathcal {I}}) \\ ({\mathbb {L}}v)''(x,i)= & {} 2 \alpha _i>0. \end{aligned}$$

Hence, we have

$$\begin{aligned} ({\mathbb {L}}v)'(x,i)\le ({\mathbb {L}}v)'(b_1-,i). \end{aligned}$$

Since $({\mathbb {L}}v)'(b_1-,2)=({\mathbb {L}}v)'(b_1+,2)\le 0$ and v is continuous, we have $({\mathbb {L}}v)(x,2)\ge ({\mathbb {L}}v)(b_1+,2)\ge 0$. For $i=1$, we note that $({\mathbb {L}}v)(x,1)=0$ for all $x \in (b_1,b_2)$. Thus, we have $({\mathbb {L}}v)'(x,1)=0$ on $(b_1,b_2)$. Hence,

$$\begin{aligned} 0= & {} ({\mathbb {L}}v)'(b_1+,1)=\frac{1}{2}\sigma _1^2v'''(b_1+,1)-\mu _1 v''(b_1+,1)-(\delta +\lambda _1)v'(b_1+,1)\\{} & {} +\lambda _1 v'(b_1+,2)+2\alpha _1 (b_1-{\mathcal {I}}_1). \end{aligned}$$

We observe that $v'''(b_1+,1)\ge 0$. Otherwise, $v'''(b_1+,1)< 0$ would imply (by the fact that $v''(b_1+,1)=0$) that $v''(b_1+ \delta ,1)<0$ for $\delta$ small enough and that would contradict the convexity. Using now that $v''(b_1+,1)=0$, $v'''(b_1+,1)\ge 0$, and that $v'(b_1+,i)=-k_i$, we obtain

$$\begin{aligned} 0 \ge (\delta +\lambda _1)k_1-\lambda _1 k_2 + 2\alpha _1 (b_1-{\mathcal {I}}_1). \end{aligned}$$

Now, let $x \le b_1$. Since $({\mathbb {L}}v)''(x,1)=2 \alpha _1 >0$ we have $({\mathbb {L}}v)'(x,1)\le ({\mathbb {L}}v)'(b_1-,1)= (\delta +\lambda _1)k_1-\lambda _1k_2+2 \alpha _1(b_1-{\mathcal {I}}_1) \le 0$. Hence, $({\mathbb {L}}v)(x,1)$ is decreasing and, by continuity, we obtain

$$\begin{aligned} ({\mathbb {L}}v)(x,1) \ge ({\mathbb {L}}v)(b_1-,x)= ({\mathbb {L}}v)(b_1+,x)=0. \end{aligned}$$

To summarize, (2) is satisfied in the interval $(-\infty , b_1)$. $\square$

We have now proven that our candidate for value function is indeed the true value function. Moreover, we have obtained the optimal production policy.

Therefore, the optimal production policy when the regime is i is the following: (a) do not produce when the inventory level is above the threshold $b_{i}$; (b) if the inventory level is equal to $b_{i}$, then the company should produce enough to prevent the inventory from being less than $b_{i}$; and (c) if the initial inventory is below $b_{i}$, then the company should produce to bring immediately the inventory to the threshold $b_{i}$ (and then continue as described above). We observe that the company should also increase production to bring immediately the inventory to the threshold $b_{2}$ just because of a change in the regime: that happens when the level of the inventory falls in the interval $(b_1, b_2)$ and the regime changes from $i=1$ to $i=2$.

3.4 Comparative statics and numerical examples

In this subsection, we study the influence of some parameters on the thresholds $b_{i}, i \in {\mathcal {S}}$ that characterize the optimal production strategy. First, we present the results that can be obtained analytically, and then those which have to be obtained numerically. To study the influence of certain parameters analytically, we apply the connection between optimal stopping and singular stochastic control problems.

Following the ideas of Karatzas and Shreve (1984) (see also Baldursson and Karatzas (1996)), it is possible to show that $V_x(x,i)=u(x,i)$, where

$$\begin{aligned} u(x,i):=\sup _{\tau } E_{x,i}\left[ \int _0^{\tau } e^{-\delta s} 2 \alpha _{\varepsilon (s)}(X^{0}_s-{\mathcal {I}}_{\varepsilon (s)})~ds-e^{-\delta \tau } k_{\varepsilon (\tau )} \right] . \end{aligned}$$

(22)

Here, $X^{0}$ is the inventory process when there is no production. Since $V_x(x,i)=u(x,i)$, we have

$$\begin{aligned} b_i \; = \; \sup \{x \in {\mathbb {R}}: u(x,i)\le -k_i\}, \end{aligned}$$

(23)

and $x \mapsto u(x,i)$ is nondecreasing since V is convex.

In the following, we will denote by $b_i(a)$ the boundary $b_i$ as a function of a parameter a. Moreover, u(x, i; a) represents u(x, i) as a function of a parameter a.

Proposition 6

For given $i \in {\mathcal {S}}=\{1,2\}$, the thresholds $b_i$ have the following properties:

• (i) ${\mathcal {I}}_j \mapsto b_i({\mathcal {I}}_j)$ is a nondecreasing function for every $j \in {\mathcal {S}}$,

• (ii) $\mu _j \mapsto b_i(\mu _j)$ is a nondecreasing function for every $j \in {\mathcal {S}}$,

• (iii) $k_i \mapsto b_i(k_i)$ is a nonincreasing function,

• (iv) $k_{3-i} \mapsto b_i(k_{3-i})$ is a nondecreasing function.

Proof

We define ${\hat{u}}(x,i):=u(x,i)+k_i$ so that

$$\begin{aligned} {\hat{u}}(x,i)&=\sup _{\tau } E_{x,i}\left[ \int _0^{\tau } e^{-\delta s} 2 \alpha _{\varepsilon (s)}(X_s-{\mathcal {I}}_{\varepsilon (s)})~ds + \left( 1- e^{-\delta \tau }\right) k_{i} \mathbb {1}_{\{\varepsilon (\tau )=i\}} \right. \\&\left. \quad + \left( k_i-e^{-\delta \tau }k_{3-i} \right) \mathbb {1}_{\{\varepsilon (\tau )=3-i\}} \right] . \end{aligned}$$

We observe that

• (1) ${\mathcal {I}}_j \mapsto u(x,i;{\mathcal {I}}_j)$ is a nonincreasing function for every $j \in {\mathcal {S}}$,

• (2) $\mu _j \mapsto u(x,i;\mu _j)$ is a nonincreasing function for every $j \in {\mathcal {S}}$ since $\mu _j \mapsto X_t^{x,0}=x - \int _0^t \mu _{\varepsilon (s)}~dt- \int _0^t \sigma _{\varepsilon (s)}~dW_s$ is a nonincreasing function,

• (3) $k_i \mapsto {\hat{u}}(x,i;k_i)$ is a nondecreasing function, and

• (4) $k_{3-i} \mapsto {\hat{u}}(x,i;k_{3-i})$ is a nonincreasing function.

Thus, we can prove each result separately.

• (i) Taking $j \in {\mathcal {S}}$, ${\mathcal {I}}_j^{1}>{\mathcal {I}}_j^{2}$ and using (23), the fact that $u(\cdot ,i)$ is increasing, and 1), we have

  $$\begin{aligned} b_i({\mathcal {I}}_j^{2}):= \sup \{x: u(x,i;{\mathcal {I}}_j^{2})\le -k_i\} \le \sup \{x: u(x,i;{\mathcal {I}}_j^{1})\le -k_i\}=: b_i({\mathcal {I}}_j^{1}). \end{aligned}$$

• (ii) Taking $j \in {\mathcal {S}}$, $\mu _j^{1}>\mu _j^{2}$ and using (23), the fact that $u(\cdot ,i)$ is increasing, and 2), we have

  $$\begin{aligned} b_i(\mu _j^{2}):= \sup \{x: u(x,i;\mu _j^{2})\le -k_i\} \le \sup \{x: u(x,i;\mu _j^{1})\le -k_i\}=: b_i(\mu _j^{1}). \end{aligned}$$

• (iii) Taking $k_i^{1}>k_i^{2}$ and using (23), the fact that $u(\cdot ,i)$ is increasing, and 3), we have

  $$\begin{aligned} b_i(k_i^{2}):= \sup \{x: {\hat{u}}(x,i;k_i^{2})\le 0\} \ge \sup \{x: u(x,i;k_i^{1})\le 0\}=: b_i(k_i^{1}). \end{aligned}$$

• (iv) It can be proved analogously to (iii).

$\square$

We obtain the following proposition for the volatilities $\sigma _i, i \in {\mathcal {S}}$,

Proposition 7

$V(\cdot ,i)$ is an increasing function of $\sigma _i$.

Proof

We know that V satisfies the Hamilton–Jacobi–Bellman equation

$$\begin{aligned} \min \left\{ L_i v(x,i)-\lambda _i\,v(x,i)+\lambda _i \,v(x,3-i) + {\alpha }_{i} (x - {{\mathcal {I}}}_{i})^2 ,\,\,\, v'(x,i) + k_{i} \right\} \,=\,0, \end{aligned}$$

(24)

where $L_i\,\psi \,:=\, \frac{1}{2}\,\sigma _i^2\,\psi '' - \mu _i\,\psi '-\delta \,\psi$. Let now $\sigma _i^2 > {\widehat{\sigma }}_i^2$ and define

$$\begin{aligned} {\widehat{L}}_i\,\psi \,:=\, \frac{1}{2}\,{\widehat{\sigma }}_i^2\,\psi '' - \mu _i\,\psi '-\delta \,\psi . \end{aligned}$$

Moreover, let V(x, i) and ${\widehat{V}}(x,i)$ be the value functions with respect to $\sigma _i$ and ${\widehat{\sigma }}_i$. Thus,

$$\begin{aligned}{} & {} L_i {\widehat{V}}(x,i)- \lambda _i\,{\widehat{V}}(x,i)+\lambda _i \,{\widehat{V}}(x,3-i) + {\alpha }_{i} (x - {{\mathcal {I}}}_{i})^2 \\{} & {} \quad ={\widehat{L}}_i {\widehat{V}}(x,i)-\lambda _i\,{\widehat{V}}(x,i)+\lambda _i \,{\widehat{V}}(x,3-i) + {\alpha }_{i} (x - {{\mathcal {I}}}_{i})^2 + \frac{1}{2}\left( \sigma _i^2 - {\widehat{\sigma }}_i^2 \right) {{\widehat{V}}}''(x,i) \\{} & {} \quad \ge {\widehat{L}}_i {\widehat{V}}(x,i)-\lambda _i\,{\widehat{V}}(x,i)+\lambda _i \,{\widehat{V}}(x,3-i) + {\alpha }_{i} (x - {{\mathcal {I}}}_{i})^2 \ge 0, \end{aligned}$$

where the penultimate inequality follows by the convexity of the value function. Moreover, we note that ${\widehat{V}}'(x,i)+k_i \ge 0$ for every $(x,i) \in {\mathbb {R}} \times {\mathcal {S}}$. Hence, ${\widehat{V}}$ solves

$$\begin{aligned} \min \left\{ L_i {\widehat{v}}(x,i)-\lambda _i\,{\widehat{v}}(x,i)+\lambda _i \,{\widehat{v}}(x,3-i) + {\alpha }_{i} (x - {{\mathcal {I}}}_{i})^2,\,\,\, {\widehat{v}}'(x,i) + k_{i} \right\} \,=\,0. \end{aligned}$$

Following now the same procedure as in the first part of the proof of Theorem 1, we obtain ${\hat{V}} \le V$, hence the claimed monotonicity. $\square$

Next, we study numerically the dependence of the thresholds $b_{i}$ with respect to $\sigma _i$ and $\lambda _i$. We introduce a benchmark case, and we use the following parameter values:

$$\begin{aligned}{} & {} \mu _1=0.2,~\mu _2=1.0,~\sigma _1=\sigma _2=0.2,~\delta =0.2,~\alpha _1=\alpha _2=0.2,~\\{} & {} \quad I_1=I_2=2.0,~ k_1=k_2=2.0,~ \lambda _1=\lambda _2=0.2. \end{aligned}$$

In this setting, the demand drift values are different in the two regimes. We will call regime 1 the “low-demand regime” and regime 2 the “high-demand regime”. We will consider different values of $\sigma _i$ in Fig. 2, and different values of $\lambda _i$ in Fig. 3.

For the above parameter values, the solution to system (17) is given by

$$\begin{aligned} A_1=-0.206,~ A_2=-15.429,~{\tilde{A}}_1=-0.308,~ {\tilde{A}}_2=0.0,~ D_1=5.844,~ D_2=8.382, \end{aligned}$$

and for the thresholds we obtain

$$\begin{aligned} b_1=0.914 \;\;\; \text{ and } \;\;\; b_2=0.977. \end{aligned}$$

The corresponding value function is presented in Fig. 1.

Fig. 1

Graphical illustration of the value function for the benchmark case

Starting from this benchmark case, we study the dependence of $b_1$ and $b_2$ with respect to $\sigma _i$. The results are presented in Fig. 2.

Fig. 2

Graphical illustration of the dependence of the production thresholds on $\sigma _i$. The other parameters are assumed to be the same as in the benchmark case

One can see that the production thresholds are decreasing in the volatility parameters $\sigma _i$. Hence, a higher uncertainty in the demand leads to a lower production threshold.

A slightly different result can be obtained for the dependence on $\lambda _i$, which gives the rate of switching from one regime to another, and is shown in Fig. 3.

Fig. 3

Graphical illustration of the dependence of the thresholds on $\lambda _i$. The other parameters are assumed to be the same as in the benchmark case

For the dependence on $\lambda _i$, we see that both production thresholds are increasing in $\lambda _1$, but decreasing in $\lambda _2$. According to the theory of continuous-time Markov chains, the amount of time that the demand is in regime i, before moving to regime $3-i$, has exponential distribution with mean $\frac{1}{ \lambda _i}$. Hence, a lower expected amount of time in the low-demand regime, before switching to the high-demand regime, increases both thresholds, and thus there is more production. On the other hand, a lower expected amount of time in the high-demand regime, before switching to the low-demand regime, decreases both thresholds, and thus there is less production.

For completeness, we introduce Fig. 4, which shows the behavior of the thresholds with respect to $\mu _i$ (as it was proven in Lemma 6). Moreover, we see that both thresholds converge to each other if the drift components converge to each other. In this case, both regimes have exactly the same parameter values, and hence the thresholds must be the same.

Fig. 4

Graphical illustration of the dependence of the production thresholds on $\mu _i$. The other parameters are assumed to be the same as in the benchmark case

4 Optimal production control when the production rates are bounded from above

In this section, we consider the model with bounded production rates. In this case, the production process $P=\{P_t,t\ge 0\}$ is such that $dP_t\,\,=\,\,p(t)\,dt$, where $p:[\,0,\infty )\times \Omega \rightarrow [\,0,\infty )$ is an ${\mathbb F}$-adapted bounded process that represents the rate of production. Let $K_{i}$, $i \in {{\mathcal {S}}}$, be positive real numbers.

Definition 3

An ${\mathbb {F}}$-adapted control process $p:[\,0,\infty )\times \Omega \rightarrow [\,0,\infty )$ that satisfies $p(t,\omega )\in [\,0,K_{\varepsilon (t)}\,]$ for every $(t,\omega )\in {[}\,0,\infty )\times \Omega$ and

$$\begin{aligned} E_{x,i}\left[ \,\int _0^{\infty }\,e^{-\delta s} \alpha _{\varepsilon (s)} (X_{s} - {{\mathcal {I}}}_{\varepsilon (s)})^{2}ds + \int _0^{\infty }e^{-\delta s} k_{\varepsilon (s)} p(s) ds \right] < \infty , \end{aligned}$$

(25)

is called an admissible classical stochastic control. The set of all admissible classical stochastic controls is denoted by ${{\mathcal {A}}}_B$.

We note that for the above process p, the inventory is given by

$$\begin{aligned} X_t\,=\,x + \int _0^t\left( -\mu _{\varepsilon (s)} + p(s)\right) \,ds-\int _0^t\sigma _{\varepsilon (s)}\,dW_s \end{aligned}$$

(26)

for every $t\in [\,0,\infty )$. Under these assumptions, Problem 1 is equivalent to the following problem.

Problem 3

The management wants to obtain the optimal production rate ${\hat{p}} \in {{\mathcal {A}}}_B$ and the value function V defined by

$$\begin{aligned} V(x,i) \;:= & {} \; \inf _{p \in \,{{\mathcal {A}}}_B }J(x,i;p) \; \nonumber \\= & {} \; \inf _{p\in \,{{\mathcal {A}}}_B } E_{x,i}\left[ \,\int _0^{\infty }\,e^{-\delta s} \alpha _{\varepsilon (s)} (X_{s} - {{\mathcal {I}}}_{\varepsilon (s)})^{2} ds + \int _0^{\infty }\,e^{-\delta s} k_{\varepsilon (s)} p(s)\,ds\,\right] .\nonumber \\ \end{aligned}$$

(27)

Problem 3 is a classical stochastic control problem with regime switching. We observe that there is a linear cost of production given by $k_{\varepsilon (s)}$. Furthermore, p is bounded from below by 0, and from above by $K_{\varepsilon (t)}$. Bensoussan et al. (1984) studied a problem in which the cost of production was quadratic and there was no regime switching (or equivalently, there was only one regime). They did not obtain an analytical solution but presented a numerical method to solve their problem. Cadenillas et al. (2013) studied a problem in which the cost of production was quadratic, and the production rates were unbounded from below and from above. They obtained an analytical solution by applying the “completing squares" method which cannot be applied to our problem. Sotomayor and Cadenillas (2011) solved a stochastic control problem with regime switching in which the control was bounded from below and from above, but their objective function was different from Sect. 4.1, because they considered a maximization problem instead of a minimization problem, and the term $\int _0^{\infty }\,e^{-\delta s} \alpha _{\varepsilon (s)} (X_{s} - {{\mathcal {I}}}_{\varepsilon (s)})^{2} ds$ did not appear in their objective function.

4.1 Verification Theorem

Let $\psi :(-\infty ,\infty )\times {{\mathcal {S}}}\rightarrow$ be a function such that $\psi (\cdot ,i) \in C^2({\mathbb {R}})$ for every $i \in {\mathcal {S}}$ and define the operators $L_i(p)$, $i\in {{\mathcal {S}}}$ by

$$\begin{aligned} L_i(p)\,\psi \,\,:=\,\, \frac{1}{2}\,\sigma _i^2\,\psi ''+(-\mu _i +p)\,\psi '-\delta \,\psi . \end{aligned}$$

Here, $\psi ''$ and $\psi '$ denote the partial derivatives $\partial ^2\psi /\partial x^2$ and $\partial \psi /\partial x$, respectively.

Theorem 8

Let $v(\cdot ,i)\in C^1((-\infty ,\infty )) \cap C^2((-\infty ,\infty ) {\setminus } N_i)$, $i\in {{\mathcal {S}}}$, where $N_i$ are finite subsets of $(-\infty ,\infty )$. Let $v(\cdot ,i)$, $i\in {{\mathcal {S}}}$, have quadratic growth on $(-\infty ,\infty )$. If the function v satisfies the equation

$$\begin{aligned} \inf _{p \in [\,0,K_{i}\,]}\{L_{i}(p)\,v(x,i)+ k_{i} p\}\,= - \alpha _{i} (x - {{\mathcal {I}}}_{i})^{2} + \,\lambda _{i}\,v(x,i)-\sum _{j\in {\mathcal S}\setminus \{i\}}q_{ij}\,v(x,j), \end{aligned}$$

(28)

for every $x\in (-\infty ,\infty )$ and $i\in {{\mathcal {S}}}$, then the control ${\hat{p}}$ defined by

$$\begin{aligned} {\hat{p}}(t)\,=\,\arg \inf _{p \,\in \,[\,0,K_{\varepsilon (t)}\,]}\{L_{\varepsilon (t)}(p)\,v(X_t,\varepsilon _t)+ k_{\varepsilon (t)}p\} \end{aligned}$$

for $t\in [\,0, \infty )$ is optimal solution for Problem 3. Moreover, v is equal to the value function V for Problem 3.

Proof

Consider the function $f(\cdot ,\cdot ,i)$, $i\in {{\mathcal {S}}}$, defined by $f(t,x,i)=e^{-\delta t}v(x,i)$ and an admissible control p. We get

$$\begin{aligned} df(t,X_t,\varepsilon _t)= & {} \left( \,\frac{1}{2}\,\sigma _{\varepsilon (t)}^2\,f_{xx}(t,X_t,\varepsilon _t)+(- \mu _{\varepsilon (t)} +p)\,f_x(t,X_t,\varepsilon _t)+f_t(t,X_t,\varepsilon _t)\right) dt\\{} & {} \,- \,f_x(t,X_t,\varepsilon _t)\,\sigma _{\varepsilon (t)}\,dW_t\,+\,\left( \,-\,\lambda _{\varepsilon (t)}\,f(t,X_t,\varepsilon _t)\right. \\{} & {} \left. +\sum _{j\ne \varepsilon (t)}q_{\varepsilon (t)j}\,f(t,X_t,j)\,\right) dt +dM_t^f. \end{aligned}$$

Here, the process $M^f=\{M^f_t,t\ge 0\}$ is a square-integrable martingale when $f(\cdot ,\cdot ,i)$, $i\in {{\mathcal {S}}}$, is bounded. In fact, $M_t^f =\sum _{i\in {{\mathcal {S}}}}\int _0^tf(s,X_s,i)dm_s^i$, where the square integrable-martingales $m^i$, $i\in {{\mathcal {S}}}$, are defined as in Eq. (5) of Björk (1980). Hence,

$$\begin{aligned} df(t,X_t,\varepsilon _t)&{=}&e^{-\delta t}\left( L_{\varepsilon (t)}(p)v(X_t,\varepsilon _t){-}\Delta _Qv(X_t,\varepsilon _t)\right) dt {-} \sigma _{\varepsilon (t)}e^{{-}\delta t}v'(X_t,\varepsilon _t)dW_t{+}\,dM_t^f, \end{aligned}$$

where $\Delta _{Q}v(X_t,\varepsilon _t):=\lambda _{\varepsilon (t)}\,v(X_t,\varepsilon _t)-\sum _{j\ne \varepsilon (t)}q_{\varepsilon (t)j}\,v(X_t,j)$. Then, for every time $t\ge 0$, we get

$$\begin{aligned} e^{-\delta t}v(X_{t},\varepsilon _{t})= & {} v(X_0,\varepsilon _0)+\int _0^{t}\!\!\!\!e^{-\delta s}\left( L_{\varepsilon (s)}(p)v(X_s,\varepsilon _s)-\Delta _Qv(X_s,\varepsilon _s)\right) ds\nonumber \\{} & {} - \int _0^{t}\!\!\!\sigma _{\varepsilon (s)}e^{-\delta s}v'(X_s,\varepsilon _s)dW_s+M_{t}^f-M_0^f. \end{aligned}$$

Applying conditional expectation, we have

$$\begin{aligned} E_{x,i}\left[ \,e^{-\delta t}v(X_{t},\varepsilon _{t})\,\right]= & {} v(x,i) +\,E_{x,i}\left[ \,\int _0^{t}\!\!\!e^{-\delta s}(L_{\varepsilon (s)}(p)v(X_s,\varepsilon _s)-\Delta _Qv(X_s,\varepsilon _s))\,ds\,\right] \nonumber \\{} & {} - \,E_{x,i}\left[ \,\int _0^{t}\!\!\!\!\!\sigma _{\varepsilon (s)}e^{-\delta s}v'(X_s,\varepsilon _s)\,dW_s\,\right] +E_{x,i}\left[ \,M_{t}^f-M_0^f\,\right] \!.\quad \end{aligned}$$

(29)

From condition (25) and the linear growth of $v^{\prime }$, we have

$$\begin{aligned} E_{x,i}\left[ \,\int _0^{t}\!\!\!\sigma _{\varepsilon (s)}e^{-\delta s}v'(X_s,\varepsilon _s)dW_s\,\right] =0. \end{aligned}$$

Then, from Eqs. (28)–(29) we get

$$\begin{aligned} v(x,i)\le & {} E_{x,i}\left[ \,e^{-\delta t} v(X_{t},\varepsilon _{t})\,\right] + E_{x,i}\left[ \,\int _0^{t}\,e^{-\delta s} \alpha _{\varepsilon (s)} (X_{s} - {{\mathcal {I}}}_{\varepsilon (s)})^{2} ds \right] \nonumber \\{} & {} +E_{x,i}\left[ \,\int _0^{t}\!\!\!e^{-\delta s} k_{\varepsilon (s)}p(s)ds\,\right] .\quad \end{aligned}$$

(30)

Taking $t\rightarrow \infty$, we observe that condition (25) and the quadratic growth of v imply $E_{x,i}[\,e^{-\delta t}v(X_{t},\varepsilon _{t})\,]\,\,\rightarrow \,\, 0$. Therefore, for every arbitrary admissible control p, $v(x,i)\le J(x,i;p)$. In particular, inequality (30) becomes an equality if $p={\hat{p}}$, and hence $v(x,i)=J(x,i;{\hat{p}})=V(x,i)$. $\square$

4.2 Construction of the solution

We want to find a function v that satisfies the conditions of Theorem 8. We note that Eq. (28) is equivalent to

$$\begin{aligned}{} & {} \frac{1}{2}\sigma _{i}^2v''(x,i) - \mu _{i}v'(x,i)-\delta v(x,i)+\inf _{p\in [\,0,K_{i}\,]}\{p[k_{i}+v'(x,i)]\}\\{} & {} \quad = - \alpha _{i}(x- {{\mathcal {I}}}_{i})^{2} + \lambda _{i}\,v(x,i)-\sum _{j\ne i}q_{ij}\,v(x,j). \end{aligned}$$

We also note also that for every $\,t\in [\,0,\infty )$:

$$\begin{aligned} {\hat{p}}(t)\,=\,\arg \inf _{p\in [\,0,K_{\varepsilon _t }]} \left\{ p(t)\left[ k_{\varepsilon _t}+v'(X_t,\varepsilon _t)\right] \right\} = \left\{ \begin{array}{ccl} K_{\varepsilon _t }&{} \text{ if } &{}v'(X_t,\varepsilon _t) < -k_{\varepsilon _t}\\ 0&{} \text{ if } &{}v'(X_t,\varepsilon _t)\ge -k_{\varepsilon _t}. \end{array}\right. \end{aligned}$$

Hence, the candidate for optimal control ${\hat{p}}$ has the form ${\hat{p}}(t)=\varphi (X_t,\varepsilon _t)$ for $t\in [\,0,\infty )$, where $\varphi (\cdot ,i)$, $i\in {{\mathcal {S}}}$, is a measurable function defined by

$$\begin{aligned} \varphi (x,i)\,=\,\left\{ \begin{array}{ccl} K_{i} &{} \text{ if } &{}v'(x,i) < -k_{i} \\ 0 &{} \text{ if } &{} v'(x,i)\ge -k_{i} \end{array} \right. \end{aligned}$$

for $x \in {\mathbb {R}}$. Thus, to solve Eq. (28) is equivalent to solve the equation

$$\begin{aligned}{} & {} \frac{1}{2}\,\sigma _i^2\,v''(x,i) - \mu _i\,v'(x,i)-\delta \, v(x,i)+\varphi (x,i)\,[k_{i}+v'(x,i)]\,\,\\{} & {} \quad = - \alpha _{i}(x- {{\mathcal {I}}}_{i})^{2} + \,\,\lambda _i\,v(x,i)-\sum _{j\ne i}q_{ij}\,v(x,j) \end{aligned}$$

for each $i\in {{\mathcal {S}}}$; or equivalently, to solve

$$\begin{aligned}{} & {} \frac{1}{2}\,\sigma _i^2\,v''(x,i) +(K _i - \mu _i)\,v'(x,i)-\delta \, v(x,i)\,\,\\{} & {} \quad = - \alpha _{i}(x- {{\mathcal {I}}}_{i})^{2} -K_{i}k_{i} + \,\,\lambda _i\,v(x,i)-\sum _{j\ne i}q_{ij}\,v(x,j) \end{aligned}$$

when $v'(x,i) < -k_{i}$, and to solve

$$\begin{aligned} \frac{1}{2}\,\sigma _i^2\,v''(x,i)- \mu _i\,v'(x,i)-\delta \, v(x,i) \,\,= - \alpha _{i}(x- {{\mathcal {I}}}_{i})^{2} + \,\,\lambda _i\,v(x,i)-\sum _{j\ne i}q_{ij}\,v(x,j) \end{aligned}$$

when $v'(x,i)\ge -k_{i}$, for each $i\in {{\mathcal {S}}}$.

For simplicity, we assume in the remainder of this section that the economy shifts only between two different regimes, that is, ${{\mathcal {S}}}=\{1,2\}$.

Our objective in the remainder of this subsection is to find a candidate for value function and a candidate for optimal control. We conjecture $v'(\cdot ,i)$ to be continuous, with linear growth such that $v'(-\infty ,i)= - \infty$ and $v'(+\infty ,i)= \infty$ for both $i=1,2$. We also conjecture $v'(\cdot ,i)$, $i=1,2$, to be non-decreasing in order to have $v(\cdot ,i)$, $i=1,2$, convex in x. We will prove in Sect. 4.3 that these conjectures are satisfied by the value function.

According to our conjecture that $v'(\cdot ,i)$ is non-decreasing, $v'(-\infty ,i)= - \infty$ and $v'(+\infty ,i)= \infty$, $i=1,2$, there exists a threshold $b_i\in (-\infty ,\infty )$ such that $v'(b_i,i)= -k_{i}$. Hence, for $x\in (-\infty ,b_i)$ we have

$$\begin{aligned} \frac{1}{2}\sigma _i^2v''(x,i){+}(K_{i}{-} \mu _i)v'(x,i){-}\delta v(x,i){+} \alpha _{i}(x{-} {{\mathcal {I}}}_{i})^{2} {+} K_{i}k_{i}{=}\lambda _i\,v(x,i){-}\lambda _i\,v(x,3-i), \end{aligned}$$

and for $x\in [\,b_i,\infty )$ we have

$$\begin{aligned} \frac{1}{2}\sigma _i^2v''(x,i) - \mu _iv'(x,i)-\delta v(x,i) + \alpha _{i}(x- {{\mathcal {I}}}_{i})^{2} = \lambda _i\,v(x,i)-\lambda _i\,v(x,3-i). \end{aligned}$$

The relationship between $b_1$ and $b_2$ depends on the relations among the drift coefficients, the volatility parameters and the rates $\lambda _1$ and $\lambda _2$. We will only consider the case $b_1<b_2$, because the case $b_1>b_2$ has a similar treatment. Thus, we consider three possibilities for the initial level of the inventory: $x\in (-\infty ,b_1)$, $x\in [\,b_1,b_2)$, and $x\in [\,b_2,\infty )$.

When $x\in [b_2, \infty )$, Eq. (28) gives the following system of differential equations:

$$\begin{aligned} -(\lambda _1+\delta )\,v(x,1) - \mu _1\,v'(x,1)+\frac{1}{2}\,\sigma _1^2\,v''(x,1)+\lambda _1\,v(x,2) + \alpha _{1}(x- {{\mathcal {I}}}_{1})^{2}= & {} 0\nonumber \\ -(\lambda _2+\delta )\,v(x,2) - \mu _2\,v'(x,2)+\frac{1}{2}\,\sigma _2^2\,v''(x,2)+\lambda _2\,v(x,1) + \alpha _{2}(x- {{\mathcal {I}}}_{2})^{2}= & {} 0. \end{aligned}$$

(31)

Consider the characteristic equation for (31), $\phi ^1_1(\gamma )\,\,\phi ^1_2(\gamma )\,\,=\,\,\lambda _1\lambda _2,$ where

$$\begin{aligned} \phi ^1_i(\gamma )\,:=\,\,-\frac{1}{2}\,\sigma _i^2\gamma ^2 + \mu _i \,\gamma +(\lambda _i+\delta ),\qquad i=1,2. \end{aligned}$$

From Lemma 2, $\phi ^1_1(\gamma )\,\,\phi ^1_2(\gamma )\,\,=\,\,\lambda _1\lambda _2$ has 4 real roots: $\gamma _1<\gamma _2<0<\gamma _3<\gamma _4$. Then, the solution for the system of Eq. (31) is

$$\begin{aligned} v(x,1)\,\,= & {} \,\,A_1\,e^{\gamma _1(x-b_2)}\,+\,A_2\,e^{\gamma _2 (x-b_2)}\,+\,A_3\,e^{\gamma _3(x-b_2)}\,+\,A_4\,e^{\gamma _4 (x-b_2)}\\{} & {} + R_{1} (x-b_2)^{2} + S_{1} (x-b_2) + T_{1} \\ v(x,2)\,\,= & {} \,\,B_1\,e^{\gamma _1(x-b_2)}\,+\,B_2\,e^{\gamma _2 (x-b_2)}\,+\,B_3\,e^{\gamma _3(x-b_2)}\,+\,B_4\,e^{\gamma _4 (x-b_2)}\\{} & {} + R_{2} (x-b_2)^{2} + S_{2} (x-b_2) + T_{2}, \end{aligned}$$

where, for each $j=1,2,3,4$,

$$\begin{aligned} B_j\,\,=\,\,\frac{\phi ^1_1(\gamma _j)}{\lambda _1}\,A_j\,\,=\,\,\frac{\lambda _2}{\phi ^1_2(\gamma _j)}\,A_j. \end{aligned}$$

(32)

Furthermore, $R_{i}$, $S_{i}$ and $T_{i}$ are the solution of the system of 6 linear equations with 6 unknowns.

$$\begin{aligned} 0= & {} - (\lambda _{1} + \delta ) R_{1} + \lambda _{1} R_{2} + \alpha _{1} \nonumber \\ 0= & {} - (\lambda _{1} + \delta ) S_{1} -2 \mu _1 R_{1} + \lambda _{1} S_{2} + 2 \alpha _{1} (b_2 - {{\mathcal {I}}}_{1}) \nonumber \\ 0= & {} - (\lambda _{1} + \delta ) T_{1} - \mu _1 S_{1} + {\sigma }_{1}^{2} R_{1} + \lambda _{1} T_{2} + \alpha _{1} (b_2 - {{\mathcal {I}}}_{1})^{2} \nonumber \\ 0= & {} - (\lambda _{2} + \delta ) R_{2} + \lambda _{2} R_{1} + \alpha _{2} \nonumber \\ 0= & {} - (\lambda _{2} + \delta ) S_{2} -2 \mu _2 R_{2} + \lambda _{2} S_{1} + 2 \alpha _{2} (b_2 - {{\mathcal {I}}}_{2}) \nonumber \\ 0= & {} - (\lambda _{2} + \delta ) T_{2} - \mu _2 S_{2} + {\sigma }_{2}^{2} R_{2} + \lambda _{2} T_{1} + \alpha _{2} (b_2 - {{\mathcal {I}}}_{2})^{2}. \end{aligned}$$

(33)

Recall that we are conjecturing that $v(\cdot ,i)$ has quadratic growth. Thus, $B_{3}=B_{4}=A_{3}=A_{4}=0$. Hence, the solution of the system (31) is given by

$$\begin{aligned} v(x,1)\,\,= & {} \,\,A_1\,e^{\gamma _1(x-b_2)}\,+\,A_2\,e^{\gamma _2 (x-b_2)}\,+ R_{1} (x-b_2)^{2} + S_{1} (x-b_2) + T_{1} \end{aligned}$$

(34)

$$\begin{aligned} v(x,2)\,\,= & {} \,\,B_1\,e^{\gamma _1(x-b_2)}\,+\,B_2\,e^{\gamma _2 (x-b_2)}\,+ R_{2} (x-b_2)^{2} + S_{2} (x-b_2) + T_{2}, \end{aligned}$$

(35)

for $x\in [b_2,\infty )$.

When $x\in [\,b_1,b_2)$, Eq. (28) gives the system of differential equations

$$\begin{aligned}{} & {} -(\lambda _1+\delta )\,v(x,1) - \mu _1\,v'(x,1)+\frac{1}{2}\,\sigma _1^2\,v''(x,1)+\lambda _1\,v(x,2) + \alpha _{1}(x- {{\mathcal {I}}}_{1})^{2} =0 \end{aligned}$$

(36)

$$\begin{aligned}{} & {} -(\lambda _2+\delta )\,v(x,2)+(K_{2}-\mu _2)\,v'(x,2)+\frac{1}{2}\,\sigma _2^2\,v''(x,2)+\lambda _2\,v(x,1)\nonumber \\{} & {} \quad \quad + K_{2}k_{2} + \alpha _{2}(x- {{\mathcal {I}}}_{2})^{2} =0. \end{aligned}$$

(37)

Consider the characteristic equation for the system (36)–(37), $\phi ^2_1({\tilde{\gamma }})\,\,\phi ^2_2({\tilde{\gamma }})\,\,=\,\,\lambda _1\lambda _2,$ where

$$\begin{aligned} \phi ^2_1({\tilde{\gamma }}):= & {} -\frac{1}{2}\,\sigma _1^2{\tilde{\gamma }}^2 + \mu _1 \,{\tilde{\gamma }}+(\lambda _1+\delta )\\ \phi ^2_2({\tilde{\gamma }}):= & {} -\frac{1}{2}\,\sigma _2^2{\tilde{\gamma }}^2 + ( \mu _2 - K_{2}) \,{\tilde{\gamma }}+(\lambda _2+\delta ). \end{aligned}$$

From Lemma 2, $\phi ^2_1({\tilde{\gamma }})\,\,\phi ^2_2({\tilde{\gamma }})\,\,=\,\,\lambda _1\lambda _2$ has 4 real roots: ${\tilde{\gamma }}_1<{\tilde{\gamma }}_2<0<{\tilde{\gamma }}_3<{\tilde{\gamma }}_4$. Then, we obtain that the solution for the system of Eqs. (36)–(37) is:

$$\begin{aligned} v(x,1)\,\,= & {} \,\,{\widetilde{A}}_1\,e^{{\tilde{\gamma }}_1 (x-b_1)}\,+\,{\widetilde{A}}_2\,e^{{\tilde{\gamma }}_2 (x-b_1)}\,+\,{\widetilde{A}}_3\,e^{{\tilde{\gamma }}_3 (x-b_1)}\,+\,{\widetilde{A}}_4\,e^{{\tilde{\gamma }}_4 (x-b_1)}\, \nonumber \\{} & {} + {\tilde{R}}_{1} (x-b_1)^{2} + {\tilde{S}}_{1} (x-b_1) + {\tilde{T}}_{1} \end{aligned}$$

(38)

$$\begin{aligned} v(x,2)\,\,= & {} \,\,{\widetilde{B}}_1\,e^{{\tilde{\gamma }}_1 (x-b_1)}\,+\,{\widetilde{B}}_2\,e^{{\tilde{\gamma }}_2 (x-b_1)}\,+\,{\widetilde{B}}_3\,e^{{\tilde{\gamma }}_3 (x-b_1)}\,+\,{\widetilde{B}}_4\,e^{{\tilde{\gamma }}_4 (x-b_1)}\, \nonumber \\{} & {} + {\tilde{R}}_{2} (x-b_1)^{2} + {\tilde{S}}_{2} (x-b_1) + {\tilde{T}}_{2}, \end{aligned}$$

(39)

where ${\tilde{R}}_{i}$, ${\tilde{S}}_{i}$ and ${\tilde{T}}_{i}$ are the solution of the following system of 6 linear equations with 6 unknowns.

$$\begin{aligned} 0= & {} - (\lambda _{1} + \delta ) {\tilde{R}}_{1} + \lambda _{1} {\tilde{R}}_{2} + \alpha _{1} \nonumber \\ 0= & {} - (\lambda _{1} + \delta ) {\tilde{S}}_{1} -2 \mu _1 {\tilde{R}}_{1} + \lambda _{1} {\tilde{S}}_{2} + 2 \alpha _{1} (b_1 - {{\mathcal {I}}}_{1}) \nonumber \\ 0= & {} - (\lambda _{1} + \delta ) {\tilde{T}}_{1} - \mu _1 {\tilde{S}}_{1} + {\sigma }_{1}^{2} {\tilde{R}}_{1} + \lambda _{1} {\tilde{T}}_{2} + \alpha _{1} (b_1 - {{\mathcal {I}}}_{1})^{2} \nonumber \\ 0= & {} - (\lambda _{2} + \delta ) {\tilde{R}}_{2} + \lambda _{2} {\tilde{R}}_{1} + \alpha _{2} \nonumber \\ 0= & {} - (\lambda _{2} + \delta ) {\tilde{S}}_{2} + 2(K_{2} - \mu _2 ) {\tilde{R}}_{2} + \lambda _{2} {\tilde{S}}_{1} + 2 \alpha _{2} (b_1 - {{\mathcal {I}}}_{2}) \nonumber \\ 0= & {} - (\lambda _{2} + \delta ) {\tilde{T}}_{2} + (K_{2} - \mu _2 ) {\tilde{S}}_{2} + {\sigma }_{2}^{2} {\tilde{R}}_{2} + \lambda _{2} {\tilde{T}}_{1} + K_{2} k_{2} + \alpha _{2} (b_1 - {{\mathcal {I}}}_{2})^{2}. \end{aligned}$$

(40)

Moreover, for each $j=1,2,3,4$, the condition for the coefficients is

$$\begin{aligned} {\widetilde{B}}_j\,\,=\,\,\frac{\phi ^2_1({\tilde{\gamma }}_j)}{\lambda _1}\,{\widetilde{A}}_j\,\,=\,\,\frac{\lambda _2}{\phi ^2_2({\tilde{\gamma }}_j)}\,{\widetilde{A}}_j. \end{aligned}$$

(41)

When $x\in (-\infty , b_{1})$, the function v is the solution of the system

$$\begin{aligned}{} & {} -(\lambda _1+\delta )\,v(x,1)+(K_{1}-\mu _1)\,v'(x,1)+\frac{1}{2}\,\sigma _1^2\,v''(x,1)+\lambda _1\,v(x,2)\nonumber \\{} & {} \quad \quad + K_{1}k_{1} + \alpha _{1}(x- {{\mathcal {I}}}_{1})^{2} =0. \end{aligned}$$

(42)

$$\begin{aligned}{} & {} -(\lambda _2+\delta )\,v(x,2)+(K_{2}-\mu _2)\,v'(x,2)+\frac{1}{2}\,\sigma _2^2\,v''(x,2)+\lambda _2\,v(x,1)\nonumber \\{} & {} \quad \quad + K_{2}k_{2} + \alpha _{2}(x- {{\mathcal {I}}}_{2})^{2} =0. \end{aligned}$$

(43)

Consider the characteristic equation for (42)–(43), $\phi ^3_1({\hat{\gamma }})\,\,\phi ^3_2({\hat{\gamma }})\,\,=\,\,\lambda _1\lambda _2,$ where

$$\begin{aligned} \phi ^3_i({\hat{\gamma }})\,\,:=\,\,-\frac{1}{2}\,\sigma _i^2{\hat{\gamma }}^2 -(K_{i} - \mu _i)\,{\hat{\gamma }}+(\lambda _i+\delta ). \end{aligned}$$

From Lemma 2, $\phi ^3_1({\hat{\gamma }})\,\,\phi ^3_2({\hat{\gamma }})\,\,=\,\,\lambda _1\lambda _2$ has 4 real roots: ${\hat{\gamma }}_1<{\hat{\gamma }}_2<0<{\hat{\gamma }}_3<{\hat{\gamma }}_4$. Then, the solution for the system of differential Eqs. (42)–(43) is given by:

$$\begin{aligned} v(x,1)= & {} {\widehat{A}}_1e^{{\hat{\gamma }}_1 (x-b_1)}+{\widehat{A}}_2e^{{\hat{\gamma }}_2 (x-b_1)}+{\widehat{A}}_3e^{{\hat{\gamma }}_3 (x-b_1)}+{\widehat{A}}_4e^{{\hat{\gamma }}_4 (x-b_1)} \\{} & {} + {\widehat{R}}_{1} (x-b_1)^{2} + {\widehat{S}}_{1} (x-b_1) + {\widehat{T}}_{1}\\ v(x,2)= & {} {\widehat{B}}_1e^{{\hat{\gamma }}_1 (x-b_1)}+{\widehat{B}}_2e^{{\hat{\gamma }}_2 (x-b_1)}+{\widehat{B}}_3e^{{\hat{\gamma }}_3 (x-b_1)}+{\widehat{B}}_4e^{{\hat{\gamma }}_4 (x-b_1)} \\{} & {} + {\widehat{R}}_{2} (x-b_1)^{2} + {\widehat{S}}_{2} (x-b_1) + {\widehat{T}}_{2}, \end{aligned}$$

where ${\widehat{R}}_{i}$, ${\widehat{S}}_{i}$ and ${\widehat{T}}_{i}$, $i \in \{1,2\}$, are the solution of the system of 6 linear equations with 6 unknowns.

$$\begin{aligned} 0= & {} - (\lambda _{1} + \delta ) {\widehat{R}}_{1} + \lambda _{1} {\widehat{R}}_{2} + \alpha _{1} \nonumber \\ 0= & {} - (\lambda _{1} + \delta ) {\widehat{S}}_{1} +2 (K_{1}-\mu _1) {\widehat{R}}_{1} + \lambda _{1} {\widehat{S}}_{2} + 2 \alpha _{1} (b_1 - {{\mathcal {I}}}_{1}) \nonumber \\ 0= & {} - (\lambda _{1} + \delta ) {\widehat{T}}_{1} +(K_{1} - \mu _1) {\widehat{S}}_{1} + {\sigma }_{1}^{2} {\widehat{R}}_{1} + \lambda _{1} {\widehat{T}}_{2} + K_{1}k_{1} + \alpha _{1} (b_1 - {{\mathcal {I}}}_{1})^{2} \nonumber \\ 0= & {} - (\lambda _{2} + \delta ) {\widehat{R}}_{2} + \lambda _{2} {\widehat{R}}_{1} + \alpha _{2} \nonumber \\ 0= & {} - (\lambda _{2} + \delta ) {\widehat{S}}_{2} + 2(K_{2} - \mu _2 ) {\widehat{R}}_{2} + \lambda _{2} {\widehat{S}}_{1} + 2 \alpha _{2} (b_1 - {{\mathcal {I}}}_{2}) \nonumber \\ 0= & {} - (\lambda _{2} + \delta ) {\widehat{T}}_{2} + (K_{2} - \mu _2 ) {\widehat{S}}_{2} + {\sigma }_{2}^{2} {\widehat{R}}_{2} + \lambda _{2} {\widehat{T}}_{1} + K_{2} k_{2} + \alpha _{2} (b_1 - {\mathcal I}_{2})^{2}. \end{aligned}$$

(44)

Furthermore, the condition for the coefficients, for each $j=1,2,3,4$, is

$$\begin{aligned} {\widehat{B}}_j\,\,=\,\,\frac{\phi ^3_1({\hat{\gamma }}_j)}{\lambda _1}\,{\widehat{A}}_j\,\,= \,\,\frac{\lambda _2}{\phi ^3_2({\hat{\gamma }}_j)}\,{\widehat{A}}_j. \end{aligned}$$

(45)

We recall that we are conjecturing that the function v has quadratic growth, which implies ${\widehat{A}}_1={\widehat{A}}_2={\widehat{B}}_1={\widehat{B}}_2=0$. Therefore,

$$\begin{aligned}{} & {} v(x,1)\,\,=\,\, {\widehat{A}}_3e^{{\hat{\gamma }}_3 (x-b_1)}+{\widehat{A}}_4e^{{\hat{\gamma }}_4 (x-b_1)} + {\widehat{R}}_{1} (x-b_1)^{2} + {\widehat{S}}_{1} (x-b_1) + {\widehat{T}}_{1} \end{aligned}$$

(46)

$$\begin{aligned}{} & {} v(x,2)\,\,=\,\, {\widehat{B}}_3e^{{\hat{\gamma }}_3 (x-b_1)}+{\widehat{B}}_4e^{{\hat{\gamma }}_4 (x-b_1)} + {\widehat{R}}_{2} (x-b_1)^{2} + {\widehat{S}}_{2} (x-b_1) + {\widehat{T}}_{2} \end{aligned}$$

(47)

is the solution for the system (42)–(43), where (45) is satisfied for $j=1,2$.

To find the thresholds $b_1$ and $b_2$, and the coefficients of v in (34), (38) and (46), we conjecture that the smooth-fit condition holds. We also want $v'(b_i,i)= - k_{i}$ for each $i=1,2$. Thus, we need to solve the following system of 10 equations with 10 unknowns:

$$\begin{aligned} \begin{array}{lcl} v(b_i-,i)&{}=&{}v(b_i+,i)\\ v(b_{3-i}-,i)&{}=&{}v(b_{3-i}+,i)\\ v'(b_i-,i)&{}=&{} - k_{i}\\ v'(b_i+,i)&{}=&{} - k_{i}\\ v'(b_{3-i}-,i)&{}=&{}v'(b_{3-i}+,i), \end{array} \end{aligned}$$

(48)

for each $i=1,2$. The solution of the system of Eq. (48) will give us the values for $b_1$ and $b_2$, and also the values for $A_j$, $j=1,2$, ${\widetilde{A}}_j$, $j=1,2,3,4$, and ${\widehat{A}}_j$, $j=3,4$. The values for the corresponding $B_j$, ${\widetilde{B}}_j$ and ${\widehat{B}}_j$ can be found from Eqs. (32), (41), and (45).

For future use, we observe here that $v^{\prime \prime }(b_{1},1) \ge 0$ is equivalent to

$$\begin{aligned} {\widehat{A}}_3 {\hat{\gamma }}_3^{2} + {\widehat{A}}_4 {\hat{\gamma }}_4^{2} + 2 {\widehat{R}}_{1} \; \ge \; 0, \end{aligned}$$

(49)

and $v^{\prime \prime }(b_{2},2) \ge 0$ is equivalent to

$$\begin{aligned} B_1 \gamma _1^2 +\,B_2\, \gamma _2^2 + 2 R_{2} \ge 0. \end{aligned}$$

(50)

4.3 Verification of the solution

In Sect. 4.2, we made some conjectures to find a candidate v for value function. In this subsection, we will prove that v is indeed the value function V of Problem 3 when ${\mathcal S}=\{1,2\}$. We will also present the optimal production policy. First, we investigate the regularity of the candidate for value function.

Lemma 9

Let $A_j$, $j \in \{1,2\}$, ${\widetilde{A}}_j$, $j \in \{1,2,3,4\}$, and ${\widehat{A}}_j$, $j \in \{3,4\}$, be the solution of the system of Eq. (48). Let $B_j$, $j \in \{1,2\}$, ${\widetilde{B}}_j$, $j \in \{1,2,3,4\}$, and ${\widehat{B}}_j$, $j \in \{3,4\}$, be defined by (32), (41) and (45). Suppose that $b_1<b_2$. Then, the function v given by

$$\begin{aligned} v(x,1)\,\,= \,\left\{ \begin{array}{lll} {\widehat{A}}_3e^{{\hat{\gamma }}_3 (x-b_1)}+{\widehat{A}}_4e^{{\hat{\gamma }}_4 (x-b_1)} + {\widehat{R}}_{1} (x-b_1)^{2} + {\widehat{S}}_{1} (x-b_1) + {\widehat{T}}_{1} &{} \text{ if } &{} x\in (-\infty ,b_1),\\ {\widetilde{A}}_1\,e^{{\tilde{\gamma }}_1 (x-b_1)}\,+\,{\widetilde{A}}_2\,e^{{\tilde{\gamma }}_2 (x-b_1)}\,+\,{\widetilde{A}}_3\,e^{{\tilde{\gamma }}_3 (x-b_1)}\,+\,{\widetilde{A}}_4\,e^{{\tilde{\gamma }}_4 (x-b_1)}\, &{} &{} \\ + {\tilde{R}}_{1} (x-b_1)^{2} + {\tilde{S}}_{1} (x-b_1) + {\tilde{T}}_{1} &{} \text{ if } &{}x\in [\,b_1,b_2),\\ A_1\,e^{\gamma _1(x-b_2)}\,+\,A_2\,e^{\gamma _2 (x-b_2)}\,+ R_{1} (x-b_2)^{2} + S_{1} (x-b_2) + T_{1} &{} \text{ if } &{}x\in [\,b_2,\infty ), \end{array}\right. \end{aligned}$$

(51)

and

$$\begin{aligned} v(x,2)\,\,=\, \left\{ \begin{array}{lll} {\widehat{B}}_3e^{{\hat{\gamma }}_3 (x-b_1)}+{\widehat{B}}_4e^{{\hat{\gamma }}_4 (x-b_1)} + {\widehat{R}}_{2} (x-b_1)^{2} + {\widehat{S}}_{2} (x-b_1) + {\widehat{T}}_{2} &{} \text{ if } &{} x\in (-\infty ,b_1),\\ {\widetilde{B}}_1\,e^{{\tilde{\gamma }}_1 (x-b_1)}\,+\,{\widetilde{B}}_2\,e^{{\tilde{\gamma }}_2 (x-b_1)}\,+\,{\widetilde{B}}_3\,e^{{\tilde{\gamma }}_3 (x-b_1)}\,+\,{\widetilde{B}}_4\,e^{{\tilde{\gamma }}_4 (x-b_1)}\, &{} &{} \\ + {\tilde{R}}_{2} (x-b_1)^{2} + {\tilde{S}}_{2} (x-b_1) + {\tilde{T}}_{2} &{} \text{ if } &{}x\in [\,b_1,b_2),\\ B_1\,e^{\gamma _1(x-b_2)}\,+\,B_2\,e^{\gamma _2 (x-b_2)}\,+ R_{2} (x-b_2)^{2} + S_{2} (x-b_2) + T_{2} &{} \text{ if } &{}x\in [\,b_2,\infty ), \end{array}\right. \end{aligned}$$

(52)

is such that $v(\cdot ,i)\in {\mathcal {C}}^{\infty }({\mathbb {R}} {\setminus }\{b_1,b_2\}) \cap {\mathcal {C}}^{4}({\mathbb {R}} {\setminus }\{b_{3-i} \})\cap {\mathcal {C}}^{2}({\mathbb {R}})$.

Proof

By construction and the smooth-fit conditions, see (48), the candidate for value function satisfies $v(\cdot ,i) \in {\mathcal {C}}^{\infty }({\mathbb {R}} {\setminus }\{b_1,b_2\}) \cap {\mathcal {C}}^1({\mathbb {R}})$. Furthermore, again by construction, for $x\in [b_2, \infty )$, one has v solves

$$\begin{aligned} \begin{aligned} -(\lambda _1+\delta )\,v(x,1) - \mu _1\,v'(x,1)+\frac{1}{2}\,\sigma _1^2\,v''(x,1)+\lambda _1\,v(x,2) + \alpha _{1}(x- {{\mathcal {I}}}_{1})^{2}&=0 \\ -(\lambda _2+\delta )\,v(x,2) - \mu _2\,v'(x,2)+\frac{1}{2}\,\sigma _2^2\,v''(x,2)+\lambda _2\,v(x,1) + \alpha _{2}(x- {{\mathcal {I}}}_{2})^{2}&=0, \end{aligned} \end{aligned}$$

(53)

for $x\in [\,b_1,b_2)$, v solves

$$\begin{aligned} \begin{aligned}&-(\lambda _1+\delta )\,v(x,1) - \mu _1\,v'(x,1)+\frac{1}{2}\,\sigma _1^2\,v''(x,1)+\lambda _1\,v(x,2) + \alpha _{1}(x- {{\mathcal {I}}}_{1})^{2} =0 \\&-(\lambda _2+\delta )\,v(x,2)+(K_{2}-\mu _2)\,v'(x,2)+\frac{1}{2}\,\sigma _2^2\,v''(x,2)+\lambda _2\,v(x,1)\\&\quad \quad + K_{2}k_{2} + \alpha _{2}(x- {{\mathcal {I}}}_{2})^{2}=0,&\end{aligned} \end{aligned}$$

(54)

and for $x\in (-\infty , b_{1})$, v solves

$$\begin{aligned} \begin{aligned}&-(\lambda _1+\delta )\,v(x,1)+(K_{1}-\mu _1)\,v'(x,1)+\frac{1}{2}\,\sigma _1^2\,v''(x,1)+\lambda _1\,v(x,2)\\&\quad \quad + K_{1}k_{1} + \alpha _{1}(x- {{\mathcal {I}}}_{1})^{2} =0 \\&-(\lambda _2+\delta )\,v(x,2)+(K_{2}-\mu _2)\,v'(x,2)+\frac{1}{2}\,\sigma _2^2\,v''(x,2)+\lambda _2\,v(x,1)\\&\quad \quad + K_{2}k_{2} + \alpha _{2}(x- {{\mathcal {I}}}_{2})^{2} =0. \end{aligned} \end{aligned}$$

(55)

By construction and system (48), we obtain that $v(\cdot ,i) \in {\mathcal {C}}^{\infty }({\mathbb {R}} {\setminus }\{b_1,b_2\}) \cap {\mathcal {C}}^1({\mathbb {R}})$ and $v'(b_2,2)=-k_2$. Hence, combining (53) and (54), we obtain first that $v(\cdot ,2) \in {\mathcal {C}}^2((b_1,\infty ))$ and further that $v(\cdot ,1) \in {\mathcal {C}}^4((b_1,\infty ))$. Using that $v'(b_1,1)=-k_1$, combining (54) and (55), gives that $v(\cdot ,1) \in {\mathcal {C}}^2((-\infty ,b_2))$ and further that $v(\cdot ,2) \in {\mathcal {C}}^4((-\infty ,b_2))$. Therefore, $v(\cdot ,i) \in {\mathcal {C}}^{\infty }({\mathbb {R}} {\setminus }\{b_1,b_2\}) \cap {\mathcal {C}}^{4}({\mathbb {R}}{\setminus }\{b_{3-i} \})\cap {\mathcal {C}}^{2}({\mathbb {R}})$. $\square$

Theorem 10

Let $A_j$, $j \in \{1,2\}$, ${\widetilde{A}}_j$, $j \in \{1,2,3,4\}$, and ${\widehat{A}}_j$, $j \in \{3,4\}$, be the solution of the system of Eq. (48). Let $B_j$, $j \in \{1,2\}$, ${\widetilde{B}}_j$, $j \in \{1,2,3,4\}$, and ${\widehat{B}}_j$, $j \in \{3,4\}$, be defined by (32), (41) and (45). Suppose that the conditions (49)–(50) are satisfied. Assume that $b_1<b_2$. Then, the function v defined by (51)–(52) is the value function V of Problem 3. Furthermore, ${\hat{p}}$ defined by

$$\begin{aligned} {\hat{p}}(t)\,=\,\left\{ \begin{array}{lllll} K_{i} &{} \text{ if } &{}\varepsilon _t=i&{} \text{ and } &{}X_t\in (- \infty , b_i),\\ 0 &{} \text{ if } &{}\varepsilon _t=i&{} \text{ and } &{}X_t\in [\,b_i,\infty ),\end{array}\right. \end{aligned}$$

(56)

is optimal production rate policy for Problem 3.

Proof

The idea of the proof is to check that the function v defined by (51)–(52), and the control ${\hat{p}}$ defined by (56) satisfy all the conditions of Theorem 4.1. That will prove that they are the value function and the optimal production rate, respectively, for Problem 3.

First, we show that v is convex. Let us denote by $X^{x,i}$ the process with $X(0)=X_0=x$ and $\varepsilon _0= \varepsilon (0)= i$. Given the regularity of v, see Lemma 9, we can apply Itô’s formula, for given $i \in {\mathcal {S}}$ and $x \ne b_i$, on $e^{-\delta s}v''(X^{x,i}_s,\varepsilon _s)$. Hence, we obtain

$$\begin{aligned} E_{x,i}\left[ e^{-\delta \tau }v''(X_{\tau },\varepsilon _{\tau })\right] = v''(x,i) + E_{x,i}\left[ \int _0^{\tau } e^{-\delta s} A^i v''(X_s,\varepsilon _s) ~ds \right] , \end{aligned}$$

where $A^i \psi (x,i):=\frac{1}{2} \sigma _i \psi ''(x,i)+(K_i \mathbbm {1}_{\{x<b_i\}}-\mu _i) \psi '(x,i)-(\lambda _i+\delta ) \psi (x,i)- \lambda _i \psi (x,3-i)$ and $\tau$ is a stopping time such that $E_{x,i}\left[ \tau \right] <\infty$. Since v solves the Hamilton–Jacobi–Bellman equation, the regularity of v implies

$$\begin{aligned} \frac{1}{2}\,\sigma _i^2\,v^{''''}(x,i) +(K - \mu _i)\,v^{'''}(x,i)-(\delta + \lambda _i) v''(x,i)+\lambda _i v''(x,3-i)= -2 \alpha _{i} \end{aligned}$$

when $x < b_{i}$, and

$$\begin{aligned} \frac{1}{2}\,\sigma _i^2\,v^{''''}(x,i) - \mu _i\,v^{'''}(x,i)-(\delta + \lambda _i) v''(x,i)+\lambda _i v''(x,3-i)= -2 \alpha _{i} \end{aligned}$$

when $x\ge b_i$.

Taking $\tau :=\inf \{t\ge 0: (X_t,\varepsilon (t)) \in \{(b_1,1),(b_2,2)\} \}$, we obtain, for every $i \in \{1,2\}$ and $x> b_i$:

$$\begin{aligned} v''(x,i)&= E_{x,i}\left[ 2\int _0^{\tau } e^{-\delta s} \alpha _{\varepsilon _s} ~ds +e^{-\delta \tau } v''(b_1,1) \mathbbm {1}_{\{ (X_{\tau }, \varepsilon _{\tau })=(b_1,1) \}} \nonumber \right. \\&\quad \left. +e^{-\delta \tau } v''(b_2,2) \mathbbm {1}_{\{ (X_{\tau }, \varepsilon _{\tau })=(b_2,2) \}}\right] . \end{aligned}$$

(57)

We recall that we are assuming (49)–(50), or equivalently $v''(b_i,i) \ge 0$. Hence, $v''(x,i) \ge 0$ (in particular $v''(x,i)>0$) for $x> b_i$ and for every $i \in \{1,2\}$.

Analogously, letting $x< b_i$ and defining $\tau :=\inf \{t\ge 0: (X_t,\varepsilon (t)) \in \{(b_1,1),(b_2,2)\} \}$ we still obtain (57), for every $i \in \{1,2\}$. Using once more (49)–(50), we find that $v''(x,i) \ge 0$ (in particular $v''(x,i)>0$) also on $x<b_i$ for every $i \in \{1,2\}$, and we thus conclude that $v(\cdot ,i)$ is convex on ${\mathbb {R}}$.

Since v is convex, it follows immediately by construction that v solves (28). That is, v solves

$$\begin{aligned} \inf _{p \in [\,0,K_i \,]}\{L_{i}(p)\,v(x,i)+ k_{i} p\}\,= - \alpha _{i} (x - {{\mathcal {I}}}_{i})^{2} + \,\lambda _{i}\,v(x,i)-\sum _{j\in {\mathcal S}\setminus \{i\}}q_{ij}\,v(x,j). \end{aligned}$$

Moreover, the quadratic growth (as well as the linear growth of $v^{\prime }$) is fulfilled by construction. $\square$

Therefore, the optimal production policy when the regime is i is the following: (a) do not produce when the inventory level is above the threshold $b_{i}$, and (b) whenever the inventory level is equal to lower than $b_{i}$, increase the production at the maximum rate $K_i$. We observe that the company should also increase production to $K_2$ just because of a change in the regime: that happens when the level of the inventory falls in the interval $(b_1, b_2)$ and the regime changes from $i=1$ to $i=2$.

4.4 Comparative statics and numerical examples

We introduce the benchmark case in which

$$\begin{aligned} \mu _1=0.2,~\mu _2=1.0,~\sigma _1=\sigma _2=0.2,~\delta =0.2,~\alpha _1=\alpha _2=0.2 \end{aligned}$$

and

$$\begin{aligned} I_1=I_2=2.0,~ k_1=k_2=2.0,~ \lambda _1=\lambda _2=0.2,~K_1=K_2=1.0. \end{aligned}$$

This benchmark case is the same as in the case in which the production rate is unbounded from above (see Sect. 3.4), but expanded by the two new parameters $K_1=K_2=1$, that represent the upper bounds on the production rate.

We obtain the following solution to the system (48):

$$\begin{aligned}{} & {} {\widehat{A}}_3=-14.267,~{\widehat{A}}_4=0.001, {\tilde{A}}_1=-0.002,~ {\tilde{A}}_2=-1.085,~ {\tilde{A}}_3=-0.006,~ {\tilde{A}}_4=0.000, \\{} & {} A_1=-0.138,~ A_2=-14.465,~ b_1=0.976,~ b_2=1.201. \end{aligned}$$

We have checked that the conditions (49)–(50) are satisfied.

Hence, our candidate v for value function is indeed the true value function V and is shown in Fig. 5. The optimal production policy is given by

$$\begin{aligned} {\hat{p}}(t,1) = \left\{ \begin{array}{ll} K_1 = 1 &{} \text{ if } X_t \le b_1=0.976 \\ 0 &{} \text{ if } X_t> b_1=0.976 \end{array} \right. \;\;\; \text{ and } \;\;\; {\hat{p}}(t,2) = \left\{ \begin{array}{ll} K_2 = 1 &{} \text{ if } X_t \le b_2=1.201 \\ 0 &{} \text{ if } X_t > b_2=1.201. \end{array} \right. \end{aligned}$$

Fig. 5

Graphical illustration of the value function for the benchmark case

We observe that the production thresholds $b_i$ are higher than in the case in which the production rates are unbounded from above. This is a consequence of the production constraints given by $K_i$. In order that the inventory levels be close to the production targets $I_i$, it is necessary to start production earlier (higher $b_i$) when the production rates are bounded from above.

Now, we want to see how the solution depends on the different parameters.

First, we study the dependence on $\mu _i$. Figure 6 shows the results.

Fig. 6

Graphical illustration of the dependence of the boundaries on $\mu _i$. The other parameters are assumed to be the same as in the benchmark case

One can observe that both boundaries increase if one of the demand drifts increases. This is plausible, since an increase in $\mu _i$ increases the future demand, so the firm starts the production earlier. It is interesting to notice that an increase in $\mu _i$ affects not only $b_i$ but also $b_{3-i}$. Moreover, as expected, $b_1=b_2$ when $\mu _1 = \mu _2$.

Next, we investigate the dependence with respect to $\lambda _i$. The results are presented in Fig. 7.

Fig. 7

Graphical illustration of the dependence of the boundaries on $\lambda _i$. The other parameters are assumed to be the same as in the benchmark case

We see that both thresholds are increasing in $\lambda _1$ and decreasing in $\lambda _2$. This is consistent with the unbounded-from-above case. We recall that the amount of time that the demand is in regime i, before moving to regime $3-i$, has exponential distribution with mean $\frac{1}{ \lambda _{i}}$. Hence, a lower expected amount of time in the low-demand regime, before switching to the high-demand regime, increases both thresholds, and thus there is more production. On the other hand, a lower expected amount of time in the high-demand regime, before switching to the low-demand regime, decreases both thresholds, and thus there is less production.

Next, we study the dependence on $\sigma _i$. Figure 8 shows two 3D plots, one for each boundary.

Fig. 8

Graphical illustration of the dependence of the production thresholds on $\sigma _i$. The other parameters are assumed to be the same as in the benchmark case

We observe an interesting effect. Both thresholds are decreasing in $\sigma _1$, as in the case in which the production rates are unbounded from above. However, both thresholds are increasing in $\sigma _2$, which differs from the case in which the production rates are unbounded from above. This means that a higher uncertainty in the low-demand regime leads the firm to start the production later (smaller $b_i$). However, a higher uncertainty in the high-demand regime leads to more productivity (bigger $b_i$).

Finally, we investigate the dependence of the production thresholds $b_i$, $i \in \{1,2\}$ on the production costs $k_i$, $i \in \{1,2\}$. Figure 9 shows that $k_i \mapsto b_i(k_i)$ is nonincreasing and $k_{3-i} \mapsto b_i(k_{3-i})$ is nondecreasing. Thus, we observe the same behavior as in the case in which the production rates are unbounded from above (see Lemma 6).

Fig. 9

Graphical illustration of the dependence of the production thresholds on the production costs $k_i$, $i \in \{1,2\}$. The other parameters are assumed to be the same as in the benchmark case

5 Comparison among different models

5.1 Comparison between the case in which the production rates are unbounded from above and the case in which the production rates are bounded from above

In this subsection, we compare the model of Sect. 3 in which the production rates are unbounded from above with the model of Sect. 4 in which the production rates are bounded from above. We start by considering again the two benchmark cases. Both have the same parameter values, except the additional parameters $K_1=1$ and $K_2=1$ in the case in which the production rates are bounded from above.

We denote by $b_i^{U}$, $i \in \{1,2\}$, the thresholds in the case in which the production rates are unbounded from above, and by $b_i^{B}$, $i \in \{1,2\}$, the thresholds in the case in which the production rates are bounded from above. For the benchmark cases, we obtain

$$\begin{aligned} b_1^{B}=0.976> 0.914=b_1^{U} \quad \text { and } \quad b_2^{B}=1.201 > 0.977=b_2^{U}. \end{aligned}$$

Hence, the production in the bounded-from-above case starts earlier than in the unbounded-from-above case. This is reasonable because in the unbounded-from-above case, production can be instantaneous and the firm can tolerate a lower inventory level, while in the bounded-from-above case the inventory level can decrease even though the production is maximal.

Next, we denote by $V_{B}$ the value function for the bounded-from-above case and by $V_{U}$ the value function for the unbounded-from-above case. One can expect that the value function for the bounded-from-above case converges to the value function for the unbounded-from-above case when $K_i$, $i \in \{1,2\}$, increases. To show this fact, we introduce Fig. 10. Here, we show the relative distance of the value functions with respect to certain values for $K_i$. For simplicity, we consider $K_1=K_2=K$ and study the influence of K.

Fig. 10

Graphical illustration of the relative change of the value functions for some values of $K_i$. The other parameters are assumed to be the same as in the benchmark case

One can see that the relative change is decreasing in K, which is exactly the expected result. Hence, if the upper bound for the production rate is high enough, the unbounded-from-above case can provide good estimates for the bounded-from-above case.

A similar result can be shown for the boundaries. This is shown in Fig. 11.

Fig. 11

Graphical illustration of the absolute difference of the thresholds for some values of $K_i$. The other parameters are assumed to be the same as in the benchmark case

5.2 The case in which the production rates are unbounded from above: comparison between regime switching and no regime switching

In this subsection, we compare the model with two regimes with the model with only one regime (equivalently, the model without regime switching) for the case in which the production rates are unbounded from above.

If there is no regime switching, we can obtain explicitly the threshold

$$\begin{aligned} b \; = \; \frac{- \delta (k - \frac{2 \alpha }{\delta \gamma _1})+2 \mu \frac{\alpha }{\delta }+2 \alpha I}{2 \alpha } \; = \; I + \frac{\mu }{\delta } + \frac{1}{\gamma _1} - \frac{\delta }{2} \left( \frac{k}{\alpha } \right) ,\end{aligned}$$

where $\gamma _1$ is the negative solution of $\frac{1}{2}\sigma ^{2} \gamma ^2-\mu \gamma -\delta =0$. We observe that b can be written as the sum of two terms: the inventory target I and a term that can be positive or negative. We recall that $\alpha$ is the weight of the cost from being far away from the inventory target I and k is the linear cost of production. Thus, $\frac{k}{\alpha }$ represents the cost of production relative to the cost of being far away from the inventory target. We observe that b is decreasing with respect to $\frac{k}{\alpha }$. Hence, the higher the cost of production relative to the cost of being far away from the inventory target, the lower the value of the threshold, and therefore the less productive the company will be. We also notice that

$$\begin{aligned} b > I \;\;\;\;\; \Longleftrightarrow \;\;\;\;\; \frac{k}{\alpha } < \frac{2}{\delta } \left( \frac{\mu }{\delta } + \frac{1}{\gamma _1} \right) . \end{aligned}$$

In the following, we denote by $b_{L}$ the threshold for the low-demand regime. That is, $b_{L}$ denotes the threshold for the one-regime case when the parameter values are $\mu =0.2$, $\sigma =0.2$, $\delta =0.2$, $\alpha =0.2$, $I=2$, and $k=2$. We denote by $b_{H}$ the threshold for the high-demand regime. That is, $b_{H}$ denotes the threshold for the one-regime case when the parameter values are $\mu =1.0$, $\sigma =0.2$, $\delta =0.2$, $\alpha =0.2$, $I=2$, and $k=2$. We want to see how the presence of regime shifts is reflected on the thresholds. For the case in which the production rates are unbounded from above, we obtain, as expected,

$$\begin{aligned} b_{L}=0.908< b_1=0.914< b_2=0.977 < b_{H}=0.980. \end{aligned}$$

5.3 The case in which the production rates are bounded from above: comparison between regime switching and no regime switching

In this subsection, we compare the model with two regimes with the model with only one regime (equivalently, the model without regime switching) for the case in which the production rates are bounded from above.

If there is no regime switching, we can obtain explicitly the threshold

$$\begin{aligned} b= & {} -\frac{2 K \alpha \gamma _1 \gamma _2 - 2 \alpha \gamma _1 \delta - 2 \alpha \gamma _2 \delta - 2I \alpha \gamma _1 \gamma _2 \delta + k \gamma _1 \gamma _2 \delta ^2 - 4 \alpha \gamma _1 \gamma _2 \mu }{2 \alpha \gamma _1 \gamma _2 \delta } \\= & {} I + \frac{2 \mu }{ \delta } + \frac{1}{\gamma _1} + \frac{1}{\gamma _2} - \frac{K}{ \delta } - \frac{ \delta }{2} \frac{k}{\alpha }, \end{aligned}$$

where $\gamma _1$ is the positive solution of $\frac{1}{2}\sigma ^2 \gamma ^2 + (K-\mu )\gamma - \delta =0$ and $\gamma _2$ is the negative solution of $\frac{1}{2}\sigma ^2 \gamma ^2 -\mu \gamma - \delta =0$. We observe that b can be written as the sum of two terms: the inventory target I and a term that can be positive or negative. We notice that b is decreasing with respect to $\frac{k}{\alpha }$. Hence, the higher the cost of production k relative to the cost $\alpha$ of being far away from the inventory target, the lower the value of the threshold, and therefore the less productive the company will be. We also notice that

$$\begin{aligned} b > I \;\;\;\;\; \Longleftrightarrow \;\;\;\;\; \frac{K}{\delta } + \frac{\delta }{2} \frac{k}{\alpha } < \frac{2 \mu }{\delta } + \frac{1}{\gamma _1} + \frac{1}{\gamma _2}. \end{aligned}$$

We denote by $b_{L}$ the threshold for the low-demand regime. That is, $b_{L}$ denotes the threshold for the one-regime case when the parameter values are $\mu =0.2$, $\sigma =0.2$, $\delta =0.2$, $\alpha =0.2$, $I=2$, $k=2$, and $K=1$. We denote by $b_H$ the threshold for the high-demand regime. That is, $b_H$ denotes the threshold for the one-regime case when the parameter values are $\mu =1$, $\sigma =0.2$, $\delta =0.2$, $\alpha =0.2$, $I=2$, $k=2$, and $K=1$. We want to see how the presence of regime shifts is reflected on the thresholds. For the case in which the production rates are bounded from above, we obtain

$$\begin{aligned} b_L=0.933< b_1=0.976< b_2=1.201 < b_H=1.296. \end{aligned}$$

6 Conclusions

We have obtained, for the first time in the literature, an analytical solution for the optimal production management when customer’s demand depends on the regime and the production rates must be nonnegative. The uncertainty is modeled by two factors: a Brownian Motion that represents instantaneous random fluctuations around the long-term drift of the demand, and a continuous-time Markov chain that represents the regime. We have assumed that the production rate must be nonnegative. We have investigated two cases: one in which the company can produce immediately any amount of a good, and one in which the production rate is bounded from above. The first case is modeled as a singular stochastic control problem and the second case as a classical stochastic control problem. We have applied the dynamic programming method to obtain an analytical solution for both problems. For both cases, the optimal production policy is to produce when the inventory level is below a threshold that depends on the regime. The thresholds for the first case are different from the thresholds for the second case.

We have applied analytical or numerical methods to investigate how the optimal production policy depends (for each regime) on the demand drifts, the volatility demands, the inventory targets, the production costs, and the production constraints. We have obtained interesting managerial insights, like the dependency of the optimal production on the demand’s volatility. Indeed, while the optimal production in the unbounded from the above model decreases (equivalently, the production thresholds decrease) in both regimes when the volatilities increase, that is not always the case in the bounded from the above model. Our numerical analysis shows that, in the bounded from the above model, the optimal production decreases in both regimes (equivalently, the production thresholds decrease) when the volatility in the low-demand regime increases, but an increase in the high-demand regime volatility leads to higher production in both regimes (equivalently, leads to higher production thresholds in both regimes). We have also shown that the optimal production policy depends dramatically on the regime.