Abstract
We study a production problem in which the cumulative consumer demand for an item follows a Brownian motion with drift, with both the drift and the variance parameters modulated by a continuoustime Markov chain that represents the regime. The company wants to maintain the inventory level as close as possible to a target inventory level, but there is a linear cost of production. We assume that the production rate is nonnegative. The company is penalized for deviations from the inventory target level and the cost of production, and the objective is to minimize the total discounted penalty costs. We consider two models. In the first model, there is no upper bound for the production rate, and in the second model there is an upper bound for the production rate. We solve both problems analytically and obtain the optimal production policy and the minimal total expected discounted cost. Our solutions allow us to obtain interesting managerial insights.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
We assume that the cumulative demand for an item follows a Brownian motion with drift modulated by a continuoustime Markov chain that alternates among regimes. We allow both the drift and the volatility to be affected by this continuoustime Markov chain, which represents the regime of the economy or the state of the world. In an application, one regime may represent a recessionary period with a low demand rate and the other may represent an expansionary period with a high demand rate. In another application, one regime may represent a high demand rate for a medical item because of an epidemic, and the other regime may represent a low demand rate for that medical item when there is not an epidemic.
The cost of having too much inventory above a preferred target level is due to inventory costs, that is, the cost of the actual maintenance of the inventory. The cost of having too little inventory is due to the perceived likelihood of depleting the inventory, which could lead to a loss of sales as well as a loss of goodwill. The perceived likelihood of such an event increases with a lower level of inventory. Thus, the management’s objective is to maintain the inventory level as close as possible to a fixed target level, so there is a running cost associated with the difference between the inventory level and its target. In addition to that cost, there is a linear cost of production, and the production rate is constrained to be nonnegative. Thus, there is a running cost of production. We consider two models. In the first model, there is no upper bound for the production rate, whereas in the second model there is an upper bound for the production rate. For instance, the second model occurs when a pharmaceutical company produces a vaccine during a pandemic. Although there is a high demand for the vaccine produced by this pharmaceutical company, the production capacity is bounded. That bound for the production capacity was evident in the recent covid crisis. The first model generates a singular stochastic control problem with regime switching, while the second model generates a classical stochastic control problem with regime switching. We determine for both models the optimal production policies by applying the dynamic programming method with regime switching. We obtain an analytical solution for both problems. This is of barriertype and prescribes to increase the production only when the inventory level is sufficiently small, in particular when the inventory level is smaller than or equal to a regimedependent barrier. On one hand, in the singular stochastic control model, the resulting optimal production policy combines jumps to singularly continuous displacements and it is mathematically described through the solution to a socalled Skorokhod reflection problem. On the other hand, in the classical control model, the optimal policy is of the “bangbang” type and switches between noproduction and production at the maximal rate.
The regimedependent boundaries that trigger the optimal production plan are endogenously determined in both models through the socalled smoothfit condition, which results in a system of highly nonlinear algebraic equations. The dependency of those trigger levels on the models’ parameters allows us to draw interesting economic conclusions (see Sects. 3.4 and 4.4, and 5). For example, while the optimal production boundaries in the unbounded from above model decrease in both regimes when the volatility increases, this is not always the case in the bounded from above model. Indeed, our numerical analysis of Sect. 4.4 reveals that while the production threshold in each regime is decreasing with respect to the volatility in regime 1 (lowdemand regime), an increase in the volatility in regime 2 (largedemand regime) leads to an increase of the production threshold in both regimes and therefore to earlier production. Given that the rate at which production is performed is capped, the manager of the company decides to produce more frequently in both regimes even when the volatility of regime 2 becomes larger, somewhat to hedge against a change to regime 1 characterized by low demand. Such a result differs from the classical message related to the “value of waiting” in problems of investment under uncertainty and represents a novel managerial result of our analysis.
There is extensive literature on the theory of optimal production control in continuoustime when the demand is deterministic; see, for instance, Sect. 6 of Sethi (2019) for some references. S Sethi and Thompson (1981) and Chapter 12 of Sethi (2019) consider the case when the demand is stochastic but the demand rate is constant. Fleming et al. (1987) allow the demand rate to be stochastic and model the demand as a continuoustime Markov chain with a finite state space. While a continuoustime Markov chain model for the demand rate is more realistic than a deterministic model, it still is not ideal. A continuoustime Markov chain model assumes that the demand can take only a finite number of values and that it remains constant for long periods of time. It is not suitable for modeling the demand for items such as gold and oil. Bensoussan et al. (1984) allow the demand to be stochastic and model it as a Brownian motion with drift. They assume that there is no regime switching, or equivalently that there is only one regime. Cadenillas et al. (2013) allow the demand to be stochastic and model the cumulative demand as a Brownian motion with both the drift and the variance parameters modulated by a continuoustime Markov chain that represents the regime of the economy. They also study the case in which the cumulative demand is a geometric Brownian motion with both the drift and the variance parameters modulated by a continuoustime Markov chain that represents the regime of the economy. Covei et al. (2022a) and Covei (2023) also studied a production problem with regime switching, while Covei et al. (2022b) studied a random timehorizon not discounted optimal inventory problem with N produced goods (without regime switching). From all the above papers, only Bensoussan et al. (1984) make the realistic assumption that the production rate is nonnegative. This motivates us to study a production problem in which the demand process changes continuously over time and is allowed to take a continuum of values, and the production rate is nonnegative. In addition, we do allow the demand to depend on macroeconomic conditions or states of the world that remain in effect for extended periods of time. Besides the production rate to be bounded below by 0, we consider two cases: the production rate is unbounded from above and the production rate is bounded from above.
We obtain, for the first time in the literature, an analytical solution to an optimal production problem in which the production rate is nonnegative, and the cumulative demand is modeled by a Brownian motion with both the drift and the variance modulated by a continuoustime Markov chain.
This paper is organized as follows. We describe the production model and management’s objectives in Sect. 2. In Sect. 3 we study and solve the problem when the production rate is unbounded from above, and in Sect. 4 we study and solve the problem when the production rate is bounded from above. In Sect. 5, we compare the results for the case in which the production rate is unbounded from above with the case in which the production rate is bounded from above. We present our conclusions in Sect. 6.
2 The management model with nonnegative production and regime switching
We consider a company in which the uncertainty in the demand is modeled by a Brownian motion W and a finitestate continuoustime Markov chain \(\epsilon \). The Brownian motion W models minuscule and continuous uncertain movements in the demand, while the finitestate continuoustime Markov chain \(\epsilon \) models uncertain longterm conditions. Thus, \(\epsilon \) models the regime, which can be the regime of the economy or the state of the world. At every point in time, the management of the company has information about these two sources of uncertainty, from the beginning up to the present. Formally, we consider a probability space \((\Omega ,{{\mathcal {F}}},{{\mathbb {P}}})\) together with a standard Brownian motion \(W=\{W_t,\,t\ge 0\}\) and an observable finitestate continuoustime Markov chain \(\varepsilon =\{\varepsilon _t,t\ge 0\}\). We assume that for every \(t\ge 0\): \(\varepsilon _t=\varepsilon (t)\in {\mathcal S}:=\{1,\cdots ,N\}\), \(N\ge 2\). Here, \(\varepsilon (t)\) represents the regime at time \(t\ge 0\).
We also assume that \(\varepsilon \) and W are independent, and that the Markov chain \(\varepsilon \) has a strongly irreducible generator \(Q=[\,q_{ij}\,]_{N\times N}\), where \(q_{ii}=\lambda _i<0\) and \(\sum _{j\in {{\mathcal {S}}}}q_{ij}=0\) for every \(i\in {{\mathcal {S}}}\). We denote by \({{\mathbb {F}}}=\{{{\mathcal {F}}}_t,t\ge 0\}\) the \(\mathbb P\)augmentation of the filtration \(\{{\mathcal F}_t^{({W,\varepsilon })},t\ge 0\}\) generated by the Brownian motion and the Markov chain, where \({\mathcal F}_t^{({W,\varepsilon })}:=\sigma \{W_s,\varepsilon _s:0\le s\le t\}\) for every \(t\ge 0\).
We assume the cumulative demand process \(D=\{D_t,t\ge 0\}\) satisfies the stochastic differential equation
where, for each \(i \in {\mathcal {S}}\), the parameter \(\mu _i\) represents the drift of the demand and \(\sigma _i >0\) represents the volatility of the demand. We allow the change dD in the cumulative demand to take negative values, which represents the return of items
The manager of a firm wants to control the inventory of a given item. Thus, the manager selects a production strategy given by an adapted process \(P=\{P_t,t\ge 0\}\), where \(P_t\) represents the cumulative production up to time t. We assume that P is a nonnegative and nondecreasing stochastic process with sample paths that are leftcontinuous with right limits. Then, the inventory process \(X=\{X_t,t\ge 0\}\) satisfies the dynamics
or equivalently
where the initial inventory level is \(X_{0}=x\).
At every point in time t, the management of the company wants to keep the inventory level \(X_t\) as close as possible to a target level \({{\mathcal {I}}}_{\varepsilon (t)} \), so there is a running cost associated with the difference between the inventory level and its target. Besides, there is a linear cost of production \(k_{\varepsilon (t)}>0\). Zhou et al. (2007) also assume a linear cost of production. Hence, the management of the company wants to select a production policy that minimizes the expected total discounted costs.
Problem 1
Select a production control \(P=\{P_t,t\ge 0\}\) that minimizes
where \(\delta > 0\) is a discount rate, \(E_{x,i}\) represents the expectation conditioned to \(X_0=x\) and \(\varepsilon _0=\varepsilon (0)=i\), and \(\alpha _{\varepsilon (t)} \) represents the importance that the inventory is far away from its target relative to the cost of production. Here, \(\alpha _{\varepsilon (t)} >0\) for every \(t \ge 0\).
We are assuming that the cumulative production is nondecreasing, or equivalently that 0 is a lower bound for the production rate. In this paper, we consider two different cases for production control. First, we consider the case that the production rate is unbounded from above, and then we consider the case in which the production rate is bounded from above. In both cases, we apply the dynamic programming method to obtain an analytical solution for the optimal production control and the value function. Moreover, we compare the solutions for these two cases (see Sect. 5 for details).
3 Optimal production control when production rates are unbounded from above
In this section, we study the case in which the production rate is unbounded from above. In this case, Problem 1 is modeled as a singular stochastic control problem with the following admissible strategies.
Definition 1
An \({\mathbb {F}}\)adapted, nonnegative, and nondecreasing control process \(P:[\,0,\infty )\times \Omega \rightarrow [\,0,\infty )\), with sample paths that are leftcontinuous with right limits, is called an admissible control if
The set of all admissible controls is denoted by \({{\mathcal {A}}}_U\).
Under these conditions, Problem 1 takes the following form.
Problem 2
The management wants to obtain the optimal production policy \({\hat{P}} \in {{\mathcal {A}}}_U\) that solves the problem
Notice that \(0\le V(x,i) \le M(1+x^2)\), for some \(M>0\), since \(P\equiv 0\) is suboptimal a priori. We observe that Problem 2 is a problem of stochastic singular control with regime switching. We are not requiring (absolute) continuity for the control P. We define \(\Lambda :=\{t\ge 0: P_{t+}\ne P_t\}\), the set of times where P has a discontinuity. The set \(\Lambda \) is countable because P can jump only a countable number of times during the interval \([\,0,\infty )\). We will denote by \(P^d\) the discontinuous part of P, that is \(P_t^d:=\sum _{0\le s\le t,\,s\in \Lambda }(P_{s^+}P_s)\); and by \(P^c\) the continuous part of P, that is \(P_t^c:=P_tP_t^d\).
3.1 Verification Theorem
Let \(\psi :(\infty ,\infty )\times {{\mathcal {S}}}\rightarrow {\mathbb {R}}\) be a function such that \(\psi (\cdot , i) \in C^2({\mathbb {R}})\) for every \(i \in {\mathcal {S}}\) and define the operators \({\widetilde{L}}_i\), for each \(i\in {{\mathcal {S}}}\), by
Here, and in the sequel, \(\psi '\), \(\psi ''\), \(\psi '''\), etc., denote the partial derivatives of \(\psi (\cdot , i)\) for any \(i\in {\mathcal {S}}\).
For a function \(v:(\infty ,\infty )\times {{\mathcal {S}}}\rightarrow {\mathbb {R}}\), consider the Hamilton–Jacobi–Bellman equation
where \(x\in (\infty ,\infty )\) and \(i\in {{\mathcal {S}}}\). We observe that (2) defines, for each \(i\in {{\mathcal {S}}}\), the continuation region
and the intervention region
Thus, it is possible to construct a control process \(P^v\) associated with v in the following sense.
Definition 2
An \({\mathbb {F}}\)adapted, nonnegative, and nondecreasing control process \(P^v\) is associated with the function v above if it is such that
Notice that if a process \(P^v\) as in Definition 2 exists, then the triple \((X^{P^v}, \epsilon , P^v)\) solves a socalled Skorokhod reflection problem at the topological boundary of the region \({\mathcal {C}}^v(i) \times \{1,2\}\). We shall see in the subsequent sections that for Problem 2, a process \(P^v\) satisfying (3) can be indeed explicitly constructed.
Next, we present the Verification Theorem that gives sufficient conditions of optimality for the solution of Problem 2.
Theorem 1
Let \(v(\cdot ,i)\in C^1((\infty ,\infty )) \cap C^2( (\infty ,\infty ) {\setminus } N_i)\), \(i\in {{\mathcal {S}}}\), where \(N_i\) are finite subsets of \((\infty ,\infty )\). Let \(v(\cdot ,i)\), \(i\in {{\mathcal {S}}}\), be a convex function on \((\infty ,\infty )\). Suppose that \(v(\cdot ,i)\), \(i\in {{\mathcal {S}}}\), have quadratic growth on \((\infty ,\infty )\). Suppose that the function \(v(\cdot ,i)\), \(i\in {{\mathcal {S}}}\), satisfies the Hamilton–Jacobi–Bellman Eq. (2) for almost every \(x \in (\infty ,\infty )\), \(i\in {{\mathcal {S}}}\), and consider the stochastic control \(P^v\) associated with v. Then, the process \({\hat{P}}=\{{\hat{P}}_t,t\ge 0\}=:P^{v}\) is the optimal production control for Problem 2. Furthermore, v is the value function for Problem 2.
Proof
Consider an admissible control P and the corresponding semimartingale
Consider also the function \(f(\cdot ,\cdot ,i)\), \(i\in {{\mathcal {S}}}\), defined by \(f(t,x,i)=e^{\delta t}v(x,i)\). We get
where \(\Delta _{Q}v(X_t,\varepsilon _t):=\lambda _{\varepsilon (t)}v(X_t,\varepsilon _t)\sum _{j\ne \varepsilon (t)}q_{\varepsilon (t)j}\,v(X_t,j)\) and \(\Lambda \) has been defined below Problem 3.1. Here, the process \(M^f=\{M^f_t,t\ge 0\}\) is a squareintegrable martingale when \(f(\cdot ,\cdot ,i)\), \(i\in {{\mathcal {S}}}\), are bounded. We observe that \(v(\cdot ,i)\) and \(v'(\cdot ,i)\), \(i\in {{\mathcal {S}}}\), are not necessarily bounded. To take care of that, we perform a localization procedure as follows. Let a and b be real numbers satisfying \(\infty< a<X_0=x<b<+\infty \), and define \(\tau _a:=\inf \{t\ge 0:X_t=a\}\), \(\tau _b:=\inf \{t\ge 0:X_t=b\}\) and \(\tau :=\tau _a\wedge \tau _b\). Then, for every time \(t\in [\,0,\infty )\), we have
Taking conditional expectation, we have
Equation (2) guarantees that \({\widetilde{L}}_{\varepsilon (t)}v(X_t,\varepsilon _t)\Delta _Qv(X_t,\varepsilon _t) \ge  {\alpha }_{ \varepsilon (s) }( X_{s}  {{\mathcal {I}}}_{ \varepsilon (s) } )^{2} \). Moreover, \(v'(x,i)\ge  k_{i}\) for \(x\in (\infty ,\infty )\) [recall Eq. (2)] and the Mean Value theorem imply that \(v(y_1,i)v(y_2,i)\ge  k_{i}(y_1y_2)\) for every \(y_1,y_2\in (\infty ,\infty )\), \(y_1>y_2\), and for every \(i\in {{\mathcal {S}}}\). Hence, replacing \(i=\varepsilon _t\), \(y_1=X_t\) and \(y_2=X_{t+}\), we obtain \(v(X_{t+},\varepsilon _t)v(X_t,\varepsilon _t)\ge  k_{ \varepsilon _t } (X_{t+}X_t)\). We also note that \(X_{s^+}X_s= P_{s^+}  P_{s}\). Then, from the last displayed equation,
We note that this inequality becomes equality for the stochastic control \(P^v\) associated with v and, hence, for the admissible \({\hat{P}}\). Indeed, condition (ii) in (3) implies that \(X^{P^v}_s\in \,\,{{\mathcal {C}}}(\varepsilon _s)\) Leb a.e. \(s\in [\,0,t\wedge \tau )\) \({\mathbb {P}}\)a.e. and, hence,
Furthermore, for the stochastic control \(P^v\) associated with v,
We note that \(v(X_s,\varepsilon _s)\) and \(v'(X_s,\varepsilon _s)\) are bounded when \(s\in [\,0,t\wedge \tau ]\). Then, \(\{M_{t\wedge \tau }^f,t\ge 0\}\) is a squareintegrable martingale and, hence, \(E_{x,i}\left[ M_{t\wedge \tau }^fM_0^f\right] =0\), for every \(t\ge 0\). Furthermore, \(\sigma ^2_{\varepsilon (s)}\,e^{2\delta s}(v'(X_s,\varepsilon _s))^2\) is bounded when \(s\in [\,0,t\wedge \tau ]\), and hence
Letting \(a\downarrow  \infty \) and \(b\uparrow +\infty \), we get \(\tau _a\rightarrow +\infty \) and \(\tau _b\rightarrow +\infty \). Then, \(\tau \rightarrow \infty \). We notice that condition (1) and the quadratic growth of v imply that \(\lim _{ t \uparrow \infty } E_{x,i}[e^{\delta t} v(X_{t^+},\varepsilon _{t})] = 0.\) Taking \(t\rightarrow \infty \), we get
In particular, for \({\hat{P}}\) we have \(v(x,i){=} E_{x,i} \left[ \, \!\!\int _0^{\infty } e^{{}\delta s} {\alpha }_{ \varepsilon (s) }( {\hat{X}}_{s}{} {{\mathcal {I}}}_{ \varepsilon (s) } )^{2} ds {+} \!\int _0^{\infty } e^{{}\delta s} k_{ \varepsilon _s }d{\hat{P}}_s\, \right] \). \(\square \)
3.2 Construction of the solution
We want to find a function v that satisfies the conditions of Theorem 1. In particular, we want Eq. (2) to be satisfied.
We conjecture that \(v'(\cdot ,i)\) are continuous and nondecreasing functions. We denote
for each \(i\in {{\mathcal {S}}}\). When \(x\in (b_i, \infty )\), \(v'(x,i)>  k_{i} \) because we conjecture above that \(v'(\cdot ,i)\) are nondecreasing functions. Hence, in view of (2), for every \(x \in [b_i, \infty )\):
When \(x\in (\infty ,b_i)\), \(v'(x,i) \le  k_{i}\). The convexity of \(v(\cdot ,i)\) and the fact that \(v'(x,i)\ge k_{i}\) [see Eq. (2)], imply that \(v'(x,i)= k_{i} \) for every \(x\in (\infty , b_i)\).
For simplicity, we assume in the remainder of this section that the economy shifts only between two regimes; that is, \({\mathcal S}=\{1,2\}\). Under this assumption, we consider only two thresholds: \(b_1\) and \(b_2\). The relation between \(b_1\) and \(b_2\) depends on the relations among the drift coefficients, the volatility parameters, and the rates \(\lambda _1\) and \(\lambda _2\). We will only consider the case \(b_1<b_2\), because the case \(b_1>b_2\) has a similar treatment. Thus, we will consider three possibilities for the initial level of the inventory: \(x\in (\infty ,b_1)\), \(x\in [\,b_1,b_2)\), and \(x\in [\,b_2,\infty )\).
We need the following lemma, which has been adapted from Remark 2.1 in Guo (2001)
Lemma 2
For \(i\in {{\mathcal {S}}}\), consider the real function \(\phi _i(z)=\sigma _i^2z^2/2 {\tilde{\mu }}_i z+(\lambda _i+\delta )\) where \({\tilde{\mu }}_i\) is a function of \(\mu _i\). Since \(\sigma _1\), \(\sigma _2\), \(\lambda _1\) and \(\lambda _2\) are positive, the equation \(\phi _1(z)\,\phi _2(z)=\lambda _1\lambda _2\) has four real roots such that \(z_1<z_2<0<z_3<z_4\).
When \(x\in [b_2, \infty )\), (5) implies that v satisfies the system of differential equations
A method to solve this kind of system or ordinary differential equations can be found, for instance, in Chapter 7 of Boyce and DiPrima (1997). According to Lemma 3.1, the solution is
where, for each \(j=1,2,3,4\),
The real values \(\gamma _1<\gamma _2<0<\gamma _3<\gamma _4\) above are the real roots of the characteristic equation \(\phi ^1_1(\gamma )\,\,\phi ^1_2(\gamma )\,\,=\,\,\lambda _1\lambda _2,\) where
Furthermore, \(R_{i}\), \(S_{i}\) and \(T_{i}\) are the solution of the system of 6 linear equations with 6 unknowns:
We recall that we are conjecturing that \(v(\cdot ,i)\) has quadratic growth. Thus, \(B_{3}=B_{4}=A_{3}=A_{4}=0\). Hence, the solution of the system (6) is given by
When \(x\in [b_1,b_2)\), we define \(v(x,2)\,:=\, k_{2} (x b_{2}) +D_2\), which satisfies \(v'(x,2)=  k_{2}\). Furthermore, (5) and the assumption \(b_1 < b_2\) imply that
Solving this ordinary differential equation, we have
where \({\tilde{\gamma }}_1<0<{\tilde{\gamma }}_2\) are the real roots of the equation
and
Finally, for every \(x\in ( \infty , b_1)\), we define
To find the thresholds \(b_1\) and \(b_2\), and the coefficients and constants in Eqs. (9)–(10), (12)–(13), and (15)–(16), we conjecture that the smoothfit condition holds. We also want \(v'(b_i,i)=  k_{i}\) and \(v''(b_i,i)=0\) for each \(i=1,2\). Thus, we need to solve the system of equations
The solution of (17) gives the values of \(b_1\) and \(b_2\), the values of the coefficients \(A_j\), \(j=1,2\), \({\widetilde{A}}_j\), \(j=1,2\), and the constants \(D_1\) and \(D_2\). The values of \(B_j\), \(j=1,2\), can be found from (7).
Lemma 3
The following statements hold:

(a)
\( \gamma _1< {\tilde{\gamma }}_1< \gamma _2< 0< \gamma _3< {\tilde{\gamma }}_2 < \gamma _4\), and

(b)
\(\phi _1^1(\gamma _1)< 0, \quad \phi ^1_1(\gamma _2)>0,\quad \phi ^1_1(\gamma _3)> 0, \quad \phi ^1_1(\gamma _4)<0\).
Proof
For the proof of (a), we remember that \(\gamma _i\) solves \(\psi (\gamma _i):=\phi _1^1(\gamma _i) \phi _2^1(\gamma _i)\lambda _1 \lambda _2=0\) and \({\tilde{\gamma }}_i\) solves \(\phi _1^2({\tilde{\gamma }}_i)=0\). Moreover, we note that \(\phi _1^1(\cdot )=\phi _1^2(\cdot )\). Since \(\lim _{\gamma \rightarrow \pm \infty }\psi (\gamma )=\infty \), \(\psi ({\tilde{\gamma }}_i)=\lambda _1 \lambda _2<0\) and \(\psi (0)=\lambda _1\delta +\lambda _2 \delta +\delta ^2>0\), the four roots \(\gamma _i\) of \(\psi \) are such that \(\gamma _1< {\tilde{\gamma }}_1< \gamma _2< 0< \gamma _3< {\tilde{\gamma }}_2 < \gamma _4\).
The proof of (b) follows immediately from the proof of (a) together with the fact that \(\lim _{\gamma \rightarrow \pm \infty } \phi _1^1=\infty \), \(\phi _1^1({\tilde{\gamma }}_i)=0\) and \(\phi _1^1(0)=\lambda _i+\delta >0\). \(\square \)
3.3 Verification of the solution
In Sect. 3.2, we made conjectures to find a candidate v for value function. In this subsection, we will prove that v is indeed the value function of Problem 2.
Before proving this, we investigate the regularity of our candidate for value function.
Lemma 4
Let \(b_i\), \(i \in \{1,2\}\), \(A_j\), \(j\in \{1,2\}\), \({\widetilde{A}}_j\), \(j \in \{1,2\}\), and \(D_i\), \( i \in \{1,2\}\), be the solution of the system of Eq. (17). Let \(B_j\), \(j \in \{1,2\}\), be defined by (7). Then, the function v defined by
and
is such that \(v(\cdot ,1)\in C^{\infty }({\mathbb {R}} {\setminus } \{b_1,b_2\}) \cap C^4({\mathbb {R}} {\setminus } \{b_1\}) \cap C^2({\mathbb {R}})\) and \(v(\cdot ,2)\in C^{\infty }({\mathbb {R}} {\setminus } \{b_2\}) \cap C^2({\mathbb {R}})\). Moreover, for fixed \(i \in \{1,2\}\) and \(x> b_i\), we have
Proof
First, we note that the system (17) implies that \(v(\cdot ,2)\in C^{\infty }({\mathbb {R}} {\setminus } \{b_2\}) \cap C^2({\mathbb {R}})\). The regularity of \(v(\cdot ,1)\) can be shown as follows. By construction, \(v(\cdot ,1) \in C^{\infty }({\mathbb {R}} {\setminus } \{b_1,b_2\})\) and solves, for \(x > b_1\),
Hence, we can rewrite this equation and obtain
Since \(v(\cdot ,2) \in C^2({\mathbb {R}})\) by (17), we obtain that \(v(\cdot ,1) \in C^4((b_1,\infty ))\) by differentiating the last equation. Moreover, by the system (17), we obtain that \(v''(b_1,1)=0=v''(b_1+,1)\). Hence, combining these two facts, we obtain that \(v(\cdot ,1)\in C^{\infty }({\mathbb {R}} {\setminus } \{b_1,b_2\}) \cap C^4({\mathbb {R}} {\setminus } \{b_1\}) \cap C^2({\mathbb {R}})\). Furthermore, fixing \(i \in \{1,2\}\) and \(x>b_i\), we know that
Therefore, the proven regularity allows us to differentiate this equation two times with respect to x, and get
\(\square \)
The proven regularity of v allows us to apply Itô’s formula for Markovmodulated processes on \(v''(\cdot ,i)\). This will allow us to prove convexity for the candidate for value function.
Theorem 5
Let \(b_i\), \(i \in \{1,2\}\), \(A_j\), \(j \in \{1,2\}\), \({\widetilde{A}}_j\), \(j \in \{1,2\}\), and \(D_i\), \(i \in \{1,2\}\), be the solution of the system of Eq. (17). Let \(B_j\), \(j \in \{1,2\}\), be defined by (7). Then, the function v given by (18)–(19) is the value function V of Problem 2. The optimal production policy \({\hat{P}}\) is determined by Definition 3.2.
Proof
To prove that the function v defined above is the solution for Problem 2, it is enough to show that it satisfies the conditions of Theorem 1. We note that \(v(\cdot ,i)\) is of quadratic growth and \(v'(\cdot ,i)\) is of linear growth by construction.
Next, we show that v is convex. We already know that \(v''(x,i)=0\) for every \(x \le b_i\) and \(i \in \{1,2\}\). By Lemma 4, we know that \(v(\cdot ,1) \in C^4({\mathbb {R}} {\setminus } \{b_1\})\) and \(v(\cdot ,2) \in C^4({\mathbb {R}} {\setminus } \{b_2\})\). Hence, for fixed \(i \in \{1,2\}\) and \(x > b_i\), we apply Itô’s formula for Markov modulated processes (using the fact that \(v(\cdot ,i)\) is of quadratic growth) and obtain
where \(A^i \psi (x,i):=\frac{1}{2} \sigma _i \psi ''(x,i)\mu _i \psi '(x,i)(\lambda _i+\delta ) \psi (x,i) \lambda _i \psi (x,3i)\) and \(\tau \) is a stopping time such that \({\mathbb {E}}\left[ \tau \right] <\infty \). From Eq. (20), taking \(\tau :=\inf \{t\ge 0: (X_t,\varepsilon (t)) \in \{(x,i):x \le b_i \} \}\), we obtain, for \(i \in \{1,2\}\) and \(x> b_i\),
Since \(v''(x,i)=0\) for all \(x \le b_i\) by construction, (21) implies, together with the fact that \(\alpha _i > 0\), that \(v''(x,i)> 0\) for all \(i \in \{1,2\}\) and \(x> b_i\). Hence, it follows that \(v(\cdot ,i)\) is convex for \(i=1,2\).
Next, we show that the candidate v for value function satisfies the HJB Eq. (2). We define
Obviously,
We will consider three cases: \(x \in [b_2,\infty )\), \(x \in [b_1,b_2)\), and \(x \in (\infty ,b_1)\).
Let \(x \in [b_2,\infty )\). By construction of v, definition of \(b_i\), the fact that \(b_1 < b_2\), and the convexity of v, we get that v satisfies \(({\mathbb {L}}v)(x,i)=0\) and \(v'(x,i)>k_i\). Hence, (2) is satisfied.
Let \(x \in [b_1,b_2)\). By construction of v(x, 1), definition of \(b_1\) and the convexity of v(x, 1), we have \(({\mathbb {L}}v)(x,1)=0\) and \(v'(x,1)>k_1\). Let us now check v(x, 2). Since \(v(x,2)=k_2(xb_2)+D_2\), we obtain
and, due to convexity,
We want to show that \(({\mathbb {L}}v)(x,2) \ge 0\). Since we know that \(({\mathbb {L}}v)(b_2,2)=({\mathbb {L}}v)(b_2+,2)= 0\), it is enough to show that \(({\mathbb {L}}v)(x,2)\) is decreasing, or equivalently \(({\mathbb {L}}v)'(x,2)\le 0\). Since \(({\mathbb {L}}v)''(x,2)\ge 0\), we have that \(({\mathbb {L}}v)'(x,2)\) is increasing, hence \(({\mathbb {L}}v)'(x,2)\le ({\mathbb {L}}v)'(b_2,2)\). Next, we show that \(({\mathbb {L}}v)'(b_2,2)\le 0\).
We have
We investigate \(\lambda _2 v'(b_2,1)\).
where the last step follows from the fact that \(v''(b_2+,2)=0\) by the smoothfit conditions and \(v'''(b_2+,2)\ge 0\). The fact that \(v'''(b_2+,2)\ge 0\) follows by convexity. If \(v'''(b_2+,2)< 0\), this would imply, since \(v''(b_2+,2)= 0\), that \(v''(b_2+ \delta ,2) < 0\) for \(\delta \) small enough, which contradicts convexity. Hence, using (8), we obtain
To summarize, (2) is satisfied in the interval \([b_1, b_2)\).
Next, let \(x<b_1\). We calculate
Hence, we have
Since \(({\mathbb {L}}v)'(b_1,2)=({\mathbb {L}}v)'(b_1+,2)\le 0\) and v is continuous, we have \(({\mathbb {L}}v)(x,2)\ge ({\mathbb {L}}v)(b_1+,2)\ge 0\). For \(i=1\), we note that \(({\mathbb {L}}v)(x,1)=0\) for all \(x \in (b_1,b_2)\). Thus, we have \(({\mathbb {L}}v)'(x,1)=0\) on \((b_1,b_2)\). Hence,
We observe that \(v'''(b_1+,1)\ge 0\). Otherwise, \(v'''(b_1+,1)< 0\) would imply (by the fact that \(v''(b_1+,1)=0\)) that \(v''(b_1+ \delta ,1)<0\) for \(\delta \) small enough and that would contradict the convexity. Using now that \(v''(b_1+,1)=0\), \(v'''(b_1+,1)\ge 0\), and that \(v'(b_1+,i)=k_i\), we obtain
Now, let \(x \le b_1\). Since \(({\mathbb {L}}v)''(x,1)=2 \alpha _1 >0\) we have \(({\mathbb {L}}v)'(x,1)\le ({\mathbb {L}}v)'(b_1,1)= (\delta +\lambda _1)k_1\lambda _1k_2+2 \alpha _1(b_1{\mathcal {I}}_1) \le 0\). Hence, \(({\mathbb {L}}v)(x,1)\) is decreasing and, by continuity, we obtain
To summarize, (2) is satisfied in the interval \((\infty , b_1)\). \(\square \)
We have now proven that our candidate for value function is indeed the true value function. Moreover, we have obtained the optimal production policy.
Therefore, the optimal production policy when the regime is i is the following: (a) do not produce when the inventory level is above the threshold \(b_{i}\); (b) if the inventory level is equal to \(b_{i}\), then the company should produce enough to prevent the inventory from being less than \(b_{i}\); and (c) if the initial inventory is below \(b_{i}\), then the company should produce to bring immediately the inventory to the threshold \(b_{i}\) (and then continue as described above). We observe that the company should also increase production to bring immediately the inventory to the threshold \(b_{2}\) just because of a change in the regime: that happens when the level of the inventory falls in the interval \((b_1, b_2)\) and the regime changes from \(i=1\) to \(i=2\).
3.4 Comparative statics and numerical examples
In this subsection, we study the influence of some parameters on the thresholds \(b_{i}, i \in {\mathcal {S}}\) that characterize the optimal production strategy. First, we present the results that can be obtained analytically, and then those which have to be obtained numerically. To study the influence of certain parameters analytically, we apply the connection between optimal stopping and singular stochastic control problems.
Following the ideas of Karatzas and Shreve (1984) (see also Baldursson and Karatzas (1996)), it is possible to show that \(V_x(x,i)=u(x,i)\), where
Here, \(X^{0}\) is the inventory process when there is no production. Since \(V_x(x,i)=u(x,i)\), we have
and \(x \mapsto u(x,i)\) is nondecreasing since V is convex.
In the following, we will denote by \(b_i(a)\) the boundary \(b_i\) as a function of a parameter a. Moreover, u(x, i; a) represents u(x, i) as a function of a parameter a.
Proposition 6
For given \(i \in {\mathcal {S}}=\{1,2\}\), the thresholds \(b_i\) have the following properties:

(i)
\({\mathcal {I}}_j \mapsto b_i({\mathcal {I}}_j)\) is a nondecreasing function for every \(j \in {\mathcal {S}}\),

(ii)
\(\mu _j \mapsto b_i(\mu _j)\) is a nondecreasing function for every \(j \in {\mathcal {S}}\),

(iii)
\(k_i \mapsto b_i(k_i)\) is a nonincreasing function,

(iv)
\(k_{3i} \mapsto b_i(k_{3i})\) is a nondecreasing function.
Proof
We define \({\hat{u}}(x,i):=u(x,i)+k_i\) so that
We observe that

(1)
\({\mathcal {I}}_j \mapsto u(x,i;{\mathcal {I}}_j)\) is a nonincreasing function for every \(j \in {\mathcal {S}}\),

(2)
\(\mu _j \mapsto u(x,i;\mu _j)\) is a nonincreasing function for every \(j \in {\mathcal {S}}\) since \(\mu _j \mapsto X_t^{x,0}=x  \int _0^t \mu _{\varepsilon (s)}~dt \int _0^t \sigma _{\varepsilon (s)}~dW_s \) is a nonincreasing function,

(3)
\(k_i \mapsto {\hat{u}}(x,i;k_i)\) is a nondecreasing function, and

(4)
\(k_{3i} \mapsto {\hat{u}}(x,i;k_{3i})\) is a nonincreasing function.
Thus, we can prove each result separately.

(i)
Taking \(j \in {\mathcal {S}}\), \({\mathcal {I}}_j^{1}>{\mathcal {I}}_j^{2}\) and using (23), the fact that \(u(\cdot ,i)\) is increasing, and 1), we have
$$\begin{aligned} b_i({\mathcal {I}}_j^{2}):= \sup \{x: u(x,i;{\mathcal {I}}_j^{2})\le k_i\} \le \sup \{x: u(x,i;{\mathcal {I}}_j^{1})\le k_i\}=: b_i({\mathcal {I}}_j^{1}). \end{aligned}$$ 
(ii)
Taking \(j \in {\mathcal {S}}\), \(\mu _j^{1}>\mu _j^{2}\) and using (23), the fact that \(u(\cdot ,i)\) is increasing, and 2), we have
$$\begin{aligned} b_i(\mu _j^{2}):= \sup \{x: u(x,i;\mu _j^{2})\le k_i\} \le \sup \{x: u(x,i;\mu _j^{1})\le k_i\}=: b_i(\mu _j^{1}). \end{aligned}$$ 
(iii)
Taking \(k_i^{1}>k_i^{2}\) and using (23), the fact that \(u(\cdot ,i)\) is increasing, and 3), we have
$$\begin{aligned} b_i(k_i^{2}):= \sup \{x: {\hat{u}}(x,i;k_i^{2})\le 0\} \ge \sup \{x: u(x,i;k_i^{1})\le 0\}=: b_i(k_i^{1}). \end{aligned}$$ 
(iv)
It can be proved analogously to (iii).
\(\square \)
We obtain the following proposition for the volatilities \(\sigma _i, i \in {\mathcal {S}}\),
Proposition 7
\(V(\cdot ,i)\) is an increasing function of \(\sigma _i\).
Proof
We know that V satisfies the Hamilton–Jacobi–Bellman equation
where \(L_i\,\psi \,:=\, \frac{1}{2}\,\sigma _i^2\,\psi ''  \mu _i\,\psi '\delta \,\psi \). Let now \(\sigma _i^2 > {\widehat{\sigma }}_i^2\) and define
Moreover, let V(x, i) and \({\widehat{V}}(x,i)\) be the value functions with respect to \(\sigma _i\) and \({\widehat{\sigma }}_i\). Thus,
where the penultimate inequality follows by the convexity of the value function. Moreover, we note that \({\widehat{V}}'(x,i)+k_i \ge 0\) for every \((x,i) \in {\mathbb {R}} \times {\mathcal {S}}\). Hence, \({\widehat{V}}\) solves
Following now the same procedure as in the first part of the proof of Theorem 1, we obtain \({\hat{V}} \le V\), hence the claimed monotonicity. \(\square \)
Next, we study numerically the dependence of the thresholds \(b_{i}\) with respect to \(\sigma _i\) and \(\lambda _i\). We introduce a benchmark case, and we use the following parameter values:
In this setting, the demand drift values are different in the two regimes. We will call regime 1 the “lowdemand regime” and regime 2 the “highdemand regime”. We will consider different values of \(\sigma _i\) in Fig. 2, and different values of \(\lambda _i\) in Fig. 3.
For the above parameter values, the solution to system (17) is given by
and for the thresholds we obtain
The corresponding value function is presented in Fig. 1.
Starting from this benchmark case, we study the dependence of \(b_1\) and \(b_2\) with respect to \(\sigma _i\). The results are presented in Fig. 2.
One can see that the production thresholds are decreasing in the volatility parameters \(\sigma _i\). Hence, a higher uncertainty in the demand leads to a lower production threshold.
A slightly different result can be obtained for the dependence on \(\lambda _i\), which gives the rate of switching from one regime to another, and is shown in Fig. 3.
For the dependence on \(\lambda _i\), we see that both production thresholds are increasing in \(\lambda _1\), but decreasing in \(\lambda _2\). According to the theory of continuoustime Markov chains, the amount of time that the demand is in regime i, before moving to regime \(3i\), has exponential distribution with mean \(\frac{1}{ \lambda _i}\). Hence, a lower expected amount of time in the lowdemand regime, before switching to the highdemand regime, increases both thresholds, and thus there is more production. On the other hand, a lower expected amount of time in the highdemand regime, before switching to the lowdemand regime, decreases both thresholds, and thus there is less production.
For completeness, we introduce Fig. 4, which shows the behavior of the thresholds with respect to \(\mu _i\) (as it was proven in Lemma 6). Moreover, we see that both thresholds converge to each other if the drift components converge to each other. In this case, both regimes have exactly the same parameter values, and hence the thresholds must be the same.
4 Optimal production control when the production rates are bounded from above
In this section, we consider the model with bounded production rates. In this case, the production process \(P=\{P_t,t\ge 0\}\) is such that \(dP_t\,\,=\,\,p(t)\,dt\), where \(p:[\,0,\infty )\times \Omega \rightarrow [\,0,\infty )\) is an \({\mathbb F}\)adapted bounded process that represents the rate of production. Let \(K_{i}\), \(i \in {{\mathcal {S}}}\), be positive real numbers.
Definition 3
An \({\mathbb {F}}\)adapted control process \(p:[\,0,\infty )\times \Omega \rightarrow [\,0,\infty )\) that satisfies \(p(t,\omega )\in [\,0,K_{\varepsilon (t)}\,]\) for every \((t,\omega )\in {[}\,0,\infty )\times \Omega \) and
is called an admissible classical stochastic control. The set of all admissible classical stochastic controls is denoted by \({{\mathcal {A}}}_B\).
We note that for the above process p, the inventory is given by
for every \(t\in [\,0,\infty )\). Under these assumptions, Problem 1 is equivalent to the following problem.
Problem 3
The management wants to obtain the optimal production rate \({\hat{p}} \in {{\mathcal {A}}}_B\) and the value function V defined by
Problem 3 is a classical stochastic control problem with regime switching. We observe that there is a linear cost of production given by \(k_{\varepsilon (s)} \). Furthermore, p is bounded from below by 0, and from above by \( K_{\varepsilon (t)} \). Bensoussan et al. (1984) studied a problem in which the cost of production was quadratic and there was no regime switching (or equivalently, there was only one regime). They did not obtain an analytical solution but presented a numerical method to solve their problem. Cadenillas et al. (2013) studied a problem in which the cost of production was quadratic, and the production rates were unbounded from below and from above. They obtained an analytical solution by applying the “completing squares" method which cannot be applied to our problem. Sotomayor and Cadenillas (2011) solved a stochastic control problem with regime switching in which the control was bounded from below and from above, but their objective function was different from Sect. 4.1, because they considered a maximization problem instead of a minimization problem, and the term \(\int _0^{\infty }\,e^{\delta s} \alpha _{\varepsilon (s)} (X_{s}  {{\mathcal {I}}}_{\varepsilon (s)})^{2} ds \) did not appear in their objective function.
4.1 Verification Theorem
Let \(\psi :(\infty ,\infty )\times {{\mathcal {S}}}\rightarrow \) be a function such that \(\psi (\cdot ,i) \in C^2({\mathbb {R}})\) for every \(i \in {\mathcal {S}}\) and define the operators \(L_i(p)\), \(i\in {{\mathcal {S}}}\) by
Here, \(\psi ''\) and \(\psi '\) denote the partial derivatives \(\partial ^2\psi /\partial x^2\) and \(\partial \psi /\partial x\), respectively.
Theorem 8
Let \(v(\cdot ,i)\in C^1((\infty ,\infty )) \cap C^2((\infty ,\infty ) {\setminus } N_i)\), \(i\in {{\mathcal {S}}}\), where \(N_i\) are finite subsets of \((\infty ,\infty )\). Let \(v(\cdot ,i)\), \(i\in {{\mathcal {S}}}\), have quadratic growth on \((\infty ,\infty )\). If the function v satisfies the equation
for every \(x\in (\infty ,\infty )\) and \(i\in {{\mathcal {S}}}\), then the control \({\hat{p}}\) defined by
for \(t\in [\,0, \infty )\) is optimal solution for Problem 3. Moreover, v is equal to the value function V for Problem 3.
Proof
Consider the function \(f(\cdot ,\cdot ,i)\), \(i\in {{\mathcal {S}}}\), defined by \(f(t,x,i)=e^{\delta t}v(x,i)\) and an admissible control p. We get
Here, the process \(M^f=\{M^f_t,t\ge 0\}\) is a squareintegrable martingale when \(f(\cdot ,\cdot ,i)\), \(i\in {{\mathcal {S}}}\), is bounded. In fact, \(M_t^f =\sum _{i\in {{\mathcal {S}}}}\int _0^tf(s,X_s,i)dm_s^i\), where the square integrablemartingales \(m^i\), \(i\in {{\mathcal {S}}}\), are defined as in Eq. (5) of Björk (1980). Hence,
where \(\Delta _{Q}v(X_t,\varepsilon _t):=\lambda _{\varepsilon (t)}\,v(X_t,\varepsilon _t)\sum _{j\ne \varepsilon (t)}q_{\varepsilon (t)j}\,v(X_t,j)\). Then, for every time \(t\ge 0\), we get
Applying conditional expectation, we have
From condition (25) and the linear growth of \(v^{\prime }\), we have
Then, from Eqs. (28)–(29) we get
Taking \(t\rightarrow \infty \), we observe that condition (25) and the quadratic growth of v imply \(E_{x,i}[\,e^{\delta t}v(X_{t},\varepsilon _{t})\,]\,\,\rightarrow \,\, 0\). Therefore, for every arbitrary admissible control p, \(v(x,i)\le J(x,i;p)\). In particular, inequality (30) becomes an equality if \(p={\hat{p}}\), and hence \(v(x,i)=J(x,i;{\hat{p}})=V(x,i)\). \(\square \)
4.2 Construction of the solution
We want to find a function v that satisfies the conditions of Theorem 8. We note that Eq. (28) is equivalent to
We also note also that for every \(\,t\in [\,0,\infty )\):
Hence, the candidate for optimal control \({\hat{p}}\) has the form \({\hat{p}}(t)=\varphi (X_t,\varepsilon _t)\) for \(t\in [\,0,\infty )\), where \(\varphi (\cdot ,i)\), \(i\in {{\mathcal {S}}}\), is a measurable function defined by
for \(x \in {\mathbb {R}}\). Thus, to solve Eq. (28) is equivalent to solve the equation
for each \(i\in {{\mathcal {S}}}\); or equivalently, to solve
when \(v'(x,i) < k_{i}\), and to solve
when \(v'(x,i)\ge k_{i}\), for each \(i\in {{\mathcal {S}}}\).
For simplicity, we assume in the remainder of this section that the economy shifts only between two different regimes, that is, \({{\mathcal {S}}}=\{1,2\}\).
Our objective in the remainder of this subsection is to find a candidate for value function and a candidate for optimal control. We conjecture \(v'(\cdot ,i)\) to be continuous, with linear growth such that \(v'(\infty ,i)=  \infty \) and \(v'(+\infty ,i)= \infty \) for both \(i=1,2\). We also conjecture \(v'(\cdot ,i)\), \(i=1,2\), to be nondecreasing in order to have \(v(\cdot ,i)\), \(i=1,2\), convex in x. We will prove in Sect. 4.3 that these conjectures are satisfied by the value function.
According to our conjecture that \(v'(\cdot ,i)\) is nondecreasing, \(v'(\infty ,i)=  \infty \) and \(v'(+\infty ,i)= \infty \), \(i=1,2\), there exists a threshold \(b_i\in (\infty ,\infty )\) such that \(v'(b_i,i)= k_{i}\). Hence, for \(x\in (\infty ,b_i)\) we have
and for \(x\in [\,b_i,\infty )\) we have
The relationship between \(b_1\) and \(b_2\) depends on the relations among the drift coefficients, the volatility parameters and the rates \(\lambda _1\) and \(\lambda _2\). We will only consider the case \(b_1<b_2\), because the case \(b_1>b_2\) has a similar treatment. Thus, we consider three possibilities for the initial level of the inventory: \(x\in (\infty ,b_1)\), \(x\in [\,b_1,b_2)\), and \(x\in [\,b_2,\infty )\).
When \(x\in [b_2, \infty )\), Eq. (28) gives the following system of differential equations:
Consider the characteristic equation for (31), \(\phi ^1_1(\gamma )\,\,\phi ^1_2(\gamma )\,\,=\,\,\lambda _1\lambda _2,\) where
From Lemma 2, \(\phi ^1_1(\gamma )\,\,\phi ^1_2(\gamma )\,\,=\,\,\lambda _1\lambda _2\) has 4 real roots: \(\gamma _1<\gamma _2<0<\gamma _3<\gamma _4\). Then, the solution for the system of Eq. (31) is
where, for each \(j=1,2,3,4\),
Furthermore, \(R_{i}\), \(S_{i}\) and \(T_{i}\) are the solution of the system of 6 linear equations with 6 unknowns.
Recall that we are conjecturing that \(v(\cdot ,i)\) has quadratic growth. Thus, \(B_{3}=B_{4}=A_{3}=A_{4}=0\). Hence, the solution of the system (31) is given by
for \(x\in [b_2,\infty )\).
When \(x\in [\,b_1,b_2)\), Eq. (28) gives the system of differential equations
Consider the characteristic equation for the system (36)–(37), \(\phi ^2_1({\tilde{\gamma }})\,\,\phi ^2_2({\tilde{\gamma }})\,\,=\,\,\lambda _1\lambda _2,\) where
From Lemma 2, \(\phi ^2_1({\tilde{\gamma }})\,\,\phi ^2_2({\tilde{\gamma }})\,\,=\,\,\lambda _1\lambda _2\) has 4 real roots: \({\tilde{\gamma }}_1<{\tilde{\gamma }}_2<0<{\tilde{\gamma }}_3<{\tilde{\gamma }}_4\). Then, we obtain that the solution for the system of Eqs. (36)–(37) is:
where \({\tilde{R}}_{i}\), \({\tilde{S}}_{i}\) and \({\tilde{T}}_{i}\) are the solution of the following system of 6 linear equations with 6 unknowns.
Moreover, for each \(j=1,2,3,4\), the condition for the coefficients is
When \(x\in (\infty , b_{1})\), the function v is the solution of the system
Consider the characteristic equation for (42)–(43), \(\phi ^3_1({\hat{\gamma }})\,\,\phi ^3_2({\hat{\gamma }})\,\,=\,\,\lambda _1\lambda _2,\) where
From Lemma 2, \(\phi ^3_1({\hat{\gamma }})\,\,\phi ^3_2({\hat{\gamma }})\,\,=\,\,\lambda _1\lambda _2\) has 4 real roots: \({\hat{\gamma }}_1<{\hat{\gamma }}_2<0<{\hat{\gamma }}_3<{\hat{\gamma }}_4\). Then, the solution for the system of differential Eqs. (42)–(43) is given by:
where \({\widehat{R}}_{i}\), \({\widehat{S}}_{i}\) and \({\widehat{T}}_{i}\), \(i \in \{1,2\}\), are the solution of the system of 6 linear equations with 6 unknowns.
Furthermore, the condition for the coefficients, for each \(j=1,2,3,4\), is
We recall that we are conjecturing that the function v has quadratic growth, which implies \({\widehat{A}}_1={\widehat{A}}_2={\widehat{B}}_1={\widehat{B}}_2=0\). Therefore,
is the solution for the system (42)–(43), where (45) is satisfied for \(j=1,2\).
To find the thresholds \(b_1\) and \(b_2\), and the coefficients of v in (34), (38) and (46), we conjecture that the smoothfit condition holds. We also want \(v'(b_i,i)=  k_{i}\) for each \(i=1,2\). Thus, we need to solve the following system of 10 equations with 10 unknowns:
for each \(i=1,2\). The solution of the system of Eq. (48) will give us the values for \(b_1\) and \(b_2\), and also the values for \(A_j\), \(j=1,2\), \({\widetilde{A}}_j\), \(j=1,2,3,4\), and \({\widehat{A}}_j\), \(j=3,4\). The values for the corresponding \(B_j\), \({\widetilde{B}}_j\) and \({\widehat{B}}_j\) can be found from Eqs. (32), (41), and (45).
For future use, we observe here that \(v^{\prime \prime }(b_{1},1) \ge 0\) is equivalent to
and \(v^{\prime \prime }(b_{2},2) \ge 0\) is equivalent to
4.3 Verification of the solution
In Sect. 4.2, we made some conjectures to find a candidate v for value function. In this subsection, we will prove that v is indeed the value function V of Problem 3 when \({\mathcal S}=\{1,2\}\). We will also present the optimal production policy. First, we investigate the regularity of the candidate for value function.
Lemma 9
Let \(A_j\), \(j \in \{1,2\}\), \({\widetilde{A}}_j\), \(j \in \{1,2,3,4\}\), and \({\widehat{A}}_j\), \(j \in \{3,4\}\), be the solution of the system of Eq. (48). Let \(B_j\), \(j \in \{1,2\}\), \({\widetilde{B}}_j\), \(j \in \{1,2,3,4\} \), and \({\widehat{B}}_j\), \(j \in \{3,4\}\), be defined by (32), (41) and (45). Suppose that \(b_1<b_2\). Then, the function v given by
and
is such that \(v(\cdot ,i)\in {\mathcal {C}}^{\infty }({\mathbb {R}} {\setminus }\{b_1,b_2\}) \cap {\mathcal {C}}^{4}({\mathbb {R}} {\setminus }\{b_{3i} \})\cap {\mathcal {C}}^{2}({\mathbb {R}})\).
Proof
By construction and the smoothfit conditions, see (48), the candidate for value function satisfies \(v(\cdot ,i) \in {\mathcal {C}}^{\infty }({\mathbb {R}} {\setminus }\{b_1,b_2\}) \cap {\mathcal {C}}^1({\mathbb {R}})\). Furthermore, again by construction, for \(x\in [b_2, \infty )\), one has v solves
for \(x\in [\,b_1,b_2)\), v solves
and for \(x\in (\infty , b_{1})\), v solves
By construction and system (48), we obtain that \(v(\cdot ,i) \in {\mathcal {C}}^{\infty }({\mathbb {R}} {\setminus }\{b_1,b_2\}) \cap {\mathcal {C}}^1({\mathbb {R}})\) and \(v'(b_2,2)=k_2\). Hence, combining (53) and (54), we obtain first that \(v(\cdot ,2) \in {\mathcal {C}}^2((b_1,\infty ))\) and further that \(v(\cdot ,1) \in {\mathcal {C}}^4((b_1,\infty ))\). Using that \(v'(b_1,1)=k_1\), combining (54) and (55), gives that \(v(\cdot ,1) \in {\mathcal {C}}^2((\infty ,b_2))\) and further that \(v(\cdot ,2) \in {\mathcal {C}}^4((\infty ,b_2))\). Therefore, \(v(\cdot ,i) \in {\mathcal {C}}^{\infty }({\mathbb {R}} {\setminus }\{b_1,b_2\}) \cap {\mathcal {C}}^{4}({\mathbb {R}}{\setminus }\{b_{3i} \})\cap {\mathcal {C}}^{2}({\mathbb {R}})\). \(\square \)
Theorem 10
Let \(A_j\), \(j \in \{1,2\}\), \({\widetilde{A}}_j\), \(j \in \{1,2,3,4\}\), and \({\widehat{A}}_j\), \(j \in \{3,4\}\), be the solution of the system of Eq. (48). Let \(B_j\), \(j \in \{1,2\}\), \({\widetilde{B}}_j\), \(j \in \{1,2,3,4\}\), and \({\widehat{B}}_j\), \(j \in \{3,4\}\), be defined by (32), (41) and (45). Suppose that the conditions (49)–(50) are satisfied. Assume that \(b_1<b_2\). Then, the function v defined by (51)–(52) is the value function V of Problem 3. Furthermore, \({\hat{p}}\) defined by
is optimal production rate policy for Problem 3.
Proof
The idea of the proof is to check that the function v defined by (51)–(52), and the control \({\hat{p}}\) defined by (56) satisfy all the conditions of Theorem 4.1. That will prove that they are the value function and the optimal production rate, respectively, for Problem 3.
First, we show that v is convex. Let us denote by \(X^{x,i}\) the process with \(X(0)=X_0=x\) and \(\varepsilon _0= \varepsilon (0)= i\). Given the regularity of v, see Lemma 9, we can apply Itô’s formula, for given \(i \in {\mathcal {S}}\) and \(x \ne b_i\), on \(e^{\delta s}v''(X^{x,i}_s,\varepsilon _s)\). Hence, we obtain
where \(A^i \psi (x,i):=\frac{1}{2} \sigma _i \psi ''(x,i)+(K_i \mathbbm {1}_{\{x<b_i\}}\mu _i) \psi '(x,i)(\lambda _i+\delta ) \psi (x,i) \lambda _i \psi (x,3i)\) and \(\tau \) is a stopping time such that \(E_{x,i}\left[ \tau \right] <\infty \). Since v solves the Hamilton–Jacobi–Bellman equation, the regularity of v implies
when \(x < b_{i}\), and
when \(x\ge b_i\).
Taking \(\tau :=\inf \{t\ge 0: (X_t,\varepsilon (t)) \in \{(b_1,1),(b_2,2)\} \}\), we obtain, for every \(i \in \{1,2\}\) and \(x> b_i\):
We recall that we are assuming (49)–(50), or equivalently \(v''(b_i,i) \ge 0\). Hence, \(v''(x,i) \ge 0\) (in particular \(v''(x,i)>0\)) for \(x> b_i\) and for every \(i \in \{1,2\}\).
Analogously, letting \(x< b_i\) and defining \(\tau :=\inf \{t\ge 0: (X_t,\varepsilon (t)) \in \{(b_1,1),(b_2,2)\} \}\) we still obtain (57), for every \(i \in \{1,2\}\). Using once more (49)–(50), we find that \(v''(x,i) \ge 0\) (in particular \(v''(x,i)>0\)) also on \(x<b_i\) for every \(i \in \{1,2\}\), and we thus conclude that \(v(\cdot ,i)\) is convex on \({\mathbb {R}}\).
Since v is convex, it follows immediately by construction that v solves (28). That is, v solves
Moreover, the quadratic growth (as well as the linear growth of \(v^{\prime }\)) is fulfilled by construction. \(\square \)
Therefore, the optimal production policy when the regime is i is the following: (a) do not produce when the inventory level is above the threshold \(b_{i}\), and (b) whenever the inventory level is equal to lower than \(b_{i}\), increase the production at the maximum rate \(K_i\). We observe that the company should also increase production to \(K_2\) just because of a change in the regime: that happens when the level of the inventory falls in the interval \((b_1, b_2)\) and the regime changes from \(i=1\) to \(i=2\).
4.4 Comparative statics and numerical examples
We introduce the benchmark case in which
and
This benchmark case is the same as in the case in which the production rate is unbounded from above (see Sect. 3.4), but expanded by the two new parameters \(K_1=K_2=1\), that represent the upper bounds on the production rate.
We obtain the following solution to the system (48):
We have checked that the conditions (49)–(50) are satisfied.
Hence, our candidate v for value function is indeed the true value function V and is shown in Fig. 5. The optimal production policy is given by
We observe that the production thresholds \(b_i\) are higher than in the case in which the production rates are unbounded from above. This is a consequence of the production constraints given by \(K_i\). In order that the inventory levels be close to the production targets \(I_i\), it is necessary to start production earlier (higher \(b_i\)) when the production rates are bounded from above.
Now, we want to see how the solution depends on the different parameters.
First, we study the dependence on \(\mu _i\). Figure 6 shows the results.
One can observe that both boundaries increase if one of the demand drifts increases. This is plausible, since an increase in \(\mu _i\) increases the future demand, so the firm starts the production earlier. It is interesting to notice that an increase in \(\mu _i\) affects not only \(b_i\) but also \(b_{3i}\). Moreover, as expected, \(b_1=b_2\) when \(\mu _1 = \mu _2\).
Next, we investigate the dependence with respect to \(\lambda _i\). The results are presented in Fig. 7.
We see that both thresholds are increasing in \(\lambda _1\) and decreasing in \(\lambda _2\). This is consistent with the unboundedfromabove case. We recall that the amount of time that the demand is in regime i, before moving to regime \(3i\), has exponential distribution with mean \(\frac{1}{ \lambda _{i}}\). Hence, a lower expected amount of time in the lowdemand regime, before switching to the highdemand regime, increases both thresholds, and thus there is more production. On the other hand, a lower expected amount of time in the highdemand regime, before switching to the lowdemand regime, decreases both thresholds, and thus there is less production.
Next, we study the dependence on \(\sigma _i\). Figure 8 shows two 3D plots, one for each boundary.
We observe an interesting effect. Both thresholds are decreasing in \(\sigma _1\), as in the case in which the production rates are unbounded from above. However, both thresholds are increasing in \(\sigma _2\), which differs from the case in which the production rates are unbounded from above. This means that a higher uncertainty in the lowdemand regime leads the firm to start the production later (smaller \(b_i\)). However, a higher uncertainty in the highdemand regime leads to more productivity (bigger \(b_i\)).
Finally, we investigate the dependence of the production thresholds \(b_i\), \(i \in \{1,2\}\) on the production costs \(k_i\), \(i \in \{1,2\}\). Figure 9 shows that \(k_i \mapsto b_i(k_i)\) is nonincreasing and \(k_{3i} \mapsto b_i(k_{3i})\) is nondecreasing. Thus, we observe the same behavior as in the case in which the production rates are unbounded from above (see Lemma 6).
5 Comparison among different models
5.1 Comparison between the case in which the production rates are unbounded from above and the case in which the production rates are bounded from above
In this subsection, we compare the model of Sect. 3 in which the production rates are unbounded from above with the model of Sect. 4 in which the production rates are bounded from above. We start by considering again the two benchmark cases. Both have the same parameter values, except the additional parameters \(K_1=1\) and \(K_2=1\) in the case in which the production rates are bounded from above.
We denote by \(b_i^{U}\), \(i \in \{1,2\}\), the thresholds in the case in which the production rates are unbounded from above, and by \(b_i^{B}\), \(i \in \{1,2\}\), the thresholds in the case in which the production rates are bounded from above. For the benchmark cases, we obtain
Hence, the production in the boundedfromabove case starts earlier than in the unboundedfromabove case. This is reasonable because in the unboundedfromabove case, production can be instantaneous and the firm can tolerate a lower inventory level, while in the boundedfromabove case the inventory level can decrease even though the production is maximal.
Next, we denote by \(V_{B}\) the value function for the boundedfromabove case and by \(V_{U}\) the value function for the unboundedfromabove case. One can expect that the value function for the boundedfromabove case converges to the value function for the unboundedfromabove case when \(K_i\), \(i \in \{1,2\}\), increases. To show this fact, we introduce Fig. 10. Here, we show the relative distance of the value functions with respect to certain values for \(K_i\). For simplicity, we consider \(K_1=K_2=K\) and study the influence of K.
One can see that the relative change is decreasing in K, which is exactly the expected result. Hence, if the upper bound for the production rate is high enough, the unboundedfromabove case can provide good estimates for the boundedfromabove case.
A similar result can be shown for the boundaries. This is shown in Fig. 11.
5.2 The case in which the production rates are unbounded from above: comparison between regime switching and no regime switching
In this subsection, we compare the model with two regimes with the model with only one regime (equivalently, the model without regime switching) for the case in which the production rates are unbounded from above.
If there is no regime switching, we can obtain explicitly the threshold
where \(\gamma _1\) is the negative solution of \(\frac{1}{2}\sigma ^{2} \gamma ^2\mu \gamma \delta =0\). We observe that b can be written as the sum of two terms: the inventory target I and a term that can be positive or negative. We recall that \(\alpha \) is the weight of the cost from being far away from the inventory target I and k is the linear cost of production. Thus, \(\frac{k}{\alpha } \) represents the cost of production relative to the cost of being far away from the inventory target. We observe that b is decreasing with respect to \(\frac{k}{\alpha } \). Hence, the higher the cost of production relative to the cost of being far away from the inventory target, the lower the value of the threshold, and therefore the less productive the company will be. We also notice that
In the following, we denote by \(b_{L}\) the threshold for the lowdemand regime. That is, \(b_{L}\) denotes the threshold for the oneregime case when the parameter values are \(\mu =0.2\), \(\sigma =0.2\), \(\delta =0.2\), \(\alpha =0.2\), \(I=2\), and \(k=2\). We denote by \(b_{H}\) the threshold for the highdemand regime. That is, \(b_{H}\) denotes the threshold for the oneregime case when the parameter values are \(\mu =1.0\), \(\sigma =0.2\), \(\delta =0.2\), \(\alpha =0.2\), \(I=2\), and \(k=2\). We want to see how the presence of regime shifts is reflected on the thresholds. For the case in which the production rates are unbounded from above, we obtain, as expected,
5.3 The case in which the production rates are bounded from above: comparison between regime switching and no regime switching
In this subsection, we compare the model with two regimes with the model with only one regime (equivalently, the model without regime switching) for the case in which the production rates are bounded from above.
If there is no regime switching, we can obtain explicitly the threshold
where \(\gamma _1\) is the positive solution of \(\frac{1}{2}\sigma ^2 \gamma ^2 + (K\mu )\gamma  \delta =0\) and \(\gamma _2\) is the negative solution of \(\frac{1}{2}\sigma ^2 \gamma ^2 \mu \gamma  \delta =0\). We observe that b can be written as the sum of two terms: the inventory target I and a term that can be positive or negative. We notice that b is decreasing with respect to \(\frac{k}{\alpha } \). Hence, the higher the cost of production k relative to the cost \(\alpha \) of being far away from the inventory target, the lower the value of the threshold, and therefore the less productive the company will be. We also notice that
We denote by \(b_{L}\) the threshold for the lowdemand regime. That is, \(b_{L}\) denotes the threshold for the oneregime case when the parameter values are \(\mu =0.2\), \(\sigma =0.2\), \(\delta =0.2\), \(\alpha =0.2\), \(I=2\), \(k=2\), and \(K=1\). We denote by \(b_H\) the threshold for the highdemand regime. That is, \(b_H\) denotes the threshold for the oneregime case when the parameter values are \(\mu =1\), \(\sigma =0.2\), \(\delta =0.2\), \(\alpha =0.2\), \(I=2\), \(k=2\), and \(K=1\). We want to see how the presence of regime shifts is reflected on the thresholds. For the case in which the production rates are bounded from above, we obtain
6 Conclusions
We have obtained, for the first time in the literature, an analytical solution for the optimal production management when customer’s demand depends on the regime and the production rates must be nonnegative. The uncertainty is modeled by two factors: a Brownian Motion that represents instantaneous random fluctuations around the longterm drift of the demand, and a continuoustime Markov chain that represents the regime. We have assumed that the production rate must be nonnegative. We have investigated two cases: one in which the company can produce immediately any amount of a good, and one in which the production rate is bounded from above. The first case is modeled as a singular stochastic control problem and the second case as a classical stochastic control problem. We have applied the dynamic programming method to obtain an analytical solution for both problems. For both cases, the optimal production policy is to produce when the inventory level is below a threshold that depends on the regime. The thresholds for the first case are different from the thresholds for the second case.
We have applied analytical or numerical methods to investigate how the optimal production policy depends (for each regime) on the demand drifts, the volatility demands, the inventory targets, the production costs, and the production constraints. We have obtained interesting managerial insights, like the dependency of the optimal production on the demand’s volatility. Indeed, while the optimal production in the unbounded from the above model decreases (equivalently, the production thresholds decrease) in both regimes when the volatilities increase, that is not always the case in the bounded from the above model. Our numerical analysis shows that, in the bounded from the above model, the optimal production decreases in both regimes (equivalently, the production thresholds decrease) when the volatility in the lowdemand regime increases, but an increase in the highdemand regime volatility leads to higher production in both regimes (equivalently, leads to higher production thresholds in both regimes). We have also shown that the optimal production policy depends dramatically on the regime.
References
Baldursson, F. M., & Karatzas, I. (1996). Irreversible investment and industry equilibrium. Finance and Stochastics, 1, 69–89.
Bensoussan, A., Sethi, S. P., Vickson, R., & Derzko, J. (1984). Stochastic production planning with production constraints. SIAM Journal on Control and Optimization, 22, 920–935.
Björk, T. (1980). Finite dimensional optimal filters for a class of Itô processes with jumping parameters. Stochastics, 4, 167–183.
Boyce, W. E., & DiPrima, R. C. (1997). Elementary differerental equations (6th ed.). John Wiley & Sons Inc.
Cadenillas, A., Lakner, P., & Pinedo, M. (2013). Optimal production management when demand depends on the business cycle. Operations Research, 58, 1046–1062.
Covei, D., Canepa, E., & Pirvu, T. (2022). Stochastic production planning with regime switching. Journal of Industrial and Management Optimization, 519(2), 1697–1713.
Covei, D., Canepa, E., & Pirvu, T. (2022). A stochastic production planning problem. Fixed Point Theory, 23, 179–198.
Covei, D. (2023). Exact solution for the production planning problem with several regimes switching over an infinite horizon time. Preprint available at https://doi.org/10.20944/preprints202309.0965.v1
Fleming, W. H., Sethi, S. P., & Soner, H. M. (1987). An optimal stochastic production planning problem with randomly fluctuating demand. SIAM Journal on Control and Optimization, 25, 1494–1502.
Fleming, W., & Soner, H. M. (1993). Controlled Markov processes and viscosity solutions. New York: Springer.
Guo, X. (2001). An explicit solution to an optimal stopping problem with regime switching. Journal of Applied Probability, 38, 464–481.
Guo, X., Miao, J., & Morellec, E. (2005). Irreversible investment with regime shifts. Journal of Economic Theory, 122, 37–59.
Jang, B. G., Koo, H. K., Liu, H., & Loewenstein, M. (2007). Liquidity premia and transaction costs. Journal of Finance, 52, 2329–2366.
Karatzas, I., & Shreve, S. E. (1984). Connections between optimal stopping and singular stochastic control I. Monotone follower problems. SIAM Journal on Control and Optimization, 22(6), 856–877.
Sethi, S. P., & Thompson, G. L. (1981). Simple models in stochastic production planning. In A. Bensoussan, P. R. Kleindorfer, & C. S. Tapiero (Eds.), Applied stochastic control in econometrics and management science. Kluwer Academic Publishers.
Sethi, S. P. (2019). Optimal control theory, applications to management science and economics (3rd ed.). Springer.
Sotomayor, L. R., & Cadenillas, A. (2009). Explicit solutions of consumptioninvestment problems in financial markets with regime switching. Mathematical Finance, 19, 251–279.
Sotomayor, L. R., & Cadenillas, A. (2011). Classical and singular stochastic control for the optimal dividend policy when there is regime switching. Insurance: Mathematics and Economics, 48, 344–354.
Vollmann, T. E., Berry, W. L., & Whybark, D. C. (1984). Manufacturing planning and control systems. Irwin.
Yong, J., & Zhou, X. Y. (1999). Stochastic controls: Hamiltonian systems and HJB equations. Springer.
Zhou, B., Zhao, Y., & Katehakis, M. (2007). Effective control policies for stochastic inventory systems with a minimum order quantity and linear costs. International Journal of Production Economics, 106, 523–531.
Acknowledgements
The research of Abel Cadenillas was funded by the Natural Sciences and Engineering Research Council of Canada (NSERC) Grant 202004065. Giorgio Ferrari and Patrick Schuhmann gratefully acknowledge financial support by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation)  SFB 1283/2 2021  317210226. The authors also thank Shihao Zhu for his help with the numerical examples.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Cadenillas, A., Ferrari, G. & Schuhmann, P. Optimal production management when there is regime switching and production constraints. Ann Oper Res (2024). https://doi.org/10.1007/s1047902405892y
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s1047902405892y