## Abstract

Operation of reservoirs is a fundamental issue in water resource management. We herein investigate well-posedness of an optimal control problem for irrigation water intake from a reservoir in an irrigation scheme, the water dynamics of which is modeled with stochastic differential equations. A prototype irrigation scheme is being developed in an arid region to harvest flash floods as a source of water. The Hamilton–Jacobi–Bellman (HJB) equation governing the value function is analyzed in the framework of viscosity solutions. The uniqueness of the value function, which is a viscosity solution to the HJB equation, is demonstrated with a mathematical proof of a comparison theorem. It is also shown that there exists such a viscosity solution. Then, an approximate value function is obtained as a numerical solution to the HJB equation. The optimal control strategy derived from the approximate value function is summarized in terms of rule curves to be presented to the operator of the irrigation scheme.

## 1 Introduction

A stock-and-flow structure is a key concept in economics as well as in water resource management. Stocks of water in reservoirs, such as dams, aquifers, lakes, ponds, and tanks, regulate flows of water, which are intrinsically uneven and uncertain (Borgomeo et al. 2014; Zhang and Babovic 2011). Stochastic processes and control theories have been applied to water resource management problems in both the design and operation stages (Leroux and Martin 2016; Cui and Schreider 2009; Zhao et al. 2014; Pelak and Porporato 2016; Basinger et al. 2010). An extreme case is being studied in a harsh environment, where a small reservoir is constructed to collect ephemeral water flows from flash floods in order to fully irrigate perennial plants, as shown in Fig. 1 (Unami et al. 2015). The structure harvesting flash floods consists of a gutter cutting across a 16 m wide valley bottom and a conveyance channel of 60 m long to guide the water to the reservoir. The conveyance channel is equipped with a spillway to release excess backwater from the reservoir. Operation of the reservoir involves an optimal control problem considering the inherently stochastic occurrence of flash floods, while the operator can make decisions on the intake flow rate from the reservoir for irrigation. The entire irrigation scheme, which consists of a reservoir with flash flood harvesting facilities and a command area of plant cultivation, is so small that the decision maker has perfect information. The water dynamics in the irrigation scheme is modeled as a set of stochastic differential equations (SDEs) representing the water balance in the reservoir as well as the uncertain occurrence and intensity of flash floods and droughts. The performance index to be minimized is the expected deficiency of water in the future. In the present paper, we attempt to establish well-posedness for such an optimal control problem with mathematical rigor, which is lacking in earlier practical papers on relevant topics (Unami et al. 2013, 2015; Sharifi et al. 2016).

In the context of dynamics programming, the Hamilton–Jacobi–Bellman (HJB) equation governs the value function from which an optimal control strategy is derived for a time-continuous problem. The versatility of the HJB equation is evident as its industrial applications are so vast, covering the fields of population dynamics (Guo and Sun 2005), financial engineering (Junca 2012; Leach et al. 2007; Bo et al. 2013), aircraft flight mechanics (Almgren and Tourin 2015), climate risk assessment (Chaumont et al. 2006), and energy systems (Sieniutycz 2009, 2012, 2015). The notion of viscosity solutions is a powerful vehicle for approaching the HJB equation, which is nonlinear and degenerate in most cases, and comparison theorems are fundamental in discussing uniqueness and stability of solutions (Fleming and Soner 2006; Kawohl and Kutev 2007; Ishii 1987; Ishii and Lions 1990; Crandall and Lions 1983). Peron’s method is a standard means of constructing viscosity solutions (Crandall et al. 1992). However, the HJB equation derived from the optimal control problem considered herein encounters some difficulties. The comparison theorems known thus far are not applicable because of irregular conditions imposed when the reservoir is empty or full. Therefore, special auxiliary functions are sought to establish a comparison theorem, which guarantees the uniqueness of a continuous viscosity solution to the HJB equation with a relaxed Hamiltonian. Another theorem is also proven to show the existence of the viscosity solution as well as to justify a numerical approach, embedding a space of weak solutions into the space of viscosity solutions, in a manner analogous to a previous study dealing with one-dimensional stationary Hamilton–Jacobi equations of first order (Guermond and Popov 2008).

An approximate value function obtained as a numerical solution to the HJB equation yields the optimal control strategy in the real world. Concurrent use of the finite difference and finite element methods is a promising discretization technique for nonlinear and degenerate partial differential operators. The optimal control strategy is the maximizer of a characteristic function depending on the value function. Assuming that the control strategy derived from the approximate value function is optimal, it is summarized in terms of rule curves for reservoir operation (Senga 1991; Khan et al. 2012; Moghaddasi et al. 2013), and a simplified chart is presented to the operator for actual implementation. This is an innovative demonstration test in the prototype irrigation scheme based on a rigorous mathematical foundation.

As usual, the notations \( C \), \( C^{1} \), \( C^{2} \), and \( C^{\infty } \) shall be used for the sets of continuous and continuously differentiable functions.

## 2 Formulation of the optimal control problem

A stochastic model is developed for water dynamics in the irrigation scheme. Then, an optimal control problem is formulated with a performance index to be minimized, in order to deduce the optimal control strategy for irrigation water intake from the reservoir harvesting flash floods.

### 2.1 Stochastic model for water dynamics

The dynamics of the storage volume \( X_{t} \) of the reservoir at time \( t \) is governed by the water balance equation

where \( Q_{\text{in}} \) is the inflow rate of the harvested flash flood, which is equal to the runoff discharge of the flash flood after subtracting the rate of overflow from the spillway, \( Q_{\text{out}} \) is the outflow rate due to evaporation and seepage, and \( u \) is the intake flow rate as a control variable constrained in a set \( U \) of admissible controls. A virtual variable \( Y_{t} \) referred to as the water flow index is considered to model the dynamics of \( Q_{\text{in}} \) and \( Q_{\text{out}} \). The one-dimensional Langevin equation is assumed to govern \( Y_{t} \) as

where \( r \) is a reversion coefficient, \( D \) is a diffusion coefficient, and \( B_{t} \) is the standard Brownian motion (Øksendal 2007). The advantages of using this virtual model (2) are capability in comprehensively representing the stochastic flow rate dynamics of flash floods as well as the occurrence of dry spells. A non-decreasing function \( Q_{\text{in}} (y) \) is assumed to define the relationship between \( Y_{t} \) and \( Q_{\text{in}} \), while \( X_{t} \) and \( Y_{t} \) determine \( Q_{\text{out}} \) with another function \( Q_{\text{out}} (x,y) \). The storage volume \( X_{t} \) of the reservoir is assumed to almost surely not exceed its capacity \( V \), because of the well-functioning spillway. It is also trivial that \( X_{t} \) cannot decrease when it is equal to 0. Consequently, the domain of \( X_{t} \) is restricted to the closed interval \( \left[ {0,V} \right] \), as is common in most reservoir operation problems. Consequently, (1) is rewritten as

with

where \( \wedge \) and \( \vee \) represent the minimum and the maximum, respectively. A target flow rate \( Q_{\text{trg}} \) of irrigation water as a function of the time \( t \) is set within the maximum capacity of the intake facility, e.g., a pump. Depending on the feasibility of intake from the reservoir, the admissible set \( U \) is prescribed as

### 2.2 Performance index and HJB equation

The current time \( s \) is assumed to be in an irrigation period \( \left[ {0,T} \right) \) (\( T < \infty \)). The performance index \( J^{u} (s,x,y) \) at time \( t = s \) with storage volume \( X_{s} = x \) and water flow index \( Y_{s} = y \) is defined as

where \( \text{E}^{s,x,y} \) represents the expectation with respect to the probability law of the stochastic processes starting at point \( \left( {s,x,y} \right) \), and \( f \) is a bounded non-negative penalty function evaluating the departure of the actual intake flow rate \( u \) from the target \( Q_{\text{trg}} \). The value \( X_{T} \) at the end of the irrigation period is the bequest to be maximized. The choice of \( u \) is optimized to attain the infimum of \( J^{u} \left( {s,x,y} \right) \). It is assumed that \( u \) is a Markov control, the choice of which at time \( t \) depends only on the current values of \( X_{t} \) and \( Y_{t} \). The infimum \( \Phi = \Phi \left( {s,x,y} \right) \) of \( J^{u} \left( {s,x,y} \right) \) exists because \( f \ge 0 \) and \( 0 \le V - X_{T} \le V \), and is referred to as the value function. The control \( u^{*} \) attaining \( \Phi \) is referred to as the optimal control. Therefore,

As mentioned in Chapters IV and V of Fleming and Soner (2006) including the verification theorem, the HJB equation

governs the value function \( \Phi \) and the optimal control \( u^{*} \) for \( \left( {s,x,y} \right) \) in the set \( G = \left[ {0,T} \right) \times \left( {0,V} \right) \times {\mathbb{R}} \), with the terminal condition

No boundary condition is imposed in the \( x \)-direction, because of the special treatment specified in (4). That value function \( \Phi \) should be understood as a viscosity solution to the HJB Eq. (8), which is degenerating. The optimal control \( u^{*} \) at any point in \( G \) is obtained as

where \( \psi \) is the characteristic function defined by

Then, the HJB equation (8) is rewritten as

where

A penalty function is chosen as

where \( \gamma \) is a positive bounded weight depending on the water flow index \( y \), which is assumed here to be

where \( K \) is a model parameter, which will be determined from physical data observed in the real world (Sect. 5). For a feasible operational flow rate \( Q_{\text{p}} \) of the intake facility, the irrigation period \( \left[ {0,T} \right) \) is divided into \( N + 1 \) non-irrigation hours \( I_{2i} = \left[ {t_{2i} ,t_{2i + 1} } \right) \) and \( N \) irrigating hours \( I_{2i + 1} = \left[ {t_{2i + 1} ,t_{2i + 2} } \right) \), where \( Q_{\text{trg}} = 0 \) and \( Q_{\text{trg}} = Q_{\text{p}} \), respectively, so that \( \left[ {0,T} \right) = \left[ {t_{0} ,t_{2N + 1} } \right) = \mathop \cup \limits_{k = 0}^{2N} I_{k} \). This partition of the entire irrigation period into a sequence of time intervals reduces the original problem into a sequence of the HJB equations with Hamiltonians independent of time. Under the above-mentioned conditions, it is easy to verify the boundedness of \( \Phi \):

###
*Remark 1*

For any \( s \in \left[ {0,T} \right) \),

When \( Q_{\text{trg}} = Q_{\text{p}} \), (10) and (11) are reduced to

and

Otherwise, these two equations are reduced to

and

The optimal control \( u^{*} \) may not be unique, as in (17). However, eliminating \( u^{*} \) reduces the HJB Eq. (12) with (18) and (20) to obtain

and

## 3 Uniqueness of viscosity solution to the HJB equation

For each non-negative integer \( k \le 2N \), the temporal variable is inverted as

and the value function \( \Phi = \Phi \left( {\tau ,x,y} \right) \) is defined on the set \( G_{k} = \left( {0,t_{k + 1} - t_{k} } \right] \times \left[ {0,V} \right] \times {\mathbb{R}} \). Then, the HJB equations (21), (22), and (23) is further rewritten as

where \( {\mathbf{x}} = \left( {\begin{array}{*{20}c} x \\ y \\ \end{array} } \right) \), and \( H \) is the Hamiltonian defined as

with

and

where \( {\mathbf{p}} = \left( {\begin{array}{*{20}c} {p_{x} } \\ {p_{y} } \\ \end{array} } \right) \) and \( M = \left( {\begin{array}{*{20}c} {\mu_{xx} } & {\mu_{xy} } \\ {\mu_{yx} } & {\mu_{yy} } \\ \end{array} } \right) \). However, the discontinuity appearing in (28) when \( x = 0 \) and \( x = V \) hinders the comparison theorem, which holds if the function \( \alpha \left( {x,y,Q} \right) \) is relaxed as

where \( \eta \) is a small positive relaxation parameter, so that \( \alpha_{\eta } \left( {x,y,Q} \right) \) uniformly approaches \( \alpha \left( {x,y,Q} \right) \) as \( \eta \to 0 \). Henceforth, this \( \alpha_{\eta } \left( {x,y,Q} \right) \) will be used in (27) instead of \( \alpha \left( {x,y,Q} \right) \). The definitions of \( a\left( {x,y,u} \right) \) and \( \Delta Q \) will be accordingly revised as \( a_{\eta } \left( {x,y,u} \right) \) and \( \Delta Q_{\eta } \) in (4) and (13), respectively. The assertion of Remark 1 is still valid for the relaxed case.

###
*Remark 2*

\( \hat{H}\left( {\tau ,x,y,p_{x} } \right) \) with the relaxed (29) is Lipschitz continuous in each \( I_{k} \) with respect to \( \tau \), \( x \), and \( y \).

Now, we move on to viscosity solution to the relaxed HJB equation. A real-valued function \( \Phi \) defined on a set \( E \) is called upper semi-continuous, if for any \( \left( {\tau ,{\mathbf{x}}} \right) \in E \subset {\mathbb{R}}^{3} \) and any \( \varepsilon > 0 \) there exists \( \delta \) such that \( \Phi \left( {\tau^{\prime},{\mathbf{x^{\prime}}}} \right) < \Phi \left( {\tau ,{\mathbf{x}}} \right) + \varepsilon \) for all \( \left( {\tau^{\prime},{\mathbf{x^{\prime}}}} \right) \in B_{\delta } \left( {\tau ,{\mathbf{x}}} \right) \cap E \), where \( B_{\delta } \left( {\tau ,{\mathbf{x}}} \right) \) represents the \( \delta \)-neighborhood of \( \left( {\tau ,{\mathbf{x}}} \right) \). Similarly, it is called lower semi-continuous in the case where the inequality is replaced by \( \Phi \left( {\tau^{\prime},{\mathbf{x^{\prime}}}} \right) > \Phi \left( {\tau ,{\mathbf{x}}} \right) - \varepsilon \). Let \( USC\left( E \right) \) and \( LSC\left( E \right) \) denote the sets of all upper and lower semi-continuous functions defined on \( E \), respectively. The upper and lower semi-continuous envelopes \( \Phi^{U} \) and \( \Phi^{L} \) of a real-valued function \( \Phi \) on \( G_{k} \) are defined as

and

respectively. Note that \( \Phi^{L} \in USC\left( {\bar{G}_{k} } \right) \) and \( \Phi^{L} \in LSC\left( {\bar{G}_{k} } \right) \). For \( \Phi^{U} \in USC\left( {\bar{G}_{k} } \right) \), being a viscosity sub-solution to (25) implies that

for any test function \( w \in C^{2} \left( {G_{k} } \right) \), such that

For \( \Phi^{L} \in LSC\left( {\bar{G}_{k} } \right) \), being a viscosity super-solution to (25) implies that

for any test function \( w \in C^{2} \left( {G_{k} } \right) \), such that

If \( \Phi^{U} \) is a viscosity sub-solution and \( \Phi^{L} \) is a viscosity super-solution, then \( \Phi \) is called a viscosity solution.

We firstly discuss the continuity of viscosity solutions, which are value functions of the optimal control problem, at \( \tau = 0 \).

###
**Theorem 1**

*Suppose that* \( \Phi_{v} \) *is a bounded viscosity solution to* (25) *with* (29) *and that* \( \Phi_{v}^{U} \left( {0,{\mathbf{x}}} \right) = \Phi_{v}^{L} \left( {0,{\mathbf{x}}} \right) \) *for* \( {\mathbf{x}} \in \left[ {0,V} \right] \times {\mathbb{R}} \)*. If* \( \Phi_{v} \left( {0,{\mathbf{x}}} \right) = \Phi_{v}^{U} \left( {0,{\mathbf{x}}} \right) = \Phi_{v}^{L} \left( {0,{\mathbf{x}}} \right) \) *as a function of* \( {\mathbf{x}} \) *is continuous in* \( \left[ {0,V} \right] \times {\mathbb{R}} \)*, then*

###
*Proof*

For \( \tau^{\prime} \in \left[ {0,t_{k + 1} - t_{k} } \right] \) and \( {\mathbf{x^{\prime}}} \in \left[ {0,V} \right] \times {\mathbb{R}} \), it holds that

with

for any admissible \( u \), because \( \Phi_{v} \) is a value function of the optimal control problem. For any \( \varepsilon > 0 \), there exists an admissible \( \tilde{u} \) such that

With the chosen penalty function (14), the first part of the expectation in the right hand side of (39) is evaluated as

and then subtracting \( \Phi_{v} \left( {0,{\mathbf{x}}} \right) \) from the inequalities (37) and (39) leads to

for any \( {\mathbf{x}} \in \left[ {0,V} \right] \times {\mathbb{R}} \). By the definition (38)

and therefore (36) holds. □

The following comparison theorem coupled with Theorem 1 proves the uniqueness of viscosity solutions of (25) with (29).

###
**Theorem 2**

*Suppose that* \( \Phi_{1} \in USC\left( {\bar{G}_{k} } \right) \) *is a bounded viscosity sub*-*solution to* (25) *with* (29) *and that* \( \Phi_{2} \in LSC\left( {\bar{G}_{k} } \right) \) *is a bounded viscosity super*-*solution to* (25) *with* (29)*. Then,*

###
*Proof*

We opt for proof by contradiction in two stages. Firstly, an auxiliary function \( \Psi \) is defined as

where \( {\varvec{\upxi}} = \left( {\begin{array}{*{20}c} \xi \\ \zeta \\ \end{array} } \right) \), and \( \varphi \left( \tau \right) \in C^{\infty } \left( {\left[ {0,t_{k + 1} - t_{k} } \right]} \right) \) is a function satisfying \( \varphi \left( \tau \right) \ge 0 \) and \( \varphi \left( \tau \right) = 0 \) if \( \tau = 0 \). Two points \( \left( {\bar{\tau },{\bar{\mathbf{x}}}} \right) \) and \( \left( {\bar{\sigma },{\bar{\mathbf{\xi }}}} \right) \) of \( \bar{G}_{k} \) are assumed to maximize \( \Psi \) as

Then, the inequality

leads to evaluations

where \( K_{1} \) is a non-negative constant given by

Assume that \( \bar{\tau } > 0 \) and \( \bar{\sigma } > 0 \). We set a smooth function \( v\left( {\tau ,{\mathbf{x}}} \right) \) as

which turns out to be a test function for a viscosity sub-solution because

and therefore

This implies that

An eligible function \( \varphi \left( \tau \right) \) is chosen here as

with \( \beta > 0 \), to set another smooth function \( \hat{v}\left( {\tau ,{\mathbf{x}}} \right) \) as

which turns out to be a test function for a viscosity super-solution because

and therefore

This implies that

Comparing (52) and (57) yields

The left-hand side of (58) remains positive for any positive \( \beta \), \( \delta \), and \( \varepsilon \), while its right-hand side approaches to zero as \( \delta ,\varepsilon \to + 0 \) because \( \hat{H} \) is continuous, to yield a contradiction. Therefore, \( \bar{\tau } = 0 \) or \( \bar{\sigma } = 0 \) or both.

Now, we prove

Assume that (59) is not true. Then, there exists a point \( \left( {\tau_{M} ,{\mathbf{x}}_{M} } \right) \in G_{k} \) such that

while one of the inequalities

holds. By the evaluations (47), it is possible to choose \( \varepsilon \) and \( \delta \) such that \( \left| {\bar{\tau } - \bar{\sigma }} \right| + \left\| {{\bar{\mathbf{x}}} - {\bar{\mathbf{\xi }}}} \right\| \le \rho \) for any \( \rho > 0 \). Then, considering the properties of upper and lower semi-continuous functions, \( \rho \) is chosen so that

for any \( \varepsilon_{0} > 0 \). Adding (62) to (61) results in

to obtain

On the other hand,

Combining (64) and (65) results in

Another choice of \( \varphi \left( \tau \right) \) as

with \( \beta > 0 \) is also eligible and leads to

as \( \beta ,\varepsilon_{0} \to + 0 \). However, (68) contradicts (45) and thus (59) is true. Consequently, we reach to (43). □

###
*Remark 3*

If there is a viscosity solution to (25) with (29) satisfying a specified continuous initial condition in the sense of Theorem 1, then its uniqueness and continuity are direct consequences of Theorem 2.

## 4 Existence of viscosity solution to the HJB equation

A weak solution to the HJB equation (25) with (29) from a specified initial condition is considered in order to show the existence of a viscosity solution as well as to provide a framework for approximate numerical solution.

Transformation of the independent variable \( y \) to \( z \) with \( z = \tan^{ - 1} \left( {\sqrt {\frac{r}{2D}} y} \right) \) makes the domain bounded. Indeed, \( G_{k} \) is mapped to \( G_{k}^{*} = \left( {0,t_{k + 1} - t_{k} } \right] \times \bar{\Omega }_{x} \times \Omega_{z} \), where \( \Omega_{x} = \left( {0,V} \right) \) and \( \Omega_{z} = \left( { - {\pi \mathord{\left/ {\vphantom {\pi 2}} \right. \kern-0pt} 2},{\pi \mathord{\left/ {\vphantom {\pi 2}} \right. \kern-0pt} 2}} \right) \). Let \( \Omega_{\tau } \) denote \( \left( {0,t_{k + 1} - t_{k} } \right) \). Then, the HJB equation (25) is formally transformed into

but \( \Phi \) at each \( \tau \in \left( {0,t_{k + 1} - t_{k} } \right] \) shall be sought in the function space \( H_{xz}^{1} \), which is completed with the norm

where

for \( * = x \) or \( z \), to satisfy the weak form

for any weights \( \hat{w}_{x} \in H^{1} \left( {\Omega_{x} } \right) \) and \( \hat{w}_{z} \in H^{1} \left( {\Omega_{z} } \right) \), where \( H^{1} \left( {\Omega_{*} } \right) \) is the Sobolev space equipped with the norm (71). For \( * = \tau \) or \( x \) or \( z \), it is known that there are embeddings \( H^{1} \left( {\Omega_{*} } \right) \to C_{B} \left( {\Omega_{*} } \right) \) and \( H^{1} \left( {\Omega_{*} } \right) \to C\left( {\bar{\Omega }_{*} } \right) \), where \( C_{B} \left( {\Omega_{*} } \right) \) is the set of all bounded continuous functions on a domain \( \Omega_{*} \), and \( C\left( {\bar{\Omega }_{*} } \right) \) is the set of all bounded uniformly continuous functions on a domain \( \Omega_{*} \). Both of \( C_{B} \left( {\Omega_{*} } \right) \) and \( C\left( {\bar{\Omega }_{*} } \right) \) are Banach spaces equipped with the uniform norm, and \( C\left( {\bar{\Omega }_{*} } \right) \) is a closed subspace of \( C_{B} \left( {\Omega_{*} } \right) \) (Adams and Fournier 2003). Let \( C_{B} \left( {G_{k}^{*} } \right) \) denote the set of all bounded continuous functions on \( G_{k}^{*} \), which is a Banach space equipped with the uniform norm \( \left\| \Phi \right\|_{\infty } = \mathop {\sup }\limits_{{\left( {\tau ,x,z} \right) \in G_{k}^{*} }} \left| \Phi \right| \). Consider the function \( g \) of \( \hat{w}_{x} \), \( \hat{w}_{z} \), and \( \Phi_{(\kappa )} \) with time increment \( \delta \tau \) as

where

For fixed \( \hat{w}_{z} \) and \( \Phi_{(\kappa )} \), \( g \) is a continuous linear functional on \( H^{1} \left( {\Omega_{x} } \right) \). For fixed \( \hat{w}_{x} \) and \( \Phi_{(\kappa )} \), \( g \) is a continuous linear functional on \( H^{1} \left( {\Omega_{z} } \right) \). Applying the Riesz representation theorem (Adams and Fournier 2003) twice, \( g \) is identified with another function \( \Phi_{(\kappa + 1)} \in H_{xz}^{1} \). Starting from an initial value \( \Phi_{(0)} \), iterations \( \left\{ {\Phi_{(\kappa )} } \right\}_{\kappa = 1,2,3 \ldots } \) with \( \delta \tau = {\tau \mathord{\left/ {\vphantom {\tau {N_{\tau } }}} \right. \kern-0pt} {N_{\tau } }} \) and \( N_{\tau } \ge 1 \) yield the Riemann sum

Equation (75) approaches

as \( N_{\tau } \) approaches \( + \infty \), which is consistent with (72).

###
*Remark 4*

There exists at least one solution \( \Phi_{w} = \Phi_{w} \left( {\tau ,x,z} \right) \) to the initial value problem (72) with an initial value \( \Phi_{(0)} \in H_{xz}^{1} \). Here, \( \Phi_{w} \in H_{xz}^{1} \) for any \( \tau \in \left( {0,t_{k + 1} - t_{k} } \right] \) and \( \Phi_{w} \in C_{B} \left( {\Omega_{\tau } } \right) \) for any \( \left( {x,z} \right) \in \bar{\Omega }_{x} \times \Omega_{z} \), implying that \( \Phi_{w} \in C_{B} \left( {G_{k}^{*} } \right) \).

The following theorem asserts that the above-mentioned weak solution accords with the viscosity solution to (25).

###
**Theorem 3**

*Suppose that* \( \Phi_{v} = \Phi_{v} \left( {\tau ,{\mathbf{x}}} \right) = \Phi_{v} \left( {\tau ,x,y} \right) \) *is a bounded viscosity solution to* (25) *with* (29) *satisfying the initial condition* \( \Phi_{v} \left( {0,{\mathbf{x}}} \right) = \Phi_{v} \left( {0,x,y} \right) = \Phi_{\left( 0 \right)} \left( {x,\tan^{ - 1} \left( {\sqrt {\frac{r}{2D}y} } \right)} \right) \) *for any* \( \left( {x,z} \right) \in \bar{\Omega }_{x} \times \Omega_{z} \) *and that* \( \Phi_{w} = \Phi_{w} \left( {\tau ,x,z} \right) \) *is a weak solution to* (72) *with* (29) *satisfying the same initial condition. Then,*

*in* \( G_{k} \).

###
*Proof*

Let \( \Phi_{v}^{U} \) and \( \Phi_{v}^{L} \) be the upper and lower semi-continuous envelopes of \( \Phi_{v} \), respectively. Namely, \( \Phi_{v}^{U} \) and \( \Phi_{v}^{L} \) are the viscosity sub-solution and the viscosity super-solution, respectively. From Theorem 1,

where \( C_{B} \left( {\left[ {0,V} \right] \times {\mathbb{R}}} \right) \) is the set of all bounded continuous functions on \( \left[ {0,V} \right] \times {\mathbb{R}} \). There exists a sequence \( \left\{ {\Phi^{\left( n \right)} \left( {\tau ,x,z} \right)} \right\} \subset C^{1} \left( {G_{k}^{*} } \right) \cap C^{2} \left( {\Omega_{z} } \right) \) converging to \( \Phi_{w} \). Namely, for any \( \varepsilon > 0 \), there exists a natural number \( N_{1} \) such that

for any \( n > N_{1} \) at all \( \tau \in \left( {0,t_{k + 1} - t_{k} } \right] \). Because of the embeddings \( H^{1} \left( {\Omega_{x} } \right) \to C\left( {\bar{\Omega }_{x} } \right) \) and \( H^{1} \left( {\Omega_{z} } \right) \to C_{B} \left( {\Omega_{z} } \right) \), there exists another natural number \( N_{2} \), such that

for any \( n > N_{2} \) at all \( \tau \in \left( {0,t_{k + 1} - t_{k} } \right] \). Consequently, \( \left\{ {\Phi^{\left( n \right)} \left( {\tau ,x,z} \right)} \right\} \) becomes a Cauchy sequence converging to a limit \( \Phi_{c} \left( {\tau ,x,z} \right) \) in \( C_{B} \left( {G_{k}^{*} } \right) \), and thus \( \Phi_{w} \left( {\tau ,x,z} \right) = \Phi_{c} \left( {\tau ,x,z} \right) \) in \( G_{k}^{*} \). Furthermore, for any \( \delta > 0 \), there exists a natural number \( N_{3} \) such that

for any \( n > N_{3} \) in \( G_{k}^{*} \). For each \( n \), let \( \delta_{i}^{\left( n \right)} \) (\( i = 1,2 \)) be real numbers such that

and

Then, test functions \( w_{i}^{\left( n \right)} \) are chosen as

so that

and

for some \( \left( {\hat{\tau }_{i} ,{\hat{\mathbf{x}}}_{i} } \right) \in G_{k} \). This implies that \( w_{i}^{\left( n \right)} \) are indeed eligible for test functions of viscosity sub-solution \( \Phi_{v}^{U} \) and viscosity super-solution \( \Phi_{v}^{L} \), resulting in

and

at respective \( \left( {\hat{\tau }_{i} ,{\hat{\mathbf{x}}}_{i} } \right) \). In order to satisfy (87) and (88), \( \delta_{i}^{\left( n \right)} \) must approach zero as \( n \to \infty \), because of (81). Finally, we obtain

in \( G_{k} \), and thus (77). □

## 5 Application with numerical demonstration

A prototype irrigation scheme including a reservoir for harvesting flash floods is being developed at the Agricultural Research Station of Mutah University, located in the Lisan Peninsula of the Dead Sea near the town of Ghor al Mazrah in Jordan. The model parameters are determined from the physical dimensions of the structures as well as from hydro-meteorological observation conducted from September 27th, 2014 through September 22nd, 2016. Indeed, the reservoir consists of two sections: a 300 m^{3} section enclosed in a greenhouse, and a 700 m^{3} section, the surface of which is exposed to open air. Once a flash flood is harvested in the open section, the water is immediately transferred to the closed section if there is room. Therefore, the sections are regarded collectively as a single reservoir of *V* = 1000 m^{3} with \( Q_{\text{out}} \left( {x,y} \right) \) varying in the \( x \)-direction. The irrigation period \( \left[ {0,T} \right) \) is set as 5.2560 × 10^{5} min of a non-leap year from May 1st through April 30th. No flash flood is expected during the months from May through October. The period from 08:00 a.m. through 08:12 a.m. is prescribed as the irrigation hours for every day throughout the irrigation period. Without loss of generality, the diffusion coefficient \( D \) is assumed to be unity. In the two consecutive winter rainy seasons of 2014–2015 and 2015–2016 included in the observation period, there were 16 events of flash floods (10 events in 2014–2015 season and 6 events in 2015–2016 season), out of which 8 events (3 events in 2014–2015 season and 5 events in 2015–2016 season) yielded substantial harvesting. The model parameter \( K \) represents the supremum of \( y \), where there is no inflow of flash flood to the reservoir, and its value is estimated to be 2.4165. The most likely value of the reversion coefficient in terms of the compatible transition probability density function is 0.0011421 per minute. The functions \( Q_{\text{in}} \left( y \right) \) and \( Q_{\text{out}} \left( {x,y} \right) \) are determined as shown in Fig. 2. The blue line in the figure indicates \( Q_{\text{in}} \) in the unit of m^{3}/min during the wet months from November through April, identified from statistical analysis of the observed data as

while \( Q_{\text{in}} \equiv 0 \) during the dry months from May to October. Seepage is negligible because of plastic sheets covering the bottom of the reservoir, and the closed section is free from evaporation. Evaporation from the water surface of the open section is estimated at \( {{10} \mathord{\left/ {\vphantom {{10} {\left( {1 + \exp \left( {y - K} \right)} \right)}}} \right. \kern-0pt} {\left( {1 + \exp \left( {y - K} \right)} \right)}} \) mm/day., which is multiplied by the water surface area depending on \( x \) to yield \( Q_{\text{out}} \left( {x,y} \right) \). Note that there is no significant difference in observed evaporation between the wet and dry months.

To approximately solve the HJB equation (72) with (29) from a specified initial condition and then to derive the optimal control strategy, a computational procedure is developed as follows. The *z*-domain \( \Omega_{z} \) is divided into \( n_{z} \) sub-domains of equal length \( \Delta z = {\pi \mathord{\left/ {\vphantom {\pi {n_{z} }}} \right. \kern-0pt} {n_{z} }} \). The *x*-domain \( \Omega_{x} \) is also divided into \( n_{x} \) sub-domains of equal length \( \Delta x = {V \mathord{\left/ {\vphantom {V {n_{x} }}} \right. \kern-0pt} {n_{x} }} \), and the unknown \( \Phi \) is attributed to each node \( \left\{ {x = i\Delta x,z = k\Delta z} \right\} \) as \( \Phi_{i,k} \). For discretization in the *z*-direction, the finite element scheme developed by Unami et al. (2015) is applied to the weak form (72). For discretization in the *x*-direction, the first-order upwind finite difference scheme is used. The mesh size \( \Delta x \) is regarded as the relaxation parameter \( \eta \). Then, the system of ordinary differential equations resulting from those discretization schemes is numerically solved in the \( \tau \)-direction using the Runge–Kutta method with a constant time step \( \Delta \tau \) to update the value of each \( \Phi_{i,k} \). The optimal control strategy \( u^{*} \) is derived from the computed \( \Phi_{i,k} \), according to (10). A computational run with \( n_{x} = 100 \), \( n_{x} = 120 \), and \( \Delta \tau = {1 \mathord{\left/ {\vphantom {1 {60}}} \right. \kern-0pt} {60}} \) minutes was completed within nine days and four hours using the supercomputer system of the Academic Center for Computing and Media Studies, Kyoto University. Distribution of computed optimal control \( u^{*} \) is delineated in Figs. 3, 4, 5, 6 and 7 for the irrigation hours of representative days, which are May 1st (Day 0), July 31st (Day 91), October 30th (Day 182), January 29th (Day 273), and April 30th (Day 364). The boundary between two adjacent cells of \( u^{*} = 0 \) and \( u^{*} = Q_{\text{p}} \) in the \( x \)-\( y \) domain for each \( \tau \in \left[ {0,t_{2i + 2} - t_{2i + 1} } \right) \) is marked as a segment in a different color at each time stage of 1-min intervals. If there is a surface of \( x_{\theta } \) such that

in the \( \tau \)–\( y \) domain, the surface is referred to as a rule curve. Possibly due to the coarse discretization, oscillations in the delineated segments appear slightly in Figs. 3, 4, 5 and 6 and more visibly near \( y = - \infty \) in Fig. 7. However, practically significant rule curves can be extracted. The rule curves are mostly monotonically increasing with respect to \( y \), and, throughout the irrigation period, water should not be withdrawn from the reservoir under sufficiently wet conditions. The restriction on intake is strictest on Day 0 and is then relaxed as time evolves. Rule curves vanish after Day 182 under dry conditions.

The chart for the rule curves actually presented to the operator, which includes only three cases of water flow index \( y = \pm 3.2935 = \pm 1.3629K = \sqrt {{{2D} \mathord{\left/ {\vphantom {{2D} r}} \right. \kern-0pt} r}} \tan \left( { \pm {\pi \mathord{\left/ {\vphantom {\pi {40}}} \right. \kern-0pt} {40}}} \right) \) and \( y = 0 \), is shown in Fig. 8. The operator has also been told that irrigation should not be performed during flash floods. On the other hand, the rule curve for \( y = - 3.2935 \) should be applied under much drier conditions.

## 6 Conclusions

A prototype irrigation scheme with a reservoir for harvesting flash floods motivated the mathematical analysis of the present paper. A water dynamics model was constructed based on practically acceptable assumptions, and the model parameters were determined from the observed data.

The optimal control problem formulated for the model was shown to have a unique value function, which solves the HJB equation in the viscosity sense. In other words, it was successfully demonstrated that the optimal control problem was well-posed. Skillful use of the properties of viscosity sub-solutions and viscosity super-solutions, as well as the choices of auxiliary functions, played key roles in the proofs of the non-trivial theorems. The innovative construction method for the weak solution rationalized the numerical approximation of the value function. The comparison theorem, Theorem 2, is independent of Theorem 1 and is applicable to discontinuous viscosity solutions in general.

The rule curves for operation of the reservoir were numerically derived, suggesting that the optimal control is also unique. This is another remarkable outcome of the present study, because optimal control in a deterministic reservoir operation problem may be not unique, but may be arbitrary. Field verification of the optimal control strategy is being initiated in the real world, cultivating a perennial plant species *Phoenix dactylifera* in the irrigated command area.

## References

Adams RA, Fournier JJF (2003) Sobolev spaces. Elsevier Science, Oxford

Almgren R, Tourin A (2015) Optimal soaring via Hamilton–Jacobi–Bellman equations. Optim Control Appl Methods 36:475–495

Basinger M, Montalto F, Lall U (2010) A rainwater harvesting system reliability model based on nonparametric stochastic rainfall generator. J Hydrol 392:105–118

Bo L, Wang Y, Yang X (2013) Stochastic portfolio optimization with default risk. J Math Anal Appl 397:467–480

Borgomeo E, Hall JW, Fung F, Watts G, Colquhoun K, Lambert C (2014) Risk-based water resources planning: incorporating probabilistic nonstationary climate uncertainties. Water Resour Res 50(8):6850–6873

Chaumont S, Imkeller P, Müller M (2006) Equilibrium trading of climate and weather risk and numerical simulation in a Markovian framework. Stoch Environ Res Risk Assess 20(3):184–205

Crandall MG, Lions PL (1983) Viscosity solutions of Hamilton–Jacobi equations. Trans Am Math Soc 277(1):1–42

Crandall MG, Ishii H, Lions PL (1992) User’s guide to viscosity solutions of second order partial differential equations. Bull Am Math Soc 27(1):1–67

Cui J, Schreider S (2009) Modelling of pricing and market impacts for water options. J Hydrol 371(1):31–41

Fleming WH, Soner HM (2006) Controlled markov processes and viscosity solutions. Springer, New York

Guermond JL, Popov B (2008)

*L*^{1}-approximation of stationary Hamilton–Jacobi equations. SIAM J Numer Anal 47(1):339–362Guo BZ, Sun B (2005) Numerical solution to the optimal birth feedback control of a population dynamics: viscosity solution approach. Optim Control Appl Methods 26:229–254

Ishii H (1987) A simple, direct proof of uniqueness for solutions of the Hamilton–Jacobi equations of Eikonal type. Proc Am Math Soc 100(2):247–251

Ishii H, Lions PL (1990) Viscosity solutions of fully nonlinear second-order elliptic partial differential equations. J Differ Equ 83(1):26–78

Junca M (2012) Optimal execution strategy in the presence of permanent price impact and fixed transaction cost. Optim Control Appl Methods 33:713–738

Kawohl B, Kutev N (2007) Comparison principle for viscosity solutions of fully nonlinear, degenerate elliptic equations. Commun Part Differ Equ 32(8):1209–1224

Khan NM, Babel MS, Tingsanchali T, Clemente RS, Luong HT (2012) Reservoir optimization-simulation with a sediment evacuation model to minimize irrigation deficits. Water Resour Manag 26(11):3173–3193

Leach PGL, O’Hara JG, Sinkala W (2007) Symmetry-based solution of a model for a combination of a risky investment and a riskless investment. J Math Anal Appl 334:368–381

Leroux AD, Martin VL (2016) Hedging supply risks: an optimal water portfolio. Am J Agric Econ 98(1):276–296

Moghaddasi M, Araghinejad S, Morid S (2013) Water management of irrigation dams considering climate variation: case study of Zayandeh-rud reservoir, Iran. Water Resour Manag 27(6):1651–1660

Øksendal B (2007) Stochastic differential equations. Springer, Berlin

Pelak N, Porporato A (2016) Sizing a rainwater harvesting cistern by minimizing costs. J Hydrol 541(B):1340–1347

Senga Y (1991) A reservoir operational rule for irrigation in Japan. Irrig Drain Syst 5(2):129–140

Sharifi E, Unami K, Yangyuoru M, Fujihara M (2016) Verifying optimality of rainfed agriculture using a stochastic model for drought occurrence. Stoch Environ Res Risk Assessm 30(5):1503–1514

Sieniutycz S (2009) Dynamic programming and Lagrange multipliers for active relaxation of resources in nonlinear non-equilibrium systems. Appl Math Model 33:1457–1478

Sieniutycz S (2012) Maximizing power yield in energy systems: a thermodynamic synthesis. Appl Math Model 36:2197–2212

Sieniutycz S (2015) Synthesizing modeling of power generation and power limits in energy systems. Energy 84(5):255–266

Unami K, Yangyuoru M, Alam AHMB, Kranjac-Berisavljevic G (2013) Stochastic control of a micro-dam irrigation scheme for dry season farming. Stoch Environ Res Risk Assessm 27(1):77–89

Unami K, Mohawesh O, Sharifi E, Takeuchi J, Fujihara M (2015) Stochastic modelling and control of rainwater harvesting systems for irrigation during dry spells. J Clean Prod 88:185–195

Zhang SX, Babovic V (2011) A real options approach to the design and architecture of water supply systems using innovative water technologies under uncertainty. J Hydroinform 14(1):13–29

Zhao T, Zhao J, Lund JR, Yang D (2014) Optimal hedging rules for reservoir flood operation from forecast uncertainties. J Water Resour Plan Manag 140(12):04014041

## Acknowledgements

The authors are grateful to Prof. Hisashi Okamoto at Gakushuin University for his valuable comments and suggestions. The present research was funded by Grants-in-Aid for Scientific Research Nos. 26257415 and 16KT0018 from the Japan Society for the Promotion of Science (JSPS).

## Author information

### Authors and Affiliations

### Corresponding author

## Rights and permissions

**Open Access** This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

## About this article

### Cite this article

Unami, K., Mohawesh, O. A unique value function for an optimal control problem of irrigation water intake from a reservoir harvesting flash floods.
*Stoch Environ Res Risk Assess* **32**, 3169–3182 (2018). https://doi.org/10.1007/s00477-018-1527-z

Published:

Issue Date:

DOI: https://doi.org/10.1007/s00477-018-1527-z

### Keywords

- Optimal control problem
- Value function
- Hamilton–Jacobi–Bellman equation
- Viscosity solution
- Irrigation scheme
- Reservoir operation