Informed production optimization in hydrocarbon reservoirs

The exploitation of subsurface hydrocarbon reservoirs is achieved through the control of production and injection wells (i.e., by prescribing time-varying pressures and flow rates) to create conditions that make the hydrocarbons trapped in the pores of the rock formation flow to the surface. The design of production strategies to exploit these reservoirs in the most efficient way requires an optimization framework that reflects the nature of the operational decisions and geological uncertainties involved. This paper introduces a new approach for production optimization in the context of closed-loop reservoir management (CLRM) by considering the impact of future measurements within the optimization framework. CLRM enables instrumented oil fields to be operated more efficiently through the systematic use of life-cycle production optimization and computer-assisted history matching. Recently, we have proposed a methodology to assess the value of information (VOI) of measurements in such a CLRM approach a-priori, i.e. during the field development planning phase, to improve the planned history matching component of CLRM. The reasoning behind the a-priori VOI analysis unveils an opportunity to also improve our approach to the production optimization problem by anticipating the fact that additional information (e.g., production measurements) will become available in the future. Here, we show how the more conventional optimization approach can be combined with VOI considerations to come up with a novel workflow, which we refer to as informed production optimization. We illustrate the concept with a simple water flooding problem in a two-dimensional five-spot reservoir and the results obtained confirm that this new approach can lead to significantly better decisions in some cases.


Introduction
Hydrocarbons are found trapped in the pores of subsurface reservoir rock. To exploit oil and gas reservoirs, wells are drilled to connect the rock formation to the surface. Production wells deplete the reservoir by imposing bottom-hole pressures (BHPs) lower than the original reservoir pressure. Injection wells are used to inject fluids (e.g., water or gas) at higher BHPs than the original reservoir pressure to replace the produced volumes and maintain the pressure of the reservoir. Together, production and injection wells create a pressure gradient in the reservoir that forces the fluids originally in-place to flow through the rock formation towards the production wells. As in other processes, an improved performance (i.e., recovery of hydrocarbons) can be achieved by designing optimal control strategies (i.e., well settings) for the system. The flow of fluids through the reservoir involves complex (nonlinear) physical phenomena that depend on many characteristics of the reservoir system whose knowledge is usually very limited. Therefore, the search of production strategies for efficient recovery of hydrocarbons requires methods to handle the inherent uncertainties, in particular geological uncertainties. In our context, reservoir management is the set of practices used to mitigate the effect of these uncertainties and optimize the reservoir performance. To achieve that, reservoir engineers and geoscientists rely on models to characterize the reservoir, integrating all the knowledge available. These models are used to simulate the reservoir response to production strategies and provide elements to support the important decisions to be taken.
Recent tools developed to support decision making in reservoir operations rely on model-based optimization and uncertainty quantification to find the best production strategy. Closed-loop reservoir management (CLRM) goes a step further and takes advantage of the frequent measurements collected throughout the reservoir producing life-time to determine the optimal set of controls (see Jansen et al. 2005 and more references in the Sect. 2). This is achieved through the systematic use of lifecycle optimization in combination with computer-assisted history matching. This combination provides the ability to react to the observations from the true reservoir (i.e., measurements gathered through the designed surveillance plan), offering the opportunity to benefit from the remaining flexibility of the production strategy and compensate for possibly wrong previous decisions, which are doomed to be suboptimal due to the presence of uncertainty.
A major challenge in determining the optimal production strategy arises from the nature of the main uncertainties inherent to the reservoir management problem. The geological uncertainties are endogenous (Jonsbråten 1998): they only get revealed at the cost of decisions. In other words, the production measurements that can help reducing the uncertainty can only be gathered when the field is operated, which requires a decision to be made (i.e., an operational strategy to be defined). In practice this means that the future state of uncertainty depends on current and previous decisions. Therefore, a truly optimal production strategy can only be determined by considering its own impact in the future measurement outcomes and their consequences to the future decisions to be made under the future state of uncertainty.

Informed production optimization in hydrocarbon reservoirs
Recently, we have proposed a methodology in which the CLRM framework is used to assess the value of future measurements (Barros et al. 2016a). Such a methodology allows one to quantify the expected value achievable with the improved decision making enabled by the selected surveillance plan. Thus, it can be used as a tool to assist operators in the design of the optimal surveillance plan (i.e., measurement type, location, frequency, precision,...).
In order to estimate the VOI for a given surveillance plan, we calculate the additional value of the future measurements in terms of the value enabled by the production strategies re-optimized in a closed-loop fashion with the new information. Because it considers the availability of future information, the VOI assessment framework can potentially also help eliminating the shortcoming of the traditional optimization approach related to the endogenous nature of the uncertainties. In this paper, we therefore propose to integrate VOI considerations into the production optimization framework to come up with a novel workflow, which we refer to as informed production optimization (IPO).
In Sect. 2 we recap the key concepts for the development of our new method and relate to previous work addressing problems of similar nature. Next, in Sect. 3, we introduce the proposed workflow for the simplest application of IPO, showing how it builds upon the VOI assessment framework we proposed in Barros et al. (2016a). Thereafter, we illustrate it with a small case study, describing details of our implementation and analyzing the improved VOI results obtained with IPO. After that, we discuss in Sect. 4 how the methodology can be extended to a more general case with multiple decision stages and observation times. Finally, in Sect. 5, we reflect on the advantages of this new concept to optimize production strategies, and we suggest directions for future work.

Closed-loop reservoir management (CLRM)
Closed-loop reservoir management (CLRM) is a combination of frequent life-cycle production optimization and data assimilation (also known as computer-assisted history matching); see Fig. 1. Life-cycle optimization aims at maximizing a financial measure, typically net present value (NPV), over the producing life of the reservoir by optimizing the production strategy. This may involve well location optimization, or, in a more restricted setting, optimization of well rates and pressures for a given configuration of wells, on the basis of one or more numerical reservoir models. Data assimilation involves modifying the parameters of one or more reservoir models, or the underlying geological models, with the aim to improve their predictive capacity, using measured data from a potentially wide variety of sources such as production data or time-lapse seismic. For further information on CLRM see, e.g., (Jansen et al. 2005(Jansen et al. , 2008Naevdal et al. 2006;Sarma et al. 2008;Chen et al. 2009;Wang et al. 2009;Foss and Jensen 2011;Hou et al. 2015).

Robust optimization
The conventional approach to optimization under geological uncertainties is the socalled robust optimization. Robust life-cycle optimization uses one or more ensembles of geological realizations (or reservoir models) to account for uncertainties and to determine the production strategy that maximizes a given objective function over the ensemble of N realizations: = { 1 , 2 , … , N } , where i is a vector of model parameters for realization i; see, e.g., Yeten et al. (2003) or Van Essen et al. (2009). Typically, the objective function to be optimized is the net present value (NPV): where NPV is the ensemble mean of the objective function values J i of the individual realizations. The objective function J i for a single realization i is defined as usual: where t is time, T is the producing life of the reservoir, q o is the oil production rate, q wp is the water production rate, q wi is the water injection rate, r o is the price of oil produced, r wp is the cost of water produced, r wi is the cost of water injected, b is the discount factor expressed as a fraction per year, and is the reference time for discounting (typically one year). The outcome of the optimization procedure is a vector containing the settings of the control variables over the producing life of the reservoir. More specifically, the vector prior = [ ⊤ 1 2 ⊤ … M ⊤ ] ⊤ comprises the schedule of well controls for each of the M control time intervals (typically months Fig. 1 Closed-loop reservoir management as a combination of life-cycle optimization and data assimilation (Jansen et al. 2008) 1 3 Informed production optimization in hydrocarbon reservoirs or quarters) through the entire reservoir life-cycle T (typically tens of years). Note that, although the optimization is based on N models, only a single strategy is obtained, under the rationale that only one strategy can be implemented in reality. Note also that, despite being popular among CLRM practitioners, the robust optimization approach presented by Van Essen et al. (2009) is only one way of dealing with uncertainty in production optimization. An alternative approach is to balance risk and return within the optimization by including well-defined risk measures or other utility functions in the objective function; see, e.g., Capolei et al. (2015) and Siraj et al. (2016).

VOI assessment in CLRM
Recently, we proposed a new methodology to assess the VOI of future measurements by making use of the CLRM framework (Barros et al. 2016a). Our approach presented there consists of closing the loop in the design phase to simulate how information obtained during the producing life-time of the reservoir comes into play in the context of optimal reservoir management. By considering both data assimilation and optimization in the procedure, we are able to not only quantify how information changes knowledge, but also how it influences the results of decision making (Barros et al. 2016a). This is possible because a new production strategy is obtained every time the models are updated with new information, and the strategies with and without additional information can be compared in terms of the value of the optimization objective function (typically NPV) obtained when applying these strategies to a virtual asset (a synthetic truth); see the flowchart in Fig. 2. We define the VOI of future measurements as the additional value realized by the asset when the future information is utilized for the optimization of the subsequent well controls. Since we do not have perfect knowledge of the true asset, we use an ensemble of plausible truths. For a problem with M control time intervals and considering the i th plausible truth i truth , the baseline for VOI calculation is the value J i prior obtained without the future information. This involves optimization of all the well controls prior = [ ⊤ prior,1 prior,2 ⊤ … prior,M ⊤ ] ⊤ under the initial state of uncertainty (i.e., over the prior ensemble of model realizations). Note that here we consider the approximation presented (Barros et al. 2016a) to accelerate the procedure: although multiple prior ensembles are used to perform the history matching for the different plausible truths, a single ensemble is used to determine the strategy prior optimal under initial uncertainty. Starting from this baseline, the additional value VOI i j of the i j measurements gathered during the j th control interval is the difference between the value J i post,j obtained through the re-optimization of the well controls in the sub- ⊤ … prior,j ⊤ ] ⊤ , are the ones determined by optimization without future information (i.e., under initial uncertainty).

Future information and optimization
The concept of accounting for the availability of future information within the optimization is not new and has been investigated for several applications in different scientific communities. In operations research, such an optimization problem is addressed from a more mathematical perspective, being referred to as stochastic programming. Two-stage or multistage models are used to account for the sequential nature of the decisions, which allows the optimization to be expressed in a nested formulation. For more information, see Birge and Louveaux (1997), Ruszczyski and Shapiro (2003), Georghiou et al. (2011). In the formalism from decision and game theory, non-myopic decision rules are the ones where the decision makers look ahead and consider future information, opposed to the myopic approach in which the influence of current decisions on the future state of uncertainty (i.e., conditional posterior distributions) is ignored (Mirrokni et al. 2012). The artificial intelligence community refers to this class of problems as partially observable Markov decision  Barros et al. (2016a) processes (POMDPs), related to applications where direct observations of the state of the uncertain processes are not available (Smallwood and Sondik 1973;Hauskrecht 2000). In these cases, the decisions are optimized not only based on their direct contribution to produce value but also to maximize the expected pay-off (or reward) of subsequent decisions Guestrin 2007, 2009). In systems and control theory, the dual control introduced by Feldbaum (1960Feldbaum ( , 1961 seeks to determine the optimal trade-off between excitation and control to promote a more active learning from the measurements while directing the system to its optimal state. This is only possible through the definition of control policies that anticipate the availability of future measurements and their learning effect. In this respect, Van Hessem (2004) discussed the steps to be taken to turn the traditional open-loop MPC (i.e., model predictive control) methods into feedback mechanisms that know how to respond to future measurements. More recently, Hanssen (2017) and Hanssen et al. (2017) proposed an implicit dual MPC controller that explicitly includes the feedback mechanism in the optimization problem. In system identification, Forgione et al. (2015) investigated the use of different model update strategies to enable batch-to-batch improvement in the control of industrial processes.
The main applications of these ideas are in: logistics and supply chain problems; network problems such as traffic control and power grids; medical decision making; planning and scheduling. In the oil and gas upstream sector, Jonsbråten (1998) applied stochastic programming to drilling sequence optimization in a simplified setting. Goel and Grossmann (2004) used stochastic models in the planning of offshore gas fields with uncertainties related to the reserves, but without considering reservoir simulation models. Foss and Jensen (2011) claimed that a conscious exploitation of the dual effect of controls can be significant in a reservoir system given the typical large uncertainties, but highlighted the fact that the dual-control problem is unsolvable in practice. Zenith et al. (2015) investigated the use of sinusoidal oscillations as a means of exciting the reservoir to obtain more informative data during well testing, but without analyzing the influence on the subsequent decisions. More recently, Abreu et al. (2015) have proposed a methodology to optimize under uncertainty the valve settings of smart wells considering the possibility of acquiring additional information in the future. Their approach uses the ideas of approximate dynamic programming to make the problem tractable. Hanssen (2017), Hanssen et al. (2017) proposed to tackle the reservoir management problem using closed-loop predictions to optimize control policies which account for the feedback mechanism instead of optimizing control sequences or set points.

Two-stage IPO
In the context of production optimization, the decision in question is the use of an optimized production strategy that maximizes the value of the asset. Despite the limited potential for learning due to the poor information content of typical production measurements, the knowledge about a reservoir changes along its producing life-time. Thus determining a single production strategy by performing robust optimization based on the initial state of uncertainty (Van Essen et al. 2009) is equivalent to ignoring the endogenous nature of the reservoir management problem. It is a myopic approach which will likely lead to suboptimal solutions. The stochastic programming ideas discussed in the previous section can be a solution to overcome this limitation.
In this paper we present a workflow for non-myopic production optimization within the CLRM framework. The proposed procedure is a combination of classical stochastic programming and our previously introduced workflow for VOI assessment in CLRM (Barros et al. 2016a). The idea is to model the decision process related to the reservoir management problem with the help of elements of the CLRM framework (i.e., ensemble-based uncertainty quantification, model-based optimization and computer-assisted history matching), which allows us to represent the sequential character of the decisions (corresponding to Markov decision process) while accounting for future measurement data with very limited information content. Because this approach considers future information when performing production optimization, we name it informed production optimization (IPO).
The ultimate goal of reservoir management is to maximize the value delivered by the asset with the implementation of production and surveillance strategies. Our methodology for VOI assessment (Barros et al. 2016a) provides a framework to quantify the value J i post,j to be produced by the asset (here, the i th plausible truth) with the incorporation of future information gathered through the designed surveillance strategy (here, measurements collected during the j th control time interval). For simplicity, we initially address the case with only a single observation time. Later, we discuss how these ideas can be extended to cases with multiple observation times.
In the conventional approach for production optimization, we seek to maximize the predicted objective function for the ensemble of models anticipating that in a closed-loop setting, once more information becomes available, we will have the opportunity to improve our predictive models and adjust our strategies to achieve the best possible J i post,j . By following this reasoning, the optimization is making use of the flexibility of the remaining degrees of freedom ⊤ ] ⊤ , while the part of the strategy prior to the future information prior (0 ∶ t j ) = [ prior,1 ⊤ prior,2 ⊤ … prior,j ⊤ ] ⊤ is deemed to be suboptimal. We propose here to use J i post,j as the cost function for our production optimization problem, as a way of reflecting the true goal of reservoir management when optimizing prior (0 ∶ t j ) . In fact, since we consider an ensemble of N truth equiprobable plausible truths, we still perform robust optimization and our new objective function is defined as where post,j is the ensemble mean of the objective function values J i post,j for the individual plausible truths. The objective function J i post,j for a single realization i of the plausible truth is calculated according to the workflow in Fig. 2, and involves the

3
Informed production optimization in hydrocarbon reservoirs solution of history matching with the future information gathered during control interval j and re-optimization of the production strategy for the subsequent control intervals i post (t j+1 ∶ T) . We can also see the optimization of this new cost function as a two-stage stochastic model or a nested optimization problem where the outer optimization concerns the production strategy up to control interval j and the inner optimization determines the remaining part of the strategy, which will be different for each one of the N truth plausible truths. Thus, the IPO problem can be formulated as where the outcome of the optimization is a single optimal production strategy IPO (0 ∶ t j ) until the observation time t j , and N truth optimal strategies i post (t j+1 , 2, … , T) , i = 1, 2, … , N truth , for the remaining producing time. The proposed workflow to solve this optimization problem iteratively is displayed in Fig. 3. The procedure resembles the original VOI assessment workflow presented in Barros et al. (2016a) (Fig. 2), but contains an outer iterative loop (shown in blue) to keep updating the part of the production strategy prior to the observation time. The inner  Fig. 2; the only difference being that, for the optimization problem, we are only interested in evaluating the new cost function J i post,j for each plausible truth and not in computing the associated VOI. Note that, for the case with a single observation time, the optimizations required in this (yellow) part of the workflow reduce to the conventional robust optimization problem, because there are no more future measurements to be considered after t j . Therefore, the optimal strategies i post (t j+1 ∶ T) can be determined by following the conventional robust optimization approach proposed by Van Essen et al. (2009). Note also that, just like the VOI workflow can be simplified to quantify the value of clairvoyance (VOC) (Barros et al. 2016a), the IPO procedure can be modified to perform optimization under the assumption that perfect revelation of the truth will be possible in the future.
By recommending a single IPO (0 ∶ t j ) and multiple i post (t j+1 ∶ T) we design a flexible production strategy that accounts for the fact that we have the opportunity to update well controls after collecting the measurements at t j . Because we consider an ensemble of plausible truths, we recognize that the outcome of the future measurements (or future clairvoyance) is unknown, and that the ensemble of i post (t j+1 ∶ T) controls should reflect this uncertainty in the production strategy to be implemented after t j . Only when we proceed with the reservoir operations (i.e., by truly implementing IPO (0 ∶ t j ) ) and the actual measurements take place at t j , the uncertainty model can be updated and the optimal strategy to be implemented for the remaining time can be determined.
From the algorithmic point of view, this nested optimization can be solved just as any other optimization problem. The same methods used in the conventional robust approach apply but the cost function to be evaluated is more complex. Here, we use gradient-based methods for both the inner and the outer optimizations. In this setting, the main challenges concern the outer optimization, in particular for the definition of the starting point and the computation of the gradient for such a complex cost function. We explain the details of our implementation when we describe our case study in Sect. 3.1.

Example
To test our approach, we used a simple two-dimensional (2D) reservoir simulation model with an inverted five-spot well pattern (i.e., 4 production wells placed at the corners of the reservoir and 1 (water) injection well placed in the middle). This case study follows the exact same configuration with heterogeneous permeability and porosity fields as the example in one of our previous papers. See Barros et al. (2016a) for a more detailed description. We used N truth = 50 plausible truths and N truth = 50 ensembles of N = 49 realizations of the porosity and permeability fields, conditioned to hard data in the wells, to model the geological uncertainties. The simulations were used to determine the set of well controls (bottom-hole pressures) that maximizes the NPV. The optimizations were run for a T = 1, 500-day time horizon with well controls updated every 150 days, i.e. M = 10 , and, with five wells, u had 50 elements. We applied bound constraints to the optimization variables (200 bar ≤ p prod ≤ 300 bar and 300 bar ≤ p inj ≤ 500 bar). The whole exercise was performed in the open-source reservoir simulator MRST (Lie et al. 2012). For the inner part of the IPO workflow (shown in yellow in Fig. 3), we used the same setup as the one used in our previous work: a CLRM environment created by combining the adjoint-based optimization and EnKF modules available with MRST (see Barros et al. 2016a).
To perform the outer optimization (shown in blue in Fig. 3), we used an implementation of the StoSAG method (Fonseca et al. 2016). The ensemble-based gradient computation was very convenient for our IPO problem, allowing us to use a black-box approach to obtain the gradients for the complex and expensive cost function while benefiting from the computational advantages for cases with the presence of uncertainties. The main difference with the use of StoSAG in a conventional production optimization problem is that here the uncertainties are characterized by the ensemble of plausible truths. Thus we pair each of the N pert = 50 perturbations of the well controls to one of the N truth = 50 plausible truths and we estimate the search direction by an approximate linear regression of their J i post,j values. We used a standard deviation of pert = 0.01 to generate the perturbations of the well controls. The starting point for the outer optimization is the solution obtained with the conventional robust optimization procedure.
For this case study, we considered the availability of oil flow rate measurements in the producers with absolute measurement errors ( oil = 5 m 3 ∕day ). Just like in our previous paper (Barros et al. 2016a), the proposed workflow considers a single observation time but was repeated for different observation times, t data = {150, 300, … , 1350} days. After applying the IPO procedure, we computed the VOI for each of the nine observation times to compare with the VOI obtained when using conventional robust optimization. In addition to that, we repeated the experiments by considering future availability of clairvoyance instead of imperfect measurements. Figure 4 illustrates the motivation for us to select oil rate measurements for this example. It compares the VOC with the value of observing total rates and water-cuts and the value of measuring only oil rates, all obtained by following the conventional optimization approach and displayed in terms of their mean values. We observe that the VOI of total rate and water-cut measurements is very close to the VOC, where the latter represents an upper bound. On the other hand, the VOI of oil rate measurements is considerably lower than the VOC, which suggests that there is room for improvement. The low values, especially for the measurements at t data = 300 days, are related to the inability of the oil rate measurements alone to accurately provide information about the water breakthrough time in the producers, whose prediction is paramount to achieve optimal reservoir management in this example. Since our goal here was to verify the potential of our proposed IPO approach to enhance the value produced by the asset, we considered the case with oil rate measurements to be a more suitable case study.
Note that the high VOI of oil rate measurements at t data = 150 days is the exception here. This is related to the nature of the waterflooding and decision making (re-optimization) processes. Initially, only oil is present in the system. The injected water flows slowly through the reservoir pores towards the location of the production wells. When the water reaches the production wells (i.e., water breakthrough), the flow of oil starts competing with the flow of water and oil rates start being affected by the fraction of water. This introduces some highly nonlinear effects to the observed response. At t data = 150 days, none of the production wells has yet experienced water breakthrough, so oil rate measurements provide better evidence to distinguish the model realizations than oil rate measurements at later times. Besides that, VOI is also a function of the timing of the measurements because a reduction in uncertainty early in time increases the chances of the re-optimization step effectively improving the remaining well controls, which will result in additional value. Figure 5 shows the results obtained with the IPO approach in comparison to those obtained when using the conventional robust optimization approach. First, we notice that the IPO does not improve significantly the VOC. This happens because, in the unrealistic case of clairvoyance, the revelation of the truth is, by definition, perfect, irrespective of the production strategy that is implemented prior to it. Thus, the gain of using the IPO approach would be more important in cases where the sub-optimality of prior (0 ∶ t j ) cannot be compensated by the reoptimization of the remaining degrees of freedom i post (t j+1 ∶ T) with perfect revelation of the truth, which does not seem to be the case for this example.
On the other hand, for the case with imperfect oil rate measurements, we observe a higher VOI for the IPO approach (Fig. 5). Note that, for the VOI calculation, the values without information are identical for both approaches. This means that the IPO was able to determine production strategies that create more value when applied to the plausible truths. Besides that, this shows that there is indeed room to achieve improved reservoir management by accounting for the availability of future information when searching for the optimal well controls. We highlight the significant increase in VOI obtained for the measurements at t data = 300 days, resulting in an improvement of approximately $ 0.8 million.
In order to explain why the strategies determined by the IPO approach are better, we looked also at how they affect the information content of the measurements. For that, we computed the observation impact I GAI (Krymskaya et al. 2010) and the uncertainty reduction NPV associated with the measurements just like we did in one of our previous papers (Barros et al. 2016a). Figure 6 displays these results for comparison with the VOI in Fig. 5. The observation impact plot indicates that the production strategies obtained with both optimization approaches produce oil rate measurements with similar information content, in terms of predicting its own outcome (i.e. observation self-sensitivity). The results in Fig. 6 (right) also suggest that both optimization approaches perform similarly in terms of producing measurements to reduce the uncertainty (spread) in NPV forecasts. We note, however, an increase in the uncertainty reduction obtained with the IPO approach for the measurements at t data = 300 and t data = 450 days, which is consistent with the improvement observed in terms of VOI for those observation times (Fig. 5).   Figure 7 shows the production strategies obtained with both optimization approaches for the case considering measurements at t data = 300 days, which correspond to the largest improvement in terms of VOI. For both approaches we observe a single production strategy defined until t data = 300 days ( prior (0 ∶ 300) and IPO (0 ∶ 300) ) and multiple strategies i post after that. We notice small differences between the prior and IPO strategies but a reasonable difference in the range of the i post strategies. Thus, for this example, small changes in prior seem to have a significant impact on the optimization of the remaining controls i post . The re-optimization of i post is performed over the posterior ensembles which will differ in both optimization approaches because the outcome of the measurements gathered at t data = 300 days correspond to the response of the true reservoirs i truth to different well controls prior (0 ∶ 300) and IPO (0 ∶ 300) . The fact that the i post derived with IPO are more scattered than in conventional optimization indicates that IPO enabled the re-optimization step to better exploit the available flexibility of the remaining degrees of freedom, thus unlocking more value.
We conclude that, for this example, the increase in VOI due to the IPO approach can be partially attributed to the ability of the improved production strategies to yield measurements that result in additional uncertainty reduction in terms of the quantity of interest (NPV). Nevertheless, there seem to be other effects that enable the IPO strategies to perform better for the plausible truths. This could be related to an enhanced controllability of the reservoir states once the measurements reveal some of the initial uncertainty. Overall, these results suggest that, when relying on measurements with limited information content, wrong reservoir management decisions cannot be entirely compensated by reoptimizing the subsequent decisions. In such cases, an approach which is merely reactive to measurements is more likely to result in poorer decisions than an approach that anticipates the availability of future information.

Multistage IPO
The results of the example above show that the IPO approach can be valuable for cases with imperfect measurements. However, the procedure depicted in Fig. 3 only addresses the case considering a single observation time, which is not realistic for well production measurements. Therefore, we discuss here what can be done to apply the same reasoning in problems with multiple observation times.
In another of our previous papers (Barros et al. 2015), we showed how our original methodology for VOI assessment can be modified to account for multiple observation times. We demonstrated that, contrary to what one would expect, the computational costs of assessing the value of multiple measurements do not increase beyond reasonable levels. The reason is that, although one might think that we need to consider new additional plausible truths for every observation time, this is not necessary. Once a realization of the initial ensemble is picked to be the plausible truth, the same realization plays the role of truth and the loop is closed as many times as the number of observation times, because that is the only way of ensuring consistency of the synthetic future measurements that we need to generate in order to assess their VOI. Thus, the computational costs of assessing the VOI for multiple observation times is the same as those of the VOI assessment for a single observation time repeated for different observation times, just like what was done in the example above.
Unfortunately, the extension of IPO to cases with multiple observation times is not that simple. In VOI assessment only, the plausible truths are not directly involved in the optimization, but they do play a role in the IPO. Thus, sticking to one of the realizations as the plausible truth throughout all the observation times would imply that we can identify the truth with the measurements obtained at the first observation time, which corresponds to the availability of clairvoyance. Since we are dealing with imperfect measurements, in order to be realistic, we need to update the uncertainty model given by our ensemble of plausible truths. This means that, in principle, the size of the problem grows exponentially with the number of observation times considered, if it is to be solved rigorously. In other words, the number of branches on the scenario tree associated with the problem would be multiplied for every new observation. Figure 8 displays a decision tree for a problem with four observation times where the uncertainty is represented by three new plausible truths each time. We notice the rapid increase in the number of scenarios, which would be even more dramatic if we consider tens of plausible truths and more observation times.
Given the already prohibitive computational costs of VOI assessment, it is safe to say that the rigorous solution of this scenario tree is not feasible for problems relying on reservoir simulation models. Gupta and Grossmann (2011) and Tarhan et al. (2013) have proposed strategies to solve multistage stochastic programming models more efficiently through approximate solutions involving decomposition and constraint relaxation methods.
Here, we propose to use a more intuitive approach to simplify this problem. The first branching before t 1 is the most important one, as it introduces the different plausible truths that will be used to generate the synthetic measurements and to evaluate the performance of the production strategies derived. The second branching before t 2 is also important because it establishes that the respective plausible truth is not perfectly revealed by the imperfect observations made at t 1 . However, the branching events associated with the remaining observation times are of smaller importance, because the one at t 2 already guarantees that there are always multiple branches below each plausible truth, reflecting the fact that at any time the uncertainty around the plausible truths is never revealed through unrealistic clairvoyance. This allows us to prune the scenario tree and reduce the size of the problem to be solved for IPO with multiple observation times. Figure 9 shows the pruned scenario tree. Note that after t 2 the number of branches does not increase, which means that, by applying this pruning strategy, we can obtain approximate solutions for three-stage and M-stage stochastic problem at similar computational costs.
To put this idea into practice and extend the two-stage IPO (Fig. 3) to M-stage IPO (with M−1 observation times), we need to make a few changes in the workflow. Figure 10 depicts the modified procedure. First, we introduce an additional ensemble of N truth truths i truth for each one of the original plausible truths i truth of our initial ensemble truth . This corresponds to the second branching of the scenario tree before t 2 (Fig. 9). From there on, the tree does not develop new branches but, for every observation time, i truth is updated through history matching of the synthetic measurements generated with i truth . After the last observation, we derive the in terms of our objective function and, finally, we compute the cost function as the mean over all the realizations. Thus, the M-stage IPO problem can be formulated as where the outcome of the optimization is a single optimal production strategy IPO 0 ∶ t 1 until the first observation time t 1 , N truth optimal strategies i post t 1 ∶ t M−1 , i = 1, 2, … , N truth , for the period between the first and the last observation, and N truth 2 optimal strategies i,j post t M−1 ∶ T , i = 1, 2, … , N truth , j = 1, 2, … , N truth , for the remaining producing time.
The M-stage IPO workflow can be realized in a similar implementation as the one described for our case study. The inner optimization can be performed with the help of the adjoint-based gradients, while the intermediate and outer optimizations can make use of the StoSAG method to estimate the required gradients.

Example
To test the IPO approach with multiple observation times, we used again the 2D five-spot example, but with a small modification: instead of M = 10 , we considered M = 3 control time intervals. We assumed oil production measurements to be available at t data = {500, 1000} days, with the same measurement error as before. In order to make the problem more tractable, we used the acceleration measures presented in Barros et al. (2016b), and we reduced the number of plausible truths to N repr truth = 10 representative ones and the size of the ensembles for robust optimization to N repr = 5 . The history matching step with the EnKF was performed over the full ensembles. Figure 11 displays the VOI of the multiple oil rate measurements obtained through the procedure from Barros et al. (2015) with the conventional optimization approach. Figure 11 (bottom) shows that the VOI for the last observation time ( t data = 1000 days) is significant lower than the VOC, which suggests that there might be scope to improve them with the IPO approach. Thus, the goal of the IPO here can be understood as an optimization of the VOI at t data = 1000 days, by maximizing the J i,j post achievable through the CLRM framework with the measurements at t data = {500, 1000} days. Figure 12 displays the results obtained in terms of the NPV achieved by the plausible truths. Figure 12 (left) compares the mean NPV for different cases: in red, we have the value attained by the production strategies optimized under prior uncertainty; in blue and in black, the NPV values corresponding to the VOI and VOC at t data = 1000 days (Fig. 11); and, in green, the value for the IPO approach. We observe an improvement of approximately $ 55,000 for the mean NPV achieved with the IPO (green) in comparison with the mean NPV for the CLRM using the conventional optimization approach, which represents an increase of 0.1%, or of 4% if expressed in terms of VOI. We note that the incremental gain is small but should be understood as an attempt to improve a solution already close to optimal for this example. This suggests that, in this example with more observations, the CLRM framework with the conventional optimization approach was able to sufficiently compensate for previous suboptimal decisions, resulting in production strategies almost as good as the ones determined through IPO. Figure 12 (right) depicts the same results but showing the empirical probability distribution function (pdf) curves derived with the NPV of the N repr truth = 10 plausible truths considered. The differences between the curves for the CLRM and IPO are small. However, the distribution for the IPO approach seems to achieve higher values at its peak, which suggests a reduction in the spread of the NPV values. Note that the pdf curves displayed here are the result of curve fitting with a small number of samples ( N repr truth = 10 ), but that the empirical histograms exhibit similar trends ( Fig. 12 (bottom)). Figure 13 shows the production strategies determined by the different approaches compared in Fig. 12. A single production strategy prior (0 ∶ 500) is derived for the conventional robust optimization approach. On the other extreme, a different optimal strategy is determined for each of the N repr truth = 10 plausible truths considered, under the assumption of clairvoyance at t = 0 . In between, for the CLRM, the solutions obtained consist of the same single strategy as prior (0 ∶ 500) for the first control time interval, and N repr truth = 10 different strategies i post (500 ∶ 1500) for the remaining of the time. Finally, we see the outcome of the IPO for this three-stage problem: a single optimal production strategy IPO (0 ∶ 500) until the first observation time t 1 = 500 days, N repr truth = 10 optimal strategies i post (500 ∶ 1000) , i = 1, 2, … , N repr truth , for the period between t 1 = 500 and t 2 = 1, 000 days, and N repr truth 2 = 100 optimal strategies i,j post (1000 ∶ 1500) , i = 1, 2, … , N repr truth , j = 1, 2, … , N repr truth , from t 2 = 1, 000 until T = 1, 500 days. The larger number of possibilities associated with future control time intervals allows the IPO strategies to account for the flexibility of the reservoir management problem. We also note that IPO (0 ∶ 500) is not the same as  Fig. 11 Results for the VOI analysis with the conventional robust optimization (2D five-spot model with M = 3 control time intervals) prior (0 ∶ 500) and that the difference between them enables the IPO approach to result in improved NPV values, as seen in Fig. 12.

Discussion and conclusions
We have proposed a new approach for production optimization under geological uncertainty. The informed production optimization (IPO) approach considers the endogenous nature of these uncertainties and includes the availability of future information to circumvent the limitations of the conventional robust optimization. To achieve this, the approach accounts for the fact that there is uncertainty about the optimal well settings to be implemented after additional measurements are processed in the future, and that this uncertainty depends on the outcome of these measurements. As a consequence, we perform robust IPO over an ensemble of plausible truths resulting in a single optimal production strategy until the moment that future measurements take place and a range of optimal production strategies for the period after them. We applied IPO to a simple case study, which resulted in a significant increase of the VOI obtained from imperfect measurements, reflecting the improvement in the production optimization. After demonstrating the potential value of the IPO approach for a case with a single  An important point that has not been addressed in this work concerns the generation of future synthetic measurements and, in particular, the contribution of the measurement noise. Here we assumed white noise with a pre-defined standard deviation to be added to the simulated measurements, but, in practice, other choices may be more appropriate. Although, in most of the cases, small when compared to the geological uncertainties, this noise contribution may have an impact on the results obtained with the proposed IPO approach. Further research is required for a better understanding of its impact.
Another point for future work concerns the computational costs of such an approach, which are comparable or higher than those of VOI assessment workflows. The IPO ideas only make sense if uncertainty quantification is addressed carefully, and this may require a prohibitive amount of reservoir simulations. For example, in the results for two-stage IPO (Fig. 5), the IPO workflow required approximately 0.5 − 1 × 10 6 (forward and backward) reservoir simulations for each t data (depending on how many iterations of the outer optimization are performed), compared to approximately 0.1 × 10 6 simulations for the VOI assessment workflow from Barros et al. (2016a). In order to be useful, the IPO needs to be tractable. In one of our previous papers (Barros et al. 2016b) we managed to reduce the number of reservoir simulations required in VOI workflows by selecting representative models, a solution that can also be applied here. Besides that, the use of proxy models or multiscale methods may help to further reduce the number of high-fidelity reservoir simulations needed and make the IPO procedure more practical. Still regarding the tractability of the problem, the procedure we proposed here to extend the IPO approach to cases with multiple observation times represents an approximation of the full multistage stochastic model. There may be other solutions to prune the decision tree and effectively reduce the problem to fewer relevant scenarios, as, e.g., suggested by Gupta and Grossmann (2011) and Tarhan et al. (2013). Therefore, also in this direction there is scope for future research.