Cluster Computing

, Volume 22, Supplement 3, pp 6267–6281 | Cite as

Constrained short-term and long-term multi-objective production optimization using general stochastic approximation algorithm

  • Hui Zhao
  • Yu ZhangEmail author
  • Lin CaoEmail author
  • Xiaodong Kang
  • Xiaoqing Xie


In the water flooding reservoir management, engineers aim to make an optimal production strategy to maximize both the net present value (NPV) in life-cycle and the short-term NPV. Previous researches of multi-objective optimization of long-term and short-term find that the short-term production optimization decreases the long-term NPV and the gradient calculation using adjoint gradient is hard to be applied. Therefore, we implement a multi-objective optimization method in which the long-term NPV is lumped into the short-term objective function as a constraint in order not to decrease the life-cycle NPV, linear and nonlinear constraints are lumped into the augmented Lagrangian objective function. Also, we propose a new gradient-free algorithm general stochastic approximation algorithm (GSA) to obtain the better approximate gradient. We proved that the research direction of GSA algorithm is always uphill, the simultaneous perturbation stochastic approximation algorithm (SPSA) and the ensemble-based optimization algorithm (EnOpt) are also two special forms of GSA algorithm. Toy problem results show that GSA has a more efficient approximation gradient than SPSA. Optimization results of a 2D reservoir show that the increase of the long-term NPV is about 86.5% and the increase of the short-term NPV is about 18.8%. The field case of Brugge field also shows that the long-term and short-term NPV increase by 64.7 and 48.9%.


Multi-objective optimization General stochastic approximation algorithm Net present value 

List of symbols

\( b \)

The annual discount rate(%)

\( C_{U} \)

The auto covariance matrix of u

\( C_{U,L} \)

The covariance matrix between u and L

\( c_{L} \)

The constraints of the long-term

\( c_{j} \)

Expression for the jth inequality equation

\( c_{v} \)

Variable of violating constraints

\( e_{i} \)

The expression for the ith equality equation

\( g \)

The true gradient

\( \hat{g} \)

The approximation gradient

\( J \)

Object function of production optimization($)

\( L \)

The augmented Lagrange function

\( M \)

The introduced optimized matrix

\( m \)

The vector consisting from the geologic parameters

\( n_{c} \)

The number of equality constraints

\( n_{e} \)

The number of inequality constraints

\( N_{L} \)

The total number of simulation timesteps of long-term

\( N_{P} \)

The total number of producers

\( N_{S} \)

The total number of simulation timesteps of short-term

\( N_{I} \)

The total number of water-injection wells

\( N_{u} \)

The dimension of control vector

\( N_{e} \)

Number of perturbation models

\( q_{o,j}^{n} \)

The average oil-production rate over the nth timestep of the jth producer (STB/d)

\( q_{w,j}^{n} \)

The water-production rate over the nth timestep of the jth producer (STB/d)

\( q_{{_{wi,i} }}^{n} \)

The water-injection rate over the nth timestep of the ith water injector (STB/d)

\( r_{o} \)

Oil price ($/L3)

\( r_{w} \)

Water production cost ($/L3)

\( r_{wi} \)

Water injection cost($/L3)

\( t^{n} \)

Nth step cumulative calculation time

\( T \)

The total time steps reservoir production control

\( u \)

Nu-dimensional column vector composed by control variable

\( u* \)

Optimal control vector of the long-term

\( u_{k}^{low} \)

Lower bound over the kth control parameters

\( u_{k}^{up} \)

Upper bound over the kth control parameters

\( u_{i}^{l} \)

The ith control variables realization of lth step

\( u_{k}^{l} \)

The control vector in the lth inner loop of the kth outer loop

\( y \)

The vector consisting from the status parameters

\( y* \)

Optimal \( y \) after the long-term optimization

\( \Delta t^{n} \)

Step of nth time step

\( \Delta \)

Random perturbation vector

Greek symbols

\( \lambda_{e,i} \)

The Lagrange multipliers for the ith equation

\( \lambda_{c,j} \)

The Lagrange multipliers for the jth equation

\( \lambda_{l} \)

The step size for search step l

\( \mu \)

The penalty factor

\( v_{j} \)

The slack variable which transformed the inequality constraint into an equality constraint

\( \phi_{j} \)

Equality constraint function

\( \varphi_{j} \)

Inequality constraint function

\( \gamma \)

The perturbation step

\( \tau ,\bar{\gamma } \)

The reduction factor

\( \alpha_{l} \)

The iteration step size

\( \alpha_{\eta } ,\beta_{\eta } \)

The intermediate variable

\( \bar{\eta },\eta^{*} \)

The variable used to detect the violation of the constraints


\( i \)

Number index

\( j \)

Number index

\( k \)

The index of the outer loop iteration

\( l \)

Iteration step number index

\( o \)

Oil index

\( r \)

Rock index

\( s \)

The gradient index of SPSA method

\( w \)

Water index

\( \text{P} \)

Producer index

\( \text{I} \)

Injector index

\( k \)

The index of the out loop iteration

\( l \)

The index of the inner loop iteration

\( i \)

The realization model

\( t^{n} \)

Cumulative time up to the nth simulator timestep



Hui Zhao would like to express his gratitude to the Project of Open Fund of State Key Laboratory of Offshore Oil Exploitation (Grant No. CCL2015RCPS0223RNN), the National Natural Science Foundation of China (No. 51674039), the China Important National Science & Technology Specific Projects (Grant No. 2016ZX05014) and the Yangtze Youth Talents found for their generous financial support of the research. Lin Cao also acknowledges the National Natural Science Foundation of China (Grant No. 51604035).

Compliance with ethical standards

Conflict of interest

We declare that we do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.


  1. 1.
    Jansen, J., S. Douma., D. Brouwer., P. V. den Hof, Heemink, A.: close-loop reservoir management. In: Proceedings of the SPE Reservoir Simulation Symposium, 2–4 Feb 2009Google Scholar
  2. 2.
    Wang, C., Li, G., Reynolds, A.C.: Production optimization in closed-loop reservoir management. In: Proceedings of the SPE Society of Petroleum Engineers, 1 September 2009Google Scholar
  3. 3.
    Chen, C., Li, G., Reynolds, A.: Close-loop reservoir management on the Brugge test case. Comput. Geoscience. 14(4), 691–703 (2010)CrossRefGoogle Scholar
  4. 4.
    Zhao, H., Chen, C., Do, S.T., Li, G., Reynolds, A.C.: Maximization of a dynamic quadratic interpolation model for production optimization. In: Paper SPE 141317 Presented at Society of Petroleum Engineers, 1 January 2011Google Scholar
  5. 5.
    Zhao, H., Li, Y., Yao, J., Zhang, K.: Theoretical research on reservoir closed-loop production management. Sci. China Tech. Sci. 54(10), 2815–2824 (2011)CrossRefGoogle Scholar
  6. 6.
    Pinto, M.A.S., Ghasemi, M., Sorek, N., Gildin, E., Schiozer, D.J.: Hybrid Optimization For Closed-Loop Reservoir Management. Society of Petroleum Engineers, Richardson (2015)CrossRefGoogle Scholar
  7. 7.
    Van Essen, G.M., van den Hof, P.M.J., Jansen, J.D.: Hierarchical long term and short term production optimization. In: Proceedings of the SPE Annual Technical Conference and Exhibition, 1 Jan 2009Google Scholar
  8. 8.
    Van Essen, G., van den Hof, P., Jansen, J.-D.: Hierarchical long-term and short-term production optimization. SPE J. 16(1), 191–199 (2011)CrossRefGoogle Scholar
  9. 9.
    Conn, A.R., Gould, N.I.M., Toint, P.L.: LANCELOT: A fortran package for large-scale nonlinear optimization (Release A), No.17. Berlin: Springer Series in Computational Mathematics, Springer-Verlag (1992)Google Scholar
  10. 10.
    Byrd, R.H., Lu, P., Nocedal, J., Zhu, C.: A Limited memory algorithm for bound constrained optimization. SIAM J. Sci. Comput. 16(5), 1190–1208 (1995)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, New York (1999)CrossRefGoogle Scholar
  12. 12.
    Aanonsen, S.I., Doublet, D., Tai, X.-C.: Efficient history matching and production optimization with the augmented lagrangian method. In: Proceedings of the SPE Reservoir Simulation Symposium, 1 Jan 2007Google Scholar
  13. 13.
    Chen, C., Wang, Y., Li, G., Reynolds, A.: Closed-loop reservoir management on the Brugge test case. Comput. Geosci. 14(4), 691–703 (2010)CrossRefGoogle Scholar
  14. 14.
    Chen, C., Li, G., Reynolds, A.: Robust constrained optimization of short- and long-term net present value for closed-loop reservoir management. In: Paper SPE141314 Presented at Society of Petroleum Engineers, 1 Sept2012Google Scholar
  15. 15.
    Zhao, H., Tang, Y. W., Li, Y. et al. Reservoir production optimization using general stochastic approximate algorithm under the mixed-linear-nonlinear constraints. J. Resid. Sci. Technol. 13(8) (2016)Google Scholar
  16. 16.
    Brouwer, D., Jansen, J.: Dynamic optimization of water flooding with smart wells using optimal control theory. SPE J. 9(4), 391–402 (2009)CrossRefGoogle Scholar
  17. 17.
    Romero, C., Carter, J., Gringarten, A., Zimmerman, R.: A modified genetic algorithm for reservoir characterization. In: Paper SPE 64765 presented at the International Oil and Gas Conference and Exhibition in China. 1 Jan 2000Google Scholar
  18. 18.
    Montes, G., Bartolome, P., Udias, A.: The use of genetic algorithms in well placement optimiaziton. In: Paper SPE 69439 presented at the SPE Latin American and Caribbean Petroleum Engineering Conference. 1 Jan 2001Google Scholar
  19. 19.
    Yeten, B.: Optimum deployment of nonconventional wells. PhD thesis, Department of Petroleum Engineering (2003)Google Scholar
  20. 20.
    Emerick, A.A., Silva, E., Messer, B., Almeida, L., Szwarcman, D., Pacheco, M., Vellasco, M.: Well placement optimization using a genetic algorithm with nonlinear constraints. In: Paper SPE 18808 presented at the SPE Reservoir Simulation Symposium, 2–4 Feb 2009Google Scholar
  21. 21.
    Parsopoulos, K., Vrahatis, M.: Recent approaches to global optimization problems 36 through particle swarm optimization. Nat. Comput. 1, 235–306 (2002)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Onwunalu, J.E.: Optimization of field development using particle swarm optimization and new well pattern optimization. PhD thesis, Department of Energy Resources Engineering, Stanford University (2002)Google Scholar
  23. 23.
    Isebor, O., Durlofsky, L., Echeverria ciaurri, D.: A derivative-free methodology with local and global search for the constrained joint optimization of well locations and controls. Comput. Geosci. (2013). CrossRefGoogle Scholar
  24. 24.
    Deutsch, C.V., Journel, A.G.: The application of simulated annealing to stochastic 5 reservoir modeling. SPE Adv. Tech. Ser. 2(2), 222–227 (1994)CrossRefGoogle Scholar
  25. 25.
    Beckner, B., Song, X.: Field development planning using simulated annealing-optimal economic well scheduling and placement. In: Paper SPE 30650 Presented at the SPE Annual Technical Conference and Exhibition Dallas (1995)Google Scholar
  26. 26.
    Yang, X., Deb, S.: Cuckoo search via Le`vy flights proceedings of world congress on nature and biologically inspired computing. Sci. World J. (2009). CrossRefGoogle Scholar
  27. 27.
    Walton, S., Hassan, O., Morgan, K., Brown, M.R.: Modified cuckoo search: a new gradient free optimization algorithm. Chaos Solitons Fract. 44, 710–718 (2011)CrossRefGoogle Scholar
  28. 28.
    Wang, C., Li, G., Reynolds A.C.: Production optimization in the context of closed-loop reservoir management. In: Paper SPE 109805 Present at Annual Technical Conference and Exhibition. 11–14 Nov 2007Google Scholar
  29. 29.
    Zhao, H.: Theoretical research on reservoir close-loop production optimization. PhD thesis China university of petroleum (2011)Google Scholar
  30. 30.
    Zhao, H., Cao, L., Li, Y., et al.: Production optimization of oil reservoirs based on an improved simultaneous perturbation stochastic approximate algorithm. Acta Pet. Sinca. 32(6), 1031–1036 (2011)Google Scholar
  31. 31.
    Chen, Y., Oliver, D.S., Zhang, D.: Efficient ensemble-based closed-loop production optimization. SPE J. 14(4), 634–645 (2009)CrossRefGoogle Scholar
  32. 32.
    Fonseca, R., Leeuwenburgh, O., Van den Hof, P., Jansen, J.-D.: Improving the Ensemble-Optimization Method Through Covariance-Matrix Adaptation. Society of Petroleum Engineers, Richardson (2014)CrossRefGoogle Scholar
  33. 33.
    Do, S.T., Reynolds, A.C.: Theoretical connections between optimization algorithms based on an approximate gradient. Comput. Geosci. 17(6), 959–973 (2013)MathSciNetCrossRefGoogle Scholar
  34. 34.
    Zhang, K., Wang, Z., Zhang, L., Yao, J., Yan, X.: A hybrid optimization method for solving bayesian inverse problems under uncertaint. PLoS ONE (2015). CrossRefGoogle Scholar
  35. 35.
    Yuan, Y., Sun, W.: Optimization Theory and Method. Science and technology press, Haidian, Beijing (1999)Google Scholar
  36. 36.
    Spall, J.C.: Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Trans. Autom. Control 37(3), 332–341 (1992)MathSciNetCrossRefGoogle Scholar
  37. 37.
    Spall, J.C.: Adaptive stochastic approximation by the simultaneous perturbation method. IEEE Trans. Autom. Control 45(10), 1839–1853 (2000)MathSciNetCrossRefGoogle Scholar
  38. 38.
    Zhao, H., Li, Y., Kang, Z.: Robust optimization in oil reservoir production. Acta Pet. Sinca. 34(5), 947–953 (2013)Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Petroleum EngineeringYangtze UniversityWuhanChina
  2. 2.State Key Laboratory of Offshore Oil ExploitationBeijingChina
  3. 3.CNOOC Research InstituteBeijingChina

Personalised recommendations