Overview of estimation methods for industrial dynamic systems


Measurement technology is advancing in the oil and gas industry. Factors such as wireless transmitters, reduced cost of measurement technology, and increased regulations that require active monitoring tend to increase the number of available measurements. There is a clear opportunity to distill the recent flood of measurements into relevant and actionable information. Common methods to do this include a filtered bias update, implicit dynamic feedback, Kalman filtering, and moving horizon estimation. The purpose of these techniques is to validate measurements and align imperfect mathematical models to the actual process. Additionally, they can determine a best-estimate of the current state of the process and any potential disturbances. These methods allow potential improvements in earlier detection of disturbances, process equipment faults, and improved state estimates for optimization and control.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14


\(\alpha \) :

Filter factor for additive bias

\(\bar{P}\) :

Predicted covariance matrix

\(\bar{x}\) :

Predicted state vector

\(\Delta P_v\) :

Differential pressure

\(\hat{d}\) :

Prior values of the parameters or disturbances

\({\hat{y}}\) :

Vector of prior model values at the sampling times (\({\hat{y}}_0,\ldots ,{\hat{y}}_n\))\(^T\)

\(\Phi \) :

Objective function value

\(\sigma _q\) :

Standard deviation of state noise

\(\sigma _r\) :

Standard deviation of measurement noise

\(\tau \) :

Time constant

\(\tau _I\) :

Integral time constant for IDF

\(\tilde{\delta }\) :

Innovation: comparison of model to measurements

A :

State matrix

B :

Control matrix

b :

Additive model bias

C :

Observation matrix

\(c_L\) :

Slack variables to penalize model value changes below the prior value

\(c_U\) :

Slack variables to penalize model value changes above the prior value

\(C_v\) :

Constant relating valve position to flow

d :

Model parameter or disturbance vector

\(e_L\) :

Slack variables to penalize model values below the measurement dead-band

\(e_U\) :

Slack variables to penalize model values above the measurement dead-band

f :

Differential equation residuals


Valve lift function

g :

Output function residuals

\(g_s\) :

Specific gravity

h :

Inequality constraint residuals

I :

Integral term in IDF

K :

Kalman gain: moderate the measurement correction

\(K_c\) :

Proportional tuning constant for IDF

n :

Sampling time index

PV :

Process variable

Q :

Estimated process error covariance

q :

Flow rate (ton/h)

\(Q_d\) :

Weighting matrix on changes of the disturbance variables

\(Q_y\) :

Inverse of the measurement error covariance

R :

Estimated measurement error covariance

S :

Innovation covariance: comparison of real error to prediction

SP :


u :

Model input vector

\(w_m\) :

Vector of weights on the model values outside a measurement dead-band

\(w_p\) :

Vector of weights to penalize deviation from the prior solution

x :

Model state vector

\(x_0\) :

Vector of initial states

y :

Vector of model values with corresponding measurements

z :

Vector of measurements


Implicit dynamic feedback


Advanced process control


Bottom hole assembly


Extended Kalman filter


Moving horizon estimation


Model predictive control


Managed pressure drilling


Nonlinear programming


Proportional integral controller


Real time optimization


Single input-single output


Unscented Kalman filter


  1. Abul-el-zeet Z, Roberts P (2002) Enhancing model predictive control using dynamic data reconciliation. AIChE J 48(2):324–333

    Article  Google Scholar 

  2. Albuquerque J, Biegler L (1995) Decomposition algorithms for on-line estimation with nonlinear models. Comput Chem Eng 19(10):1031–1039

    Article  Google Scholar 

  3. Allgöwer F, Badgwell, TA, Qin JS, Rawlings JB, Wright SJ (1999) Nonlinear predictive control and moving horizon estimationan introductory overview. In: Advances in control. Springer, Berlin, pp 391–449

  4. Asgharzadeh Shishavan R, Hubbell C, Perez HD, Hedengren JD, Pixton, DS, Pink AP (2015) Multivariate control for managed pressure drilling systems using high speed telemetry. SPE J. doi:10.2118/170962-PA

  5. Biegler L, Yang X, Fischer g (2015) Advances in sensitivity-based nonlinear model predictive control and dynamic real-time optimization. J Process Control 30:104–116

    Article  Google Scholar 

  6. Binder T, Blank L, Bock H, Burlisch R, Dahmen W, Diehl M, Kronseder T, Marquardt W, Schlöder J, Stryk O (2001) Online optimization of large scale systems. In: Introduction to model based optimization of chemical processes on moving horizons. Springer, Berlin, pp 295–339

  7. Brower D, Hedengren J, Loegering C, Brower A, Witherow K, Winter K (2012) Fiber optic monitoring of subsea equipment. In: Ocean, offshore & arctic engineering OMAE, 84143. Rio de Janiero, Brazil

  8. Carey G, Finlayson B (1975) Othogonal collocation on finite elements. Chem Eng Sci 30:587–596

    Article  Google Scholar 

  9. Darby M, Nikolaou M, Jones J, Nicholson D (2011) RTO: an overview and assessment of current practice. J Process Control 21:874–884

    Article  Google Scholar 

  10. Eaton A, Safdarnejad S, Hedengren J, Moffat K, Hubbell C, Brower D, Brower A (2015) Post-installed fiber optic pressure sensors on subsea production risers for severe slugging control. In: ASME 34th international conference on ocean, offshore, and arctic engineering (OMAE), 42196. St. John’s, Newfoundland, Canada

  11. Ellis M, Durand H, Christofides PD (2014) A tutorial review of economic model predictive control methods. J Process Control 24(8):1156–1178. doi:10.1016/j.jprocont.2014.03.010. Economic nonlinear model predictive control

    Article  Google Scholar 

  12. Findeisen R, Allgöwer F, Biegler L (2007) Assessment and future directions of nonlinear model predictive control. Springer, Berlin

    Book  MATH  Google Scholar 

  13. Hallac B, Kayvanloo K, Hedengren J, Hecker W, Argyle M (2015) An optimized simulation model for iron-based Fischer–Tropsch catalyst design: transfer limitations as functions of operating and design conditions. Chem Eng J 263:268–279

    Article  Google Scholar 

  14. Haseltine E, Rawlings J (2005) Critical evaluation of extended kalman filtering and moving-horizon estimation. Ind Eng Chem Res 44(8):2451–2460

    Article  Google Scholar 

  15. Hedengren J, Brower D (2012) Advanced process monitoring of flow assurance with fiber optics. In: AIChE spring meeting. Houston, TX

  16. Hedengren J, Edgar T (2005) Order reduction of large scale DAE models. In: IFAC 16th world congress. Prague, Czechoslovakia

  17. Hedengren J, Edgar T (2006) Moving horizon estimation—the explicit solution. In: Proceedings of chemical process control (CPC) VII conference. Lake Louise, Alberta, Canada

  18. Hedengren JD, Allsford KV, Ramlal J (2007) Moving horizon estimation and control for an industrial gas phase polymerization reactor. Proceedings of the American Control Conference (ACC). New York, NY, pp 1353–1358

  19. Hedengren JD, Shishavan RA, Powell KM, Edgar TF (2014) Nonlinear modeling, estimation and predictive control in APMonitor. Comput Chem Eng 70:133–148. doi:10.1016/j.compchemeng.2014.04.013. Manfred Morari Special Issue

    Article  Google Scholar 

  20. Hutin R, Tennent R, Kashikar S (2001) New mud pulse telemetry techniques for deepwater applications and improved real-time data capabilities. In: SPE/IADC drilling conference, 67762-MS. Society of Petroleum Engineers, Amsterdam, Netherlands

  21. Jacobsen L, Spivey B, Hedengren J (2013) Model predictive control with a rigorous model of a solid oxide fuel cell. In: Proceedings of the American control conference (ACC). Washington, D.C., pp 3747–3752

  22. Jang S, Joseph B, Mukai H (1986) Comparison of two approaches to on-line parameter and state estimation of nonlinear systems. Ind Eng Chem Process Des Dev 25:809–814

    Article  Google Scholar 

  23. Jeffrey K, Forward K (2009) Improvements with broadband networked drill string. Digit Energy J 18:7–8

    Google Scholar 

  24. Jensen K, Hedengren J (2012) Improved load following of a boiler with advanced process control. In: AIChE spring meeting. Houston, TX

  25. Kelly J, Hedengren J (2013) A steady-state detection (SSD) algorithm to detect non-stationary drifts in processes. J Process Control 23(3):326–331

    Article  Google Scholar 

  26. Kelly J, Zyngier D (2008) Continuously improve the performance of planning and scheduling models with parameter feedback. In: FOCAPO 08—foundations of computer aided process operations. Boston, MA

  27. Lambert R, Nascu I, Pistikopoulos E (2013) Simultaneous reduced order multi-parametric moving horizon estimation and model predictive control. Dyn Control Process Syst 10(1):267–278

    Google Scholar 

  28. Lewis NR, Hedengren JD, Haseltine EL (2015) Hybrid dynamic optimization methods for systems biology with efficient sensitivities, Special Issue on Algorithms and Applications in Dynamic Optimization. Processes 3(3):701–729. doi:10.3390/pr3030701

  29. Liebman M, Edgar T, Lasdon L (1992) Efficient data reconciliation and estimation for dynamic processes using nonlinear programming techniques. Comput Chem Eng 16:963–986

    Article  Google Scholar 

  30. Long R, Veeningen D (2011) Networked drill pipe offers along-string pressure evaluation in real time. World Oil 232(9):91–94

    Google Scholar 

  31. Moraal P, Grizzle J (1995) Observer design for nonlinear systems with discrete-time measurements. IEEE Trans Autom Control 40(3):395–404

    MathSciNet  Article  MATH  Google Scholar 

  32. Muske KR, Badgwell TA (2002) Disturbance modeling for offset-free linear model predictive control. J Process Control 12:617–632

    Article  Google Scholar 

  33. Nybø R, Frøyen J, Lauvsnes AD, Korsvold T, Choate M (2012) The overlooked drilling hazard: Decision making from bad data. In: SPE intelligent energy international, SPE-150306. Society of Petroleum Engineers, Utrecht

  34. Odelson B, Rajamani M, Rawlings J (2006) A new autocovariance least-squares method for estimating noise covariances. Automatica 42(2):303–308

    MathSciNet  Article  MATH  Google Scholar 

  35. Pannocchia G, Kerrigan E (2003) Offset-free control of constrained linear discrete-time systems subject to persistent unmeasured disturbances. In: Proceedings of the 42nd IEEE conference on decision and control. Maui, Hawaii, pp 3911–3916

  36. Pannocchia G, Rawlings J (2002) Disturbance models for offset-free MPC control. AIChE J 49(2):426–437

    Article  Google Scholar 

  37. Pixton DS, Craig A (2014) Drillstring network 2.0: an enhanced drillstring network based on 100 wells of experience. In: IADC/SPE drilling conference and exhibition, SPE-167965-MS. Society of Petroleum Engineers, Fort Worth, TX. doi:10.2118/167965-MS

  38. Pixton DS, Shishavan RA, Perez HD, Hedengren JD, Craig A (2014) Addressing UBO and MPD challenges with wired drill pipe telemetry. In: SPE/IADC managed pressure drilling & underbalanced operations conference & exhibition, SPE-168953-MS. Society of Petroleum Engineers

  39. Powell KM, Hedengren JD, Edgar TF (2014) Dynamic optimization of a hybrid solar thermal and fossil fuel system. Sol Energy 108:210–218. doi:10.1016/j.solener.2014.07.004

    Article  Google Scholar 

  40. Prata DM, Lima EL, Pinto JC (2009) Nonlinear dynamic data reconciliation in real time in actual processes. In: do Nascimento CAO, de Brito Alves RM, Biscaia EC (eds) 10th international symposium on process systems engineering: part A, vol 27. Computer Aided Chemical Engineering, pp 47–54. doi:10.1016/S1570-7946(09)70228-7

  41. Qin S, Badgwell T (2000) Nonlinear model predictive control, chap. In: An overview of nonlinear model predictive control applications. Birkhäuser Verlag, Boston, pp. 369–392

  42. Ramamurthi Y, Sistu P, Bequette B (1993) Control-relevant dynamic data reconciliation and parameter estimation. Comput Chem Eng 17(1):41–59

    Article  Google Scholar 

  43. Ramlal J, Naidoo V, Allsford K, Hedengren J (2007) Moving horizon estimation for an industrial gas phase polymerization reactor. In: Proceedings of the IFAC symposium on nonlinear control systems design (NOLCOS). Pretoria, South Africa

  44. Rao C, Rawlings J, Lee J (2001) Constrained linear state estimation—a moving horizon approach. Automatica 37:1619–1628

    Article  MATH  Google Scholar 

  45. Rawlings J, Angeli D, Bates C (2012) Fundamentals of economic model predictive control. In: 2012 IEEE 51st annual conference on decision and control (CDC), pp 3851–3861. doi:10.1109/CDC.2012.6425822

  46. Rawlings J, Mayne D (2009) Model predictive control: theory and design. Nob Hill Publishing, LLC, Madison

    Google Scholar 

  47. Renfro J, Morshedi A, Asbjornsen O (1987) Simultaneous optimization and solution of systems described by differential/algebraic equations. Comput Chem Eng 11(5):503–517

    Article  Google Scholar 

  48. Safdarnejad SM, Hedengren JD, Baxter LL (2015) Plant-level dynamic optimization of cryogenic carbon capture with conventional and renewable power sources. Appl Energy 149:354–366. doi:10.1016/j.apenergy.2015.03.100

    Article  Google Scholar 

  49. Safdarnejad SM, Hedengren JD, Lewis NR, Haseltine E (2015) Initialization strategies for optimization of dynamic systems. Comput Chem Eng 78:39–50. doi:10.1016/j.compchemeng.2015.04.016

    Article  Google Scholar 

  50. Shishavan RA, Hubbell C, Perez H, Hedengren JD, Pixton DS (2015) Combined rate of penetration and pressure regulation for drilling optimization using high speed telemetry. SPE Drill Complet J (SPE-170275-MS). doi:10.2118/170275-PA

  51. Soderstrom T, Edgar T, Russo L, Young R (2000) Industrial application of a large-scale dynamic data reconciliation strategy. Ind Eng Chem Res 39:1683–1693

    Article  Google Scholar 

  52. Soroush M (1998) State and parameter estimations and their applications in process control. Comput Chem Eng 23:229–245

    Article  Google Scholar 

  53. Spivey B, Hedengren J, Edgar T (2010) Constrained nonlinear estimation for industrial process fouling. Ind Eng Chem Res 49(17):7824–7831

    Article  Google Scholar 

  54. Sugiura J, Samuel R, Oppelt J, Ostermeyer GP, Hedengren JD, Pastusek P (2015) Drilling modeling and simulation: current state and future goals. SPE/IADC-173045-MS. London, UK

  55. Sui D, Nybø R, Gola G, Roverso D, Hoffmann M (2011) Ensemble methods for process monitoring in oil and gas industry operations. J Nat Gas Sci Eng 3(6):748–753. doi:10.1016/j.jngse.2011.05.004 Artificial Intelligence and Data Mining

    Article  Google Scholar 

  56. Sun L, Hedengren JD, Beard RW (2014) Optimal trajectory generation using model predictive control for aerially towed cable systems. J Guidance Control Dyn 37(2):525–539

    Article  Google Scholar 

  57. Taylor J, del Pilar Moreno R (2013) Nonlinear dynamic data reconciliation: in-depth case study. In: 2013 IEEE international conference on control applications (CCA), pp. 746–753 (2013). doi:10.1109/CCA.2013.6662839

  58. Vachhani P, Rengaswamy R, Gangwal V, Narasimhan S (2005) Recursive estimation in constrained nonlinear dynamical systems. AIChE J 51(3):946–959

    Article  Google Scholar 

  59. Zavala V, Biegler L (2009) Nonlinear programming strategies for state estimation and model predictive control. In: Magni L, Raimondo D, Allgöwer F (eds) Nonlinear model predictive control, vol 384., Lecture notes in control and information sciences Springer, Berlin, pp 419–432

    Chapter  Google Scholar 

Download references


The authors would like to acknowledge the financial and technical assistance of National Oilwell Varco (NOV) and SINTEF in projects related to modeling and control design for automated drilling systems.

Author information



Corresponding author

Correspondence to John D. Hedengren.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hedengren, J.D., Eaton, A.N. Overview of estimation methods for industrial dynamic systems. Optim Eng 18, 155–178 (2017). https://doi.org/10.1007/s11081-015-9295-9

Download citation


  • Dynamic data reconciliation
  • Wired drillpipe
  • Industrial data
  • Moving horizon estimation
  • Kalman filter