Skip to main content
SpringerLink
Log in

Toward an optimal ensemble of kernel-based approximations with engineering applications

  • Research Paper
  • Published:
Structural and Multidisciplinary Optimization Aims and scope Submit manuscript

Abstract

This paper presents a general approach toward the optimal selection and ensemble (weighted average) of kernel-based approximations to address the issue of model selection. That is, depending on the problem under consideration and loss function, a particular modeling scheme may outperform the others, and, in general, it is not known a priori which one should be selected. The surrogates for the ensemble are chosen based on their performance, favoring non-dominated models, while the weights are adaptive and inversely proportional to estimates of the local prediction variance of the individual surrogates. Using both well-known analytical test functions and, in the surrogate-based modeling of a field scale alkali-surfactant-polymer enhanced oil recovery process, the ensemble of surrogates, in general, outperformed the best individual surrogate and provided among the best predictions throughout the domains of interest.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Balabanov VO, Haftka RT, Grossman B, Mason WH, Watson LT (1998) Multidisciplinary response model for HSCT wing bending material weight. In: 7th AIAA/ USAF/NASA/ISSMO symposium on multidisciplinary analysis and optimization, St. Louis, MO, AIAA Paper 98-4804

  • Bishop C (1995) Neural networks for pattern recognition. Oxford University Press, Oxford

    Google Scholar 

  • Bourrel M, Salager JL, Schechter RS, Wade WH (1980) A correlation for phase behavior of nonionic surfactants. J Colloid Interface Sci 75(2):451–461

    Article  Google Scholar 

  • Buckland ST, Burnham KP, Augustin NH (1997) Model selection: an integral part of inference. Biometrics 53:275–290

    Google Scholar 

  • Carrero E, Queipo N, Pintos S, Zerpa L (2007) Global sensibility analysis of asp enhanced oil recovery process. J Pet Sci Eng doi:10.1016/j.petrol.2006.11.007

  • Cherkassky V, Ma Y (2003) Comparison of model selection for regression. Neural Comput 15:1691–1714

    Article  MATH  Google Scholar 

  • Cherkassky V, Ma Y (2004) Practical selection of SVM parameters and noise estimation for SVM regression. Neural Netw 17(1):113–126

    Article  MATH  Google Scholar 

  • Cherkassky V, Shao X, Mulier F, Vapnik V (1999) Model complexity control for regression using VC generalization bounds. IEEE Trans Neural Netw 10(5):1075–1089

    Article  Google Scholar 

  • Clarke SM, Griebsch JH, Simpson TW (2005) Analysis of support vector regression for approximation of complex engineering analyses. ASME J Mech Des 127(6):1077–1087

    Article  Google Scholar 

  • Craig KJ, Stander N, Dooge DA, Varadappa S (2002) MDO of automotive vehicles for crashworthiness using response surface methods. In: 9th AIAA/ISSMO symposium on multidisciplinary analysis and optimization, Atlanta, GA, AIAA Paper 2002-5607

  • Dosher TM, Wise FA (1976) Enhanced oil recovery potential: an estimate. J Pet Technol, p 575, Paper SPE 5800

  • Engelsen S, Lake LW, Lin EC, Ohno T, Pope GA, Camilleri D, Sepehrnoori K (1987) Description of an improved compositional micellar/polymer simulator. Paper SPE 13967, SPE Reserv Eng 427–432

  • Girosi F (1998) An equivalence between sparse approximation and support vector machines. Neural Comput 10(6):1455–1480

    Article  Google Scholar 

  • Giunta AA, Balabanov V, Grossman B, Burgee S, Haftka RT, Mason WH, Watson LT (1997) Multidisciplinary optimization of a supersonic transport using design of experiments theory and response surface modelling. Aeronaut J 101(1008):347–356

    Google Scholar 

  • Goel T, Haftka RT, Shyy W, Queipo NV (2007) Ensemble of surrogates. Struct Multidisc Optim 33(3):199–216

    Article  Google Scholar 

  • Gunn SR (1998) Matlab support vector machine toolbox. http://www.isis.ecs.soton.ac.uk/isystems/kernel (Mar)

  • Hernández C, Chacón L, Anselmi L, Baldonedo A, Qi J, Phillip C, Pitts MJ (2001) ASP system design for an offshore application in the La Salina Field, Lake Maracaibo. In: SPE latin american and caribbean petroleum engineering conference, Buenos Aires, Argentina, Paper SPE 69544, March 2001

  • Hoeting J, Madigan D, Raftery A, Volinsky CT (1999) Bayesian model averaging: a tutorial. Stat Sci 14(4):382–417

    Article  MATH  MathSciNet  Google Scholar 

  • Jin R, Chen W, Simpson TW (2001) Comparative studies of metamodelling techniques under multiple modelling criteria. Struct Multidisc Optim 23(1):1–13

    Article  Google Scholar 

  • Kass RE, Raftery AE (1995) Bayes factors. J Am Stat Assoc 90:773–795

    Article  MATH  Google Scholar 

  • Krogh A, Sollich P (1997) Statistical Mechanics of Ensemble Learning. Phys Rev E 55(1):811–825

    Article  Google Scholar 

  • Kurtaran H, Eskandarian A, Marzougui D, Bedewi NE (2002) Crashworthiness design optimization using successive response surface approximations. Comput Mech 29:409–421

    Article  MATH  Google Scholar 

  • Lake LW (1989) Enhanced oil recovery. Prentice Hall, Englewood Cliffs, NJ

    Google Scholar 

  • Lake LW, Bhuyan D, Pope GA (1990) Mathematical modelling of high-ph chemical flooding. SPE Paper 17398, SPE Reserv Eng 213–220

  • Li W, Padula S (2004) Approximation methods for conceptual design of complex systems. In: Schumaker L, Chui C, Neaumtu M (eds) Eleventh international conference on approximation theory, May 2004

  • Madigan D, Raftery AE (1994) Model selection and accounting for model uncertainty in graphic models using Occam’s window. J Am Stat Assoc 89:1535–1546

    Article  MATH  Google Scholar 

  • Manrique E, De Carvajal G, Anselmi L, Romero C, Chacón L (2000) Alkali/Surfactant/Polymer at VLA 6/9/21 field in Maracaibo Lake: experimental results and pilot project design. In: SPE/DOE improved oil recovery symposium, Tulsa, OK, Paper SPE 59363, April 2000

  • Martin JD, Simpson TW (2005) On the use of kriging models to approximate deterministic computer models. AIAA J 43(4):853–863

    Article  Google Scholar 

  • Müller K, Mika S, Rätsch G, Tsuda K, Schölkopf B (2001) An introduction to kernel-based learning algorithms. IEEE Trans Neural Netw 12(2):181–201

    Article  Google Scholar 

  • Perrone M (1994) General averaging results for convex optimization. In: Mozer MC et al. (ed) Proceedings of the 1993 connectionist models summer school. Erlbaum, Hillsdale, NJ

    Google Scholar 

  • Perrone M, Cooper L (1993) When networks disagree: ensemble methods for hybrid neural networks. In: Mammone RJ (ed) Artificial neural networks for speech and vision. Chapman & Hall, FL, pp 126–142

    Google Scholar 

  • Poggio T, Smale S (2003) The mathematics of learning: dealing with data. Not Am Math Soc 50(5):537–544

    MATH  MathSciNet  Google Scholar 

  • Pope GA, Nelson RC (1978) A chemical flooding compositional simulator. Paper SPE 6725, Soc Pet Eng J, p 18

  • Qi Q, Hongjun G, Dongwen L, Ling D (2000) The pilot test of ASP combination flooding in Karamay oil field. In: SPE international oil and gas conference an exhibition, Beijing, China, Paper SPE 64726, November 2000

  • Queipo NV, Haftka R, Shyy W, Goel T, Vaidyanathan R, Kevin Tucker P (2005) Surrogate-based analysis and optimization. J Progress Aerospace Sci 41:1–28

    Article  Google Scholar 

  • Queipo NV, Goicochea J, Pintos S (2002) Surrogate modeling-based optimization of SAGD processes. J Pet Sci Eng 35 (1–2):83–93

    Article  Google Scholar 

  • Queipo NV, Verde A, Canelón J, Pintos S (2002) Efficient global optimization of hydraulic fracturing designs. J Pet Sci Eng 35(3–4):151–166

    Article  Google Scholar 

  • Salager JL (1996) Quantifying the concept of physico-chemical formulation in surfactant–oil–water systems-state of the art. Progr Colloid Polym Sci 100:137–142

    Article  Google Scholar 

  • Salager JL, Bourrel M, Schechter RS, Wade WH (1979a) Mixing rules for optimum phase behavior formulation of surfactant–oil–water systems. Soc Pet Eng J 19:271–278

    Google Scholar 

  • Salager JL, Morgan J, Schechter RS, Wade WH, Vasquez E (1979b) Optimum formulation of surfactant-oil-water systems for minimum tension and phase behavior. Soc Pet Eng J 19:107–115

    Google Scholar 

  • Schölkopf B, Smola AJ (2002) Learning with kernels. MIT Press, Cambridge, MA

    Google Scholar 

  • Simpson TW, Peplinski JD, Koch PN, Allen JK (2001) Metamodels for computer based engineering design: survey and recommendations. Eng Comput 17(2):129–150

    Article  MATH  Google Scholar 

  • UTCHEM-9.0. (2000) Utchem-9.0 a three-dimensional chemical flood simulator. http://www.cpge.utexas.edu/utchem/ (Jul)

  • Vapnik V (1998) Statistical learning theory. Wiley, New York

    MATH  Google Scholar 

  • Wahba G (2000) An introduction to model building with reproducing kernel Hilbert spaces. Technical Report 1020, University of Wisconsin-Madison Statistics Department (Apr)

  • Wei-Ju W (1996) Optimum design of field-scale chemical flooding using reservoir simulation. PhD thesis, The University of Texas at Austin, TX

  • Zerpa L, Queipo NV, Pintos S, Salager J (2005) An optimization methodology of alkaline-surfactant-polymer flooding processes using field scale numerical simulation and multiple surrogates. J Pet Sci Eng 47:197–208

    Article  Google Scholar 

  • Zhijian Q, Yigen Z, Xiansong Z, Jialin D (1998) A successful ASP flooding in gudong oil field. In: SPE/DOE Improved Oil Recovery Symposium, Tulsa, OK, Paper SPE 39613, April 1998

Download references

Author information

Authors and Affiliations

  1. Applied Computing Institute, Faculty of Engineering, University of Zulia, Maracaibo, 4011, Venezuela

    Egar Sanchez, Salvador Pintos & Nestor V. Queipo

Authors
  1. Egar Sanchez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Salvador Pintos
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nestor V. Queipo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nestor V. Queipo.

Additional information

This work was supported in part by the Fondo Nacional de Ciencia, Tecnología e Innovación (FONACIT), Venezuela under Grant F-2005000210. N. Q. Author also acknowledges that this material is based upon work supported by National Science Foundation under Grant DDM-423280.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sanchez, E., Pintos, S. & Queipo, N.V. Toward an optimal ensemble of kernel-based approximations with engineering applications. Struct Multidisc Optim 36, 247–261 (2008). https://doi.org/10.1007/s00158-007-0159-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00158-007-0159-6

Keywords

Search

Navigation