Abstract
Surrogate-based optimization (SBO) is the main focus of this book. We provide a brief introduction to the subject in this chapter. In particular, we recall the SBO concept and the optimization flow, discuss the principles of surrogate modeling and typical approaches to construct surrogate models. We also discuss the distinction between function approximation (or data-driven) surrogates and physics-based surrogates, as well as outline the algorithm of SBO exploiting the two aforementioned classes of models. More detailed information about the selected types of SBO algorithms (especially those involving response correction techniques) as well as illustration and application examples in various fields of engineering are provided in the remaining part of the book.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Alba, E., and Marti, R., (Eds) (2006) Metaheuristic procedures for training neural networks, Springer.
Alexandrov, N.M., Dennis, J.E., Lewis, R.M., Torczon, V. (1998) A trust region framework for managing use of approximation models in optimization. Struct. Multidisciplinary Optim. 15, pp. 16–23.
Alexandrov, N.M., Lewis, R.M. (2001) An overview of first-order model management for engineering optimization. Optimization and Engineering. 2, 413–430.
Andrés, E., Salcedo-Sanz, S., Monge, F., Pérez-Bellido, A.M., (2012) Efficient aerodynamic design through evolutionary programming and support vector regression algorithms. Int. J. Expert Systems with Applications. 39, 10700–10708.
Angiulli, G., Cacciola, M., Versaci, M. (2007) Microwave devices and antennas modelling by support vector regression machines. IEEE Trans. Magnetics, 43, pp. 1589–1592.
Bakr, M.H., Bandler, J.W., Biernacki, R.M., Chen, S.H., Madsen, K. (1998) A trust region aggressive space mapping algorithm for EM optimization. IEEE Trans. Microwave Theory Tech., 46, pp. 2412–2425.
Bakr, M.H., Bandler, J.W., Georgieva, N.K., Madsen, K. (1999) A hybrid aggressive space-mapping algorithm for EM optimization. IEEE Trans. Microwave Theory Tech., 47, pp. 2440–2449.
Bakr, M.H., Bandler, J.W., and Georgieva, N. (1999) “Modeling of microwave circuits exploiting space derivative mapping,” IEEE MTT-S Int. Microwave Symp. Dig., (Anaheim, CA), pp. 715–718.
Bandler, J.W., Biernacki, R.M., Chen, S.H., Grobelny, P.A., Hemmers, R.H., (1994) Space mapping technique for electromagnetic optimization. IEEE Trans. Microwave Theory Tech. 42, 2536–2544.
Bandler, J.W., Biernacki, R.M., Chen, S.H., Hemmers, R.H., Madsen, K., (1995) Electromagentic optimization exploiting aggressive space mapping. IEEE Trans. Microwave Theory Tech., 43, 2874–2882.
Bandler, J.W., Cheng, Q.S., Gebre-Mariam, D.H., Madsen, K., Pedersen, F., Søndergaard, J. (2003) EM-based surrogate modeling and design exploiting implicit, frequency and output space mappings. IEEE Int. Microwave Symp. Digest, Philadelphia, PA, pp. 1003–1006.
Bandler, J.W., Cheng, Q.S., Dakroury, S.A., Mohamed, A.S., Bakr, M.H., Madsen, K., Søndergaard, J. (2004a) Space mapping: the state of the art. IEEE Trans. Microwave Theory Tech., 52, pp. 337−361.
Bandler, J.W., Cheng, Q.S., Nikolova, N.K., Ismail, M.A. (2004b) Implicit space mapping optimization exploiting preassigned parameters. IEEE Trans. Microwave Theory Tech., 52, pp. 378–385.
Bandler, J.W., Cheng, Q.S., Hailu, D.M., and Nikolova, N.K. (2004c) “A space-mapping design framework,” IEEE Trans. Microwave Theory Tech., vol. 52, no. 11, pp. 2601–2610.
Beachkofski, B., and Grandhi, R. (2002) Improved distributed hypercube sampling. American Institute of Aeronautics and Astronautics. Paper AIAA 2002–1274.
Booker, A.J., Dennis, J.E., Frank, P.D., Serafini, D.B., Torczon, V., Trosset, M.W. (1999) A rigorous framework for optimization of expensive functions by surrogates. Structural Optimization. 17, 1–13.
Broyden, C.G. (1965) A class of methods for solving nonlinear simultaneous equations. Math. Comp., 19, pp. 577–593.
Ceperic, V., Baric, A. (2004) Modeling of analog circuits by using support vector regression machines. Proc. 11 th Int. Conf. Electronics, Circuits, Syst., Tel-Aviv, Israel, pp. 391–394.
Cheng, Q.S., Koziel, S., and Bandler, J.W.(2006) Simplified space mapping approach to enhancement of microwave device models. Int. J. RF and Microwave Computer-Aided Eng., 16, 518–535.
Conn, A.R., Gould, N.I.M., Toint, P.L. (2000) Trust Region Methods. MPS-SIAM Series on Optimization.
Couckuyt, I., (2013) Forward and inverse surrogate modeling of computationally expensive problems. PhD Thesis, Ghent University.
Devabhaktuni, V.K., Yagoub, M.C.E., and Zhang, Q.J., (2001) A robust algorithm for automatic development of neural-network models for microwave applications. IEEE Trans. Microwave Theory Tech., 49, 2282-2291.
Echeverria, D., Hemker, P.W. (2005) Space mapping and defect correction. CMAM Int. Mathematical Journal Computational Methods in Applied Mathematics, 5, pp. 107–136.
Forrester, A.I.J., Sóbester, A., and Keane, A.J. (2007) Multi-Fidelity Optimization via Surrogate Modeling. Proceedings of the Royal Society A. 463, 3251-3269.
Forrester, A.I.J., Keane, A.J. (2009) Recent advances in surrogate-based optimization, Prog. in Aerospace Sciences, 45, pp. 50−79.
Geisser, S. (1993) Predictive Inference. Chapman and Hall.
Giunta, A.A., (1997) Aircraft multidisciplinary design optimization using design of experiments theory and response surface modeling methods. PhD Thesis, Virginia Polytechnic Institute and State University.
Giunta, A.A., Eldred, M.S., (2000) Implementation of a trust region model management strategy in the DAKOTA optimization toolkit. Proc. AIAA/USAF/NASA/ISSMO Symp. Multidisciplinary Analysis and Optimization, Long Beach, CA, AIAA-2000-4935.
Giunta, A.A., Wojtkiewicz, S.F., Eldred, M.S. (2003) Overview of modern design of experiments methods for computational simulations. American Institute of Aeronautics and Astronautics, paper AIAA 2003—0649.
Golub, G.H., Van Loan, Ch.F. (1996) Matrix Computations. (3rd ed.), The Johns Hopkins University Press.
Gorissen, D., Crombecq, K., Couckuyt, I., Dhaene, T., Demeester, P. (2010) A surrogate modeling and adaptive sampling toolbox for computer based design. Journal of Machine Learning Research, 11, pp. 2051-2055.
Gunn, S.R. (1998) Support vector machines for classification and regression. Technical Report. School of Electronics and Computer Science, University of Southampton.
Haykin, S. (1998) Neural Networks: A Comprehensive Foundation. 2nd ed. Prentice Hall.
Hosder, S., Watson, L.T., Grossman, B., Mason, W.H., Kim, H., (2001) Polynomial response surface approximations for the multidisciplinary design optimization of a high speed civil transport. Optimization and Engineering, 2, 431–452.
Huang, L., Gao, Z., (2012) Wing-body optimization based on multi-fidelity surrogate model. 28 th Int. Congress of the Aeronautical Sciences, Brisbane, Australia.
Jacobs, J.H., Etman, L.F.P., van Keulen, F., Rooda, J.E., (2004) Framework for sequential approximate optimization. Struct. Multidisc. Optimization, 27, 384–400.
Jones, D., Schonlau, M., Welch, W. (1998) Efficient global optimization of expensive black-box functions. Journal of Global Optimization. 13, pp. 455–492.
Journel, A.G., Huijbregts, Ch.J. (1981) Mining Geostatistics. Academic Press.
Kabir, H., Wang, Y., Yu, M., Zhang, Q.J. (2008). Neural network inverse modeling and applications to microwave filter design. IEEE Trans. Microwave Theory Tech., 56, pp. 867–879.
Kleijnen, J., (2008) Design and Analysis of Simulation Experiments. Springer.
Kleijnen, J.P.C. (2009) Kriging metamodeling in simulation: a review. European Journal of Operational Research. 192, 707–716.
Koehler, J.R., Owen, A.B. (1996) Computer experiments. In S. Ghosh and C. R. Rao (Eds.) Handbook of Statistics. Elsevier Science B.V. 13, pp. 261—308.
Kolda, T.G., Lewis, R.M., Torczon, V. (2003). Optimization by direct search: new perspectives on some classical and modern methods. SIAM Rev., 45, pp. 385−482.
Koziel, S., Bandler, J.W., Madsen, K. (2006) A space mapping framework for engineering optimization: theory and implementation. IEEE Trans. Microwave Theory Tech., 54. 3721-3730.
Koziel, S., Bandler, J.W., Madsen, K. (2006b) Theoretical justification of space-mapping-based modeling utilizing a data base and on-demand parameter extraction. IEEE Trans. Microwave Theory Tech., vol. 54, no. 12, pp. 4316-4322.
Koziel, S., and Bandler, J.W. (2007b) Space-mapping optimization with adaptive surrogate model. IEEE Trans. Microwave Theory Tech., 55, pp. 541–547.
Koziel, S., and Bandler, J.W. (2007d) A space-mapping approach to microwave device modeling exploiting fuzzy systems. IEEE Trans. Microwave Theory Tech., vol. 55, no. 12, pp. 2539–2547.
Koziel, S., Cheng, Q.S., Bandler, J.W. (2008) Space mapping. IEEE Microwave Magazine, 9, pp. 105–122.
Koziel, S., Bandler, J.W., Madsen, K. (2008b) Quality assessment of coarse models and surrogates for space mapping optimization. Optimization Eng. 9, 375–391.
Koziel, S., Bandler, J.W., Cheng, Q.S. (2010a) Robust trust-region space-mapping algorithms for microwave design optimization. IEEE Trans. Microwave Theory and Tech., 58, pp. 2166–2174.
Koziel, S., Cheng, Q.S., Bandler, J.W. (2010b) Implicit space mapping with adaptive selection of preassigned parameters. IET Microwaves, Antennas & Propagation, 4, pp. 361–373.
Koziel, S., Bandler, J.W., Cheng, Q.S. (2010c) Adaptively constrained parameter extraction for robust space mapping optimization of microwave circuits. IEEE MTT-S Int. Microwave Symp. Dig., 205–208.
Koziel, S., Echeverría-Ciaurri, D., Leifsson, L. (2011) Surrogate-based methods, in S. Koziel and X.S. Yang (Eds.) Computational Optimization, Methods and Algorithms, Series: Studies in Computational Intelligence, Springer-Verlag, pp. 33–60.
Koziel, S., Ogurtsov, S., Couckuyt, I., Dhaene, T. (2013b) Variable-fidelity electromagnetic simulations and co-kriging for accurate modeling of antennas. IEEE Trans. Antennas Prop., 61, pp. 1301–1308.
Koziel, S., Ogurtsov, S, Bandler, J.W., and Cheng, Q.S. (2013c) Reliable space mapping optimization integrated with EM-based adjoint sensitivities. IEEE Trans. Microwave Theory Tech., 61, pp. 3493–3502.
Koziel, S., and Leifsson, L. (2012e) Response correction techniques for surrogate-based design optimization of microwave structures. Int. J. RF and Microwave CAE, 22, 211–223.
Koziel, S., and Leifsson, L. (Eds.) (2013a) Surrogate-Based Modeling and Optimization. Applications in Engineering. Springer.
Koziel, S., and Leifsson, L. (2013b) Multi-level Airfoil Shape Optimization with Automated Low-fidelity Model Selection. Int. Conf. Comp. Science, Barcelona, Spain, June 5–7.
Koziel, S., and Leifsson, L. (2013c) Surrogate-Based Aerodynamic Shape Optimization by Variable-Resolution Models. AIAA Journal, vol. 51, no. 1, pp. 94–106.
Koziel, S., Ogurtsov, S. (2011a) Simulation-driven design in microwave engineering: application case studies, in Yang, X.S., and Koziel, S. (eds), Computational Optimization and Applications in Engineering and Industry, Series: Studies in Computational Intelligence, Springer-Verlag.
Koziel, S., Ogurtsov, S. (2012a) Model management for cost-efficient surrogate-based optimization of antennas using variable-fidelity electromagnetic simulations. IET Microwaves Ant. Prop., 6, pp. 1643–1650.
Koziel, S., and Ogurtsov, S. (2012b) Fast simulation-driven design of microwave structures using improved variable-fidelity optimization technique. Engineering Optimization, 44, pp. 1007–1019.
Koziel, S., and Ogurtsov, S. (2013a) Multi-level microwave design optimization with automated model fidelity adjustment. Int. J. RF and Microwave CAE.
Koziel, S., and Ogurtsov, S. (2013c) Multi-objective design of antennas using variable-fidelity simulations and surrogate models. IEEE Trans. Antennas Prop., 61, 5931–5939.
Koziel, S., Ogurtsov, S., Bandler, J.W., and Cheng, Q.S. (2013d) “Reliable space-mapping optimization integrated with EM-based adjoint sensitivities,” IEEE Trans. Microwave Theory Tech., vol. 61, no. 10, pp. 3493–3502.
Laurenceau, J., and Sagaut, P., (2008) Building efficient response surfaces of aerodynamic functions with kriging and cokriging. AIAA Journal, 46, 498–507.
Leary, S., Bhaskar, A., Keane, A. (2003) Optimal orthogonal-array-based latin hypercubes. Journal of Applied Statistics. 30, 585–598.
Leifsson, L., and Koziel, S. (2014) Variable-resolution shape optimization: low-fidelity model selection and scalability. Int. J. Mathematical Modeling and Numerical Optimization.
Leifsson, L., and Koziel, S. (2015) Simulation-driven aerodynamic design using variable-fidelity models. Imperial College Press.
Leifsson, L., and Koziel, S. (2015b) Variable-resolution shape optimization: Low-fidelity model selection and scalability. Int. J. Mathematical Modeling and Numerical Optimization, vol. 6, no. 1.
Leifsson, L., and Koziel, S. (2015c) Surrogate modeling and optimization using shape-preserving response prediction: A review. Engineering Optimization, May 6, pp. 1–21.
Levin, D. (1998) The approximation power of moving least-squares. Mathematics of Computation. 67, 1517–1531.
Li, Y.F., and Lan, C.C. (1989) Development of fuzzy algorithms for servo systems. IEEE Contr. Syst. Mag., vol. 9, no. 3, pp. 65–72.
Liu, J., Han, Z., Song, W., (2012) Comparison of infill sampling criteria in kriging-based aerodynamic optimization. 28 th Int. Congress of the Aeronautical Sciences, Brisbane, Australia.
Liu, B., Zhang, Q., and Gielen, G. (2014) A gaussian process surrogate model assisted evolutionary algorithm for medium scale expensive black box optimization problems. IEEE Trans. Evol. Comp., 18, pp. 180–192.
Lophaven, S.N., Nielsen, H.B., and Søndergaard, J., (2002) DACE: a Matlab kriging toolbox. Technical University of Denmark.
Marheineke, N., Pinnau, R., Reséndiz, E., (2012) Space mapping-focused control techniques for particle dispersions in fluids. Optimization and Engineering, 13, 101–120.
Marsden, A.L., Wang, M., Dennis, J.E., Moin, P. (2004) Optimal aeroacoustic shape design using the surrogate management framework. Optimization and Engineering. 5, 235–262.
McKay M, Conover W, Beckman R. (1979) A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics. 21, 239–245.
Meng, J., Xia, L. (2007) Support-vector regression model for millimeter wave transition. Int. J. Infrared and Millimeter Waves, 28, pp. 413–421.
Minsky, M.I., and Papert, S.A. (1969) Perceptrons: An Introduction to Computational Geometry. The MIT Press.
O'Hagan, A. (1978) Curve fitting and optimal design for predictions. Journal of the Royal Statistical Society B. 40, 1–42.
Palmer, K., Tsui, K.-L. (2001) A minimum bias latin hypercube design. IIE Transactions. 33, 793–808.
Passino, K.M., and Yurkovich, S. (1998) Fuzzy Control. Addison Wesley Longman Inc., Menlo Park, CA, USA.
Pérez, V.M., Renaud, J.E., Watson, L.T., (2002) Interior point sequential approximate optimization methodology. Proc. 9 th AIAA/ISSMO Symp. Multidisciplinary Analysis and Optimization, Atlanta, GA, AIAA-2002-5505.
Priess, M., Koziel, S., and Slawig, T., (2011) Surrogate-based optimization of climate model parameters using response correction. J. Comp. Science., 2, 335–344.
Tu, S., Cheng, Q.S., Zhang, Y., Bandler, J.W., Nikolova, N.K. (2013). Space mapping optimization of handset antennas exploiting thin-wire models, IEEE Trans. Ant. Prop., 61, 3797–3807.
Queipo, N.V., Haftka, R.T., Shyy, W., Goel, T., Vaidynathan, R., Tucker, P.K. (2005) Surrogate-based analysis and optimization. Progress in Aerospace Sciences, 41, pp. 1–28.
Rasmussen, C.E., Williams, C.K.I. (2006) Gaussian Processes for Machine Learning. MIT Press, Cambridge, Massachussets.
Rayas-Sanchez, J.E. (2004) EM-based optimization of microwave circuits using artificial neural networks: the state-of-the-art. IEEE Trans. Microwave Theory Tech. 52, 420–435.
Redhe, M., Nilsson, L. (2004) Optimization of the new Saab 9–3 exposed to impact load using a space mapping technique. Structural and Multidisciplinary Optimization. 27, 411–420.
Robinson, T.D., Eldred, M.S., Willcox, K.E., and Haimes, R., (2008) Surrogate-Based Optimization Using Multifidelity Models with Variable Parameterization and Corrected Space Mapping. AIAA Journal, 46, 2316–2326.
Rojo-Alvarez, J.L., Camps-Valls, G., Martinez-Ramon, M., Soria-Olivas, E., Navia-Vazquez, A., Figueiras-Vidal, A.R. (2005) Support vector machines framework for linear signal processing. Signal Processing, 85, pp. 2316–2326.
Roux, W.J., Stander, N., Haftka, R.T., (1998) Response surface approximations for structural optimization. Int. J. Numerical Methods in Engineering, 42, 517–534.
Salleh, M.K.M., Pringent, G., Pigaglio, O., and Crampagne, R. (2008) Quarter-wavelength side-coupled ring resonator for bandpass filters. IEEE Trans. Microwave Theory Tech., 56, pp. 156–162.
Sans, M., Selga, J., Rodriguez, A., Bonache, J., Boria, V.E., Martin, F. (2014) Design of planar wideband bandpass filters from specifications using a two-step aggressive space mapping (ASM) optimization algorithm. IEEE Trans. Microwave Theory Tech., 62, pp. 3341–3350.
Santner, T.J., Williams, B., Notz, W. (2003) The Design and Analysis of Computer Experiments. Springer-Verlag.
Shaker, G.S.A., Bakr, M.H., Sangary, N., Safavi-Naeini, S. (2009) Accelerated antenna design methodology exploiting parameterized Cauchy models. J. Progress in Electromagnetic Research (PIER B), 18, pp. 279–309.
Simpson, T.W., Maurey, T.M., Korte, J.J., and Mistree, F. (2001) Kriging models for global approximation in simulation-based multidisciplinary design optimization. AIAA Journal, vol. 39, no. 12, pp. 2233–2241.
Simpson, T.W., Peplinski, J., Koch, P.N., Allen, J.K. (2001) Metamodels for computer-based engineering design: survey and recommendations. Engineering with Computers, 17, pp. 129−150.
Smola, A.J., Schölkopf, B. (2004) A tutorial on support vector regression. Statistics and Computing, 14, pp. 199−222.
Sobieszczanski-Sobieski, J., Haftka, R.T., (1997). Multidisciplinary aerospace design optimization: survey of recent developments. Structural Optimization, 14, 1–23.
Søndergaard, J. (2003) Optimization using surrogate models – by the space mapping technique. Ph.D. Thesis, Informatics and Mathematical Modelling, Technical University of Denmark, Lyngby.
Tesfahunegn, Y.A., Koziel, S., and Leifsson, L. (2015) Surrogate-based airfoil design with multi-level optimization and adjoint sensitivities. 53rd AIAA Aerospace Sciences Meeting, Science and Technology Forum, Kissimee, Florida, Jan 5–9.
Toal, D.J.J., and Keane, A.J. (2011) Efficient Multipoint Aerodynamic Design Optimization via Cokriging. Journal of Aircraft. 48, 1685–1695.
Toropov, V.V., Filatov, A.A., Polynkin, A.A., (1993). Multiparameter structural optimization using FEM and multipoint explicit approximations. Struct. Optim., 6, 7–14.
Wang, L.-X., and Mendel, J.M. (1992) Generating fuzzy rules by learning from examples. IEEE Trans. Systems, Man, Cybernetics, vol. 22, no. 6, pp. 1414–1427, Nov./Dec. 1992.
Wild, S.M., Regis, R.G., Shoemaker, C.A. (2008) ORBIT: Optimization by radial basis function interpolation in trust-regions. SIAM J. Sci. Comput., 30, pp. 3197−3219.
Xia, L., Xu, R.M., Yan, B. (2007) Ltcc interconnect modeling by support vector regression. Progress In Electromagnetics Research, 69, pp. 67–75.
Yang, Y., Hu, S.M., Chen, R.S., (2005) A combination of FDTD and least-squares support vector machines for analysis of microwave integrated circuits. Microwave Opt. Technol. Lett., 44, 296–299.
Ye, K.Q., (1998) Orthogonal column latin hypercubes and their application in computer experiments. Journal of the American Statistical Association. 93, 1430–1439.
Zadeh, L.A. (1965) Fuzzy sets. Inform. Contr., vol. 8, no. 3, pp. 338–353.
Zhang, K., and Han, Z., (2013) Support vector regression-based multidisciplinary design optimization in aircraft conceptual design. AIAA Aerospace Sciences Meeting. AIAA paper 2013–1160.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Koziel, S., Leifsson, L. (2016). Introduction to Surrogate Modeling and Surrogate-Based Optimization. In: Simulation-Driven Design by Knowledge-Based Response Correction Techniques. Springer, Cham. https://doi.org/10.1007/978-3-319-30115-0_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-30115-0_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-30113-6
Online ISBN: 978-3-319-30115-0
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)