Journal of Global Optimization

, Volume 26, Issue 3, pp 297–319 | Cite as

A Knowledge-Based Approach To Response Surface Modelling in Multifidelity Optimization

  • Stephen J. Leary
  • Atul Bhaskar
  • Andy J. Keane


This paper is concerned with approximations for expensive function evaluation – the expensive functions arising in an engineering design context. The problem of reducing the computational cost of generating sufficient learning samples is addressed. Several approaches of using a priori knowledge to achieve computational economy are presented. In all these, the results of a cheap model are treated as knowledge to be incorporated in the training process. Several approaches are described here: in particular, we focus on neural based systems. This approach is then developed as a new knowledge-based kriging model which is shown to be as accurate as neural based alternatives while being much easier to train. Examples from the domain of structural optimization are given to demonstrate the approach.

Multifidelity modelling Knowledge-based neural networks Kriging Expensive function optimization 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bakr, M. H., Bandler, J. W., Madsen, K., Rayas-Sanchez, J. E. and Sondergaard, J. (2000a), Space-mapping optimization of microwave circuits exploiting surrogate models, IEEE Transactions on Microwave Theory and Techniques 48: 2297–2306.Google Scholar
  2. 2.
    Bakr, M. H., Bandler, J.W., Ismail, M. A., Rayas-Sanchez, J. E. and Zhang., Q. (2000b). Neiral space-mapping for EM-based design, IEEE Transactions on Microwave Theory and Techniques 48: 2307–2315.Google Scholar
  3. 3.
    Bandler, J. W., Ismail, M. A., Rayas-Sanchez, J. E. and Zhang., Q. (1999). Neuromodelling of microwave circuits exploiting space-mapping technology, IEEE Transactions on Microwave Theory and Techniques 47: 2417–2427.Google Scholar
  4. 4.
    Bandler, J. W., Georgieva, N., Ismail, M. A., Rayas-Sanchez, J. E. and Zhang., Q. (2001), A generalized space-mapping tableau approach to device modelling, IEEE Transactions on Microwave Theory and Techniques 49: 67–79.Google Scholar
  5. 5.
    Chang, K. J., Haftka, R. T., Giles, G. L. and Kao, P.-J. (1993). Sensitivity-based scaling for approximating structural response, Journal of Aircraft 30: 283–287.Google Scholar
  6. 6.
    Cressie, N. A. C. (1993). Statistics for Spatial Data, John Wiley and Sons Inc, New York.Google Scholar
  7. 7.
    Gibbs, M. N. and Mackay, D. J. C. (1997). Efficient implementation of Gaussian processes, Scholar
  8. 8.
    Haftka, R. T. (1991). Combining global and local approximations, AIAA Journal 29: 1523–1525.Google Scholar
  9. 9.
    Hutchinson, M. G., Unger, E. R., Mason, W. H., Grossman, B. and Haftka, R. T. (1994). Variable-complexity aerodynamic optimization of a high speed civil transport wing, Journal of Aircraft 31: 110–116.Google Scholar
  10. 10.
    Jones, D. R., Schonlau, M. and Welch, W. J., (1998). Efficient global optimization of expensive black-box functions, Journal of Global Optimization 13: 455–492.Google Scholar
  11. 11.
    Myers, R. H. and Montgomery, D. C. (1995). Response surface methodology: Process and product optimization using designed experiments, John Wiley and Sons inc, New York.Google Scholar
  12. 12.
    Press, W. H., Teukolsky, S. A., Vellering, W. T. and Flannery, B. P. (1992). Numerical recipes in fortran, 2nd Edition, Cambridge University Press, Cambridge.Google Scholar
  13. 13.
    Rasmussen, C. E. (1996). Evaluation of Gaussian processes and other methods for nonlinear regression, PhD Thesis, University of Toronto.Google Scholar
  14. 14.
    Rumelhart, D. E., Hinton, G. E. and Williams, R. J. (1986). Learning representations by backpropagating errors, Nature 323: 533–536.Google Scholar
  15. 15.
    Sacks, J., Welch, W. J., Mitchell, T. J. and Wynn, H. P. (1989). Design and Analysis of Computer Experiments, Statistical Science 4: 409–435.Google Scholar
  16. 16.
    Towell, G. G. and Shavlik, J. W. (1994). Knowledge-based artificial neural networks, Artificial Intelligence 70: 119–165.Google Scholar
  17. 17.
    Vitali, R., Haftka, R. T. and Sankar, B. V. (1999). Multifidelity design of a stiffened composite panel with a crack, 4th World Congress of Structural and Multidisciplinary Optimization, Buffalo, NY.Google Scholar
  18. 18.
    Wang, F. and Zhang, Q. (1997). Knowledge-based neural models for microwave design, IEEE Transactions on Microwave Theory and Techniques 45: 2333–2343.Google Scholar
  19. 19.
    Watson, P. M. and Gupta, K. C. (1996). EM-ANN models for microstrip vias and interconnects in dataset circuits, IEEE Transactions on Microwave Theory and Techniques 44: 2495–2503.Google Scholar
  20. 20.
    White, H., Gallant, A. R., Kornik, K., Stinchcombe, M. and Wooldridge, J. (1992). Artificial Neural Networks: Approximation and Learning Theory, Blackwell publishers, Oxford.Google Scholar
  21. 21.
    Zhu, C., Byrd R. H., Lu P. and Nocedal J. (1994). L-BFGS-B: a limited memory FORTRAN code for solving bound constrained optimization problems, Tech. Report, NAM-11, EECS Department, Northwestern University.Google Scholar

Copyright information

© Kluwer Academic Publishers 2003

Authors and Affiliations

  • Stephen J. Leary
    • 1
  • Atul Bhaskar
    • 1
  • Andy J. Keane
    • 1
  1. 1.Computational Engineering and Design Centre, School of Engineering SciencesThe University of SouthamptonHighfield, SouthamptonUK

Personalised recommendations