The application of gradient-only optimization methods for problems discretized using non-constant methods

  • Daniel N. Wilke
  • Schalk Kok
  • Albert A. Groenwold
Research Paper


We study the minimization of objective functions containing non-physical jump discontinuities. These discontinuities arise when (partial) differential equations are discretized using non-constant methods and the resulting numerical solutions are used in computing the objective function. Although the functions may become discontinuous, gradient information may be computed at every point. Gradient information is computable everywhere since every point has an associated discretization for which (semi) analytical sensitivities can be calculated. Rather than the construction of global approximations using only function value information to overcome the discontinuities, we propose to use only the gradient information. We elaborate on the modifications of classical gradient based optimization algorithms for use in gradient-only approaches, and we then present gradient-only optimization strategies using both BFGS and a new spherical quadratic approximation for sequential approximate optimization (SAO). We then use the BFGS and SAO algorithms to solve three problems of practical interest, both unconstrained and constrained.


Gradient-only optimization Step discontinuity Partial differential equation Non-constant discretization 


  1. Allaire G, Jouve F, Toader A-M (2004) Structural optimization using sensitivity analysis and a level-set method. J Comput Phys 194(1):363–393zbMATHCrossRefMathSciNetGoogle Scholar
  2. Barthelemy J-FM, Haftka RT (1993) Approximation concepts for optimum structural design—a review. Struct Optim 5:129–144CrossRefGoogle Scholar
  3. Bazaraa MS, Sherali HD, Shetty CM (1993) Nonlinear programming—theory and algorithms, 2nd edn. Wiley, New YorkzbMATHGoogle Scholar
  4. Brandstatter BR, Ring W, Magele Ch, Richter KR (1998) Shape design with great geometrical deformations using continuously moving finite element nodes. IEEE Trans Magn 34(5):2877–2880CrossRefGoogle Scholar
  5. Forrester AIJ, Keane AJ (2009) Recent advances in surrogate-based optimization. Prog Aerosp Sci 45(1–3):50–79CrossRefGoogle Scholar
  6. Garcia MJ, Gonzalez CA (2004) Shape optimisation of continuum structures via evolution strategies and fixed grid finite element analysis. Struct Multidisc Optim V26(1):92–98CrossRefMathSciNetGoogle Scholar
  7. Gould N, Orban D, Toint P (2005) Numerical methods for Large-Scale nonlinear optimization. Acta Numer 14(1):299–361zbMATHCrossRefMathSciNetGoogle Scholar
  8. Groenwold AA, Etman LFP, Snyman JA, Rooda JE (2007) Incomplete series expansion for function approximation. Struct Multidisc Optim 34:21–40CrossRefMathSciNetGoogle Scholar
  9. Haftka RT, Gürdal Z (1991) Elements of structural optimization, solid mechanics and its applications, 3rd edn, vol 11. Kluwer Academic, DordrechtGoogle Scholar
  10. Kocks UF (1976) Laws for work-hardening and low-temperature creep. J Eng Mater Technol Trans ASME 98 Ser H(1):76–85Google Scholar
  11. Kodiyalam S, Thanedar PB (1993) Some practical aspects of shape optimization and its influence on intermediate mesh refinement. Finite Elem Anal Des 15(2):125–133CrossRefGoogle Scholar
  12. Kok S, Beaudoin AJ, Tortorelli DA (2002) On the development of stage IV hardening using a model based on the mechanical threshold. Acta Mater 50(7):1653–1667CrossRefGoogle Scholar
  13. Lui DC, Nocedal J (1989) On the limited memory BFGS method for large scale optimization. Math Program 54(1–3):503–528Google Scholar
  14. Olhoff N, Rasmussen J, Lund E (1993) A method of exact numerical differentiation for error elimination in finite element based semi-analytical shape sensitivity analysis. Mechan Struct Mach 21:1–66CrossRefMathSciNetGoogle Scholar
  15. Persson P-O, Strang G (2004) A simple mesh generator in matlab. SIAM Rev 46(2):329–345zbMATHCrossRefMathSciNetGoogle Scholar
  16. Potra FA, Shi Y (1995) Efficient line search algorithm for unconstrained optimization. J Optim Theory Appl 85(3):677–704zbMATHCrossRefMathSciNetGoogle Scholar
  17. Quapp W (1996) A gradient-only algorithm for tracing a reaction path uphill to the saddle of a potential energy surface. Chem Phys Lett 253(3–4):286–292CrossRefGoogle Scholar
  18. Sacks J, Welch WJ, Mitchell TJ, Wynn HP (1989) Design and analysis of computer experiments. Stat Sci 4:409–435zbMATHCrossRefMathSciNetGoogle Scholar
  19. Schleupen A, Maute K, Ramm E (2000) Adaptive FE-procedures in shape optimization. Struct Multidisc Optim 19(4):282302CrossRefGoogle Scholar
  20. Shor NZ, Kiwiel KC, Ruszcaynski A (1985) Minimization methods for non-differentiable functions. Springer, New YorkzbMATHGoogle Scholar
  21. Simpson TW, Toropov V, Balabanov V, Viana FAC (2008) Design and analysis of computer experiments in multidisciplinary design optimization: a review of how far we have come—or not. In: Proc 12th AIAA/ISSMO multidisciplinary analysis and optimization conference, Victoria, 10–12 September 2008Google Scholar
  22. Snyman JA (1982) A new and dynamic method for unconstrained minimization. Appl Math Model 6(6):449–462zbMATHCrossRefMathSciNetGoogle Scholar
  23. Snyman JA (2005a) A gradient-only line search method for the conjugate gradient method applied to constrained optimization problems with severe noise in the objective function. Int J Numer Methods Eng 62(1):72–82zbMATHCrossRefMathSciNetGoogle Scholar
  24. Snyman JA (2005b) Practical mathematical optimization: an introduction to basic optimization theory and classical and new gradient-based algorithms. Applied optimization, 2nd edn, vol 97. Springer, New YorkGoogle Scholar
  25. Snyman JA, Hay AM (2001) The spherical quadratic steepest descent (SQSD) method for unconstrained minimization with no explicit line searches. Comput Math Appl 42(1–2):169–178zbMATHMathSciNetGoogle Scholar
  26. Snyman JA, Hay AM (2002) The dynamic-Q optimization method: an alternative to SQP? Comput Math Appl 44(12):1589–1598zbMATHCrossRefMathSciNetGoogle Scholar
  27. Svanberg K (2002) A class of globally convergent optimization methods based on conservative convex separable approximations. SIAM J Optim 12:555–573zbMATHCrossRefMathSciNetGoogle Scholar
  28. Toropov VV (1989) Simulation approach to structural optimization. Struct Optim 1:37–46CrossRefGoogle Scholar
  29. Van Miegroet L, Mos N, Fleury C, Duysinx P (2005) Generalized shape optimization based on the level set method. In: 6th world congresses of structural and multidisciplinary optimization. International Society for Structural and Multidisciplinary Optimization, Daejeon, pp 1–10Google Scholar
  30. Voce E (1955) A practical strain-hardening function. Metallurgica 51:219–226Google Scholar
  31. Wallis J (1685) A treatise of algebra, both historical and practical. LondonGoogle Scholar
  32. Wilke DN, Kok S, Groenwold AA (2006) A quadratically convergent unstructured remeshing strategy for shape optimization. Int J Numer Methods Eng 65(1):1–17zbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2009

Authors and Affiliations

  • Daniel N. Wilke
    • 1
  • Schalk Kok
    • 1
  • Albert A. Groenwold
    • 2
  1. 1.Department of Mechanical and Aeronautical EngineeringUniversity of PretoriaPretoriaSouth Africa
  2. 2.Department of Mechanical EngineeringUniversity of StellenboschStellenboschSouth Africa

Personalised recommendations