Advertisement

Benchmarking and Evaluating MATLAB Derivative-Free Optimisers for Single-Objective Applications

  • Lin Li
  • Yi Chen
  • Qunfeng Liu
  • Jasmina Lazic
  • Wuqiao Luo
  • Yun Li
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10362)

Abstract

MATLAB® builds in a number of derivative-free optimisers (DFOs), conveniently providing tools beyond conventional optimisation means. However, with the increase of available DFOs and being compounded by the fact that DFOs are often problem dependent and parameter sensitive, it has become challenging to determine which one would be most suited to the application at hand, but there exist no comparisons on MATLAB DFOs so far. In order to help engineers use MATLAB for their applications without needing to learn DFOs in detail, this paper evaluates the performance of all seven DFOs in MATLAB and sets out an amalgamated benchmark of multiple benchmarks. The DFOs include four heuristic algorithms - simulated annealing, particle swarm optimization (PSO), the genetic algorithm (GA), and the genetic algorithm with elitism (GAe), and three direct-search algorithms - Nelder-Mead’s simplex search, pattern search (PS) and Powell’s conjugate search. The five benchmarks presented in this paper exceed those that have been reported in the literature. Four benchmark problems widely adopted in assessing evolutionary algorithms are employed. Under MATLAB’s default settings, it is found that the numerical optimisers Powell is the aggregative best on the unimodal Quadratic Problem, PSO on the lower dimensional Scaffer Problem, PS on the lower dimensional Composition Problem, while the extra-numerical genotype GAe is the best on the Varying Landscape Problem and on the other two higher dimensional problems. Overall, the GAe offers the highest performance, followed by PSO and Powell. The amalgamated benchmark quantifies the advantage and robustness of heuristic and population-based optimisers (GAe and PSO), especially on multimodal problems.

Keywords

Evolutionary algorithms Heuristic search Direct search methods Derivative-free optimisation Benchmarking 

References

  1. 1.
    Lo, V.M.: Heuristic algorithm for task assignment in distributed systems. IEEE Trans. Comput. 37(11), 1384–1397 (1988)MathSciNetCrossRefGoogle Scholar
  2. 2.
    A. R. Conn, K. Scheinberg and L. N. Vicente. Introduction to Derivative-Free Optimization, SIAM (2009)Google Scholar
  3. 3.
    Powell, M.J.D.: Direct search algorithms for optimisation calculations. Acta Numer. 7, 287–336 (1998)CrossRefzbMATHGoogle Scholar
  4. 4.
  5. 5.
    Feng, W., Brune, T., Chan, L., Chowdhury, M., Kuek, C.K., Li, Y.: Benchmarks for testing evolutionary algorithms. In: Asia-Pacific Conference on Control and Measurement, pp. 134–138 (1998)Google Scholar
  6. 6.
    Luo, W., Li, Y.: Benchmarking heuristic search and optimisation algorithms in matlab. In: 22th International Conference on Automation & Computing, Colchester city, UK, 7 September 2016Google Scholar
  7. 7.
    Chen, Q., Liu, B., Zhang, Q., Liang, J., Suganthan, P., Qu, B.: Problem definitions and evaluation criteria for CEC 2015 special session on bound constrained single-objective computationally expensive numerical optimization. In: 2015 IEEE Congress on Evolutionary Computation, Sendai, Japan, 25 May (2015)Google Scholar
  8. 8.
    Liang, J., Qu, B., Suganthan, P.: Problem definitions and evaluation criteria for the CEC 2014 special session and competition on single objective real-parameter numerical optimization. In: 2013 IEEE Congress on Evolutionary Computation, Cancun, Mexico, 21 June (2013)Google Scholar
  9. 9.
    Michelewicz, Z.: Genetic algorithm + data structure = evolutionary programs, vol. 1, p. 996. Springer-Verlag, New York (1996)Google Scholar
  10. 10.
    Renders, J.-M., Bersini, H.: Hybridizing genetic algorithms with hill-climbing methods for global optimisation: two possible ways. In: 1994 IEEE World Congress on Computational Intelligence, Florida, USA, 26 June 1994Google Scholar
  11. 11.
    Zhan, Z.-H., Zhang, J., Li, Y., Chung, H.S.-H.: Adaptive particle swarm optimisation. IEEE Trans. Syst. Man Cybernet. Part B: Cybernet. 39(6), 1362–1381 (2009)CrossRefGoogle Scholar
  12. 12.
    Yao, X., Liu, Y., Lin, G.: Evolutionary programming made faster. IEEE Trans. Evol. Comput. 3(2), 82–102 (1999)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Lin Li
    • 1
  • Yi Chen
    • 1
  • Qunfeng Liu
    • 1
  • Jasmina Lazic
    • 2
  • Wuqiao Luo
    • 3
  • Yun Li
    • 1
    • 3
  1. 1.School of Computer Science and Network SecurityDongguan University of TechnologyDongguanChina
  2. 2.MathWorks, Matrix HouseCambridgeUK
  3. 3.School of EngineeringUniversity of GlasgowGlasgowUK

Personalised recommendations