Abstract
Direct Multisearch is a well-established class of algorithms, suited for multiobjective derivative-free optimization. In this work, we analyze the worst-case complexity of this class of methods in its most general formulation for unconstrained optimization. Considering nonconvex smooth functions, we show that to drive a given criticality measure below a specific positive threshold, Direct Multisearch takes at most a number of iterations proportional to the square of the inverse of the threshold, raised to the number of components of the objective function. This number is also proportional to the size of the set of linked sequences between the first unsuccessful iteration and the iteration immediately before the one where the criticality condition is satisfied. We then focus on a particular instance of Direct Multisearch, which considers a more strict criterion for accepting new nondominated points. In this case, we can establish a better worst-case complexity bound, simply proportional to the square of the inverse of the threshold, for driving the same criticality measure below the considered threshold.
Similar content being viewed by others
References
Marler, R.T., Arora, J.S.: Survey of multi-objective optimization methods for engineering. Struct. Multidiscip. Optim. 26, 369–395 (2004)
Ehrgott, M.: Multicriteria Optimization. Springer, Heidelberg (2005)
Custódio, A.L., Emmerich, M., Madeira, J.F.A.: Recent developments in derivative-free multiobjective optimization. Comput. Technol. Rev. 5, 1–30 (2012)
Kolda, T.G., Lewis, R.M., Torczon, V.: Optimization by direct search: new perspectives on some classical and modern methods. SIAM Rev. 45, 385–482 (2003)
Conn, A.R., Scheinberg, K., Vicente, L.N.: Introduction to Derivative-Free Optimization. MPS-SIAM Series on Optimization. SIAM, Philadelphia (2009)
Audet, C., Hare, W.: Derivative-Free and Blackbox Optimization. Springer Series in Operations Research and Financial Engineering. Springer, Cham (2017)
Custódio, A.L., Madeira, J.F.A., Vaz, A.I.F., Vicente, L.N.: Direct multisearch for multiobjective optimization. SIAM J. Optim. 21, 1109–1140 (2011)
Cartis, C., Gould, N.I.M., Toint, Ph.L.: On the oracle complexity of first-order and derivative-free algorithms for smooth nonconvex minimization. SIAM J. Optim. 22, 66–86 (2012)
Dodangeh, M., Vicente, L.N.: Worst case complexity of direct search under convexity. Math. Program. 155, 307–332 (2016)
Fliege, J., Vaz, A.I.F., Vicente, L.N.: Complexity of gradient descent for multiobjective optimization. Optim. Methods Softw. 34, 949–959 (2019)
Garmanjani, R., Vicente, L.N.: Smoothing and worst-case complexity for direct-search methods in nonsmooth optimization. IMA J. Numer. Anal. 33, 1008–1028 (2013)
Grapiglia, G.N., Yuan, J., Yuan, Y.: On the convergence and worst-case complexity of trust-region and regularization methods for unconstrained optimization. Math. Program. 152, 491–520 (2015)
Gratton, S., Royer, C.W., Vicente, L.N., Zhang, Z.: Direct search based on probabilistic descent. SIAM J. Optim. 25, 1515–1541 (2015)
Konečný, J., Richtárik, P.: Simple Complexity Analysis of Simplified Direct Search. Technical reports, arXiv:1410.0390 (2014)
Nesterov, Y.: Introductory Lectures on Convex Optimization. Applied Optimization. Kluwer Academic Publishers, Boston (2004)
Vicente, L.N.: Worst case complexity of direct search. EURO J. Comput. Optim. 1, 143–153 (2013)
Gratton, S., Sartenaer, A., Toint, Ph.L.: Recursive trust-region methods for multiscale nonlinear optimization. SIAM J. Optim. 19, 414–444 (2008)
Cartis, C., Sampaio, Ph.R., Toint, Ph.L.: Worst-case evaluation complexity of non-monotone gradient-related algorithms for unconstrained optimization. Optimization 64, 1349–1361 (2015)
Nesterov, Y., Polyak, B.T.: Cubic regularization of Newton method and its global performance. Math. Program. 108, 177–205 (2006)
Cartis, C., Gould, N.I.M., Toint, Ph.L.: Adaptive cubic regularisation methods for unconstrained optimization. Part II: worst-case function- and derivative-evaluation complexity. Math. Program. 130, 295–319 (2011)
Bergou, E.H., Gorbunov, E., Richtárik, P.: Stochastic three points method for unconstrained smooth optimization. SIAM J. Optim. 30, 2726–2749 (2020)
Calderón, L., Diniz-Ehrhardt, M.A., Martínez, J.M.: On high-order model regularization for multiobjective optimization. Optim. Methods Softw. (2020). https://doi.org/10.1080/10556788.2020.1719408
Žilinskas, A.: On the worst-case optimal multi-objective global optimization. Optim. Lett. 7, 1921–1928 (2013)
Calvin, J.M., Žilinskas, A.: On efficiency of a single variable bi-objective optimization algorithm. Optim. Lett. 14, 259–267 (2020)
Villacorta, K.D.V., Oliveira, P.R., Soubeyran, A.: A trust-region method for unconstrained multiobjective problems with applications in satisficing processes. J. Optim. Theory Appl. 160, 865–889 (2014)
Fliege, J., Svaiter, B.F.: Steepest descent methods for multicriteria optimization. Math. Methods Oper. Res. 51, 479–494 (2000)
Zitzler, E.: Evolutionary Algorithms for Multiobjective Optimization: Methods and Applications. Ph.D. Thesis, Swiss Federal Institute of Technology Zurich, Switzerland (1999)
Thomann, J., Eichfelder, G.: A trust-region algorithm for heteregeneous multiobjective optimization. SIAM J. Optim. 29, 1017–1047 (2019)
Dennis Jr., J.E., Woods, D.J.: Optimization on microcomputers: the Nelder–Mead simplex algorithm. In: Wouk, A. (ed.) New Computing Environments: Microcomputers in Large-Scale Computing, pp. 116–122. SIAM, Philadelphia (1987)
Liuzzi, G., Lucidi, S., Rinaldi, F.: A derivative-free approach to constrained multiobjective nonsmooth optimization. SIAM J. Optim. 26, 2744–2774 (2016)
Acknowledgements
The authors would like to thank the three anonymous referees, whose comments and suggestions much improved the quality of the paper. Funding support for Ana Luísa Custódio and Rohollah Garmanjani was provided by national funds through FCT–Fundação para a Ciência e a Tecnologia I. P., under the scope of Projects PTDC/MAT-APL/28400/2017 and UIDB/00297/2020. Support for Elisa Riccietti was provided by TOTAL E&P.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Xinmin Yang.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Custódio, A.L., Diouane, Y., Garmanjani, R. et al. Worst-Case Complexity Bounds of Directional Direct-Search Methods for Multiobjective Optimization. J Optim Theory Appl 188, 73–93 (2021). https://doi.org/10.1007/s10957-020-01781-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10957-020-01781-z
Keywords
- Multiobjective unconstrained optimization
- Derivative-free optimization methods
- Directional direct-search
- Worst-case complexity
- Nonconvex smooth optimization