Optimization Letters

, Volume 10, Issue 4, pp 699–708 | Cite as

On the optimal order of worst case complexity of direct search

Original Paper

Abstract

The worst case complexity of direct-search methods has been recently analyzed when they use positive spanning sets and impose a sufficient decrease condition to accept new iterates. For smooth unconstrained optimization, it is now known that such methods require at most \(\mathcal {O}(n^2\epsilon ^{-2})\) function evaluations to compute a gradient of norm below \(\epsilon \in (0,1)\), where n is the dimension of the problem. Such a maximal effort is reduced to \(\mathcal {O}(n^2\epsilon ^{-1})\) if the function is convex. The factor \(n^2\) has been derived using the positive spanning set formed by the coordinate vectors and their negatives at all iterations. In this paper, we prove that such a factor of \(n^2\) is optimal in these worst case complexity bounds, in the sense that no other positive spanning set will yield a better order of n. The proof is based on an observation that reveals the connection between cosine measure in positive spanning and sphere covering.

Keywords

Direct search Worst case complexity Optimal order Sphere covering Positive spanning set Cosine measure 

References

  1. 1.
    Bárány, I., Füredi, Z.: Approximation of the sphere by polytopes having few vertices. Proc. Am. Math. Soc. 102, 651–659 (1988)MathSciNetCrossRefMATHGoogle Scholar
  2. 2.
    Böröczky Jr, K.: Finite Packing and Covering. Cambridge University Press, New York (2004)CrossRefMATHGoogle Scholar
  3. 3.
    Böröczky Jr, K.: April 25, 2014 (private communication)Google Scholar
  4. 4.
    Böröczky Jr, K., Wintsche, G.: Covering the sphere by equal spherical balls. In: Aronov, B., Basu, S., Pach, J., Sharir, M. (eds.) Discrete and Computational Geometry. Algorithms and Combinatorics, vol. 25, pp. 235–251. Springer, Berlin (2003)CrossRefGoogle Scholar
  5. 5.
    Bourgain, J., Lindenstrauss, J., Milman, V.: Approximation of zonoids by zonotopes. Acta Math. 162, 73–141 (1989)MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    Carl, B., Pajor, A.: Gelfand numbers of operators with values in a Hilbert space. Invent. Math. 94, 479–504 (1988)MathSciNetCrossRefMATHGoogle Scholar
  7. 7.
    Conn, A.R., Scheinberg, K., Vicente, L.N.: Introduction to Derivative-Free Optimization. MPS-SIAM Series on Optimization. SIAM, Philadelphia (2009)CrossRefMATHGoogle Scholar
  8. 8.
    Dalla, L., Larman, D.G., Mani-Levitska, P., Zong, C.: The blocking numbers of convex bodies. Discret. Comput. Geom. 24, 267–278 (2000)MathSciNetCrossRefMATHGoogle Scholar
  9. 9.
    Dodangeh, M., Vicente, L.N.: Worst case complexity of direct search under convexity. Math. Program. doi:10.1007/s10107-014-0847-0
  10. 10.
    Dolan, E.D., Lewis, R.M., Torczon, V.: On the local convergence of pattern search. SIAM J. Optim. 14, 567–583 (2003)MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    Fejes Tóth, L.: Regular Figures. Pergman Press, London (1964)MATHGoogle Scholar
  12. 12.
    Fercoq, O., Richtárik, P.: Accelerated, parallel and proximal coordinate descent. Technical report, School of Mathematics, University of Edinburgh, December 2013. arXiv:1312.5799v2
  13. 13.
    Gluskin, E.D.: Extremal properties of orthogonal parallelepipeds and their applications to the geometry of Banach spaces. Math. USSR-Sbornik 64, 85–96 (1989)MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    Gratton, S., Royer, C.W., Vicente, L.N., Zhang, Z.: Direct search based on probabilistic descent. SIAM J. Optim. (to appear)Google Scholar
  15. 15.
    Kolda, T.G., Lewis, R.M., Torczon, V.: Optimization by direct search: new perspectives on some classical and modern methods. SIAM Rev. 45, 385–482 (2003)MathSciNetCrossRefMATHGoogle Scholar
  16. 16.
    Konečný, J., Richtárik, P.: Simple complexity analysis of simplified direct search. Technical report, School of Mathematics, University of Edinburgh, November 2014. arXiv:1410.0390v2
  17. 17.
    Nesterov, Y.: Efficiency of coordinate descent methods on huge-scale optimization problems. SIAM J. Optim. 22, 341–362 (2012)MathSciNetCrossRefMATHGoogle Scholar
  18. 18.
    Tikhomirov, K.E.: On the distance of polytopes with few vertices to the Euclidean ball. Discret. Comput. Geom. 53, 173–181 (2015)MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    Torczon, V.: On the convergence of pattern search algorithms. SIAM J. Optim. 7, 1–25 (1997)MathSciNetCrossRefMATHGoogle Scholar
  20. 20.
    Vicente, L.N.: Worst case complexity of direct search. Euro J. Comput. Optim. 1, 143–153 (2013)CrossRefMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.Department of MathematicsUniversity of CoimbraCoimbraPortugal
  2. 2.CMUC, Department of MathematicsUniversity of CoimbraCoimbraPortugal
  3. 3.CERFACS-IRIT Joint LabToulouseFrance

Personalised recommendations