Optimization Letters

, Volume 10, Issue 4, pp 699–708 | Cite as

On the optimal order of worst case complexity of direct search

  • M. Dodangeh
  • L. N. Vicente
  • Z. ZhangEmail author
Original Paper


The worst case complexity of direct-search methods has been recently analyzed when they use positive spanning sets and impose a sufficient decrease condition to accept new iterates. For smooth unconstrained optimization, it is now known that such methods require at most \(\mathcal {O}(n^2\epsilon ^{-2})\) function evaluations to compute a gradient of norm below \(\epsilon \in (0,1)\), where n is the dimension of the problem. Such a maximal effort is reduced to \(\mathcal {O}(n^2\epsilon ^{-1})\) if the function is convex. The factor \(n^2\) has been derived using the positive spanning set formed by the coordinate vectors and their negatives at all iterations. In this paper, we prove that such a factor of \(n^2\) is optimal in these worst case complexity bounds, in the sense that no other positive spanning set will yield a better order of n. The proof is based on an observation that reveals the connection between cosine measure in positive spanning and sphere covering.


Direct search Worst case complexity Optimal order Sphere covering Positive spanning set Cosine measure 



We would like to thank Professors Károly Böröczky, Jr., Ilya Dumer, Gabor Fejes Tóth, and Tibor Tarnai, with whom we had helpful discussions on sphere covering.


  1. 1.
    Bárány, I., Füredi, Z.: Approximation of the sphere by polytopes having few vertices. Proc. Am. Math. Soc. 102, 651–659 (1988)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Böröczky Jr, K.: Finite Packing and Covering. Cambridge University Press, New York (2004)CrossRefzbMATHGoogle Scholar
  3. 3.
    Böröczky Jr, K.: April 25, 2014 (private communication)Google Scholar
  4. 4.
    Böröczky Jr, K., Wintsche, G.: Covering the sphere by equal spherical balls. In: Aronov, B., Basu, S., Pach, J., Sharir, M. (eds.) Discrete and Computational Geometry. Algorithms and Combinatorics, vol. 25, pp. 235–251. Springer, Berlin (2003)CrossRefGoogle Scholar
  5. 5.
    Bourgain, J., Lindenstrauss, J., Milman, V.: Approximation of zonoids by zonotopes. Acta Math. 162, 73–141 (1989)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Carl, B., Pajor, A.: Gelfand numbers of operators with values in a Hilbert space. Invent. Math. 94, 479–504 (1988)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Conn, A.R., Scheinberg, K., Vicente, L.N.: Introduction to Derivative-Free Optimization. MPS-SIAM Series on Optimization. SIAM, Philadelphia (2009)CrossRefzbMATHGoogle Scholar
  8. 8.
    Dalla, L., Larman, D.G., Mani-Levitska, P., Zong, C.: The blocking numbers of convex bodies. Discret. Comput. Geom. 24, 267–278 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Dodangeh, M., Vicente, L.N.: Worst case complexity of direct search under convexity. Math. Program. doi: 10.1007/s10107-014-0847-0
  10. 10.
    Dolan, E.D., Lewis, R.M., Torczon, V.: On the local convergence of pattern search. SIAM J. Optim. 14, 567–583 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Fejes Tóth, L.: Regular Figures. Pergman Press, London (1964)zbMATHGoogle Scholar
  12. 12.
    Fercoq, O., Richtárik, P.: Accelerated, parallel and proximal coordinate descent. Technical report, School of Mathematics, University of Edinburgh, December 2013. arXiv:1312.5799v2
  13. 13.
    Gluskin, E.D.: Extremal properties of orthogonal parallelepipeds and their applications to the geometry of Banach spaces. Math. USSR-Sbornik 64, 85–96 (1989)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Gratton, S., Royer, C.W., Vicente, L.N., Zhang, Z.: Direct search based on probabilistic descent. SIAM J. Optim. (to appear)Google Scholar
  15. 15.
    Kolda, T.G., Lewis, R.M., Torczon, V.: Optimization by direct search: new perspectives on some classical and modern methods. SIAM Rev. 45, 385–482 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Konečný, J., Richtárik, P.: Simple complexity analysis of simplified direct search. Technical report, School of Mathematics, University of Edinburgh, November 2014. arXiv:1410.0390v2
  17. 17.
    Nesterov, Y.: Efficiency of coordinate descent methods on huge-scale optimization problems. SIAM J. Optim. 22, 341–362 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Tikhomirov, K.E.: On the distance of polytopes with few vertices to the Euclidean ball. Discret. Comput. Geom. 53, 173–181 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Torczon, V.: On the convergence of pattern search algorithms. SIAM J. Optim. 7, 1–25 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Vicente, L.N.: Worst case complexity of direct search. Euro J. Comput. Optim. 1, 143–153 (2013)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.Department of MathematicsUniversity of CoimbraCoimbraPortugal
  2. 2.CMUC, Department of MathematicsUniversity of CoimbraCoimbraPortugal
  3. 3.CERFACS-IRIT Joint LabToulouseFrance

Personalised recommendations