Optimization and Engineering

, Volume 17, Issue 1, pp 229–262 | Cite as

On the properties of positive spanning sets and positive bases

Article
First Online:
Received:
Revised:
Accepted:

Abstract

The concepts of positive span and positive basis are important in derivative-free optimization. In fact, a well-known result is that if the gradient of a continuously differentiable objective function on \(\mathbb{R}^n\) is nonzero at a point, then one of the vectors in any positive basis (or any positive spanning set) of \(\mathbb{R}^n\) is a descent direction for the objective function from that point. This article summarizes the basic results and explores additional properties of positive spanning sets, positively independent sets and positive bases that are potentially useful in the design of derivative-free optimization algorithms. In particular, it provides construction procedures for these special sets of vectors that were not previously mentioned in the literature. It also proves that invertible linear transformations preserve positive independence and the positive spanning property. Moreover, this article introduces the notion of linear equivalence between positive spanning sets and between positively independent sets to simplify the analysis of their structures. Linear equivalence turns out to be a generalization of the concept of structural equivalence between positive bases that was introduced by Coope and Price (SIAM J Optim 11:859–869, 2001). Furthermore, this article clarifies which properties of linearly independent sets, spanning sets and ordinary bases carry over to positively independent sets, positive spanning sets, and positive bases. For example, a linearly independent set can always be extended to a basis of a linear space but a positively independent set cannot always be extended to a positive basis. Also, the maximum size of a linearly independent set in \(R^n\) is n but there is no limit to the size of a positively independent set in \(\mathbb{R}^n\) when \(n \ge 3\). Whenever possible, the results are proved for the more general case of frames of convex cones instead of focusing only on positive bases of linear spaces. In addition, this article discusses some algorithms for determining whether a given set of vectors is positively independent or whether it positively spans a linear subspace of \(\mathbb{R}^n\). Finally, it provides an algorithm for extending any finite set of vectors to a positive spanning set of \(\mathbb{R}^n\) using only a relatively small number of additional vectors.

Keywords

Positive independence Positive spanning set Positive basis Derivative-free optimization Direct search  Linear equivalence 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgments

This paper is dedicated to the memory of my undergraduate math mentor Dr. Jose A. Marasigan. I am grateful to Saint Joseph’s University for providing me with a sabbatical in Fall 2014 to pursue work on this paper. I would also like to thank the anonymous referees for their comments.

References

  1. Abramson MA, Audet C, Dennis JE Jr, Le Digabel S (2009) OrthoMADS: a deterministic mads instance with orthogonal directions. SIAM J Optim 20(2):948–966CrossRefMathSciNetMATHGoogle Scholar
  2. Alberto P, Nogueira F, Rocha H, Vicente LN (2004) Pattern search methods for user-provided points: application to molecular geometry problems. SIAM J Optim 14(4):1216–1236CrossRefMathSciNetMATHGoogle Scholar
  3. Audet C (2011) A short proof on the cardinality of maximal positive bases. Optim Lett 5(1):191–194CrossRefMathSciNetMATHGoogle Scholar
  4. Audet C, Ianni A, Le Digabel S, Tribes C (2014) Reducing the number of function evaluations in mesh adaptive direct search algorithms. SIAM J Optim 24(2):621–642CrossRefMathSciNetMATHGoogle Scholar
  5. Audet C, Dennis JE Jr (2006) Mesh adaptive direct search algorithms for constrained optimization. SIAM J Optim 17(2):188–217CrossRefMathSciNetMATHGoogle Scholar
  6. Conn AR, Scheinberg K, Vicente LN (2009) Introduction to derivative-free optimization. SIAM, PhiladelphiaCrossRefMATHGoogle Scholar
  7. Coope ID, Price CJ (2001) On the convergence of grid-based methods for unconstrained optimization. SIAM J Optim 11:859–869CrossRefMathSciNetMATHGoogle Scholar
  8. Coope ID, Price CJ (2002) Positive bases in numerical optimization. Comput Optim Appl 21(2):169–175CrossRefMathSciNetMATHGoogle Scholar
  9. Custódio AL, Dennis JE Jr, Vicente LN (2008) Using simplex gradients of nonsmooth functions in direct search methods. IMA J Numer Anal 28(4):770–784CrossRefMathSciNetMATHGoogle Scholar
  10. Davis C (1954) Theory of positive linear dependence. Am J Math 76(4):733–746CrossRefMATHGoogle Scholar
  11. Dulá JH, Helgason RV, Venugopal N (1998) An algorithm for identifying the frame of a pointed finite conical hull. INFORMS J Comput 10(3):323–330CrossRefMathSciNetMATHGoogle Scholar
  12. Huang HX, Pardalos PM (2001) New algorithm for the conical combination representation problem of a vector. J Optim Theory Appl 109(3):495–519CrossRefMathSciNetMATHGoogle Scholar
  13. Kelley CT (2011) Implicit filtering (software, environments and tools). SIAM, PhiladelphiaCrossRefGoogle Scholar
  14. Kolda TG, Lewis RM, Torczon V (2003) Optimization by direct search: new perspectives on some classical and modern methods. SIAM Rev 45(3):385–482CrossRefMathSciNetMATHGoogle Scholar
  15. Lewis RM, Torczon V (1999) Rank ordering and positive bases in pattern search algorithms. Technical report TR96-71, ICASE, NASA langley research centerGoogle Scholar
  16. McKinney RL (1962) Positive bases for linear spaces. Trans Am Math Soc 103:131–148CrossRefMathSciNetMATHGoogle Scholar
  17. Reay JR (1965) A new proof of the Bonnice-Klee theorem. Proc Am Math Soc 16:585–587MathSciNetMATHGoogle Scholar
  18. Reay JR (1966) Unique minimal representations with positive bases. Am Math Month 73(3):253–261CrossRefMathSciNetMATHGoogle Scholar
  19. Romanowicz Z (1987) Geometric structure of positive bases in linear spaces. Zastos Mat Appl Math XIX(3–4):557–567MathSciNetGoogle Scholar
  20. Shepard GC (1971) Diagrams for positive bases. J Lon Math Soc S2–4(1):165–175CrossRefMathSciNetGoogle Scholar
  21. Torczon V (1997) On the convergence of pattern search algorithms. SIAM J Optim 7(1):1–25CrossRefMathSciNetMATHGoogle Scholar
  22. Vaz AIF, Vicente LN (2009) PSwarm: a hybrid solver for linearly constrained global derivative-free optimization. Optim Methods Softw 24(4–5):669–685CrossRefMathSciNetMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.Department of MathematicsSaint Joseph’s UniversityPhiladelphiaUSA

Personalised recommendations