Abstract
The concepts of positive span and positive basis are important in derivative-free optimization. In fact, a well-known result is that if the gradient of a continuously differentiable objective function on \(\mathbb{R}^n\) is nonzero at a point, then one of the vectors in any positive basis (or any positive spanning set) of \(\mathbb{R}^n\) is a descent direction for the objective function from that point. This article summarizes the basic results and explores additional properties of positive spanning sets, positively independent sets and positive bases that are potentially useful in the design of derivative-free optimization algorithms. In particular, it provides construction procedures for these special sets of vectors that were not previously mentioned in the literature. It also proves that invertible linear transformations preserve positive independence and the positive spanning property. Moreover, this article introduces the notion of linear equivalence between positive spanning sets and between positively independent sets to simplify the analysis of their structures. Linear equivalence turns out to be a generalization of the concept of structural equivalence between positive bases that was introduced by Coope and Price (SIAM J Optim 11:859–869, 2001). Furthermore, this article clarifies which properties of linearly independent sets, spanning sets and ordinary bases carry over to positively independent sets, positive spanning sets, and positive bases. For example, a linearly independent set can always be extended to a basis of a linear space but a positively independent set cannot always be extended to a positive basis. Also, the maximum size of a linearly independent set in \(R^n\) is n but there is no limit to the size of a positively independent set in \(\mathbb{R}^n\) when \(n \ge 3\). Whenever possible, the results are proved for the more general case of frames of convex cones instead of focusing only on positive bases of linear spaces. In addition, this article discusses some algorithms for determining whether a given set of vectors is positively independent or whether it positively spans a linear subspace of \(\mathbb{R}^n\). Finally, it provides an algorithm for extending any finite set of vectors to a positive spanning set of \(\mathbb{R}^n\) using only a relatively small number of additional vectors.
Similar content being viewed by others
References
Abramson MA, Audet C, Dennis JE Jr, Le Digabel S (2009) OrthoMADS: a deterministic mads instance with orthogonal directions. SIAM J Optim 20(2):948–966
Alberto P, Nogueira F, Rocha H, Vicente LN (2004) Pattern search methods for user-provided points: application to molecular geometry problems. SIAM J Optim 14(4):1216–1236
Audet C (2011) A short proof on the cardinality of maximal positive bases. Optim Lett 5(1):191–194
Audet C, Ianni A, Le Digabel S, Tribes C (2014) Reducing the number of function evaluations in mesh adaptive direct search algorithms. SIAM J Optim 24(2):621–642
Audet C, Dennis JE Jr (2006) Mesh adaptive direct search algorithms for constrained optimization. SIAM J Optim 17(2):188–217
Conn AR, Scheinberg K, Vicente LN (2009) Introduction to derivative-free optimization. SIAM, Philadelphia
Coope ID, Price CJ (2001) On the convergence of grid-based methods for unconstrained optimization. SIAM J Optim 11:859–869
Coope ID, Price CJ (2002) Positive bases in numerical optimization. Comput Optim Appl 21(2):169–175
Custódio AL, Dennis JE Jr, Vicente LN (2008) Using simplex gradients of nonsmooth functions in direct search methods. IMA J Numer Anal 28(4):770–784
Davis C (1954) Theory of positive linear dependence. Am J Math 76(4):733–746
Dulá JH, Helgason RV, Venugopal N (1998) An algorithm for identifying the frame of a pointed finite conical hull. INFORMS J Comput 10(3):323–330
Huang HX, Pardalos PM (2001) New algorithm for the conical combination representation problem of a vector. J Optim Theory Appl 109(3):495–519
Kelley CT (2011) Implicit filtering (software, environments and tools). SIAM, Philadelphia
Kolda TG, Lewis RM, Torczon V (2003) Optimization by direct search: new perspectives on some classical and modern methods. SIAM Rev 45(3):385–482
Lewis RM, Torczon V (1999) Rank ordering and positive bases in pattern search algorithms. Technical report TR96-71, ICASE, NASA langley research center
McKinney RL (1962) Positive bases for linear spaces. Trans Am Math Soc 103:131–148
Reay JR (1965) A new proof of the Bonnice-Klee theorem. Proc Am Math Soc 16:585–587
Reay JR (1966) Unique minimal representations with positive bases. Am Math Month 73(3):253–261
Romanowicz Z (1987) Geometric structure of positive bases in linear spaces. Zastos Mat Appl Math XIX(3–4):557–567
Shepard GC (1971) Diagrams for positive bases. J Lon Math Soc S2–4(1):165–175
Torczon V (1997) On the convergence of pattern search algorithms. SIAM J Optim 7(1):1–25
Vaz AIF, Vicente LN (2009) PSwarm: a hybrid solver for linearly constrained global derivative-free optimization. Optim Methods Softw 24(4–5):669–685
Acknowledgments
This paper is dedicated to the memory of my undergraduate math mentor Dr. Jose A. Marasigan. I am grateful to Saint Joseph’s University for providing me with a sabbatical in Fall 2014 to pursue work on this paper. I would also like to thank the anonymous referees for their comments.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Regis, R.G. On the properties of positive spanning sets and positive bases. Optim Eng 17, 229–262 (2016). https://doi.org/10.1007/s11081-015-9286-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11081-015-9286-x