Annals of Operations Research

, Volume 226, Issue 1, pp 301–349

# A characterization theorem and an algorithm for a convex hull problem

• Bahman Kalantari
Article

## Abstract

Given $$S= \{v_1, \dots , v_n\} \subset {\mathbb {R}}^m$$ and $$p \in {\mathbb {R}}^m$$, testing if $$p \in conv(S)$$, the convex hull of $$S$$, is a fundamental problem in computational geometry and linear programming. First, we prove a Euclidean distance duality, distinct from classical separation theorems such as Farkas Lemma: $$p$$ lies in $$conv(S)$$ if and only if for each $$p' \in conv(S)$$ there exists a pivot, $$v_j \in S$$ satisfying $$d(p',v_j) \ge d(p,v_j)$$. Equivalently, $$p \not \in conv(S)$$ if and only if there exists a witness, $$p' \in conv(S)$$ whose Voronoi cell relative to $$p$$ contains $$S$$. A witness separates $$p$$ from $$conv(S)$$ and approximate $$d(p, conv(S))$$ to within a factor of two. Next, we describe the Triangle Algorithm: given $$\epsilon \in (0,1)$$, an iterate, $$p' \in conv(S)$$, and $$v \in S$$, if $$d(p, p') < \epsilon d(p,v)$$, it stops. Otherwise, if there exists a pivot $$v_j$$, it replace $$v$$ with $$v_j$$ and $$p'$$ with the projection of $$p$$ onto the line $$p'v_j$$. Repeating this process, the algorithm terminates in $$O(mn \min \{ \epsilon ^{-2}, c^{-1}\ln \epsilon ^{-1} \})$$ arithmetic operations, where $$c$$ is the visibility factor, a constant satisfying $$c \ge \epsilon ^2$$ and $$\sin (\angle pp'v_j) \le 1/\sqrt{1+c}$$, over all iterates $$p'$$. In particular, the geometry of the input data may result in efficient complexities such as $$O(mn \root t \of {\epsilon ^{-2}} \ln \epsilon ^{-1})$$, $$t$$ a natural number, or even $$O(mn \ln \epsilon ^{-1})$$. Additionally, (i) we prove a strict distance duality and a related minimax theorem, resulting in more effective pivots; (ii) describe $$O(mn \ln \epsilon ^{-1})$$-time algorithms that may compute a witness or a good approximate solution; (iii) prove generalized distance duality and describe a corresponding generalized Triangle Algorithm; (iv) prove a sensitivity theorem to analyze the complexity of solving LP feasibility via the Triangle Algorithm. The Triangle Algorithm is practical and competitive with the simplex method, sparse greedy approximation and first-order methods. Finally, we discuss future work on applications and generalizations.

## Keywords

Convex hull Linear programming Duality Approximation algorithms Gilbert’s algorithm Frank–Wolfe algorithm Minimax

## Notes

### Acknowledgments

I thank Iraj Kalantari for a discussion on the use of Voronoi diagram argument to prove Theorem 1, resulting in a simpler geometric proof than the one given in an earlier version of this article. I thank Tong Zhang for a discussion regarding the Greedy Algorithm for general convex minimization over a simplex. I also thank Günter Rote for bringing to my attention the work of Kuhn (2011) and Durier and Michelot (1986) (Remark 4). I thank a referee for bringing to my attention Dantzig (1992) technical report on analysis of von Neumann algorithm. Also, when the article was being refereed Mike Grigoriadis brought to my attention Dantzig’s analysis in Epelman and Freund (1997). I thank a second anonymous referee for his comments.

## References

1. Asano, T., Matoušek, J., & Tokuyama, T. (2007). Zone diagrams: Existence, uniqueness, and algorithmic challenge. SIAM Journal on Computing, 37, 1182–1198.
2. Aurenhammer, F. (1991). Voronoi diagrams—a survey of fundamental geometric data structure. ACM Computing Surveys, 23, 345–405.
3. Berkowitz, R., Kalantari, B., Memendendez, D., & Kalantari, I. (2012). On properties of forbidden zone of polyhedrons and polytopes. In Proceedings of the ninth annual international symposium on voronoi diagrams in science and engineering, pp. 56–65.Google Scholar
4. Burges, C. (1998). A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 2, 121–167.
5. Chan, T. M. (1996). Output-sensitive results on convex hulls, extreme points, and related problems. Discrete & Computational Geometry, 16, 369–387.
6. Chazelle, B. (1993). An optimal convex hull algorithm in any fixed dimension. Discrete & Computational Geometry, 10, 377–409.
7. Chvátal, V. (1983). Linear programming. New York: W.H. Freeman and Company.Google Scholar
8. Clarkson, K. L. (1988). Las Vegas algorithms for linear and integer programming when the dimension is small. In: Proceedings of 29th annual IEEE symposium on foundations of computer science, pp. 452–456.Google Scholar
9. Clarkson, K. L. (1994). More output-sensitive geometric algorithm. In Proceedings of the 35-th annual IEEE symposium on foundations of computer science, pp. 695–702.Google Scholar
10. Clarkson, K. L. (2008). Coresets, sparse greedy approximation, and the Frank-Wolfe algorithm. In SODA ’08: Proceedings of the nineteenth annual ACM-SIAM symposium on discrete algorithms (pp. 922–931). Philadelphia: Society for Industrial and Applied Mathematics.Google Scholar
11. Dantzig, G. B. (1963). Linear programming and extensions. Princeton, NJ: Princeton University Press.Google Scholar
12. Dantzig, G. B. (1992). An $$\epsilon$$-precise feasible solution to a linear program with a convexity constraint in $$1/\epsilon ^2$$ iterations independent of problem size. Technical Report, Stanford University.Google Scholar
13. de Biasi, S. C., Kalantari, B., & Kalantari, I. (2010). Maximal zone diagrams and their computation. In Proceedings of the seventh annual international symposium on voronoi diagrams in science and engineering, pp. 171–180.Google Scholar
14. de Biasi, S. C., Kalantari, B., & Kalantari, I. (2011). Mollified zone diagrams and their computation. In Transactions on computational science XIV: Lecture notes in computer science (Vol. 6970, pp. 31–59).Google Scholar
15. Durier, R., & Michelot, C. (1986). Sets of efficient points in a normed space. Journal of Mathematical Analysis and Applications, 117, 506–528.
16. Dyer, M. E., & Frieze, A. M. (1989). A randomized algorithm for fixed-dimensional linear programming. Mathematical Programming, 44, 203–212.
17. Epelman, M., & Freund, R. M. (1997). Condition number complexity of an elementary algorithm for resolving a conic linear system, Technical Report. http://dspace.mit.edu/bitstream/handle/1721.1/5254/or-319-97-37692575?sequence=1.
18. Frank, M., & Wolfe, P. (1956). An algorithm for quadratic programming. Naval Research Logistics Quarterly, 3, 95–110.
19. Gärtner, B., & Jaggi, M. (2009). Coresets for polytope distance. In Symposium on computational geometry, pp. 33–42.Google Scholar
20. Gibson, T., & Kalantari, B. (2013). Experiments with the triangle algorithm for linear systems, 2-page Extended Abstract, 23nd annual fall workshop on computational geometry, City College of New York.Google Scholar
21. Gilbert, E. G. (1966). An iterative procedure for computing the minimum of a quadratic form on a convex set. SIAM Journal on Control, 4, 61–80.
22. Goodman, J. E., & O’Rourke, J. (Eds.). (2004). Handbook of discrete and computational geometry, 2nd Edn (Discrete Mathematics and Its Applications). Boca Raton: Chapman & Hall.Google Scholar
23. Har-Peled, S., Roth, D., & Zimak, D. (2007). Maximum margin coresets for active and noise tolerant learning. IJCAI.Google Scholar
24. Jin, Y., & Kalantari, B. (2006). A procedure of Chvátal for testing feasibility in linear programming and matrix scaling. Linear Algebra and its Applications, 416, 795–798.
25. Kalai, G. (1992). A subexponential randomized simplex algorithm. In Proceedings of the 24-th annual ACM symposium on theory of computing, pp. 475–482.Google Scholar
26. Kalantari, B. (2014). An algorithmic separating hyperplane theorem and its applications.Google Scholar
27. Kalantari, B. (1990). Canonical problems for quadratic programming and projective methods for their solution. Contemporary Mathematics, 114, 243–263.
28. Kalantari, B. (2003). Semidefinite programming and matrix scaling over the semidefinite cone. Linear Algebra and its Applications, 375, 221–243.
29. Kalantari, B. (2012a). A characterization theorem and an algorithm for a convex hull problem. arXiv:1204.1873v2.
30. Kalantari, B. (2012b). Finding a lost treasure in convex hull of points from known distances. In The Proceedings of the 24th Canadian conference on computational geometry, pp. 271–276.Google Scholar
31. Kalantari, B. (2012c). Solving linear system of equations via a convex hull algorithm. arXiv:1210.7858v1.
32. Kalantari, B., & Emamy-K, M. R. (1997). On linear programming and matrix scaling over the algebraic numbers. Linear Algebra and its Applications, 262, 283–306.
33. Kalantari, B., & Saks, M. (2013). On the triangle algorithm for the convex hull membership, 2-page Extended Abstract, 23nd annual fall workshop on computational geometry, City College of New York.Google Scholar
34. Karmarkar, N. (1984). A new polynomial time algorithm for linear programming. Combinatorica, 4, 373–395.
35. Kelner, J. A., & Spielman, D. A. (2006). A randomized polynomial-time simplex algorithm for linear programming, In Proceedings of the 38th annual ACM symposium on theory of computing, pp 51–60.Google Scholar
36. Khachiyan, L. G. (1979). A polynomial algorithm in linear programming. Doklady Akademia Nauk SSSR, 244, 1093–1096 (translated in Soviet Mathematics Doklady 20, 191–194, 1979).Google Scholar
37. Khachiyan, L., & Kalantari, B. (1992). Diagonal matrix scaling and linear programming. SIAM Journal on Optimization, 4, 668–672.
38. Klee, V., & Minty, G. J. (1972). How good is the simplex algorithm? In O. Shisha (Ed.), Inequalities III (pp. 159–175). London: Academic Press.Google Scholar
39. Kuhn, H. W. (2011). On a pair of dual nonlinear programs. In J. Abadie (Ed.), Nonlinear programming (pp. 37–54). New York: Wiley, 1967, see also, Locational problems and mathematical programming, in Mathematical Optimiation in Economics C.I.M.E. Summer Schools, Vol. 38, pp. 57–76.Google Scholar
40. Lan, G., Lu, Z., & Monteiro, R. D. C. (2011). Primal-dual first-order methods with $$O(1/\epsilon )$$ iteration-complexity for cone programming. Mathematical Programming Series A, 126, 1–29.
41. Li, M., & Kalantari, B. (2013). Experimental study of the convex hull decision problem via a new geometric algorithm, 2-page Extended Abstract, 23nd annual fall workshop on computational geometry, City College of New York.Google Scholar
42. Matoušek, J. (1993). Linear optimization queries. Journal of Algorithms, 14, 432–448.
43. Matoušek, J., Sharir, M., & Welzl, E. (1992). A subexponential bound for linear programming. In Proceedings of the 8th annual ACM symposium on computational geometry, pp. 1–8.Google Scholar
44. Megiddo, N. (1984). Linear programming in linear time when the dimension is fixed. Journal of the ACM, 31, 114–127.
45. Motawani, R., & Raghavan, R. (1995). Randomized algorithms. Cambridge: Cambridge University Press.
46. Nesterov, Y. E. (2013). Private communications.Google Scholar
47. Nesterov, Y. E. (2005). Smooth minimization of nonsmooth functions. Mathematical Programming, 103, 127–152.
48. Preparata, F. P., & Shamos, M. I. (1985). Computational geometry: An introduction. New York: Springer.
49. Rockafellar, R. T. (1997). Convex analysis. Princeton, New Jersey: Princeton University Press. [1970].Google Scholar
50. Schrijver, A. (1986). Theory of linear and integer programming. New York, Chichester, Brisbane, Toronto and Singapore: Wiley.Google Scholar
51. Seidel, R. (1991). Small-dimensional linear programming and convex hulls made easy. Discrete Computational Geometry, 6, 423–434.
52. Shamir, R. (1987). The efficiency of the simplex method: A survey. Management Science, 33, 301–334.
53. Sharir, M., & Welzl, E. (1992). A combinatorial bound for linear programming and related problems. In Proceedings of the 9th symposium on theoretical aspects of computer science, pp. 569–579.Google Scholar
54. Todd, M. J. (2002). The many facets of linear programming. Mathematical Programming, 91, 417–436.
55. Zhang, T. (2003). Sequential greedy approximation for certain convex optimization problems. IEEE Transactions on Information Theory, 49, 682–691.
56. Zimak, D. A. (2006). Algorithms and Analysis for Multi-Category Classification, Ph.D. thesis, Department of Computer Science, University of Illinois at Urbana-Champaign.Google Scholar