Abstract
We propose in this paper a new method processing the projection of an arbitrary size vector onto the probabilistic simplex or the \(\ell _1\) ball. Our method merges two principles. The first one is an original search of the projection using a bucket algorithm. The second one is a filtering, on the fly, of the values that cannot be part of the projection. The combination of these two principles offers a simple and efficient algorithm whose worst-case complexity is linear with respect to the vector size. Furthermore, the proposed algorithm exploits the representation of numeric values in digital computers to define the number of buckets and to accelerate the filtering.
This is a preview of subscription content, log in via an institution to check access.
Notes
The code used for these experiments is available at www.perezguillau.me/code/projection.zip. Note that part of this code is based on the generously provided code from https://www.gipsa-lab.grenoble-inp.fr/~laurent.condat/software.html.
References
Barlaud, M., Belhajali, W., Combettes, P.L., Fillatre, L.: Classification and regression using an outer approximation projection-gradient method. IEEE Trans. Signal Process. 65(17), 4635–4644 (2017)
Candès, E.J., Tao, T.: Decoding by linear programming. IEEE Trans. Inf. Theory 51(12), 4203–4215 (2005)
Condat, L.: Fast projection onto the simplex and the \(\ell _1\) ball. Math. Program. Ser. A 158(1), 575–585 (2016)
Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C.: Introduction to Algorithms., vol. 6. MIT Press, Cambridge (2001)
Donoho, D.L.: De-noising by soft-thresholding. IEEE Trans. Inf. Theory 41(3), 613–627 (1995)
Donoho, D.L., Elad, M.: Optimally sparse representation in general (nonorthogonal) dictionaries via \(\ell _1\) minimization. Proc. Natl. Acad. Sci. 100(5), 2197–2202 (2003)
Duchi, J., Shalev-Shwartz, S., Singer, Y., Chandra, T.: Efficient projections onto the \(\ell _1\)-ball for learning in high dimensions. In: Proceedings of the 25th International Conference on Machine Learning. ACM (2008), pp. 272–279
Held, M., Wolfe, P., Crowder, H.P.: Validation of subgradient optimization. Math. Program. 6(1), 62–88 (1974)
Kiwiel, K.C.: Breakpoint searching algorithms for the continuous quadratic knapsack problem. Math. Program. 112(2), 473–491 (2008)
Michelot, C.: A finite algorithm for finding the projection of a point onto the canonical simplex of \({\mathbb{R}}^n\). J. Optim. Theory Appl. 50(1), 195–200 (1986)
Natarajan, B.: Sparse approximate solutions to linear systems. SIAM J. Comput. 24(2), 227–234 (1995)
Perez, G., Barlaud, M., Fillatre, l., Régin, J.-C.: A filtered bucket-clustering method for Projection onto the Simplex and the \(\ell _1\) ball. In: XXVIème Colloque Gretsi, Juan-Les-Pins (2017)
Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B (Methodol.) 267–288 (1996)
Tibshirani, R.: The lasso method for variable selection in the COX model. Stat. Med. (1997)
Van Den Berg, E., Friedlander, M.P.: Probing the pareto frontier for basis pursuit solutions. SIAM J. Sci. Comput. 31(2), 890–912 (2008)
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Part of this work has been already published [12].
Appendix A
Appendix A
See Table 3.
Rights and permissions
About this article
Cite this article
Perez, G., Barlaud, M., Fillatre, L. et al. A filtered bucket-clustering method for projection onto the simplex and the \(\ell _1\) ball. Math. Program. 182, 445–464 (2020). https://doi.org/10.1007/s10107-019-01401-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10107-019-01401-3