Advertisement

A filtered bucket-clustering method for projection onto the simplex and the \(\ell _1\) ball

  • Guillaume PerezEmail author
  • Michel Barlaud
  • Lionel Fillatre
  • Jean-Charles Régin
Short Communication Series A
  • 60 Downloads

Abstract

We propose in this paper a new method processing the projection of an arbitrary size vector onto the probabilistic simplex or the \(\ell _1\) ball. Our method merges two principles. The first one is an original search of the projection using a bucket algorithm. The second one is a filtering, on the fly, of the values that cannot be part of the projection. The combination of these two principles offers a simple and efficient algorithm whose worst-case complexity is linear with respect to the vector size. Furthermore, the proposed algorithm exploits the representation of numeric values in digital computers to define the number of buckets and to accelerate the filtering.

Mathematics Subject Classification

49M30 65C60 65K05 90C25 

Notes

References

  1. 1.
    Barlaud, M., Belhajali, W., Combettes, P.L., Fillatre, L.: Classification and regression using an outer approximation projection-gradient method. IEEE Trans. Signal Process. 65(17), 4635–4644 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Candès, E.J., Tao, T.: Decoding by linear programming. IEEE Trans. Inf. Theory 51(12), 4203–4215 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Condat, L.: Fast projection onto the simplex and the \(\ell _1\) ball. Math. Program. Ser. A 158(1), 575–585 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C.: Introduction to Algorithms., vol. 6. MIT Press, Cambridge (2001)zbMATHGoogle Scholar
  5. 5.
    Donoho, D.L.: De-noising by soft-thresholding. IEEE Trans. Inf. Theory 41(3), 613–627 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Donoho, D.L., Elad, M.: Optimally sparse representation in general (nonorthogonal) dictionaries via \(\ell _1\) minimization. Proc. Natl. Acad. Sci. 100(5), 2197–2202 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Duchi, J., Shalev-Shwartz, S., Singer, Y., Chandra, T.: Efficient projections onto the \(\ell _1\)-ball for learning in high dimensions. In: Proceedings of the 25th International Conference on Machine Learning. ACM (2008), pp. 272–279Google Scholar
  8. 8.
    Held, M., Wolfe, P., Crowder, H.P.: Validation of subgradient optimization. Math. Program. 6(1), 62–88 (1974)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Kiwiel, K.C.: Breakpoint searching algorithms for the continuous quadratic knapsack problem. Math. Program. 112(2), 473–491 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Michelot, C.: A finite algorithm for finding the projection of a point onto the canonical simplex of \({\mathbb{R}}^n\). J. Optim. Theory Appl. 50(1), 195–200 (1986)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Natarajan, B.: Sparse approximate solutions to linear systems. SIAM J. Comput. 24(2), 227–234 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Perez, G., Barlaud, M., Fillatre, l., Régin, J.-C.: A filtered bucket-clustering method for Projection onto the Simplex and the \(\ell _1\) ball. In: XXVIème Colloque Gretsi, Juan-Les-Pins (2017)Google Scholar
  13. 13.
    Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B (Methodol.) 267–288 (1996)Google Scholar
  14. 14.
    Tibshirani, R.: The lasso method for variable selection in the COX model. Stat. Med. (1997)Google Scholar
  15. 15.
    Van Den Berg, E., Friedlander, M.P.: Probing the pareto frontier for basis pursuit solutions. SIAM J. Sci. Comput. 31(2), 890–912 (2008)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature and Mathematical Optimization Society 2019

Authors and Affiliations

  1. 1.Cornell UniversityIthacaUSA
  2. 2.Université Côte d’Azur, CNRSSophia AntipolisFrance

Personalised recommendations