Computational Optimization and Applications

, Volume 70, Issue 2, pp 443–478 | Cite as

A primal-dual homotopy algorithm for \(\ell _{1}\)-minimization with \(\ell _{\infty }\)-constraints

  • Christoph Brauer
  • Dirk A. Lorenz
  • Andreas M. Tillmann


In this paper we propose a primal-dual homotopy method for \(\ell _1\)-minimization problems with infinity norm constraints in the context of sparse reconstruction. The natural homotopy parameter is the value of the bound for the constraints and we show that there exists a piecewise linear solution path with finitely many break points for the primal problem and a respective piecewise constant path for the dual problem. We show that by solving a small linear program, one can jump to the next primal break point and then, solving another small linear program, a new optimal dual solution is calculated which enables the next such jump in the subsequent iteration. Using a theorem of the alternative, we show that the method never gets stuck and indeed calculates the whole path in a finite number of steps. Numerical experiments demonstrate the effectiveness of our algorithm. In many cases, our method significantly outperforms commercial LP solvers; this is possible since our approach employs a sequence of considerably simpler auxiliary linear programs that can be solved efficiently with specialized active-set strategies.


Convex optimization Dantzig selector Homotopy methods Nonsmooth optimization Primal-dual methods 

Mathematics Subject Classification

90C05 90C25 65K05 

Supplementary material


  1. 1.
    Appa, G., Smith, C.: On \(L_{1}\) and Chebyshev estimation. Math. Program. 5, 73–87 (1973). CrossRefzbMATHGoogle Scholar
  2. 2.
    Asif, M.S.: Primal dual pursuit—a homotopy based algorithm for the Dantzig selector. Master’s thesis, Georgia Institute of Technology (2008)Google Scholar
  3. 3.
    Asif, M.S., Romberg, J.: Dantzig selector homotopy with dynamic measurements. In: Proceedings on SPIE 7246, Computational Imaging VII, 72460E (2009)Google Scholar
  4. 4.
    Asif, M.S., Romberg, J.: On the Lasso and Dantzig selector equivalence. In: Proceedings on CISS. IEEE (2010)Google Scholar
  5. 5.
    Blum, L.: A new simple homotopy algorithm for linear programming I. J. Complex. 4, 124–136 (1988)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Brauer, C., Gerkmann, T., Lorenz, D.A.: Sparse reconstruction of quantized speech signals. In: Proceedings on ICASSP. IEEE (2016)Google Scholar
  7. 7.
    Cai, T., Liu, W.: A direct estimation approach to sparse linear discriminant analysis. J. Am. Stat. Assoc. 106(496), 1566–1577 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Cai, T., Liu, W., Luo, X.: A constrained \(\ell _1\) minimization approach to sparse precision matrix estimation. J. Am. Stat. Assoc. 106(494), 594–607 (2011)CrossRefzbMATHGoogle Scholar
  9. 9.
    Candés, E.J., Tao, T.: Decoding by linear programming. IEEE Trans. Inf. Theory 51(12), 4203–4215 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Candés, E.J., Tao, T.: The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). Ann. Stat. 35(6), 2313–2351 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic decomposition by basis pursuit. SIAM J. Sci. Comput. 20(1), 33–61 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Dantzig, G.B.: Linear Programming and Extensions. Princeton University Press, Princeton (1963)CrossRefzbMATHGoogle Scholar
  13. 13.
    Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Eldar, Y.C., Kutyniok, G. (eds.): Compressed Sensing: Theory and Applications. Cambridge University Press, Cambridge (2012)Google Scholar
  15. 15.
    Foucart, S., Rauhut, H.: A Mathematical Introduction to Compressive Sensing. Birkhäuser, Basel (2013)CrossRefzbMATHGoogle Scholar
  16. 16.
    Jacques, L., Hammond, D.K., Fadili, J.M.: Dequantizing compressed sensing: when oversampling and non-Gaussian constraints combine. IEEE Trans. Inf. Theory 57(1), 559–571 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    James, G.M., Radchenko, P., Lv, J.: DASSO: connections between the Dantzig selector and Lasso. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 71(1), 127–142 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Kohavi, R.: A study of cross-validation and bootstrap for accuracy estimation and model selection. In: International Joint Conference on Artificial Intelligence (IJCAI) (1995)Google Scholar
  19. 19.
    Lorenz, D.A., Pfetsch, M.E., Tillmann, A.M.: Solving basis pursuit: heuristic optimality check and solver comparison. ACM Trans. Math. Softw. 41(2), 8 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Mairal, J., Yu, B.: Complexity analysis of the lasso regularization path. ArXiv preprint arXiv:1205.0079 (2012)
  21. 21.
    Nazareth, J.L.: The homotopy principle and algorithms for linear programming. SIAM J. Optim. 1(3), 316–332 (1991)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Nocedal, J., Wright, S.J.: Numerical Optimization, 2nd edn. Springer, Berlin (2006)zbMATHGoogle Scholar
  23. 23.
    Osborne, M.R., Presnell, B., Turlach, B.A.: A new approach to variable selection in least squares problems. IMA J. Numer. Anal. 20, 389–404 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Pang, H., Zhao, T., Vanderbei, R.J., Liu, H.: A parametric simplex approach to statistical learning problems. Unpublished manuscript. (2015)
  25. 25.
    Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton (1970)CrossRefzbMATHGoogle Scholar
  26. 26.
    Schrijver, A.: Theory of Linear and Integer Programming. Wiley, Hoboken (1986)zbMATHGoogle Scholar
  27. 27.
    Späth, H.: Mathematical Algorithms for Linear Regression. Computer Science and Scientific Computing. Translated and Revised from the 1987 German Original by the Author. Academic Press Inc., Boston (1992)zbMATHGoogle Scholar
  28. 28.
    Stiefel, E.: Über diskrete und lineare tschebyscheff-approximationen. Numerische Mathematik 1(1), 1–28 (1959). MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Vanderbei, R.J.: Linear Programming: Foundations and Extensions, 2nd edn. Kluwer Academic Publishers, Dordrecht (2001)CrossRefzbMATHGoogle Scholar
  30. 30.
    Zheng, S., Liu, W.: An experimental comparison of gene selection by Lasso and Dantzig selector for cancer classification. Comput. Biol. Med. 41(11), 1033–1040 (2011)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Institut für Analysis und AlgebraTechnische Universität BraunschweigBraunschweigGermany
  2. 2.Visual Computing Institute & Chair of Operations ResearchRWTH Aachen UniversityAachenGermany

Personalised recommendations