Skip to main content
SpringerLink
Log in

A Predual Proximal Point Algorithm Solving a Non Negative Basis Pursuit Denoising Model

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

This paper develops an implementation of a Predual Proximal Point Algorithm (PPPA) solving a Non Negative Basis Pursuit Denoising model. The model imposes a constraint on the l 2 norm of the residual, instead of penalizing it. The PPPA solves the predual of the problem with a Proximal Point Algorithm (PPA). Moreover, the minimization that needs to be performed at each iteration of PPA is solved with a dual method. We can prove that these dual variables converge to a solution of the initial problem.

Our analysis proves that we turn a constrained non differentiable convex problem into a short sequence of nice concave maximization problems. By nice, we mean that the functions which are maximized are differentiable and their gradient is Lipschitz.

The algorithm is easy to implement, easier to tune and more general than the algorithms found in the literature. In particular, it can be applied to the Basis Pursuit Denoising (BPDN) and the Non Negative Basis Pursuit Denoising (NNBPDN) and it does not make any assumption on the dictionary. We prove its convergence to the set of solutions of the model and provide some convergence rates.

Experiments on image approximation show that the performances of the PPPA are at the current state of the art for the BPDN.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Bect, J., Blanc-Féraud, L., Aubert, G., & Chambolle, A. (2004). A l1-unified variational framework for image restoration. In Lecture notes in computer science. Proc. ECCV 2004. Berlin: Springer.

    Google Scholar 

  • Berg, E. V., Friedlander, M. P., Hennenfent, G., Herrmann, F., Saab, R., & Yılmaz, Ö. (2007). Sparco: A testing framework for sparse reconstruction (Tech. Rep. TR-2007-20). Dept. Computer Science, University of British Columbia, Vancouver.

  • Bertsekas, D. P. (2003). Nonlinear programming (2nd ed.). Belmont: Athena Scientific.

    Google Scholar 

  • Bioucas-Dias, J. (2006). Bayesian wavelet-based image deconvolution: A gem algorithm exploiting a class of heavy-tailed priors. IEEE Transactions on Image Processing, 15(4), 937–951.

    Article  Google Scholar 

  • Brown, M., & Costen, N. (2005). Exploratory basis pursuit classification. Pattern Recognition Letters, 26, 1907–1915.

    Article  Google Scholar 

  • Candes, E., Romberg, J., & Tao, T. (2006). Robust uncertainty principles : Exact signal reconstruction from highly incomplete frequency information. IEEE Transactions on Information Theory, 52(2), 489–509.

    Article  MathSciNet  Google Scholar 

  • Chen, S. S., Donoho, D. L., & Saunders, M. A. (1999). Atomic decomposition by basis pursuit. SIAM Journal on Scientific Computing, 20(1), 33–61.

    Article  MATH  MathSciNet  Google Scholar 

  • Ciarlet, P. (1989). Introduction to numerical linear algebra and optimisation. Cambridge: Cambridge University Press.

    Google Scholar 

  • Combettes, P. L., & Pesquet, J. C. (2007). Proximal thresholding algorithm for minimization over orthonormal bases. SIAM Journal on Optimization, 18(4), 1351–1376.

    Article  MATH  MathSciNet  Google Scholar 

  • Combettes, P., & Wajs, V. (2005). Signal recovery by proximal forward-backward splitting. SIAM Journal on Multiscale Modeling and Simulation, 4(4), 1168–1200.

    Article  MATH  MathSciNet  Google Scholar 

  • Daubechies, I., Defrise, M., & Mol, C. D. (2004). An iterative thresholding algorithm for linear inverse problem with sparsity constraint. Communication on Pure and Applied Mathematics, 57(11), 1413–1457.

    Article  MATH  Google Scholar 

  • Donoho, D. (2005). Neighborly polytopes and sparse solution of underdetermined linear equations (Tech. Rep. 2005-04). Dept. of Statistics, Stanford University.

  • Donoho, D. (2006). Compressed sensing. IEEE Transactions on Information Theory, 52(4), 1289–1306.

    Article  MathSciNet  Google Scholar 

  • Donoho, D., & Tanner, J. (2005). Sparse nonnegative solution of underdetermined linear equations by linear programming. Proceedings of the National Academy of Sciences, 102(27), 9446–9451.

    Article  MathSciNet  Google Scholar 

  • Donoho, D., & Tsaig, Y. (2006). Fast solution of l1-norm minimization problems when the solution may be sparse (Tech. Rep. 2006-18). Stanford, Dept. of Statistics.

  • Donoho, D., Elad, M., & Temlyakov, V. (2006). Stable recovery of sparse overcomplete representation in the presence of noise. IEEE Transactions on Information Theory, 52, 6–18.

    Article  MathSciNet  Google Scholar 

  • Efron, B., Hastie, T., Johnstone, I., & Tibshirani, R. (2004). Least angle regression. Annals of Statistics, 32(2), 407–499.

    Article  MATH  MathSciNet  Google Scholar 

  • Elad, M. (2006). Why simple shrinkage is still relevant for redundant transforms. IEEE Transactions on Information Theory, 52(12), 5559–5569.

    Article  MathSciNet  Google Scholar 

  • Elad, M., Matalon, B., & Zibulevsky, M. (2007). Coordinate and subspace optimization methods for linear least squares with non-quadratic regularization. Journal on Applied and Computational Harmonic Analysis, 23, 346–367.

    Article  MATH  MathSciNet  Google Scholar 

  • Figueiredo, M., & Nowak, R. (2003). An em algorithm for wavelet-based image restoration. IEEE Transactions on Image Processing, 12(8), 906–916.

    Article  MathSciNet  Google Scholar 

  • Figueiredo, M., & Nowak, R. (2005). A bound optimization approach to wavelet-based image deconvolution. In ICIP 2005 (Vol. 2, pp. 782–785).

  • Figueiredo, M., Nowak, R., & Wright, S. (2007). Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems. IEEE Journal of Selected Topics in Signal Processing, 1(4), 586–598.

    Article  Google Scholar 

  • Figueiredo, M., Nowak, R., & Wright, S. (2007). Gpsr 5.0. Matlab toolbox. Available at http://www.lx.it.pt/~mtf/GPSR/.

  • Güler, O. (1991). On the convergence of the proximal point algorithm for convex minimization. SIAM Journal of Control and Optimization, 29(2), 403–419.

    Article  MATH  Google Scholar 

  • Hale, E., Yin, W., & Zhang, Y. (2007). A fixed-point continuation method for l1-regularized minimization with applications to compressed sensing (CAAM TR07-07). Rice University.

  • Kim, S. J., Koh, K., Lustig, M., Boyd, S., & Gorinevsky, D. (2007). A method for large-scale l1-regularized least squares. IEEE Journal on Selected Topics in Signal Processing, 1(4), 606–617.

    Article  Google Scholar 

  • Lemarechal, C., & Sagastizabal, C. (1997). Practical aspects of the Moreau-Yoshida regularization 1: theoretical properties. SIAM Journal of Optimization, 7, 867–895.

    Article  MathSciNet  Google Scholar 

  • Malgouyres, F. (2006). Projecting onto a polytope simplifies data distributions (Tech. Rep. 2006-1). University Paris 13.

  • Malgouyres, F. (2007). Rank related properties for basis pursuit and total variation regularization. Signal Processing, 87(11), 2695–2707.

    Article  Google Scholar 

  • Malgouyres, F. (2008) Codes and scripts on basis pursuit denoising. http://www.math.univ-paris13.fr/~malgouy/software/index.html.

  • Maria, S., & Fuchs, J. (2006). Application of the global matched filter to stap data: an efficient algorithmic approach. In Proceedings of ICASSP 2006 (Vol. 4, pp. 1013–1016). Toulouse, France.

  • Nesterov, Y. (2004). Introductory lectures on convex optimization: A basic course. Norwell: Kluwer Academic.

    MATH  Google Scholar 

  • Rockafellar, R. (1970). Convex analysis. Princeton: Princeton University Press.

    MATH  Google Scholar 

  • Rockafellar, R. (1976). Monotone operators and the proximal point algorithm. SIAM Journal of Control and Optimization, 14(5), 877–898.

    Article  MATH  MathSciNet  Google Scholar 

  • Sardy, S., Bruce, A., & Tseng, P. (2000). Block coordinate relaxation methods for nonparametric wavelet denoising. Journal of Computational and Graphical Statistics, 9(2), 361–379.

    Article  MathSciNet  Google Scholar 

  • Starck, J. L., Elad, M., & Donoho, D. (2005). Image decomposition via the combination of sparse representations and a variational approach. IEEE Transactions on Image Processing, 14(10), 1570–1582.

    Article  Google Scholar 

  • Zeng, T. (2007). Études de modèles variationnels et apprentissage de dictionnaires. Ph.D. thesis, Université Paris 13.

Download references

Author information

Authors and Affiliations

  1. LAGA, CNRS UMR 7539, Université Paris 13, 99, avenue Jean-Baptiste Clement, 93430, Villetaneuse, France

    F. Malgouyres

  2. Dept. of Mathematics and Institute for Computational Mathematics, Hong Kong Baptist University, Kowloon Tong, Hong Kong

    T. Zeng

Authors
  1. F. Malgouyres
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. T. Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to F. Malgouyres.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Malgouyres, F., Zeng, T. A Predual Proximal Point Algorithm Solving a Non Negative Basis Pursuit Denoising Model. Int J Comput Vis 83, 294–311 (2009). https://doi.org/10.1007/s11263-009-0227-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-009-0227-z

Keywords

Search

Navigation