A cyclic projected gradient method
In recent years, convex optimization methods were successfully applied for various image processing tasks and a large number of first-order methods were designed to minimize the corresponding functionals. Interestingly, it was shown recently in Grewenig et al. (2010) that the simple idea of so-called “superstep cycles” leads to very efficient schemes for time-dependent (parabolic) image enhancement problems as well as for steady state (elliptic) image compression tasks. The “superstep cycles” approach is similar to the nonstationary (cyclic) Richardson method which has been around for over sixty years.
In this paper, we investigate the incorporation of superstep cycles into the projected gradient method. We show for two problems in compressive sensing and image processing, namely the LASSO approach and the Rudin-Osher-Fatemi model that the resulting simple cyclic projected gradient algorithm can numerically compare with various state-of-the-art first-order algorithms. However, due to the nonlinear projection within the algorithm convergence proofs even under restrictive assumptions on the linear operators appear to be hard. We demonstrate the difficulties by studying the simplest case of a two-cycle algorithm in ℝ2 with projections onto the Euclidean ball.
KeywordsConstrained optimization Projected gradient methods Fast explicit diffusion Image denoising Sparse recovery
- 2.Anderssen, R.S., Golub, G.H.: Richardson’s non-stationary matrix iterative procedure. Tech. rep., Stanford University, Stanford, CA, USA (1972) Google Scholar
- 21.Gentzsch, W.: Numerical solution of linear and non-linear parabolic differential equations by a time discretisation of third order accuracy. In: Hirschel, E.H. (ed.) Proceedings of the Third GAMM Conference on Numerical Methods in Fluid Dynamics, pp. 109–117. Vieweg, Wiesbaden (1979) Google Scholar
- 28.Lorenz, D.A.: Constructing test instances for basis pursuit denoising. Tech. rep., TU Braunschweig (2011). arXiv:1103.2897
- 40.Zhu, M.: Fast numerical algorithms for total variation based image restoration. Ph.D. thesis, University of California, Los Angeles, USA (2008) Google Scholar
- 41.Zhu, M., Chan, T.: An efficient primal-dual hybrid gradient algorithm for total variation image restoration. Tech. rep., UCLA, Center for Applied Math (2008) Google Scholar