Computational Optimization and Applications

, Volume 48, Issue 2, pp 233–253 | Cite as

Convex constrained optimization for large-scale generalized Sylvester equations

Article

Abstract

We propose and study the use of convex constrained optimization techniques for solving large-scale Generalized Sylvester Equations (GSE). For that, we adapt recently developed globalized variants of the projected gradient method to a convex constrained least-squares approach for solving GSE. We demonstrate the effectiveness of our approach on two different applications. First, we apply it to solve the GSE that appears after applying left and right preconditioning schemes to the linear problems associated with the discretization of some partial differential equations. Second, we apply the new approach, combined with a Tikhonov regularization term, to restore some blurred and highly noisy images.

Keywords

Convex optimization Spectral projected gradient method Generalized Sylvester equation Image restoration 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Ahmed, N.V., Teo, K.L.: Optimal Control of Distributed Parameter Systems. North-Holland, Amsterdam (1981) MATHGoogle Scholar
  2. 2.
    Anderson, G.L., Netravali, A.N.: Image restoration based on a subjective criterion. IEEE Trans. Syst. Man Cybern. 6, 845–856 (1976) CrossRefGoogle Scholar
  3. 3.
    Bartels, R.H., Stewart, G.W.: Solution of the matrix equation AX+XB=C. Commun. ACM 15, 820–826 (1972) CrossRefGoogle Scholar
  4. 4.
    Barzilai, J., Borwein, J.M.: Two point step size gradient methods. IMA J. Numer. Anal. 8, 141–148 (1988) CrossRefMATHMathSciNetGoogle Scholar
  5. 5.
    Bertsekas, D.P.: On The Goldstein-Levitin-Polyak gradient projection method. IEEE Trans. Automat. Contr. 21, 174–184 (1976) CrossRefMATHMathSciNetGoogle Scholar
  6. 6.
    Birgin, E.G., Martínez, J.M., Raydan, M.: Nonmonotone spectral projected gradient methods on convex sets. SIAM J. Opt. 10, 1196–1211 (2000) CrossRefMATHGoogle Scholar
  7. 7.
    Birgin, E.G., Martínez, J.M., Raydan, M.: Algorithm 813: SPG—software for convex-constrained optimization. ACM Trans. Math. Softw. 27, 340–349 (2001) CrossRefMATHGoogle Scholar
  8. 8.
    Birgin, E.G., Martínez, J.M., Raydan, M.: Inexact spectral gradient method for convex-constrained optimization. IMA J. Numer. Anal. 23, 539–559 (2003) CrossRefMATHMathSciNetGoogle Scholar
  9. 9.
    Bouhamidi, A., Jbilou, K.: Sylvester Tikhonov-regularization methods in image restoration. J. Comput. Math. 206, 86–98 (2007) CrossRefMATHMathSciNetGoogle Scholar
  10. 10.
    Boyle, J.P., Dykstra, R.L.: A method for finding projections onto the intersection of convex sets in Hilbert spaces. Lect. Notes Stat. 37, 28–47 (1986) MathSciNetGoogle Scholar
  11. 11.
    Calvetti, D., Levenberg, N., Reichel, L.: Iterative methods for XAXB=C. J. Comput. Appl. Math. 86, 73–101 (1997) CrossRefMATHMathSciNetGoogle Scholar
  12. 12.
    Calvetti, D., Golub, G.H., Reichel, L.: Estimation of the L-curve via Lanczos bidiagonalization. BIT 39, 603–619 (1999) CrossRefMATHMathSciNetGoogle Scholar
  13. 13.
    Calvetti, D., Lewis, B., Reichel, L.: GMRES, L-curves and discrete ill-posed problems. BIT 42, 44–65 (2002) CrossRefMATHMathSciNetGoogle Scholar
  14. 14.
    Castellanos, J.L., Gómez, S., Guerra, V.: The triangle method for finding the corner of the L-curve. Appl. Numer. Math. 43, 359–373 (2002) CrossRefMATHMathSciNetGoogle Scholar
  15. 15.
    Chan, T.F., Shen, J.: Image Processing and Analysis: Variational, PDE, Wavelet, and Stochastic Methods. SIAM, Philadelphia (2005) MATHGoogle Scholar
  16. 16.
    Dai, Y.H., Liao, L.Z.: R-linear convergence of the Barzilai and Borwein gradient method. IMA J. Numer. Anal. 22, 1–10 (2002) CrossRefMATHMathSciNetGoogle Scholar
  17. 17.
    Datta, B.N.: Numerical Methods for Linear Control Systems Design and Analysis. Elsevier, Amsterdam (2003) Google Scholar
  18. 18.
    Demmel, J.W., Kågström, B.: Accurate solutions of ill-posed problems in control theory. SIAM J. Matrix Anal. Appl. 9, 126–145 (1988) CrossRefMATHMathSciNetGoogle Scholar
  19. 19.
    Dou, A.: Method of undetermined coefficients in linear differential systems and the matrix equation YAAY=F. SIAM J. Appl. Math. 14, 691–696 (1966) CrossRefMATHMathSciNetGoogle Scholar
  20. 20.
    El Guennouni, A., Jbilou, K., Riquet, A.J.: Block Krylov subspace methods for solving large Sylvester equations. Numer. Algorithms 29, 75–96 (2002) CrossRefMATHMathSciNetGoogle Scholar
  21. 21.
    Epton, M.A.: Methods for the solution of AXDBXC=E and its application in the numerical solution of implicit ordinary differential equations. BIT 20, 341–345 (1980) CrossRefMATHMathSciNetGoogle Scholar
  22. 22.
    Fletcher, R.: Low storage methods for unconstrained optimization. Lect. Appl. Math. 26, 165–179 (1990) MathSciNetGoogle Scholar
  23. 23.
    Fletcher, R.: On the Barzilai-Borwein method. In: Qi, L., Teo, K.L., Yang, X.Q. (eds.) Optimization and Control with Applications, pp. 235–256. Springer, Berlin (2005) CrossRefGoogle Scholar
  24. 24.
    Goldstein, A.A.: Convex Programming in Hilbert Space. Bull. Am. Math. Soc. 70, 709–710 (1964) CrossRefMATHGoogle Scholar
  25. 25.
    Golub, G.H., von Matt, U.: Tikhonov regularization for large scale problems. In: Golub, G.H., Lui, S.H., Luk, F., Plemmons, R. (eds.) Workshop on Scientific Computing, pp. 3–26. Springer, New York (1997) Google Scholar
  26. 26.
    Golub, G.H., Heath, M., Wahba, G.: Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics 21, 215–223 (1979) CrossRefMATHMathSciNetGoogle Scholar
  27. 27.
    Golub, G.H., Nash, S., Van Loan, C.: A Hessenberg-Schur method the problem AX+XB=C. IEEE Trans. Automat. Contr. AC 24, 909–913 (1979) CrossRefMATHGoogle Scholar
  28. 28.
    Grippo, L., Lampariello, F., Lucidi, S.: A nonmonotone line search technique for Newton’s method. SIAM J. Numer. Anal. 23, 707–716 (1986) CrossRefMATHMathSciNetGoogle Scholar
  29. 29.
    Hammarling, S.J.: Numerical solution of the stable, nonnegative definite Lyapunov equations. IMA J. Numer. Anal. 2, 303–323 (1982) CrossRefMATHMathSciNetGoogle Scholar
  30. 30.
    Hanke, M., Hansen, P.C.: Regularization methods for large-scale problems. Surv. Math. Ind. 3, 253–315 (1993) MATHMathSciNetGoogle Scholar
  31. 31.
    Hansen, P.C.: Analysis of discrete ill-posed problems by means of the L-curve. SIAM Rev. 34, 561–580 (1992) CrossRefMATHMathSciNetGoogle Scholar
  32. 32.
    Hansen, P.C., Nagy, J.G., O’Leary, D.P.: Deblurring Images: Matrices, Spectra, and Filtering. SIAM, Philadelphia (2006) MATHGoogle Scholar
  33. 33.
    Hu, D.Y., Reichel, L.: Krylov subspace methods for the Sylvester equation. Linear Algebra Appl. 174, 283–314 (1992) CrossRefMathSciNetGoogle Scholar
  34. 34.
    Jaimoukha, I.M., Kasenally, E.M.: Krylov subspace methods for solving large Lyapunov equations. SIAM J. Matrix Anal. Appl. 31, 227–251 (1994) MATHMathSciNetGoogle Scholar
  35. 35.
    Jbilou, K.: Low rank approximate solutions to large Sylvester matrix equations. Appl. Math. Comput. 177, 365–376 (2006) CrossRefMATHMathSciNetGoogle Scholar
  36. 36.
    Jbilou, K., Riquet, A.: Projection methods for large Lyapunov matrix equations. Linear Algebra Appl. 415, 344–358 (2006) CrossRefMATHMathSciNetGoogle Scholar
  37. 37.
    Jbilou, K., Messaoudi, A., Sadok, H.: Global FOM and GMRES algorithms for matrix equations. Appl. Numer. Math. 31, 49–63 (1999) CrossRefMATHMathSciNetGoogle Scholar
  38. 38.
    La Cruz, W., Martínez, J.M., Raydan, M.: Spectral residual method without gradient information for solving large-scale nonlinear systems of equations. Math. Comput. 75, 1449–1466 (2006) CrossRefGoogle Scholar
  39. 39.
    Levitin, E.S., Polyak, B.T.: Constrained minimization problems. U.S.S.R. Comput. Math. Math. Phys. 6, 1–50 (1966) CrossRefGoogle Scholar
  40. 40.
    Lions, J.L.: Optimal Control of Systems Governed by Partial Differential Equations. Springer, Berlin (1971) MATHGoogle Scholar
  41. 41.
    Lucy, L.B.: An iterative technique for the rectification of observed distributions. Astron. J. 79, 745–754 (1974) CrossRefGoogle Scholar
  42. 42.
    Monsalve, M.: Block linear method for large scale Sylvester equations. Comput. Appl. Math. 27, 47–59 (2008) MATHMathSciNetGoogle Scholar
  43. 43.
    Raydan, M.: On the Barzilai and Borwein choice of steplength for the gradient method. IMA J. Numer. Anal. 13, 321–326 (1993) CrossRefMATHMathSciNetGoogle Scholar
  44. 44.
    Richardson, W.H.: Bayesian-based iterative method of image restoration. J. Opt. Soc. Am. 62, 55–59 (1972) CrossRefGoogle Scholar
  45. 45.
    Simoncini, V.: On the numerical solution of AXXB=C. BIT 36, 814–830 (1996) CrossRefMATHMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  1. 1.L.M.P.A., Université du LittoralCalais-CedexFrance
  2. 2.Departamento de Cómputo Científico y EstadísticaUniversidad Simón Bolívar (USB)CaracasVenezuela

Personalised recommendations