Computational Optimization and Applications

, Volume 48, Issue 2, pp 233–253 | Cite as

Convex constrained optimization for large-scale generalized Sylvester equations

  • A. Bouhamidi
  • K. Jbilou
  • M. Raydan


We propose and study the use of convex constrained optimization techniques for solving large-scale Generalized Sylvester Equations (GSE). For that, we adapt recently developed globalized variants of the projected gradient method to a convex constrained least-squares approach for solving GSE. We demonstrate the effectiveness of our approach on two different applications. First, we apply it to solve the GSE that appears after applying left and right preconditioning schemes to the linear problems associated with the discretization of some partial differential equations. Second, we apply the new approach, combined with a Tikhonov regularization term, to restore some blurred and highly noisy images.


Convex optimization Spectral projected gradient method Generalized Sylvester equation Image restoration 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ahmed, N.V., Teo, K.L.: Optimal Control of Distributed Parameter Systems. North-Holland, Amsterdam (1981) zbMATHGoogle Scholar
  2. 2.
    Anderson, G.L., Netravali, A.N.: Image restoration based on a subjective criterion. IEEE Trans. Syst. Man Cybern. 6, 845–856 (1976) CrossRefGoogle Scholar
  3. 3.
    Bartels, R.H., Stewart, G.W.: Solution of the matrix equation AX+XB=C. Commun. ACM 15, 820–826 (1972) CrossRefGoogle Scholar
  4. 4.
    Barzilai, J., Borwein, J.M.: Two point step size gradient methods. IMA J. Numer. Anal. 8, 141–148 (1988) CrossRefzbMATHMathSciNetGoogle Scholar
  5. 5.
    Bertsekas, D.P.: On The Goldstein-Levitin-Polyak gradient projection method. IEEE Trans. Automat. Contr. 21, 174–184 (1976) CrossRefzbMATHMathSciNetGoogle Scholar
  6. 6.
    Birgin, E.G., Martínez, J.M., Raydan, M.: Nonmonotone spectral projected gradient methods on convex sets. SIAM J. Opt. 10, 1196–1211 (2000) CrossRefzbMATHGoogle Scholar
  7. 7.
    Birgin, E.G., Martínez, J.M., Raydan, M.: Algorithm 813: SPG—software for convex-constrained optimization. ACM Trans. Math. Softw. 27, 340–349 (2001) CrossRefzbMATHGoogle Scholar
  8. 8.
    Birgin, E.G., Martínez, J.M., Raydan, M.: Inexact spectral gradient method for convex-constrained optimization. IMA J. Numer. Anal. 23, 539–559 (2003) CrossRefzbMATHMathSciNetGoogle Scholar
  9. 9.
    Bouhamidi, A., Jbilou, K.: Sylvester Tikhonov-regularization methods in image restoration. J. Comput. Math. 206, 86–98 (2007) CrossRefzbMATHMathSciNetGoogle Scholar
  10. 10.
    Boyle, J.P., Dykstra, R.L.: A method for finding projections onto the intersection of convex sets in Hilbert spaces. Lect. Notes Stat. 37, 28–47 (1986) MathSciNetGoogle Scholar
  11. 11.
    Calvetti, D., Levenberg, N., Reichel, L.: Iterative methods for XAXB=C. J. Comput. Appl. Math. 86, 73–101 (1997) CrossRefzbMATHMathSciNetGoogle Scholar
  12. 12.
    Calvetti, D., Golub, G.H., Reichel, L.: Estimation of the L-curve via Lanczos bidiagonalization. BIT 39, 603–619 (1999) CrossRefzbMATHMathSciNetGoogle Scholar
  13. 13.
    Calvetti, D., Lewis, B., Reichel, L.: GMRES, L-curves and discrete ill-posed problems. BIT 42, 44–65 (2002) CrossRefzbMATHMathSciNetGoogle Scholar
  14. 14.
    Castellanos, J.L., Gómez, S., Guerra, V.: The triangle method for finding the corner of the L-curve. Appl. Numer. Math. 43, 359–373 (2002) CrossRefzbMATHMathSciNetGoogle Scholar
  15. 15.
    Chan, T.F., Shen, J.: Image Processing and Analysis: Variational, PDE, Wavelet, and Stochastic Methods. SIAM, Philadelphia (2005) zbMATHGoogle Scholar
  16. 16.
    Dai, Y.H., Liao, L.Z.: R-linear convergence of the Barzilai and Borwein gradient method. IMA J. Numer. Anal. 22, 1–10 (2002) CrossRefzbMATHMathSciNetGoogle Scholar
  17. 17.
    Datta, B.N.: Numerical Methods for Linear Control Systems Design and Analysis. Elsevier, Amsterdam (2003) Google Scholar
  18. 18.
    Demmel, J.W., Kågström, B.: Accurate solutions of ill-posed problems in control theory. SIAM J. Matrix Anal. Appl. 9, 126–145 (1988) CrossRefzbMATHMathSciNetGoogle Scholar
  19. 19.
    Dou, A.: Method of undetermined coefficients in linear differential systems and the matrix equation YAAY=F. SIAM J. Appl. Math. 14, 691–696 (1966) CrossRefzbMATHMathSciNetGoogle Scholar
  20. 20.
    El Guennouni, A., Jbilou, K., Riquet, A.J.: Block Krylov subspace methods for solving large Sylvester equations. Numer. Algorithms 29, 75–96 (2002) CrossRefzbMATHMathSciNetGoogle Scholar
  21. 21.
    Epton, M.A.: Methods for the solution of AXDBXC=E and its application in the numerical solution of implicit ordinary differential equations. BIT 20, 341–345 (1980) CrossRefzbMATHMathSciNetGoogle Scholar
  22. 22.
    Fletcher, R.: Low storage methods for unconstrained optimization. Lect. Appl. Math. 26, 165–179 (1990) MathSciNetGoogle Scholar
  23. 23.
    Fletcher, R.: On the Barzilai-Borwein method. In: Qi, L., Teo, K.L., Yang, X.Q. (eds.) Optimization and Control with Applications, pp. 235–256. Springer, Berlin (2005) CrossRefGoogle Scholar
  24. 24.
    Goldstein, A.A.: Convex Programming in Hilbert Space. Bull. Am. Math. Soc. 70, 709–710 (1964) CrossRefzbMATHGoogle Scholar
  25. 25.
    Golub, G.H., von Matt, U.: Tikhonov regularization for large scale problems. In: Golub, G.H., Lui, S.H., Luk, F., Plemmons, R. (eds.) Workshop on Scientific Computing, pp. 3–26. Springer, New York (1997) Google Scholar
  26. 26.
    Golub, G.H., Heath, M., Wahba, G.: Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics 21, 215–223 (1979) CrossRefzbMATHMathSciNetGoogle Scholar
  27. 27.
    Golub, G.H., Nash, S., Van Loan, C.: A Hessenberg-Schur method the problem AX+XB=C. IEEE Trans. Automat. Contr. AC 24, 909–913 (1979) CrossRefzbMATHGoogle Scholar
  28. 28.
    Grippo, L., Lampariello, F., Lucidi, S.: A nonmonotone line search technique for Newton’s method. SIAM J. Numer. Anal. 23, 707–716 (1986) CrossRefzbMATHMathSciNetGoogle Scholar
  29. 29.
    Hammarling, S.J.: Numerical solution of the stable, nonnegative definite Lyapunov equations. IMA J. Numer. Anal. 2, 303–323 (1982) CrossRefzbMATHMathSciNetGoogle Scholar
  30. 30.
    Hanke, M., Hansen, P.C.: Regularization methods for large-scale problems. Surv. Math. Ind. 3, 253–315 (1993) zbMATHMathSciNetGoogle Scholar
  31. 31.
    Hansen, P.C.: Analysis of discrete ill-posed problems by means of the L-curve. SIAM Rev. 34, 561–580 (1992) CrossRefzbMATHMathSciNetGoogle Scholar
  32. 32.
    Hansen, P.C., Nagy, J.G., O’Leary, D.P.: Deblurring Images: Matrices, Spectra, and Filtering. SIAM, Philadelphia (2006) zbMATHGoogle Scholar
  33. 33.
    Hu, D.Y., Reichel, L.: Krylov subspace methods for the Sylvester equation. Linear Algebra Appl. 174, 283–314 (1992) CrossRefMathSciNetGoogle Scholar
  34. 34.
    Jaimoukha, I.M., Kasenally, E.M.: Krylov subspace methods for solving large Lyapunov equations. SIAM J. Matrix Anal. Appl. 31, 227–251 (1994) zbMATHMathSciNetGoogle Scholar
  35. 35.
    Jbilou, K.: Low rank approximate solutions to large Sylvester matrix equations. Appl. Math. Comput. 177, 365–376 (2006) CrossRefzbMATHMathSciNetGoogle Scholar
  36. 36.
    Jbilou, K., Riquet, A.: Projection methods for large Lyapunov matrix equations. Linear Algebra Appl. 415, 344–358 (2006) CrossRefzbMATHMathSciNetGoogle Scholar
  37. 37.
    Jbilou, K., Messaoudi, A., Sadok, H.: Global FOM and GMRES algorithms for matrix equations. Appl. Numer. Math. 31, 49–63 (1999) CrossRefzbMATHMathSciNetGoogle Scholar
  38. 38.
    La Cruz, W., Martínez, J.M., Raydan, M.: Spectral residual method without gradient information for solving large-scale nonlinear systems of equations. Math. Comput. 75, 1449–1466 (2006) CrossRefGoogle Scholar
  39. 39.
    Levitin, E.S., Polyak, B.T.: Constrained minimization problems. U.S.S.R. Comput. Math. Math. Phys. 6, 1–50 (1966) CrossRefGoogle Scholar
  40. 40.
    Lions, J.L.: Optimal Control of Systems Governed by Partial Differential Equations. Springer, Berlin (1971) zbMATHGoogle Scholar
  41. 41.
    Lucy, L.B.: An iterative technique for the rectification of observed distributions. Astron. J. 79, 745–754 (1974) CrossRefGoogle Scholar
  42. 42.
    Monsalve, M.: Block linear method for large scale Sylvester equations. Comput. Appl. Math. 27, 47–59 (2008) zbMATHMathSciNetGoogle Scholar
  43. 43.
    Raydan, M.: On the Barzilai and Borwein choice of steplength for the gradient method. IMA J. Numer. Anal. 13, 321–326 (1993) CrossRefzbMATHMathSciNetGoogle Scholar
  44. 44.
    Richardson, W.H.: Bayesian-based iterative method of image restoration. J. Opt. Soc. Am. 62, 55–59 (1972) CrossRefGoogle Scholar
  45. 45.
    Simoncini, V.: On the numerical solution of AXXB=C. BIT 36, 814–830 (1996) CrossRefzbMATHMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  1. 1.L.M.P.A., Université du LittoralCalais-CedexFrance
  2. 2.Departamento de Cómputo Científico y EstadísticaUniversidad Simón Bolívar (USB)CaracasVenezuela

Personalised recommendations