A preconditioning technique for a class of PDE-constrained optimization problems



We investigate the use of a preconditioning technique for solving linear systems of saddle point type arising from the application of an inexact Gauss–Newton scheme to PDE-constrained optimization problems with a hyperbolic constraint. The preconditioner is of block triangular form and involves diagonal perturbations of the (approximate) Hessian to insure nonsingularity and an approximate Schur complement. We establish some properties of the preconditioned saddle point systems and we present the results of numerical experiments illustrating the performance of the preconditioner on a model problem motivated by image registration.


Constrained optimization KKT conditions Saddle point problems Hyperbolic PDEs Krylov subspace methods Preconditioning Monge–Kantorovich problem Image registration 

Mathematics Subject Classifications (2010)

65F08 65F22 49M05 49M15 90C30 


  1. 1.
    Akcelik, V., Biros, G., Ghattas, O.: Parallel multiscale Gauss–Newton–Krylov methods for inverse wave propagation. In: Proceedings of the IEEE/ACM Conference, pp. 1–15 (2002)Google Scholar
  2. 2.
    Akcelik, V., Biros, G., Ghattas, O., et al.: High resolution forward and inverse earthquake modeling on terascale computers. In: Proceedings of the IEEE/ACM Conference, pp. 1–52 (2003)Google Scholar
  3. 3.
    Angenent, S., Haker, S., Tannenbaum, A.: Minimizing flows for the Monge–Kantorovich problem. SIAM J. Math. Anal. 35, 61–97 (2003)MathSciNetMATHCrossRefGoogle Scholar
  4. 4.
    Arridge, S.R.: Optical tomography in medical imaging. Inverse Probl. 15, R41–R93 (1999)MathSciNetMATHCrossRefGoogle Scholar
  5. 5.
    Ascher, U.M., Haber, E.: Grid refinement and scaling for distributed parameter estimation problems. Inverse Probl. 17, 571–590 (2001)MathSciNetMATHCrossRefGoogle Scholar
  6. 6.
    Benamou, J.D., Brenier, Y.: A computational fluid mechanics solution to the Monge-Kantorovich mass transfer problem. Numer. Math. 84, 375–393 (2000)MathSciNetMATHCrossRefGoogle Scholar
  7. 7.
    Benzi, M., Golub, G.H., Liesen, J.: Numerical solution of saddle point problems. Acta Numer. 14, 1–137 (2005)MathSciNetMATHCrossRefGoogle Scholar
  8. 8.
    Biegler, L., Ghattas, O., Heinkenschloss, M., Waanders, B.: Large-scale PDE-constrained optimization. In: Lecture Notes in Computational Science and Engineering, vol. 30. Springer, New York (2003)Google Scholar
  9. 9.
    Biros, G., Ghattas, O.; Parallel Lagrange–Newton–Krylov–Schur methods for PDE-constrained optimization. Parts I–II. SIAM J. Sci. Comput. 27, 687–738 (2005)MathSciNetMATHCrossRefGoogle Scholar
  10. 10.
    Bouchouev, I., Isakov, V.: Uniqueness, stability and numerical methods for the inverse problem that arises in financial markets. Inverse Probl. 15, R95–R116 (1999)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Casanova, R., Silva, A., Borges, A.R.: A quantitative algorithm for parameter estimation in magnetic induction tomography. Meas. Sci. Technol. 15, 1412–1419 (2004)CrossRefGoogle Scholar
  12. 12.
    Cheney, M., Isaacson, D., Newell, J.C.: Electrical impedance tomography. SIAM Rev. 41, 85–101 (1999)MathSciNetMATHCrossRefGoogle Scholar
  13. 13.
    Deuflhard, P., Potra, F.: Asymptotic mesh independence of Newton-Galerkin methods via a refined Mysovskii theorem. SIAM J. Numer. Anal. 29, 1395–1412 (1992)MathSciNetMATHCrossRefGoogle Scholar
  14. 14.
    Dollar, H.S.: Properties of Linear Systems in PDE-Constrained Optimization. Part I: Distributed Control. Tech. Rep. RAL-TR-2009-017, Rutherford Appleton Laboratory (2009)Google Scholar
  15. 15.
    Dupire, B.: Pricing with a smile. Risk 7, 32–39 (1994)Google Scholar
  16. 16.
    Egger, H., Engl, H.W.: Tikhonov regularization applied to the inverse problem of option pricing: convergence analysis and rates. Inverse Probl. 21, 1027–1045 (2005)MathSciNetMATHCrossRefGoogle Scholar
  17. 17.
    Elman, H., Silvester, D., Wathen, A.: Finite Elements and Fast Iterative Solvers with Applications in Incompressible Fluid Dynamics. Oxford University Press, Oxford (2005)MATHGoogle Scholar
  18. 18.
    El-Qady, G., Ushijima, K.: Inversion of DC resistivity data using neural networks. Geophys. Prospect. 49, 417–430 (2001)CrossRefGoogle Scholar
  19. 19.
    Golub, G.H., Van Loan, C.F.: Matrix Computations, 3rd edn. John Hopkins University Press, Baltimore (1996)MATHGoogle Scholar
  20. 20.
    Haber, E., Ascher, U.M.: Preconditioned all-at-once methods for large, sparse parameter estimation problems. Inverse Probl. 17, 1847–1864 (2001)MathSciNetMATHCrossRefGoogle Scholar
  21. 21.
    Haber, E., Modersitzki, J.: A multilevel method for image registration. SIAM J. Sci. Comput. 27, 1594–1607 (2006)MathSciNetMATHCrossRefGoogle Scholar
  22. 22.
    Hanson, L.R.: Techniques in Constrained Optimization Involving Partial Differential Equations. PhD thesis, Emory University, Atlanta, GA (2007)Google Scholar
  23. 23.
    Hu, J., Sala, M., Tong, C., Tuminaro, R., et al.: ML: Multilevel Preconditioning Package, The Trilinos Project, Sandia National Laboratories (2006). http://trilinos.sandia.gov/packages/ml/
  24. 24.
    Kantorovich, L.V.: On the translocation of masses. Dokl. Akad. Nauk SSSR 37, 227–229 (1942) (in Russian). English translation in J. Math. Sci. 133, 1381–1382 (2006)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Kantorovich, L.V.: On a problem of Monge. Usp. Mat. Nauk 3, 225–226 (1948) (in Russian). English translation in J. Math. Sci. 133, 1383 (2006)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Klibanov, M.V., Lucas, T.R.: Numerical solution of a parabolic inverse problem in optical tomography using experimental data. SIAM J. Appl. Math. 59, 1763–1789 (1999)MathSciNetMATHCrossRefGoogle Scholar
  27. 27.
    Laird, C.D., Biegler, L.T., Waanders, B., Bartlett, R.A.: Time-dependent contaminant source determination for municipal water networks using large scale optimization. ASCE J. Water Resour. Manage. Plan. 131, 125–134 (2005)CrossRefGoogle Scholar
  28. 28.
    LeVeque, R.J.: Numerical Methods for Conservation Laws. Birkhauser, New York (1990)MATHGoogle Scholar
  29. 29.
    Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, New York (1999)MATHCrossRefGoogle Scholar
  30. 30.
    Orozco, C., Ghattas, O.; Massively parallel aerodynamic shape optimization. Comput. Syst. Eng. 1, 311–320 (1992)CrossRefGoogle Scholar
  31. 31.
    Parker, R.L.: Geophysical Inverse Theory. Princeton University Press, Princeton (1994)MATHGoogle Scholar
  32. 32.
    Rees, T., Dollar, H.S., Wathen, A.J.: Optimal solvers for PDE-constrained optimization. SIAM J. Sci. Comput. 32, 271–298 (2010)MathSciNetMATHCrossRefGoogle Scholar
  33. 33.
    Rees, T., Stoll, M.: Block-triangular preconditioners for PDE-constrained optimization. Numer. Linear Algebra Appl. 17, 977–996 (2010). doi:10.1002/nla.693 MathSciNetCrossRefGoogle Scholar
  34. 34.
    Saad, Y.: A flexible inner-outer preconditioned GMRES algorithm. SIAM J. Sci. Comput. 14, 451–469 (1993)MathSciNetCrossRefGoogle Scholar
  35. 35.
    Saad, Y., Schultz, M.H.: GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM J. Sci. Stat. Comput. 7, 856–869 (1986)MATHCrossRefGoogle Scholar
  36. 36.
    Shenoy, A., Heinkenschloss, M., Cliff, E.M.: Airfoil design by an all-at-once method. Int. J. Comput. Fluid Dyn. 11, 3–25 (1998)MathSciNetMATHCrossRefGoogle Scholar
  37. 37.
    Vogel, C.R.: Sparse matrix computations arising in distributed parameter identification. SIAM J. Matrix Anal. Appl. 20, 1027–1037 (1999)MathSciNetMATHCrossRefGoogle Scholar
  38. 38.
    Vogel, C.R.: Computational Methods for Inverse Problems, Frontiers in Applied Mathematics Series. Society for Industrial and Applied Mathematics, Philadelphia (2002)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC. 2011

Authors and Affiliations

  1. 1.Department of Mathematics and Computer ScienceEmory UniversityAtlantaUSA
  2. 2.Department of MathematicsUniversity of British ColumbiaVancouverCanada
  3. 3.Quantitative Analytics Research GroupStandard & Poor’sNew YorkUSA

Personalised recommendations