Skip to main content
SpringerLink
Log in

Damped Dynamical Systems for Solving Equations and Optimization Problems

  • Reference work entry
  • First Online:
Handbook of the Mathematics of the Arts and Sciences

  • 273 Accesses

Abstract

We present an approach for solving optimization problems with or without constrains which we call Dynamical Functional Particle Method (DFMP). The method consists of formulating the optimization problem as a second order damped dynamical system and then applying symplectic method to solve it numerically. In the first part of the chapter, we give an overview of the method and provide necessary mathematical background. We show that DFPM is a stable, efficient, and given the optimal choice of parameters, competitive method. Optimal parameters are derived for linear systems of equations, linear least squares, and linear eigenvalue problems. A framework for solving nonlinear problems is developed and numerically tested. In the second part, we adopt the method to several important applications such as image analysis, inverse problems for partial differential equations, and quantum physics. At the end, we present open problems and share some ideas of future work on generalized (nonlinear) eigenvalue problems, handling constraints with reflection, global optimization, and nonlinear ill-posed problems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

References

  • Abdullaev F Kh, Ögren M, Sørensen MP (2018, Submitted) Collective dynamics of Fermi-Bose mixtures with an oscillating scattering length

    Google Scholar 

  • Afraites L, Dambrine M, Kateb D (2007) Conformal mappings and shape derivatives for the transmission problem with a single measurement. Numer Func Anal Opt 28:519–551

    Article  MATH  Google Scholar 

  • Alvarez F (2000) On the minimizing property of a second order dissipative system in Hilbert spaces. SIAM J Control Opt 38(4):1102–1119

    Article  MathSciNet  MATH  Google Scholar 

  • Alvarez F, Attouch H, Bolte J, Redont P (2002) A second-order gradient-like dissipative dynamical system with hessian-driven damping. J de Mathematiques Pures et Applicuees 81(8):747–779

    Article  MATH  Google Scholar 

  • Andersen HC (1983) Rattle: a velocity version of the shake algorithm for molecular dynamics calculations. J Comput Phys 52:24–34

    Article  MATH  Google Scholar 

  • Ascher U, van den DK, Huang H (2007) Artificial time integration. BIT 47:3–25

    Google Scholar 

  • Attouch H, Alvarez F (2000) The heavy ball with friction dynamical system for convex constrained minimization problems. Lect Notes Econ Math Syst 481:25–35

    Article  MathSciNet  MATH  Google Scholar 

  • Attouch H, Chbani Z (2016) Combining fast inertial dynamics for convex optimization with Tikhonov regularization. 39(2). arXiv:1602.01973

    Google Scholar 

  • Attouch H, Goudou X, Redont P (2000) The heavy ball with friction method, I. The continuous dynamical system: global exploration of the local minima of a real-valued function by asymptotic analysis of a dissipative dynamical system. Commun Contemp Math 2(1):1–34

    Article  MathSciNet  MATH  Google Scholar 

  • Baravdish G, Svensson O, Åström F (2015) On backward p(x)-parabolic equations for image enhancement. Numer Funct Anal Optim 36(2):147–168

    Article  MathSciNet  MATH  Google Scholar 

  • Baravdish G, Svensson O, Gulliksson M, Zhang Y (2018) A damped flow for image denoising. ArXiv e-prints

    MATH  Google Scholar 

  • Begout P, Bolte J, Jendoubi M (2015) On damped second-order gradient systems. J Differ Equ 259(7):3115–3143

    Article  MathSciNet  MATH  Google Scholar 

  • Bertsekas DP (2015) Convex optimization algorithms. Athena Scientific, Belmont

    MATH  Google Scholar 

  • Bhatt A, Floyd D, Moore BE (2016) Second order conformal symplectic schemes for damped Hamiltonian systems. J Sci Comput 66(3):1234–1259

    Article  MathSciNet  MATH  Google Scholar 

  • Chadan K, Colton D, Paivarinta L, Rundell W (1997) An introduction to inverse scattering and inverse spectral problems. SIAM, Philadelphia

    Book  MATH  Google Scholar 

  • Cheng X, Gong R, Han W, Zheng W (2014) A novel coupled complex boundary method for inverse source problems. Inverse Prob 30:055002

    Article  MathSciNet  MATH  Google Scholar 

  • Cheng X, Lin G, Zhang Y, Gong R, Gulliksson M (2018) A modified coupled complex boundary method for an inverse chromatography problem. J Inverse Ill-Posed Prob 26:33–49

    Article  MathSciNet  MATH  Google Scholar 

  • Chu MT (2008) Numerical linear algebra algorithms as dynamical systems. Acta Numer 17:1–86

    Article  MathSciNet  MATH  Google Scholar 

  • Edvardsson S, Neuman M, Edström P, Olin H (2015) Solving equations through particle dynamics. Comput Phys Commun 197:169–181

    Article  MathSciNet  MATH  Google Scholar 

  • Engl H, Hanke M, Neubauer A (1996) Regularization of inverse problems, vol 375. Springer, New York

    Book  MATH  Google Scholar 

  • Gulliksson M (2017) The discrete dynamical functional particle method for solving constrained optimization problems. Dolomites Res Notes Approx 10:6–12

    MathSciNet  MATH  Google Scholar 

  • Gulliksson M, Edvardsson S, Persson J (2012) The dynamical functional particle method: an approach for boundary value problems. J Appl Mech 79(2):021012

    Article  Google Scholar 

  • Gulliksson M, Edvardsson S, Lind A (2013) The dynamical functional particle method. ArXiv e-prints, 2013

    Google Scholar 

  • Gulliksson M, Holmbom A, Persson J, Zhang Y (2018) A separating oscillation method of recovering the g-limit in standard and non-standard homogenization problems. Inverse Prob 32:025005

    Article  MathSciNet  MATH  Google Scholar 

  • Hairer E, Lubich C, Wanner G (2006) Geometric numerical integration, 2nd edn. Springer, Berlin/Heidelberg

    MATH  Google Scholar 

  • Han W, Cong W, Wang G (2006) Mathematical theory and numerical analysis of bioluminescence tomography. Inverse Prob 22:1659–1675

    Article  MathSciNet  MATH  Google Scholar 

  • Kaltenbacher B, Neubauer A, Scherzer O (2008) Iterative regularization methods for nonlinear ill-posed problems. Walter de Gruyter GmbH & Co. KG, Berlin

    Book  MATH  Google Scholar 

  • Karafyllis I, Grüne L (2013) Lyapunov function based step size control for numerical ode solvers with application to optimization algorithms. In: Hüper K, Trumpf J (eds) Mathematical system theory – festschrift in honor of Uwe Helmke on the occasion of his 60th birthday. CreateSpace, pp 183–210. http://num.math.unibayreuth.de/de/publications/2013/gruene_karafyllis_lyapunov_function_based_step_size_control_2013/index.html

  • Kaufman D, Pai D (2012) Geometric numerical integration of inequality constrained, nonsmooth hamiltonian systems. SIAM J Sci Comput 34(5):A2670–A2703

    Article  MathSciNet  MATH  Google Scholar 

  • Lin G, Cheng X, Zhang Y (2018a) A parametric level set based collage method for an inverse problem in elliptic partial differential equations. J Comput Appl Math 340:101–121

    Article  MathSciNet  MATH  Google Scholar 

  • Lin G, Zhang Y, Cheng X, Gulliksson M, Forssen P, Fornstedt T (2018b) A regularizing Kohn-Vogelius formulation for the model-free adsorption isotherm estimation problem in chromatography. Appl Anal 97:13–40

    Article  MathSciNet  MATH  Google Scholar 

  • Lions J, Magenes E (1972) Non-homogeneous boundary value problems and applications, vol I. Springer, Berlin

    Book  MATH  Google Scholar 

  • Mclachlan R, Reinout G, Quispel W (2006) Geometric integrators for odes. J Phys A 39: 5251–5285

    Article  MathSciNet  MATH  Google Scholar 

  • McLachlan R, Modin K, Verdier O, Wilkins M (2014) Geometric generalisations of shake and rattle. Found Comput Math J Soc Found Comput Math 14(2):339

    Article  MathSciNet  MATH  Google Scholar 

  • Nesterov Y (1983) A method of solving a convex programming problem with convergence rate. Sov Math Doklady 27:372–376

    MATH  Google Scholar 

  • Neubauer A (2000) On Landweber iteration for nonlinear ill-posed problems in Hilbert scales. Numer Math 85:309–328

    Article  MathSciNet  MATH  Google Scholar 

  • Neubauer A (2017) On Nesterov acceleration for Landweber iteration of linear ill-posed problems. J Inverse Ill-Posed Prob 25:381–390

    MathSciNet  MATH  Google Scholar 

  • Neuman M, Edvardsson S, Edström P (2015) Solving the radiative transfer equation with a mathematical particle method. Opt Lett 40(18):4325–4328

    Article  Google Scholar 

  • Poljak BT (1964) Some methods of speeding up the convergence of iterative methods. Akademija Nauk SSSR. Zurnal Vycislitel nli Matematiki i Matematicoskoi Fiziki 4:791

    Google Scholar 

  • Rieder A (2005) Runge-Kutta integrators yield optimal regularization schemes. Inverse Prob 21:453–471

    Article  MathSciNet  MATH  Google Scholar 

  • Roubíček T (2013) Nonlinear partial differential equations with applications, vol 153. Springer Science & Business Media, Basel

    MATH  Google Scholar 

  • Sandin P, Ögren M, Gulliksson M (2016) Numerical solution of the stationary multicomponent nonlinear schrödinger equation with a constraint on the angular momentum. Phys Rev E 93:033301

    Article  MathSciNet  Google Scholar 

  • Sandro I, Valerio P, Francesco Z (1979) A new method for solving nonlinear simultaneous equations. SIAM J Numer Anal 16(5):779–11. 10

    Google Scholar 

  • Scherzer O, Grasmair M, Grossauer H, Haltmeier M, Lenzen F (2009) Variational methods in imaging. Springer, New York

    MATH  Google Scholar 

  • Schock E (1985) Approximate solution of ill-posed equations: arbitrarily slow convergence vs. superconvergence. Construct Methods Pract Treat Integral Equ 73:234–243

    Article  MathSciNet  MATH  Google Scholar 

  • Smyrlis G, Zisis V (2004) Local convergence of the steepest descent method in Hilbert spaces. J Math Anal Appl 300(2):436–453

    Article  MathSciNet  MATH  Google Scholar 

  • Song S, Huang J (2012) Solving an inverse problem from bioluminescence tomography by minimizing an energy-like functional. J Comput Anal Appl 14:544–558

    MathSciNet  MATH  Google Scholar 

  • Tautenhahn U (1994) On the asymptotical regularization of nonlinear ill-posed problems. Inverse Prob 10:1405–1418

    Article  MathSciNet  MATH  Google Scholar 

  • Tikhonov A, Leonov A, Yagola A (1998) Nonlinear ill-posed problems, vol I and II. Chapman and Hall, London

    MATH  Google Scholar 

  • Tsai C-C, Liu C-S, Yeih W-C (2010) Fictious time integration method of fundamental solutions with Chebyshev polynomials for solving Poisson-type nonlinear pdes. CMES 56(2):131–151

    MathSciNet  MATH  Google Scholar 

  • Vainikko G, Veretennikov A (1986) Iteration procedures in ill-posed problems. Moscow: Nauka (In Russian)

    Google Scholar 

  • Wang Y, Zhang Y, Lukyanenko D, Yagola A (2012) A method of restoring the aerosol particle size distribution function on the set of piecewise-convex functions. Vychislitelnye Metody i Programmirovanie 13:49–66

    Google Scholar 

  • Wang Y, Zhang Y, Lukyanenko D, Yagola A (2013) Recovering aerosol particle size distribution function on the set of bounded piecewise-convex functions. Inverse Prob Sci Eng 21:339–354

    Article  MathSciNet  MATH  Google Scholar 

  • Watson L, Sosonkina M, Melville R, Morgan A, Walker H (1997) Alg 777:hompack90: a suite of fortan 90 codes for globally convergent homotopy algorithms. ACM Trans Math Softw 23(4):514–549

    Article  MATH  Google Scholar 

  • Yao Z, Zhang Y, Bai Z, Eddy WF (2018) Estimating the number of sources in magnetoencephalography using spiked population eigenvalues. J Am Stat Assoc 113(522):505–518

    Article  MathSciNet  MATH  Google Scholar 

  • Zhang Y, Hofmann B (2018) On the second order asymptotical regularization of linear ill-posed inverse problems. Applicable Analysis, pp 1–26. https://doi.org/10.1080/00036811.2018.1517412

  • Zhang Y, Lukyanenko D, Yagola A (2013) Using Lagrange principle for solving linear ill-posed problems with a priori information. Vychislitelnye Metody i Programmirovanie 14:468–482

    Google Scholar 

  • Zhang Y, Lukyanenko D, Yagola A (2015) An optimal regularization method for convolution equations on the sourcewise represented set. J Inverse Ill-Posed Prob 23:465–475

    MathSciNet  MATH  Google Scholar 

  • Zhang Y, Gulliksson M, Hernandez Bennetts V, Schaffernicht E (2016a) Reconstructing gas distribution maps via an adaptive sparse regularization algorithm. Inverse Prob Sci Eng 24:1186–1204

    Article  MathSciNet  MATH  Google Scholar 

  • Zhang Y, Lin G, Forssen P, Gulliksson M, Fornstedt T, Cheng X (2016b) A regularization method for the reconstruction of adsorption isotherms in liquid chromatography. Inverse Prob 32:105005

    Article  MathSciNet  MATH  Google Scholar 

  • Zhang Y, Lukyanenko D, Yagola A (2016c) Using Lagrange principle for solving two-dimensional integral equation with a positive kernel. Inverse Prob Sci Eng 24:811–831

    Article  MathSciNet  MATH  Google Scholar 

  • Zhang Y, Forssen P, Fornstedt T, Gulliksson M, Dai X (2017a) An adaptive regularization algorithm for recovering the rate constant distribution from biosensor data. Inverse Prob Sci Eng 24:1–26

    MATH  Google Scholar 

  • Zhang Y, Lin G, Forssen P, Gulliksson M, Fornstedt T, Cheng X (2017b) An adjoint method in inverse problems of chromatography. Inverse Prob Sci Eng 25:1112–1137

    Article  MathSciNet  MATH  Google Scholar 

  • Zhang Y, Gong R, Cheng X, Gulliksson M (2018a) A dynamical regularization algorithm for solving inverse source problems of elliptic partial differential equations. Inverse Prob 34:065001

    Article  MathSciNet  MATH  Google Scholar 

  • Zhang Y, Gong R, Gulliksson M, Cheng X (2018b) A coupled complex boundary expanding compacts method for inverse source problems. J Inverse Ill-Posed Prob, pp 1–20. https://doi.org/10.1515/jiip-2017-0002

Download references

Author information

Authors and Affiliations

  1. Mathematics, School of Engineering and Technology, Örebro, Sweden

    Mårten Gulliksson & Magnus Ögren

  2. Department of Mathematics, University of Bergen, Bergen, Norway

    Anna Oleynik

  3. Faculty of Mathematics, Chemnitz University of Technology, Chemnitz, Germany

    Ye Zhang

Authors
  1. Mårten Gulliksson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Magnus Ögren
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anna Oleynik
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ye Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mårten Gulliksson .

Editor information

Editors and Affiliations

  1. Department of Mathematical Sciences, University of Montana, Missoula, MT, USA

    Bharath Sriraman

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this entry

Check for updates. Verify currency and authenticity via CrossMark

Cite this entry

Gulliksson, M., Ögren, M., Oleynik, A., Zhang, Y. (2021). Damped Dynamical Systems for Solving Equations and Optimization Problems. In: Sriraman, B. (eds) Handbook of the Mathematics of the Arts and Sciences. Springer, Cham. https://doi.org/10.1007/978-3-319-57072-3_32

Download citation

Publish with us

Policies and ethics

Search

Navigation