Damped Dynamical Systems for Solving Equations and Optimization Problems

  • Mårten GullikssonEmail author
  • Magnus Ögren
  • Anna Oleynik
  • Ye Zhang
Living reference work entry


We present an approach for solving optimization problems with or without constrains which we call Dynamical Functional Particle Method (DFMP). The method consists of formulating the optimization problem as a second order damped dynamical system and then applying symplectic method to solve it numerically. In the first part of the chapter, we give an overview of the method and provide necessary mathematical background. We show that DFPM is a stable, efficient, and given the optimal choice of parameters, competitive method. Optimal parameters are derived for linear systems of equations, linear least squares, and linear eigenvalue problems. A framework for solving nonlinear problems is developed and numerically tested. In the second part, we adopt the method to several important applications such as image analysis, inverse problems for partial differential equations, and quantum physics. At the end, we present open problems and share some ideas of future work on generalized (nonlinear) eigenvalue problems, handling constraints with reflection, global optimization, and nonlinear ill-posed problems.


Optimization Damped dynamical systems Convex problems Eigenvalue problems Image analysis Inverse problems Quantum physics Schrödinger equation 


  1. Abdullaev F Kh, Ögren M, Sørensen MP (2018, Submitted) Collective dynamics of Fermi-Bose mixtures with an oscillating scattering lengthGoogle Scholar
  2. Afraites L, Dambrine M, Kateb D (2007) Conformal mappings and shape derivatives for the transmission problem with a single measurement. Numer Func Anal Opt 28:519–551CrossRefGoogle Scholar
  3. Alvarez F (2000) On the minimizing property of a second order dissipative system in Hilbert spaces. SIAM J Control Opt 38(4):1102–1119MathSciNetCrossRefGoogle Scholar
  4. Alvarez F, Attouch H, Bolte J, Redont P (2002) A second-order gradient-like dissipative dynamical system with hessian-driven damping. J de Mathematiques Pures et Applicuees 81(8):747–779CrossRefGoogle Scholar
  5. Andersen HC (1983) Rattle: a velocity version of the shake algorithm for molecular dynamics calculations. J Comput Phys 52:24–34CrossRefGoogle Scholar
  6. Ascher U, van den DK, Huang H (2007) Artificial time integration. BIT 47:3–25Google Scholar
  7. Attouch H, Alvarez F (2000) The heavy ball with friction dynamical system for convex constrained minimization problems. Lect Notes Econ Math Syst 481:25–35MathSciNetCrossRefGoogle Scholar
  8. Attouch H, Chbani Z (2016) Combining fast inertial dynamics for convex optimization with Tikhonov regularization. 39(2). arXiv:1602.01973Google Scholar
  9. Attouch H, Goudou X, Redont P (2000) The heavy ball with friction method, I. The continuous dynamical system: global exploration of the local minima of a real-valued function by asymptotic analysis of a dissipative dynamical system. Commun Contemp Math 2(1):1–34MathSciNetCrossRefGoogle Scholar
  10. Baravdish G, Svensson O, Åström F (2015) On backward p(x)-parabolic equations for image enhancement. Numer Funct Anal Optim 36(2):147–168MathSciNetCrossRefGoogle Scholar
  11. Baravdish G, Svensson O, Gulliksson M, Zhang Y (2018) A damped flow for image denoising. ArXiv e-printsGoogle Scholar
  12. Begout P, Bolte J, Jendoubi M (2015) On damped second-order gradient systems. J Differ Equ 259(7):3115–3143MathSciNetCrossRefGoogle Scholar
  13. Bertsekas DP (2015) Convex optimization algorithms. Athena Scientific, BelmontzbMATHGoogle Scholar
  14. Bhatt A, Floyd D, Moore BE (2016) Second order conformal symplectic schemes for damped Hamiltonian systems. J Sci Comput 66(3):1234–1259MathSciNetCrossRefGoogle Scholar
  15. Chadan K, Colton D, Paivarinta L, Rundell W (1997) An introduction to inverse scattering and inverse spectral problems. SIAM, PhiladelphiaCrossRefGoogle Scholar
  16. Cheng X, Gong R, Han W, Zheng W (2014) A novel coupled complex boundary method for inverse source problems. Inverse Prob 30:055002MathSciNetCrossRefGoogle Scholar
  17. Cheng X, Lin G, Zhang Y, Gong R, Gulliksson M (2018) A modified coupled complex boundary method for an inverse chromatography problem. J Inverse Ill-Posed Prob 26:33–49MathSciNetCrossRefGoogle Scholar
  18. Chu MT (2008) Numerical linear algebra algorithms as dynamical systems. Acta Numer 17:1–86MathSciNetCrossRefGoogle Scholar
  19. Edvardsson S, Neuman M, Edström P, Olin H (2015) Solving equations through particle dynamics. Comput Phys Commun 197:169–181MathSciNetCrossRefGoogle Scholar
  20. Engl H, Hanke M, Neubauer A (1996) Regularization of inverse problems, vol 375. Springer, New YorkCrossRefGoogle Scholar
  21. Gulliksson M (2017) The discrete dynamical functional particle method for solving constrained optimization problems. Dolomites Res Notes Approx 10:6–12MathSciNetCrossRefGoogle Scholar
  22. Gulliksson M, Edvardsson S, Persson J (2012) The dynamical functional particle method: an approach for boundary value problems. J Appl Mech 79(2):021012CrossRefGoogle Scholar
  23. Gulliksson M, Edvardsson S, Lind A (2013) The dynamical functional particle method. ArXiv e-prints, 2013Google Scholar
  24. Gulliksson M, Holmbom A, Persson J, Zhang Y (2018) A separating oscillation method of recovering the g-limit in standard and non-standard homogenization problems. Inverse Prob 32:025005MathSciNetCrossRefGoogle Scholar
  25. Hairer E, Lubich C, Wanner G (2006) Geometric numerical integration, 2nd edn. Springer, Berlin/HeidelbergzbMATHGoogle Scholar
  26. Han W, Cong W, Wang G (2006) Mathematical theory and numerical analysis of bioluminescence tomography. Inverse Prob 22:1659–1675MathSciNetCrossRefGoogle Scholar
  27. Kaltenbacher B, Neubauer A, Scherzer O (2008) Iterative regularization methods for nonlinear ill-posed problems. Walter de Gruyter GmbH & Co. KG, BerlinCrossRefGoogle Scholar
  28. Karafyllis I, Grüne L (2013) Lyapunov function based step size control for numerical ode solvers with application to optimization algorithms. In: Hüper K, Trumpf J (eds) Mathematical system theory – festschrift in honor of Uwe Helmke on the occasion of his 60th birthday. CreateSpace, pp 183–210.
  29. Kaufman D, Pai D (2012) Geometric numerical integration of inequality constrained, nonsmooth hamiltonian systems. SIAM J Sci Comput 34(5):A2670–A2703MathSciNetCrossRefGoogle Scholar
  30. Lin G, Cheng X, Zhang Y (2018a) A parametric level set based collage method for an inverse problem in elliptic partial differential equations. J Comput Appl Math 340:101–121MathSciNetCrossRefGoogle Scholar
  31. Lin G, Zhang Y, Cheng X, Gulliksson M, Forssen P, Fornstedt T (2018b) A regularizing Kohn-Vogelius formulation for the model-free adsorption isotherm estimation problem in chromatography. Appl Anal 97:13–40MathSciNetCrossRefGoogle Scholar
  32. Lions J, Magenes E (1972) Non-homogeneous boundary value problems and applications, vol I. Springer, BerlinCrossRefGoogle Scholar
  33. Mclachlan R, Reinout G, Quispel W (2006) Geometric integrators for odes. J Phys A 39: 5251–5285MathSciNetCrossRefGoogle Scholar
  34. McLachlan R, Modin K, Verdier O, Wilkins M (2014) Geometric generalisations of shake and rattle. Found Comput Math J Soc Found Comput Math 14(2):339MathSciNetCrossRefGoogle Scholar
  35. Nesterov Y (1983) A method of solving a convex programming problem with convergence rate. Sov Math Doklady 27:372–376zbMATHGoogle Scholar
  36. Neubauer A (2000) On Landweber iteration for nonlinear ill-posed problems in Hilbert scales. Numer Math 85:309–328MathSciNetCrossRefGoogle Scholar
  37. Neubauer A (2017) On Nesterov acceleration for Landweber iteration of linear ill-posed problems. J Inverse Ill-Posed Prob 25:381–390MathSciNetzbMATHGoogle Scholar
  38. Neuman M, Edvardsson S, Edström P (2015) Solving the radiative transfer equation with a mathematical particle method. Opt Lett 40(18):4325–4328CrossRefGoogle Scholar
  39. Poljak BT (1964) Some methods of speeding up the convergence of iterative methods. Akademija Nauk SSSR. Zurnal Vycislitel nli Matematiki i Matematicoskoi Fiziki 4:791Google Scholar
  40. Rieder A (2005) Runge-Kutta integrators yield optimal regularization schemes. Inverse Prob 21:453–471MathSciNetCrossRefGoogle Scholar
  41. Roubíček T (2013) Nonlinear partial differential equations with applications, vol 153. Springer Science & Business Media, BaselCrossRefGoogle Scholar
  42. Sandin P, Ögren M, Gulliksson M (2016) Numerical solution of the stationary multicomponent nonlinear schrödinger equation with a constraint on the angular momentum. Phys Rev E 93:033301MathSciNetCrossRefGoogle Scholar
  43. Sandro I, Valerio P, Francesco Z (1979) A new method for solving nonlinear simultaneous equations. SIAM J Numer Anal 16(5):779–11. 10Google Scholar
  44. Scherzer O, Grasmair M, Grossauer H, Haltmeier M, Lenzen F (2009) Variational methods in imaging. Springer, New YorkzbMATHGoogle Scholar
  45. Schock E (1985) Approximate solution of ill-posed equations: arbitrarily slow convergence vs. superconvergence. Construct Methods Pract Treat Integral Equ 73:234–243MathSciNetCrossRefGoogle Scholar
  46. Smyrlis G, Zisis V (2004) Local convergence of the steepest descent method in Hilbert spaces. J Math Anal Appl 300(2):436–453MathSciNetCrossRefGoogle Scholar
  47. Song S, Huang J (2012) Solving an inverse problem from bioluminescence tomography by minimizing an energy-like functional. J Comput Anal Appl 14:544–558MathSciNetzbMATHGoogle Scholar
  48. Tautenhahn U (1994) On the asymptotical regularization of nonlinear ill-posed problems. Inverse Prob 10:1405–1418MathSciNetCrossRefGoogle Scholar
  49. Tikhonov A, Leonov A, Yagola A (1998) Nonlinear ill-posed problems, vol I and II. Chapman and Hall, LondonzbMATHGoogle Scholar
  50. Tsai C-C, Liu C-S, Yeih W-C (2010) Fictious time integration method of fundamental solutions with Chebyshev polynomials for solving Poisson-type nonlinear pdes. CMES 56(2):131–151MathSciNetzbMATHGoogle Scholar
  51. Vainikko G, Veretennikov A (1986) Iteration procedures in ill-posed problems. Moscow: Nauka (In Russian)Google Scholar
  52. Wang Y, Zhang Y, Lukyanenko D, Yagola A (2012) A method of restoring the aerosol particle size distribution function on the set of piecewise-convex functions. Vychislitelnye Metody i Programmirovanie 13:49–66Google Scholar
  53. Wang Y, Zhang Y, Lukyanenko D, Yagola A (2013) Recovering aerosol particle size distribution function on the set of bounded piecewise-convex functions. Inverse Prob Sci Eng 21:339–354MathSciNetCrossRefGoogle Scholar
  54. Watson L, Sosonkina M, Melville R, Morgan A, Walker H (1997) Alg 777:hompack90: a suite of fortan 90 codes for globally convergent homotopy algorithms. ACM Trans Math Softw 23(4):514–549CrossRefGoogle Scholar
  55. Yao Z, Zhang Y, Bai Z, Eddy WF (2018) Estimating the number of sources in magnetoencephalography using spiked population eigenvalues. J Am Stat Assoc 113(522):505–518MathSciNetCrossRefGoogle Scholar
  56. Zhang Y, Hofmann B (2018) On the second order asymptotical regularization of linear ill-posed inverse problems. Applicable Analysis, pp 1–26.
  57. Zhang Y, Lukyanenko D, Yagola A (2013) Using Lagrange principle for solving linear ill-posed problems with a priori information. Vychislitelnye Metody i Programmirovanie 14:468–482Google Scholar
  58. Zhang Y, Lukyanenko D, Yagola A (2015) An optimal regularization method for convolution equations on the sourcewise represented set. J Inverse Ill-Posed Prob 23:465–475MathSciNetzbMATHGoogle Scholar
  59. Zhang Y, Gulliksson M, Hernandez Bennetts V, Schaffernicht E (2016a) Reconstructing gas distribution maps via an adaptive sparse regularization algorithm. Inverse Prob Sci Eng 24:1186–1204MathSciNetCrossRefGoogle Scholar
  60. Zhang Y, Lin G, Forssen P, Gulliksson M, Fornstedt T, Cheng X (2016b) A regularization method for the reconstruction of adsorption isotherms in liquid chromatography. Inverse Prob 32:105005MathSciNetCrossRefGoogle Scholar
  61. Zhang Y, Lukyanenko D, Yagola A (2016c) Using Lagrange principle for solving two-dimensional integral equation with a positive kernel. Inverse Prob Sci Eng 24:811–831MathSciNetCrossRefGoogle Scholar
  62. Zhang Y, Forssen P, Fornstedt T, Gulliksson M, Dai X (2017a) An adaptive regularization algorithm for recovering the rate constant distribution from biosensor data. Inverse Prob Sci Eng 24:1–26Google Scholar
  63. Zhang Y, Lin G, Forssen P, Gulliksson M, Fornstedt T, Cheng X (2017b) An adjoint method in inverse problems of chromatography. Inverse Prob Sci Eng 25:1112–1137MathSciNetCrossRefGoogle Scholar
  64. Zhang Y, Gong R, Cheng X, Gulliksson M (2018a) A dynamical regularization algorithm for solving inverse source problems of elliptic partial differential equations. Inverse Prob 34:065001MathSciNetCrossRefGoogle Scholar
  65. Zhang Y, Gong R, Gulliksson M, Cheng X (2018b) A coupled complex boundary expanding compacts method for inverse source problems. J Inverse Ill-Posed Prob, pp 1–20.

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Mårten Gulliksson
    • 1
    Email author
  • Magnus Ögren
    • 1
  • Anna Oleynik
    • 2
  • Ye Zhang
    • 3
  1. 1.MathematicsSchool of Engineering and TechnologyÖrebroSweden
  2. 2.Department of MathematicsUniversity of BergenBergenNorway
  3. 3.Faculty of MathematicsChemnitz University of TechnologyChemnitzGermany

Section editors and affiliations

  • Torsten Lindström
  • Bharath Sriraman
    • 1
  1. 1.Department of Mathematical SciencesThe University of MontanaMissoulaUSA

Personalised recommendations