Skip to main content
SpringerLink
Log in

Using Covariance Matrix Adaptation Evolution Strategies for solving different types of differential equations

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

A novel mesh-free heuristic method for solving differential equations is proposed. The new approach can cope with linear, nonlinear, and partial differential equations (DE), and systems of DEs. Candidate solutions are expressed using a linear combination of kernel functions. Thus, the original problem is transformed into an optimization problem that consists in finding the parameters that define each kernel. The new optimization problem is solved applying a Covariance Matrix Adaptation Evolution Strategy. To increase the accuracy of the results, a Downhill Simplex local search is applied to the best solution found by the mentioned evolutionary algorithm. Our method is applied to 32 differential equations extracted from the literature. All problems are successfully solved, achieving competitive accuracy levels when compared to other heuristic methods. A simple comparison with numerical methods is performed using two partial differential equations to show the pros and cons of the proposed algorithm. To verify the potential of this approach with a more practical problem, an electric circuit is analyzed in depth. The method can obtain the dynamic behavior of the circuit in a parametric way, taking into account different component values.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  • Allahviranloo T, Salahshour S, Abbasbandy S (2012) Explicit solutions of fractional differential equations with uncertainty. Soft Comput 16:297–302

    Article  MATH  Google Scholar 

  • Babaei M (2013) A general approach to approximate solutions of nonlinear differential equations using particle swarm optimization. Appl Soft Comput 13(7):3354–3365

    Article  Google Scholar 

  • Balasubramaniam P, Kumar AVA (2009) Solution of matrix Riccati differential equation for nonlinear singular system using genetic programming. Genet Program Evolvable Mach 10:71–89

    Article  Google Scholar 

  • Bronson JR, Sastry SP, Levine JA, Whitaker RT (2014) Adaptive and unstructured mesh cleaving. In: 23rd International meshing roundtable (IMR23), vol 82, pp 266–278

  • Bryden KM, Ashlock DA, Corns S, Willson SJ (2006) Graph-based evolutionary algorithms. IEEE Trans Evol Comput 10(5):550–567

    Article  Google Scholar 

  • Chaquet JM, Carmona EJ (2012) Solving differential equations with Fourier series and evolution strategies. Appl Soft Comput 12:3051–3062

    Article  Google Scholar 

  • Chen H, Kong L, Leng WJ (2011) Numerical solution of PDEs via integrated radial basis function networks with adaptive training algorithm. Appl Soft Comput 11:855–860

    Article  Google Scholar 

  • Choi B, Lee JH (2009) Comparison of generalization ability on solving differential equations using backpropagation and reformulated radial basis function networks. Neurocomputing 73:115–118

    Article  Google Scholar 

  • Colutto S, Fruhauf F, Fuchs M, Scherzer O (2010) The CMA-ES on Riemannian manifolds to reconstruct shapes in 3-D voxel images. IEEE Trans Evol Comput 14(2):227–245

    Article  Google Scholar 

  • El-Emam NN, Al-Rabeh RH (2011) An intelligent computing technique for fluid flow problems using hybrid adaptive neural network and genetic algorithm. Appl Soft Comput 11:3283–3296

    Article  Google Scholar 

  • Guo R, Guo D (2009) Random fuzzy variable foundation for grey differential equation modeling. Soft Comput 13:185–201

    Article  MATH  Google Scholar 

  • Hangelbroek T, Ron A (2010) Nonlinear approximation using Gaussian kernels. J Funct Anal 259(1):203–219

    Article  MathSciNet  MATH  Google Scholar 

  • Hansen N (2006) The CMA evolution strategy: a comparing review. In: Lozano JA, Larrañaga P, Inza I, Bengoetxea E (eds) Towards a new evolutionary computation, vol 192. Studies in fuzziness and soft computing. Springer, Berlin, pp 75–102

    Chapter  Google Scholar 

  • Hansen N (June 2011) The CMA evolution strategy: A tutorial. https://www.lri.fr/~hansen/cmatutorial.pdf

  • Hansen N, Kern S (2004) Evaluating the CMA evolution strategy on multimodal test functions. In: Eighth international conference on parallel problem solving from nature PPSN VIII, pp 282–291. Springer

  • He J, Xu J, Yao X (2000a) Solving equations by hybrid evolutionary computation techniques. IEEE Trans Evol Comput 4(3):295–304

    Article  Google Scholar 

  • He S, Reif K, Unbehauen R (2000b) Multilayer neural networks for solving a class of partial differential equations. Neural Netw 13:385–396

    Article  Google Scholar 

  • Howard D, Roberts SC (2001) Genetic programming solution of the Convection-diffusion equation. In: Proceedings of the 3rd annual conference on genetic and evolutionary computation, GECCO’01, pp 34–41. Morgan Kaufmann Publishers Inc

  • Howard D, Brezulianu A, Kolibal J (2011) Genetic programming of the stochastic interpolation framework: convection-diffusion equation. Soft Comput 15:71–78

    Article  Google Scholar 

  • Khan JA, Zahoor RMA, Qureshi IM (2009) Swarm intelligence for the solution of problems in differential equations. In: Second international conference on environmental and computer science, 2009. ICECS ’09, pp 141–147

  • Kirstukas SJ, Bryden KM, Ashlock DA (2005) A hybrid genetic programming approach for the analytical solution of differential equations. Int J Gen Syst 34:279–299

    Article  MATH  Google Scholar 

  • Kumar M, Yadav N (2011) Multilayer perceptrons and radial basis function neural network methods for the solution of differential equations. A survey. Comput Math Appl 62:3796–3811

    Article  MathSciNet  MATH  Google Scholar 

  • Lagaris IE, Likas A, Fotiadis DI (1998) Artificial neural networks for solving ordinary and partial differential equations. IEEE Trans Neural Netw 5:987–1000

    Article  Google Scholar 

  • Leveque RJ (2002) Finite volume methods for hyperbolic problems. Cambridge University Press, Cambridge

    Book  MATH  Google Scholar 

  • Liu GR (2010) Meshfree methods: moving beyond the finite element method. CRC Press, Inc., Boca Raton

    MATH  Google Scholar 

  • MacNeil PE (2012) A technique for generating approximate solutions and its application to coulomb interactions. In: Southeastcon, 2012 Proceedings of IEEE, pp 1–5

  • Mehrkanoon S, Suykens JAK (2015) Learning solutions to partial differential equations using LS-SVM. Neurocomputing 159:105–116

    Article  Google Scholar 

  • Mosleh M (2013) Fuzzy neural network for solving a system of fuzzy differential equations. Appl Soft Comput 13(8):3597–3607

    Article  Google Scholar 

  • Nelder JA, Mead R (1965) A simplex method for function minimization. Comput J 7(4):308–313

    Article  MathSciNet  MATH  Google Scholar 

  • Ozisik MN (1994) Finite difference methods in heat transfer. CRC Press, Inc., Boca Raton

    MATH  Google Scholar 

  • Panagant N, Bureerat S (2014) Solving partial differential equations using a new differential evolution algorithm. Math Probl Eng 2014(747490):1–10

    Article  Google Scholar 

  • Parisi DR, Mariani MC, Laborde MA (2003) Solving differential equations with unsupervised neural networks. Chem Eng Process 42:715–721

    Article  Google Scholar 

  • Peterson LE (2011) Covariance matrix self-adaptation evolution strategies and other metaheuristic techniques for neural adaptive learning. Soft Comput 15:598

    Article  Google Scholar 

  • Press WH, Vetterling WT, Teukolsky SA, Flannery BP (2002) Numerical Recipes in C++: the art of scientific computing, 2nd edn. Cambridge University Press, New York

    MATH  Google Scholar 

  • Puffer F, Tetzlaff R, Wolf D (1995) A learning algorithm for cellular neural networks (CNN) solving nonlinear partial differential equations. In: International symposium on signals, systems, and electronics (ISSSE ’95), pp 501–504

  • Rudd K, Ferrari S (2015) A constrained integration (cint) approach to solving partial differential equations using artificial neural networks. Neurocomputing 155:277–285

    Article  Google Scholar 

  • Seaton T, Brown G, Miller JF (2010) Analytic solutions to differential equations under graph-based genetic programming. In: Esparcia-Alcázar Anna Isabel, Ekárt Anikó, Sara Silva, Dignum Stephen, Şima Uyar A (eds) Genetic Programming: 13th European Conference, EuroGP 2010, pp 232–243. Springer Berlin Heidelberg

  • Shirvany Y, Hayati M, Moradian R (2009) Multilayer perceptron neural networks with novel unsupervised training method for numerical solution of the partial differential equations. Appl Soft Comput 9:20–29

    Article  Google Scholar 

  • Sobester A, Nair PB, Keane AJ (2008) Genetic programming approaches for solving elliptic partial differential equations. IEEE Trans Evol Comput 12:469–478

    Article  Google Scholar 

  • Spivak M (1980) Calculus. Publish or Perish, 4 edition

  • Suli E, Mayers DF (2003) An introduction to numerical analysis. Cambridge University Press, Cambridge

    Book  MATH  Google Scholar 

  • Sun M, Yan X, Sclabassi RJ (2003) Solving partial differential equations in real-time using artificial neural network signal processing as an alternative to finite-element analysis. In: International conference on neural networks and signal processing, 2003, vol 1, pp 381–384

  • Thompson JF, Soni BK, Weatherill NP (1999) Handbook of grid generation. CRC Press, Boca Raton

    MATH  Google Scholar 

  • Tsoulos IG, Lagaris IE (2006) Solving differential equations with genetic programming. Genet Program Evolvable Mach 7:33–54

    Article  Google Scholar 

  • Tsoulos IG, Gavrilis D, Glavas E (2009) Solving differential equations with constructed neural networks. Neurocomputing 72:2385–2391

    Article  Google Scholar 

  • Yao K (2015) Uncertain differential equation with jumps. Soft Comput 19:2063–2069

    Article  MATH  Google Scholar 

  • Yazdi HS, Pourreza R (2010) Unsupervised adaptive neural-fuzzy inference system for solving differential equations. Appl Soft Comput 10:267–275

    Article  Google Scholar 

  • Yazdi HS, Pakdaman M, Modaghegh H (2011) Unsupervised kernel least mean square algorithm for solving ordinary differential equations. Neurocomputing 74:2062–2071

    Article  Google Scholar 

Download references

Acknowledgements

The authors wish to thank Nikolaus Hansen for sharing with the research community CMA-ES implementations in several programming languages. In particular, the ANSI C implementation (https://www.lri.fr/~hansen/cmaes_inmatlab.html) has been used in the present work.

Author information

Authors and Affiliations

  1. Dpto. de Inteligencia Artificial, Escuela Técnica Superior de Ingeniería Informática, Universidad Nacional de Educación a Distancia, Madrid, Spain

    Jose M. Chaquet & Enrique J. Carmona

Authors
  1. Jose M. Chaquet
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Enrique J. Carmona
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jose M. Chaquet.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Communicated by V. Loia.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chaquet, J.M., Carmona, E.J. Using Covariance Matrix Adaptation Evolution Strategies for solving different types of differential equations. Soft Comput 23, 1643–1666 (2019). https://doi.org/10.1007/s00500-017-2888-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-017-2888-9

Keywords

Search

Navigation