Inference of hidden variables in systems of differential equations with genetic programming
 Theodore W. Cornforth,
 Hod Lipson
 … show all 2 hide
Rent the article at a discount
Rent now* Final gross prices may vary according to local VAT.
Get AccessAbstract
The datadriven modeling of dynamical systems is an important scientific activity, and many studies have applied genetic programming (GP) to the task of automatically constructing such models in the form of systems of ordinary differential equations (ODEs). These previous studies assumed that data measurements were available for all variables in the system, whereas in realworld settings, it is typically the case that one or more variables are unmeasured or “hidden.” Here, we investigate the prospect of automatically constructing ODE models of dynamical systems from time series data with GP in the presence of hidden variables. Several examples with both synthetic and physical systems demonstrate the unique challenges of this problem and the circumstances under which it is possible to reverseengineer both the form and parameters of ODE models with hidden variables.
 Adachi, F, Washio, T, Motoda, H (2006) Scientific discovery of dynamic models based on scaletype constraints. IPSJ Digit. Cour. 2: pp. 607619 CrossRef
 Aguirre, LA, Freitas, US, Letellier, C, Maquet, J (2001) Structureselection techniques applied to continuoustime nonlinear models. Physica D 158: pp. 118 CrossRef
 Aguirre, LA, Letellier, C (2009) Modeling nonlinear dynamics and chaos: a review. Math. Probl. Eng..
 D.P. Ahalpara, A. Sen, in Proceedings of the EuroGP 2011, ed. by S. Silva, J.A. Foster, M. Nicolau, P. Machado, M. Giacobini. A sniffer technique for an efficient deduction of model dynamical equations using genetic programming (Springer, Berlin, 2011), pp. 1–12
 Ando, S, Sakamoto, E, Iba, H (2002) Evolutionary modeling and inference of gene network. Inf. Sci. 145: pp. 237259 CrossRef
 H. Andrew, in Proceedings of the 2nd International Conference on Adaptive Computing in Engineering Design and Control. System identification using genetic programming (University of Plymouth, Plymouth, 1996), pp. 57–62
 P.J. Angeline, D.B. Fogel, in Proceedings of the SPIE. An evolutionary program for the identification of dynamical systems (1997), pp. 409–417
 Argemí, B, Saurina, J (2007) Study of the degradation of 5azacytidine as a model of unstable drugs using a stoppedflow method and further data analysis with multivariate curve resolution. Talanta 74: pp. 176182 CrossRef
 Åström, KJ, Eykhoff, P (1971) System identification—a survey. Automatica 7: pp. 123162 CrossRef
 Baake, E, Baake, M, Bock, HG, Briggs, KM (1992) Fitting ordinary differential equations to chaotic data. Phys. Rev. A 45: pp. 55245529 CrossRef
 V. Babovic, M. Keijzer, in Proceedings of the 4th International Conference on Hydroinformatics. Evolutionary algorithms approach to induction of differential equations (Iowa City, USA, 2000), pp. 251–258
 Bakker, R, Schouten, JC, Giles, CL, Takens, F, Bleek, CM (2000) Learning chaotic attractors. Neural Comput. 12: pp. 23552383 CrossRef
 Banzhaf, W, Nordin, P, Keller, RE, Francone, FD (1997) Genetic Programming: An Introduction on the Automatic Evolution of Computer Programs and Its Applications. Morgan Kaufmann, San Francisco
 Beisler, JA (1978) Isolation, characterization, and properties of a labile hydrolysis product of the antitumor nucleoside, 5azacytidine. J. Med. Chem. 21: pp. 204208 CrossRef
 Bernardino, HS, Barbosa, HJC (2011) Inferring systems of ordinary differential equations via grammarbased immune programming. LNCS 6825: pp. 198211
 Bishop, CM (2006) Pattern Recognition and Machine Learning. Springer, New York
 Boccaletti, S (2008) The Synchronized Dynamics of Complex Systems. Elsevier, Amsterdam
 Bongard, J, Lipson, H (2007) Automated reverse engineering of nonlinear dynamical systems. PNAS 104: pp. 99439948 CrossRef
 Bradley, E, Easley, M, Stolle, R (2001) Reasoning about nonlinear system identification. Artif. Intell. 133: pp. 139188 CrossRef
 Brameier, M, Banzhaf, W (2007) Linear Genetic Programming. Springer, New York
 Breeden, JL, Hübler, A (1990) Reconstructing equations of motion from experimental data with unobserved variables. Phys. Rev. A 42: pp. 58175826 CrossRef
 Bridewell, W, Langley, P, Todorovski, L, Dzeroski, S (2008) Inductive process modeling. Mach. Learn. 71: pp. 132 CrossRef
 Cao, H, Kang, L, Chen, Y, Yu, J (2000) Evolutionary modeling of systems of ordinary differential equations with genetic programming. Genet. Program Evolvable Mach. 1: pp. 309337 CrossRef
 Chis, OT, Banga, JR, BalsaCanto, E (2011) Structural identifiability of systems biology models: a critical comparison of methods. PLoS ONE 6: pp. e27755 CrossRef
 Cho, KH, Shin, SY, Kim, HW, Wolkenhauer, O, McFerran, B, Kolch, W (2003) Mathematical modeling of the influence of RKIP on the ERK signaling pathway. LNCS 2602: pp. 127141
 Clery, D, Voss, D (2005) All for one and one for all. Science 308: pp. 809 CrossRef
 Cremers, J, Hübler, A (1987) Construction of differential equations from experimental data. Zeitschrift für Naturforschung A 42: pp. 797802
 Crutchfield, JP, McNamara, BS (1987) Equations of motion from a data series. Complex Syst. 1: pp. 417452
 S. Džeroski, L. Todorovski, in Proceedings of the 10th International Conference on Machine Learning. Discovering dynamics (Kaufmann, Amherst, 1993), pp. 97–103
 Evans, J, Rzhetsky, A (2010) Machine science. Science 329: pp. 3994400 CrossRef
 Gouesbet, G (1991) Reconstruction of the vector fields of continuous dynamical systems from numerical scalar time series. Phys. Rev. A 43: pp. 53215331 CrossRef
 Grassberger, P, Procaccia, I (1983) Measuring the strangeness of strange attractors. Physica D 9: pp. 189208 CrossRef
 G.J. Gray, D.J. MurraySmith, Y. Li, K.C. Sharman, in Proceedings of the 2nd International Conference on Genetic Algorithms in Engineering Systems: Innovations and Applications. Nonlinear model structure identification using genetic programming (Inst of Electrical Engineers, London, 1997), pp. 308–313
 Gray, GJ, MurraySmith, DJ, Li, Y, Sharman, KC, Weinbrenner, T (1998) Nonlinear model structure identification using genetic programming. Control Eng. Pract. 6: pp. 13411352 CrossRef
 Grewal, MS, Glover, K (1976) Identifiability of linear and nonlinear dynamical systems. IEEE Trans. Automat. Control 21: pp. 833837 CrossRef
 Hengl, S, Kreutz, C, Timmer, J, Maiwald, T (2007) Databased identifiability analysis of nonlinear dynamical models. Bioinformatics 23: pp. 26122618 CrossRef
 Hodgkin, AL, Huxley, AF (1952) A quantitative description of membrane current and its application to conduction and excitation in nerve. J. Phys. 117: pp. 500544
 Hong, X, Mitchell, RJ, Chen, S, Harris, CJ, Li, K, Irwin, GW (2008) Model selection approaches for nonlinear system identification: a review. Int. J. Syst. Sci. 39: pp. 925946 CrossRef
 H. Iba, H. de Garis, T. Sato, in Proceedings of the PPSN III, ed. by Y. Davidor, H.P. Schwefel, R. Männer. Genetic programming with local hillclimbing (Springer, London, 1994), pp. 302–311
 Iba, H (2008) Inference of differential equation models by genetic programming. Inf. Sci. 178: pp. 44534468 CrossRef
 S.J. Julier, J.K. Uhlmann, in Proceedings of the International Symposium on Aerospace/Defense Sensing, Simulation, and Controls. A new extension of the Kalman filter to nonlinear systems (1997), pp. 182–193
 Kaboudan, MA (2000) Genetic programming prediction of stock prices. Comput. Econ. 16: pp. 207236 CrossRef
 Kennel, MB, Brown, R, Abarbanel, HD (1992) Determining embedding dimension for phasespace reconstruction using a geometrical construction. Phys. Rev. A 45: pp. 34033411 CrossRef
 King, RD, Whelan, KE, Jones, FM, Reiser, PGK, Bryant, CH, Muggleton, SH, Kell, DB, Oliver, SG (2004) Functional genomic hypothesis generation and experimentation by a robot scientist. Nature 427: pp. 247252 CrossRef
 Kissinger, LD, Stemm, NL (1986) Determination of the antileukemia agents cytarabine and azacitidine and their respective degradation products by highperformance liquid chromatography. J. Chromatogr. 353: pp. 309318 CrossRef
 Koza, J (1992) Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge
 Langley, P (1981) Datadriven discovery of physical laws. Cognitive Sci. 5: pp. 3154 CrossRef
 Langley, P, Simon, HA, Bradshaw, GL, Zytkow, JM (1987) Scientific Discovery: Computational Explorations of the Creative Processes. MIT Press, Cambridge
 P. Langley, J. Sanchez, L. Todorovski, S. Džeroski, in Proceedings of the 19th International Conference on Machine Learning. Inducing process models (Morgan Kaufmann, Sydney, 2002), pp. 347–354
 Ljung, L (1999) System Identification—Theory For the User. Prentice Hall, Upper Saddle River, NJ
 Lorenz, EN (1963) Deterministic nonperiodic flow. J. Atmos. Sci. 20: pp. 130141 CrossRef
 Mamani, G, Becedas, J, Batlle, VF, SiraRamírez, H (2008) Algebraic observer to estimate unmeasured state variables of DC motors. Eng. Lett. 16: pp. 248255
 Montana, DJ (1995) Strongly typed genetic programming. Evol. Comput. 3: pp. 199230 CrossRef
 B.S. Mulloy, R.L. Riolo, R.S. Savit, in Proceedings of the GECCO 1996. Dynamics of genetic programming and chaotic time series prediction (MIT Press, Cambridge, MA, 1996), pp. 166–174
 Neely, C, Weller, P, Dittmara, R (1997) Is technical analysis in the foreign exchange market profitable? A genetic programming approach. J. Financ. Quant. Anal. 32: pp. 405426 CrossRef
 Oakley, H Two scientific applications of genetic programming: stack filters and nonlinear equation fitting to chaotic data. In: Kinnear, KE eds. (1994) Advances Genetic Programming. MIT Press, Cambridge, MA, pp. 369389
 Packard, NH, Crutchfield, JP, Farmer, JD, Shaw, RS (1980) Geometry from a time series. Phys. Rev. Lett. 45: pp. 712716 CrossRef
 Pnevmatikos, SN eds. (1985) Singularities & Dynamical Systems. Elsevier, Amsterdam
 Press, WH, Flannery, BP, Teukolsky, SA, Vetterling, WT (2007) Numerical Recipes: The Art of Scientific Computing. Cambridge University Press, Cambridge
 Qian, L, Wang, H, Dougherty, ER (2008) Inference of noisy nonlinear differential equation models for gene regulatory networks using genetic programming and Kalman filtering. IEEE Trans. Signal Process. 56: pp. 33273339 CrossRef
 Raue, A, Becker, V, Klingmüller, U, Timmer, J (2010) Identifiability and observability analysis for experimental design in nonlinear dynamical models. Chaos 20: pp. 045105 CrossRef
 RodriguezVázquez, K, Fleming, PJ (2005) Evolution of mathematical models of chaotic systems based on multiobjective genetic programming. Knowl. Inf. Syst. 8: pp. 235256 CrossRef
 Routray, K, Deo, G (2005) Kinetic parameter estimation for a multiresponse nonlinear reaction model. AIChE J. 51: pp. 17331746 CrossRef
 Russell, S, Norvig, P (2009) Artificial Intelligence: A Modern Approach. Prentice Hall, Upper Saddle River, NJ
 C. Ryan, J.J. Collins, M. O’Neill, in Proceedings of the EuroGP 1998, ed. by W. Banzhaf, R. Poli, M. Schoenauer, T.C. Fogarty. Grammatical evolution: evolving programs for an arbitrary language (Springer, London, 1998), pp. 83–96
 E. Sakamoto, H. Iba, in Proceedings of the CEC 2001. Inferring a system of differential equations for a gene regulatory network by using genetic programming (2001), pp. 720–726
 Schmidt, MD, Lipson, H (2008) Coevolution of fitness predictors. IEEE Trans. Evol. Comput. 12: pp. 736749 CrossRef
 Schmidt, MD, Lipson, H (2009) Distilling freeform natural laws from experimental data. Science 324: pp. 8185 CrossRef
 M.D. Schmidt, H. Lipson, in Proceedings of the GECCO 2010. Agefitness Pareto optimization (2010), pp. 543–544
 Schmidt, MD, Lipson, H (2010) Agefitness Pareto optimization. Genet. Program. Theory Pract. 8: pp. 129146
 M. Schwabacher, P. Langley, in Proceedings of the ICML 2001, ed. by C.E. Brodley, A.P. Danyluk. Discovering communicable scientific knowledge from spatiotemporal data (Morgan Kaufmann Publishers Inc., San Francisco, CA, 2001), pp. 489–496
 Sharp, KV, Adrian, RJ (2004) Transition from laminar to turbulent flow in liquid filled microtubes. Exp. Fluids 36: pp. 741747 CrossRef
 Sjöberg, J, Zhang, Q, Ljung, L, Benveniste, A, Delyon, B, Glorennec, PY, Hjalmarsson, H, Juditsky, A (1995) Nonlinear blackbox modeling in system identification: a unified overview. Automatica 31: pp. 16911724 CrossRef
 Stewart, WE, Caracotsios, M, Sørensen, JP (1992) Parameter estimation from multiresponse data. AIChE J. 38: pp. 641650 CrossRef
 Strogatz, SH (2001) Exploring complex networks. Nature 410: pp. 268276 CrossRef
 Szalay, A, Gray, J (2006) 2020 Computing: science in an exponential world. Nature 440: pp. 413414 CrossRef
 Takens, F (1981) Detecting strange attractors in turbulence. Lect. Notes Math. 898: pp. 366381 CrossRef
 L. Todorovski, S. Džeroski, in Proceedings of the ICML 1997. Declarative bias in equation discovery (Kaufmann, Nashville, 1997), pp 376–384
 Vajda, S, Rabitz, H (1994) Identifiability and distinguishability of general reaction systems. J. Phys. Chem. 98: pp. 52655271 CrossRef
 Voss, HU, Timmer, J, Kurths, J (2004) Nonlinear dynamical system identification from uncertain and indirect measurements. Int. J. Bifurcat. Chaos 14: pp. 19051933 CrossRef
 Walter, É, Pronzato, L (1997) Identification of Parametric Models From Experimental Data. Springer, New York
 Waltz, D, Buchanan, BG (2009) Automating science. Science 324: pp. 4344 CrossRef
 T. Washio, H. Motoda, Y. Niwa, in Proceedings of the ICML 2000. Enhancing the plausibility of law equation discovery (2000), pp. 1127–1134
 Wei, HL, Billings, SA (2008) Model structure selection using an integrated forward orthogonal search algorithm assisted by squared correlation and mutual information. Int. J. Model. Identif. Control 3: pp. 341356 CrossRef
 P.A. Whigham, in Proceedings of the Workshop on Genetic Programming: From Theory to RealWorld Applications, ed. by J.P. Rosca. Grammaticallybased genetic programming (Tahoe City, California, 1995), pp. 33–41
 Winkler, S, Affenzeller, M, Wagner, S (2005) New methods for the identification of nonlinear model structures based upon genetic programming techniques. J. Syst. Sci. 31: pp. 514
 Yoshida, T, Jones, LE, Ellner, SP, Fussmann, GF, Hairston, NG (2003) Rapid evolution drives ecological dynamics in a predatorprey system. Nature 424: pp. 303306 CrossRef
 Title
 Inference of hidden variables in systems of differential equations with genetic programming
 Journal

Genetic Programming and Evolvable Machines
Volume 14, Issue 2 , pp 155190
 Cover Date
 20130601
 DOI
 10.1007/s1071001291754
 Print ISSN
 13892576
 Online ISSN
 15737632
 Publisher
 Springer US
 Additional Links
 Topics
 Keywords

 Genetic programming
 Ordinary differential equations
 Hidden variables
 Modeling
 Symbolic identification
 Authors

 Theodore W. Cornforth ^{(1)}
 Hod Lipson ^{(2)}
 Author Affiliations

 1. Biological Statistics and Computational Biology, Cornell University, 1198 Comstock Hall, Ithaca, NY, 148532601, USA
 2. Mechanical and Aerospace Engineering, Cornell University, 239 Upson Hall, Ithaca, NY, 148537501, USA