Skip to main content
SpringerLink
Log in

Data-driven discovery of linear dynamical systems from noisy data

  • Article
  • Published:
Science China Technological Sciences Aims and scope Submit manuscript

Abstract

In modern science and engineering disciplines, data-driven discovery methods play a fundamental role in system modeling, as data serve as the external representations of the intrinsic mechanisms within systems. However, empirical data contaminated by process and measurement noise remain a significant obstacle for this type of modeling. In this study, we have developed a data-driven method capable of directly uncovering linear dynamical systems from noisy data. This method combines the Kalman smoothing and sparse Bayesian learning to decouple process and measurement noise under the expectation-maximization framework, presenting an analytical method for alternate state estimation and system identification. Furthermore, the discovered model explicitly characterizes the probability distribution of process and measurement noise, as they are essential for filtering, smoothing, and stochastic control. We have successfully applied the proposed algorithm to several simulation systems. Experimental results demonstrate its potential to enable linear dynamical system discovery in practical applications where noise-free data are intractable to capture.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bergen K J, Johnson P A, de Hoop M V, et al. Machine learning for data-driven discovery in solid Earth geoscience. Science, 2019, 363: eaau0323

    Article  Google Scholar 

  2. Xie X, Samaei A, Guo J, et al. Data-driven discovery of dimensionless numbers and governing laws from scarce measurements. Nat Commun, 2022, 13: 7562

    Article  Google Scholar 

  3. Floryan D, Graham M D. Data-driven discovery of intrinsic dynamics. Nat Mach Intell, 2022, 4: 1113–1120

    Article  Google Scholar 

  4. Wang Y, Fang H, Jin J, et al. Data-driven discovery of stochastic differential equations. Engineering, 2022, 17: 244–252

    Article  Google Scholar 

  5. Shen T, Dong Y L, He D X, et al. Online identification of time-varying dynamical systems for industrial robots based on sparse Bayesian learning. Sci China Tech Sci, 2022, 65: 386–395

    Article  Google Scholar 

  6. Shen T, Qiao X, Dong Y, et al. Deep adaptive control with online identification for industrial robots. Sci China Tech Sci, 2022, 65: 2593–2604

    Article  Google Scholar 

  7. Wu J, Li W, Xiong Z. Identification of robot dynamic model and joint frictions using a baseplate force sensor. Sci China Tech Sci, 2022, 65: 30–40

    Google Scholar 

  8. Li B, Liu H, Wang R. Data-driven sensor placement for efficient thermal field reconstruction. Sci China Tech Sci, 2021, 64: 1981–1994

    Article  Google Scholar 

  9. Li H, Wu P, Zeng N, et al. A survey on parameter identification, state estimation and data analytics for lateral flow immunoassay: From systems science perspective. Int J Syst Sci, 2022, 53: 3556–3576

    Article  Google Scholar 

  10. Li X, Feng S, Hou N, et al. Surface microseismic data denoising based on sparse autoencoder and Kalman filter. Syst Sci Control Eng, 2022, 10: 616–628

    Article  Google Scholar 

  11. Gao T T, Yan G. Autonomous inference of complex network dynamics from incomplete and noisy data. Nat Comput Sci, 2022, 2: 160–168

    Article  Google Scholar 

  12. Yuan Y, Tang X, Zhou W, et al. Data driven discovery of cyber physical systems. Nat Commun, 2019, 10: 4894

    Article  Google Scholar 

  13. Aalto A, Viitasaari L, Ilmonen P, et al. Gene regulatory network inference from sparsely sampled noisy data. Nat Commun, 2020, 11: 3493

    Article  Google Scholar 

  14. Schmidt M, Lipson H. Distilling free-form natural laws from experimental data. Science, 2009, 324: 81–85

    Article  Google Scholar 

  15. Brunton S L, Proctor J L, Kutz J N. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proc Natl Acad Sci USA, 2016, 113: 3932–3937

    Article  MathSciNet  Google Scholar 

  16. Ouala S, Nguyen D, Drumetz L, et al. Learning latent dynamics for partially observed chaotic systems. Chaos-An Interdiscip J Nonlinear Sci, 2020, 30: 103121

    Article  MathSciNet  Google Scholar 

  17. Lu P Y, Ariño Bernad J, Soljačiá M. Discovering sparse interpretable dynamics from partial observations. Commun Phys, 2022, 5: 206

    Article  Google Scholar 

  18. Bakarji J, Tartakovsky D M. Data-driven discovery of coarse-grained equations. J Comput Phys, 2021, 434: 110219

    Article  MathSciNet  Google Scholar 

  19. Rudy S H, Brunton S L, Proctor J L, et al. Data-driven discovery of partial differential equations. Sci Adv, 2017, 3: e1602614

    Article  Google Scholar 

  20. Raissi M, Karniadakis G E. Hidden physics models: Machine learning of nonlinear partial differential equations. J Comput Phys, 2018, 357: 125–141

    Article  MathSciNet  Google Scholar 

  21. Li X, Li L, Yue Z, et al. Sparse learning of partial differential equations with structured dictionary matrix. Chaos-An Interdiscip J Nonlinear Sci, 2019, 29: 043130

    Article  MathSciNet  Google Scholar 

  22. Jin J, Yuan Y, Goncalves J. High precision variational Bayesian inference of sparse linear networks. Automatica, 2020, 118: 109017

    Article  MathSciNet  Google Scholar 

  23. Jin J, Yuan Y, Gonçalves J. A full Bayesian approach to sparse network inference using heterogeneous datasets. IEEE Trans Automat Contr, 2020, 66: 3282–3288

    Article  MathSciNet  Google Scholar 

  24. Liu H, Shang Z, Ren Z, et al. Recovering unknown topology in a two-layer multiplex network: One layer infers the other layer. Sci China Tech Sci, 2022, 65: 1493–1505

    Article  Google Scholar 

  25. Reinbold P A K, Grigoriev R O. Data-driven discovery of partial differential equation models with latent variables. Phys Rev E, 2019, 100: 022219

    Article  MathSciNet  Google Scholar 

  26. Kaheman K, Brunton S L, Kutz J N. Automatic differentiation to simultaneously identify nonlinear dynamics and extract noise probability distributions from data. Mach Learn-Sci Technol, 2022, 3: 015031

    Article  Google Scholar 

  27. Chen Z, Liu Y, Sun H. Physics-informed learning of governing equations from scarce data. Nat Commun, 2021, 12: 6136

    Article  Google Scholar 

  28. Sun F, Liu Y, Wang Q, et al. PiSL: Physics-informed Spline Learning for data-driven identification of nonlinear dynamical systems. Mech Syst Signal Process, 2023, 191: 110165

    Article  Google Scholar 

  29. Wei B. Sparse dynamical system identification with simultaneous structural parameters and initial condition estimation. Chaos Solitons Fractals, 2022, 165: 112866

    Article  MathSciNet  Google Scholar 

  30. Schaeffer H. Learning partial differential equations via data discovery and sparse optimization. Proc R Soc A, 2017, 473: 20160446

    Article  MathSciNet  Google Scholar 

  31. Särkkä S, Svensson L. Bayesian Filtering and Smoothing. Cambridge: Cambridge University Press, 2023. 17

    Google Scholar 

  32. Shumway R H, Stoffer D S, Stoffer D S. Time Series Analysis and Its Applications. New York: Springer, 2000. 3

    Book  Google Scholar 

  33. Barfoot T D. State Estimation for Robotics. Cambridge: Cambridge University Press, 2017

    Book  Google Scholar 

  34. Song Y, Xu W, Niu L. Multiplicative Levy noise-induced transitions in gene expression. Sci China Tech Sci, 2022, 65: 1700–1709

    Article  Google Scholar 

  35. Gibson S, Ninness B. Robust maximum-likelihood estimation of multivariable dynamic systems. Automatica, 2005, 41: 1667–1682

    Article  MathSciNet  Google Scholar 

  36. Einicke G A, Malos J T, Reid D C, et al. Riccati equation and EM algorithm convergence for inertial navigation alignment. IEEE Trans Signal Process, 2008, 57: 370–375

    Article  MathSciNet  Google Scholar 

  37. Einicke G A, Falco G, Malos J T. EM algorithm state matrix estimation for navigation. IEEE Signal Process Lett, 2010, 17: 437–440

    Article  Google Scholar 

  38. Schön T B, Wills A, Ninness B. System identification of nonlinear state-space models. Automatica, 2011, 47: 39–49

    Article  MathSciNet  Google Scholar 

  39. Wang Y, Cheng C, Sun H, et al. Data augmentation-based statistical inference of diffusion processes. Chaos-An Interdiscip J Nonlinear Sci, 2023, 33: 033115

    Article  MathSciNet  Google Scholar 

  40. Wu C F J. On the convergence properties of the EM algorithm. Ann Statist, 1983, 11: 95–103

    Article  MathSciNet  Google Scholar 

  41. Mauroy A, Goncalves J. Koopman-based lifting techniques for nonlinear systems identification. IEEE Trans Automat Contr, 2019, 65: 2550–2565

    Article  MathSciNet  Google Scholar 

  42. Lusch B, Kutz J N, Brunton S L. Deep learning for universal linear embeddings of nonlinear dynamics. Nat Commun, 2018, 9: 4950

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan, 430074, China

    YaSen Wang & Han Ding

  2. State Key Lab of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan, 430074, China

    YaSen Wang & Han Ding

  3. School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, 430074, China

    Ye Yuan

  4. Department of Mechanical Engineering, University of Kansas, Lawrence, 66045, USA

    HuaZhen Fang

Authors
  1. YaSen Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ye Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. HuaZhen Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Han Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ye Yuan.

Additional information

This work was supported by the National Natural Science Foundation of China (Grant No. 92167201).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Y., Yuan, Y., Fang, H. et al. Data-driven discovery of linear dynamical systems from noisy data. Sci. China Technol. Sci. 67, 121–129 (2024). https://doi.org/10.1007/s11431-023-2520-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11431-023-2520-6

Search

Navigation