Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Entropy optimization based filtering for non-Gaussian stochastic systems


This paper is concerned with the entropy optimization based filter design for a class of multivariate dynamic stochastic systems with simultaneous presence of non-Gaussian process noise and measurement noise. The filter consists of time update and measurement update two steps, where the selection of the filter gain in the measurement update equation is a key issue to be addressed. Different from the classic Kalman filter theory, entropy rather than variance is employed as the filtering performance criterion due to the non-Gaussian characteristic of the estimation error. Following the establishment of the relationship between the probability density functions of random noises and estimation error, two kinds of entropy based performance indices are provided. On this basis, the corresponding optimal filter gains are obtained respectively by using the gradient optimization technique. Finally, some numerical simulations are provided to demonstrate the effectiveness of the proposed filtering algorithms.

This is a preview of subscription content, log in to check access.


  1. 1

    Kalman R E. A new approach to linear filtering and prediction problems. Trans ASME J Basic Eng, 1960, 82: 35–45

  2. 2

    Anderson B D, Moore J B. Optimal Filtering. Englewood Cliffs: Prentice Hall, 1979

  3. 3

    Simon D. Optimal State Estimation: Kalman, H , and Nonlinear Approaches. New York: Wiley, 2006

  4. 4

    Ding D R, Wang Z D, Ho D W C, et al. Distributed recursive filtering for stochastic systems under uniform quantizations and deception attacks through sensor networks. Automatica, 2017, 78: 231–240

  5. 5

    Ding D R, Wei G L, Zhang S J, et al. On scheduling of deception attacks for discrete-time networked systems equipped with attack detectors. Neurocomputing, 2017, 219: 99–106

  6. 6

    Fridman E, Shaked U. An improved delay-dependent H filtering of linear neutral systems. IEEE Trans Signal Process, 2004, 52: 668–673

  7. 7

    Wang Z D, Liu Y R, Liu X H. H filtering for uncertain stochastic time-delay systems with sector-bounded nonlinearities. Automatica, 2008, 44: 1268–1277

  8. 8

    Yang F W, Li Y M, Liu X H. Robust error square constrained filter design for systems with non-Gaussian noises. IEEE Signal Process Lett, 2008, 15: 930–933

  9. 9

    Wei G L, Wang Z D, Shen B. Error-constrained filtering for a class of nonlinear time-varying delay systems with non-Gaussian noises. IEEE Trans Autom Control, 2010, 55: 2876–2882

  10. 10

    Yue H, Wang H. Minimum entropy control of closed-loop tracking error for dynamic stochastic systems. IEEE Trans Autom Control, 2003, 48: 118–122

  11. 11

    Guo L, Wang H. Minimum entropy filtering for multivariate stochastic systems with non-Gaussian noises. IEEE Trans Autom Control, 2006, 51: 695–700

  12. 12

    Zhou J L, Zhou D H, Wang H, et al. Distribution function tracking filter design using hybrid characteristic functions. Automatica, 2010, 46: 101–109

  13. 13

    Saridis G N. Entropy formulation of optimal and adaptive control. IEEE Trans Autom Control, 1988, 33: 713–721

  14. 14

    Petersen I R, James M R, Dupuis P. Minimax optimal control of stochastc uncertain systems with relative entropy constraint. IEEE Trans Autom Control, 2000, 45: 398–412

  15. 15

    Zhang J H, Chu C C, Munoz J, et al. Minimum entropy based run-to-run control for semiconductor processes with uncertain metrology delay. J Process Control, 2009, 19: 1688–1697

  16. 16

    Yi Y, Guo L, Wang H. Adaptive statistic tracking control based on two steps neural networks with time delays. IEEE Trans Neural Netw, 2009, 20: 420–429

  17. 17

    Ding J L, Chai T Y, Wang H. Offline modeling for product quality prediction of mineral processing using modeling error PDF shaping and entropy minimization. IEEE Trans Neural Netw, 2011, 22: 408–419

  18. 18

    Liu Y L, Wang H, Guo L. Observer-based feedback controller design for a class of stochastic systems with non-Gaussian variables. IEEE Trans Autom Control, 2015, 60: 1445–1450

  19. 19

    Wang H. Bounded Dynamic Stochastic Systems: Modeling and Control. London: Springer-Verlag, 2000

  20. 20

    Guo L, Wang H. Stochastic Distribution Control System Design: a Convex Optimization Approach. London: Springer-Verlag, 2010

  21. 21

    Papoulis A. Probability, Random Variables and Stochastic Processes. 3rd ed. New York: McGraw-Hill, 1991

  22. 22

    Guo L, Yin L P, Wang H, et al. Entropy optimization filtering for fault isolation of nonlinear non-Gaussian stochastic systems. IEEE Trans Autom Control, 2009, 54: 804–810

  23. 23

    Afshar P, Wang H, Chai T Y. An ILC-based adaptive control for general stochastic systems with strictly decreasing entropy. IEEE Trans Neural Netw, 2009, 20: 471–482

  24. 24

    Renyi A. A Diary on Information Theory. New York: Wiley, 1987

  25. 25

    Ren M F, Zhang J H, Wang H. Minimized tracking error randomness control for nonlinear multivariate and non- Gaussian systems using the generalized density evolution equation. IEEE Trans Autom Control, 2014, 59: 2486–2490

  26. 26

    Karny M. Toward fully probabilistic control design. Automatica, 1996, 32: 1719–1722

  27. 27

    Yin L P, Zhang H Y, Guo L. Data driven output joint probability density function control for multivariate non-linear non-Gaussian systems. IET Control Theory Appl, 2015, 9: 2697–2703

  28. 28

    Liu Y, Wang H, Hou C H. Sliding-mode control design for nonlinear systems using probability density function shaping. IEEE Trans Neural Netw Learn Syst, 2014, 25: 332–343

Download references


This work was supported by National Natural Science Foundation of China (Grant Nos. 61320106010, 61573019, 61627810). The authors would like to thank the anonymous reviewers for their constructive comments which helped to improve the quality and presentation of this paper significantly.

Author information

Correspondence to Yan Wang.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Tian, B., Wang, Y. & Guo, L. Entropy optimization based filtering for non-Gaussian stochastic systems. Sci. China Inf. Sci. 60, 120203 (2017).

Download citation


  • non-Gaussian systems
  • joint probability density function (JPDF)
  • quadratic information potential
  • relative entropy
  • optimal filtering