Iterative parameter estimate with batched binary-valued observations

基于给定二值观测样本数据的迭代参数估计算法

Abstract

In this paper, we consider linear system identification with batched binary-valued observations. We constructed an iterative parameter estimate algorithm to achieve the maximum likelihood (ML) estimate. The first interesting result was that there exists at most one finite ML solution for this specific maximum likelihood problem, which was induced by the fact that the Hessian matrix of the log-likelihood function was negative definite under binary data and Gaussian system noises. The global concave property and local strongly concave property of the log-likelihood function were obtained. Under mild conditions on the system input, we proved that the ML function has a unique maximum point. The second main result was that the ML estimate was consistent under persistent excitation inputs, which infers the effectiveness of ML estimate. Finally, the proposed iterative estimate algorithm converged to a fixed vector with an exponential rate that was proved by constructing a Lyapunov function. A more interesting result was that the limit of the iterative algorithm achieved the maximization of the ML function. Numerical simulations are illustrated to support the theoretical results obtained in this paper well.

摘要

创新点

针对给定二值观测样本数据的参数辨识问题, 我们构造了一种迭代算法并进行了相关的理论研究。 首先, 找出了一个似然函数存在极大值点的充分必要条件, 并通过计算对数似然函数的 Hessian 矩阵, 证明了该问题最多只有一个极大似然估计点。 然后, 通过构建Lyapunov函数, 证明了在持续激励条件下, 该迭代算法能达到指数收敛速度, 而且迭代的极限点就是似然函数唯一的最大值点, 这说明了迭代算法具有极强的稳健性。

This is a preview of subscription content, access via your institution.

References

  1. 1

    Wang L Y, Zhang J F, Yin G, et al. System Identification With Quantized Observations. Boston: BirkhAauser, 2010

    Book  MATH  Google Scholar 

  2. 2

    Wang L Y, Zhang J F, Yin G G. System identification using binary sensors. IEEE Trans Autom Control, 2003, 48: 1892–1907

    MathSciNet  Article  Google Scholar 

  3. 3

    Ribeiro A, Giannakis G B, Roumeliotis S I. Soi-kf: distributed kalman filtering with low-cost communications using the sign of innovations. IEEE Trans Signal Process, 2006, 54: 4782–4795

    Article  Google Scholar 

  4. 4

    Marelli D, You K Y, Fu M Y. Identification of arma models using intermittent and quantized output observations. Automatica, 2013, 49: 360–369

    MathSciNet  Article  MATH  Google Scholar 

  5. 5

    Chen T S, Zhao Y L, Ljung L. Impulse response estimation with binary measurements: a regularized fir model approach. In: Proceedings of the 16th IFAC Symposium on System Identification, Brussels, 2012. 113–118

    Google Scholar 

  6. 6

    Godoya B I, Goodwin G C, Agueroa J C, et al. On identification of fir systems having quantized output data. Automatica, 2011, 47: 1905–1915

    MathSciNet  Article  Google Scholar 

  7. 7

    Severini T A. Likelihood Methods in Statistics. Oxford: Oxford University Press, 2000

    MATH  Google Scholar 

  8. 8

    Mclachlan G J, Krishnan T. The EM Algorithm and Extensions. 2nd ed. Hoboken: John Wiley & Sons Inc, 2008

    Book  MATH  Google Scholar 

  9. 9

    Newcomb S. A generalized theory of the combination of observations so as to obtain the best result. American J Math, 1886, 8: 343–366

    MathSciNet  Article  MATH  Google Scholar 

  10. 10

    Beale E M L, Little R J A. Missing values in multivariate analysis. J Royal Stat Soc Ser B (Methodol), 1975, 37: 129–145

    MathSciNet  MATH  Google Scholar 

  11. 11

    Buck S F. A method of estimation of missing values in multivariate data suitable for use with an electronic computer. J Royal Stat Soc B (Methodol), 1960, 22: 302–306

    MathSciNet  MATH  Google Scholar 

  12. 12

    Hartley H O. Maximum likelihood estimation from incomplete data. Biometrics, 1958, 14: 174–194

    Article  MATH  Google Scholar 

  13. 13

    Healy M J R, Westmacott M. Missing values in experimaents analyzed on automatic computers. Appl Stat, 1966, 5: 203–206

    Article  Google Scholar 

  14. 14

    Mckendrick A G. Applications of mathematics to medical problems. In: Proceedings of the Edinburgh Mathematical Society. Cambridge: Cambridge University Press, 1926, 44: 98–130

    Article  Google Scholar 

  15. 15

    Orchard T, Woodbury M A. A missing information principle: theory and applications. In: Proceedings of the 6th Berkeley Symposium on Mathematical Statistics and Probability. Berkeley: University of California Press, 1972, 1: 697–715

    MathSciNet  MATH  Google Scholar 

  16. 16

    Dempster A P, Laird N M, Rubin D B. Maximum likelihood from incomplete data via the em algorithm. J Royal Stat Soc Ser B (Methodol), 1977, 39: 1–38

    MathSciNet  MATH  Google Scholar 

  17. 17

    Render R A, Walker H F. Mixture densities, maximum likelihood and the em algorithm. Soc Indust Appl Math, 1984, 26: 195–239

    MathSciNet  MATH  Google Scholar 

  18. 18

    Wolynetz M S. Maximum likelihood estimation from confined and censored normal data. J Royal Stat Soc Ser C (Appl Stat), 1979, 28: 185–195

    MATH  Google Scholar 

  19. 19

    Byrne W. Alternating minimization and boltzmann machine learning. IEEE Trans Neural Netw, 1992, 3: 612–620

    Article  Google Scholar 

  20. 20

    Boyles R A. On the convergence of the em algorithm. J Royal Stat Soc Ser B (Methodol), 1983, 45: 47–50

    MathSciNet  MATH  Google Scholar 

  21. 21

    Wu C F. On the convergence properties of the em algorithm. Annals Stat, 1983, 11: 95–103

    MathSciNet  Article  MATH  Google Scholar 

  22. 22

    Ljung L. System Identification. Boston: BirkhAauser, 1998

    Book  MATH  Google Scholar 

  23. 23

    Bi W J, Zhao Y L, Liu C X, et al. Set-valued analysis for genome-wide association studies of complex diseases. In: Proceedings of the 32st Chinese Control Conference, Xi’an, 2013. 8262–8267

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Yanlong Zhao.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zhao, Y., Bi, W. & Wang, T. Iterative parameter estimate with batched binary-valued observations. Sci. China Inf. Sci. 59, 052201 (2016). https://doi.org/10.1007/s11432-015-5304-z

Download citation

Keywords

  • Binary-valued observation
  • maximum likelihood estimate
  • strongly convex
  • system identification
  • exponential rate

关键词

  • 二值观测
  • 极大似然估计
  • 强凸性
  • 系统辨识
  • 指数速度