A Maximum Entropy Approach for Estimating Nonlinear Dynamic Models

  • Amos Golan
Conference paper
Part of the Fundamental Theories of Physics book series (FTPH, volume 105)


Given the objective of estimating the unknown parameters of a finite (and relatively small) data set, generated by some (possibly nonlinear) dynamic discrete time process, it is common to use a Kalman filter Maximum Likelihood (ML) approach, ML-type estimators or more recently a GMM (Imbens, Spady and Johnson, 1998) or BMOM (Zellner 1997, Tobias and Zellner 1997) estimators. All of the above ML-type methods (except the BMOM) require some distributional assumptions while the moment-type estimators require some data (or assumptions) on the moments of the underlying distribution that generated the data. Recently, Golan, Judge and Miller (1996) and Golan, Judge and Karp (1996) developed a Maximum Entropy (ME) framework for estimating dynamic models without distributional assumptions. In this paper the above ME formulation is extended for estimating nonlinear (chaotic) dynamic models. Under this new formulation, one views the errors as another set of unknown parameters to be estimated. Thus, for any data set, the estimation problem is ill-posed (under-determined) where the number of unknowns is always greater than the number of data points and therefore the ME approach is a natural way to estimate the parameters. After developing the basic primal entropy model, a computationally efficient dual, unconstrained approach is developed. Under this dual method, the optimization is done with respect to the Lagrange multipliers associated with each observation. The dual is also used to contrast this ME approach with the more traditional ML-type estimators. Statistics and inference procedures are developed as well. Monte Carlo and empirical results for estimating the parameters of noisy, chaotic systems are presented as well.


information maximum entropy nonlinear dynamic models 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Dufour, J.M. and E. Ghysels, eds. (1996), Special issue of the Journal of Econometrics, “Recent Developments in Econometrics of Structural Change” Vol, no. 70 Google Scholar
  2. Golan, A. and G. Judge, (1996) A Maximum Entropy Approach to Empircal Likelihood and Inference (Working Paper), UC BerkeleyGoogle Scholar
  3. Golan, A., G. Judge and L. Karp (1996) A Maximum Entropy Approach to Estimation and Inference in Dynamic Models, Journal of Economic Dynamics and Control, Vol. no, 20, pp. 559–582Google Scholar
  4. Golan, A., G. Judge and D. Miller (1996), Maximum Entropy Econometrics: Robust Estimation with Limited Data. John Wiley.Google Scholar
  5. Golan, A., G. Judge and J. Perloff (1997) Estimation and Inference with Censored and Ordered Multinomial Response Data, Journal of Econometrics, Vol. no. 79, pp. 23–51Google Scholar
  6. Hansen, L. (1982) Large Sample Properties of Generalized Method of Moments Estimators, Econometrica, Vol. no. 50, pp. 1029–1054Google Scholar
  7. Imbens, G.W., R. Spady and P. Johnson (1998) Information Theoretic Approaches to Inference in Moment Condition Models, Econometrica, Vol. no. 66, pp. 333–357Google Scholar
  8. Mittelhammer, R.C. and N.S. Cardell (1996) On the Consistency and Asymptotic Normality of the Data-Constrained GME Estimator in the GLM (Working Paper), Washington State U.Google Scholar
  9. Montgomery, A.L., V. Zarnowitz, R.S. Tsay and G.C. Tiao (1998) Foreccasting the U.S. Unemployment Rate, Journal of the American Statistical Association, Vol. no. 93, pp. 478–493Google Scholar
  10. Owen, A. (1990) Empirical Likelihood Ratio Confidence Regions, The Annals of Statistics, Vol. no. 18, pp. 90–120Google Scholar
  11. Qin, J. and J. Lawless (1994) Empirical Likelihood and General Estimating Equations, The Annals of Statistics, Vol. no. 22, pp. 300–325Google Scholar
  12. Tobias, J. and A. Zellner (l997) Further Results on BMOM Analysis of the Multiple Regression Analysis, H.G.B. Alexander Research Foundation. University of Chicago Zellner, A. (1997) The Bayesian Method of Moments (BMOM): Theory and Applications, in Advances in Econometrics, T. Fomby and R. C. Hill eds pp. 85-106Google Scholar
  13. Zellner, A. (1994) Time-series Analysis, Forecasting, and Econometric Modeling: the Structural Econometric Modeling, Time-Series Analysis (SEMTSA) Approach Journal of Forecasting, Vol. no. 13, pp. 215–233Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 1999

Authors and Affiliations

  • Amos Golan
    • 1
  1. 1.American UniversityUSA

Personalised recommendations