Adaptive Monte Carlo Maximum Likelihood
We consider Monte Carlo approximations to the maximum likelihood estimator in models with intractable norming constants. This paper deals with adaptive Monte Carlo algorithms, which adjust control parameters in the course of simulation. We examine asymptotics of adaptive importance sampling and a new algorithm, which uses resampling and MCMC. This algorithm is designed to reduce problems with degeneracy of importance weights. Our analysis is based on martingale limit theorems. We also describe how adaptive maximization algorithms of Newton-Raphson type can be combined with the resampling techniques. The paper includes results of a small scale simulation study in which we compare the performance of adaptive and non-adaptive Monte Carlo maximum likelihood algorithms.
KeywordsMaximum likelihood Importance sampling Adaptation MCMC Resampling
This work was partially supported by Polish National Science Center No. N N201 608 740.
- 1.Attouch H (1984) Variational convergence of functions and operators. Pitman, BostonGoogle Scholar
- 6.Hall P, Heyde CC (1980) Martinagale limit theory and its application. Academic Press, New YorkGoogle Scholar
- 7.Miasojedow B, Niemiro W (2014) Debiasing MCMC via importance sampling-resampling. In preparationGoogle Scholar
- 10.Møller BJ, Pettitt AN, Reeves R, Berthelsen KK (2006) An efficient Markov chain Monte Carlo method for distributions with intractable normalising constants. Biometrika 93:451–458Google Scholar
- 14.Rockafellar TJ, Wets RJ-B (2009) Variational analysis, 3rd edn. Springer, New YorkGoogle Scholar
- 16.Younes L (1988) Estimation and annealing for Gibbsian fields. Annales de l’I H P sec B 24(2):269–294Google Scholar