Statistics and Computing

, Volume 22, Issue 5, pp 1031–1040 | Cite as

Improved cross-entropy method for estimation

  • Joshua C. C. Chan
  • Dirk P. Kroese


The cross-entropy (CE) method is an adaptive importance sampling procedure that has been successfully applied to a diverse range of complicated simulation problems. However, recent research has shown that in some high-dimensional settings, the likelihood ratio degeneracy problem becomes severe and the importance sampling estimator obtained from the CE algorithm becomes unreliable. We consider a variation of the CE method whose performance does not deteriorate as the dimension of the problem increases. We then illustrate the algorithm via a high-dimensional estimation problem in risk management.


Cross-entropy Variance minimization Importance sampling Kullback-Leibler divergence Rare-event simulation Likelihood ratio degeneracy t copula 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Asmussen, S., Glynn, P.W.: Stochastic Simulation: Algorithms and Analysis. Springer, New York (2007) MATHGoogle Scholar
  2. Asmussen, S., Rubinstein, R.Y., Kroese, D.P.: Heavy tails, importance sampling and cross-entropy. Stoch. Models 21, 57–76 (2005) MathSciNetMATHCrossRefGoogle Scholar
  3. Bassamboo, A., Juneja, S., Zeevi, A.: Portfolio credit risk with extremal dependence: asymptotic analysis and efficient simulation. Oper. Res. 56(3), 593–606 (2008) MathSciNetMATHCrossRefGoogle Scholar
  4. Chan, J.C.C., Eisenstat, E.: Marginal likelihood estimation with the cross-entropy method (2011, submitted) Google Scholar
  5. Chan, J.C.C., Kroese, D.P.: Efficient estimation of large portfolio loss probabilities in t-copula models. Eur. J. Oper. Res. 205, 361–367 (2010) MathSciNetMATHCrossRefGoogle Scholar
  6. Chan, J.C.C., Kroese, D.P.: Rare-event probability estimation with conditional Monte Carlo. Ann. Oper. Res. (2011, forthcoming). doi: 10.1007/s10479-009-0539-y
  7. Chan, J.C.C., Glynn, P.W., Kroese, D.P.: A comparison of cross-entropy and variance minimization strategies. J. Appl. Probab. 48A, 183–194 (2011) MathSciNetMATHCrossRefGoogle Scholar
  8. Chib, S.: Marginal likelihood from the Gibbs output. J. Am. Stat. Assoc. 90, 1313–1321 (1995) MathSciNetMATHGoogle Scholar
  9. Chib, S., Jeliazkov, I.: Marginal likelihood from the Metropolis-Hastings output. J. Am. Stat. Assoc. 96, 270–281 (2001) MathSciNetMATHCrossRefGoogle Scholar
  10. Cornebise, J., Moulines, E., Olsson, J.: Adaptive methods for sequential importance sampling with application to state space models. Stat. Comput. 18, 461–480 (2008) MathSciNetCrossRefGoogle Scholar
  11. de Boer, P.T., Kroese, D.P., Rubinstein, R.Y.: A fast cross-entropy method for estimating buffer overflows in queueing networks. Manag. Sci. 50, 883–895 (2004) MATHCrossRefGoogle Scholar
  12. Gelfand, A.E., Dey, D.K.: Bayesian model choice: asymptotics and exact calculations. J. R. Stat. Soc. B 56(3), 501–514 (1994) MathSciNetMATHGoogle Scholar
  13. Gelman, A., Meng, X.: Simulating normalizing constants: from importance sampling to bridge sampling to path sampling. Stat. Sci. 13, 163–185 (1998) MathSciNetMATHCrossRefGoogle Scholar
  14. Hui, K.P., Bean, N., Kraetzl, M., Kroese, D.P.: The cross-entropy method for network reliability estimation. Ann. Oper. Res. 134, 101–118 (2005) MathSciNetMATHCrossRefGoogle Scholar
  15. Juneja, S., Shahabuddin, P.: Simulating heavy tailed processes using delayed hazard rate twisting. ACM Trans. Model. Comput. Simul. 12, 94–118 (2002) CrossRefGoogle Scholar
  16. Keith, J.M., Kroese, D.P., Sofronov, G.Y.: Adaptive independence samplers. Stat. Comput. 18, 409–420 (2008) MathSciNetCrossRefGoogle Scholar
  17. Kroese, D.P.: The cross-entropy method. In: Wiley Encyclopedia of Operations Research and Management Science. Wiley, New York (2011) Google Scholar
  18. Kroese, D.P., Rubinstein, R.Y.: The transform likelihood ratio method for rare event simulation with heavy tails. Queueing Syst. 46, 317–351 (2004) MathSciNetMATHCrossRefGoogle Scholar
  19. Newton, M.A., Raftery, A.E.: Approximate bayesian inference with the weighted likelihood bootstrap. J. R. Stat. Soc. B 56, 3–48 (1994) MathSciNetMATHGoogle Scholar
  20. Orsak, G.C.: A note on estimating false alarm rates via importance sampling. IEEE Trans. Commun. 41(9), 1275–1277 (1993) MATHCrossRefGoogle Scholar
  21. Philippe, A.: Simulation of right and left truncated gamma distribution by mixtures. Stat. Comput. 7, 173–181 (1997) CrossRefGoogle Scholar
  22. Ridder, A.: Importance sampling simulations of Markovian reliability systems using cross-entropy. Ann. Oper. Res. 134, 119–136 (2005) MathSciNetMATHCrossRefGoogle Scholar
  23. Robert, C.P.: Simulation of truncated normal variables. Stat. Comput. 5, 121–125 (1995) MATHCrossRefGoogle Scholar
  24. Rubinstein, R.Y.: Optimization of computer simulation models with rare events. Eur. J. Oper. Res. 99, 89–112 (1997) CrossRefGoogle Scholar
  25. Rubinstein, R.Y.: The cross-entropy method for combinatorial and continuous optimization. Methodol. Comput. Appl. Probab. 1(2), 127–190 (1999) MathSciNetMATHCrossRefGoogle Scholar
  26. Rubinstein, R.Y., Glynn, P.W.: How to deal with the curse of dimensionality of likelihood ratios in Monte Carlo simulation. Stoch. Models 25, 547–568 (2009) MathSciNetMATHCrossRefGoogle Scholar
  27. Rubinstein, R.Y., Kroese, D.P.: The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization Monte-Carlo Simulation, and Machine Learning. Springer, New York (2004) MATHGoogle Scholar
  28. Smith, P.J., Shafi, M., Gao, H.: Quick simulation: a review of importance sampling techniques in communications systems. IEEE J. Sel. Areas Commun. 15(4), 597–613 (1997) CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  1. 1.Research School of EconomicsAustralian National UniversityCanberraAustralia
  2. 2.Department of MathematicsUniversity of QueenslandBrisbaneAustralia

Personalised recommendations