Abstract
Estimation of rare-event probabilities in high-dimensional settings via importance sampling is a difficult problem due to the degeneracy of the likelihood ratio. In fact, it is generally recommended that Monte Carlo estimators involving likelihood ratios should not be used in such settings. In view of this, we develop efficient algorithms based on conditional Monte Carlo to estimate rare-event probabilities in situations where the degeneracy problem is expected to be severe. By utilizing an asymptotic description of how the rare event occurs, we derive algorithms that involve generating random variables only from the nominal distributions, thus avoiding any likelihood ratio. We consider two settings that occur frequently in applied probability: systems involving bottleneck elements and models involving heavy-tailed random variables. We first consider the problem of estimating ℙ(X 1+⋅⋅⋅+X n >γ), where X 1,…,X n are independent but not identically distributed (ind) heavy-tailed random variables. Guided by insights obtained from this model, we then study a variety of more general settings. Specifically, we consider a complex bridge network and a generalization of the widely popular normal copula model used in managing portfolio credit risk, both of which involve hundreds of random variables. We show that the same conditioning idea, guided by an asymptotic description of the way in which the rare event happens, can be used to derive estimators that outperform existing ones.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Asmussen, S., & Binswanger, K. (1997). Simulation of ruin probabilities for subexponential claims. ASTIN Bulletin, 27, 297–318.
Asmussen, S., & Glynn, P. W. (2007). Stochastic simulation: algorithms and analysis. New York: Springer.
Asmussen, S., & Kroese, D. P. (2006). Improved algorithms for rare event simulation with heavy tails. Advances in Applied Probability, 38, 545–558.
Asmussen, S., Binswanger, K., & Højgaard, B. (2000). Rare events simulation for heavy-tailed distributions. Bernoulli, 6, 303–322.
Asmussen, S., Rubinstein, R. Y., & Kroese, D. P. (2005). Heavy tails, importance sampling and cross-entropy. Stochastic Models, 21, 57–76.
Bassmaboo, A., Juneja, S., & Zeevi, A. (2008). Portfolio credit risk with extremal dependence: Asymptotic analysis and efficient simulation. Operations Research, 56(3), 593–606.
Botev, Z. I., & Kroese, D. P. (2008). An efficient algorithm for rare-event probability estimation, combinatorial optimization, and counting. Methodology and Computing in Applied Probability, 10(4), 471–505.
Cruz, M. (2002). Modeling, measuring, and hedging operational risk. New York: Wiley.
Geweke, J. (1993). Bayesian treatment of the independent student-t linear model. Journal of Applied Econometrics, 8, S19–S40.
Glasserman, P. (2004). Monte Carlo methods in financial engineering. New York: Springer.
Glasserman, P., & Li, J. (2005). Importance sampling for portfolio credit risk. Management Science, 51(11), 1643–1656.
Glasserman, P., Heidelberger, P., & Shahabuddin, P. (2002). Portfolio value-at-risk with heavy-tailed risk factors. Mathematical Finance, 12(3), 239–269.
Goodman, L. A. (1962). The variance of the product of k random variables. Journal of the American Statistical Association, 57(297), 54–60.
Gupton, G., Finger, C., & Bhatia, M. (1997). Creditmetrics technical document (Technical report). J. P. Morgan & Co., New York.
Juneja, S. (2007). Estimating tail probabilities of heavy tailed distributions with asymptotically zero relative error. Queueing Systems, 57(2–3), 115–127.
Juneja, S., & Shahabuddin, P. (2002). Simulating heavy tailed processes using delayed hazard rate twisting. ACM Transactions on Modeling and Computer Simulation, 12(2), 94–118. ISSN 1049-3301.
Kroese, D. P., & Rubinstein, R. Y. (2004). The transform likelihood ratio method for rare event simulation with heavy tails. Queueing Systems, 46, 317–351.
Li, D. (2000). On default correlations: a copula function approach. Journal of Fixed Income, 9, 43–54.
Mashal, R., & Zeevi, A. (2002). Beyond correlation: Extreme co-movements between financial assets (Technical report). Columbia University, New York.
Rubinstein, R. Y. (2007). How to deal with the curse of dimensionality of likelihood ratios in Monte Carlo simulation (Technical report). Technion, Haifa, Israel.
Rubinstein, R. Y., & Kroese, D. P. (2004). The cross-entropy method: a unified approach to combinatorial optimization Monte-Carlo simulation, and machine learning. New York: Springer.
Rubinstein, R. Y., & Kroese, D. P. (2007). Simulation and the Monte Carlo method (2nd ed.). New York: Wiley.
Author information
Authors and Affiliations
Corresponding author
Additional information
This work is supported by the Australian Research Council (Discovery Grant DP0985177).
J.C.C. Chan would like to acknowledge financial support from the Australian Government and the University of Queensland through IPRS and UQRS scholarships.
Rights and permissions
About this article
Cite this article
Chan, J.C.C., Kroese, D.P. Rare-event probability estimation with conditional Monte Carlo. Ann Oper Res 189, 43–61 (2011). https://doi.org/10.1007/s10479-009-0539-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10479-009-0539-y