Abstract
The need for expressing uncertainty in stochastic simulation systems is widely recognized. However, the emphasis in uncertainty has been directed toward assessing simulation model input parameter uncertainty, while the analysis of simulation output uncertainty is deduced from the input uncertainty. Most recently used methods to assess uncertainty include Delta-Method approaches, Resampling method, Bayesian Analysis method and so on. The problem for all these methods is that the typical simulation user is not particularly proficient in statistics, and so is unlikely to be aware of appropriate sensitivity and/or uncertainty analyses. This suggests the need for a transparent, implementable and efficient method for understanding uncertainty, especially for simulation output uncertainty. In this paper, we propose a simple and straightforward framework to assess stochastic simulation output uncertainty based on Bayesian Melding. We firstly assume the form of probability distribution function of simulation output. We also assume that the final output uncertainty is the weight sum of uncertainty for every simulation output and the weight of each simulation run is proportional to its probability. The advantage of these assumptions is that to describe the simulation output uncertainty in the form of probability distribution function after limited simulation runs, we need only to do two things (1) to estimate parameters in the simulation output probability distribution function and (2) to calculate weight for each simulation. Both of them are discussed in detail in this paper.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Morgan, M.G., Henrion, M.: Uncertainty: A guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis, 1st edn. Cambridge University Press, New York (1990)
Beck, M.B.: Water Quality Modeling: A Review of the Analysis of Uncertainty. Water Resources Research 23(8), 1393–1442 (1987)
Bogen, K.T., Spear, R.C.: Integrating Uncertainty and Interindividual Variability in Environmental Risk Assessment. Risk Analysis 7(4), 427–436 (1987)
Oberkampf, W.L., et al.: Estimation of Total Uncertainty in Modeling and Simulation. SANDIA REPORT, SAND2000-0824, Unlimited Release (2000)
Refsgaard, J.C., Henriksen, H.J.: Modelling guidelines – terminology and guiding principles. Advances in Water Resources 27(1), 71–82 (2004)
Van Asselt, M.B.A., Rotmans, J.: Uncertainty in integrated assessment modelling – from positivism to pluralism. Climatic Change, 75–105 (2002)
Beven, K.: Rainfall-runoff Modelling: The Primer. John Wiley & Sons, Chichester (2000)
Beven, K., Binley, A.: The future of distributed models: model calibration and uncertainty prediction. Hydrological Processes 6(3), 279–298 (1992)
Christensen, S.: A synthetic groundwater modelling study of the accuracy of GLUE uncertainty intervals. Nordic Hydrology 35(1), 45–59 (2003)
Neuman, S.P.: Maximum likelihood Bayesian averaging of uncertain model predictions. Stochastic Environmental Research and Risk Assessment 17(5), 291–305 (2003)
Korving, H., Van Noortwijk, J.M., Van Gelder, P.H.A.J.M., Parkhi, R.S.: Coping with uncertainty in sewer system rehabilitation In: van Gelder, B. (eds.) Safety and Reliability. Swets & Zeitlinger, Lisse (2003)
Casman, E.A., Morgan, M.G., Dowlatabadi, H.: Mixed levels of uncertainty in complex policy models. Risk Analysis 19(1), 33–42 (1999)
Raftery, A.E., Givens, G.H., Zeh, J.E.: Inference from a deterministic population dynamics model for bowhead whales (with discussion). Journal of the American Statistical Association , 402–416 (1995)
Poole, D., Raftery, A.E.: Inference for deterministic simulation models: the Bayesian melding approach. Journal of the American Statistical Association, 1244–1255 (2000)
Sevcikova, H., et al.: Assessing uncertainty in urban simulations using Bayesian Melding. Transport. Res. Part B (2006), doi:10.1016/j.trb.2006.11.001
Cheng, R.C.H.: Selecting input models. In: Tew, J.D., Manivannan, S., Sadowski, D.A., Seila, A.F. (eds.) Proceedings of the 1994 Winter Simulation Conference, pp. 184–191. IEEE, Piscataway, NJ (1994)
Cheng, R.C.H., Holland, W.: Sensitivity of computer simulation experiments to errors in input data. Journal of Statistical Computation and Simulation 57, 219–241 (1997)
Cheng, R.C.H., Holland, W.: Two-point methods for assessing variability in simulation output. Journal of Statistical Computation and Simulation, 183–205 (1998)
Cheng, R.C.H., Holland, W.: Calculation of confidence intervals for simulation output. submitted for publication, 2003
Zouaoui, F., Wilson, J.R.: Accounting for input model and parameter uncertainty in simulation. In: Peters, B.A., Smith, J.S., Medeiros, D.J., Rohrer, M.W. (eds.) Proceedings of the 2001 Winter Simulation Conference, pp. 290–299. IEEE, Piscataway, NJ (2001)
Zouaoui, F., Wilson, J.R.: Accounting for parameter uncertainty in simulation input modeling. In: Peters, B.A., Smith, J.S., Medeiros, D.J., Rohrer, M.W. (eds.) Proceedings of the 2001 Winter Simulation Conference, pp. 354–363. IEEE, Piscataway, NJ (2001)
Cheng, R.C.H.: Analysis of simulation output by resampling. International Journal of Simulation: Systems, Science & Technology 1, 51–58 (2000)
Cheng, R.C.H.: Analysis of simulation experiments by bootstrap resampling. In: Peters, B.A., Smith, J.S., Medeiros, D.J., Rohrer, M.W. (eds.) Proceedings of the 2001 Winter Simulation Conference, pp. 179–186. IEEE, Piscataway, NJ (2001)
Barton, R.R., Schruben, L.W.: Uniform and bootstrap resampling of empirical distributions. In: Evans, G.W., Mollaghasemi, M., Russell, E.C., Biles, W.E. (eds.) Proceedings of the 1993 Winter Simulation Conference, pp. 503–508. IEEE, Piscataway, NJ (1993)
Chick, S.E.: Bayesian analysis for simulation input and output. In: Andradóttir, S., Healy, K.J., Withers, D.H., Nelson, B.L. (eds.) Proceedings of the 1997 Winter Simulation Conference, pp. 253–260. IEEE, Piscataway, NJ (1997)
Chick, S.E.: Steps to implement Bayesian input distribution selection. In: Farrington, P.A., Black Nembhard, H., Sturrock, D.T., Evans, G.W. (eds.) Proceedings of the 1999 Winter Simulation Conference, pp. 317–324. IEEE, Piscataway, NJ (1999)
Chick, S.E.: Bayesian methods for simulation. In: Joines, J.A., Barton, R.R., Kang, K., Fishwick, P.A. (eds.) Proceedings of the 2000 Winter Simulation Conference, pp. 109–118. IEEE, Piscataway, NJ (2000)
Chick, S.E.: Input distribution selection for simulation experiments: accounting for input uncertainty. Operations Research, 744–758 (2001)
Chick, S.E., Ng, S.H.: Joint criterion for factor identification and parameter estimation. In: Yücesan, E., Chen, C.-H., Snowdon, J.L., Charnes, J.M. (eds.) Proceedings of the 2002 Winter Simulation Conference, pp. 400–406. IEEE, Piscataway, NJ (2002)
Kleijnen, J.P.C.: Sensitivity analysis versus uncertainty analysis: when to use what? In: Grasman, J., van Straten, G. (eds.) Predictability and Nonlinear Modeling in Natural Sciences and Economics, Kluwer Academic, Dordrecht (1994)
Kleijnen, J.P.C.: Five-stage procedure for the evaluation of simulation models through statistical techniques. In: Charnes, J.M., Morrice, D.J., Brunner, D.T., Swain, J.J. (eds.) Proceedings of the 1996 Winter Simulation Conference, pp. 248–254. IEEE, Piscataway, NJ (1996)
Shane, G.: Henderson: Input Model Uncertainty: Why Do We Care And What Should We Do About It? In: Proceedings of the 2003 Winter Simulation Conference (2003)
Sun, T., Wang, J.: A traffic cellular automata model based on road network grids and its spatial & temporal resolution’s influences on simulation. Simulation Modelling Practice and Theory (2007), http://dx.doi.org/10.1016/j.simpat.2007.04.010
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sun, T., Wang, J. (2007). A Simple Model for Assessing Output Uncertainty in Stochastic Simulation Systems. In: Gelbukh, A., Kuri Morales, Á.F. (eds) MICAI 2007: Advances in Artificial Intelligence. MICAI 2007. Lecture Notes in Computer Science(), vol 4827. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-76631-5_32
Download citation
DOI: https://doi.org/10.1007/978-3-540-76631-5_32
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-76630-8
Online ISBN: 978-3-540-76631-5
eBook Packages: Computer ScienceComputer Science (R0)