Abstract
This chapter describes simulation output analysis, specifically estimating means, probabilities and quantiles of simulation output, along with measures of error on these estimates. The distinction between risk and error is emphasized, along with the impact of uncertainty about the input processes that drive the simulation.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
This central limit theorem is obtained by noting that \(\Pr \{\sqrt{n}(\widehat{{\vartheta}} - {\vartheta}) \leq y\} =\Pr \{\widehat{ {\vartheta}} \leq {\vartheta} + y/\sqrt{n}\} =\Pr \{\widehat{ F}({\vartheta} + y/\sqrt{n}) > q\}\) and then using the fact that the central limit theorem for averages applies to \(\widehat{F}\).
References
Albright, S. C. (2007). VBA for modelers: Developing decision support systems using Microsoft Excel (2nd ed.). Belmont: Thompson Higher Eduction.
Alexopoulos, C. (2006). Statistical estimation in computer simulation. In S. G. Henderson & B. L. Nelson (Eds.), Handbooks in operations research and management science: Simulation. New York: North-Holland.
Andradóttir, S. (1999). Accelerating the convergence of random search methods for discrete stochastic optimization. ACM Transactions on Modeling and Computer Simulation, 9, 349–380.
Andradóttir, S. (2006a). An overview of simulation optimization via random search. In S. G. Henderson & B. L. Nelson (Eds.), Handbooks in operations research and management science: Simulation. New York: North-Holland.
Andradóttir, S. (2006b). Simulation optimization with countably infinite feasible regions: Efficiency and convergence. ACM Transactions on Modeling and Computer Simulation, 16, 357–374.
Andradóttir, S., & Kim, S. (2010). Fully sequential procedures for comparing constrained systems via simulation. Naval Research Logistics, 57, 403–421.
Ankenman, B. E., & Nelson, B. L. (2012, in press). A quick assessment of input uncertainty. Proceedings of the 2012 Winter Simulation Conference, Berlin.
Asmussen, S. & Glynn, P. W., (2007). Stochastic simulation: Algorithms and analysis. New York: Springer.
Batur, D., & Kim, S. (2010). Finding feasible systems in the presence of constraints on multiple performance measures. ACM Transactions on Modeling and Computer Simulation, 20, 13:1–13:26.
Bechhofer, R. E., Santner, T. J., & Goldsman, D. (1995). Design and analysis of experiments for statistical selection, screening and multiple comparisons. New York: Wiley.
Biller, B., & Corlu, C. G. (2012). Copula-based multivariate input modeling. Surveys in Operations Research and Management Science, 17, 69–84.
Biller, B., & Ghosh, S. (2006). Multivariate input processes. In S. G. Henderson & B. L. Nelson (Eds.), Handbooks in operations research and management science: Simulation. New York: North-Holland.
Biller, B., & Nelson, B. L. (2003). Modeling and generating multivariate time-series input processes using a vector autoregressive technique. ACM Transactions on Modeling and Computer Simulation, 13, 211–237.
Biller, B., & Nelson, B. L. (2005). Fitting time series input processes for simulation. Operations Research, 53, 549–559.
Billingsley, P. (1995). Probability and measure (3rd ed.). New York: Wiley.
Boesel, J., Nelson, B. L., & Kim, S. (2003). Using ranking and selection to “clean up” after simulation optimization. Operations Research, 51, 814–825.
Bratley, P., Fox, B. L., & Schrage, L. E. (1987). A guide to simulation (2nd ed.). New York: Springer.
Burt, J. M., & Garman, M. B. (1971). Conditional Monte Carlo: A simulation technique for stochastic network analysis. Management Science, 19, 207–217.
Cario, M. C. & Nelson, B. L. (1998). Numerical methods for fitting and simulating autoregressive-to-anything processes. INFORMS Journal on Computing, 10, 72–81.
Cash, C., Nelson, B. L., Long, J., Dippold, D., & Pollard, W. (1992). Evaluation of tests for initial-condition bias. Proceedings of the 1992 Winter Simulation Conference (pp. 577–585). Piscataway, New Jersey: IEEE.
Chatfield, C. (2004). The analysis of time series: An introduction (6th ed.). Boca Raton: Chapman & Hall/CRC.
Chen, H. (2001). Initialization for NORTA: Generation of random vectors with specified marginals and correlations. INFORMS Journal on Computing, 13, 312–331.
Chow, Y. S., & Robbins, H. (1965). On the asymptotic theory of fixed-width sequential confidence intervals for the mean. The Annals of Mathematical Statistics, 36, 457–462.
Devroye, L. (1986). Non-uniform random variate generation. New York: Springer.
Devroye, L. (2006). Nonuniform random variate generation. In S. G. Henderson & B. L. Nelson (Eds.), Handbooks in operations research and management science: Simulation. New York: North-Holland.
Efron, B., & Tibshirani, R. J. (1993). An introduction to the bootstrap. Boca Raton: Chapman & Hall/CRC.
Elizandro, D., & Taha, H. (2008). Simulation of industrial systems: Discrete event simulation using Excel/VBA. New York: Auerbach Publications.
Frazier, P. I. (2010). Decision-theoretic foundations of simulation optimization. In J. J. Cochran (Ed.), Wiley encyclopedia of operations research and management sciences. New York: Wiley.
Fu, M. C. (2006). Gradient estimation. In S. G. Henderson & B. L. Nelson (Eds.), Handbooks in operations research and management science: Simulation. New York: North-Holland.
Gerhardt, I., & Nelson, B. L. (2009). Transforming renewal processes for simulation of nonstationary arrival processes. INFORMS Journal on Computing, 21, 630–640.
Ghosh, S., & Henderson, S. G. (2002). Chessboard distributions and random vectors with specified marginals and covariance matrix. Operations Research, 50, 820–834.
Glasserman, P. (2004). Monte Carlo methods in financial engineering. New York: Springer.
Glasserman, P., & Yao, D. D. (1992). Some guidelines and guarantees for common random numbers. Management Science, 38, 884–908.
Glynn, P. W. (2006). Simulation algorithms for regenerative processes. In S. G. Henderson & B. L. Nelson (Eds.), Handbooks in operations research and management science: Simulation. New York: North-Holland.
Glynn, P. W., & Whitt, W. (1992). The asymptotic validity of sequential stopping rules for stochastic simulations. The Annals of Applied Probability, 2, 180–198.
Goldsman, D., Kim, S., Marshall, S. W., & Nelson, B. L. (2002). Ranking and selection for steady-state simulation: Procedures and perspectives. INFORMS Journal on Computing, 14, 2–19.
Goldsman, D., & Nelson, B. L. (1998). Comparing systems via simulation. In J. Banks (Ed.), Handbook of simulation (pp. 273–306). New York: Wiley.
Goldsman, D., & Nelson, B. L. (2006). Correlation-based methods for output analysis. In S. G. Henderson & B. L. Nelson (Eds.), Handbooks in operations research and management science: Simulation. New York: North-Holland.
Gross, D., Shortle, J. F., Thompson, J. M., & Harris, C. M. (2008). Fundamentals of queueing theory (4th ed.). New York: Wiley.
Haas, P. J. (2002). Stochastic petri nets: Modeling, stability, simulation. New York: Springer.
Henderson, S. G. (2003). Estimation of nonhomogeneous Poisson processes from aggregated data. Operations Research Letters, 31, 375–382.
Henderson, S. G. (2006). Mathematics for simulation. In S. G. Henderson & B. L. Nelson (Eds.), Handbooks in operations research and management science: Simulation. New York: North-Holland.
Henderson, S. G., & Nelson, B. L. (2006). Stochastic computer simulation. In S. G. Henderson & B. L. Nelson (Eds.), Handbooks in operations research and management science: Simulation. New York: North-Holland.
Hill, R. R., & Reilly, C. H. (2000). The effects of coefficient correlation structure in two-dimensional knapsack problems on solution procedure performance. Management Science, 46, 302–317.
Hong, L. J., & Nelson, B. L. (2007a). Selecting the best system when systems are revealed sequentially. IIE Transactions, 39, 723–734.
Hong, L. J., & Nelson, B. L. (2007b). A framework for locally convergent random-search algorithms for discrete optimization via simulation. ACM Transactions on Modeling and Computer Simulation, 17, 19/1-19/22.
Hörmann, W. (1993). The transformed rejection method for generating Poisson random variables. Insurance: Mathematics and Economics, 12, 39–45.
Iravani, S. M. R., & Krishnamurthy, V. (2007). Workforce agility in repair and maintenance environments. Manufacturing and Service Operations Management, 9, 168–184.
Iravani, S. M., Van Oyen, M. P., & Sims, K. T. (2005). Structural flexibility: A new perspective on the design of manufacturing and service operations. Management Science, 51, 151–166.
Johnson, M. E. (1987). Multivariate statistical simulation. New York: Wiley.
Johnson, N. L., Kemp, A. W., & Kotz, S. (2005). Univariate discrete distributions (3rd ed.). New York: Wiley.
Johnson, N. L., Kotz, S., & Balakrishnan, N. (1994). Continuous univariate distributions (2nd ed., Vol. 1). New York: Wiley.
Johnson, N. L., Kotz, S., & Balakrishnan, N. (1995). Continuous univariate distributions (2nd ed., Vol. 2). New York: Wiley.
Johnson, N. L., Kotz, S., & Balakrishnan, N. (1997). Discrete multivariate distributions. New York: Wiley.
Kachitvichyanukul, V., & Schmeiser, B. (1990). Noninverse correlation induction: Guidelines for algorithm development. Journal of Computational and Applied Mathematics, 31, 173–180.
Karian, A. Z., & Dudewicz, E. J. (2000). Fitting statistical distributions: The generalized lambda distribution and generalized bootstrap methods. New York: CRC.
Karlin, S., & Taylor, H. M. (1975). A first course in stochastic processes (2nd ed.). New York: Academic.
Kelton, W. D. (2006). Implementing representations of uncertainty. In S. G. Henderson & B. L. Nelson (Eds.), Handbooks in operations research and management science: Simulation. New York: North-Holland.
Kelton, W. D., Smith, J. S., & Sturrock, D. T. (2011). Simio and simulation: Modeling, analysis and applications (2nd ed.). New York: McGraw-Hill.
Kim, S., & Nelson, B. L. (2001). A fully sequential procedure for indifference-zone selection in simulation. ACM Transactions on Modeling and Computer Simulation, 11, 251–273.
Kim, S., & Nelson, B. L. (2006). Selecting the best system. In S. G. Henderson & B. L. Nelson (Eds.), Handbooks in operations research and management science: Simulation. New York: North-Holland.
Knuth, D. E. (1998). The art of computer programming, Vol. 2: Seminumerical algorithms (3rd ed.). Boston: Addison-Wesley.
Kotz, S., Balakrishnan, N., & Johnson, N. L. (2000). Continuous multivariate distributions, Vol. 1, models and applications (2nd ed.). New York: Wiley.
Kulkarni, V. G. (1995). Modeling and analysis of stochastic systems. London: Chapman & Hall.
Lakhany, A., & Mausser, H. (2000). Estimating the parameters of the generalized lambda distribution. ALGO Research Quarterly, 3, 47–58.
Law, A. M. (2007). Simulation modeling and analysis (4th ed.). New York: McGraw-Hill.
Law, A. M., & Kelton, W. D. (2000). Simulation modeling and analysis (3rd ed.). New York: McGraw-Hill.
L’Ecuyer, P. (1988). Efficient and portable combined random number generators. Communications of the ACM, 31, 742–749.
L’Ecuyer, P. (1990). A unified view of IPA, SF, and LR gradient estimation techniques. Management Science, 36, 1364–1383.
L’Ecuyer, P. (1999). Good parameters and implementations for combined multiple recursive random number generators. Operations Research, 47, 159–164.
L’Ecuyer, P. (2006). Uniform random number generation. In S. G. Henderson & B. L. Nelson (Eds.), Handbooks in operations research and management science: Simulation. New York: North-Holland.
L’Ecuyer, P., & Simard, R. (2001). On the performance of birthday spacings tests for certain families of random number generators. Mathematics and Computers in Simulation, 55, 131–137.
L’Ecuyer, P., Simard, R., Chen, E. J., & Kelton, W. D. (2002). An object-oriented random-number package with many long streams and substreams. Operations Research, 50, 1073-1075.
Lee, S., Wilson, J. R., & Crawford, M. M. (1991). Modeling and simulation of a nonhomogeneous Poisson process having cyclic behavior. Communications in Statistics-Simulation and Computation, 20, 777–809.
Leemis, L. M. (1991). Nonparameteric estimation of the cumulative intensity function for a nonhomogeneous Poisson process. Management Science, 37, 886–900.
Leemis, L. M. (2006). Arrival processes, random lifetimes and random objects. In S. G. Henderson & B. L. Nelson (Eds.), Handbooks in operations research and management science: Simulation. New York: North-Holland.
Leemis, L. M., & McQueston, J. T. (2008). Univariate distribution relationships. The American Statistician, 62, 45–53.
Lehmann, E. L. (2010). Elements of large-sample theory. New York: Springer.
Lewis, T. A. (1981). Confidence intervals for a binomial parameter after observing no successes. The American Statistician, 35, 154.
Marse, K., & Roberts, S. D. (1983). Implementing a portable FORTRAN uniform (0,1) generator. Simulation, 41, 135–139.
Montgomery, D. C. (2009). Design and analysis of experiments (7th ed.). New York: Wiley.
Nádas, A. (1969). An extension of a theorem of Chow and Robbins on sequential confidence intervals for the mean. The Annals of Mathematical Statistics, 40, 667–671.
Nelson, B. L. (1990). Control-variate remedies. Operations Research, 38, 974–992.
Nelson, B. L. (1995). Stochastic modeling: Analysis and simulation. Mineola: Dover Publications, Inc.
Nelson, B. L. (2008). The MORE plot: Displaying measures of risk and error from simulation output. Proceedings of the 2008 Winter Simulation Conference (pp. 413–416). Piscataway, New Jersey: IEEE.
Nelson, B. L., & Taaffe, M. R. (2004). The Ph t ∕ Ph t ∕ ∞ queueing system: Part I: The single node. INFORMS Journal on Computing, 16, 266–274.
Nelson, B. L., Swann, J., Goldsman, D., & Song, W. (2001). Simple procedures for selecting the best simulated system when the number of alternatives is large. Operations Research, 49, 950–963.
Pasupathy, R., & Schmeiser, B. (2010). The initial transient in steady-state point estimation: Contexts, a bibliography, the MSE criterion, and the MSER statistic. Proceedings of the 2010 Winter Simulation Conference (pp. 184–197). Piscataway, New Jersey: IEEE.
Sargent, R. G. (2011). Verification and validation of simulation models. Proceedings of the 2011 Winter Simulation Conference (pp. 183–198). Piscataway, New Jersey: IEEE.
Schmeiser, B. (1982). Batch size effects in the analysis of simulation output. Operations Research, 30, 556–568.
Schruben, L. (1982). Detecting initialization bias in simulation output. Operations Research, 30, 569–590.
Schruben, L., Singh, H., & Tierney, L. (1983). Optimal tests for initialization bias in simulation output. Operations Research, 31, 1167–1178.
Shapiro, A., Dentcheva, D., & Ruszczyński, A. (2009). Lectures on stochastic programming: Modeling and theory. Philadelphia: Society for Industrial and Applied Mathematics.
Shechter, S. M., Schaefer, A. J., Braithwaite, R. S., & Roberts, M. S. (2006). Increasing the efficiency of Monte Carlo cohort simulations with variance reduction techniques. Medical Decision Making, 26, 550–553.
Snell, M., & Schruben, L. (1985). Weighting simulation data to reduce initialization effects. IIE Transactions, 17, 354–363.
Steiger, N. M., & Wilson, J. R. (2001). Convergence properties of the batch means method for simulation output analysis. INFORMS Journal on Computing, 13, 277–293.
Stigler, S. M. (1986). The history of statistics: The measurement of uncertainty before 1900. Cambridge, MA: Belknap.
Swain, J. J., Venkatraman, S., & Wilson, J. R. (1988). Least-squares estimation of distribution functions in Johnson’s translation system. Journal of Statistical Computation and Simulation, 29, 271–297.
Tafazzoli, A., & Wilson, J. R. (2011). Skart: A skewness-and-autoregression-adjusted batch-means procedure for simulation analysis. IIE Transactions, 43, 110-128.
Walkenbach, J. (2010). Excel 2010 power programming with VBA. New York: Wiley.
White, K. P. (1997). An effective truncation heuristic for bias reduction in simulation output. Simulation, 69, 323–334.
Whitt, W. (1981). Approximating a point process by a renewal process: The view through a queue, an indirect approach. Management Science, 27, 619–636.
Whitt, W. (1989). Planning queueing simulations. Management Science, 35, 1341–1366.
Whitt, W. (2006). Analysis for design. In S. G. Henderson & B. L. Nelson (Eds.), Handbooks in operations research and management science: Simulation. New York: North-Holland.
Whitt, W. (2007). What you should know about queueing models to set staffing requirements in service systems. Naval Research Logistics, 54, 476–484.
Xu, J., Hong, L. J., & Nelson, B. L. (2010). Industrial strength COMPASS: A comprehensive algorithm and software for optimization via simulation. ACM Transactions on Modeling and Computer Simulation, 20, 1–29.
Xu, J., Nelson, B. L., & Hong, L. J. (2012, in press). An adaptive hyperbox algorithm for high-dimensional discrete optimization via simulation problems. INFORMS Journal on Computing.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer Science+Business Media New York
About this chapter
Cite this chapter
Nelson, B.L. (2013). Simulation Output. In: Foundations and Methods of Stochastic Simulation. International Series in Operations Research & Management Science, vol 187. Springer, Boston, MA. https://doi.org/10.1007/978-1-4614-6160-9_7
Download citation
DOI: https://doi.org/10.1007/978-1-4614-6160-9_7
Published:
Publisher Name: Springer, Boston, MA
Print ISBN: 978-1-4614-6159-3
Online ISBN: 978-1-4614-6160-9
eBook Packages: Business and EconomicsBusiness and Management (R0)