Discrete Event Dynamic Systems

, Volume 16, Issue 3, pp 385–403 | Cite as

Comparison with a Standard via All-Pairwise Comparisons

  • E. Jack Chen


We develop two sequential procedures to compare a finite number of designs with respect to a single standard. The goal is to identify a good design, and ensure that the standard is chosen when other alternatives are not better than the standard. We give preference to the standard since there are costs and time involved when replacing the standard. These procedures can be used when the expected performance of the standard is known or unknown, when variances across designs are unequal, and with the variance reduction technique of common random numbers. An experimental performance evaluation demonstrates the validity and efficiency of these sequential procedures.


Simulaton Comparison with a standard Sample-size allocation Indifference-zone selection 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Banks J, (1998). Handbook of Simulation. Wiley, New York.Google Scholar
  2. Bechhofer RE, Santner TJ, and Goldsman DM (1995). Design and Analysis of Experiments for Statistical Selection, Screening and Multiple Comparisons. Wiley, New York.Google Scholar
  3. Chen EJ (2004). Using ordinal optimization approach to improve efficiency of selection procedures. Journal of Discrete Event Dynamic Systems 14(2): 153–170.zbMATHCrossRefGoogle Scholar
  4. Chen EJ (2005). Using parallel and distributed computing to increase the capability of selection procedures. Proceedings of the 2005 Winter Simulation Conference. Institute of Electrical and Electronics Engineers, Piscataway, New Jersey, pp. 723–731.Google Scholar
  5. Chen EJ, Kelton WD (2003). Inferences from indifference-zone selection procedures. Proceedings of the 2003 Winter Simulation Conference, Institute of Electrical and Electronic Engineers, Piscataway, New Jersey, pp. 456–464.Google Scholar
  6. Chen EJ, Kelton WD (2005). Sequential selection procedures: Using sample means to improve efficiency. European Journal of Operational Research 166(1):133–153.zbMATHCrossRefGoogle Scholar
  7. Chen CH, Lin J, Yücesan E, Chick SE (2000). Simulation budget allocation for further enhancing the efficiency of ordinal optimization. Journal of Discrete Event Dynamic Systems 10(3):251–270.zbMATHCrossRefGoogle Scholar
  8. Chick SE, Inoue K (2001). New two-stage and sequential procedures for selecting the best system. Operations Research 49:732–743.CrossRefGoogle Scholar
  9. Edwards DG, Hsu JC (1983). Multiple comparisons with the best treatment. Journal of the American Statistical Association 78:965–971.zbMATHMathSciNetCrossRefGoogle Scholar
  10. Hastings C Jr (1955). Approximations for Digital Computers. Princeton University Press, Princeton, New Jersey.zbMATHGoogle Scholar
  11. Ho YC, Sreenivas RS, Vakili P (1992). Ordinal optimization of DEDS. Journal of Discrete Event Dynamic Systems 2:61–68.zbMATHCrossRefGoogle Scholar
  12. Kim S-H (2005). Comparison with a standard via fully sequential procedures. ACM Transactions on Modeling and Computer Simulation 15:155–174.CrossRefGoogle Scholar
  13. Kim S-H, Nelson BL (2001). A fully sequential procedure for indifference-zone selection in simulation. ACM Transactions on Modeling and Computer Simulation 11:251–273.CrossRefGoogle Scholar
  14. Law AM, Kelton WD (2000). Simulation Modeling and Analysis. 3rd edn. McGraw-Hill, New YorkzbMATHGoogle Scholar
  15. Nakayama MK (1997). Multiple-comparison procedures for steady-state simulations. Annals of Statistics 25:2433–2450.zbMATHMathSciNetCrossRefGoogle Scholar
  16. Nelson BL, Goldsman D (2001). Comparisons with a standard in simulation experiments. Management Science 47:449–463.CrossRefGoogle Scholar
  17. Nelson BL, Matejcik FJ (1995). Using common random numbers for indifference-zone selection and multiple comparisons in simulation. Management Science 41:1935–1945.zbMATHCrossRefGoogle Scholar
  18. Press WH, Flannery BP, Teukolsky SA, Vetterling WT (1992). Numerical Recipes in C: The Art of Scientific Computing. 2nd edn. Cambridge University Press, Cambridge, England.Google Scholar
  19. Rice JA (1995). Mathematical Statistics and Data Analysis. 2nd edn. Duxbury, Belmont, California.zbMATHGoogle Scholar
  20. Rinott Y (1978). On two-stage selection procedures and related probability inequalities. Communications in Statistics A7:799–811.zbMATHMathSciNetCrossRefGoogle Scholar
  21. Swisher JR, Jacobson SH, and Yücesan E (2003). Discrete-event simulation optimization using ranking, selection, and multiple comparison procedures: A survey. ACM Transactions on Modeling and Computer Simulation 13(2):134–154.CrossRefGoogle Scholar
  22. Tong YL (1980). Probability Inequalities in Multivariate Distributions. Academic, New York.zbMATHGoogle Scholar

Copyright information

© Springer Science + Business Media, LLC 2006

Authors and Affiliations

  1. 1.BASF CorporationRockawayUSA

Personalised recommendations