Simulation-Based Optimality Tests for Stochastic Programs

  • Güzin Bayraksan
  • David P. Morton
  • Amit Partani
Part of the International Series in Operations Research & Management Science book series (ISOR, volume 150)


Assessing whether a solution is optimal, or near-optimal, is fundamental in optimization. We describe a simple simulation-based procedure for assessing the quality of a candidate solution to a stochastic program. The procedure is easy to implement, widely applicable, and yields point and interval estimators on the candidate solutions optimality gap. Our simplest procedure allows for significant computational improvements. The improvements we detail aim to reduce computational effort through single- and two-replication procedures, reduce bias via a class of generalized jackknife estimators, and reduce variance by using a randomized quasi-Monte Carlo scheme.


Monte Carlo Candidate Solution Stochastic Program Latin Hypercube Sampling Interval Estimator 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



The authors thank Georg Pflug for valuable discussions, particularly with respect to Example 3.2. This research was supported by the National Science Foundation under grants CMMI-0653916 and EFRI-0835930.


  1. Ahmed, S., Shapiro, A.: The sample average approximation method for stochastic programs with integer recourse. Optim. Online,, 2002, accessed September 26, (2002)
  2. Attouch, H., Wets, R.J.-B.: Approximation and convergence in nonlinear optimization. In: Mangasarian, O., Meyer, R., Robinson, S., (eds.) Nonlinear Programming 4, pp. 367–394. Academic, New York, NY (1981)Google Scholar
  3. Bailey, T.G., Jensen, P., Morton, D.P.: Response surface analysis of two-stage stochastic linear programming with recourse. Naval Res. Logistics, 46, 753–778 (1999)MathSciNetCrossRefMATHGoogle Scholar
  4. Bayraksan, G., Morton, D.P.: Assessing solution quality in stochastic programs. Math. Program. 108, pp. 495–514 (2006)MathSciNetCrossRefMATHGoogle Scholar
  5. Bayraksan, G., Morton, D.P.: A sequential sampling procedure for stochastic programming Oper. Res. (2010)Google Scholar
  6. Bayraksan, G., Morton, D.P.: Assessing solution quality in stochastic programs via sampling. In Oskoorouchi, M. R. (ed.) Tutorials in Operations Research, pp. 102–122. INFORMS, Hanover, MD (2009b)Google Scholar
  7. Beale, E.M.L.: On minimizing a convex function subject to linear inequalities. J. R. Stat. Soc. 17B, 173–184 (1955)MathSciNetGoogle Scholar
  8. Bertocchi, M., Dupačová, J., Moriggia, V.: Sensitivity of bond portfolio’s behavior with respect to random movements in yield curve: A simulation study. Ann. Oper. Res. 99, 267–286 (2000)MathSciNetCrossRefMATHGoogle Scholar
  9. Brooke, A., Kendrick, D., Meeraus, A., Raman, R.: GAMS, A User’s Guide, 2006. GAMS Development Corporation, Washington, DC,, accessed September 26 (2006)
  10. Caflisch, R.E., Morokoff, W.J., Owen, A.B.: Valuation of mortgage backed securities using Brownian bridges to reduce effective dimension. J. Comput. Finance 1, 27–46 (1997)Google Scholar
  11. Campi, M.C., Garatti, S.: The exact feasibility of randomized solutions of robust convex programs. Optim. Online., accessed September 26 (2007)
  12. Casella, G., Berger, R.L.: Statistical Inference. Duxbury Press, Belmont, CA (1990)Google Scholar
  13. Dantzig, G.B.: Linear programming under uncertainty. Manage. Sci. 1, 197–206 (1955)MathSciNetCrossRefMATHGoogle Scholar
  14. Dantzig, G.B., Glynn, P.W.: Parallel processors for planning under uncertainty. Ann. Oper. Res. 22, 1–21 (1990)MathSciNetCrossRefMATHGoogle Scholar
  15. Dantzig, G.B., Infanger, G.: A probabilistic lower bound for two-stage stochastic programs, Department of Operations Research, Stanford University, Stanford, CA, November (1995)CrossRefGoogle Scholar
  16. Diwekar, U.M., Kalagnanam, J.R.: An efficient sampling technique for optimization under uncertainty. Am. Inst. Chem. Eng. J. 43, 440 (1997)CrossRefGoogle Scholar
  17. Dupačová, J.: On non-normal asymptotic behavior of optimal solutions for stochastic programming problems and on related problems of mathematical statistics. Kybernetika. 27, 38–52 (1991)MathSciNetMATHGoogle Scholar
  18. Dupačová, J., Wets, R.J.-B.: Asymptotic behavior of statistical estimators and of optimal solutions of stochastic optimization problems. Ann. Stat. 16, 1517–1549 (1988)CrossRefMATHGoogle Scholar
  19. Fox, B.L.: Strategies for Quasi-Monte Carlo. Kluwer, Boston, MA (1999)CrossRefGoogle Scholar
  20. Freimer, M., Thomas, D., Linderoth, J.T.: Reducing bias in stochastic linear programming with sampling methods, Technical Report 05T-002, Industrial and Systems Engineering, Lehigh University (2005)Google Scholar
  21. Gray, H.L., Schucany, W.R.: The Generalized Jackknife Statistic. Marcel Dekker, New York, NY (1972)Google Scholar
  22. Higle, J.L.: Variance reduction and objective function evaluation in stochastic linear programs. INFORMS J. Comput. 10, 236–247 (1998)MathSciNetCrossRefMATHGoogle Scholar
  23. Higle, J.L., Sen, S.: Statistical verification of optimality conditions for stochastic programs with recourse. Ann. Oper. Res. 30, 215–240 (1991a)MathSciNetCrossRefMATHGoogle Scholar
  24. Higle, J.L., Sen, S.: Stochastic decomposition: An algorithm for two-stage linear programs with recourse. Math. Oper. Res. 16, 650–669 (1991b)MathSciNetCrossRefMATHGoogle Scholar
  25. Higle, J.L., Sen, S.: Stochastic Decomposition: A Statistical Method for Large Scale Stochastic Linear Programming. Kluwer, Dordrecht (1996a)Google Scholar
  26. Higle, J.L., Sen, S.: Duality and statistical tests of optimality for two stage stochastic programs. Math. Program. 75, 257–275 (1996b)MathSciNetMATHGoogle Scholar
  27. Higle, J.L., Sen, S.: Statistical approximations for stochastic linear programming problems. Ann. Oper. Res. 85, 173–192 (1999)MathSciNetCrossRefMATHGoogle Scholar
  28. Infanger, G.: Monte Carlo (importance) sampling within a Benders decomposition algorithm for stochastic linear programs. Ann. Oper. Res. 39, 69–95 (1992)MathSciNetCrossRefMATHGoogle Scholar
  29. Infanger, G.: Planning Under Uncertainty: Solving Large-Scale Stochastic Linear Programs. The Scientific Press Series, Boyd & Fraser, Danvers, MA (1994)Google Scholar
  30. Kall, P.: On approximations and stability in stochastic programming. In: Guddat, J., Jongen, H.Th., Kummer, B., Nožička, F. (eds.) Parametric Optimization and Related Topics, pp. 387–407. Akademie-Verlag, Berlin (1987)Google Scholar
  31. Kenyon, A.S., Morton, D.P.: Stochastic vehicle routing with random travel times. Transport. Sci. 37, 69–82 (2003)CrossRefGoogle Scholar
  32. King, A.J., Rockafellar, R.T.: Asymptotic theory for solutions in statistical estimation and stochastic programming. Math. Oper. Res. 18, 148–162 (1993)MathSciNetCrossRefMATHGoogle Scholar
  33. King, A.J., Wets, R.J.-B.: Epi-consistency of convex stochastic programs. Stochastics and Stochastics Reports 34, 83–92 (1991)MathSciNetMATHGoogle Scholar
  34. Kleywegt, A.J., Shapiro, A., Homem-de-Mello, T.: The sample average approximation method for stochastic discrete optimization. SIAM J. Optim. 12, 479–502 (2001)MathSciNetCrossRefMATHGoogle Scholar
  35. Korf, L.A., Wets, R.J.-B.: Random lsc functions: an ergodic theorem. Math. Oper. Res. 26, 421–445 (2001)MathSciNetCrossRefMATHGoogle Scholar
  36. Lan, G., Nemirovski, A., Shapiro, A.: Validation analysis of robust stochastic approximation method. Optim. Online. (2008)
  37. Law, A.M.: Simulation Modeling and Analysis, 4th edn. McGraw-Hill, Boston, MA (2007)Google Scholar
  38. L’Ecuyer, P., Lemieux, C.: Variance reduction via lattice rules. Manage. Sci. 46, 1214–1235 (2000)CrossRefMATHGoogle Scholar
  39. Linderoth, J.T., Shapiro, A., Wright, S.: The empirical behavior of sampling methods for stochastic programming. Ann. Oper. Res. 142, 215–241 (2006)MathSciNetCrossRefMATHGoogle Scholar
  40. Mak, W.K., Morton, D.P., Wood, R.K.: Monte Carlo bounding techniques for determining solution quality in stochastic programs. Oper. Res. Lett. 24, 47–56 (1999)MathSciNetCrossRefMATHGoogle Scholar
  41. Morton, D.P., Popova, E.: A Bayesian stochastic programming approach to an employee scheduling problem. IIE Trans. Oper. Eng. 36, 155–167 (2003)CrossRefGoogle Scholar
  42. Morton, D.P., Wood, R.K.: On a stochastic knapsack problem and generalizations. In: Woodruff, D.L. (ed.) Advances in Computational and Stochastic Optimization, Logic Programming, and Heuristic Search: Interfaces in Computer Science and Operations Research, pp. 149–168. Kluwer, Boston, MA (1998)CrossRefGoogle Scholar
  43. Morton, D.P., Popova, E., Popova, I.: Efficient fund of hedge funds construction under downside risk measures. J. Bank. Finance 30, 503–518 (2006)CrossRefGoogle Scholar
  44. Niederreiter, H.: Random Number Generation and Quasi-Monte Carlo Methods. CBMS-NSF Regional Conference Series in Applied Mathematics, SIAM (1992)Google Scholar
  45. Norkin, V.I., Pflug, G.Ch., Ruszczyński, A.: A branch and bound method for stochastic global optimization. Math. Program. 83, 425–450 (1998)MATHGoogle Scholar
  46. Partani, A.: Adaptive Jackknife Estimators for Stochastic Programming. PhD thesis, The University of Texas at Austin (2007)Google Scholar
  47. Partani, A., Morton, D. P., Popova, I.: Jackknife estimators for reducing bias in asset allocation. In: Proceedings of the Winter Simulation Conference Monterey, California (2006)Google Scholar
  48. Pennanen, T., Koivu, M.: Epi-convergent discretizations of stochastic programs via integration quadratures. Numerische Math. 100, 141–163 (2005)MathSciNetCrossRefMATHGoogle Scholar
  49. Quenouille, M.H.: Approximate tests of correlation in time-series. J. R. Stat. Soc. Series B 11, 68–84 (1949a)MathSciNetMATHGoogle Scholar
  50. Quenouille, M.H.: Problems in plane sampling. Ann. Math. Stat. 20, 355–375 (1949b)MathSciNetCrossRefMATHGoogle Scholar
  51. Robinson, S.M.: Analysis of sample-path optimization. Math. Oper. Res. 21, 513–528 (1996)MathSciNetCrossRefMATHGoogle Scholar
  52. Robinson, S.M., Wets, R.J.-B.: Stability in two-stage stochastic programming. SIAM J. Control Optim. 25, 1409–1416 (1987)MathSciNetCrossRefMATHGoogle Scholar
  53. Rubinstein, R.Y., Shapiro, A.: Discrete Event Systems: Sensitivity and Stochastic Optimization by the Score Function Method. Wiley, Chichester (1993)Google Scholar
  54. Santoso, T., Ahmed, S., Goetschalckx, M., Shapiro, A.: A stochastic programming approach for supply chain network design under uncertainty. Eur. J. Oper. Res. 167, 96–115 (2005)MathSciNetCrossRefMATHGoogle Scholar
  55. Shapiro, A.: Asymptotic properties of statistical estimators in stochastic programming. Ann. Stat. 17, 841–858 (1989)CrossRefMATHGoogle Scholar
  56. Shapiro, A.: Monte Carlo sampling methods. In: Ruszczyński, A., Shapiro, A. (eds.) Stochastic Programming, Handbooks in Operations Research and Management Science. Elsevier, Amsterdam (2003)Google Scholar
  57. Shapiro, A., Homem-de-Mello, T.: A simulation-based approach to two-stage stochastic programming with recourse. Math. Program. 81, 301–325 (1998)MathSciNetMATHGoogle Scholar
  58. Shapiro, A., Homem-de-Mello, T., Kim, J.: Conditioning of convex piecewise linear stochastic programs. Math. Program. 94, 1–19 (2002)MathSciNetCrossRefMATHGoogle Scholar
  59. Shapiro, A., Dentcheva, D., Ruszczyński, A.: Lectures on Stochastic Programming: Modeling and Theory. MPS-SIAM Series on Optimization, Philadelphia, PA (2009)CrossRefGoogle Scholar
  60. Sloan, I.H., Joe, S.: Lattice Methods for Multiple Integration. Clarendon Press, Oxford (1994)Google Scholar
  61. Verweij, B., Ahmed, S., Kleywegt, A., Nemhauser, G., Shapiro, A.: The sample average approximation method applied to stochastic vehicle routing problems: A computational study. Comput. Appl. Optim. 24, 289–333 (2003)MathSciNetCrossRefMATHGoogle Scholar
  62. Wallace, S.W., Ziemba, W.T. (eds.) Applications of Stochastic Programming. MPS-SIAM Series on Optimization, Philadelphia, PA (2005)Google Scholar
  63. Watkins, D.W., Jr., McKinney, D.C., Morton, D.P.: Groundwater pollution control. In: Wallace, S.W., Ziemba, W.T. (eds.) Applications of Stochastic Programming, pp. 409–424. MPS-SIAM Series on Optimization, Philadelphia, PA (2005)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  • Güzin Bayraksan
    • 1
  • David P. Morton
    • 2
  • Amit Partani
    • 2
  1. 1.Systems and Industrial EngineeringUniversity of ArizonaTucsonUSA
  2. 2.Graduate Program in Operations ResearchThe University of Texas at AustinAustinUSA

Personalised recommendations