Skip to main content
SpringerLink
Log in

Validation analysis of mirror descent stochastic approximation method

  • Full Length Paper
  • Series A
  • Published:
Mathematical Programming Submit manuscript

Abstract

The main goal of this paper is to develop accuracy estimates for stochastic programming problems by employing stochastic approximation (SA) type algorithms. To this end we show that while running a Mirror Descent Stochastic Approximation procedure one can compute, with a small additional effort, lower and upper statistical bounds for the optimal objective value. We demonstrate that for a certain class of convex stochastic programs these bounds are comparable in quality with similar bounds computed by the sample average approximation method, while their computational cost is considerably smaller.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bregman L.M.: The relaxation method of finding the common points of convex sets and its application to the solution of problems in convex programming. Comput. Math. Math. Phys. 7, 200–217 (1967)

    Article  Google Scholar 

  2. Kleywegt A.J., Shapiro A., Homem-de-Mello T.: The sample average approximation method for stochastic discrete optimization. SIAM J. Optim. 12, 479–502 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  3. Law A.M.: Simulation Modeling and Analysis. McGraw Hill, New York (2007)

    Google Scholar 

  4. Lemarechal C., Nemirovski A., Nesterov Yu.: New variants of bundle methods. Math. Program. 69, 111–148 (1995)

    Article  MATH  Google Scholar 

  5. Linderoth J., Shapiro A., Wright S.: The empirical behavior of sampling methods for stochastic programming. Ann. Oper. Res. 142, 215–241 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  6. Mak W.K., Morton D.P., Wood R.K.: Monte Carlo bounding techniques for determining solution quality in stochastic programs. Oper. Res. Lett. 24, 47–56 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  7. Nemirovskii, A., Yudin, D.: On Cezari’s convergence of the steepest descent method for approximating saddle point of convex-concave functions. (in Russian)—Doklady Akademii Nauk SSSR, 239, 5 (1978) (English translation: Soviet Math. Dokl. 19, 2 (1978))

  8. Nemirovski A., Yudin D.: Problem complexity and method efficiency in optimization. Wiley-Interscience Series in Discrete Mathematics, Wiley, XV (1983)

  9. Nemirovski A., Juditsky A., Lan G., Shapiro A.: Robust stochastic approximation approach to stochastic programming. SIAM J. Optim. 19, 1574–1609 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  10. Norkin V.I., Pflug G.Ch., Ruszczyński A.: A branch and bound method for stochastic global optimization. Math. Program. 83, 425–450 (1998)

    MATH  Google Scholar 

  11. Polyak B.T.: New stochastic approximation type procedures. Automat. i Telemekh. 7, 98–107 (1990) (English translation: Automation and Remote Control)

    MathSciNet  Google Scholar 

  12. Polyak B.T., Juditsky A.B.: Acceleration of stochastic approximation by averaging. SIAM J. Control Optim. 30, 838–855 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  13. Robbins H., Monro S.: A stochastic spproximation method. Ann. Math. Stat. 22, 400–407 (1951)

    Article  MathSciNet  MATH  Google Scholar 

  14. Rockafellar R.T., Uryasev S.P.: Optimization of conditional value-at-risk. The Journal of Risk 2, 21–41 (2000)

    Google Scholar 

  15. Shapiro A.: Monte Carlo sampling methods. In: Ruszczyński, A., Shapiro, A. (eds) Stochastic Programming, Handbook in OR & MS, vol. 10, North-Holland Publishing Company, Amsterdam (2003)

    Google Scholar 

  16. Shapiro A., Nemirovski A.: On complexity of stochastic programming problems. In: Jeyakumar, V., Rubinov, A.M. (eds) Continuous Optimization: Current Trends and Applications, pp. 111–144. Springer, New York (2005)

    Google Scholar 

  17. Verweij B., Ahmed S., Kleywegt A.J., Nemhauser G., Shapiro A.: The sample average approximation method applied to stochastic routing problems: a computational study. Comput. Optim. Appl. 24, 289–333 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  18. Wang, W., Ahmed, S.: Sample average approximation of expected value constrained stochastic programs, E-print available at: http://www.optimization-online.org (2007)

Download references

Author information

Authors and Affiliations

  1. University of Florida, Gainesville, FL, 32611, USA

    Guanghui Lan

  2. Georgia Institute of Technology, Atlanta, GA, 30332, USA

    Arkadi Nemirovski & Alexander Shapiro

Authors
  1. Guanghui Lan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arkadi Nemirovski
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alexander Shapiro
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guanghui Lan.

Additional information

G. Lan research of this author was partly supported by the ONR Grant N000140811104 during his Ph.D. study. A. Nemirovski and A. Shapiro research of this author was partly supported by the NSF awards DMI-0619977 and DMS-0914785.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lan, G., Nemirovski, A. & Shapiro, A. Validation analysis of mirror descent stochastic approximation method. Math. Program. 134, 425–458 (2012). https://doi.org/10.1007/s10107-011-0442-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10107-011-0442-6

Keywords

Mathematics Subject Classification (2000)

Search

Navigation