Minimizing buffered probability of exceedance by progressive hedging

  • 40 Accesses


Stochastic programming problems have for a long time been posed in terms of minimizing the expected value of a random variable influenced by decision variables, but alternative objectives can also be considered, such as minimizing a measure of risk. Here something different is introduced: minimizing the buffered probability of exceedance for a specified loss threshold. The buffered version of the traditional concept of probability of exceedance has recently been developed with many attractive properties that are conducive to successful optimization, in contrast to the usual concept, which is often posed simply as the probability of failure. The main contribution here is to demonstrate that in minimizing buffered probability of exceedance the underlying convexities in a stochastic programming problem can be maintained and the progressive hedging algorithm can be employed to compute a solution.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Subscribe to journal

Immediate online access to all issues from 2019. Subscription will auto renew annually.

US$ 99

This is the net price. Taxes to be calculated in checkout.


  1. 1.

    This represention goes back to [27] and was extended from stochastic programming to stochastic variational inequalities in [29].

  2. 2.

    An extension of progressive hedging to iterate also on such multipliers with extra proximal terms is available in [22].

  3. 3.

    The second part of this has already been addressed; the existence part can be handled by compactness of the sets \(C(\xi )\) or more generality some joint aspects of these sets and growth properties of the functions \(f(\cdot ,\xi )\). For more on these issues, see [28, §4].

  4. 4.

  5. 5.

    Portfolio Safeguard (PSG),

  6. 6.


  1. 1.

    Davis, R.A., Uryasev, S.: Analysis of tropical storm damage using buffered probability of exceedance. Nat. Hazards (2016).

  2. 2.

    Kouri, D.P., Shapiro, A.: Optimization of PDEs with uncertain inputs. In: Antil, H., Kouri, D., Lacasse, M.D., Ridzal, D. (eds.) Frontiers in PDE-Constrained Optimization. The IMA Volumes in Mathematics and Its Applications, vol. 163. Springer, New York, NY (2018)

  3. 3.

    Kouri, D.P.: Higher-moment buffered probability. Optm. Lett. 13, 657–672 (2019)

  4. 4.

    Mafusalov, A., Uryasev, S.: Buffered probability of exceedance: mathematical properties and optimization. SIAM. J. Optim. 28, 1077–1103 (2018)

  5. 5.

    Mafusalov, A., Shapiro, A., Uryasev, S.: Estimation and asymptotics for buffered probability of exceedance. Eur. J. Oper. Res. 270, 826–836 (2018)

  6. 6.

    Norton, M., Uryasev, S.: Maximization of AUC and Buffered AUC in binary classification. Math. Program. (2018).

  7. 7.

    Norton, M., Mafusalov, A., Uryasev, S.: Soft margin support vector classification as buffered probability minimization. J. Mach. Learn. Res. 18, 1–43 (2017)

  8. 8.

    Norton, M., Mafusalov, A., Uryasev, S.: Cardinality of upper average and application to network optimization. SIAM J. Optim. 28(2), 1726–1750 (2018)

  9. 9.

    Norton, M., Khokhlov, V., Uryasev, S.: Calculating CVaR and bPOE for common probability distributions with application to portfolio optimization and density estimation. Ann. Oper. Res. arXiv:1811.11301 (submitted for publication)

  10. 10.

    Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton (1970)

  11. 11.

    Rockafellar, R.T.: Solving stochastic programming problems with risk measures by progessive hedging. Set-valued Variational Anal. 26, 759–768 (2017)

  12. 12.

    Rockafellar, R.T., Royset, J.O.: On buffered failure probability in design and optimization of structures. J. Reliab. Eng. Syst. Saf. 95, 499–510 (2010)

  13. 13.

    Rockafellar, R.T., Royset, J.O.: Superquantiles and their applications to risk, random variables, and regression. In: Tutorials in Operations Research, pp. 151–167. INFORMS (2013)

  14. 14.

    Rockafellar, R.T., Royset, J.O., Miranda, S.J.: Superquantile regression with applications to buffered reliability, uncertainty quantification and conditional value-at-risk. Eur. J. Oper. Res. 234, 140–154 (2014)

  15. 15.

    Rockafellar, R.T., Royset, J.O.: Engineering decisions under risk averseness. J. Risk Uncertain. Eng. Syst. Part A Civ. Eng. 1(2), 04015005 (2015).

  16. 16.

    Rockafellar, R.T., Royset, J.O.: Risk measures in engineering design under uncertainty. In: Proceedings of the 12th International Conference on Applications of Statistics and Probability in Civil Engineering (ICASP), Vancouver (2015).

  17. 17.

    Rockafellar, R.T., Royset, J.O., Harajli, M.M.: Importance sampling in the evaluation and optimization of buffered probability of failure. In: Proceedings of the 12th International Conference on Applications of Statistics and Probability in Civil Engineering (ICASP), Vancouver (2015).

  18. 18.

    Rockafellar, R.T., Royset, J.O.: Measures of residual risk with connections to regression, risk tracking, surrogate models and ambiguity. SIAM J. Optim. 25, 1179–1208 (2015)

  19. 19.

    Rockafellar, R.T., Royset, J.O.: Superquantile/CVaR risk measures: second-order theory. Ann. Oper. Res. 262, 3–29 (2018)

  20. 20.

    Rockafellar, R.T., Royset, J.O.: Random variables, monotone relations and convex analysis. Math. Program. B 128, 297–331 (2014)

  21. 21.

    Rockafellar, R.T., Sun, J.: Soving monotone stochastic variational inequalities and complementarity problems by progressive hedging. Math. Program. B 174, 453–471 (2018)

  22. 22.

    Rockafellar, R.T., Sun, J.: Solving Lagrangian variational inequalities with applications to stochastic programming. Math. Program. (2020).

  23. 23.

    Rockafellar, R.T., Uryasev, S.: Optimization of conditional value-at-risk. J. Risk 2, 21–43 (2000)

  24. 24.

    Rockafellar, R.T., Uryasev, S.: Conditional value-at-risk for general loss distributions. J. Bank. Finance 26, 1443–1471 (2002)

  25. 25.

    Rockafellar, R.T., Uryasev, S.: The fundamental risk quadrangle in risk management, optimization and statistical esimation. Surv. Oper. Res. Manag. Sci. 18, 33–53 (2013)

  26. 26.

    Rockafellar, R.T., Uryasev, S., Zabarankin, M.: Generalized deviations in risk analysis. Finance Stoch. 10, 51–74 (2006)

  27. 27.

    Rockafellar, R.T., Wets, R.J.-B.: Nonanticipativity and \({\cal{L}}^1\)-martingales in stochastic optimization problems. Stoch. Syst. Model. Identif. Optim. Math. Programm. Study 6, 170–187 (1976)

  28. 28.

    Rockafellar, R.T., Wets, R.J.-B.: Scenarios and policy aggregation in optimization under uncertainty. Math. Oper. Res. 16, 119–147 (1991)

  29. 29.

    Rockafellar, R.T., Wets, R.J.-B.: Stochastic variational inequalities: single-stage to multistage. Math. Program. B 165, 291–330 (2017)

  30. 30.

    Shang, D., Kuzmenko, V., Uryasev, S.: Cash flow matching with risks controlled by buffered probability of exceedance and conditional value-at-risk. Ann. Oper. Res. 260, 501–514 (2016)

  31. 31.

    Uryasev, S.: Buffered probability of exceedance and buffered service level: definitions and properties, Research Report 2014-3. ISE Department, University of Florida (2014)

  32. 32.

    Watson, J.-P., Woodruff, D.L.: Progressive hedging innovations for a class of stochastic mixed-integer resource allocation problems. Comput. Manag. Sci. 8, 355–370 (2010)

  33. 33.

    Zhitlukhin, M.: Monotone Sharpe ratios and related measures of investment performance (2018). arXiv:1809.10193

Download references


This research was sponsored for both authors by DARPA EQUiPS Grant SNL 014150709. The authors are grateful also to Dr. Viktor Kuzmeno for help with conducting the numerical case study.

Author information

Correspondence to R. Tyrrell Rockafellar.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Rockafellar, R.T., Uryasev, S. Minimizing buffered probability of exceedance by progressive hedging. Math. Program. (2020) doi:10.1007/s10107-019-01462-4

Download citation


  • Convex stochastic programming problems
  • Probability of failure
  • Probability of exceedance
  • Buffered probability of failure
  • Buffered probability of exceedance
  • Quantiles
  • Superquantiles
  • Conditional value-at-risk
  • Progressive hedging algorithm

Mathematics Subject Classification

  • 90C15