Skip to main content

Data-driven tuning for chance constrained optimization: analysis and extensions

Abstract

Many optimization problems involve uncertain parameters which, if not appropriately accounted for, can cause solution infeasiblity. In this work, we consider joint chance-constrained optimization problems, which require all constraints to hold with a given probability, and a two-step solution method based on iterative tuning. Previous work established an a priori feasibility guarantee for this tuning approach, which relies on an assumption that must be verified on a case-by-case basis. In this paper, we propose an empirical methodology using statistical hypothesis testing to assess the validity of this assumption, thus providing further insight into the validity of the a priori guarantee. In addition, we provide sample complexity results to assess the requisite amount of data for the tuning method. We find that for large scale optimization problems, the tuning approach may require significantly less samples than the scenario approach. We numerically assess these results via application to the optimal power flow problem as well as further assess the scalability of the method and the optimality and feasibility of solutions obtained from tuning.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

References

  • ARPA-E Grid Optimization Competition Datasets. https://gocompetition.energy.gov/datasets/c2finalevent

  • Ahmed S, Luedtke J, Song Y, Xie W (2017) Nonanticipative duality, relaxations, and formulations for chance-constrained stochastic programs. Math. Programm. 162(1):51–81

    Article  Google Scholar 

  • Alamo T, Tempo R, Luque A (2010) On the sample complexity of randomized approaches to the analysis and design under uncertainty. In: Proceedings of the 2010 American Control Conference, pp. 4671–4676 . https://doi.org/10.1109/ACC.2010.5531078

  • Baker K, Toomey B (2017) Efficient relaxations for joint chance constrained AC optimal power flow. Electr. Power Syst. Res. 148:230–236. https://doi.org/10.1016/j.epsr.2017.04.001

    Article  Google Scholar 

  • Ben-Tal A, El Ghaoui L, Nemirovski A (2009) Robust Optimization. Princeton University Press, Princeton, NJ

    Book  Google Scholar 

  • Bertsimas D, Gupta V, Kallus N (2018) Data-driven robust optimization. Math. Programm. 167(2):235–292. https://doi.org/10.1007/s10107-017-1125-8

    Article  Google Scholar 

  • Bienstock D, Chertkov M, Harnett S (2014) Chance-constrained optimal power flow: Risk-aware network control under uncertainty. SIAM Rev. 56(3):461–495

    Article  Google Scholar 

  • Boole G (1854) An Investigation of the Laws of Thought: on Which Are Founded the Mathematical Theories of Logic and Probabilities. Dover, New York, NY

    Google Scholar 

  • Campi M, Garatti S (2011) A sampling-and-discarding approach to chance-constrained optimization: feasibility and optimality. J. Opt. Theory Appl. 148:257–280. https://doi.org/10.1007/s10957-010-9754-6

    Article  Google Scholar 

  • Campi MC, Garatti S, Prandini M (2009) The scenario approach for systems and control design. Ann. Rev. Control 33(2):149–157

    Article  Google Scholar 

  • Calafiore GC, El Ghaoui L (2006) On distributionally robust chance-constrained linear programs. J. Opt. Theory Appl. 130(1):1–22

    Article  Google Scholar 

  • Calafiore GC, Campi MC (2006) The scenario approach to robust control design. IEEE Trans. Automatic Control 51(5):742–753

    Article  Google Scholar 

  • Cantelli FP (1910) Intorno ad un teorema fondamentale della teoria del rischio

  • Chen W, Sim M, Sun J, Teo C-P (2010) From cvar to uncertainty set: implications in joint chance-constrained optimization. Oper. Res. 58(2):470–485

    Article  Google Scholar 

  • Coffrin C, Bent R, Sundar K, Ng Y, Lubin M (2018) Powermodels.jl: An open-source framework for exploring power flow formulations. In: 2018 Power Systems Computation Conference (PSCC), pp. 1–8 . https://doi.org/10.23919/PSCC.2018.8442948

  • Diniz AL, Henrion R (2017) On probabilistic constraints with multivariate truncated gaussian and lognormal distributions. Energy Syst. 8(1):149–167

    Article  Google Scholar 

  • Dunning I, Huchette J, Lubin M (2017) Jump: a modeling language for mathematical optimization. SIAM Rev. 59(2):295–320. https://doi.org/10.1137/15M1020575

    Article  Google Scholar 

  • Geng X, Xie L (2019) Data-driven decision making in power systems with probabilistic guarantees: theory and applications of chance-constrained optimization. Ann. Rev. Control 47:341–363. https://doi.org/10.1016/j.arcontrol.2019.05.005

    Article  Google Scholar 

  • Gurobi Optimization, LLC: Gurobi Optimizer Reference Manual (2021). https://www.gurobi.com

  • IEEE 118-bus, 54-unit, 24-hour system. https://icseg.iti.illinois.edu/ieee-118-bus-system/

  • Hou AM, Roald LA (2022) Data-driven tuning for chance-constrained optimization: two steps towards probabilistic performance guarantees. IEEE Control Syst. Lett. 6:1400–1405. https://doi.org/10.1109/LCSYS.2021.3096826

    Article  Google Scholar 

  • Hou AM, Roald LA (2020) Chance constraint tuning for optimal power flow. In: 2020 International Conference on Probabilistic Methods Applied to Power Systems (PMAPS), pp. 1–6 . https://doi.org/10.1109/PMAPS47429.2020.9183552

  • Hoeffding W (1963) Probability inequalities for sums of bounded random variables. J. Am. Stat. Assoc. 58(301):13–30. https://doi.org/10.1080/01621459.1963.10500830

    Article  Google Scholar 

  • Jabr RA (2013) Adjustable robust opf with renewable energy sources. IEEE Trans. Power Syst. 28(4):4742–4751. https://doi.org/10.1109/TPWRS.2013.2275013

    Article  Google Scholar 

  • Jia M, Hug G, Shen C (2020) Iterative Decomposition of Joint Chance Constraints in OPF

  • Jiang N, Xie W (2022) Also-x and also-x+: Better convex approximations for chance constrained programs. Oper. Res.

  • Lam H, Qian H (2019) Combating conservativeness in data-driven optimization under uncertainty: A solution path approach. arXiv preprint arXiv:1909.06477

  • Li B, Vrakopoulou M, Mathieu JL (2019) Chance constrained reserve scheduling using uncertain controllable loads part ii: analytical reformulation. IEEE Trans. Smart Grid 10(2):1618–1625. https://doi.org/10.1109/TSG.2017.2773603

    Article  Google Scholar 

  • Lorca A, Sun XA (2015) Adaptive robust optimization with dynamic uncertainty sets for multi-period economic dispatch under significant wind. IEEE Trans. Power Syst. 30(4):1702–1713. https://doi.org/10.1109/TPWRS.2014.2357714

    Article  Google Scholar 

  • Lubin M, Dvorkin Y, Backhaus S (2016) A robust approach to chance constrained optimal power flow with renewable generation. IEEE Trans. Power Syst. 31(5):3840–3849. https://doi.org/10.1109/TPWRS.2015.2499753

    Article  Google Scholar 

  • Luedtke J, Ahmed S, Nemhauser GL (2010) An integer programming approach for linear programs with probabilistic constraints. Math. Programm. 122(2):247–272

    Article  Google Scholar 

  • Luedtke J, Ahmed S (2008) A sample approximation approach for optimization with probabilistic constraints. SIAM J. Opt. 19(2):674–699

    Article  Google Scholar 

  • Margellos K, Goulart P, Lygeros J (2014) On the road between robust optimization and the scenario approach for chance constrained optimization problems. IEEE Trans. Auto. Control 59(8):2258–2263

    Article  Google Scholar 

  • Mezghani I, Misra S, Deka D (2020) Stochastic ac optimal power flow: a data-driven approach. Electr. Power Syst. Res. 189:106567

    Article  Google Scholar 

  • Nemirovski A, Shapiro A (2007) Convex approximations of chance constrained programs. SIAM J. Opt. 17(4):969–996

    Article  Google Scholar 

  • Oldewurtel F, Roald L, Andersson G, Tomlin C (2015) Adaptively constrained stochastic model predictive control applied to security constrained optimal power flow. In: American Control Conference, pp. 931–936

  • Pena-Ordieres A, Molzahn DK, Roald L, Waechter A (2020) DC optimal power flow with joint chance constraints. IEEE Transactions on Power Systems

  • Peña-Ordieres A, Luedtke JR, Wächter A (2019) Solving chance-constrained problems via a smooth sample-based nonlinear approximation. arXiv preprint arXiv:1905.07377

  • Roald LA, Molzahn DK, Tobler AF (2017) Power system optimization with uncertainty and ac power flow: Analysis of an iterative algorithm. In: 10th IREP Symp. Bulk Power Syst. Dynamics Control

  • Roald L, Andersson G (2017) Chance-constrained AC optimal power flow: reformulations and efficient algorithms. IEEE Trans. Power Syst. 33(3):2906–2918

    Article  Google Scholar 

  • Roald L, Misra S, Krause T, Andersson G (2017) Corrective control to handle forecast uncertainty: a chance constrained optimal power flow. IEEE Trans. Power Syst. 32(2):1626–1637. https://doi.org/10.1109/TPWRS.2016.2602805

    Article  Google Scholar 

  • Roald L, Oldewurtel F, Van Parys B, Andersson G (2015) Security constrained optimal power flow with distributionally robust chance constraints. arXiv preprint arXiv:1508.06061

  • Roald L, Oldewurtel F, Krause T, Andersson G (2013) Analytical reformulation of security constrained optimal power flow with probabilistic constraints. In: 2013 IEEE Grenoble Conference, pp. 1–6 . IEEE

  • Summers T, Warrington J, Morari M, Lygeros J (2015) Stochastic optimal power flow based on conditional value at risk and distributional robustness. Int. J. Electr. Power Energy Syst. 72:116–125

    Article  Google Scholar 

  • Vrakopoulou M, Li B, Mathieu JL (2019) Chance constrained reserve scheduling using uncertain controllable loads part i: formulation and scenario-based analysis. IEEE Trans. Smart Grid 10(2):1608–1617. https://doi.org/10.1109/TSG.2017.2773627

    Article  Google Scholar 

  • Vrakopoulou M, Margellos K, Lygeros J, Andersson G (2013) In: Billinton, R., Karki, R., Verma, A.K. (eds.) Probabilistic Guarantees for the N-1 Security of Systems with Wind Power Generation, pp. 59–73. Springer, India . https://doi.org/10.1007/978-81-322-0987-4_5

  • Wang W, Ahmed S (2008) Sample average approximation of expected value constrained stochastic programs. Oper. Res. Lett. 36(5):515–519

    Article  Google Scholar 

  • Wald A (1945) Sequential tests of statistical hypotheses. Ann. Math. Stat. 16(2):117–186

    Article  Google Scholar 

  • Xie W, Ahmed S (2018) On deterministic reformulations of distributionally robust joint chance constrained optimization problems. SIAM J. Opt. 28(2):1151–1182

    Article  Google Scholar 

  • Zhang Y, Shen S, Mathieu JL (2015) Data-driven optimization approaches for optimal power flow with uncertain reserves from load control. In: 2015 American Control Conference (ACC), pp. 3013–3018 . IEEE

  • Zimmerman RD, Murillo-Sánchez CE, Thomas RJ (2011) Matpower: Steady-state operations, planning, and analysis tools for power systems research and education. IEEE Trans. Power Syst. 26(1):12–19. https://doi.org/10.1109/TPWRS.2010.2051168

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ashley M. Hou.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

A Case study comparison formulations

A Case study comparison formulations

A.1 Scenario approach

The scenario approach (Campi et al. 2009; Calafiore and Campi 2006) is a common approach used to solve chance-constrained problems. While it requires no assumptions on the underlying distribution of the uncertainty \(\xi\), it relies on the availability of uncertainty samples or scenarios \(\Xi _\text {SA} = \{ \xi ^{(1)}, \dots , \xi ^{(N_\text {SA})} \}\) to transform probabilistic constraints into deterministic constraints. The JCCP is reformulated by enforcing that the chance constraint must hold for all uncertainty samples, that is

$$\begin{aligned}&\min _{x} \sum _{i \in {\mathcal {N}}} c_{2, i} x_i^2 + c_{1,i} x_i + c_{0,i} \end{aligned}$$
(34a)
$$\begin{aligned}&\text {s.t.} \sum _{i \in {\mathcal {N}}} x_i - d_i = 0, \end{aligned}$$
(34b)
$$\begin{aligned}&{\Omega} _\xi ^{(k)} := \sum _{i \in {\mathcal {N}}} \xi _i^{(k)}, ~\forall k\in [N_\text {SA}] \end{aligned}$$
(34c)
$$\begin{aligned}&x_i - \alpha _i {\Omega} _\xi ^{(k)} \le p_{G,i}^{\max }, ~~\forall i \in {\mathcal {N}},~k\in \Xi _\text {SA} \end{aligned}$$
(34d)
$$\begin{aligned}&x_i - \alpha _i {\Omega} _\xi ^{(k)} \ge p_{G,i}^{\min }, ~~\forall i \in {\mathcal {N}},~k\in \Xi _\text {SA} \end{aligned}$$
(34e)
$$\begin{aligned}&{M}_{(ij,\cdot )}(x - \alpha {\Omega} _\xi ^{(k)} + \xi ^{(k)} - d) \le p_{ij}^{\max }, ~\forall ij \in {\mathcal {L}},~k\in \Xi _\text {SA} \end{aligned}$$
(34f)
$$\begin{aligned}&{M}_{(ij,\cdot )}(x - \alpha {\Omega} _\xi ^{(k)} + \xi ^{(k)} - d) \ge p_{ij}^{\min }, \forall ij \in {\mathcal {L}},~k\in \Xi _\text{SA}. \end{aligned}$$
(34g)

Here, the objective function and nodal balance constraint remain the same as in the deterministic problem. Constraint (34c) defines \({\Omega} _{\xi }^{(k)}\) as the total uncertainty summed across all nodes \({\mathcal {N}}\) for sample draw \(k \in [N_{\text {SA}}]\). Constraints (34d)–(34g) enforce the generator and line constraints for each uncertainty realization.

A.2 CVaR inner approximation

The CVaR reformulation (Nemirovski and Shapiro 2007) is a conservative (inner) convex approximation of the JCCP. To solve the CVaR reformulation, we use the sample average approximation approach (Wang and Ahmed 2008), using an available sample set \(\Xi _{\text {CVaR}} = \{ \xi ^{(1)}, \dots , \xi ^{(N)} \}\) to solve the following deterministic approximate problem:

$$\begin{aligned}&\min _{\begin{array}{c} x, \beta \le 0, s \end{array}} \sum _{i \in {\mathcal {N}}} c_{2,i} x_i^2 + c_{1,i} x_i + c_{0,i} \end{aligned}$$
(35a)
$$\begin{aligned}&\text {s.t.} \sum _{i \in {\mathcal {N}}} x_i - d_i = 0 \end{aligned}$$
(35b)
$$\begin{aligned}&{\Omega} _\xi ^{(k)} = \sum _{i \in {\mathcal {N}}} \xi _i^{(k)}, \forall k \in [N] \end{aligned}$$
(35c)
$$\begin{aligned}&x_i - \alpha _i {\Omega} _\xi ^{(k)} - p_{G,i}^{\max } \le s_k, \forall i \in {\mathcal {N}}, \forall k \in [N] \end{aligned}$$
(35d)
$$\begin{aligned}&p_{G,i}^{\min } - x_i + \alpha _i {\Omega} _\xi ^{(k)} \le s_k, \forall i \in {\mathcal {N}}, \forall k \in [N] \end{aligned}$$
(35e)
$$\begin{aligned}&{M}_{(ij,\cdot )}(x \!-\!\alpha {\Omega} _\xi ^{(k)} \!+\! \xi ^{(k)} \!-\! d) \!-\! p_{ij}^{\max } \le s_k, \forall ij \in {\mathcal {L}}, \forall k \in [N] \end{aligned}$$
(35f)
$$\begin{aligned}&p_{ij}^{\min } - {M}_{(ij,\cdot )}(x \!-\! \alpha {\Omega} _\xi ^{(k)} \!+\! \xi ^{(k)} \!-\! d) \le s_k, \forall ij \in {\mathcal {L}}, \forall k \in [N] \end{aligned}$$
(35g)
$$\begin{aligned}&\frac{1}{N} \sum _{k = 1}^{N} s_k - (1 - \epsilon ) \beta \le 0, \end{aligned}$$
(35h)
$$\begin{aligned}&s_k \ge \beta,~\forall k \in [N]. \end{aligned}$$
(35i)

The objective function, nodal balance constraint, and definition of \({\Omega} _{k}^{(k)}\) remain the same as previously for the scenario approach. Here, we introduce non-negative auxiliary variable \(s_k\) for each uncertainty realization \(k = 1, \dots , N\) to denote uncertain constraint violations in the generator and line flow constraints (35d)–(35g). Auxiliary variable \(\beta \le 0\) and constraints (35h) and (35i) result directly from the definition of a CVaR constraint (Nemirovski and Shapiro 2007).

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hou, A.M., Roald, L.A. Data-driven tuning for chance constrained optimization: analysis and extensions. TOP (2022). https://doi.org/10.1007/s11750-022-00639-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11750-022-00639-z

Keywords

  • Chance-constrained optimization
  • Optimal power flow

Mathematics Subject Classification

  • 90C15
  • 90C90