Abstract
Mehrotra and Özevin (SIAM J Optim 19:1846–1880, 2009) computationally found that a weighted barrier decomposition algorithm for two-stage stochastic conic programs achieves significantly superior performance when compared to standard barrier decomposition algorithms existing in the literature. Inspired by this motivation, Mehrotra and Özevin (SIAM J Optim 20:2474–2486, 2010) theoretically analyzed the iteration complexity for a decomposition algorithm based on the weighted logarithmic barrier function for two-stage stochastic linear optimization with discrete support. In this paper, we extend the aforementioned theoretical paper and its self-concordance analysis from the polyhedral case to the semidefinite case and analyze the iteration complexity for a weighted logarithmic barrier decomposition algorithm for two-stage stochastic convex quadratic SDP with discrete support.
Similar content being viewed by others
References
Alzalg, B.: Decomposition-based interior point methods for stochastic quadratic second-order cone programming. Appl. Math. Comput. 249, 1–18 (2014)
Alzalg, B.: Volumetric barrier decomposition algorithms for stochastic quadratic second-order cone programming. Appl. Math. Comput. 265, 494–508 (2015)
Alzalg, B.: Primal interior-point decomposition algorithms for two-stage stochastic extended second-order cone programming. Optimization 67, 2291–2323 (2018)
Alzalg, B.: Combinatorial and Algorithmic Mathematics: From Foundation to Optimization, 1st edn. Kindle Direct Publishing. Seatle, WA (2022)
Alzalg, B., Ariyawansa, K.A.: Logarithmic barrier decomposition-based interior point methods for stochastic symmetric programming. J. Math. Anal. Appl. 409, 973–995 (2014)
Ariyawansa, K.A., Zhu, Y.: A class of polynomial volumetric barrier decomposition algorithms for stochastic semidefinite programming. Math. Comp. 80, 1639–1661 (2011)
Birge, J.R., Louveaux, F.: Introduction to Stochastic Programming. Springer Series in Operations Research and Financial Engineering. Springer, New York (1997)
Chang, X., Liu, S., Li, X.: Modified alternating direction method of multipliers for convex quadratic semidefinite programming. Neurocomputing 214, 575–586 (2016)
Cheng, C.-H., Zhu, Y.: Stochastic semidefinite programming for emergency relief logistics. In: 12th INFORMS Computer Society Conference, Monterey, CA, USA, pp. 108–119 (2011)
Correa, R.: A global algorithm for nonlinear semidefinite programming. SIAM J. Optim. 15, 303–318 (2004)
Freund, R.W., Jarre, F., Vogelbusch, C.H.: Nonlinear semidefinite programming: sensitivity, convergence, and an application in passive reduced-order modeling. Math. Program. 109, 581–611 (2007)
Graham, A.: Kronecker Products and Matrix Calculus: With Applications. Ellis Horwood, Chichester (1981)
King, A.J., Rockafellar, R.T.: Asymptotic theory for solutions in statistical estimation and stochastic programming. Math. Oper. Res. 18, 148–162 (1993)
Mehrotra, S., Özevin, M.G.: Decomposition-based interior point methods for two-stage stochastic semidefinite programming. SIAM J. Optim. 18, 206–222 (2007)
Mehrotra, S., Özevin, M.G.: Decomposition-based interior point methods for two-stage stochastic convex quadratic programs with recourse. Oper. Res. 57, 964–974 (2009)
Mehrotra, S., Özevin, M.G.: On the implementation of interior point decomposition algorithms for two-stage stochastic conic programs. SIAM J. Optim. 19, 1846–1880 (2009)
Mehrotra, S., Özevin, M.G.: Convergence of a weighted barrier decomposition algorithm for two-stage stochastic programming with discrete support. SIAM J. Optim. 20, 2474–2486 (2010)
Nesterov, Y.E., Nemirovskii, A.S.: Interior Point Polynomial Algorithms in Convex Programming. SIAM Publications, Philadelphia, PA (1994)
Nie, J.W., Yuan, Y.X.: A predictor-corrector algorithm for QSDP combining Dikin type and Newton centering steps. Ann. Oper. Res. 103, 115–133 (2001)
Renegar, J.: A Mathematical View of Interior-Point Methods in Convex Optimization. SIAM Publications, Philadelphia, PA (2001)
Schultz, R., Wollenberg, T.: Unit commitment under uncertainty in AC transmission systems via risk averse semidefinite stochastic programs. RAIRO-Oper. Res. 51, 391–416 (2017)
Shapiro, A.: Asymptotic behavior of optimal solutions in stochastic programming. Math. Oper. Res. 18, 829–845 (1993)
Sun, W., Li, C., Sampaio, R.J.: On duality theory for non-convex semidefinite programming. Ann. Oper. Res. 186, 331–343 (2011)
Todd, M.J.: Scaling, shifting and weighting in interior-point methods. Comput. Optim. Appl. 3, 305–315 (1994)
Todd, M.J.: Semidefinite optimization. Acta Numer. 10, 515–560 (2001)
Toh, K.C., Tutuncu, R.H., Todd, M.J.: Inexact primal-dual path-following algorithms for a special class of convex quadratic SDP and related problems. Pac. J. Optim. 3, 135–164 (2007)
Toh, K.C.: An inexact primal-dual path-following algorithm for convex quadratic SDP. Math. Program. 112, 221–254 (2008)
Zhang, L., Li, Y., Wu, J.: Nonlinear rescaling Lagrangians for nonconvex semidefinite programming. Optimization 63, 899–920 (2014)
Zhao, G.: A log barrier method with Benders decomposition for solving two-stage stochastic linear programs. Math. Program. Ser. A 90, 507–536 (2001)
Zhu, Y., Ariyawansa, K.A.: A preliminary set of applications leading to stochastic semidefinite programs and chance-constrained semidefinite programs. Appl. Math. Model. 35, 2425–2442 (2011)
Zhu, Y., Zhang, J., Partel, K.: Location-aided routing with uncertainty in mobile ad hoc networks: A stochastic semidefinite programming approach. Math. Comp. Model. 53, 2192–2203 (2011)
Acknowledgements
This study was done during the year 2021 when the first author was on Sabbatical leave in Ohio, USA. The authors thank the two anonymous expert referees for their valuable suggestions from which the paper has benefited.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Alper Yildirim.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
A Complexity Proofs
A Complexity Proofs
In this appendix, we give proofs of Theorems 6.1 and 6.2. The following proposition, which is due to [18, Theorem 2.1.1], follows directly from the definition of self-concordance.
Proposition A.1
Let \(\mu > 0\), \(X \in \mathcal {P}\), and \(\delta \triangleq \sqrt{(1/\mu )\nabla ^2\eta (\mu ,X)[\Delta X,\Delta X]}\). Then, for \(\delta < 1\), \(\tau \in [0,1]\), and \(D \in \mathbb {S}^{n_1}\), we have
We use two different merit functions to measure the speed of Newton method to estimate the number of Newton steps required for recentering. For the short-step algorithm, we utilize \(\delta (\mu ,X)\), and for the long-step algorithm, we use the first-stage objective \(\eta (\mu ,X)\). The following lemma is derived from Theorem 2.2.3 in [18] and describes the behavior of the Newton method when applied to \(\eta (\mu ,\cdot )\).
Lemma A.1
For any \(\mu > 0\) and \(X \in \mathcal {P}_\circ \), let \(\Delta X\) be the Newton direction calculated by (30), and \(\delta \) be defined as \(\delta \triangleq \delta (\mu ,X)\triangleq \sqrt{1/(\mu \alpha \min _k \{\pi ^{(k)}\}) \nabla ^2\eta (\mu ,X)[\Delta X,\Delta X]}\). Then, the following statements hold:
-
(i)
If \(\delta < 2-\sqrt{3}\), then \(\delta (\mu ,X+\Delta X)\le (\delta /(1-\delta ))^2\le \delta /2\).
-
(ii)
If \(\delta \ge 2-\sqrt{3}\), then \(\eta (\mu ,X)-\eta (\mu ,X+\bar{\theta }\Delta X)\ge \mu \alpha \min _k\{ \pi ^{(k)}\} (\delta -\ln (1+\delta ))\), where \(\bar{\theta }=1/(1+\delta )\).
1.1 A.1 Complexity Proof of the Short-Step Algorithm
Proving Theorem 6.1 makes use of Lemma A.2.
Lemma A.2
Let \(\mu ^+\triangleq \gamma \mu \), where \(\gamma = 1 - \sigma \sqrt{\alpha \min _k \left\{ \pi ^{(k)}\right\} /(n_1+\alpha n_2)}\) and \(\sigma \le 0.1\). Furthermore, let \(\beta = (2 - \sqrt{3})/2\). If \(\delta (\mu ,X)\le \beta \), then \(\delta (\mu ^+,X)\le 2\beta \).
Proof
Let \(\kappa =2\beta = 2 - \sqrt{3}\). It is easy to verify that, for \(\sigma < 0.1\), \(\mu ^+\) satisfies
Applying Theorem 3.1.1 in [18], we get \(\delta (\mu ^+,X) < \kappa =2\beta \) as desired. \(\square \)
From Lemmas A.1 and A.2, we conclude that at each iteration we can reduce the barrier parameter \(\mu \) by the factor \( \gamma = 1 -\sigma \sqrt{\alpha \min _k\left\{ \pi ^{(k)} \right\} /(n_1+\alpha n_2)}\), with \(\sigma \le 0.1\), and that only one Newton step is sufficient to restore proximity to the central path. This proves Theorem 6.1.
1.2 A.2 Complexity Proof of the Long-Step Algorithm
We use \(\eta \) as the merit function for the analysis of the long-step algorithm because the iterations that are generated by the less conservative long-step algorithm may violate the condition \(\delta <2 - \sqrt{3}\), which is required in item (i) of Lemma A.1.
We assume that we have a point \(\mu ^{(j-1)}\) sufficiently close to \(X(\mu ^{(j-1)})\). Then, we reduce the barrier parameter from \(\mu ^{(j-1)}\) to \(\mu ^{(j)} = \gamma \mu ^{(j-1)}\), where \(\gamma \in (0,1)\). While searching for a point \(X^{(j)}\) that is sufficiently close to \(X(\mu ^{(j)})\), our algorithm will generate a finite sequence of points \(P_1,P_2,\ldots , P_N \in \mathcal {P}\) and we finally set \(X(\mu ^{(j)})= P_N\) . We need to determine an upper bound on N; the number of Newton iterations needed for recentering. We begin by determining an upper bound on the difference \(\phi (\mu ^{(j)},X^{(j-1)}) \triangleq \eta (\mu ^{(j)},X^{(j-1)}) -\eta (\mu ^{(j)},X(\mu ^{(j)}))\). Then, by item (ii) of Lemma A.1, we know that at \(P_k \in \mathcal {P}\), a Newton step with step size \(\bar{\theta } =( 1 + \delta )^{-1}\) decreases \( \eta (\mu ^{(j)},P_k)\) at least by a certain amount, which depends on the current value of \(\delta \) and \(\mu \). A line search might yield even a larger decrease. Consequently, we will get an upper bound on N.
Proving Theorem 6.2 makes use of the following propositions and lemmas. We give Proposition A.2 without proof because the proof is quite similar to that of [29, Lemma 6].
Proposition A.2
For any \(\mu > 0\) and \(X\in \mathcal {P}_\circ \), we denote \(\tilde{\Delta }X \triangleq X-X(\mu ) \) and define the parameter \(\tilde{\delta }(\mu ,X)\) as \(\tilde{\delta }(\mu ,X)\triangleq \sqrt{ 1/(\mu \alpha \min _k \{\pi ^{(k)}\})\nabla ^2\eta (\mu ,X)[\tilde{\Delta }X,\tilde{\Delta } X]}\). If \(\tilde{\delta }<1\), then \(\phi (\mu ,X)\le \mu ( \tilde{\delta }/(1-\tilde{\delta }) +\ln (1-\tilde{\delta }))\).
Lemma A.3
Let \(\tilde{\Delta } X\) and \(\tilde{\delta }\) be as defined in Proposition A.2. Let also \(\mu > 0\) and \(X\in \mathcal {P}_\circ \). If \(\tilde{\delta }<1\), then we have \(|\phi ^\prime (\mu ,X)|\le -\sqrt{n_1+\alpha n_2}\ln (1-\tilde{\delta })\).
Proof
For any \(\mu > 0\), using the chain rule, we can write \(\phi ^\prime (\mu ,X)=\eta ^\prime (\mu ,X)-\eta ^\prime (\mu ,X(\mu ))-\nabla \eta ^\prime (\mu ,X(\mu ))[ X^\prime (\mu )]\). According to the optimality conditions (29), we have \(\nabla \eta ^\prime (\mu ,X(\mu )) [ X^\prime (\mu )]= 0\). Consequently,
From (24), we get
From (32), applying the mean-value theorem, we get
Using (34), (33), and (31) and noting that \(X(\mu )+\tau \tilde{\Delta }X=X-(1-\tau )\tilde{\Delta }X\), we have
The proof is complete. \(\square \)
Lemma A.4
Let \(\mu > 0\) and \(X\in \mathcal {P}_\circ \) be such that \(\tilde{\delta }<1\), where \(\tilde{\delta }\) is as defined in Proposition A.2. Let also \(\mu ^+=\gamma \mu \) with \(\gamma \in (0,1)\). Then, \(\eta (\mu ^+,X) -\eta (\mu ^+,X(\mu ^+)) \le \mathcal {O}(n_1+\alpha n_2)\,\mu ^+\).
Proof
Differentiating (32), we get
We take the derivative of the optimality conditions (29) of the first-stage problem (2) to get \(X^\prime (\mu )\):
where \(\Sigma =\nabla ^2\eta (\mu ,X(\mu ))\). Solving (36), we have \(X^\prime (\mu )=-(\Sigma ^{-1}- \Sigma ^{-1}\mathcal {A}^\textsf{T }( \mathcal {A}\Sigma ^{-1}\mathcal {A}^\textsf{T })\mathcal {A}\Sigma ^{-1}) [\{\nabla \eta (\mu ,X(\mu ))\}^\prime ]. \) Next, we have
where the last inequality is followed from (24).
Now, we bound the first two terms in the right-hand side of (35). From the definition of \(\eta (\mu ,X)\) (see (2)), it is clear that \(\eta ^{\prime \prime }(\mu ,X)=\sum _{k=1}^K\rho ^{(k)^{\prime \prime }}(\mu ,X)\).
Now, by differentiating (10) and using (9), we get
Therefore,
and
Thus, for any \( X\in \mathcal {P}_\circ \), we have \(\eta ^{\prime \prime }(\mu ,X)\ge \alpha n_2/\mu \). Since \(\eta (\mu ,X)\) is strictly concave in \(\mu \) for any \( X\in \mathcal {P}_\circ \), we also have \(\eta ^{\prime \prime }(\mu ,X)\le 0\). Now, using the above bounds and the one given in (37), from (35) we obtain \(\phi ^{\prime \prime }(\mu ,X)\le (n_1+2\alpha n_2)/\mu \). Then, by Proposition A.2 and Lemma A.3, we get
Since \(\tilde{\delta }\) and \(\gamma \) are arbitrary constants, (38) implies that \( \eta (\mu ^+,X)-\eta (\mu ^+,X(\mu ^+))\le \mathcal {O}(n_1+\alpha n_2)\mu ^+\) as desired. \(\square \)
Note that Proposition A.2 and Lemmas A.3 and A.2 all require \(\tilde{\delta }\) to be less than one. However, we cannot evaluate \(\tilde{\delta }\) because we do not explicitly know the points \(X(\mu )\) forming the central path. Nonetheless, we find that \(\delta \) is proportional to \(\tilde{\delta }\) as shown in the following proposition, which is due to [14, Lemma 5.5].
Proposition A.3
For any \(\mu > 0, X\in \mathcal {P}_\circ \), and \(\lambda \in \mathbb {R}^{m_1}\), let \((\Delta X,\Delta \lambda )\) be the Newton direction defined in (30) and \( (\tilde{\Delta }X,\tilde{\Delta } \lambda )\triangleq (X-X(\mu ),\lambda -\lambda (\mu ))\). We denote \(\delta (\mu ,X)=(\Sigma [\Delta X,\Delta X]/\mu )^{1/2}, \,\, \, \text {and} \,\, \, \tilde{\delta }(\mu ,X)=( \Sigma [\tilde{\Delta } X,\tilde{\Delta } X]/\mu )^{1/2}\), where \(\Sigma =\nabla ^2\eta (\mu ,X)\). If \({\delta }\le 1/6\) then \(2\delta /3 \le \tilde{\delta }\le 2{\delta }\).
Lemma A.1 implies that each line search should decrease the value of \(\eta \) by at least \(\mu (\delta -\ln (1-\delta ))\). Therefore, in view of Lemmas A.1 and A.4, it is clear that after reducing \(\mu \) by a factor \(\gamma \in (0,1)\), at most \( \mathcal {O}((n_1+\alpha n_2)/ (\alpha \min _k\{\pi ^{(k)}\}))\) Newton iterations are needed for recentering. In the long-step version of the proposed algorithm, we need to update barrier parameter \(\mu \) no more than \( \mathcal {O}(\ln \mu ^{(0)}/\epsilon )\) times. Theorem 6.2 now follows from Lemmas A.1(ii) and A.4 and Proposition A.3.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Alzalg, B., Gafour, A. Convergence of a Weighted Barrier Algorithm for Stochastic Convex Quadratic Semidefinite Optimization. J Optim Theory Appl 196, 490–515 (2023). https://doi.org/10.1007/s10957-022-02128-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10957-022-02128-6
Keywords
- Quadratic semidefinite programming
- Two-stage stochastic programming
- Large-scale optimization
- Interior-point methods
- Decomposition