Skip to main content

Solving nearly-separable quadratic optimization problems as nonsmooth equations

Abstract

An algorithm for solving nearly-separable quadratic optimization problems (QPs) is presented. The approach is based on applying a semismooth Newton method to solve the implicit complementarity problem arising as the first-order stationarity conditions of such a QP. An important feature of the approach is that, as in dual decomposition methods, separability of the dual function of the QP can be exploited in the search direction computation. Global convergence of the method is promoted by enforcing decrease in component(s) of a Fischer–Burmeister formulation of the complementarity conditions, either via a merit function or through a filter mechanism. The results of numerical experiments when solving convex and nonconvex instances are provided to illustrate the efficacy of the method.

This is a preview of subscription content, access via your institution.

References

  1. 1.

    Bai, L., Raghunathan, A. U.: Semismooth equation approach to network utility maximization (NUM). In American Control Conference (ACC), pp. 4795–4801, (2013)

  2. 2.

    Bertsekas, D.P.: Nonlinear Programming, 2nd edn. Athena Scientific, Belmont (1999)

    MATH  Google Scholar 

  3. 3.

    Bertsekas, D.P.: Convex Optimization Theory. Athena Scientific, Belmont (2009)

    MATH  Google Scholar 

  4. 4.

    Birge, J.R., Louveaux, F.V.: Introduction to Stochastic Programming. Springer, New York (1997)

    MATH  Google Scholar 

  5. 5.

    Bragalli, C., Ambrosio, C.D., Lee, J., Lodi, A., Toth, P.: On the optimal design of water distribution networks: a practical MINLP approach. Opt. Eng. 13(2), 219–246 (2012)

    MathSciNet  Article  MATH  Google Scholar 

  6. 6.

    Ralph, D., Dempe, S.: Directional derivatives of the solution of a parametric nonlinear program. Math. Prog. 70, 159–172 (1995)

    MathSciNet  MATH  Google Scholar 

  7. 7.

    Deng, W., Lai, M.-J., Peng, Z., Yin, W.: Parallel Multi-Block ADMM with \(o(1/k)\) Convergence. Technical report, arXiv:1312.3040 (2014)

  8. 8.

    Dinh, Q.T., Necoara, I., Diehl, M.: Path-following gradient-based decomposition algorithms for separable convex optimization. J. Global Optim. 59(1), 59–80 (2014)

    MathSciNet  Article  MATH  Google Scholar 

  9. 9.

    Dinh, Q.T., Necoara, I., Savorgnan, C., Diehl, M.: An inexact perturbed path-following method for lagrangian decomposition in large-scale separable convex optimization. SIAM J. Optim. 23(1), 95–125 (2013)

    MathSciNet  Article  MATH  Google Scholar 

  10. 10.

    Dinh, Q.T., Savorgnan, C., Diehl, M.: Combining lagrangian decomposition and excessive gap smoothing technique for solving large-scale separable convex optimization problems. Comput. Optim. Appl. 55(1), 75–111 (2013)

    MathSciNet  Article  MATH  Google Scholar 

  11. 11.

    Facchinei, F., Fischer, A., Kanzow, C.: Regularity properties of a semismooth reformulation of variational inequalities. SIAM J. Optim. 8, 850–869 (1998)

    MathSciNet  Article  MATH  Google Scholar 

  12. 12.

    Facchinei, F., Soares, J.: A new merit function for nonlinear complementarity problems and a related algorithm. SIAM J. Optim. 7, 225–247 (1997)

    MathSciNet  Article  MATH  Google Scholar 

  13. 13.

    Fiacco, A.V.: Sensitivity analysis for nonlinear programming using penalty methods. Math. Program. 10, 287–311 (1976)

    MathSciNet  Article  MATH  Google Scholar 

  14. 14.

    Fiacco, A.V., McCormick, G.P.: Nonlinear programming: sequential unconstrained minimization techniques. Class. Appl. Math. Soc. Ind. Appl. Math. (1987)

  15. 15.

    Fischer, A.: A special Newton-type optimization method. Optimization 24, 269–284 (1992)

    MathSciNet  Article  MATH  Google Scholar 

  16. 16.

    Fletcher, R., Gould, N.I.M., Leyffer, S., Toint, PhL, Wächter, A.: Global convergence of trust-region SQP-filter algorithms for nonlinear programming. SIAM J. Opt. 13, 635–659 (2002)

    MathSciNet  Article  MATH  Google Scholar 

  17. 17.

    Fletcher, R., Leyffer, S.: Nonlinear programming without a penalty function. Math. Program. 91, 239–269 (2002)

    MathSciNet  Article  MATH  Google Scholar 

  18. 18.

    Fletcher, R., Leyffer, S.: Filter-type algorithms for solving systems of algebraic equations and inequalities. In: High-Performance Algorithms and Software in Nonlinear Optimization, pp. 259–278. Kluwer, Dordrecht, The Netherlands (2003)

  19. 19.

    Fletcher, R., Leyffer, S., Toint, PhL: On the global convergence of a filter-SQP algorithm. SIAM J. Opt. 13, 44–59 (2002)

    MathSciNet  Article  MATH  Google Scholar 

  20. 20.

    Frasch, J.V., Sager, S., Diehl, M.: A parallel quadratic programming method for dynamic optimization problems. Math. Prog. Comput. 7(3), 289–329 (2015)

    MathSciNet  Article  MATH  Google Scholar 

  21. 21.

    Gondzio, J., Grothey, A.: Parallel interior-point solver for structured quadratic programs: Application to financial planning problems. Ann. Oper. Res. 152(1), 319–339 (2007)

    MathSciNet  Article  MATH  Google Scholar 

  22. 22.

    Gould, N.I.M., Leyffer, S., Toint, PhL: A multidimensional filter algorithm for nonlinear equations and nonlinear least squares. SIAM J. Opt. 15, 17–38 (2004)

    MathSciNet  Article  MATH  Google Scholar 

  23. 23.

    Jiang, H., Qi, L.: A new nonsmooth equations approach to nonlinear complementarity problems. SIAM J Control Opt. 35, 178–193 (1997)

    MathSciNet  Article  MATH  Google Scholar 

  24. 24.

    Kanzow, C., Petra, S.: Projected filter trust region methods for a semismooth least squares formulation of mixed complementarity problems. Opt. Methods Softw. 22(5), 713–735 (2007)

    MathSciNet  Article  MATH  Google Scholar 

  25. 25.

    Qi, L., Sun, J.: A nonsmooth version of Newton’s method. Math. Progr. 58, 353–368 (1993)

    MathSciNet  Article  MATH  Google Scholar 

  26. 26.

    Laird, C.D., Biegler, L.T.: Large-scale nonlinear programming for multi-scenario optimization. In: Bock, H.G., Kostina, E., Phu, H.X., Ranacher, R. (eds.) Modeling, simulation and optimization of complex processes, pp. 323–336. Springer, Berlin (2008)

    Chapter  Google Scholar 

  27. 27.

    De Luca, T., Facchinei, F., Kanzow, C.: A semismooth equation approach to the solution of nonlinear complementarity problems. Math. Progr. 75, 407–439 (1996)

    MathSciNet  MATH  Google Scholar 

  28. 28.

    Mas-Colell, A., Whinston, M.D., Green, J.R.: Microeconomic Theory. Oxford University Press, Oxford (1991)

    MATH  Google Scholar 

  29. 29.

    Munson, T.S., Facchinei, F., Ferris, M., Fischer, A., Kanzow, C.: The Semismooth algorithm for large scale complementarity problems. J. Comput. 13, 294–311 (2001)

    MathSciNet  MATH  Google Scholar 

  30. 30.

    Necoara, I., Patrascu, A.: Iteration complexity analysis of dual first-order methods for conic convex programming. Opt. Methods Softw. 31(3), 645–678 (2016)

    MathSciNet  Article  MATH  Google Scholar 

  31. 31.

    Necoara, I., Suykens, J.A.K.: Application of a smoothing technique to decomposition in convex optimization. IEEE Trans. Autom. Control 53(11), 2674–2679 (2008)

    MathSciNet  Article  Google Scholar 

  32. 32.

    Necoara, I., Suykens, J.A.K.: Application of a smoothing technique to decomposition in convex optimization. IEEE Trans. Autom. Control 53(11), 2674–2679 (2008)

    MathSciNet  Article  Google Scholar 

  33. 33.

    Necoara, I., Suykens, J.A.K.: Interior-point lagrangian decomposition method for separable convex optimization. J. Optim. Theory Appl. 143(3), 567–588 (2009)

    MathSciNet  Article  MATH  Google Scholar 

  34. 34.

    Nesterov, Y.: Introductory Lectures on Convex Optimization: A Basic Course. Kluwer, Boston (2004)

    Book  MATH  Google Scholar 

  35. 35.

    Nesterov, Y.: Excessive gap technique in nonsmooth convex minimization. SIAM J Opt. 16(1), 235–249 (2005)

    MathSciNet  Article  MATH  Google Scholar 

  36. 36.

    Nesterov, Y.: Smooth minimization of non-smooth functions. Math. Progr. (A) 103(1), 127–152 (2005)

    MathSciNet  Article  MATH  Google Scholar 

  37. 37.

    Nesterov, Y., Nemirovskii, A.: Interior Point Polynomial Algorithms in Convex Programming. Society for Industrial and Applied Mathematics (SIAM Studies in Applied Mathematics), Philadelphia (1994)

    Book  MATH  Google Scholar 

  38. 38.

    Neumaier, A.: MINQ—General Definite and Bound Constrained Indefinite Quadratic Programming (1998)

  39. 39.

    Nie, Pu-yan, Lai, Ming yong, Zhu, Shu jin, Zhang, Pei ai: A line search filter approach for the system of nonlinear equations. Comput. Math. Appl. 55(9), 2134–2141 (2008)

    MathSciNet  Article  MATH  Google Scholar 

  40. 40.

    Qi, L.: Convergence analysis of some algorithms for solving nonsmooth equations. Math. Oper. Res. 51, 101–131 (1991)

    Google Scholar 

  41. 41.

    Mifflin, R.: Semismooth and semiconvex functions in constrained optimization. SIAM J. Control Opt. 15(6), 959–972 (1977)

    MathSciNet  Article  MATH  Google Scholar 

  42. 42.

    Cottle, R.W., Pang, J.-S., Stone, R.E.: The Linear Complementarity Problem. Classics in Applied Mathematics. SIAM, Philadelphia (2009)

    Book  Google Scholar 

  43. 43.

    Raghunathan, A.U.: Global optimization of nonlinear network design. SIAM J. Optim. 23(1), 268–295 (2013)

    MathSciNet  Article  MATH  Google Scholar 

  44. 44.

    Raghunathan, A. U., Curtis, F. E., Takaguchi, Y., Hashimoto, H.: Accelerating convergence to competitive equilibrium in electricity markets. In: IEEE Power & Energy Society General Meeting, PESGM2016-000221 (2016)

  45. 45.

    Rockafellar, R.T.: Monotone Operators and the Proximal Point Algorithm. SIAM J. Control Optim. 14(5), 877–898 (1976)

    MathSciNet  Article  MATH  Google Scholar 

  46. 46.

    Shor, N.Z.: Minimization Methods for Non-differentiable Functions. Springer, New York (1985)

    Book  MATH  Google Scholar 

  47. 47.

    De Luca, T., Facchinei, F., Kanzow, C.: A theoretical and numerical comparison of some semismooth algorithms for complementarity problems. Comput. Opt. Appl. 16(2), 173–205 (2000)

    MathSciNet  Article  MATH  Google Scholar 

  48. 48.

    Ulbrich, M., Ulbrich, S., Vicente, L.N.: A globally convergent primal-dual interior-point filter method for nonlinear programming. Math. Progr. 100(2), 379–410 (2004)

    MathSciNet  Article  MATH  Google Scholar 

  49. 49.

    Wächter, A., Biegler, L.T.: On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Math. Program. 106(1), 25–57 (2006)

    MathSciNet  Article  MATH  Google Scholar 

  50. 50.

    Wang, G., Negrete-Pincetic, M., Kowli, A., Shafieepoorfard, E., Meyn, S., Shanbhag, U.V.: Dynamic competitive equilibria in electricity markets. In: Chakrabortty, Aranya, Ili, Marija D. (eds.) Control and Optimization Methods for Electric Smart Grids, volume 3 of Power Electronics and Power Systems, pp. 35–62. Springer, New York (2012)

    Chapter  Google Scholar 

  51. 51.

    Wood, A.J., Wollenberg, B.F.: Power Generation Operation and Control, 2nd edn. Wiley, Hoboken (1996)

    Google Scholar 

  52. 52.

    Zavala, V.M., Laird, C.D., Biegler, L.T.: Interior-point decomposition approaches for parallel solution of large-scale nonlinear parameter estimation problems. Chem. Eng. Sci. 63(19), 4834–4845 (2008)

    Article  Google Scholar 

  53. 53.

    Zimmerman, R.D., Murillo-Sanchez, C.E., Thomas, R.J.: MATPOWER: steady-state operations, planning and analysis tools for power systems research and education. IEEE Trans. Power Syst. 26(1), 12–19 (2011)

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Frank E. Curtis.

Additional information

Frank E. Curtis: This author was partially supported by the U.S. Department of Energy, Office of Science, Applied Mathematics, Early Career Research Program under Award Number DE-SC0010615.

Appendix: Stationarity, differentiability, regularity, and (1.1) versus (1.5)

Appendix: Stationarity, differentiability, regularity, and (1.1) versus (1.5)

In this appendix, we discuss stationarity conditions for the QPs (1.1) and (\(1.2_i\)), differentiability properties of the subproblem solutions \(x_i(\cdot )\) with respect to \(\lambda \), regularity properties of implicit complementarity problem (1.5), and relationships between solutions of (1.1) and (1.5).

Stationarity conditions for (1.1) and (\(1.2_i\))

First-order stationarity conditions for the QPs (1.1) and (\(1.2_i\)) can be derived from standard theory on Karush-Kuhn-Tucker conditions; e.g., see [2]. A point \(\varvec{x}^*\) is a first-order KKT point of (1.1) if there exist multipliers \((\lambda ^*,\varvec{\xi }^*)\) such that

$$\begin{aligned} \begin{aligned} \varvec{Q}\varvec{x}^* + \varvec{q}+ \varvec{A}^T\lambda ^* + \varvec{B}^T\varvec{\xi }^* =&\; 0 \\ 0 \le b - \varvec{A}\varvec{x}^* \perp \lambda ^* \ge&\; 0 \\ 0 \le \varvec{c}- \varvec{B}\varvec{x}^* \perp \varvec{\xi }^* \ge&\; 0. \end{aligned} \end{aligned}$$
(6.6b)

Similarly, a point \(x_i^*\) is a first-order KKT point of (\(1.2_i\)) if there exists multipliers \(\xi _i^*\) such that

$$\begin{aligned} \begin{aligned} Q_ix_i^* + q_i + A_i^T\lambda + B_i^T\xi _i^*&= 0 \\ 0 \le c_i - B_ix_i^* \perp \xi _i^*&\ge 0. \end{aligned} \end{aligned}$$
(7.1)

Second-order KKT points are defined as those at which the first-order KKT conditions can be satisfied and at which a curvature condition on the Hessian holds over a critical cone defined by the corresponding active constraints. To formally state these second-order conditions, let us first define the following: Given a primal point \(\varvec{x}\in \mathbb {R}^{nN}\), a Lagrange multiplier vector for the coupling constraint \(\lambda \in \mathbb {R}^m\), and Lagrange multipliers for the subproblem constraints \(\varvec{\xi }:= (\xi _1,\dots ,\xi _N) \in \mathbb {R}^{pN}\), let subsets of the indices for the coupling constraints be

$$\begin{aligned} \begin{aligned} \alpha (\varvec{x},\lambda )&:= \{ j : (\varvec{A}\varvec{x}- b)_{[j]} = 0< \lambda _{[j]} \}, \\ \beta (\varvec{x},\lambda )&:= \{ j : (\varvec{A}\varvec{x}- b)_{[j]} = 0 = \lambda _{[j]} \}, \\ \text {and}\ \ \gamma (\varvec{x},\lambda )&:= \{ j : ( \varvec{A}\varvec{x}-b)_{[j]} < 0 = \lambda _{[j]} \}, \end{aligned} \end{aligned}$$
(7.2)

and, for all \(i \in \{1,\dots ,N\}\), let subsets of the indices for the inequalities in \(\mathcal{X}_i\) be

$$\begin{aligned} \begin{aligned} \alpha _i(x_i,\xi _i)&:= \{ j : (B_ix_i - c_i)_{[j]} = 0< \xi _{i[j]} \}, \\ \beta _i(x_i,\xi _i)&:= \{ j : (B_ix_i - c_i)_{[j]} = 0 = \xi _{i[j]} \}, \\ \text {and}\ \ \gamma _i(x_i,\xi _i)&:= \{ j : (B_ix_i - c_i)_{[j]} < 0 = \xi _{i[j]} \}. \end{aligned} \end{aligned}$$
(7.3)

Using these definitions, given \((\varvec{x}^*,\lambda ^*,\varvec{\xi }^*)\) satisfying (7.1), we define the critical cone

$$\begin{aligned} \mathcal{T}(\varvec{x}^*,\lambda ^*,\varvec{\xi }^*) := \left\{ \varvec{d}= \begin{bmatrix} d_1 \\ \vdots \\ d_N \end{bmatrix} \in \mathbb {R}^{nN} : \left\{ \begin{aligned} \sum \limits _{i=1}^NA_{i[\alpha \cdot ]}d_i = 0,\; \sum \limits _{i=1}^NA_{i[\beta \cdot ]}d_i&\le 0, \\ \text {and}\ \ B_{i[\alpha _i\cdot ]}d_i = 0, \; B_{i[\beta _i\cdot ]}d_i&\le 0 \\ \text {for all}\ \ i \in \{1,\dots ,N\} \end{aligned} \right\} \right\} . \end{aligned}$$
(7.4)

Similarly, corresponding to a point \((x_i^*,\xi _i^*)\) satisfying (7.2), we define the critical cone

$$\begin{aligned} {\mathcal {T}}_i(x_i^*,\xi _i^*) := \left\{ d_i \in \mathbb {R}^n : B_{i[\alpha _i\cdot ]}d_i = 0\ \ \text {and}\ \ B_{i[\beta _i\cdot ]}d_i \le 0 \right\} , \end{aligned}$$
(7.5)

We may now state that \(\varvec{x}^*\) is a second-order KKT point of (1.1) if there exist multipliers \((\lambda ^*,\varvec{\xi }^*)\) such that (7.1) holds and

$$\begin{aligned} \varvec{d}^T\varvec{Q}\varvec{d}\ge 0\ \ \text {for all}\ \ \varvec{d}\in \mathcal{T}(\varvec{x}^*,\varvec{\xi }^*,\lambda ^*), \end{aligned}$$
(7.6)

and \(x_i^*\) is a second-order KKT point of (\(1.2_i\)) if there exists \(\xi _i^*\) such that (7.2) holds and

$$\begin{aligned} d_i^TQ_id_i \ge 0\ \ \text {for all}\ \ d_i \in \mathcal{T}_i(x_i^*,\xi _i^*). \end{aligned}$$
(7.7)

If these curvature conditions hold strictly for nonzero elements in the critical cone, then one can conclude that the corresponding point is a strict local minimizer. In particular, \(\varvec{x}^*\) is a strict local minimizer of (1.1) if there exist multipliers \((\lambda ^*,\varvec{\xi }^*)\) such that (7.1) holds and

$$\begin{aligned} \varvec{d}^T\varvec{Q}\varvec{d}> 0\ \ \text {for all}\ \ \varvec{d}\in \mathcal{T}(\varvec{x}^*,\varvec{\xi }^*,\lambda ^*) \setminus \{0\}, \end{aligned}$$
(7.8)

while \(x_i^*\) is a strict local minimizer of (\(1.2_i\)) if there exists \(\xi _i^*\) such that (7.2) holds and

$$\begin{aligned} d_i^TQ_id_i > 0\ \ \text {for all}\ \ d_i \in \mathcal{T}_i(x_i^*,\xi _i^*) \setminus \{0\}. \end{aligned}$$
(7.9)

Overall, one can observe that if (1.1) (resp. (\(1.2_i\))) is convex in that \(\varvec{Q}\succeq 0\) (resp. \(Q_i \succeq 0\)), then any first-order KKT point is a second-order KKT point, from which convexity implies that it is a global minimizer of (1.1) (resp. (\(1.2_i\))). Similarly, if (1.1) (resp. (\(1.2_i\))) is strictly convex in that \(\varvec{Q}\succ 0\) (resp. \(Q_i \succ 0\)), then any first-order KKT point is a strict local minimizer, from which convexity implies that it is a strict global minimizer of (1.1) (resp. (\(1.2_i\))).

Differentiability concepts of locally Lipschitz functions

Let \({\varPhi } : \mathbb {R}^m \rightarrow \mathbb {R}^m\) be locally Lipschitz. Then, by Rademacher’s theorem, \({\varPhi }\) is differentiable almost everywhere in \(\mathbb {R}^m\). If \(D_{\varPhi }\) denotes the set of \(\lambda \in \mathbb {R}^m\) at which \({\varPhi }\) is differentiable, then the B-subdifferential of \({\varPhi }\) at \(\lambda \) can be defined (see [25]) as

$$\begin{aligned} \partial _B{\varPhi }(\lambda ) := \left\{ H \in \mathbb {R}^{m\times m} : H = \lim \limits _{\lambda ^k \in D_{\varPhi },\lambda ^k \rightarrow \lambda } \nabla {\varPhi }(\lambda ^k) \right\} . \end{aligned}$$
(7.10)

The convex hull of this set is the generalized Jacobian of \({\varPhi }\) at \(\lambda \) [25], written as

$$\begin{aligned} \partial {\varPhi } := \text {conv}\;\partial _B{\varPhi }(\lambda ). \end{aligned}$$

Such a function is semismooth [25, 41] at \(\lambda \) if it is directionally differentiable at \(\lambda \) and

$$\begin{aligned} H\delta - {\varPhi }'(\lambda ;\delta ) = o(\Vert \delta \Vert )\ \ \text {for any}\ \ \delta \rightarrow 0\ \ \text {and}\ \ H \in \partial {\varPhi }(\lambda +\delta ). \end{aligned}$$

Further, \({\varPhi }\) is strongly semismooth at \(\lambda \) if

$$\begin{aligned} H\delta - {\varPhi }'(\lambda ;\delta ) = o(\Vert \delta \Vert ^2)\ \ \text {for any}\ \ \delta \rightarrow 0\ \ \text {and}\ \ H \in \partial {\varPhi }(\lambda +\delta ). \end{aligned}$$

Regularity properties of solutions of (1.5)

A square matrix \(M\in \mathbb {R}^{m\times m}\) is said to be a P-matrix if all of its principal minors are positive [42, Def. 3.3.1]. If \(M \in \mathbb {R}^{m \times m}\) is a symmetric P-matrix, then M is positive definite. This property is of importance for matrices arising in the following definition, which relates to regularity of solutions for (1.5); see [47, Def. 2.1].

Definition 7.1

(b- and R-regular solutions of (1.5)) Suppose that \(F \in C^1\). Let \(\lambda ^*\) be a solution of (1.5) and let \(\alpha \), \(\beta \), and \(\gamma \) be defined as in (7.3). Then, \(\lambda ^*\) is a

  • b-regular solution of (1.5) if \(\nabla F_{[\sigma \sigma ]}(\lambda ^*)\) is nonsingular whenever \(\alpha \subseteq \sigma \subseteq \alpha \cup \beta \);

  • R-regular solution of (1.5) if \(\nabla F_{[\alpha \alpha ]}(\lambda ^*)\) is nonsingular and its Schur complement w.r.t.

    $$\begin{aligned} \begin{bmatrix} \nabla F_{[\alpha \alpha ]}(\lambda ^*)&\nabla F_{[\alpha \beta ]}(\lambda ^*) \\ \nabla F_{[\beta \alpha ]}(\lambda ^*)&\nabla F_{[\beta \beta ]}(\lambda ^*) \end{bmatrix}, \end{aligned}$$

    namely

    $$\begin{aligned} \begin{aligned} (\nabla F_{[\beta \beta ]}/\nabla F_{[\alpha \alpha ]})(\lambda ^*) := \nabla F_{[\beta \beta ]}(\lambda ^*) - \nabla F_{[\beta \alpha ]}(\lambda ^*) (\nabla F_{[\alpha \alpha ]}(\lambda ^*))^{-1} \nabla F_{[\alpha \beta ]}(\lambda ^*) \end{aligned} \end{aligned}$$
    (7.11)

    is a P-matrix.

Differentiability properties of \(x_i(\lambda )\)

The differentiability properties of a first-order KKT point \(x_i^*(\lambda )\) of (\(1.2_i\)) and its corresponding objective value as functions of \(\lambda \) follow from the theory of sensitivity analysis of parametric nonlinear optimization problems. The study of solution and objective sensitivity to variations in parameters in such problems has been studied extensively since the 1970’s. Since the constraints of (1.1) and (\(1.2_i\)) are affine, we have that the conditions in Appendix 1 are necessary for stationarity with the last conditions sufficient for (local) optimality. However, for certain results below, we rely on additional regularity conditions for (\(1.2_i\)).

We begin by stating two types of regularity conditions on the constraints in (\(1.2_i\)).

Definition 7.2

((\(1.2_i\))-LICQ) If \(B_{i[(\alpha _i\cup \beta _i)\cdot ]}\) has full row rank (with \((\alpha _i,\beta _i)\) from (7.4)), then the Linear Independence Constraint Qualification (LICQ) for (\(1.2_i\)) holds at \(x_i \in \mathcal{X}_i\).

Definition 7.3

((\(1.2_i\))-SCQ) If there exists \(x_i \in \mathbb {R}^n\) such that \(B_i x_i < c_i\), then the Slater Constraint Qualification (SCQ) holds for (\(1.2_i\)).

If a problem has a convex feasible region, as does (\(1.2_i\)), the SCQ is equivalent to the Mangasarian-Fromovitz Constraint Qualification (MFCQ) holding at every feasible point. If the LICQ holds at any feasible point, then the SCQ holds. For a local minimizer \(x_i^*(\lambda )\) of (\(1.2_i\)), the LICQ implies uniqueness of the optimal Lagrange multiplier vector \(\xi ^*_i(\lambda )\); however, the SCQ only implies that the set of optimal multipliers at \(x_i^*(\lambda )\), call it \(\Xi _i(x_i^*(\lambda ))\), is bounded.

Now, given a first-order KKT point of (\(1.2_i\)) and the critical cone

$$\begin{aligned} \mathcal{T}^{sc}_i(x_i^*,\xi _i^*) := \{ d_i \in \mathbb {R}^n : B_{i[\alpha _i\cdot ]}d_i = 0\} \supseteq \mathcal{T}_i(x_i^*,\xi _i^*) \end{aligned}$$
(7.12)

with \(\alpha _i\) defined in (7.4), we define the following.

Definition 7.4

((\(1.2_i\))-SSOSC) If \((x_i^*,\xi _i^*)\) is a first-order KKT point of (\(1.2_i\)), then the strong second-order sufficient condition (SSOSC) for (\(1.2_i\)) holds at \((x_i^*,\xi _i^*)\) if

$$\begin{aligned} d_i^TQ_id_i > 0\ \ \text {for all}\ \ d_i \in \mathcal{T}^{sc}_i(x_i^*,\xi _i^*). \end{aligned}$$
(7.13)

An early result on differentiability of \(x_i(\cdot )\) for our purposes can be derived from the work of Fiacco [13, Thm. 2.1]. We state it as the following lemma.

Lemma 7.1

Given \(\bar{\lambda } \in \mathbb {R}^m\), suppose that \(x_i^*(\bar{\lambda })\) is a strict local solution of (\(1.2_i\)) with \(\lambda =\bar{\lambda }\) such that with the corresponding Lagrange multiplier \(\xi _i^*(\bar{\lambda })\) the following hold:

  • \(\beta _i(x_i^*(\bar{\lambda }),\xi _i^*(\bar{\lambda })) = \emptyset \),

  • the (\(1.2_i\))-LICQ holds at \(x_i^*(\bar{\lambda })\), and

  • the (\(1.2_i\))-SSOSC holds at \((x_i^*(\bar{\lambda }),\xi _i^*(\bar{\lambda }))\).

Then, there exist open neighborhoods \(U_i\), \(V_i\), and \(W_i\) centered at \(x_i^*\), \(\xi _i^*\), and \(\bar{\lambda }\), respectively, and functions \(x_i(\cdot ) : W_i \rightarrow U_i \subseteq \mathcal{X}_i\) and \(\xi _i(\cdot ) : W_i \rightarrow V_i\) such that:

  1. (i)

    \(x_i(\cdot )\) and \(\xi _i(\cdot )\) are \(C^1\) functions of \(\lambda \in W_i\);

  2. (ii)

    for each \(\lambda \in W_i\), the point \(x_i(\lambda )\) is a strict local solution of (\(1.2_i\)) with Lagrange multiplier \(\xi _i(\lambda )\); and

  3. (iii)

    for each \(\lambda \in W_i\), the Jacobian of \((x_i(\cdot ),\xi _i(\cdot ))\) at \(\lambda \) is given by

    $$\begin{aligned} \begin{bmatrix} Q_i&B^T_{i[\alpha _i\cdot ]} \\ B_{i[\alpha _i\cdot ]}&0 \end{bmatrix} \begin{bmatrix} \nabla x_i(\lambda ) \\ \nabla \xi _{i[\alpha _i]}(\lambda ) \end{bmatrix} = \begin{bmatrix} -A_i^T \\ 0 \end{bmatrix}. \end{aligned}$$
    (7.14)

The following result, due to Ralph and Dempe [6, Thms. 1 & 2 and Cor. 4(2)], represents an extension of this result in which the strict complementarity requirement (i.e., \(\beta _i = \emptyset \)) is relaxed and the LICQ is replaced by the (weaker) MFCQ along with the Constant Rank Constraint Qualification (CRCQ). The CRCQ holds automatically for (1.1), so we do not state it explicitly. Moreover, the MFCQ is equivalent to the SCQ for (1.1), so we refer to the latter.

Lemma 7.2

Given \(\bar{\lambda } \in \mathbb {R}^m\), suppose that \(x_i^*(\bar{\lambda })\) is a strict local solution of (\(1.2_i\)) with \(\lambda =\bar{\lambda }\) such that the following hold:

  • the (\(1.2_i\))-SCQ holds and

  • the (\(1.2_i\))-SSOSC holds for all \(\xi _i \in \Xi _i(x_i^*(\bar{\lambda }))\).

Then, for all \(i \in \{1,\dots ,N\}\), there exist open neighborhoods \(U_i\) and \(W_i\) centered at \(x_i^*(\bar{\lambda })\) and \(\bar{\lambda }\), respectively, and a function \(x_i(\cdot ) : W_i \rightarrow U_i \subseteq \mathcal{X}_i\) such that:

  1. (i)

    \(x_i(\cdot )\) is a \(PC^1\) function of \(\lambda \in W_i\);

  2. (ii)

    for each \(\lambda \in W_i\), the point \(x_i(\lambda )\) is a strict local solution of (\(1.2_i\)); and

  3. (iii)

    the directional derivative \(x'_i(\lambda ;\cdot )\) is a piecewise linear function such that for each \(\lambda \in W_i\), \(d \in \mathbb {R}^m\), and \(\xi _i \in \Xi _i(x_i(\lambda ))\) the value \(x_i'(\lambda ;\delta )\) is given by the unique solution of

    $$\begin{aligned} \min \limits _v\; \tfrac{1}{2}v^TQ_iv + v^TA_i^T\delta \ \ \text {s.t.}\ \ B_{i[\alpha _i\cdot ]}v = 0,\ \ B_{i[\beta _i\cdot ]}v \le 0. \end{aligned}$$
    (7.15)

We remark that the conclusion of Lemma 7.2, namely, that \(x_i(\cdot )\) is a \(PC^1\) function of \(\lambda \in W_i\), implies that \(x_i(\cdot )\) is locally Lipschitz and directionally differentiable.

Relationship between the QP and the ICP formulations

A solution to (1.5) easily yields a first-order KKT point of (1.1). In particular, we have the following result whose proof follows simply by combining (7.2) with (1.5).

Lemma 7.3

Suppose \(\lambda ^*\) solves (1.5) with \(x_i^*\) denoting a first-order KKT point of (\(1.2_i\)) for \(\lambda = \lambda ^*\) for all \(i \in \{1,\dots ,N\}\). Then, \(\varvec{x}^*\) is a first-order KKT point of (1.1).

Similarly, a solution to (1.5) can also yield a second-order KKT point of (1.1). For this claim, we have the following result whose proof follows by combining (7.8) and the fact that the Cartesian product of the \(\mathcal{T}_i\)’s represents a superset of \(\mathcal{T}\).

Lemma 7.4

Suppose \(\lambda ^*\) solves (1.5) with \(x_i^*\) denoting a second-order KKT point of (\(1.2_i\)) for \(\lambda = \lambda ^*\) for all \(i \in \{1,\dots ,N\}\). Then, \(\varvec{x}^*\) is a second-order KKT point of (1.1).

The reverse implications are the subjects of the following lemmas, the proofs of which are straightforward; hence, we state them without proof.

Lemma 7.5

Suppose that \(\varvec{x}^*\) is a first-order KKT point of (1.1) in that there exist multipliers \((\lambda ^*,\varvec{\xi }^*)\) such that (7.1) holds. Then, the following hold:

  1. (i)

    for each \(i \in \{1,\dots ,N\}\), \(x_i^*\) is a first-order KKT point for (\(1.2_i\)) for \(\lambda = \lambda ^*\), and

  2. (ii)

    \(\lambda ^*\) solves (1.5).

Lemma 7.6

Suppose that \(\varvec{x}^*\) is a second-order KKT point of (1.1) in that there exist multipliers \((\varvec{\xi }^*,\lambda ^*)\) such that (7.1) and (7.7) hold. Moreover, suppose that

$$\begin{aligned} \varvec{d}^T\varvec{Q}\varvec{d}\ge 0\ \ \text {for all}\ \ \varvec{d}= (d_1,\ldots ,d_N) \in \prod \limits _{i=1}^N\mathcal{T}_i(x_i^*,\xi _i^*). \end{aligned}$$
(7.16)

Then, the following hold:

  1. (i)

    for each \(i \in \{1,\dots ,N\}\), \(x_i^*\) is a second-order KKT point for (\(1.2_i\)) for \(\lambda = \lambda ^*\), and

  2. (ii)

    \(\lambda ^*\) solves (1.5).

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Curtis, F.E., Raghunathan, A.U. Solving nearly-separable quadratic optimization problems as nonsmooth equations. Comput Optim Appl 67, 317–360 (2017). https://doi.org/10.1007/s10589-017-9895-8

Download citation

Keywords

  • Quadratic optimization problems
  • Dual decomposition
  • Complementarity problems
  • Semismooth Newton methods
  • Fischer–Burmeister function

Mathematics Subject Classification

  • 49M05
  • 49M15
  • 49M27
  • 49M29
  • 65K05
  • 65K10
  • 90C20
  • 90C33