Sum-of-Squares Relaxations in Robust DC Optimization and Feature Selection

Abstract

This paper presents sum-of-squares (SOS) relaxation results to a difference-of-convex-max (DC-max) optimization involving SOS-convex polynomials in the face of constraint data uncertainty and their applications to robust feature selection. The main novelty of the present work in relation to the recent research in robust convex and DC optimization is the derivation of a new form of minimally exact SOS relaxations for robust DC-max problems. This leads to the identification of broad classes of robust DC-max problems with finitely exact SOS relaxations that are numerically tractable. They allow one to find the optimal values of these classes of DC-max problems by solving a known finite number of semi-definite programs (SDPs) for certain concrete cases of commonly used uncertainty sets in robust optimization. In particular, we derive relaxation results for a class of robust fractional programs. Also, we provide a finitely exact SDP relaxation for a DC approximation problem of an NP-hard robust feature selection model which gives computable upper bounds for the global optimal value.

Appendix A: Supplementary Proofs for Sect. 4

Lemma A.1

Let $\varphi _k: [0,1]^n \rightarrow {{\mathbb {R}}}$ be

$$\begin{aligned}&\varphi _k (r_1,\ldots ,r_n) \\&\quad =\max _{ \begin{array}{c} \mu \in {{\mathbb {R}}}, \\ {\widetilde{\xi }}_i, {\overline{\xi }}_i \ge 0, i=1,\ldots ,m, \\ \overline{p}_j, \overline{q}_j \ge 0, j=1,\ldots ,n \end{array} } \Big \{\mu : (1-\lambda ) \sum _{i=1}^m\sigma _i \xi _i + \lambda \alpha \sum _{j=1}^n(p_j+q_j) - \lambda \alpha \sum _{j=1}^nr_j (p_j+q_j) \\&\qquad + \lambda \sum _{j=1}^nr_j + \sum _{i=1}^m{\widetilde{\xi }}_i [1-\xi _i - y_i ( \langle p-q,c_k\rangle + b ) ] \\&\qquad -\sum _{i=1}^m{\overline{\xi }}_i \xi _i - \sum _{j=1}^n\overline{p}_j p_j - \sum _{j=1}^n\overline{q}_j q_j \ge \mu ,\ \forall (p,q,\xi ,b) \in {{\mathbb {R}}}^{2n+m+1} \Big \}. \end{aligned}$$

Then, $\varphi _k$ is concave.

Proof

Take $(r_1,\ldots ,r_n), (r_1',\ldots ,r_n') \in [0,1]^n$ and $\theta \in (0,1)$. There exist $\mu \in {{\mathbb {R}}}, {\widetilde{\xi }}_i,{\overline{\xi }}_i, \overline{p}_j, \overline{q}_j \ge 0$ such that $\varphi (r_1,\ldots , r_n) = \mu$ and

$$\begin{aligned}&(1-\lambda ) \sum _{i=1}^m\sigma _i \xi _i + \lambda \alpha \sum _{j=1}^n(p_j+q_j) - \lambda \alpha \sum _{j=1}^nr_j (p_j+q_j) \\ {}&\quad + \lambda \sum _{j=1}^nr_j + \sum _{i=1}^m{\widetilde{\xi }}_i [1-\xi _i - y_i ( \langle p-q,c_k\rangle + b ) ] \\ {}&\quad - \sum _{i=1}^m{\overline{\xi }}_i \xi _i - \sum _{j=1}^n\overline{p}_j p_j - \sum _{j=1}^n\overline{q}_j q_j \ge \mu ,\ \forall (p,q,\xi ,b) \in {{\mathbb {R}}}^{2n+m+1}. \end{aligned}$$

There exist $\mu '\in {{\mathbb {R}}}, {\widetilde{\xi }}_i',{\overline{\xi }}_i', \overline{p}_j', \overline{q}_j' \ge 0$ and

$$\begin{aligned}&(1-\lambda ) \sum _{i=1}^m\sigma _i \xi _i + \lambda \alpha \sum _{j=1}^n(p_j+q_j) - \lambda \alpha \sum _{j=1}^nr_j' (p_j+q_j) \\&\quad + \lambda \sum _{j=1}^nr_j' + \sum _{i=1}^m{\widetilde{\xi }}_i' [1-\xi _i - y_i ( \langle p-q,c_k\rangle + b ) ]\\&\quad - \sum _{i=1}^m{\overline{\xi }}_i' \xi _i - \sum _{j=1}^n\overline{p}_j' p_j - \sum _{j=1}^n\overline{q}_j' q_j \ge \mu ',\ \forall (p,q,\xi ,b) \in {{\mathbb {R}}}^{2n+m+1}. \end{aligned}$$

Now,

$$\begin{aligned}{} & {} \theta \varphi (r_1,\ldots , r_n) + (1-\theta ) \varphi (r_1', \ldots , r_n')= \theta \mu + (1-\theta ) \mu ' \\{} & {} \quad \le \theta \Big [(1-\lambda ) \sum _{i=1}^m\sigma _i \xi _i + \lambda \alpha \sum _{j=1}^n(p_j+q_j) - \lambda \alpha \sum _{j=1}^nr_j (p_j+q_j) + \lambda \sum _{j=1}^nr_j \\{} & {} \qquad + \sum _{i=1}^m{\widetilde{\xi }}_i [1-\xi _i - y_i ( \langle p-q,c_k\rangle + b ) ] - \sum _{i=1}^m{\overline{\xi }}_i \xi _i - \sum _{j=1}^n\overline{p}_j p_j - \sum _{j=1}^n\overline{q}_j q_j \Big ] \\{} & {} \qquad + (1-\theta ) \Big [(1-\lambda ) \sum _{i=1}^m\sigma _i \xi _i + \lambda \alpha \sum _{j=1}^n(p_j+q_j) - \lambda \alpha \sum _{j=1}^nr_j' (p_j+q_j) \\{} & {} \qquad + \lambda \sum _{j=1}^nr_j' + \sum _{i=1}^m{\widetilde{\xi }}_i' [1-\xi _i - y_i ( \langle p-q,c_k\rangle + b ) ] - \sum _{i=1}^m{\overline{\xi }}_i' \xi _i - \sum _{j=1}^n\overline{p}_j' p_j - \sum _{j=1}^n\overline{q}_j' q_j \Big ]\\{} & {} \quad = (1-\lambda ) \sum _{i=1}^m\sigma _i \xi _i + \lambda \alpha \sum _{j=1}^n(p_j+q_j) - \lambda \alpha \sum _{j=1}^n(\theta r_j + (1-\theta ) r_j') (p_j+q_j) \\{} & {} \qquad + \lambda \sum _{j=1}^n(\theta r_j + (1-\theta ) r_j') + \sum _{i=1}^m(\theta {\widetilde{\xi }}_i + (1-\theta ) {\widetilde{\xi }}_i') \Big [1-\xi _i - y_i ( \langle p-q,c_k\rangle + b ) \Big ] \\{} & {} \qquad - \sum _{i=1}^m(\theta {\overline{\xi }}_i + (1-\theta ) {\overline{\xi }}_i') \xi _i - \sum _{j=1}^n(\theta \overline{p}_j + (1-\theta ) \overline{p}_j') p_j - \sum _{j=1}^n(\theta \overline{q}_j + (1-\theta ) \overline{q}_j') q_j \\{} & {} \quad \le \varphi (\theta r_1 + (1-\theta ) r_1, \ldots , \theta r_n + (1-\theta ) r_n). \end{aligned}$$

By definition, $\varphi _k$ is concave. $\square$

Infimum of a concave function over the convex hull of finitely many points is equal to the infimum over the set of those points [31, Theorem 32.2].