Simple and fast algorithm for binary integer and online linear programming

Abstract

In this paper, we develop a simple and fast online algorithm for solving a class of binary integer linear programs (LPs) arisen in general resource allocation problem. The algorithm requires only one single pass through the input data and is free of matrix inversion. It can be viewed as both an approximate algorithm for solving binary integer LPs and a fast algorithm for solving online LP problems. The algorithm is inspired by an equivalent form of the dual problem of the relaxed LP and it essentially performs (one-pass) projected stochastic subgradient descent in the dual space. We analyze the algorithm in two different models, stochastic input and random permutation, with minimal technical assumptions on the input data. The algorithm achieves $O\left( m \sqrt{n}\right)$ expected regret under the stochastic input model and $O\left( (m+\log n)\sqrt{n}\right)$ expected regret under the random permutation model, and it achieves $O(m \sqrt{n})$ expected constraint violation under both models, where n is the number of decision variables and m is the number of constraints. The algorithm enjoys the same performance guarantee when generalized to a multi-dimensional LP setting which covers a wider range of applications. In addition, we employ the notion of the permutational Rademacher complexity and derive regret bounds for two earlier online LP algorithms for comparison. Both algorithms improve the regret bound by a factor of $\sqrt{m}$ by paying more computational cost. Furthermore, we demonstrate how to convert the possibly infeasible solution to a feasible one through a randomized procedure. Numerical experiments illustrate the general applicability and effectiveness of the algorithms.

Appendices

Appendix

Concentration inequalities under random permutations

Lemma 5

Let $U_1 , ... , U_n$ be a random sample without replacement from the real numbers $\{c_1,...,c_N\}$. Then for every $s > 0$,

$$\begin{aligned} \begin{aligned} {\mathbb {P}}(\vert \bar{U_n}-{\bar{c}}_N\vert \ge s) \le \left\{ \begin{array}{lll} 2\exp \left( -\frac{2ns^2}{\Delta _N^2}\right) &{}\quad \text {(Hoeffding)},\\ 2\exp \left( -\frac{2ns^2}{(1-(n-1)/N)\Delta _N^2}\right) &{}\quad \text {(Serfling)},\\ 2\exp \left( -\frac{ns^2}{2\sigma ^2_N+s\Delta _N}\right) &{}\quad \text {(Hoeffding-Bernstein)},\\ 2\exp \left( -\frac{ns^2}{m\sigma _N^2}\right) &{}\quad \text {if N = mn (Massart)}, \end{array} \right. \end{aligned} \end{aligned}$$

where ${\bar{c}}_N = \frac{1}{N}\sum \nolimits _{i=1}^{N}c_i$, $\sigma ^2_N=\frac{1}{N}\sum \nolimits _{i=1}^{N}(c_i-{\bar{c}}_N)^2$ and $\Delta _N=\max \limits _{1\le i \le N}c_i-\min \limits _{1\le i \le N}c_i$.

Proof

See Theorem 2.14.19 in [53]. $\square$

Proof of Lemma 1

Proof

For ${\varvec{p}}^*$, the optimal solution of (SP), we have

$$\begin{aligned} {\underline{d}}\Vert {\varvec{p}}^*\Vert _1 \le {\varvec{d}}^T{\varvec{p}}^* \overset{(a)}{\le } {\varvec{E}} r^+\le {\bar{r}}, \end{aligned}$$

where inequality (a) is due to that if otherwise, ${\varvec{p}}^*$ cannot be the optimal solution because it will give a larger objective value of $f({\varvec{p}})$ than setting ${\varvec{p}}={\varvec{0}}.$ Given the non-negativeness of ${\varvec{p}}^*$, we have $\Vert {\varvec{p}}^*\Vert _2 \le \Vert {\varvec{p}}^*\Vert _1.$ So we obtain the first inequality in the lemma.

For ${\varvec{p}}_t$ specified by Algorithm 1, we have,

$$\begin{aligned} \Vert {\varvec{p}}_{t+1}\Vert ^2_2&\le \left\| {\varvec{p}}_{t}+\gamma _t\left( {\varvec{a}}_tx_t-{\varvec{d}} \right) \right\| ^2_2 \\&= \Vert {\varvec{p}}_{t}\Vert _2^2 + \gamma _t^2 \Vert {\varvec{a}}_tx_t - {\varvec{d}}\Vert _2^2 + 2\gamma _t ({\varvec{a}}_t x_t - {\varvec{d}})^\top {\varvec{p}} _{t}\\&\le \Vert {\varvec{p}}_{t}\Vert _2^2 + \gamma _t^2 m({\bar{a}}+{\bar{d}})^2 + 2\gamma _t {\varvec{a}}_t^\top {\varvec{p}} _{t} x_t - 2\gamma _t{\varvec{d}}^\top {\varvec{p}}_{t} \end{aligned}$$

where the first inequality comes from the projection (into the non-negative orthant) step in the algorithm. Note that

$$\begin{aligned} {\varvec{a}}_t^\top {\varvec{p}}_{t}x_t = {\varvec{a}}_t^\top {\varvec{p}}_{t}I(r_t>{\varvec{a}}_t^\top {\varvec{p}}_{t}) \le r_t \le {\bar{r}}. \end{aligned}$$

Therefore,

$$\begin{aligned} \Vert {\varvec{p}}_{t+1}\Vert ^2_2&\le \Vert {\varvec{p}}_{t}\Vert _2^2 + \gamma _t^2 m({\bar{a}} +{\bar{d}})^2 + 2\gamma _t {\bar{r}} - 2\gamma _t{\varvec{d}}^\top {\varvec{p}}_{t}, \end{aligned}$$

and it holds with probability 1.

Next, we establish that when $\Vert {\varvec{p}}_{t}\Vert _2$ is large enough, then it must hold that $\Vert {\varvec{p}}_{t+1}\Vert _2 \le \Vert {\varvec{p}}_{t}\Vert _2$. Specifically, when $\Vert {\varvec{p}}_{t}\Vert _2\ge \frac{2{\bar{r}}+m({\bar{a}}+{\bar{d}})^2}{{\underline{d}}}$, we have

$$\begin{aligned} \Vert {\varvec{p}} _{t+1}\Vert _2^2 - \Vert {\varvec{p}} _t\Vert _2^2&\le \gamma _t^2 m({\bar{a}} +{\bar{d}})^2 + 2\gamma _t {\bar{r}} - 2\gamma _t{\varvec{d}}^\top {\varvec{p}}_{t} \\&\le \gamma _t^2 m({\bar{a}}+{\bar{d}})^2 + 2\gamma _t {\bar{r}} - 2\gamma _t{\underline{d}} \Vert {\varvec{p}}_{t}\Vert _1\\&\le \gamma _t^2 m({\bar{a}}+{\bar{d}})^2 + 2\gamma _t {\bar{r}} - 2\gamma _t{\underline{d}} \Vert {\varvec{p}}_{t}\Vert _2\\&\le 0 \end{aligned}$$

when $\gamma _t\le 1.$ On the other hand, when $\Vert {\varvec{p}}_{t}\Vert _2\le \frac{2{\bar{r}}+m({\bar{a}}+{\bar{d}})^2}{{\underline{d}}}$,

$$\begin{aligned} \Vert {\varvec{p}}_{t+1}\Vert _2&\le \left\| {\varvec{p}}_{t}+\gamma _t\left( {\varvec{a}}_tx_t-{\varvec{d}} \right) \right\| _2 \\&\overset{(b)}{\le } \Vert {\varvec{p}}_{t}\Vert _2 + \gamma _t\Vert {\varvec{a}}_tx_t-{\varvec{d}}\Vert _2 \\&\le {\frac{2{\bar{r}}+m({\bar{a}}+{\bar{d}})^2}{{\underline{d}}}} + m({\bar{a}}+{\bar{d}}) \end{aligned}$$

where (b) comes from the triangle inequality of the norm.

Combining these two scenarios with the fact that ${\varvec{p}} _1 = {\varvec{0}}$, we have

$$\begin{aligned} \Vert {\varvec{p}}_{t}\Vert _2 \le {\frac{2{\bar{r}}+m({\bar{a}}+{\bar{d}})^2}{{\underline{d}}}} + m({\bar{a}}+{\bar{d}}) \end{aligned}$$

for $t=1,...,n$ with probability 1.

$\square$

Proof of Theorem 1

Proof

First, the primal optimal objective value is no greater than the dual objective with ${\varvec{p}}={\varvec{p}}^*.$ Specifically,

$$\begin{aligned} R_n^* = \text {P-LP}&= \text {D-LP}\\&\le n{\varvec{d}}^\top {\varvec{p}}^* + \sum _{j=1}^n \left( r_j-{\varvec{a}}_j^\top {\varvec{p}}^*\right) ^+. \end{aligned}$$

Taking expectation on both sides,

$$\begin{aligned} {\varvec{E}} \left[ R_n^*\right]&\le {\varvec{E}} \left[ n{\varvec{d}}^\top {\varvec{p}}^* + \sum _{t=1}^n \left( r_t-{\varvec{a}}_t^\top {\varvec{p}}^*\right) ^+\right] \\&\le nf({\varvec{p}}^*). \end{aligned}$$

Thus, the optimal objective value of the stochastic program (by a factor of n) is an upper bound for the expected value of the primal optimal objective. Hence

$$\begin{aligned} {\varvec{E}} [R_n^* - R_n]&\le nf({\varvec{p}}^*) - \sum _{j=1}^n {\varvec{E}} \left[ r_tI(r_t> {\varvec{a}}_t^\top {\varvec{p}} _{t})\right] \nonumber \\&\le \sum _{t=1}^n {\varvec{E}} \left[ f({\varvec{p}}_t)\right] - \sum _{t=1}^n {\varvec{E}} \left[ r_tI(r_t> {\varvec{a}}_t^\top {\varvec{p}} _{t})\right] \nonumber \\&\le \sum _{t=1}^n {\varvec{E}} \left[ {\varvec{d}}^\top {\varvec{p}}_t + \left( r_t -{\varvec{a}}_t^\top {\varvec{p}}_t\right) ^+- r_tI(r_t> {\varvec{a}}_t^\top {\varvec{p}} _{t}) \right] \nonumber \\&= \sum _{t=1}^n {\varvec{E}} \left[ \left( {\varvec{d}}^\top - {\varvec{a}}_tI(r_t > {\varvec{a}}_t^\top {\varvec{p}} _{t})\right) ^\top {\varvec{p}}_t\right] . \end{aligned}$$

(1)

where the expectation is taken with respect to $(r_t, {\varvec{a}}_t)$’s. In above, the second line comes from the optimality of ${\varvec{p}}^*$, while the third line is valid because of the independence between ${\varvec{p}}_t$ and $(r_t, {\varvec{a}}_t).$

The importance of the above inequality lies in that it relates and represents the primal optimality gap in the dual prices ${\varvec{p}}_t$ – which is the core of Algorithm 1. From the updating formula in Algorithm 1, we know

$$\begin{aligned} \Vert {\varvec{p}}_{t+1}\Vert _2^2&\le \Vert {\varvec{p}} _{t}\Vert _2^2 - \frac{2}{\sqrt{n}}\left( {\varvec{d}} - {\varvec{a}}_tI(r_t> {\varvec{a}}_t^\top {\varvec{p}} _{t})\right) ^\top {\varvec{p}}_t + \frac{1}{n} \left\| {\varvec{d}}- {\varvec{a}}_tI(r_t> {\varvec{a}}_t^\top {\varvec{p}} _{t})\right\| ^2_2\\&\le \Vert {\varvec{p}} _{t}\Vert _2^2 - \frac{2}{\sqrt{n}}\left( {\varvec{d}}- {\varvec{a}}_tI(r_t > {\varvec{a}}_t^\top {\varvec{p}} _{n})\right) ^\top {\varvec{p}}_t + \frac{m({\bar{a}}+{\bar{d}})^2}{n}. \end{aligned}$$

Moving the cross-term to the right-hand side, we have

$$\begin{aligned} \sum _{t=1}^n \left( {\varvec{d}}- {\varvec{a}}_tI(r_t > {\varvec{a}}_t^\top {\varvec{p}} _{t})\right) ^\top {\varvec{p}}_t&\le \sum _{t=1}^n \left( \sqrt{n}\Vert {\varvec{p}} _t\Vert _2^2 - \sqrt{n}\Vert {\varvec{p}} _{t+1} \Vert _2^2 + \frac{m({\bar{a}}+{\bar{d}})^2}{\sqrt{n}}\right) \\&\le m({\bar{a}}+{\bar{d}})^2\sqrt{n}. \end{aligned}$$

Consequently,

$$\begin{aligned} {\varvec{E}} [R_n^* - R_n] \le m({\bar{a}}+{\bar{d}})^2\sqrt{n} \end{aligned}$$

hold for all n and distribution ${\mathcal {P}}\in \Xi .$

For the constraint violation, if we revisit the updating formula, we have

$$\begin{aligned} {\varvec{p}}_{t+1} \ge {\varvec{p}}_t + \frac{1}{\sqrt{n}}\left( {\varvec{a}}_tx_t-{\varvec{d}}\right) \end{aligned}$$

where the inequality is elementwise. Therefore,

$$\begin{aligned} \sum _{t=1}^n {\varvec{a}}_tx_t&\le n{\varvec{d}} + \sum _{t=1}^n \sqrt{n}({\varvec{p}}_{t+1} -{\varvec{p}}_{t}) \\&\le {\varvec{b}} + \sqrt{n}{\varvec{p}}_{n+1} \end{aligned}$$

In the second line, we remove the term involve ${\varvec{p}} _1$ with the algorithm specifying ${\varvec{p}} _1={\varvec{0}}.$ Then with Lemma 1, we have

$$\begin{aligned} {\varvec{E}} \left[ v({\varvec{x}})\right] = {\varvec{E}} \left[ \Vert \left( {\varvec{A}}{\varvec{x}}-{\varvec{b}}\right) ^+ \Vert _2\right] \le \sqrt{n}{\varvec{E}} \Vert {\varvec{p}}_{n+1}\Vert _2 \le \left( {\frac{2{\bar{r}}+m({\bar{a}}+{\bar{d}})^2}{{\underline{d}}}} + m({\bar{a}}+{\bar{d}})\right) \sqrt{n}. \end{aligned}$$

$\square$

Proof of Proposition 2

Proof

Define SLP$(s, {\varvec{b}}_0)$ as the following LP

$$\begin{aligned} \max&\sum _{j=1}^s r_jx_j \\ \text {s.t. }&\sum _{j=1}^s a_{ij}x_j \le \frac{sb_i}{n} + b_{0i} \\&0 \le x_j \le 1\ \text { for } j=1,...,s. \end{aligned}$$

where ${\varvec{b}}_0 = (b_{01},...,b_{0m})$ denotes the constraint relaxation quantity for the scaled LP. Denote the optimal objective value of SLP$(s, {\varvec{b}}_0)$ as $R^*(s, {\varvec{b_0}}).$ Also, denote ${\varvec{x}}({\varvec{p}}) = (x_1({\varvec{p}}),...,x_n({\varvec{p}}))$ and $x_j({\varvec{p}})=I(r_j>{\varvec{a}}_j^\top {\varvec{p}}).$ It denotes the decision variables we obtain with a dual price ${\varvec{p}}.$

We prove the following three results:

• (i) The following bounds hold for $R_n^*,$

  $$\begin{aligned} \sum _{j=1}^n r_jx_j({\varvec{p}}_n^*)-m{\bar{r}}+ \le R_n^* \le \sum _{j=1}^n r_jx_j({\varvec{p}}_n^*) + m{\bar{r}}. \end{aligned}$$

• (ii) When $s\ge \max \{16{\bar{a}}^2,\exp {\{16{\bar{a}}^2\}},e\},$ then the ${\varvec{x}}({\varvec{p}}_n^*)$ is a feasible solution to SLP$\left( s, \left( \sqrt{s}\log s+\frac{ms}{n}\right) {\varvec{1}}\right)$ with probability no less than $1-\frac{m}{s}$.

• (iii) The following inequality holds for the optimal objective values to the scaled LP and its relaxation

  $$\begin{aligned} R_s^* \ge R^*\left( s, \left( \sqrt{s}\log s+\frac{ms}{n}\right) {\varvec{1}} \right) - \frac{{\bar{r}}\sqrt{s}\log s }{{\underline{d}}}-\frac{{\bar{r}}sm}{{\underline{d}}n}. \end{aligned}$$

For part (i), this inequality replace the optimal value with bounds containing the objective values obtained by adopting optimal dual solution. The left-hand side of the inequality comes from the complementarity condition while the right-hand side can be shown from Lemma 2.

For part (ii), the motivation to introduce a relaxed form of the scaled LP is to ensure the feasibility of ${\varvec{p}}_n^*$. The key idea for the proof is to utilize the feasibility of ${\varvec{p}}_n^*$ for (P-LP). To see that, let $\alpha _{ij}=a_{ij}I(r_j>{\varvec{a}}_j^T{\varvec{p}}^*)$ and

$$\begin{aligned} \begin{aligned} c_\alpha&= \max \limits _{i,j} \alpha _{ij} - \min \limits _{i,j}\alpha _{ij}\le 2{\bar{a}},\\ {\bar{\alpha }}_i&= \frac{1}{n}\sum \limits _{j=1}^{n} \alpha _{ij} =\frac{1}{n}\sum \limits _{j=1}^{n} a_{ij}x_t({\varvec{p}}_n^*) \le d_i+\frac{m}{n}, \\ \sigma ^2_i&= \frac{1}{n}\sum \limits _{j=1}^{n}(\alpha _{ij}-{\bar{\alpha }}_i)^2 \le 4{\bar{a}}^2. \end{aligned} \end{aligned}$$

(2)

Here the first and third inequality comes from the bounds on $a_{ij}$’s, and the second one comes from the feasibility of the optimal solution for (P-LP).

Then, when $k>\max \{16{\bar{a}}^2,\exp {\{16{\bar{a}}^2\}},e\}$, by applying Hoeffding-Bernstein’s Inequality

$$\begin{aligned} \begin{aligned} {\mathbb {P}} \left( \sum \limits _{j=1}^k \alpha _{ij} - kd_i-\frac{km}{n} \ge \sqrt{k}\log k \right)&\overset{(e)}{\le } {\mathbb {P}} \left( \sum \limits _{j=1}^k \alpha _{ij} - k{\bar{\alpha }}_i \ge \sqrt{k}\log k \right) \\&\overset{(f)}{\le } \exp \left( -\frac{k\log ^2 k}{8k{\bar{a}}^2+2{\bar{a}}\sqrt{k}\log k} \right) \\&\overset{(g)}{\le } \frac{1}{k} \end{aligned} \end{aligned}$$

for $i=1,...,m$. Here inequality (e) comes from (2), (f) comes from applying Lemma 5, and (g) holds when $k>\max \{16{\bar{a}}^2,\exp {\{16{\bar{a}}^2\}},e\}$.

Let event

$$\begin{aligned} E_i=\left\{ \sum _{j=1}^s \alpha _{ij}-sd_i-\frac{sm}{n}<\sqrt{s}\log s\right\} \end{aligned}$$

and $E=\bigcap \nolimits _{i=1}^{m}E_i$. The above derivation denotes the case where the i-th constraint is satisfied for the LP with constraint relaxation. By applying union bound, we obtain ${\mathbb {P}}(E)\ge 1-\frac{m}{s}$ and it completes the proof of part (ii).

For part(iii), denote the dual optimal solution to SLP$\left( s, \left( \sqrt{s}\log s+\frac{ms}{n}\right) {\varvec{1}}\right)$ as $\tilde{{\varvec{p}}}_s.$

$$\begin{aligned} \begin{aligned} R^*\left( s, \left( \sqrt{s}\log s+\frac{ms}{n}\right) {\varvec{1}} \right)&= s\left( {\varvec{d}}+\frac{\log s}{\sqrt{s}}{\varvec{1}}+\frac{m}{n}{\varvec{1}}\right) ^\top \tilde{{\varvec{p}}}^*_{s} + \sum _{j=1}^s\left( r_j-{\varvec{a}}_j^\top \tilde{{\varvec{p}}}^*_{s}\right) ^{+}\\&\le s\left( {\varvec{d}}+\frac{\log s}{\sqrt{s}}{\varvec{1}}+\frac{m}{n}{\varvec{1}}\right) ^\top {{\varvec{p}}}^*_{s} + \sum _{j=1}^s\left( r_j-{\varvec{a}}_j^\top {{\varvec{p}}}^*_{s}\right) ^{+} \\&\le \frac{{\bar{r}}\sqrt{s}\log s}{{\underline{d}}} + \frac{{\bar{r}}sm}{{\underline{d}}n} + R_s^*. \end{aligned} \end{aligned}$$

where the first inequality comes from dual optimality of $\tilde{{\varvec{p}}}_s^*$ and the second inequality comes from the upper bound of $\Vert {\varvec{p}}_s^*\Vert$ and the duality of the scaled LP $R_s^*$. Therefore,

$$\begin{aligned} R_s^* \ge R^*\left( s, \left( \sqrt{s}\log s+\frac{ms}{n}\right) {\varvec{1}} \right) - \frac{{\bar{r}}\sqrt{s}\log s }{{\underline{d}}} - \frac{{\bar{r}}sm}{{\underline{d}}n}. \end{aligned}$$

Finally, we complete the proof with the help of the above three results.

$$\begin{aligned} \frac{1}{s}{\varvec{E}} \left[ {\mathbb {I}}_ER_s^*\right]&\ge \frac{1}{s}{\varvec{E}} \left[ {\mathbb {I}}_ER^*\left( s, \left( \sqrt{s}\log s+\frac{ms}{n}\right) \cdot {\varvec{1}} \right) \right] - \frac{{\bar{r}}\log s }{{\underline{d}}\sqrt{s}}-\frac{{\bar{r}}m}{{\underline{d}}n} \\&\ge \frac{1}{s}{\varvec{E}} \left[ {\mathbb {I}}_E\sum _{j=1}^sr_jx_j({\varvec{p}}^*)\right] - \frac{{\bar{r}}\log s }{{\underline{d}}\sqrt{s}}-\frac{{\bar{r}}m}{{\underline{d}}n} - \frac{m{\bar{r}}}{s} \end{aligned}$$

where ${\mathbb {I}}_E$ denotes an indicator function for event E. The first line comes from applying part (iii) while the second line comes from the feasibility of ${\varvec{p}}^*$ on event E. Then,

$$\begin{aligned} \frac{1}{s}{\varvec{E}} \left[ R_s^*\right]&\ge \frac{1}{s}{\varvec{E}} \left[ \sum _{j=1}^sr_jx_j({\varvec{p}}^*)\right] - \frac{{\bar{r}}\log s }{{\underline{d}}\sqrt{s}} -\frac{{\bar{r}}m}{{\underline{d}}n}-\frac{2m{\bar{r}}}{s} \\&= \frac{1}{n}{\varvec{E}} \left[ \sum _{j=1}^n r_jx_j({\varvec{p}}^*)\right] - \frac{{\bar{r}}\log s }{{\underline{d}}\sqrt{s}} -\frac{{\bar{r}}m}{{\underline{d}}n}-\frac{2m{\bar{r}}}{s} \\&\ge \frac{1}{n} R_n^* - \frac{{\bar{r}}\log s }{{\underline{d}}\sqrt{s}} - \frac{2m{\bar{r}}}{s} - \frac{{\bar{r}}m}{n} -\frac{{\bar{r}}m}{{\underline{d}}n} \end{aligned}$$

where the first line comes from part (ii) – the probability bound on event E, the second line comes from the symmetry of the random permutation probability space, and the third line comes from part (i). We complete the proof. $\square$

Proof of Theorem 3

Proof

For the regret bound,

$$\begin{aligned} R_n^* - {\varvec{E}} \left[ R_n\right]&= R_n^* - \sum _{t=1}^n {\varvec{E}} \left[ r_t x_t\right] \end{aligned}$$

where $x_t$’s are specified according to Algorithm 1. Then

$$\begin{aligned} R_n^* - {\varvec{E}} \left[ R_n\right]&= R_n^* - \sum _{t=1}^n \frac{1}{t} {\varvec{E}} \left[ R_t^*\right] + \sum _{t=1}^n \frac{1}{t} {\varvec{E}} \left[ R_t^*\right] - \sum _{t=1}^n {\varvec{E}} \left[ r_t x_t\right] \nonumber \\&= \sum _{t=1}^n\left( \frac{1}{n}R_n^*-\frac{1}{t} {\varvec{E}} \left[ R_t^*\right] \right) + \sum _{t=1}^n{\varvec{E}} \left[ \frac{1}{n+1-t}{\tilde{R}}_{n-t+1}^* - r_tx_t\right] \end{aligned}$$

(3)

where ${\tilde{R}}_{n-t+1}^*$ is defined as the optimal value of the following LP

$$\begin{aligned} \max&\sum _{j=t}^n r_j x_j \\ \text {s.t. }&\sum _{j=t}^n a_{ij}x_j \le \frac{(n-t+1)b_i}{n} \\&0 \le x_j \le 1\ \text { for } j=1,...,m. \end{aligned}$$

For the first part of (3), we can apply Proposition 2. Meanwhile, the analysis of the second part takes a similar form as (1) in the analysis of the previous stochastic input model. Specifically,

$$\begin{aligned} {\varvec{E}} \left[ \frac{1}{n+1-t}{\tilde{R}}_{n-t+1}^* - r_tx_t\right] \le \left( {\varvec{d}}- {\varvec{a}}_tI(r_t > {\varvec{a}}_t^\top {\varvec{p}} _{t})\right) ^\top {\varvec{p}}_t. \end{aligned}$$

Similar to the stochastic input model,

$$\begin{aligned} \Vert {\varvec{p}}_{t+1}\Vert _2^2&\le \Vert {\varvec{p}} _{t}\Vert _2^2 - \frac{2}{\sqrt{n}}\left( {\varvec{d}}- {\varvec{a}}_tI(r_t> {\varvec{a}}_t^\top {\varvec{p}} _{t})\right) ^\top {\varvec{p}}_t + \frac{1}{n} \left\| {\varvec{d}}- {\varvec{a}}_tI(r_t> {\varvec{a}}_t^\top {\varvec{p}} _{t})\right\| ^2_2\\&\le \Vert {\varvec{p}} _{t}\Vert _2^2 - \frac{2}{\sqrt{n}}\left( {\varvec{d}}- {\varvec{a}}_tI(r_t > {\varvec{a}}_t^\top {\varvec{p}} _{t})\right) ^\top {\varvec{p}}_t + \frac{m({\bar{a}}+{\bar{d}})^2}{n}. \end{aligned}$$

Thus, we have

$$\begin{aligned} \begin{aligned} \sum \limits _{t=1}^{n}{\varvec{E}} \left[ \left( {\varvec{d}}- {\varvec{a}}_tI(r_t > {\varvec{a}}_t^\top {\varvec{p}} _{t})\right) ^\top {\varvec{p}}_t\right]&\le \sum \limits _{t=1}^{n}{\varvec{E}} \left[ \sqrt{n}(\Vert {\varvec{p}}_t\Vert ^2_2 -\Vert {\varvec{p}}_{t+1}\Vert ^2_2)\right] +\sum \limits _{t=1}^{n}\frac{m({\bar{a}} +{\bar{d}})^2}{\sqrt{n}}\\&\le m({\bar{a}}+{\bar{d}})^2\sqrt{n}. \end{aligned} \end{aligned}$$

Combine two parts above, finally we have

$$\begin{aligned} \begin{aligned} R_n^* - {\varvec{E}}[R_n(\pi )]&\le m{\bar{r}} + \frac{{\bar{r}}\log n\sqrt{n}}{{\underline{d}}} + m{\bar{r}}\log n + \frac{\max \{16{\bar{a}}^2,\exp {\{16{\bar{a}}^2\}},e\}{\bar{r}}}{n}\\&\quad +m({\bar{a}} +{\bar{d}})^2\sqrt{n}= O((m+\log n)\sqrt{n}) \end{aligned} \end{aligned}$$

Thus, we complete the proof for the regret. The proof for the constraint violation part follows exactly the same way as the stochastic input model. $\square$

Proof for Theorem 4

Proof

The proof follows mostly the proof of Theorem 1 and Theorem 3. We only highlight the difference here. First, the sample average approximation form of the dual problem takes a slightly different form but it is still convex in ${\varvec{p}}.$

$$\begin{aligned} \min _{{\varvec{p}}}&f_n({\varvec{p}}) = {\varvec{d}}^\top {\varvec{p}} + \frac{1}{n} \sum _{j=1}^n \left( \max _{s=1,...,k}\left\{ r_{js}-{\varvec{a}}_{js}^\top {\varvec{p}}\right\} \right) ^+ \nonumber \\ \text {s.t. }&{\varvec{p}}\ge {\varvec{0}}. \end{aligned}$$

(multi-D-SAA)

The updating formula for ${\varvec{p}}_t$ is different but we can achieve the same relation between ${\varvec{p}}_t$ and ${\varvec{p}}_{t+1}.$ At time t, if $\max \limits _{l=1,...,k}\left\{ r_{jl}-{\varvec{a}}_{jl}^\top {\varvec{p}}\right\} >0$, we have

$$\begin{aligned} \Vert {\varvec{p}}_{t+1}\Vert ^2_2&\le \left\| {\varvec{p}}_{t}+\frac{1}{\sqrt{n}}\left( {\varvec{A}}_tx_t-{\varvec{d}} \right) \right\| ^2_2 \\&= \left\| {\varvec{p}}_{t}+\frac{1}{\sqrt{n}}\left( {\varvec{a}}_{tl_t}-{\varvec{d}} \right) \right\| ^2_2 \\&= \Vert {\varvec{p}}_{t}\Vert _2^2 + \frac{1}{n} \Vert {\varvec{a}}_{tl_t}x_t - {\varvec{d}}\Vert _2^2 + \frac{2}{\sqrt{n}} ({\varvec{a}}_{tl_t} x_t - {\varvec{d}})^\top {\varvec{p}} _{t}\\&\le \Vert {\varvec{p}}_{t}\Vert _2^2 + \frac{m({\bar{a}}+{\bar{d}})^2}{n} + \frac{2}{\sqrt{n}} {\varvec{a}}_t^\top {\varvec{p}} _{t} x_t - \frac{2}{\sqrt{n}}{\varvec{d}}^\top {\varvec{p}}_{t}\\&\le \Vert {\varvec{p}}_{t}\Vert _2^2 + \frac{m({\bar{a}}+{\bar{d}})^2}{n} + \frac{2{\bar{r}}}{\sqrt{n}}- \frac{2}{\sqrt{n}}{\varvec{d}}^\top {\varvec{p}}_{t}, \end{aligned}$$

while if $\max \limits _{l=1,...,k}\left\{ r_{jl}-{\varvec{a}}_{jl}^\top {\varvec{p}}\right\} \le 0$, we have

$$\begin{aligned} \Vert {\varvec{p}}_{t+1}\Vert ^2_2&\le \left\| {\varvec{p}}_{t}+\frac{1}{\sqrt{n}}\left( {\varvec{A}}_tx_t-{\varvec{d}} \right) \right\| ^2_2 \\&= \left\| {\varvec{p}}_{t}-\frac{1}{\sqrt{n}}{\varvec{d}}\right\| ^2_2 \\&\le \Vert {\varvec{p}}_{t}\Vert _2^2 + \frac{m({\bar{a}}+{\bar{d}})^2}{n} - \frac{2}{\sqrt{n}}{\varvec{d}}^\top {\varvec{p}}_{t}. \end{aligned}$$

Combining those two parts, we obtain

$$\begin{aligned} \Vert {\varvec{p}}_{t+1}\Vert ^2_2&\le \Vert {\varvec{p}}_{t}\Vert _2^2 + \frac{m({\bar{a}}+{\bar{d}})^2}{n} + \frac{2{\bar{r}}}{\sqrt{n}}- \frac{2}{\sqrt{n}}{\varvec{d}}^\top {\varvec{p}}_{n}, \end{aligned}$$

which is the same formula as the one-dimensional setting. With the above results, the rest of the proof simply follows the same approach as the one-dimensional case.

$\square$

Proofs for results in Section 6

1.1 Proof for Lemma 3

Proof

We have

$$\begin{aligned} \begin{aligned} {\varvec{E}}\left[ \sup \limits _{f\in {\mathcal {F}}}\left| {\bar{f}}({\varvec{Z}}_t) \!-\! {\bar{f}}(\tilde{{\varvec{Z}}}_{t'}) \right| \Big \vert {\varvec{Z}}_n\!\right]&= {\varvec{E}}\left[ \sup \limits _{f\in {\mathcal {F}}}\left| {\varvec{E}}\left[ {\bar{f}}({\varvec{Z}}_{t-s})\vert {\varvec{Z}}_t\right] - {\varvec{E}}\left[ {\bar{f}}(\tilde{{\varvec{Z}}}_s)\vert \tilde{{\varvec{Z}}}_{t'}\right] \right| \Big \vert {\varvec{Z}}_n \right] \\&\le \! {\varvec{E}}\left[ \sup \limits _{f\in {\mathcal {F}}}\left| {\bar{f}}({\varvec{Z}}_{t-s}) \!-\! {\bar{f}}(\tilde{{\varvec{Z}}}_s) \right| \Big \vert {\varvec{Z}}_n \!\right] \!=\! {\varvec{E}} \left[ {Q}_{t,s}({\mathcal {F}},{\varvec{Z}}_t)\big \vert {\varvec{Z}}_n\right] . \end{aligned} \end{aligned}$$

For the first line, on the right-hand side, the two inner expectations are taken with respect to a uniform random sampling on ${\varvec{Z}}_t$ and $\tilde{{\varvec{Z}}}_{t'}$ respectively. Specifically, ${\varvec{Z}}_{t-s}$ (or $\tilde{{\varvec{Z}}}_{s}$) can be viewed as a random sampled subset from ${\varvec{Z}}_t$ (or $\tilde{{\varvec{Z}}}_{t'}$). For the second line, the first part comes from Jensen’s inequality and the expectation in the second part is taken with respect to the random sampling of ${\varvec{Z}}_t$ from ${\varvec{Z}}_n.$ $\square$

1.2 Proof for Lemma 4

Proof

For the first inequality, we refer to Theorem 3 in [44]. For the second inequality, it is a direct application of Massart’s Lemma (See Lemma 26.8 of [54]). $\square$

1.3 Proof for Proposition 5

Proof

Let ${\mathcal {F}}_{{\varvec{p}}} = \left\{ f_{{\varvec{p}}}: f_{{\varvec{p}}}(r,{\varvec{a}}) = rI\left( r>{\varvec{a}}^T{\varvec{p}} \right) \right\}$, ${\varvec{Z}}_t = \{(r_1,{\varvec{a}}_1),...,(r_t,{\varvec{a}}_t)\},$ and ${\tilde{Z}}_{n-t} = \{(r_{t+1},{\varvec{a}}_{t+1}),...,(r_n,{\varvec{a}}_n)\}.$ Also, we assume $n-t>t$ without loss of generality. Then,

$$\begin{aligned}&{\varvec{E}} \left[ \left| \frac{1}{n-t}\sum _{j=t+1}^n r_{j}I(r_j>{\varvec{a}}_j^\top {\varvec{p}}) - \frac{1}{t}\sum _{j=1}^t r_{j}I(r_j>{\varvec{a}}_j^\top {\varvec{p}}) \right| \right] \\&\quad \le {\varvec{E}} \left[ \sup \limits _{f\in {\mathcal {F}}_{{\varvec{p}}}}\left| {\bar{f}}({\varvec{Z}}_t) - {\bar{f}}(\tilde{{\varvec{Z}}}_{n-t}) \right| \Big \vert {\varvec{Z}}_n = {\mathcal {D}}\right] \\&\quad \le {\varvec{E}}\left[ Q_{t,\lfloor t/2\rfloor }({\mathcal {F}},{\varvec{Z}}_{t})\Big \vert {\varvec{Z}}_n \right] \\&\quad \le \frac{4{\bar{r}}}{t} + \frac{2\sqrt{2{\bar{r}}^2m\log n}}{\sqrt{t}}. \end{aligned}$$

Here the first line comes from taking maximum over ${\mathcal {F}}_{{\varvec{p}}}$, the second line comes from Lemma 3 and the third line comes from lemma 4.

Similarly, we can show that the inequality on $a_{ij}$’s holds. Thus the proof is completed. $\square$

1.4 Proof for Theorem 6

Proof

At time $t+1,$

$$\begin{aligned} \begin{aligned} {\varvec{E}}\left[ r_{t+1}x_{t+1}\right]&= {\varvec{E}}\left[ r_{t+1}I(r_{t+1}>{\varvec{a}}_{t+1}^\top {\varvec{p}} _{t+1})\right] \\&=\frac{1}{n-t} {\varvec{E}}\left[ \sum _{j=t+1}^n r_jI(r_{j}>{\varvec{a}}_j{\varvec{p}} _{t+1})\right] \\&\ge \frac{1}{t}{\varvec{E}}\left[ \sum _{j=1}^t r_jI(r_{j}\!>\!{\varvec{a}}_j{\varvec{p}} _{t+1}) \!\right] - \frac{4{\bar{r}}}{\sqrt{\min \{t,n\!-\!t\}}} -\frac{2\sqrt{2{\bar{r}}^2m\log n}}{\sqrt{\min \{t,n-t\}}}, \end{aligned} \end{aligned}$$

where the expectation is taken with respect to the random permutation. The first line comes from the algorithm design, the second line comes from the symmetry over the last $n-t$ terms, and the last line comes from the application of Proposition 5. To relate the first term in the last line with the offline optimal $R_n^*,$ we utilize Proposition 2. Then the optimality gap of Algorithm 3 is as follows,

$$\begin{aligned} \begin{aligned}&R_n^* - {\varvec{E}} \left[ \sum _{t=1}^n r_t x_t\right] = R_n^* - \sum _{t=1}^n {\varvec{E}} \left[ r_t x_t\right] \\&\quad \le R_n^* - \sum \limits _{t=2}^{n} \left( \frac{1}{t}{\varvec{E}} \left[ \sum _{j=1}^t r_jI(r_{j}>{\varvec{a}}_j{\varvec{p}} _t) \right] - \frac{4{\bar{r}}+2\sqrt{2{\bar{r}}^2m\log n}}{\sqrt{\min \{t,n-t\}}} \right) \\&\quad \le m{\bar{r}} + \frac{{\bar{r}}}{{\underline{d}}}\sqrt{n}\log n + m{\bar{r}}\log n + {\bar{r}}\max \{16{\bar{a}}^2,e^{16{\bar{a}}^2},e\} \\&\quad \quad + \left( 8{\bar{r}}+ 4\sqrt{2{\bar{r}}^2m\log n} \right) \sqrt{n}\\&\quad = O(\sqrt{mn}\log n) \end{aligned} \end{aligned}$$

where the third line comes from that $r_1x_1\ge 0$ because $x_1=1$ if and only if $r_1>0$, and the last line comes from an application of Proposition 2. Next, we analyze the constraint; again, from Proposition 5, we know

$$\begin{aligned}&\frac{1}{n-t}{\varvec{E}} \left[ \sum _{j=t+1}^n a_{ij}I(r_j>{\varvec{a}}_j^\top {\varvec{p}} _t)\right] \\&\quad \le {\varvec{E}} \left[ \frac{1}{t}\sum _{j=1}^t a_{ij}I(r_j>{\varvec{a}}_j^\top {\varvec{p}}_{t}) \right] + \frac{4{\bar{a}}}{\sqrt{\min \{t,n-t\}}} +\frac{2\sqrt{2{\bar{a}}^2m\log n}}{\sqrt{\min \{t,n-t\}}}\\&\quad \le d_i + {\bar{a}}{\varvec{E}}\left[ \frac{1}{t}\sum \limits _{j=1}^{t}I(r_j ={\varvec{a}}_j^\top {\varvec{p}}_{t}) \right] +\frac{6\sqrt{2{\bar{a}}^2m\log n}}{\sqrt{\min \{t,n-t\}}}\\&\quad \le d_i+\frac{m}{t}+\frac{6\sqrt{2{\bar{a}}^2m\log n}}{\sqrt{\min \{t,n-t\}}}, \end{aligned}$$

where the first inequality comes from Proposition 5, the second inequality comes from the feasibility of the boundedness assumption and the scaled LP solved at time t, and the last inequality comes from the general position assumption that there are at most m columns such that $r_j={\varvec{a}}_j^\top {\varvec{p}}_{t}$ for $j=1,...,n$ and any fixed $t=1,...,T$. Due to the symmetry of the random permutation,

$$\begin{aligned} {\varvec{E}} \left[ a_{i,t+1}I(r_{t+1}>{\varvec{a}}_{t+1}^\top {\varvec{p}}_{t+1})\right]&\le d_i+\frac{m}{t} +\frac{6\sqrt{2{\bar{a}}^2m\log n}}{\sqrt{\min \{t,n-t\}}}. \end{aligned}$$

Summing up the inequality, we have

$$\begin{aligned} {\varvec{E}} [{\varvec{A}}{\varvec{x}}-{\varvec{b}}] \le O(\sqrt{mn}\log n+m\log n), \end{aligned}$$

which implies

$$\begin{aligned} {\varvec{E}}[v({\varvec{x}})]={\varvec{E}} [\Vert {\varvec{A}}{\varvec{x}}-{\varvec{b}}\Vert _2] \le O(m\sqrt{n}\log n+m^{3/2}\log n), \end{aligned}$$

$\square$

Proof of Theorem 7

Proof

At time t, the optimal solution to the scaled LP is $\tilde{{\varvec{x}}}^{(t)}=({\tilde{x}}_1^{(t)},...,{\tilde{x}}_t^{(t)})$. We have

$$\begin{aligned} \begin{aligned} {\varvec{E}}\left[ r_tx_t\right]&= {\varvec{E}}\left[ r_t{\tilde{x}}_t^{(t)}\right] \\&= \frac{1}{t}{\varvec{E}}\left[ \sum _{j=1}^tr_s {\tilde{x}}^{(t)}_j\right] . \end{aligned} \end{aligned}$$

Then, for the objective,

$$\begin{aligned} \begin{aligned} R_n^* - \sum _{t=1}^n {\varvec{E}} \left[ r_t x_t\right]&= R_n^* - \sum \limits _{t=1}^{n} \frac{1}{t}{\varvec{E}}\left[ \sum _{j=1}^tr_s {\tilde{x}}^{(t)}_j\right] \\&\le m{\bar{r}} + \frac{{\bar{r}}}{{\underline{d}}}\log n\sqrt{n} + m{\bar{r}}\log n + {\bar{r}}\max \{16{\bar{a}}^2,e^{16{\bar{a}}^2},e\}. \end{aligned} \end{aligned}$$

where the second line comes from an application of Proposition 2. Then, we analyze the constraint violation. From the construction of the algorithm, we have that ${\varvec{E}}[a_{it}x_t]\le d_i$. Let

$$\begin{aligned} A_{it} = a_{it}x_t - d_i \end{aligned}$$

and then we know

$$\begin{aligned} M_{it} = \sum _{j=n-t+1}^{n} A_{ij} \end{aligned}$$

is a supermartingale with $\vert A_{ij}\vert \le {\bar{a}}+{\bar{d}}$. Then if we apply Hoeffding’s lemma for supermartingale, we have

$$\begin{aligned} \begin{aligned} {\mathbb {P}}\left( M_{in}\ge 2({\bar{a}}+{\bar{d}})\sqrt{n}\log n \right)&\le \exp \left\{ - \frac{2({\bar{a}}+{\bar{d}})^2n\log ^2 n}{n({\bar{a}}+{\bar{d}})^2} \right\} \\&\le \exp \{-2\log ^2 n\} \le \frac{1}{n}, \end{aligned} \end{aligned}$$

when $n>3$. Thus,

$$\begin{aligned} \begin{aligned} {\varvec{E}} \left[ \left( \sum \limits _{t=1}^{n}a_{it}x_t-d_i\right) ^+ \right]&= {\varvec{E}}\left[ (M_{in})^+ \right] \\&\le 2({\bar{a}}+{\bar{d}})\sqrt{n}\log n {\mathbb {P}}\left( M_{in}< 2({\bar{a}}+{\bar{d}})\sqrt{n}\log n \right) \\&\ \ \ \ + {\bar{a}}n {\mathbb {P}}\left( M_{in}\ge 2({\bar{a}}+{\bar{d}})\sqrt{n}\log n \right) \\&\le 2({\bar{a}}+{\bar{d}})\sqrt{n}\log n+{\bar{a}}\\ {\varvec{E}} \left[ v({\varvec{x}}) \right]&\le 2({\bar{a}}+{\bar{d}})\sqrt{mn}\log n + {\bar{a}}\sqrt{m}. \end{aligned} \end{aligned}$$

$\square$

Proof of Theorem 8

Proof

For the upper bound of v, lemma 1 and

$$\begin{aligned} \sum \limits _{t=1}^{n}a_{it}x_t-b_i \le \sqrt{n}p_{n+1,i} \le \sqrt{n}\Vert {\varvec{p}}_{n+1}\Vert _{2}, \end{aligned}$$

can give the result.

For the fesibility, first, we prove that if the initial solution is infeasible for the i-th constraint, i.e., $\sum \nolimits _{t=1}^{n} a_{it}x_{t}>nd_i$, then

$$\begin{aligned} \sum \limits _{t\in S_0}a_{it} \ge v\sqrt{n}\log n \end{aligned}$$

(4)

with high probability. According to the definition of v, (4) says the corresponding ${\hat{x}}_t$ is feasible. To start, (4) holds with probability 1 if $\vert S_0\vert =n_+$ which implies $S_0=S_+$. Otherwise, the infeasibility indicates $n_+\ge \max \{1,frac{{\underline{d}}n}{{\bar{a}}}\}$, and by definition $n_+ \le n.$

This implies bounds on the cardinality of $S_0$,

$$\begin{aligned} \vert S_0\vert \ge \frac{2v\sqrt{n}\log n}{{\bar{a}}} \end{aligned}$$

and

$$\begin{aligned} \vert S_0\vert \le \frac{2v\sqrt{n}\log n}{{\underline{d}}}+1 \le \frac{3v\sqrt{n}\log n}{{\underline{d}}}. \end{aligned}$$

Consequently,

$$\begin{aligned} \frac{\vert S_0\vert }{n_+}\sum \limits _{t\in S_+}a_{it} \ge \frac{2v\log n}{{\underline{d}}\sqrt{n}}n{\underline{d}} \ge 2v\sqrt{n}\log n. \end{aligned}$$

Then, since $n\ge \left( \frac{6{\bar{a}}}{{\underline{d}}}\right) ^4$

$$\begin{aligned} \begin{aligned} {\mathbb {P}} \left( \sum \limits _{t\in S_0}a_{it} < v\sqrt{n}\log n \right)&\le {\mathbb {P}} \left( \sum \limits _{t\in S_0} a_{it} - \frac{\vert S_0\vert }{n_+} \sum \limits _{t\in S_+} a_{it} \le -v\sqrt{n}\log n \right) \\&\le \exp \left\{ -\frac{2v^2n^2\log ^2n}{4{\bar{a}}^2\vert S_0\vert ^2} \right\} \\&\le \exp \left\{ -\frac{n{\underline{d}}^2}{18{\bar{a}}^2} \right\} \le \frac{1}{n^2}. \end{aligned} \end{aligned}$$

Moreover, for any i s.t. $\sum \nolimits _{t=1}^n a_{it} < \frac{n{\underline{d}}}{2}$, since $n>16$ and $\sqrt{n}>\frac{12{\bar{a}}({\bar{r}}+({\bar{a}}+{\underline{d}})^2m)\log n}{{\underline{d}}^2}$, we have

$$\begin{aligned} \sum \limits _{t\in S_+\backslash S_0} a_{it} \le \frac{n{\underline{d}}}{2} + {\bar{a}}\vert S_0\vert \le n{\underline{d}}. \end{aligned}$$

Then, for any i s.t. $\frac{n{\underline{d}}}{2}\le \sum \nolimits _{t=1}^{n}a_{it}\le {n{\underline{d}}}$. Similarly, we can find that

$$\begin{aligned} \frac{\vert S_0\vert }{n_+} \sum \limits _{t\in S_+}a_{it}&\ge \frac{v\log n}{{\underline{d}}\sqrt{n}}n{\underline{d}} \ge v\sqrt{n}\log n,\\ {\mathbb {P}} \left( \sum \limits _{t\in S_0}a_{it} < 0 \right)&\le {\mathbb {P}} \left( \sum \limits _{t\in S_0} a_{it} - \frac{\vert S_0\vert }{n_+} \sum \limits _{t\in S_+} a_{it} \le -v\sqrt{n}\log n \right) \\&\le \exp \left\{ -\frac{2v^2n^2\log ^2 n}{4{\bar{a}}^2\vert S_0\vert ^2} \right\} \\&\le \exp \left\{ -\frac{n{\underline{d}}^2}{18{\bar{a}}^2} \right\} \le \frac{1}{n^2}. \end{aligned}$$

Combining three parts above, we have that with probability at least $1-\frac{2m}{n^2}$, the modified solution is feasible. Given the fact that the modified solution change the original solution for at most $O(\sqrt{n}\log n)$ entries, the modified solution can achieve $O((m+\log n)\sqrt{n})$ regret. $\square$