Appendix A: Proof of Theorem 1
Proof
Let us prove that the Hessian matrix H associated with the Markowitz portfolio given in Eq. (14) is strictly negative definite, i.e., we prove that for all \(c\in \bar{C}_{adm}\), \(\upsilon \in {\varUpsilon } _{adm}\) and some \(\mu ,\delta >0\)
$$\begin{aligned} \begin{bmatrix} \frac{\partial ^{2}}{\partial c^{2}}{\varPhi } _{\mu ,\delta }\left( \upsilon ,c\right)&\frac{\partial ^{2}}{\partial \upsilon \partial c}{\varPhi } _{\mu ,\delta }\left( \upsilon ,c\right) \\ \frac{\partial ^{2}}{\partial c\partial \upsilon }{\varPhi } _{\mu ,\delta }\left( \upsilon ,c\right)&\frac{\partial ^{2}}{\partial \upsilon ^{2}}{\varPhi } _{\mu ,\delta }\left( \upsilon ,c\right) \end{bmatrix} <0 \end{aligned}$$
(34)
Defining
$$\begin{aligned} V\left( \upsilon \right) :=\frac{1}{2}\dfrac{\partial }{\partial \upsilon } \bar{\upsilon }^{\intercal }\left( \upsilon \right) ={\mathrm {diag}}\left( \upsilon _{1|1},\ldots ,\upsilon _{1|M};\upsilon _{2|1},\ldots ,\upsilon _{2|M};\ldots ;\upsilon _{N|1},\ldots ,\upsilon _{N|M}\right) \end{aligned}$$
and noting that
$$\begin{aligned} \dfrac{{ \partial }}{{ \partial c}}\left( \left\| { c} \right\| ^{2}{ +}\left\| { \upsilon }\right\| ^{2}\right) ^{2}{ }&= {} 4\left( \left\| c\right\| ^{2}{ +} \left\| { \upsilon }\right\| ^{2}\right) { c} \\ \dfrac{{ \partial }^{2}}{{ \partial c}^{2}}\left( \left\| { c}\right\| ^{2}{ +}\left\| { \upsilon }\right\| ^{2}\right) ^{2}{ }&= {} 4\dfrac{{ \partial }}{{ \partial c}} \left[ \left( \left\| { c}\right\| ^{2}{ +}\left\| { \upsilon }\right\| ^{2}\right) { c}\right] { =4} \left( \left\| { c}\right\| ^{2}{ +}\left\| { \upsilon }\right\| ^{2}\right) { I}_{{ NM\times NM}}{ +8cc}^{\intercal } \\ \dfrac{{ \partial }}{{ \partial \upsilon }}\left( \left\| { c}\right\| ^{2}{ +}\left\| { \upsilon }\right\| ^{2}\right) ^{2}&= {} { 4}\left( \left\| { c}\right\| ^{2} { +}\left\| { \upsilon }\right\| ^{2}\right) { \upsilon } \\ \dfrac{{ \partial }^{2}}{{ \partial \upsilon }^{2}}\left( \left\| { c}\right\| ^{2}{ +}\left\| { \upsilon } \right\| ^{2}\right) ^{2}{}&= {} 4\dfrac{{ \partial }}{{ \partial \upsilon }}\left[ \left( \left\| c\right\| ^{2}{ +} \left\| \upsilon \right\| ^{2}\right) { \upsilon }\right] { =4}\left( \left\| c\right\| ^{2}{ +}\left\| \upsilon \right\| ^{2}\right) { I}_{NM\times NM}{ +8\upsilon \upsilon } ^{\intercal } \end{aligned}$$
by (14) we have
$$\begin{aligned} \dfrac{\partial }{\partial \upsilon }{\varPhi } _{\mu ,\delta }\left( \upsilon ,c\right)&= {} \mu \left[ \tilde{W}c+\xi \left( \upsilon ^{\intercal }\tilde{W} c\right) \tilde{W}c-\xi V\left( \upsilon \right) \bar{W}c\right] \nonumber \\&\quad -\, 2\left[ \upsilon ^{\intercal }e-\upsilon ^{+}\right] _{+}e-\left[ c^{\intercal }{\varPsi } \upsilon -b_{ineq}\right] _{+}{\varPsi } c-4\delta \left( \left\| c\right\| ^{2}+\left\| \upsilon \right\| ^{2}\right) \upsilon \end{aligned}$$
(35)
$$\begin{aligned} \dfrac{\partial }{\partial c}{\varPhi } _{\mu ,\delta }\left( \upsilon ,c\right)&= {} \mu \left[ \tilde{W}\upsilon +\xi \left( \upsilon ^{\intercal }\tilde{W} c\right) \tilde{W}\upsilon -\dfrac{\xi }{2}\bar{W}\bar{\upsilon }\left( \upsilon \right) \right] \nonumber \\&\quad -\,\sum \limits _{j=1}^{N}\left( \bar{\pi }_{j}-\bar{e}_{j}\right) \left( \bar{\pi }_{j}-\bar{e}_{j}\right) ^{\intercal }c-\left( e^{\intercal }c-1\right) e \nonumber \\&\quad -\,\left[ c^{\intercal }{\varPsi } \upsilon -b_{ineq}\right] _{+}{\varPsi } \upsilon -4\delta \left( \left\| c\right\| ^{2}+\left\| \upsilon \right\| ^{2}\right) c \end{aligned}$$
(36)
implying
$$\begin{aligned} \dfrac{\partial ^{2}}{\partial \upsilon ^{2}}{\varPhi } _{\mu ,\delta }\left( \upsilon ,c\right)&= {} \mu \xi \left[ \tilde{W}cc^{\intercal }\tilde{W}-\mathrm { diag}\left\{ \bar{W}c\right\} \right] \nonumber \\&\quad -\, 2\chi \left( \upsilon ^{\intercal }e-\upsilon ^{+}>0\right) ee^{\intercal }-\chi \left( c^{\intercal }{\varPsi } \upsilon -b_{ineq}>0\right) {\varPsi } cc^{\intercal }{\varPsi } \nonumber \\&\quad -\,\delta \left[ 4\left( \left\| c\right\| ^{2}+\left\| \upsilon \right\| ^{2}\right) I_{NM\times NM}+8\upsilon \upsilon ^{\intercal } \right] \end{aligned}$$
(37)
$$\begin{aligned} \dfrac{\partial ^{2}}{\partial c^{2}}{\varPhi } _{\mu ,\delta }\left( \upsilon ,c\right)&= {} \mu \xi \tilde{W}\upsilon \upsilon ^{\intercal }\tilde{W} -\sum \limits _{j=1}^{N}\left( \bar{\pi }_{j}-\bar{e}_{j}\right) \left( \bar{\pi }_{j}-\bar{e}_{j}\right) ^{\intercal }-ee^{\intercal } \nonumber \\&\quad -\,\chi \left( c^{\intercal }{\varPsi } \upsilon -b_{ineq}>0\right) {\varPsi } \upsilon \upsilon ^{\intercal }{\varPsi } \nonumber \\&\quad -\,\delta \left[ 4\left( \left\| c\right\| ^{2}+\left\| \upsilon \right\| ^{2}\right) I_{NM\times NM}+8cc^{\intercal }\right] \end{aligned}$$
(38)
$$\begin{aligned} \dfrac{\partial ^{2}}{\partial c\partial \upsilon }{\varPhi } _{\mu ,\delta }\left( \upsilon ,c\right)&= {} \left[ \dfrac{\partial ^{2}}{\partial \upsilon \partial c}{\varPhi } _{\mu ,\delta }\left( \upsilon ,c\right) \right] ^{\intercal }\nonumber \\ {}&= {} \mu \left[ \tilde{W}+\xi \left[ \tilde{W}\upsilon c^{\intercal }\tilde{W} +\left( \upsilon ^{\intercal }\tilde{W}c\right) \tilde{W}\right] -\xi \bar{W} V\left( \upsilon \right) \right] \nonumber \\&\quad -\, \underset{\left[ c^{\intercal }{\varPsi } \upsilon -b_{ineq}\right] _{+}\left( {\varPsi } +{\varPsi } \upsilon c^{\intercal }{\varPsi } ^{\intercal }\right) }{\underbrace{ \dfrac{\partial }{\partial c}\left( \left[ c^{\intercal }{\varPsi } \upsilon -b_{ineq}\right] _{+}{\varPsi } c\right) }}-8\delta c\upsilon ^{\intercal }\nonumber \\&= {} \mu \left[ \tilde{W}+\xi \left[ \tilde{W}\upsilon c^{\intercal }\tilde{W} +\left( \upsilon ^{\intercal }\tilde{W}c\right) \tilde{W}\right] -\xi \bar{W} V\left( \upsilon \right) \right] \nonumber \\&\quad -\,\left[ c^{\intercal }{\varPsi } \upsilon -b_{ineq}\right] _{+}\left( {\varPsi } +{\varPsi } \upsilon c^{\intercal }{\varPsi } ^{\intercal }\right) -8\delta c\upsilon ^{\intercal } \end{aligned}$$
(39)
Notice that (34) is fulfilled if
$$\begin{aligned}&\left[ \begin{array}{cc} \frac{\partial ^{2}}{\partial c^{2}}{ {\varPhi } }_{\mu ,\delta }\left( \upsilon ,c\right) &{}\quad \frac{\partial ^{2}}{\partial \upsilon \partial c} { {\varPhi } }_{\mu ,\delta }\left( \upsilon ,c\right) \\ \frac{\partial ^{2}}{\partial c\partial \upsilon }{ {\varPhi } }_{\mu ,\delta }\left( \upsilon ,c\right) &{}\quad \frac{\partial ^{2}}{\partial \upsilon ^{2}}{ {\varPhi } }_{\mu ,\delta }\left( \upsilon ,c\right) \end{array}\right] \\&\quad = -\,\delta \left[ \begin{array}{cc} { 4}\left( \left\| c\right\| ^{2}+\left\| \upsilon \right\| ^{2}\right) I_{NM\times NM}+8cc^{\intercal } &{}\quad 8c\upsilon ^{\intercal } \\ 8\upsilon c^{\intercal } &{}\quad 4\left( \left\| c\right\| ^{2}+\left\| \upsilon \right\| ^{2}\right) I_{NM\times NM}+8\upsilon \upsilon ^{\intercal } \end{array} \right] \\&\qquad +F_{\mu ,\delta }^{0}\left( \upsilon ,c\right) + \left[ \begin{array}{ll} F_{\mu ,\delta }^{1}\left( \upsilon ,c\right) &{}\quad 0 \\ 0 &{}\quad F_{\mu ,\delta }^{2}\left( \upsilon ,c\right) \end{array}\right] \\&\qquad +\,F_{\mu ,\delta }^{3}\left( \upsilon ,c\right) +F_{\mu ,\delta }^{4}\left( \upsilon ,c\right) \\&\quad \le -4\delta \left[ \begin{array}{cc} \left( \left\| c\right\| ^{2}+\left\| \upsilon \right\| ^{2}\right) I_{NM\times NM} &{} \quad 0 \\ 0 &{} \quad \left( \left\| c\right\| ^{2}+\left\| \upsilon \right\| ^{2}\right) I_{NM\times NM} \end{array} \right] +F_{\mu ,\delta }^{3}\left( \upsilon ,c\right) <0 \end{aligned}$$
where
$$\begin{aligned} F_{\mu ,\delta }^{0}\left( \upsilon ,c\right)&= {} -8\delta \begin{bmatrix} cc^{\intercal }&\quad c\upsilon ^{\intercal } \\ \upsilon c^{\intercal }&\quad \upsilon \upsilon ^{\intercal } \end{bmatrix} =-8\delta \begin{bmatrix} c \\ \upsilon \end{bmatrix} \begin{bmatrix} c \\ \upsilon \end{bmatrix} ^{\intercal }\le 0 \\ F_{\mu ,\delta }^{1}\left( \upsilon ,c\right)&= {} -ee^{\intercal }\le 0 \\ F_{\mu ,\delta }^{2}\left( \upsilon ,c\right)&= {} -2\chi \left( \upsilon ^{\intercal }e-\upsilon ^{+}>0\right) ee^{\intercal }\le 0 \\ F_{\mu ,\delta }^{3}\left( \upsilon ,c\right)&= {} \begin{bmatrix} { \mu \xi \tilde{W}\upsilon \upsilon }^{\intercal }{ \tilde{W}} ^{\intercal }&\tau \\ \tau ^{\intercal }&{ \mu \xi }\left[ { \tilde{W}cc}^{\intercal }{ \tilde{W}}^{\intercal }-{\mathrm {diag}}\left\{ \bar{W}c\right\} \right] \end{bmatrix} \end{aligned}$$
where
$$\begin{aligned}\tau&= {} { \mu }\left[ { \tilde{W}+\xi }\left( { \tilde{W} \upsilon c}^{\intercal }{ \tilde{W}}^{\intercal }+\left( { \upsilon }^{\intercal }{ \tilde{W}c}\right) \tilde{W}-{ \bar{W}V} \left( { \upsilon }\right) \right) \right] \\ \tau ^{\intercal }&= {} { \mu }\left[ { \tilde{W}+\xi }\left( { \tilde{W}\upsilon c}^{\intercal }{ \tilde{W}}^{\intercal }+\left( { \upsilon }^{\intercal }{ \tilde{W}c}\right) \tilde{W}-{ \bar{W}V}\left( { \upsilon }\right) \right) \right] ^{\intercal } \end{aligned}$$
Notice that
$$\begin{aligned} F_{\mu ,\delta }^{4}\left( \upsilon ,c\right) =-\left[ c^{\intercal }{\varPsi } \upsilon -b_{ineq}\right] _{+}\left[ \begin{array}{cc} {\varPsi } \upsilon \upsilon ^{\intercal }{\varPsi } &{} \left( {\varPsi } +{\varPsi } \upsilon c^{\intercal }{\varPsi } ^{\intercal }\right) \\ \left( {\varPsi } +{\varPsi } c\upsilon ^{\intercal }{\varPsi } ^{\intercal }\right) &{} {\varPsi } cc^{\intercal }{\varPsi } \end{array} \right] \le 0 \end{aligned}$$
since
$$\begin{aligned}&\left( \begin{array}{l} \upsilon \\ c \end{array} \right) ^{\intercal }\left[ \begin{array}{cc} {\varPsi } \upsilon \upsilon ^{\intercal }{\varPsi } &{} \left( {\varPsi } +{\varPsi } \upsilon c^{\intercal }{\varPsi } ^{\intercal }\right) \\ \left( {\varPsi } +{\varPsi } c\upsilon ^{\intercal }{\varPsi } ^{\intercal }\right) &{} {\varPsi } cc^{\intercal }{\varPsi } \end{array} \right] \left( \begin{array}{l} \upsilon \\ c \end{array} \right) \\&\quad = \left( \begin{array}{l} \upsilon \\ c \end{array} \right) ^{\intercal }\left[ \begin{array}{l} {\varPsi } \upsilon \left( \upsilon ^{\intercal }{\varPsi } \upsilon \right) +{\varPsi } c+{\varPsi } \upsilon \left( c^{\intercal }{\varPsi } ^{\intercal }c\right) \\ {\varPsi } \upsilon +{\varPsi } c\left( \upsilon ^{\intercal }{\varPsi } ^{\intercal }\upsilon \right) +{\varPsi } c\left( c^{\intercal }{\varPsi } c\right) \end{array} \right] \\&\quad = \left( \upsilon ^{\intercal }{\varPsi } \upsilon \right) ^{2}+\left( \upsilon ^{\intercal }{\varPsi } c+c^{\intercal }{\varPsi } \upsilon \right) +2\left( \upsilon ^{\intercal }{\varPsi } \upsilon \right) \left( c^{\intercal }{\varPsi } ^{\intercal }c\right) +\left( c^{\intercal }{\varPsi } c\right) ^{2}\\&\quad = \left( \upsilon ^{\intercal }{\varPsi } \upsilon +c^{\intercal }{\varPsi } c\right) ^{2}+\upsilon ^{\intercal }{\varPsi } \upsilon +c^{\intercal }{\varPsi } ^{\intercal }c\ge 0 \end{aligned}$$
This implies
$$\begin{aligned} 4\delta \left( \left\| c\right\| ^{2}+\left\| \upsilon \right\| ^{2}\right) \left[ \begin{array}{cc} I_{NM\times NM} &{} 0 \\ 0 &{} I_{NM\times NM} \end{array} \right] >F_{\mu ,\delta }^{3}\left( \upsilon ,c\right) \end{aligned}$$
and
$$\begin{aligned}&\delta \left[ \begin{array}{cc} I_{NM\times NM} &{} 0 \\ 0 &{} I_{NM\times NM} \end{array} \right] >\dfrac{1}{4}\dfrac{F_{\mu ,\delta }^{3}\left( \upsilon ,c\right) }{ \left\| c\right\| ^{2}+\left\| \upsilon \right\| ^{2}}\\&\quad = \dfrac{{ \mu }}{4} \begin{bmatrix} { \xi \tilde{W}}\dfrac{{ \upsilon \upsilon }^{\intercal }}{ \left\| c\right\| ^{2}+\left\| \upsilon \right\| ^{2}}{ \tilde{W}}^{\intercal }&\phi \\ \phi ^{\intercal }&\dfrac{{ \xi }}{\left\| c\right\| ^{2}+\left\| \upsilon \right\| ^{2}}\left[ { \tilde{W}cc} ^{\intercal }{ \tilde{W}}-{\mathrm {diag}}\left\{ \bar{W}c\right\} \right] \end{bmatrix} \\&{\text {where}} \\&\quad \phi =\dfrac{\left[ { \tilde{W}+\xi }\left( { \tilde{W}\upsilon c }^{\intercal }{ \tilde{W}}+\left( { \upsilon }^{\intercal } { \tilde{W}c}\right) \tilde{W}-{ \bar{W}V}\left( { \upsilon }\right) \right) \right] }{\left\| c\right\| ^{2}+\left\| \upsilon \right\| ^{2}} \\&\quad \phi ^{\intercal }=\dfrac{\left[ { \tilde{W}+\xi }\left( { \tilde{W}\upsilon c}^{\intercal }{ \tilde{W}}+\left( { \upsilon } ^{\intercal }{ \tilde{W}c}\right) \tilde{W}-{ \bar{W}V}\left( { \upsilon }\right) \right) \right] ^{\intercal }}{\left\| c\right\| ^{2}+\left\| \upsilon \right\| ^{2}} \end{aligned}$$
where
$$\begin{aligned}\phi &=\dfrac{\left[ { \tilde{W}+\xi }\left( { \tilde{W}\upsilon c }^{\intercal }{ \tilde{W}}+\left( { \upsilon }^{\intercal } { \tilde{W}c}\right) \tilde{W}-{ \bar{W}V}\left( { \upsilon }\right) \right) \right] }{\left\| c\right\| ^{2}+\left\| \upsilon \right\| ^{2}} \\ \phi ^{\intercal }&=\dfrac{\left[ { \tilde{W}+\xi }\left( { \tilde{W}\upsilon c}^{\intercal }{ \tilde{W}}+\left( { \upsilon } ^{\intercal }{ \tilde{W}c}\right) \tilde{W}-{ \bar{W}V}\left( { \upsilon }\right) \right) \right] ^{\intercal }}{\left\| c\right\| ^{2}+\left\| \upsilon \right\| ^{2}} \end{aligned}$$
or equivalently,
$$\begin{aligned} \begin{bmatrix} \delta I_{NM\times NM}-\dfrac{{ \mu }}{4}{ \xi \tilde{W}}\tfrac{ { \upsilon \upsilon }^{\intercal }}{\left\| c\right\| ^{2}+\left\| \upsilon \right\| ^{2}}{ \tilde{W}}^{\intercal }&- \dfrac{{ \mu }}{4}\tfrac{\left[ { \tilde{W}+\xi }\left( { \tilde{W}\upsilon c}^{\intercal }{ \tilde{W}}+\left( { \upsilon } ^{\intercal }{ \tilde{W}c}\right) \tilde{W}-{ \bar{W}V}\left( { \upsilon }\right) \right) \right] }{\left\| c\right\| ^{2}+\left\| \upsilon \right\| ^{2}} \\ -\dfrac{{ \mu }}{4}\tfrac{\left[ { \tilde{W}+\xi }\left( { \tilde{W}\upsilon c}^{\intercal }{ \tilde{W}}+\left( { \upsilon } ^{\intercal }{ \tilde{W}c}\right) \tilde{W}-{ \bar{W}V}\left( { \upsilon }\right) \right) \right] ^{\intercal }}{\left\| c\right\| ^{2}+\left\| \upsilon \right\| ^{2}}&\delta I_{NM\times NM}-\dfrac{{ \mu }}{4}\tfrac{{ \xi }}{\left\| c\right\| ^{2}+\left\| \upsilon \right\| ^{2}}\left[ { \tilde{W}cc} ^{\intercal }{ \tilde{W}}-{\mathrm {diag}}\left\{ \bar{W}c\right\} \right] \end{bmatrix} >0 \end{aligned}$$
for all \(\upsilon \in {\varUpsilon } _{adm}\) and \(c\in \bar{C}_{adm}\). By Schur’s complement to fulfill this condition it is necessary and sufficient to satisfy that
$$\begin{aligned} A&:= {} \delta I_{NM\times NM}-\dfrac{{ \mu }}{4}{ \xi \tilde{W}} \tfrac{{ \upsilon \upsilon }^{\intercal }}{\left\| c\right\| ^{2}+\left\| \upsilon \right\| ^{2}}{ \tilde{W}}^{\intercal }\\&>\dfrac{{ \mu }}{4}\tfrac{\left[ { \tilde{W}+\xi }\left( { \tilde{W}\upsilon c}^{\intercal }{ \tilde{W}}+\left( { \upsilon } ^{\intercal }{ \tilde{W}c}\right) \tilde{W}-{ \bar{W}V}\left( { \upsilon }\right) \right) \right] ^{\intercal }}{\left\| c\right\| ^{2}+\left\| \upsilon \right\| ^{2}}\\&\times \left[ \delta I_{NM\times NM}-\dfrac{{ \mu }}{4}\tfrac{{ \xi }}{ \left\| c\right\| ^{2}+\left\| \upsilon \right\| ^{2}}\left[ { \tilde{W}cc}^{\intercal }{ \tilde{W}}-{\mathrm {diag}}\left\{ \bar{W}c\right\} \right] \right] ^{-1}\\&\times \dfrac{{ \mu }}{4}\dfrac{\left[ { \tilde{W}+\xi }\left( { \tilde{W}\upsilon c}^{\intercal }{ \tilde{W}}+\left( { \upsilon } ^{\intercal }{ \tilde{W}c}\right) \tilde{W}-{ \bar{W}V}\left( { \upsilon }\right) \right) \right] }{\left\| c\right\| ^{2}+\left\| \upsilon \right\| ^{2}}:=B \end{aligned}$$
The last matrix inequality holds for all \(\upsilon \in {\varUpsilon } _{adm}\) and \(c\in \bar{C}_{adm}\) if (here we use \(A\ge A^{\prime }>B^{\prime }\ge B\))
$$\begin{aligned} \delta -\dfrac{{ \mu }}{4}{ \xi }\underset{s}{\max }\left( { \tilde{W}}_{s}^{2}\right)\ge & {} 0 \\ \delta -\dfrac{{ \mu }}{4}{ \xi }\underset{s}{\max }\left( { \tilde{W}}_{s}^{2}\right)> & {} \dfrac{{ \mu }}{4}\left\| \dfrac{ { \tilde{W}}}{\varepsilon ^{2}\left( NM\right) }{ +\xi \tilde{W}} ^{2}\right\| \end{aligned}$$
being fulfilled if
$$\begin{aligned} \delta >{ \mu }\left\| { \tilde{W}}\right\| \left( \frac{ { \xi }}{2}\left\| { \tilde{W}}\right\| +\dfrac{1}{ 4\varepsilon ^{2}\left( NM\right) }\right) \end{aligned}$$
So, \(H<0\) which means that the penalty function (14) is strongly concave for all \(\upsilon \in {\varUpsilon } _{adm}\) and \(c\in \bar{C}_{adm}\), and, hence, has a unique maximal point defined below as \(\upsilon ^{*}\left( \mu ,\delta \right)\), \(c^{*}\left( \mu ,\delta \right)\). \(\square\)
Appendix B: Proof of Theorem 2
Proof
Before we prove Theorem 2, let us introduce the following notation
$$\begin{aligned} c={\mathrm {col}}\left[ c_{i|k}\right] ,\quad \upsilon ={\mathrm {col}}\left[ \upsilon _{i|k}\right] , \quad \omega ={\mathrm {col}}\left[ \upsilon _{i|k}\eta _{i|k}\right] ,\quad \eta ={\mathrm {col}}\left[ \eta _{i|k}\right] , \end{aligned}$$
$$\begin{aligned} A_{eq}c&= {} \left( \begin{array}{l} \left[ \sum \limits _{i=1}^{N}\sum \limits _{k=1}^{M}\pi _{j|ik}c_{i|k}-\sum \limits _{k=1}^{M}c_{j|k}\right] _{j=1,N} \\ {\mathbf {e}}_{NM}^{\intercal } \end{array} \right) =b_{eq} \\ A_{ineq}&:= {} \sum \limits _{i=1}^{N}\sum \limits _{k=1}^{M}\upsilon _{i|k}c_{i|k}\eta _{i|k}=\omega ^{\intercal }c \\ A_{ineq1}&:= {} \frac{\partial }{\partial c}A_{ineq}=\omega \\ A_{ineq2}&:= {} \frac{\partial }{\partial \upsilon }A_{ineq}={\mathrm {col}} \left[ c_{i|k}\eta _{i|k}\right] \end{aligned}$$
so that
$$\begin{aligned}&\dfrac{\partial }{\partial \upsilon }\left( \frac{1}{2}\left[ \sum \limits _{i=1}^{N}\sum \limits _{k=1}^{M}\upsilon _{i|k}c_{i|k}\eta _{i|k} -b_{ineq}\right] _{+}^{2}\right) \\&\quad =\left[ \sum \limits _{i=1}^{N}\sum \limits _{k=1}^{M}\left[ \upsilon _{i|k}\eta _{i|k}c_{i|k}-b_{ineq}\right] \right] _{+}{\mathrm {col}}[c_{i|k}\eta _{i|k}]\\&\quad = A_{ineq2}\left[ A_{ineq2}^{\intercal }\upsilon _{n}-b_{ineq}\right] _{+}=0 \\&\dfrac{\partial }{\partial c}\left( \frac{1}{2}\sum \limits _{j=1}^{N}\left( \left[ \sum \limits _{i=1}^{N}\sum \limits _{k=1}^{M}\pi _{j|ik}c_{i|k}-\sum \limits _{k=1}^{M}c_{j|k}\right] -\left( b_{eq}\right) _{j}\right) ^{2}\right) ={\mathrm {col}}\left[ Q_{\alpha |\beta } \right] \\&Q_{\alpha |\beta }:=\sum \limits _{j=1}^{N}\left[ \left( \left[ \sum \limits _{i=1}^{N}\sum \limits _{k=1}^{M}\pi _{j|ik}c_{i|k}-\sum \limits _{k=1}^{M}c_{j|k}\right] -\left( b_{eq}\right) _{j}\right) \left[ \pi _{\alpha j|\beta }-\chi _{j,\alpha } \right] \right] \\ \end{aligned}$$
where \(\chi _{j,\alpha }=1\) if \(\alpha =j\) and 0 otherwise
$$\begin{aligned} \dfrac{\partial }{\partial c}\left( \frac{1}{2}\left( \sum \limits _{i=1}^{N}\sum \limits _{k=1}^{M}c_{i|k}-1\right) ^{2}\right)&= {} \left[ \sum \limits _{i=1}^{N}\sum \limits _{k=1}^{M}c_{i|k}-1\right] \mathbf {e,} \quad \mathbf {e:}=(1,\ldots ,1)^{\top }\in {\mathbb {R}}^{NM} \\ \frac{\partial }{\partial c}\left[ \frac{1}{2}\left( \omega ^{\intercal }c-b_{ineq}\right) ^{2}\right] _{+}&= {} \left[ \omega ^{\intercal }c-b_{ineq} \right] _{+}\omega =A_{ineq1}\left[ A_{ineq1}^{\intercal }c-b_{ineq}\right] _{+} \end{aligned}$$
By the strict convexity property (15) for any \(y:=\left( c^{\intercal },\upsilon ^{\intercal }\right) ^{\intercal }\) \((\upsilon \in {\varUpsilon } _{adm}\) and \(c\in \bar{C}_{adm})\) and the extremal vector \(y_{n}^{*}\) we have
$$\begin{aligned} 0\ge & {} \left( y_{n}^{*}-y\right) ^{\intercal }\dfrac{\partial }{\partial y }{\varPhi } _{\mu _{n},\delta _{n}}\left( y_{n}^{*}\right) \nonumber \\&= {} \left( \upsilon _{n}^{*}-\upsilon \right) ^{\intercal }\dfrac{\partial }{ \partial \upsilon }{\varPhi } _{\mu _{n},\delta _{n}}\left( \upsilon _{n}^{*},c_{n}^{*}\right) +\left( c_{n}^{*}-c\right) ^{\intercal }\dfrac{ \partial }{\partial c}{\varPhi } _{\mu _{n},\delta _{n}}\left( \upsilon _{n}^{*},c_{n}^{*}\right) \nonumber \\&= {} \left( \upsilon _{n}^{*}-\upsilon \right) ^{\intercal }\left( \mu _{n} \dfrac{\partial }{\partial \upsilon }f\left( \upsilon _{n}^{*},c_{n}^{*}\right) +A_{ineq2}\left[ A_{ineq2}^{\intercal }\upsilon _{n}^{*}-b_{ineq}\right] _{+}+\delta _{n}^{*}\upsilon _{n}\right) \nonumber \\&\quad +\left( c_{n}^{*}-c\right) ^{\intercal }\left( \mu _{n}\dfrac{ \partial }{\partial c}f\left( \upsilon _{n}^{*},c_{n}^{*}\right) +A_{eq}\left[ A_{eq}^{\intercal }c_{n}^{*}-b_{eq}\right] \right. \nonumber \\&\quad + \left. \left[ \sum \limits _{i=1}^{N}\sum \limits _{k=1}^{M}c_{n,i|k}^{*}-1 \right] \mathbf {e+}A_{ineq1}\left[ A_{ineq1}^{\intercal }c_{n}^{*}-b_{ineq}\right] _{+}+\delta _{n}c_{n}^{*}\right) \end{aligned}$$
(40)
Selecting in (40) \(c:=c^{*}\in \bar{C}_{adm}^{*}\) (\(c^{*}\) is one of the admissible portfolio solutions such that \(A_{eq}c^{*}=b_{eq}\) and \(A_{ineq1}c^{*}-b_{ineq}=0\)), and \(\upsilon =\upsilon ^{*}\) (satisfying \(A_{ineq2}\upsilon ^{*}-b_{ineq}\le 0\)), we have that
$$\begin{aligned} 0& \ge {} \left( \upsilon _{n}^{*}-\upsilon ^{*}\right) ^{\intercal }\left( \mu _{n}\dfrac{\partial }{\partial \upsilon }f\left( \upsilon _{n}^{*},c_{n}^{*}\right) +A_{ineq2}\left[ A_{ineq2}^{\intercal }\upsilon _{n}^{*}-b_{ineq}\right] _{+}+\delta _{n}^{*}\upsilon _{n}\right) \\&\quad + \left( c_{n}^{*}-c^{*}\right) ^{\intercal }\left( \mu _{n}\dfrac{ \partial }{\partial c}f\left( \upsilon _{n}^{*},c_{n}^{*}\right) +A_{eq}\left[ A_{eq}^{\intercal }c_{n}^{*}-b_{eq}\right] +\left[ \sum \limits _{i=1}^{N}\sum \limits _{k=1}^{M}c_{n,i|k}^{*}-1\right] \mathbf { e}\right. \\&\quad +\left. A_{ineq1}\left[ A_{ineq1}^{\intercal }c_{n}^{*}-b_{ineq}\right] _{+}+\delta _{n}c_{n}^{*}\right) \\&= {} \mu _{n}\left( \upsilon _{n}^{*}-\upsilon ^{*}\right) ^{\intercal } \dfrac{\partial }{\partial \upsilon }f\left( \upsilon _{n}^{*},c_{n}^{*}\right) +\left( \upsilon _{n}^{*}-\upsilon ^{*}\right) ^{\intercal }\left( A_{ineq2}\left[ A_{ineq2}^{\intercal }\upsilon _{n}^{*}-b_{ineq}\right] _{+}+\delta _{n}^{*}\upsilon _{n}^{*}\right) \\&\mu _{n}\left( c_{n}^{*}-c^{*}\right) ^{\intercal }\dfrac{\partial }{ \partial c}f\left( \upsilon _{n}^{*},c_{n}^{*}\right) +\underset{ \ge 0}{\underbrace{\left( c_{n}^{*}-c^{*}\right) ^{\intercal }A_{eq}A_{eq}^{\intercal }\left( c_{n}^{*}-c^{*}\right) }}\\&\quad +\underset{\ge 0}{\underbrace{\left[ \sum \limits _{i=1}^{N}\sum \limits _{k=1}^{M}c_{n,i|k}^{*}-1\right] ^{2}}}+\left( c_{n}^{*}-c^{*}\right) ^{\intercal }A_{ineq1}^{\intercal }\left[ A_{ineq1}^{\intercal }\left( c_{n}^{*}-c^{*}\right) \right] _{+}\\&\quad +\left( c_{n}^{*}-c^{*}\right) ^{\intercal }A_{ineq1}^{\intercal } \left[ A_{ineq1}c_{n}^{*}-b_{ineq}\right] _{+}+\delta _{n}\left( c_{n}^{*}-c^{*}\right) ^{\intercal }c_{n}^{*} \end{aligned}$$
implying
$$\begin{aligned} 0\ge & {} \mu _{n}\left( \upsilon _{n}^{*}-\upsilon ^{*}\right) ^{\intercal }\dfrac{\partial }{\partial \upsilon }f\left( \upsilon _{n}^{*},c_{n}^{*}\right) +\left( \upsilon _{n}^{*}-\upsilon ^{*}\right) ^{\intercal }\left( A_{ineq2}\left[ A_{ineq2}^{\intercal }\upsilon _{n}^{*}-b_{ineq}\right] _{+}+\delta _{n}^{*}\upsilon _{n}^{*}\right) \nonumber \\&\mu _{n}\left( c_{n}^{*}-c^{*}\right) ^{\intercal }\dfrac{\partial }{ \partial c}f\left( \upsilon _{n}^{*},c_{n}^{*}\right) +\left( c_{n}^{*}-c^{*}\right) ^{\intercal }A_{ineq1}\left[ A_{ineq1}^{\intercal }c_{n}^{*}-b_{ineq}\right] _{+}+\delta _{n}\left( c_{n}^{*}-c^{*}\right) ^{\intercal }c_{n}^{*} \end{aligned}$$
(41)
Notice that
$$\begin{aligned}&\left( c_{n}^{*}-c^{*}\right) ^{\intercal }A_{ineq1}\left[ A_{ineq1}^{\intercal }c_{n}^{*}-b_{ineq}\right] _{+}\\&\quad =\left( A_{ineq1}^{\intercal }c_{n}^{*}-b_{ineq}-A_{ineq1}^{\intercal }c^{*}+b_{ineq}\right) \left[ A_{ineq1}^{\intercal }c_{n}^{*}-b_{ineq}\right] _{+}\\&\quad = \underset{\ge 0}{\underbrace{\left( A_{ineq1}^{\intercal }c_{n}^{*}-b_{ineq}\right) \left[ A_{ineq1}^{\intercal }c_{n}^{*}-b_{ineq}\right] _{+}}}\\&\qquad -\, \underset{\le 0}{\underbrace{\left( A_{ineq1}^{\intercal }c^{*}-b_{ineq}\right) }}\left[ A_{ineq1}^{\intercal }c_{n}^{*}-b_{ineq} \right] _{+}\ge 0 \end{aligned}$$
Analogously,
$$\begin{aligned} \left( \upsilon _{n}^{*}-\upsilon ^{*}\right) ^{\intercal }A_{ineq2} \left[ A_{ineq2}^{\intercal }\upsilon _{n}^{*}-b_{ineq}\right] _{+}\ge 0 \end{aligned}$$
Using these both inequalities in (41) implies
$$\begin{aligned}&0\ge \mu _{n}\left( \upsilon _{n}^{*}-\upsilon ^{*}\right) ^{\intercal }\dfrac{\partial }{\partial \upsilon }f\left( \upsilon _{n}^{*},c_{n}^{*}\right) +\left( \upsilon _{n}^{*}-\upsilon ^{*}\right) ^{\intercal }\delta _{n}^{*}\upsilon _{n}^{*} \\&\quad \mu _{n}\left( c_{n}^{*}-c^{*}\right) ^{\intercal }\dfrac{\partial }{ \partial c}f\left( \upsilon _{n}^{*},c_{n}^{*}\right) +\delta _{n}\left( c_{n}^{*}-c^{*}\right) ^{\intercal }c_{n}^{*} \end{aligned}$$
Dividing both sides of this inequality by \(\delta _{n}\) we get
$$\begin{aligned} 0\ge & {} \dfrac{\mu _{n}}{\delta _{n}}\left( \upsilon _{n}^{*}-\upsilon ^{*}\right) ^{\intercal }\dfrac{\partial }{\partial \upsilon }f\left( \upsilon _{n}^{*},c_{n}^{*}\right) +\dfrac{\mu _{n}}{\delta _{n}} \left( c_{n}^{*}-c^{*}\right) ^{\intercal }\dfrac{\partial }{ \partial c}f\left( \upsilon _{n}^{*},c_{n}^{*}\right) \nonumber \\&+\,\left( \upsilon _{n}^{*}-\upsilon ^{*}\right) ^{\intercal }\upsilon _{n}^{*}+\left( c_{n}^{*}-c^{*}\right) ^{\intercal }c_{n}^{*} \end{aligned}$$
(42)
Notice that the sequences \(\left\{ c_{n}^{*}\right\}\), \(\left\{ \upsilon _{n}^{*}\right\}\) are bounded on \(\bar{C}_{adm}\otimes {\varUpsilon } _{adm}\). Considering that by the supposition (19 ) \(\dfrac{\mu _{n}}{\delta _{n}}\underset{n\rightarrow \infty }{\rightarrow } 0\), from (42) we may conclude that for any partial limit points (which must exist for any bounded sequence by the Weierstrass theorem) \(c_{\infty }^{*}\in \bar{C}_{adm}^{*}\) and \(\upsilon _{\infty }^{*}\in {\varUpsilon } _{adm}^{*}\) (obviously, these partial limits may be not unique) that
$$\begin{aligned} 0\ge \left( c_{\infty }^{*}-c^{*}\right) ^{\intercal }c_{\infty }^{*}+\left( \upsilon _{\infty }^{*}-\upsilon ^{*}\right) ^{\intercal }\upsilon _{\infty }^{*} \end{aligned}$$
(43)
By the identities
$$\begin{aligned} \left( c_{\infty }^{*}-c^{*}\right) ^{\intercal }c_{\infty }^{*}&= {} \left\| c_{\infty }^{*}-c^{*}\right\| ^{2}+\left( c_{\infty }^{*}-c^{*}\right) ^{\intercal }c^{*} \\ \left( \upsilon _{\infty }^{*}-\upsilon ^{*}\right) ^{\intercal }\upsilon _{\infty }^{*}&= {} \left\| \upsilon _{\infty }^{*}-\upsilon ^{*}\right\| ^{2}+\left( \upsilon _{\infty }^{*}-\upsilon ^{*}\right) ^{\intercal }\upsilon ^{*} \end{aligned}$$
the inequality (43) leads to
$$\begin{aligned} 0\ge & {} \left( c_{\infty }^{*}-c^{*}\right) ^{\intercal }c_{\infty }^{*}+\left( \upsilon _{\infty }^{*}-\upsilon ^{*}\right) ^{\intercal }\upsilon _{\infty }^{*}=\left\| c_{\infty }^{*}-c^{*}\right\| ^{2}+\left\| \upsilon _{\infty }^{*}-\upsilon ^{*}\right\| ^{2}\\&+\left( c_{\infty }^{*}-c^{*}\right) ^{\intercal }c^{*}+\left( \upsilon _{\infty }^{*}-\upsilon ^{*}\right) ^{\intercal }\upsilon ^{*}\\\ge & {} \left( c_{\infty }^{*}-c^{*}\right) ^{\intercal }c^{*}+\left( \upsilon _{\infty }^{*}-\upsilon ^{*}\right) ^{\intercal }\upsilon ^{*}=\left( y_{\infty }^{*}-y^{*}\right) y^{*} \end{aligned}$$
The inequality \(0\ge \left( y_{\infty }^{*}-y^{*}\right) y^{*}\) exactly represents the necessary and sufficient condition that the point \(y^{*}\) is the minimum point of the function \(\left\| y_{\infty }^{*}\right\| ^{2}\) on the set \({\mathcal {Y}}_{adm}=\bar{C}_{adm}^{*}\otimes {\varUpsilon } _{adm}^{*}\) corresponding to \(\delta =0\). Indeed, the function \(\left\| y\right\| ^{2}\) has the minimum \(y^{*}\) at the set \({\mathcal {Y}}_{adm}\) which satisfies the necessary and sufficient condition
$$\begin{aligned} 0\ge \left( y-y^{*}\right) \frac{\partial }{\partial y}\left\| y^{*}\right\| ^{2}=2\left( y-y^{*}\right) y^{*} \end{aligned}$$
valid for any admissible \(y\in {\mathcal {Y}}_{adm}\). \(\square\)
Appendix C: Proof of Lemma 1
Proof
From (41) we have
$$\begin{aligned} \left\| A_{ineq1}c_{n}^{*}-b_{ineq}\right\| \le K_{1}\sqrt{\delta _{n}},\quad K_{1}={\mathrm {const}}>0 \end{aligned}$$
implying
$$\begin{aligned} A_{ineq1}c_{n}^{*}-b_{ineq}\le K_{1}\sqrt{\delta _{n}}e-u_{n}^{*}\le K_{1}\sqrt{\delta _{n}}e,\quad \left\| e\right\| =1 \end{aligned}$$
where the vector inequality is treated component-wise. We have also
$$\begin{aligned} \left\| c_{n}^{*}-\hat{c}_{n}\right\| ^{2}\le \underset{ A_{ineq1}{ c}-{ b}_{ineq}{ \le K}_{1}\sqrt{\delta _{n}} { e},\quad { c\in \bar{C}}_{{ adm}}}{\max }\quad \underset{y\in \bar{C}_{adm}}{\min }\left\| c-y\right\| ^{2}:=d\left( \delta _{n}\right) \end{aligned}$$
We introduce the new variable
$$\begin{aligned} \tilde{c}:=\left( 1-\nu _{n}\right) c+\nu _{n}{\mathring{c}}\in \bar{C}_{adm} \end{aligned}$$
(44)
where
$$\begin{aligned} 0<\nu _{n}:=\frac{K_{1}\sqrt{\delta _{n}}}{K_{1}\sqrt{\delta _{n}}+\underset{ j=1,\ldots ,M_{1}}{\min }\left| \left( A_{ineq1}{\mathring{c}}-b_{ineq}\right) _{j}\right| }<1 \end{aligned}$$
and \({\mathring{c}}\) satisfies the Slater condition (18). For the new variable \(c=\dfrac{\tilde{c}-\nu _{n}{\mathring{c}}}{1-\nu _{n}}\) we have
$$\begin{aligned} A_{ineq1}\tilde{c}-b_{ineq}&= {} \left( 1-\nu _{n}\right) A_{ineq1}c+\nu _{n}A_{ineq1}{\mathring{c}}-b_{ineq}\\&= {} \left( 1-\nu _{n}\right) \left( A_{ineq1}c-b_{ineq}\right) +\left( 1-\nu _{n}\right) b_{ineq}+\nu _{n}\left( A_{ineq1}{\mathring{c}}-b_{ineq}\right) \\ &\quad +\,\nu _{n}b_{ineq}-b_{ineq}=\left( 1-\nu _{n}\right) \left( A_{ineq1}c-b_{ineq}\right) +\nu _{n}\left( A_{ineq1}{\mathring{c}} -b_{ineq}\right) \\ &\le {} \left( 1-\nu _{n}\right) K_{1}\sqrt{\delta _{n}}e+\dfrac{K_{1}\sqrt{\delta _{n}}}{K_{1}\sqrt{\delta _{n}}+\underset{j=1,\ldots ,M_{1}}{\min }\left| \left( A_{ineq1}{\mathring{c}}-b_{ineq}\right) _{j}\right| }\left( A_{ineq1}{\mathring{c}}-b_{ineq}\right) \\ &= {} \dfrac{K_{1}\sqrt{\delta _{n}}}{K_{1}\sqrt{\delta _{n}}+\underset{ j=1,\ldots ,M_{1}}{\min }\left| \left( A_{ineq1}{\mathring{c}}-b_{ineq}\right) _{j}\right| } \\&\quad \times \left( \underset{j=1,\ldots ,M_{1}}{\min }\left| \left( A_{ineq1}{\mathring{c}} -b_{ineq}\right) _{j}\right| e+\left( A_{ineq1}{\mathring{c}} -b_{ineq}\right) \right) \le 0 \end{aligned}$$
Therefore,
$$\begin{aligned} d\left( \delta _{n}\right)&= {} \underset{A_{ineq1}{ c}-{ b}_{ineq} { \le K}_{1}\sqrt{\delta _{n}}{ e},{ c\in Cadm}}{ \max }\quad \underset{y\in Cadm}{\min }\left\| c-y\right\| ^{2}\\\le & {} \underset{A_{ineq1}\tilde{c}-{ b}_{ineq}{ \le 0},\tilde{ c}{ \in Cadm}}{\max }\left\| \dfrac{\tilde{c}-\nu _{n}{\mathring{c}}}{ 1-\nu _{n}}-\tilde{c}\right\| ^{2}\\&= {} \dfrac{\nu _{n}^{2}}{\left( 1-\nu _{n}\right) ^{2}}\underset{A_{ineq1}\tilde{ c}-{ b}_{ineq}{ \le 0},\quad \tilde{c}{ \in Cadm}}{\max }\left\| \tilde{c}-{\mathring{c}}\right\| ^{2}\le K_{2}\delta _{n}, \quad 0<K_{2}<\infty \end{aligned}$$
Thus, \(\left\| c_{n}^{*}-\hat{c}_{n}\right\|\) \(\le \sqrt{d\left( \delta _{n}\right) } \le\) \(\sqrt{K_{2}}\sqrt{\delta _{n}}\) which proves (23). \(\square\)
Appendix D: Proof of Theorem 3
Proof
(Theorem 3 on the convergence of the projection gradient method)
In view of (21) and using the projection property
$$\begin{aligned} \left\| \Pr \left\{ z_{n-1}+{\varGamma } _{n}\dfrac{\partial }{\partial z}{\varPhi } _{\mu _{n},\delta _{n}}\left( z_{n-1}\right) \right\} -z_{n}^{*}\right\| \le \left\| z_{n-1}+{\varGamma } _{n}\dfrac{\partial }{\partial z} {\varPhi } _{\mu _{n},\delta _{n}}\left( z_{n-1}\right) -z_{n}^{*}\right\| \end{aligned}$$
it follows that
$$\begin{aligned} Z_{n}&= {} \left\| z_{n}-z_{n}^{*}\right\| ^{2}\nonumber \\ &\le {} \left\| \left( z_{n-1}-z_{n-1}^{*}\right) -\gamma _{n}\dfrac{\partial }{\partial z}{\varPhi } _{\mu _{n},\delta _{n}}\left( z_{n-1}\right) +\left( z_{n-1}^{*}-z_{n}^{*}\right) \right\| ^{2}\nonumber \\&= {} Z_{n-1}+\gamma _{n}^{2}\left\| \dfrac{\partial }{\partial z}{\varPhi } _{\mu _{n},\delta _{n}}\left( z_{n-1}\right) \right\| ^{2}+\left\| \left( z_{n-1}^{*}-z_{n}^{*}\right) \right\| ^{2}\nonumber \\&\quad -\,2\gamma _{n}\left( z_{n-1}-z_{n-1}^{*}\right) ^{\intercal }\dfrac{ \partial }{\partial z}{\varPhi } _{\mu _{n},\delta _{n}}\left( z_{n-1}\right) \nonumber \\&\quad +\,2\left( z_{n-1}-z_{n-1}^{*}\right) ^{\intercal }\left( z_{n-1}^{*}-z_{n}^{*}\right) -2\gamma _{n}\left( z_{n-1}^{*}-z_{n}^{*}\right) ^{\intercal }\dfrac{\partial }{\partial z}{\varPhi } _{\mu _{n},\delta _{n}}\left( z_{n-1}\right) \end{aligned}$$
(45)
By the inequalities [see inequalities (21.17) and (21.36) in Poznyak (2008)] we can conclude that
$$\begin{aligned}&\left\| \dfrac{\partial }{\partial z}{\varPhi } _{\mu _{n},\delta _{n}}\left( z_{n-1}\right) \right\| ^{2}\\&\quad =\left\| \left[ \dfrac{\partial }{\partial z}{\varPhi } _{\mu _{n},\delta _{n}}\left( z_{n-1}\right) -\dfrac{\partial }{\partial z}{\varPhi } _{\mu _{n},\delta _{n}}\left( z_{n-1}^{*}\right) \right] +\dfrac{\partial }{ \partial z}{\varPhi } _{\mu _{n},\delta _{n}}\left( z_{n-1}^{*}\right) \right\| ^{2}\\&\quad \le \left( 1+\vartheta _{n}\right) \left\| \dfrac{\partial }{\partial z}{\varPhi } _{\mu _{n},\delta _{n}}\left( z_{n-1}\right) -\dfrac{\partial }{\partial z} {\varPhi } _{\mu _{n},\delta _{n}}\left( z_{n-1}^{*}\right) \right\| ^{2}\\&\qquad + \left( 1+\vartheta _{n}^{-1}\right) \left\| \dfrac{\partial }{\partial z} {\varPhi } _{\mu _{n},\delta _{n}}\left( z_{n-1}^{*}\right) \right\| ^{2}\le \left( 1+\vartheta _{n}\right) L_{\nabla }Z_{n-1}+\left( 1+\vartheta _{n}^{-1}\right) d \end{aligned}$$
where \(\left\| \dfrac{\partial }{\partial z}{\varPhi } _{\mu _{n},\delta _{n}}\left( z_{n-1}^{*}\right) \right\| ^{2}\le d\) and
$$\begin{aligned}&\left( z_{n-1}-z_{n-1}^{*}\right) ^{\intercal }\dfrac{\partial }{\partial z}\Phi _{\mu _{n},\delta _{n}}\left( z_{n-1}\right) \ge l_{n}Z_{n-1},\quad l_{n}=\left( \mu _{n}\lambda ^{-}+\delta _{n}\right) \\&\left| \left( z_{n-1}-z_{n-1}^{*}\right) ^{\intercal }\left( z_{n-1}^{*}-z_{n}^{*}\right) \right| \le \left\| z_{n-1}^{*}-z_{n}^{*}\right\| \sqrt{Z_{n-1}} \\&\left| \left( z_{n-1}^{*}-z_{n}^{*}\right) ^{\intercal }\dfrac{\partial }{\partial z}\Phi _{\mu _{n},\delta _{n}}\left( z_{n-1}\right) \right| \\&\quad \overset{\vartheta >0}{\le }\left\| z_{n-1}^{*}-z_{n}^{*}\right\| \sqrt{\left( 1+\vartheta \right) L_{\nabla }Z_{n-1}+\left( 1+\vartheta ^{-1}\right) d}\\&\quad \le \left\| z_{n-1}^{*}-z_{n}^{*}\right\| \left[ \left( 1+\vartheta ^{1/2}\right) \sqrt{L_{\nabla }}\sqrt{Z_{n-1}}+\left( 1+\vartheta ^{-1/2}\right) \sqrt{d}\right] \end{aligned}$$
Then, from (45) for \(m=n-1\) we obtain
$$\begin{aligned} Z_{n}&\le {} Z_{n-1}+\gamma _{n}^{2}\left[ \left( 1+\vartheta \right) L_{\nabla }Z_{n-1}+\left( 1+\vartheta ^{-1}\right) d\right] \\&\quad +\, 2\left( K_{1}^{2}\left| \mu _{n}-\mu _{n-1}\right| ^{2}+K_{2}^{2}\left| \delta _{n}-\delta _{n-1}\right| ^{2}\right) -2\gamma _{n}\left( \mu _{n}\lambda ^{-}+\delta _{n}\right) Z_{n-1}\\&\quad +\, 2\left( K_{1}\left| \mu _{n}-\mu _{n-1}\right| +K_{2}\left| \delta _{n}-\delta _{n-1}\right| \right) \sqrt{Z_{n-1}}\\&\quad +\, 2\gamma _{n}\left( K_{1}\left| \mu _{n}-\mu _{n-1}\right| +K_{2}\left| \delta _{n}-\delta _{n-1}\right| \right) \\ \quad &\times \left[ \left( 1+\vartheta ^{1/2}\right) \sqrt{L_{\nabla }}\sqrt{Z_{n-1}} +\left( 1+\vartheta ^{-1/2}\right) \sqrt{d}\right] \end{aligned}$$
or, equivalently,
$$\begin{aligned} Z_{n}\le Z_{n-1}\left( 1-\alpha _{n-1}\right) +\bar{\delta }_{n-1}\sqrt{ Z_{n-1}}+\beta _{n-1} \end{aligned}$$
(46)
where
$$\begin{aligned} \left. \begin{aligned} \alpha _{n-1}&=2\gamma _{n}\left( \mu _{n}\lambda ^{-}+\delta _{n}\right) -\gamma _{n}^{2}\left( 1+\vartheta \right) L_{\nabla } \\&=2\gamma _{n}\left( \mu _{n}\lambda ^{-}+\delta _{n}\right) \left[ 1-\dfrac{ \gamma _{n}\left( 1+\vartheta \right) L_{\nabla }}{2\left( \mu _{n}\lambda ^{-}+\delta _{n}\right) }\right] \\&\ge \gamma _{n}\delta _{n}2\left( 1+o\left( 1\right) \right) \left[ 1-\dfrac{ \gamma _{n}\left( 1+\vartheta \right) L_{\nabla }}{2\delta _{n}\left( o\left( 1\right) +1\right) }\right] \ge K_{\alpha }\gamma _{n}\delta _{n} \\ \bar{\delta }_{n-1}&=2\left( K_{1}\left| \mu _{n}-\mu _{n-1}\right| +K_{2}\left| \delta _{n}-\delta _{n-1}\right| \right) \left[ 1+\gamma _{n}\left( 1+\vartheta ^{1/2}\right) \sqrt{L_{\nabla }}\right] \\&\le K_{\delta }\left( \left| \mu _{n}-\mu _{n-1}\right| +\left| \delta _{n}-\delta _{n-1}\right| \right) \\ \beta _{n-1}&=\gamma _{n}^{2}\left( 1+\vartheta ^{-1}\right) d+\left( K_{1}^{2}\left| \mu _{n}-\mu _{n-1}\right| ^{2}+K_{2}^{2}\left| \delta _{n}-\delta _{n-1}\right| ^{2}\right) \\&\quad +\,2\gamma _{n}\left( K_{1}\left| \mu _{n}-\mu _{n-1}\right| +K_{2}\left| \delta _{n}-\delta _{n-1}\right| \right) \left( 1+\vartheta ^{-1/2}\right) \sqrt{d} \\&\le \gamma _{n}^{2}K_{\beta ,1}+\gamma _{n}\left( \left| \mu _{n}-\mu _{n-1}\right| +\left| \delta _{n}-\delta _{n-1}\right| \right) K_{\beta ,2} \\&\quad +\left( \left| \mu _{n}-\mu _{n-1}\right| ^{2}+\left| \delta _{n}-\delta _{n-1}\right| ^{2}\right) K_{\beta ,3} \end{aligned} \right\} \end{aligned}$$
(47)
Using the inequality
$$\begin{aligned} Z_{n}^{r}\le \left( 1-r\right) \theta _{n}^{r}+\frac{r}{\theta _{n}^{1-r}} Z_{n}, r\in \left( 0,1\right) ,\quad \theta _{n}>0 \end{aligned}$$
for \(r=1/2\) and \(\sqrt{\theta _{n}}=\) \(\dfrac{\bar{\delta }_{n-1}}{2\alpha _{n-1}\left( 1-\rho \right) }\) , \(\rho \in \left( 0,1\right)\), inequality (46) can be reduced to the following one:
$$\begin{aligned} Z_{n}\le & {} Z_{n-1}\left( 1-\alpha _{n-1}\left[ 1-\dfrac{\bar{\delta }_{n-1}}{ 2\alpha _{n-1}\sqrt{\theta _{n}}}\right] \right) +\left[ \beta _{n-1}+\frac{1 }{2}\bar{\delta }_{n-1}\sqrt{\theta _{n}}\right] \nonumber \\&= {} Z_{n-1}\left( 1-\alpha _{n-1}\rho \right) +\left[ \beta _{n-1}+\dfrac{\bar{ \delta }_{n-1}^{2}}{4\left( 1-\rho \right) \alpha _{n-1}}\right] \end{aligned}$$
(48)
By Theorem 16.14 in Poznyak (2008) \(Z_{n}\underset{n\rightarrow \infty }{\rightarrow }0\) if
$$\begin{aligned} \sum \limits _{n=0}^{\infty }\alpha _{n}=\infty ,\quad \dfrac{\beta _{n-1}}{ \alpha _{n-1}}+\dfrac{\bar{\delta }_{n-1}^{2}}{\alpha _{n-1}^{2}}\underset{ n\rightarrow \infty }{\rightarrow }0 \end{aligned}$$
which is equivalent to (25). This completes the proof. \(\square\)
