An efficient arc-search interior-point algorithm for convex quadratic programming with box constraints

Abstract

This paper proposes an arc-search interior-point algorithm for convex quadratic programming with box constraints. The problem has many applications, such as optimal control with actuator saturation. It is shown that an explicit feasible starting point exists for this problem. Therefore, an efficient feasible interior-point algorithm is proposed to tackle the problem. It is proved that the proposed algorithm is polynomial and has the best known complexity bound $O(\sqrt {n}\log (1/\epsilon ))$. Moreover, the search neighborhood for this special problem is wider than an algorithm for general convex quadratic programming problems, which implies that longer steps and faster convergence are expected. Finally, an engineering design problem is considered and the algorithm is applied to solve the engineering problem.

Appendix: : Proofs of technical lemmas

Proof Proof of Lemma 4.2

From (16), we have $\dot {x}^{\mathrm {T}} (\dot {\gamma }-\dot {\lambda })= \dot {z}^{\mathrm {T}} \dot {\gamma }+ \dot {y}^{\mathrm {T}} \dot {\lambda } =\dot {p}^{\mathrm {T}} \dot {\omega }$, $\ddot {x}^{\mathrm {T}} (\ddot {\gamma }-\ddot {\lambda })= \ddot {z}^{\mathrm {T}} \ddot {\gamma }+ \ddot {y}^{\mathrm {T}} \ddot {\lambda } =\ddot {p}^{\mathrm {T}} \ddot {\omega }$, $\ddot {x}^{\mathrm {T}} (\dot {\gamma }-\dot {\lambda })=\ddot {p}^{\mathrm {T}} \dot {\omega }$, and $\dot {x}^{\mathrm {T}} (\ddot {\gamma }-\ddot {\lambda })=\dot {p}^{\mathrm {T}} \ddot {\omega }$. Pre-multiplying $\dot {x}^{\mathrm {T}}$ and $\ddot {x}^{\mathrm {T}}$ to (15) gives

$$ \dot{x}^{\mathrm{T}} (\dot{\gamma}-\dot{\lambda}) =\dot{x}^{\mathrm{T}} H \dot{x}, $$

$$ \ddot{x}^{\mathrm{T}} (\ddot{\gamma}-\ddot{\lambda})=\ddot{x}^{\mathrm{T}} H \ddot{x}, $$

$$ \ddot{x}^{\mathrm{T}} (\dot{\gamma}-\dot{\lambda})=\ddot{x}^{\mathrm{T}} H \dot{x} = \dot{x}^{\mathrm{T}} H \ddot{x} =\dot{x}^{\mathrm{T}} (\ddot{\gamma}-\ddot{\lambda}). $$

Equations (27) and (28) follow from the first two equations and the fact that H is positive definite. The last equation gives (29). Using (27), (28), and (29) gives

$$ \begin{array}{@{}rcl@{}} & & (\dot{x}(1-\cos(\alpha))+ \ddot{x}\sin(\alpha))^{\mathrm{T}}H (\dot{x}(1-\cos(\alpha))+ \ddot{x}\sin(\alpha)) \\ & = & (\dot{x}^{\mathrm{T}} H \dot{x})(1-\cos(\alpha))^{2} +2(\dot{x}^{\mathrm{T}} H \ddot{x})\sin(\alpha)(1-\cos(\alpha)) +(\ddot{x}^{\mathrm{T}} H\ddot{x}) \sin^{2}(\alpha) \\ & = & (\dot{x}^{\mathrm{T}} H \dot{x})(1-\cos(\alpha))^{2} +(\ddot{x}^{\mathrm{T}} H \ddot{x}) \sin^{2}(\alpha) + (\ddot{x}^{\mathrm{T}} (\dot{\gamma}-\dot{\lambda})+\dot{x}^{\mathrm{T}} (\ddot{\gamma}-\ddot{\lambda}) ) \sin(\alpha) (1-\cos(\alpha)) \ge 0, \end{array} $$

which is the first inequality of (30). Using (27), (28), and (29) also gives

$$ \begin{array}{@{}rcl@{}} & & (\dot{x}(1-\cos(\alpha))- \ddot{x}\sin(\alpha))^{\mathrm{T}}H (\dot{x}(1-\cos(\alpha))- \ddot{x}\sin(\alpha)) \\ & = & (\dot{x}^{\mathrm{T}} H \dot{x})(1-\cos(\alpha))^{2} -2(\dot{x}^{\mathrm{T}} H \ddot{x})\sin(\alpha)(1-\cos(\alpha)) +(\ddot{x}^{\mathrm{T}} H\ddot{x}) \sin^{2}(\alpha) \\ & = & (\dot{x}^{\mathrm{T}} H \dot{x})(1-\cos(\alpha))^{2} +(\ddot{x}^{\mathrm{T}} H\ddot{x}) \sin^{2}(\alpha) - (\ddot{x}^{\mathrm{T}} (\dot{\gamma}-\dot{\lambda})+\dot{x}^{\mathrm{T}} (\ddot{\gamma}-\ddot{\lambda}) ) \sin(\alpha) (1-\cos(\alpha)) \ge 0, \end{array} $$

which is the second inequality of (30). Replacing $\dot {x}(1-\cos \limits (\alpha ))$ and $\ddot {x}\sin \limits (\alpha )$ by $\dot {x}\sin \limits (\alpha )$ and $\ddot {x}(1-\cos \limits (\alpha ))$, and following the same method, we can obtain (31). □

Proof Proof of Lemma 4.3

From the last two rows of (13) or equivalently (17), we have

$$ \begin{array}{@{}rcl@{}} {\Lambda} \dot{y}+Y \dot{\lambda} = {\Lambda} Y e \\ {\Gamma} \dot{z}+Z \dot{\gamma} = {\Gamma} Z e. \end{array} $$

Pre-multiplying $Y^{-\frac {1}{2}}{\Lambda }^{-\frac {1}{2}}$ on both sides of the first equality gives

$$ \begin{array}{@{}rcl@{}} Y^{-\frac{1}{2}}{\Lambda}^{\frac{1}{2}}\dot{y} + Y^{\frac{1}{2}}{\Lambda}^{-\frac{1}{2}}\dot{\lambda}=Y^{\frac{1}{2}}{\Lambda}^{\frac{1}{2}}e. \end{array} $$

Pre-multiplying $Z^{-\frac {1}{2}}{\Gamma }^{-\frac {1}{2}}$ on both sides of the second equality gives

$$ \begin{array}{@{}rcl@{}} Z^{-\frac{1}{2}}{\Gamma}^{\frac{1}{2}}\dot{z} + Z^{\frac{1}{2}}{\Gamma}^{-\frac{1}{2}}\dot{\gamma}=Z^{\frac{1}{2}}{\Gamma}^{\frac{1}{2}}e. \end{array} $$

(79)

Let $u=\left [ \begin {array}{c} Y^{-\frac {1}{2}}{\Lambda }^{\frac {1}{2}}\dot {y} \\ Z^{-\frac {1}{2}}{\Gamma }^{\frac {1}{2}}\dot {z} \end {array} \right ]$, $v=\left [ \begin {array}{c} Y^{\frac {1}{2}}{\Lambda }^{-\frac {1}{2}}\dot {\lambda } \\ Z^{\frac {1}{2}}{\Gamma }^{-\frac {1}{2}}\dot {\gamma } \end {array} \right ]$, and $w=\left [ \begin {array}{c} Y^{\frac {1}{2}}{\Lambda }^{\frac {1}{2}}e \\ Z^{\frac {1}{2}}{\Gamma }^{\frac {1}{2}}e \end {array} \right ]$, and using (16) andLemma 4.2, we have $u^{\mathrm {T}}v =\dot {y}^{\mathrm {T}} \dot {\lambda } + \dot {z}^{\mathrm {T}} \dot {\gamma } = \dot {x}^{\mathrm {T}} (\dot {\gamma }-\dot {\lambda }) \ge 0$. Using Lemma 4.3 and (9), we have

$$ \|u\|^{2} + \|v\|^{2}={\sum}_{i=1}^{n} \left( \frac{\dot{y}_{i}^{2}\lambda_{i}}{y_{i}} +\frac{\dot{z}_{i}^{2}\gamma_{i}}{z_{i}} \right) + {\sum}_{i=1}^{n} \left( \frac{\dot{\lambda}_{i}^{2}y_{i}}{\lambda_{i}} + \frac{\dot{\gamma}_{i}^{2}z_{i}}{\gamma_{i}} \right) \le {\sum}_{i=1}^{n} \left( y_{i} \lambda_{i} + z_{i} \gamma_{i} \right) = {\sum}_{i=1}^{2n} p_{i} \omega_{i} = 2n\mu. $$

Since p_i > 0 and ω_i > 0, dividing both sides of the inequality by $\min \limits _{j} p_{i} \omega _{i}$ and using (11) gives

$$ {\sum}_{i=1}^{n} \left( \frac{\dot{y}_{i}^{2}}{{y_{i}^{2}}} +\frac{\dot{z}_{i}^{2}}{{z_{i}^{2}}} \right) + {\sum}_{i=1}^{n} \left( \frac{\dot{\gamma}_{i}^{2}}{{\gamma_{i}^{2}}} +\frac{\dot{\lambda}_{i}^{2}}{{\lambda_{i}^{2}}} \right) = \Bigl\lVert \frac{\dot{p}}{p} \Bigr\rVert^{2} + \Bigl\lVert \frac{\dot{\omega}}{\omega} \Bigr\rVert^{2} \le \frac{2n\mu}{\min_{j} p_{i}\omega_{i}} \le \frac{2n}{1-\theta}. $$

(80)

This proves (33). Combining (33) and Lemma 4.1 yields

$$ \Bigl\lVert \frac{{\dot{p}}}{p} \Bigr\rVert^{2} \Bigl\lVert \frac{{\dot{\omega}}}{\omega} \Bigr\rVert^{2} \le \left( \frac{n}{(1-\theta)} \right)^{2}. $$

This leads to

$$ \Bigl\lVert \frac{{\dot{p}}}{{p}} \Bigr\rVert \Bigl\lVert \frac{{\dot{\omega}}}{\omega} \Bigr\rVert \le \frac{n}{(1-\theta)}. $$

(81)

Therefore, using (11) and Cauchy-Schwarz inequality yields

$$ \frac{\dot{p}^{\mathrm{T}} \dot{\omega}} {\mu} \le \frac{|\dot{p}|^{\mathrm{T}} |\dot{\omega}|} {\mu} \le (1+\theta)\frac{|\dot{p}|^{\mathrm{T}} |\dot{\omega}|}{\max_{i} p_{i}\omega_{i}} \le (1+\theta) \left( \frac{|\dot{p}|}{p} \right)^{\mathrm{T}} \left( \frac{|\dot{\omega}|}{\omega}\right) \le (1+\theta) \Bigl\lVert \frac{{\dot{p}}}{p} \Bigr\rVert \Bigl\lVert \frac{{\dot{\omega}}}{\omega} \Bigr\rVert \le \frac{1+\theta}{1-\theta}n, $$

(82)

which is the second inequality of (35). From Lemma 4.2, $\dot {p}^{\mathrm {T}} \dot {\omega } = \dot {x}^{\mathrm {T}} (\dot {\gamma }-\dot {\lambda }) = \dot {x}^{\mathrm {T}}H \dot {x} \ge 0$, we have the first inequality of (35).□

Proof Proof of Lemma 4.4

Similar to the proof of Lemma 4.3, from (18), we have

$$ \begin{array}{@{}rcl@{}} & & {\Lambda} \ddot{y}+Y \ddot{\lambda} =-2\left( \dot{y} \circ \dot{\lambda}\right) \\ & \Longleftrightarrow & Y^{-\frac{1}{2}}{\Lambda}^{\frac{1}{2}}\ddot{y} + Y^{\frac{1}{2}}{\Lambda}^{-\frac{1}{2}}\ddot{\lambda}=-2Y^{-\frac{1}{2}}{\Lambda}^{-\frac{1}{2}} \left( \dot{y} \circ \dot{\lambda} \right), \end{array} $$

and

$$ \begin{array}{@{}rcl@{}} & & {\Gamma} \ddot{z}+Z \ddot{\gamma} =-2 \left( \dot{z} \circ \dot{\gamma} \right) \\ & \Longleftrightarrow & Z^{-\frac{1}{2}}{\Gamma}^{\frac{1}{2}}\ddot{z} + Z^{\frac{1}{2}}{\Gamma}^{-\frac{1}{2}}\ddot{\gamma}=-2Z^{-\frac{1}{2}}{\Gamma}^{-\frac{1}{2}} \left( \dot{z} \circ \dot{\gamma} \right). \end{array} $$

Let $u=\left [ \begin {array}{c} Y^{-\frac {1}{2}}{\Lambda }^{\frac {1}{2}}\ddot {y} \\ Z^{-\frac {1}{2}}{\Gamma }^{\frac {1}{2}}\ddot {z} \end {array} \right ]$, $v=\left [ \begin {array}{c} Y^{\frac {1}{2}}{\Lambda }^{-\frac {1}{2}}\ddot {\lambda } \\ Z^{\frac {1}{2}}{\Gamma }^{-\frac {1}{2}}\ddot {\gamma } \end {array} \right ]$, and $w=\left [ \begin {array}{c} -2Y^{-\frac {1}{2}}{\Lambda }^{-\frac {1}{2}} \left (\dot {y} \circ \dot {\lambda } \right ) \\ -2Z^{-\frac {1}{2}}{\Gamma }^{-\frac {1}{2}} \left (\dot {z} \circ \dot {\gamma } \right ) \end {array} \right ]$, using (16) and Lemma 4.2, we have $u^{\mathrm {T}}v = \ddot {y}^{\mathrm {T}} \ddot {\lambda } + \ddot {z}^{\mathrm {T}} \ddot {\gamma } = \ddot {x}^{\mathrm {T}} (\ddot {\gamma }-\ddot {\lambda }) \ge 0$. Using Lemma 4.3, we have

$$ \begin{array}{@{}rcl@{}} \|u\|^{2} + \|v\|^{2} & = & {\sum}_{i=1}^{n} \left( \frac{\ddot{y}_{i}^{2}\lambda_{i}}{y_{i}} + \frac{\ddot{z}_{i}^{2}\gamma_{i}}{z_{i}} \right) + {\sum}_{i=1}^{n} \left( \frac{\ddot{\lambda}_{i}^{2}y_{i}}{\lambda_{i}} + \frac{\ddot{\gamma}_{i}^{2} z_{i}}{\gamma_{i}} \right) \\ & \le & \Bigl\lVert -2Y^{-\frac{1}{2}}{\Lambda}^{-\frac{1}{2}} \left( \dot{y} \circ \dot{\lambda} \right) \Bigr\rVert^{2} + \Bigl\lVert -2Z^{-\frac{1}{2}}{\Gamma}^{-\frac{1}{2}} \left( \dot{z} \circ \dot{\gamma} \right) \Bigr\rVert^{2} \\ & = & 4{\sum}_{i=1}^{n} \left( \frac{{\dot{y}_{i}^{2}}}{y_{i}} \frac{{\dot{\lambda}_{i}^{2}}}{{\lambda_{i}}} + \frac{{\dot{z}_{i}^{2}}}{z_{i}} \frac{{\dot{\gamma}_{i}^{2}}}{{\gamma_{i}}} \right). \end{array} $$

Dividing both sides of the inequality by μ and using (11) gives

$$ \begin{array}{@{}rcl@{}} & & (1-\theta) \left( {\sum}_{i=1}^{n} \left( \frac{\ddot{y}_{i}^{2}}{{y_{i}^{2}}} + \frac{\ddot{z}_{i}^{2}}{{z_{i}^{2}}} \right) + {\sum}_{i=1}^{n} \left( \frac{\ddot{\lambda}_{i}^{2}}{{\lambda_{i}^{2}}} +\frac{\ddot{\gamma}_{i}^{2}}{{\gamma_{i}^{2}}} \right) \right) \\ & = & (1-\theta) \left( \Bigl\lVert \frac{{\ddot{p}}}{p} \Bigr\rVert^{2} +\Bigl\lVert \frac{{\ddot{\omega}}}{\omega} \Bigr\rVert^{2} \right) \\ & \le & 4(1+\theta) \left( {\sum}_{i=1}^{n} \left( \frac{{\dot{y}_{i}^{2}}}{{{y_{i}^{2}}}}\frac{{\dot{\lambda}_{i}^{2}}}{{{\lambda_{i}^{2}}}} + \frac{{\dot{z}_{i}^{2}}}{{{z_{i}^{2}}}}\frac{{\dot{\gamma}_{i}^{2}}}{{\gamma_{i}^{2}}} \right) \right), \end{array} $$

in view of Lemma 4.3, this leads to

$$ \Bigl\lVert \frac{\ddot{p}}{p} \Bigr\rVert^{2} + \Bigl\lVert \frac{\ddot{\omega}}{\omega} \Bigr\rVert^{2} \le 4 \frac{1+\theta}{1-\theta} \Bigl\lVert \frac{\dot{p}}{p} \circ \frac{\dot{\omega}}{\omega} \Bigr\rVert^{2} \le 4 \frac{1+\theta}{1-\theta} \Bigl\lVert \frac{\dot{p}}{p} \Bigr\rVert^{2} \Bigl\lVert \frac{\dot{\omega}}{\omega} \Bigr\rVert^{2} \le \frac{4(1+\theta)n^{2}}{(1-\theta)^{3}}. $$

(83)

This proves (36). Combining (36) and Lemma 4.1 yields

$$ \Bigl\lVert \frac{\ddot{p}}{p} \Bigr\rVert^{2} \Bigl\lVert \frac{\ddot{\omega}}{\omega} \Bigr\rVert^{2} \le \left( \frac{2(1+\theta)n^{2}}{(1-\theta)^{3}} \right)^{2}. $$

Using (11) and Cauchy-Schwarz inequality yields

$$ \frac{\ddot{p}^{\mathrm{T}} \ddot{\omega}} {\mu} \le \frac{|\ddot{p}|^{\mathrm{T}} |\ddot{\omega}|} {\mu} \le (1+\theta)\frac{|\ddot{p}|^{\mathrm{T}} |\ddot{\omega}|}{\max_{i} p_{i}\omega_{i}} \le (1+\theta) \left( \frac{|\ddot{p}|}{p} \right)^{\mathrm{T}} \left( \frac{|\ddot{\omega}|}{\omega} \right) \le (1+\theta) \Bigl\lVert \frac{\ddot{p}}{p} \Bigr\rVert \Bigl\lVert \frac{\ddot{\omega}}{\omega} \Bigr\rVert \le \frac{2n^{2}(1+\theta)^{2}}{(1-\theta)^{3}}, $$

which is the second inequality of (38). Using (16) and Lemma 4.2, we have $\ddot {p}^{\mathrm {T}} \ddot {\omega } = \ddot {y}^{\mathrm {T}} \ddot {\lambda }+ \ddot {z}^{\mathrm {T}} \ddot {\gamma } =\ddot {x}^{\mathrm {T}}(\ddot {\gamma }-\ddot {\lambda }) =\ddot {x}^{\mathrm {T}}H \ddot {x} \ge 0$. This proves the first inequality of (38). Finally, using (11), Cauchy-Schwarz inequality, (33), and (36) yields

$$ \begin{array}{@{}rcl@{}} \frac{\left| \dot{p}^{\mathrm{T}} \ddot{\omega} \right|} {\mu} \le \frac{|\dot{p}|^{\mathrm{T}} |\ddot{\omega}|} {\mu} \le (1+\theta)\frac{|\dot{p}|^{\mathrm{T}} |\ddot{\omega}|}{\max_{i} p_{i}\omega_{i}} \le (1+\theta) \left( \frac{|\dot{p}|}{p} \right)^{\mathrm{T}} \left( \frac{|\ddot{\omega}|}{\omega} \right) \\ \le (1+\theta) \Bigl\lVert \frac{\dot{p}}{p} \Bigr\rVert \Bigl\lVert \frac{\ddot{\omega}}{\omega} \Bigr\rVert \le (1+\theta) \left( \frac{2n}{1-\theta} \right)^{\frac{1}{2}} \left( \frac{4(1+\theta)n^{2}}{(1+\theta)^{3}} \right)^{\frac{1}{2}} \le \frac{(2n(1+\theta))^{\frac{3}{2}}}{(1-\theta)^{2}}. \end{array} $$

This proves the first inequality of (39). Replacing $\dot {p}$ by $\ddot {p}$ and $\ddot {\omega }$ by $\dot {\omega }$, then using the same reasoning, we can prove the second inequality of (39).□

Proof Proof of Lemma 4.5

Using (20), (22), (17), and (18), we have

$$ \begin{array}{@{}rcl@{}} && y^{\mathrm{T}}({\alpha})\lambda({\alpha}) \\ &= & \Big(y^{\mathrm{T}}-\dot{y}^{\mathrm{T}}\sin({\alpha})+\ddot{y}^{\mathrm{T}}(1-\cos({\alpha})) \Big) \Big(\lambda-\dot{\lambda}\sin({\alpha})+\ddot{\lambda}(1-\cos({\alpha})) \Big) \\ & = & y^{\mathrm{T}}\lambda-y^{\mathrm{T}}\dot{\lambda}\sin({\alpha}) +y^{\mathrm{T}}\ddot{\lambda}(1-\cos({\alpha})) \\ && -\dot{y}^{\mathrm{T}}\lambda \sin({\alpha})+\dot{y}^{\mathrm{T}}\dot{\lambda}\sin^{2}({\alpha}) -\dot{y}^{\mathrm{T}}\ddot{\lambda}\sin({\alpha})(1-\cos({\alpha})) \\ & & +\ddot{y}^{\mathrm{T}}{\lambda}(1-\cos({\alpha})) -\ddot{y}^{\mathrm{T}}\dot{\lambda}\sin({\alpha})(1-\cos({\alpha})) +\ddot{y}^{\mathrm{T}}\ddot{\lambda}(1-\cos({\alpha}))^{2} \\ & = & y^{\mathrm{T}}\lambda-(y^{\mathrm{T}}\dot{\lambda}+\lambda^{\mathrm{T}}\dot{y})\sin({\alpha}) + (y^{\mathrm{T}}\ddot{\lambda}+\lambda^{\mathrm{T}}\ddot{y})(1-\cos({\alpha})) \\ && - (\dot{y}^{\mathrm{T}}\ddot{\lambda}+\dot{\lambda}^{\mathrm{T}}\ddot{y})\sin({\alpha})(1-\cos({\alpha})) +\dot{y}^{\mathrm{T}}\dot{\lambda}\sin^{2}({\alpha})+\ddot{y}^{\mathrm{T}}\ddot{\lambda}(1-\cos({\alpha}))^{2} \\ & = & y^{\mathrm{T}}\lambda(1-\sin({\alpha})) -2\dot{y}^{\mathrm{T}}\dot{\lambda}(1-\cos({\alpha}))\\ && - (\dot{y}^{\mathrm{T}}\ddot{\lambda}+\dot{\lambda}^{\mathrm{T}}\ddot{y})\sin({\alpha})(1-\cos({\alpha}))\\ & & + \dot{y}^{\mathrm{T}}\dot{\lambda}(1-\cos^{2}({\alpha})) + \ddot{y}^{\mathrm{T}}\ddot{\lambda}(1-\cos({\alpha}))^{2} \\ & = & y^{\mathrm{T}}\lambda (1-\sin({\alpha}))+(\ddot{y}^{\mathrm{T}}\ddot{\lambda}-\dot{y}^{\mathrm{T}}\dot{\lambda}) (1-\cos({\alpha}))^{2} - (\dot{y}^{\mathrm{T}}\ddot{\lambda}+\dot{\lambda}^{\mathrm{T}}\ddot{y})\sin({\alpha})(1-\cos({\alpha})). \end{array} $$

(84)

Using (21), (23), (17), (18), and a similar derivation of (84), we have

$$ z^{\mathrm{T}}({\alpha})\gamma({\alpha})=z^{\mathrm{T}}\gamma (1-\sin({\alpha}))+ (\ddot{z}^{\mathrm{T}}\ddot{\gamma}-\dot{z}^{\mathrm{T}}\dot{\gamma})(1-\cos({\alpha}))^{2} - (\dot{z}^{\mathrm{T}}\ddot{\gamma}+\dot{\gamma}^{\mathrm{T}}\ddot{z})\sin({\alpha})(1-\cos({\alpha})). $$

(85)

Combining (84) and (85) gives

$$ \begin{array}{@{}rcl@{}} && {}2n\mu({\alpha}) =p^{\mathrm{T}}(\alpha) \omega(\alpha) \\ &= & y^{\mathrm{T}}({\alpha})\lambda({\alpha}) + z^{\mathrm{T}}({\alpha})\gamma({\alpha}) \\ &= & (y^{\mathrm{T}}\lambda+z^{\mathrm{T}}\gamma) (1-\sin({\alpha})) +(\ddot{y}^{\mathrm{T}}\ddot{\lambda}+\ddot{z}^{\mathrm{T}}\ddot{\gamma} -\dot{y}^{\mathrm{T}}\dot{\lambda}-\dot{z}^{\mathrm{T}}\dot{\gamma}) (1-\cos({\alpha}))^{2} \\ && -(\dot{y}^{\mathrm{T}} \ddot{\lambda}+\dot{z}^{\mathrm{T}} \ddot{\gamma} +\ddot{y}^{\mathrm{T}} \dot{\lambda}+\ddot{z}^{\mathrm{T}} \dot{\gamma})\sin(\alpha)(1-\cos({\alpha})) \\ &= & (y^{\mathrm{T}}\lambda+z^{\mathrm{T}}\gamma)(1-\sin({\alpha})) + (\ddot{x}^{\mathrm{T}}(\ddot{\gamma}-\ddot{\lambda}) -\dot{x}^{\mathrm{T}}(\dot{\gamma}-\dot{\lambda}))(1-\cos({\alpha}))^{2} {\kern3.9pc}\text{use (16)} \\ && -(\dot{x}^{\mathrm{T}} (\ddot{\gamma}-\ddot{\lambda}) +\ddot{x}^{\mathrm{T}} (\dot{\gamma}-\dot{\lambda}))\sin(\alpha)(1-\cos({\alpha})) \\ &\le & (y^{\mathrm{T}}\lambda+z^{\mathrm{T}}\gamma) \left( 1-\sin({\alpha}) \right) +(\ddot{x}^{\mathrm{T}} H \ddot{x}- \dot{x}^{\mathrm{T}} H \dot{x})(1-\cos({\alpha}))^{2}{\kern2pc}\text{use (30) in Lemma 4.2} \\ && + \dot{x}^{\mathrm{T}} H \dot{x}(1-\cos({\alpha}))^{2} + \ddot{x}^{\mathrm{T}} H \ddot{x}\sin^{2}(\alpha) \\ &= & (y^{\mathrm{T}}\lambda+z^{\mathrm{T}}\gamma) \left( 1-\sin({\alpha}) \right) + \ddot{x}^{\mathrm{T}} H \ddot{x} (1-\cos({\alpha}))^{2} + \ddot{x}^{\mathrm{T}} H \ddot{x}\sin^{2}(\alpha). \end{array} $$

(86)

Dividing both sides by 2n proves the second inequality of the lemma. Combining (86) and (31) proves the first inequality of the lemma.□□

Proof Proof of Lemma 4.6

From the second inequality of (40), we have

$$ \mu(\alpha) - \mu \le \mu \sin(\alpha) \left( -1 + \frac{\ddot{x}^{\mathrm{T}}H\ddot{x}}{2n\mu} \sin(\alpha) + \frac{\ddot{x}^{\mathrm{T}}H\ddot{x}}{2n\mu} \sin^{3}(\alpha) \right). $$

Clearly, if $\frac {\ddot {x}^{\mathrm {T}}H\ddot {x}}{2n\mu } \le \frac {1}{2}$, for any $\alpha \in [0, \frac {\pi }{2}]$, the function

$$ f(\alpha) := \left( -1 + \frac{\ddot{x}^{\mathrm{T}}H\ddot{x}}{2n\mu} \sin(\alpha) + \frac{\ddot{x}^{\mathrm{T}}H\ddot{x}}{2n\mu} \sin^{3}(\alpha) \right) \le 0, $$

and μ(α) ≤ μ. If $\frac {\ddot {x}^{\mathrm {T}}H\ddot {x}}{2n\mu } > \frac {1}{2}$, using Lemma 2.5, the function f has one real solution $\sin \limits (\alpha ) \in (0,1)$. The solution is given as

$$ \sin(\hat{\alpha}) = \sqrt[3]{\frac{n\mu}{\ddot{x}^{\mathrm{T}}H\ddot{x}} + \sqrt{\left( \frac{n\mu}{\ddot{x}^{\mathrm{T}}H\ddot{x}} \right)^{2} + \left( \frac{1}{3} \right)^{3} } } +\sqrt[3]{\frac{n\mu}{\ddot{x}^{\mathrm{T}}H\ddot{x}} - \sqrt{\left( \frac{n\mu}{\ddot{x}^{\mathrm{T}}H\ddot{x}} \right)^{2} + \left( \frac{1}{3} \right)^{3} } }. $$

This proves the Lemma.□

Proof Proof of Lemma 4.7

Since $\sin \limits (\tilde {\alpha })$ is the only positive real solution of (42) in [0, 1] and q(0) < 0, substituting a₀,a₁,a₂,a₃, and a₄ into (42), we have, for all $\sin \limits (\alpha ) \le \sin \limits (\tilde {\alpha })$,

$$ \begin{array}{@{}rcl@{}} && \left( \Big\lVert \ddot{p} \circ \ddot{\omega}-\dot{\omega} \circ \dot{p} -\frac{1}{2n}(\ddot{p}^{\mathrm{T}}\ddot{\omega}-\dot{\omega}^{\mathrm{T}}\dot{p})e \Big\rVert \right) \sin^{4}(\alpha) +\left( \Big\lVert \dot{p} \circ \ddot{\omega}+\dot{\omega} \circ \ddot{p} -\frac{1}{2n}(\dot{p}^{\mathrm{T}}\ddot{\omega}+\dot{\omega}^{\mathrm{T}}\ddot{p})e \Big\rVert \right)\sin^{3}(\alpha) \\ &\le & - \left( 2\theta \frac{\dot{p}^{\mathrm{T}}\dot{\omega}}{2n} \right) \sin^{4}(\alpha) -\left( 2\theta \frac{\dot{p}^{\mathrm{T}}\dot{\omega}}{2n} \right)\sin^{2}(\alpha) +\theta \mu (1-\sin(\alpha)). \end{array} $$

(87)

Using (24), (25), (17), (18), (44), Lemma 2.2, (87), and the first inequality of (40), we have

$$ \begin{array}{@{}rcl@{}} && \Big\lVert p(\alpha) \circ \omega(\alpha) - \mu(\alpha) e \Big\rVert \\ &= & \Big\lVert \Big(p-\dot{p}\sin(\alpha) +\ddot{p} (1-\cos(\alpha)) \Big) \circ \Big(\omega-\dot{\omega}\sin(\alpha) +\ddot{\omega} (1-\cos(\alpha)) \Big) -\mu(\alpha) e \Big\rVert \\ &= & \Big\lVert (p \circ \omega - {\mu} e)(1-\sin(\alpha)) + \left( \ddot{p} \circ \ddot{\omega} -\dot{p} \circ \dot{\omega} - \frac{1}{2n}(\ddot{p}^{\mathrm{T}} \ddot{\omega} -\dot{p}^{\mathrm{T}} \dot{\omega})e \right) (1-\cos(\alpha))^{2} \\ && - \left( \dot{p} \circ \ddot{\omega}+\dot{\omega} \circ \ddot{p} - \frac{1}{2n}(\dot{p}^{\mathrm{T}} \ddot{\omega} +\ddot{p}^{\mathrm{T}} \dot{\omega})e \right) \sin(\alpha) (1-\cos(\alpha)) \Big\rVert\\ &\le & (1-\sin(\alpha))\Big\lVert {p} \circ {\omega} - {\mu} e \Big\rVert + \Big\lVert (\ddot{p} \circ \ddot{\omega}-\dot{p} \circ \dot{\omega} - \frac{1}{2n}(\ddot{p}^{\mathrm{T}} \ddot{\omega} -\dot{p}^{\mathrm{T}} \dot{\omega}) )e \Big\rVert (1-\cos(\alpha))^{2} \\ && + \Big\lVert (\dot{p} \circ \ddot{\omega}+\dot{\omega} \circ \ddot{p} - \frac{1}{2n}(\dot{p}^{\mathrm{T}} \ddot{\omega} +\ddot{p}^{\mathrm{T}} \dot{\omega})e \Big\rVert \sin(\alpha) (1-\cos(\alpha)) \end{array} $$

(88)

$$ \begin{array}{@{}rcl@{}} &\le & \theta {\mu} (1-\sin(\alpha)) + \Big\lVert (\ddot{p} \circ \ddot{\omega}-\dot{p} \circ \dot{\omega} - \frac{1}{2n}(\ddot{p}^{\mathrm{T}} \ddot{s} -\dot{p}^{\mathrm{T}} \dot{\omega}) )e \Big\rVert \sin^{4}(\alpha) +a_{3}\sin^{3}(\alpha) \\ &\le & 2 \theta \mu (1-\sin(\alpha))-\Big(2\theta \frac{\dot{p}^{\mathrm{T}} \dot{\omega}}{2n} \Big) (\sin^{4}(\alpha)+\sin^{2}(\alpha)) \\ &\le & 2 \theta \left( \mu (1-\sin(\alpha))-\Big(\frac{\dot{x}^{\mathrm{T}} H \dot{x}}{2n} \Big) \left( \left( 1-\cos(\alpha) \right)^{2}+\sin^{2}(\alpha) \right) \right) \\ &\le & 2 \theta \mu(\alpha). \end{array} $$

(89)

Hence, the point (x(α),p(α),ω(α)) satisfies the proximity condition for $\mathcal {N}_{2}(2\theta )$. To check the positivity condition (p(α),ω(α)) > 0, note that the initial condition (p,ω) > 0. It follows from (89) and Corollary 4.1 that, for $\sin \limits (\alpha ) \le \sin \limits (\bar {\alpha })$ and 𝜃 < 0.5,

$$ p_{i}(\alpha)\omega_{i}(\alpha) \ge (1-2\theta)\mu(\alpha) >0. $$

(90)

Therefore, we cannot have p_i(α) = 0 or ω_i(α) = 0 for any index i when $\alpha \in [0, \sin \limits ^{-1}(\bar {\alpha })]$. This proves (p(α),ω(α)) > 0.□

Proof Proof of Lemma 4.6

Since

$$\Bigl\lVert \frac{\dot{p}}{p} \Bigr\rVert^{2} ={\sum}_{i=1}^{2n} \left( \frac{\dot{p_{i}}}{p_{i}} \right)^{2}, \Bigl\lVert \frac{\dot{\omega}}{\omega} \Bigr\rVert^{2} ={\sum}_{i=1}^{2n} \left( \frac{\dot{\omega_{i}}}{\omega_{i}} \right)^{2}, $$

from Lemma 4.3 and (11), we have

$$ \begin{array}{@{}rcl@{}} & & \left( \frac{n}{1-\theta} \right)^{2} \\ & \ge & \Bigl\lVert \frac{{\dot{p}}}{p} \Bigr\rVert^{2} \Bigl\lVert \frac{\dot{\omega}}{\omega} \Bigr\rVert^{2} =\left( {\sum}_{i=1}^{2n} \left( \frac{\dot{p_{i}}}{p_{i}} \right)^{2} \right) \left( {\sum}_{i=1}^{2n} \left( \frac{\dot{\omega_{i}}}{\omega_{i}} \right)^{2} \right) \\ & \ge & {\sum}_{i=1}^{2n} \left( \frac{\dot{p_{i}}}{p_{i}} \frac{\dot{\omega_{i}}}{\omega_{i}} \right)^{2} =\Bigl\lVert \frac{\dot{p}}{p} \circ \frac{\dot{\omega}}{\omega} \Bigr\rVert^{2} \\ & \ge & {\sum}_{i=1}^{2n} \left( \frac{\dot{p_{i}}\dot{\omega_{i}}}{(1+\theta) \mu} \right)^{2} =\frac{1}{(1+\theta)^{2} \mu^{2}}\Bigl\lVert \dot{p} \circ \dot{\omega} \Bigr\rVert^{2}, \end{array} $$

i.e.,

$$ \Bigl\lVert \dot{p} \circ \dot{\omega} \Bigr\rVert^{2} \le \left( \frac{1+\theta}{1-\theta} n \mu \right)^{2} $$

This proves (45). Using

$$\Bigl\lVert \frac{\ddot{p}}{p} \Bigr\rVert^{2} ={\sum}_{i=1}^{2n} \left( \frac{\ddot{p_{i}}}{p_{i}} \right)^{2}, \Bigl\lVert \frac{\ddot{\omega}}{\omega} \Bigr\rVert^{2} ={\sum}_{i=1}^{2n} \left( \frac{\ddot{\omega_{i}}}{\omega_{i}} \right)^{2}, $$

and Lemma 4.4, then following the same procedure, it is easy to verify (46). From (33) and (36), we have

$$ \begin{array}{@{}rcl@{}} & & \left( \frac{2n}{(1-\theta)} \right) \left( \frac{4(1+\theta)n^{2}}{(1-\theta)^{3}} \right) \ge \left( \Bigl\lVert \frac{\dot{p}}{p}\Bigr\rVert^{2}+ \Bigl\lVert \frac{\dot{\omega}}{\omega} \Bigr\rVert^{2} \right) \left( \Bigl\lVert \frac{\ddot{p}}{p}\Bigr\rVert^{2}+ \Bigl\lVert \frac{\ddot{\omega}}{\omega} \Bigr\rVert^{2} \right) \\ & \ge & \Bigl\lVert \frac{\ddot{p}}{p} \Bigr\rVert^{2} \Bigl\lVert \frac{\dot{\omega}}{\omega} \Bigr\rVert^{2} +\Bigl\lVert \frac{\dot{p}}{p} \Bigr\rVert^{2} \Bigl\lVert \frac{\ddot{\omega}}{\omega} \Bigr\rVert^{2} \\ & = &\left( {\sum}_{i=1}^{2n} \left( \frac{\ddot{p_{i}}}{p_{i}} \right)^{2} \right) \left( {\sum}_{i=1}^{2n} \left( \frac{\dot{\omega_{i}}}{\omega_{i}} \right)^{2} \right) +\left( {\sum}_{i=1}^{2n} \left( \frac{\dot{p_{i}}}{p_{i}} \right)^{2} \right) \left( {\sum}_{i=1}^{2n} \left( \frac{\ddot{\omega_{i}}}{\omega_{i}} \right)^{2} \right) \\ & \ge & {\sum}_{i=1}^{2n} \left( \frac{\ddot{p_{i}}\dot{\omega_{i}}}{p_{i}\omega_{i}} \right)^{2} +{\sum}_{i=1}^{2n} \left( \frac{\dot{p_{i}}\ddot{\omega_{i}}}{p_{i}\omega_{i}} \right)^{2} \\ & \ge & {\sum}_{i=1}^{2n} \left( \frac{\ddot{p_{i}}\dot{\omega_{i}}}{(1+\theta) \mu} \right)^{2} +{\sum}_{i=1}^{2n} \left( \frac{\dot{p_{i}}\ddot{\omega_{i}}}{(1+\theta) \mu} \right)^{2} \\ & = & \frac{1}{(1+\theta)^{2}\mu^{2}} \left( \Bigl\lVert \ddot{p} \circ \dot{\omega} \Bigr\rVert^{2} +\Bigl\lVert \dot{p} \circ \ddot{\omega} \Bigr\rVert^{2} \right), \\ \end{array} $$

(91)

i.e.,

$$ \Bigl\lVert \ddot{p} \circ \dot{\omega} \Bigr\rVert^{2} +\Bigl\lVert \dot{p} \circ \ddot{\omega} \Bigr\rVert^{2} \le \frac{(2n)^{3}(1+\theta)^{3}}{(1-\theta)^{4}} \mu^{2}. $$

This proves the lemma.□

Proof Proof of Lemma 4.9

First notice that $q(\sin \limits (\alpha ))$ is a monotonic increasing function of $\sin \limits (\alpha )$ for $\alpha \in [0, \frac {\pi }{2}]$ and $q(\sin \limits (0))<0$; therefore, we need only to show that $q(\frac {\theta }{\sqrt {n}}) <0$ for 𝜃 ≤ 0.22. Using Lemma 2.6, we have

$$ \Big\lVert \dot{p} \circ \ddot{\omega}+\dot{\omega} \circ \ddot{p} -\frac{1}{2n}(\dot{p}^{\mathrm{T}}\ddot{\omega}+\dot{\omega}^{\mathrm{T}}\ddot{p})e \Big\rVert \le \Big\lVert \dot{p} \circ \ddot{\omega} \Big\rVert+ \Big\lVert \dot{\omega} \circ \ddot{p} \Big\rVert, $$

$$ \Big\lVert \ddot{p} \circ \ddot{\omega}-\dot{\omega} \circ \dot{p} -\frac{1}{2n}(\ddot{p}^{\mathrm{T}}\ddot{\omega}-\dot{\omega}^{\mathrm{T}}\dot{p})e \Big\rVert \le \Big\lVert \ddot{p} \circ \ddot{\omega} \Big\rVert+ \Big\lVert \dot{\omega} \circ \dot{p} \Big\rVert. $$

In view of Lemmas 4.8, 4.3, and 4.4, from (42), we have, for $\alpha \in [0, \frac {\pi }{2}]$,

$$ \begin{array}{@{}rcl@{}} q(\sin(\alpha)) &\le & \left( \Big\lVert \ddot{p} \circ \ddot{\omega} \Big\rVert+ \Big\lVert \dot{\omega} \circ \dot{p} \Big\rVert +2\theta \frac{\dot{p}^{\mathrm{T}}\dot{\omega}}{2n} \right) \sin^{4}(\alpha) +\left( \Big\lVert \dot{p} \circ \ddot{\omega} \Big\rVert+ \Big\lVert \dot{\omega} \circ \ddot{p} \Big\rVert\right) \sin^{3}(\alpha) \\ & &+ 2\theta \frac{\dot{p}^{\mathrm{T}}\dot{\omega}}{2n}\sin^{2}(\alpha) +\theta \mu\sin(\alpha)-\theta \mu \\ &\le & \mu \Bigg(\left( \frac{2(1+\theta)^{2}}{(1-\theta)^{3}}n^{2}+\frac{n(1+\theta)}{(1-\theta)}+ \frac{\theta(1+\theta)}{(1-\theta)} \right) \sin^{4}(\alpha) +4\sqrt{2}\frac{(1+\theta)^{\frac{3}{2}}}{(1-\theta)^{2}}n^{\frac{3}{2}}\sin^{3}(\alpha) \\ && {\kern1.7pc}+ \frac{\theta(1+\theta)}{(1-\theta)} \sin^{2}(\alpha) +\theta \sin(\alpha)-\theta \Bigg). \end{array} $$

Since n ≥ 1 and 𝜃 > 0, substituting $\sin \limits (\alpha )=\frac {\theta }{\sqrt {n}}$ gives

$$ \begin{array}{@{}rcl@{}} q\Big(\frac{\theta}{\sqrt{n}} \Big) &\le & \mu \Bigg(\left( \frac{2(1+\theta)^{2}}{(1-\theta)^{3}}n^{2} +\frac{n(1+\theta)}{(1-\theta)}+ \frac{\theta(1+\theta)}{(1-\theta)} \right) \frac{\theta^{4}}{n^{2}} +4\sqrt{2}\frac{(1+\theta)^{\frac{3}{2}}n^{\frac{3}{2}}}{(1-\theta)^{2}} \frac{\theta^{3}}{n^{\frac{3}{2}}} \\ && {\kern2pc}+\frac{\theta(1+\theta)}{(1-\theta)} \frac{\theta^{2}}{n} +\theta \frac{\theta}{\sqrt{n}}-\theta \Bigg) \\ &= & \theta\mu\Bigg(\frac{2\theta^{3}(1+\theta)^{2}}{(1-\theta)^{3}} +\frac{ \theta^{3}(1+\theta)}{n(1-\theta)}+\frac{ \theta^{4}(1+\theta)}{(1-\theta)n^{2}} \\ && {\kern2pc}+\frac{4\sqrt{2} \theta^{2}(1+\theta)^{\frac{3}{2}}}{(1-\theta)^{2}} +\frac{ \theta^{2}(1+\theta)}{n(1-\theta)}+\frac{ \theta}{\sqrt{n}} -1 \Bigg) \\ &\le & \theta\mu\Bigg(\frac{2\theta^{3}(1+\theta)^{2}}{(1-\theta)^{3}} +\frac{ \theta^{3}(1+\theta)}{(1-\theta)}+\frac{ \theta^{4}(1+\theta)}{(1-\theta)} \\ & &{\kern1.6pc}+\frac{4\sqrt{2} \theta^{2}(1+\theta)^{\frac{3}{2}}}{(1-\theta)^{2}} +\frac{ \theta^{2}(1+\theta)}{(1-\theta)}+ { \theta} -1 \Bigg) :=\theta\mu p(\theta). \end{array} $$

(92)

Since p(𝜃) is a monotonic increasing function of 𝜃 ∈ [0, 1), p(0) < 0, it is easy to verify that p(0.22) < 0. This proves the lemma. □

Proof Proof of Lemma 4.10

Using Lemma 2.6, we have

$$ 0 \le \Big\lVert {\Delta} p \circ {\Delta} \omega - \frac{1}{2n}({\Delta} p^{\mathrm{T}}{\Delta} \omega) e \Big\rVert^{2} \le \| {\Delta} p \circ {\Delta} \omega \|^{2}. $$

(93)

Pre-multiplying $\Big (P(\alpha ){\Omega }(\alpha )\Big )^{-\frac {1}{2}}$ on both sides of (52) yields

$$ D {\Delta} \omega+D^{-1} {\Delta} p=\Big(P(\alpha){\Omega}(\alpha)\Big)^{-\frac{1}{2}} \Big(\mu(\alpha) e -P(\alpha){\Omega}(\alpha)e\Big). $$

Let u = DΔω, v = D^− 1Δp, from (49), we have

$$ u^{\mathrm{T}}v= {\Delta} p^{\mathrm{T}} {\Delta} \omega = {\Delta} y^{\mathrm{T}} {\Delta} \lambda + {\Delta} z^{\mathrm{T}} {\Delta} \gamma={\Delta} x^{\mathrm{T}} ({\Delta} \gamma - {\Delta} \lambda ) = {\Delta} x^{\mathrm{T}} H {\Delta} x \ge 0. $$

(94)

Use Lemma 2.4 and the assumption of $(x(\alpha ), p(\alpha ), \omega (\alpha )) \in \mathcal {N}_{2}(2\theta )$, we have

$$ \begin{array}{@{}rcl@{}} \Big\lVert {\Delta} p \circ {\Delta} \omega \Big\rVert & = & \Big\lVert u \circ v \Big\rVert \le 2^{-\frac{3}{2}} \Big\lVert \Big(P(\alpha){\Omega}(\alpha) \Big)^{-\frac{1}{2}} \Big(\mu(\alpha) e-P(\alpha){\Omega}(\alpha)e \Big) \Big\rVert^{2} \\ & = & 2^{-\frac{3}{2}} {\sum}_{i=1}^{2n} \frac{\left( \mu(\alpha)-p_{i}(\alpha)\omega_{i}(\alpha)\right)^{2}} {p_{i}(\alpha)\omega_{i}(\alpha)} \\ & \le & 2^{-\frac{3}{2}} \frac{\| \mu(\alpha) e-p(\alpha)\circ \omega(\alpha) \|^{2}} {\min_{i}{p_{i}(\alpha)\omega_{i}(\alpha)}} \\ & \le & 2^{-\frac{3}{2}} \frac{(2\theta)^{2}\mu(\alpha)^{2}} {(1-2\theta)\mu(\alpha)} =2^{\frac{1}{2}} \frac{\theta^{2}\mu(\alpha)}{(1-2\theta)}. \end{array} $$

(95)

Define (p^k+ 1(t),ω^k+ 1(t)) = (p(α),ω(α)) + t(Δp,Δω). From (52) and (26), we have

$$ p(\alpha)^{\mathrm{T}} {\Delta} \omega+ \omega(\alpha)^{\mathrm{T}} {\Delta} p = 2n \mu -{\sum}_{i=1}^{2n} p_{i}(\alpha) \omega_{i}(\alpha)=0. $$

(96)

Therefore,

$$ {\mu}^{k+1}(t)=\frac{ \Big(p(\alpha)+t{\Delta} p \Big)^{\mathrm{T}}\Big(\omega(\alpha)+t{\Delta} \omega \Big)}{2n} =\frac{ p(\alpha)^{\mathrm{T}}\omega(\alpha)+t^{2}{\Delta} p^{\mathrm{T}}{\Delta} \omega}{2n} = \mu(\alpha) + t^{2}\frac{ {\Delta} p^{\mathrm{T}}{\Delta} \omega}{2n}. $$

(97)

Since Δp^TΔω = Δx^THΔx ≥ 0, we conclude that μ^k+ 1(t) ≥ μ(α). Using (97), (52), (93), and (95), we have

$$ \begin{array}{@{}rcl@{}} \\ & & \Big\lVert p^{k+1}(t) \circ \omega^{k+1}(t) - {\mu}^{k+1}(t)e \Big\rVert \\ & = & \Big\lVert (p(\alpha)+t{\Delta} p) \circ (\omega(\alpha)+t{\Delta} \omega) - \mu(\alpha) e - \frac{t^{2}}{2n} \left( {\Delta} p^{\mathrm{T}} {\Delta} \omega \right) e \Big\rVert \\ & = & \Big\lVert p(\alpha)\circ \omega(\alpha) +t(\omega(\alpha) \circ {\Delta} p + p(\alpha) \circ {\Delta} \omega) + t^{2} {\Delta} p \circ {\Delta} \omega - \mu(\alpha) e -\frac{t^{2}}{2n} \left( {\Delta} p^{\mathrm{T}} {\Delta} \omega \right) e \Big\rVert \\ & = & \Big\lVert p(\alpha)\circ \omega(\alpha) +t(\mu(\alpha) e-p(\alpha)\circ \omega(\alpha)) + t^{2} {\Delta} p \circ {\Delta} \omega - \mu(\alpha) e -\frac{t^{2}}{2n} \left( {\Delta} p^{\mathrm{T}} {\Delta} \omega \right) e \Big\rVert \\ & = & \Big\lVert (1-t)\left( p(\alpha)\circ \omega(\alpha)- \mu(\alpha) e \right) + t^{2}\left( {\Delta} p \circ {\Delta} \omega - \frac{1}{2n}\left( {\Delta} p^{\mathrm{T}} {\Delta} \omega \right) e \right) \Big\rVert \\ & \le & (1-t)(2\theta) \mu(\alpha) + t^{2} \frac{2^{\frac{1}{2}}\theta^{2}}{(1-2\theta)}\mu(\alpha) \\ & \le & \left( (1-t)(2\theta) + t^{2} \frac{2^{\frac{1}{2}}\theta^{2}}{(1-2\theta)} \right) \mu^{k+1} :=f(t, \theta)\mu^{k+1}. \end{array} $$

(98)

Therefore, taking t = 1 gives $\Big \lVert p^{k+1} \circ \omega ^{k+1} - {\mu }^{k+1}e \Big \rVert \le \frac {2^{\frac {1}{2}}\theta ^{2}}{(1-2\theta )}\mu ^{k+1}$. It is easy to see that, for 𝜃 ≤ 0.29,

$$ \frac{2^{\frac{1}{2}}\theta^{2}}{(1-2\theta)} = 0.2832 < \theta. $$

For 𝜃 ≤ 0.29 and t ∈ [0, 1], noticing 0 ≤ f(t,𝜃) ≤ f(t, 0.29) ≤ 0.58(1 − t) + 0.2832t² < 1, and using Corollary 4.1, we have, for an additional condition $\sin \limits (\alpha ) \le \sin \limits ^{-1}(\bar {\alpha })$,

$$ \begin{array}{@{}rcl@{}} p_{i}^{k+1}(t)\omega_{i}^{k+1}(t) & \ge& \left( 1-f(t, \theta) \right) \mu^{k+1}(t) \\ & =&\left( 1-f(t, \theta) \right) \left( \mu(\alpha)+\frac{t^{2}}{n}{\Delta} p^{\mathrm{T}}{\Delta} \omega \right) \\ & \ge& \left( 1-f(t, \theta) \right)\mu(\alpha) \\ & >&0. \end{array} $$

(99)

Therefore, (p^k+ 1(t),ω^k+ 1(t)) > 0 for t ∈ [0, 1], i.e., (p^k+ 1,ω^k+ 1) > 0. This finishes the proof.□

Proof Proof of Lemma 4.11

The first inequality of (53) follows from (94). Pre-multiplying both sides of (52) by $P^{-\frac {1}{2}}(\alpha ){\Omega }^{-\frac {1}{2}}(\alpha )$ gives

$$ P^{-\frac{1}{2}}(\alpha){\Omega}^{\frac{1}{2}}(\alpha) {\Delta} p +P^{\frac{1}{2}}(\alpha){\Omega}^{-\frac{1}{2}}(\alpha) {\Delta} \omega =P^{-\frac{1}{2}}(\alpha){\Omega}^{-\frac{1}{2}}(\alpha)\Big(\mu(\alpha) e - P(\alpha){\Omega}(\alpha)e\Big). $$

Let $u=P^{-\frac {1}{2}}(\alpha ){\Omega }^{\frac {1}{2}}(\alpha ) {\Delta } p$, $v=P^{\frac {1}{2}}(\alpha ){\Omega }^{-\frac {1}{2}}(\alpha ) {\Delta } \omega$, and $w=P^{-\frac {1}{2}}(\alpha ){\Omega }^{-\frac {1}{2}}(\alpha )$ (μ(α)e − P(α)Ω(α)e), from (94), we have u^Tv = Δp^TΔω ≥ 0. Using Lemma 4.3 and the assumption of $(x(\alpha ), p(\alpha ), \omega (\alpha )) \in \mathcal {N}_{2}(2\theta )$, we have

$$ \begin{array}{@{}rcl@{}} && \| u\|^{2} + \| v \|^{2}={\sum}_{i=1}^{2n} \left( \frac{({\Delta} p_{i})^{2}\omega_{i}(\alpha)}{p_{i}(\alpha)} +\frac{({\Delta} \omega_{i})^{2}p_{i}(\alpha)}{\omega_{i}(\alpha)} \right) \\ &\le & \| w \|^{2} = {\sum}_{i=1}^{2n} \frac{(\mu(\alpha)-p_{i}(\alpha)\omega_{i}(\alpha))^{2}} {p_{i}(\alpha)\omega_{i}(\alpha)} \\ &\le & \frac{{\sum}_{i=1}^{2n} (\mu(\alpha)-p_{i}(\alpha)\omega_{i}(\alpha))^{2}} {\min_{i}{p_{i}(\alpha)\omega_{i}(\alpha)}} \\ &\le & \frac{(2\theta)^{2} \mu^{2}(\alpha)}{(1-2\theta)\mu(\alpha)} = \frac{(2\theta)^{2} \mu(\alpha)}{(1-2\theta)}. \\ \end{array} $$

(100)

Dividing both sides by μ(α) and using p_i(α)ω_i(α) ≥ μ(α)(1 − 2𝜃) yield

$$ \begin{array}{@{}rcl@{}} & &{\sum}_{i=1}^{2n} (1-2\theta)\left( \frac{({\Delta} p_{i})^{2}}{{p_{i}^{2}}(\alpha)} +\frac{({\Delta} \omega_{i})^{2}}{{\omega_{i}^{2}}(\alpha)} \right) \\ &= & (1-2\theta) \left( \Big\lVert \frac{\Delta p }{p(\alpha) } \Big\rVert^{2} +\Big\lVert \frac{\Delta \omega }{\omega(\alpha) } \Big\rVert^{2} \right) \\ &\le & \frac{(2\theta)^{2}}{(1-2\theta)}, \\ \end{array} $$

(101)

i.e.,

$$ \begin{array}{@{}rcl@{}} \Big\lVert \frac{\Delta p }{p(\alpha) } \Big\rVert^{2} +\Big\lVert \frac{\Delta \omega }{\omega(\alpha) } \Big\rVert^{2} \le \left( \frac{2\theta}{1-2\theta} \right)^{2}. \end{array} $$

(102)

Invoking Lemma 4.1, we have

$$ \begin{array}{@{}rcl@{}} \Big\lVert \frac{\Delta p }{p(\alpha) } \Big\rVert^{2} \cdot\Big\lVert \frac{\Delta \omega }{\omega(\alpha) } \Big\rVert^{2} \le \frac{1}{4} \left( \frac{2\theta}{1-2\theta}\right)^{4}. \end{array} $$

(103)

This gives

$$ \begin{array}{@{}rcl@{}} \Big\lVert \frac{\Delta p }{p(\alpha) } \Big\rVert \cdot\Big\lVert \frac{\Delta \omega }{\omega(\alpha) } \Big\rVert \le \frac{2\theta^{2}}{(1-2\theta)^{2}}. \end{array} $$

(104)

Using Cauchy-Schwarz inequality, we have

$$ \begin{array}{@{}rcl@{}} && \frac{({\Delta} p)^{\mathrm{T}}({\Delta} \omega)}{\mu(\alpha)} \\ &\le & {\sum}_{i=1}^{2n} \frac{|{\Delta} p_{i}||{\Delta} \omega_{i}|}{\mu(\alpha)} \\ &\le & (1+2\theta){\sum}_{i=1}^{2n} \frac{|{\Delta} p_{i}|}{p_{i}(\alpha)} \frac{|{\Delta} \omega_{i}|}{\omega_{i}(\alpha)} \\ &= & (1+2\theta)\Big\lvert \frac{\Delta p}{p(\alpha)} \Big\rvert^{\mathrm{T}} \Big\lvert \frac{\Delta \omega}{\omega(\alpha)} \Big\rvert \\ &\le & (1+2\theta) \Big\lVert \frac{\Delta p }{p(\alpha) } \Big\rVert \cdot\Big\lVert \frac{\Delta \omega }{\omega(\alpha) } \Big\rVert \\ &\le & \frac{2\theta^{2}(1+2\theta)}{(1-2\theta)^{2}}. \end{array} $$

(105)

Therefore,

$$ \begin{array}{@{}rcl@{}} \frac{({\Delta} p)^{\mathrm{T}}({\Delta} \omega)}{2n} \le \frac{\theta^{2}(1+2\theta)}{n(1-2\theta)^{2}}\mu(\alpha). \end{array} $$

(106)

This proves the lemma.□

Proof Proof of Lemma 4.13

Using Lemmas 4.12, 4.5, 2.2, 4.2, 4.3, and 4.4, and noticing $\ddot {p}^{\mathrm {T}}\ddot {\omega } \ge 0$ and $\dot {p}^{\mathrm {T}}\dot {\omega } \ge 0$, we have

$$ \begin{array}{@{}rcl@{}} {}\mu^{k+1}& \le &\mu(\alpha) \left( 1+\frac{\theta^{2}(1+2\theta)}{n(1-2\theta)^{2}} \right) =\mu(\alpha) \left( 1+\frac{\delta_{0}}{n}\right) \end{array} $$

(107a)

$$ \begin{array}{@{}rcl@{}} &= & \mu^{k} \left( 1-\sin({\alpha})+ \left( \frac{\ddot{p}^{\mathrm{T}}\ddot{\omega}}{2n\mu}- \frac{\dot{p}^{\mathrm{T}}\dot{\omega}}{2n\mu} \right)(1-\cos(\alpha))^{2}\right.\\ &&{\kern14pt}\left.-\left( \frac{\dot{p}^{\mathrm{T}}\ddot{\omega}}{2n\mu}+ \frac{\dot{\omega}^{\mathrm{T}}\ddot{p}}{2n\mu} \right)\sin(\alpha)(1-\cos(\alpha)) \right) \left( 1+\frac{\delta_{0}}{n}\right) \\ &\le & \mu^{k} \left( 1-\sin({\alpha})+ \frac{\ddot{p}^{\mathrm{T}}\ddot{\omega}}{2n\mu}\sin^{4}(\alpha) +\left( \left| \frac{\dot{p}^{\mathrm{T}}\ddot{\omega}}{2n\mu} \right| + \left| \frac{\dot{\omega}^{\mathrm{T}}\ddot{p}}{2n\mu} \right| \right)\sin^{3}(\alpha) \right) \left( 1+\frac{\delta_{0}}{n}\right) \\ &\le & \mu^{k} \left( 1-\sin(\alpha)+\frac{n(1+\theta)^{2}}{(1-\theta)^{3}}\sin^{4}(\alpha) +\frac{2(2n)^{\frac{1}{2}}(1+\theta)^{\frac{3}{2}}}{(1-\theta)^{2}}\sin^{3}(\alpha) \right) \left( 1+\frac{\delta_{0}}{n}\right) \end{array} $$

(107b)

Substituting $\sin \limits (\alpha ) =\frac {\theta }{\sqrt {n}}$ into (107b) gives

$$ \begin{array}{@{}rcl@{}} \mu^{k+1}{} &\le &{} \mu^{k} \left( 1-\frac{\theta}{\sqrt{n}}+\frac{n(1+\theta)^{2}}{(1-\theta)^{3}}\frac{\theta^{4}}{{n}^{2}} +\frac{2(2n)^{\frac{1}{2}}(1+\theta)^{\frac{3}{2}}}{(1-\theta)^{2}}\frac{\theta^{3}}{{n}^{\frac{3}{2}}} \right) \left( 1+\frac{\delta_{0}}{n}\right) \\ {}&=&{} \mu^{k} \left( 1-\frac{\theta}{\sqrt{n}}+\frac{\theta^{4}(1+\theta)^{2}}{n(1-\theta)^{3}} +\frac{2^{\frac{3}{2}}\theta^{3}(1+\theta)^{\frac{3}{2}}}{n(1-\theta)^{2}} \right) \left( 1+\frac{\delta_{0}}{n}\right) \\ {}&=& {}\mu^{k} \left( 1-\frac{\theta}{\sqrt{n}}+\frac{\delta_{0}}{n} +\frac{\theta^{4}(1+\theta)^{2}}{n(1-\theta)^{3}} +\frac{2^{\frac{3}{2}}\theta^{3}(1+\theta)^{\frac{3}{2}}}{n(1-\theta)^{2}} -\frac{\theta \delta_{0}}{n^{\frac{3}{2}}} +\frac{\delta_{0}}{n}\left[ \frac{\theta^{4}(1+\theta)^{2}}{n(1-\theta)^{3}} +\frac{2^{\frac{3}{2}}\theta^{3}(1+\theta)^{\frac{3}{2}}}{n(1-\theta)^{2}} \right] \right) \\ {}&=& {}\mu^{k} \left( {}1-\frac{\theta}{\sqrt{n}} {}\left[ {}1{\kern-.5pt}-{\kern-.5pt}\frac{\delta_{0}}{\sqrt{n} \theta} {\kern-.5pt}-{\kern-.5pt}\frac{\theta^{3}(1{\kern-.5pt}+{\kern-.5pt}\theta)^{2}}{\sqrt{n}(1{\kern-.5pt}-{\kern-.5pt}\theta)^{3}} -\frac{2^{\frac{3}{2}}\theta^{2}(1{\kern-.5pt}+{\kern-.5pt}\theta)^{\frac{3}{2}}}{\sqrt{n}(1-\theta)^{2}} {}\right]{}-\frac{\theta \delta_{0}}{n^{\frac{3}{2}}} \left[ {}1- \frac{\theta^{3}(1{}+{}\theta)^{2}}{\sqrt{n}(1{}-{}\theta)^{3}} -\frac{2^{\frac{3}{2}}\theta^{2}(1{}+{}\theta)^{\frac{3}{2}}}{\sqrt{n}(1{}-{}\theta)^{2}} {}\right]{} \right) \end{array} $$

Since

$$ 1- \frac{\theta^{3}(1+\theta)^{2}}{\sqrt{n}(1-\theta)^{3}} -\frac{2^{\frac{3}{2}}\theta^{2}(1+\theta)^{\frac{3}{2}}}{\sqrt{n}(1-\theta)^{2}} \ge 1- \frac{\theta^{3}(1+\theta)^{2}}{(1-\theta)^{3}} -\frac{2^{\frac{3}{2}}\theta^{2}(1+\theta)^{\frac{3}{2}}}{(1-\theta)^{2}} := f(\theta), $$

where f(𝜃) is a monotonic decreasing function of 𝜃, it follows that for 𝜃 ≤ 0.37, f(𝜃) > 0. Therefore, for 𝜃 ≤ 0.37,

$$ \begin{array}{@{}rcl@{}} \mu^{k+1} \le & \mu^{k} \left( 1-\frac{\theta}{\sqrt{n}} \left[ 1-\frac{\delta_{0}}{\sqrt{n} \theta} -\frac{\theta^{3}(1+\theta)^{2}}{\sqrt{n}(1-\theta)^{3}} -\frac{2^{\frac{3}{2}}\theta^{2}(1+\theta)^{\frac{3}{2}}}{\sqrt{n}(1-\theta)^{2}} \right] \right) \\ =& \mu^{k} \left( 1-\frac{\theta}{\sqrt{n}} \left[ 1-\frac{\theta (1+2\theta)}{\sqrt{n}(1-2\theta)^{2}} -\frac{\theta^{3}(1+\theta)^{2}}{\sqrt{n}(1-\theta)^{3}} -\frac{2^{\frac{3}{2}}\theta^{2}(1+\theta)^{\frac{3}{2}}}{\sqrt{n}(1-\theta)^{2}} \right] \right) \end{array} $$

(108)

Since

$$ 1-\frac{\theta (1+2\theta)}{\sqrt{n}(1-2\theta)^{2}} -\frac{\theta^{3}(1+\theta)^{2}}{\sqrt{n}(1-\theta)^{3}} -\frac{2^{\frac{3}{2}}\theta^{2}(1+\theta)^{\frac{3}{2}}}{\sqrt{n}(1-\theta)^{2}} \ge 1-\frac{\theta (1+2\theta)}{(1-2\theta)^{2}} -\frac{\theta^{3}(1+\theta)^{2}}{(1-\theta)^{3}} -\frac{2^{\frac{3}{2}}\theta^{2}(1+\theta)^{\frac{3}{2}}}{(1-\theta)^{2}}:=g(\theta), $$

where g(𝜃) is a monotonic decreasing function of 𝜃, it follows that for 𝜃 ≤ 0.19, g(𝜃) > 0.0976 > 0. For 𝜃 = 0.19, 𝜃g(𝜃) > 0.0185 and

$$ \mu^{k+1} \le \mu^{k} \left( 1-\frac{0.0185}{\sqrt{n}} \right). $$

This proves (55). □