Appendix
1.1 Proof of Lemma 1
Proof
(1) For the M-P law \(F^{y}\left( 1/(x+1)\right) \), according to Proposition 2.10 in Yao et al. (2015), we can deduce that
$$\begin{aligned} F^{y}\left( \frac{1}{1+x}\right)&=\int _{(1-\sqrt{y})^2}^{(1+\sqrt{y})^2}\frac{\frac{1}{1+x}}{2\pi xy}\sqrt{(x-(1-\sqrt{y})^2)((1+\sqrt{y})^2-x)} dx\\&=-\frac{1}{4\pi i} \oint _{|z|=1} \frac{(1-z^2)^2}{\left( 1+|1+\sqrt{y}z|^2\right) z^2(1+\sqrt{y}z)(z+\sqrt{y})}dz. \end{aligned}$$
Observe that on the unit circle \(|z|=1\), \(1+|1+\sqrt{y}z|^2=1+\left( 1+\sqrt{y}z\right) \left( 1+\sqrt{y}/z\right) \). Setting \(z\left( 1+|1+\sqrt{y}z|^2\right) \)=\(\sqrt{y}z^2+(y+2)z+\sqrt{y}=0\), the two roots are
$$\begin{aligned} y_1=\frac{-(y+2)+\sqrt{y^2+4}}{2\sqrt{y}}\in (-1,0),~~~ y_2=\frac{-(y+2)-\sqrt{y^2+4}}{2\sqrt{y}}\notin (-1,0). \end{aligned}$$
Conclusively, by Cauchy’s residue theorem, we arrive at
$$\begin{aligned}&F^{y}\left( \frac{1}{1+x}\right) \nonumber \\&\quad =-\frac{1}{4\pi i} \oint _{|z|=1} \frac{(1-z^2)^2}{y(z-y_1)(z-y_2)z(z+\frac{1}{\sqrt{y}})(z+\sqrt{y})}dz\nonumber \\&\quad =-\frac{1}{2}\left[ \frac{1}{y}+\frac{y-1}{y(y_1+\sqrt{y})(y_2+\sqrt{y})}+\frac{(1-y_1^2)^2}{y(y_1-y_2)y_1\left( y_1+\frac{1}{\sqrt{y}}\right) \left( y_1+\sqrt{y}\right) }\right] , \end{aligned}$$
(16)
where in the contour integral \(z=0\), \(z=-\sqrt{y}\), \(z=y_1\) are poles of order one, whereas \(y_2\notin (-1,0)\) and hence is not a isolated singular point.
Next we calculate the M-P law of \(\log (x+1)\). As \(y\in (0,+\infty )\), it is straightforward to verify that
$$\begin{aligned} \log {\left( |1+\sqrt{y}z|^2+1\right) }= \log \left[ (1+a_{1}(y)z)(b_{1}(y)+\sqrt{y}z^{-1})\right] , \end{aligned}$$
where
$$\begin{aligned} a_{1}(y):=\frac{y+2-\sqrt{y^2+4}}{2\sqrt{y}}\in (0,1),~~~ b_{1}(y):=\frac{2y}{y+2-\sqrt{y^2+4}}=\frac{\sqrt{y}}{a_1(y)}\notin (0,1). \end{aligned}$$
Then, according to Proposition 2.10 in Yao et al. (2015) and Cauchy’s residue theorem, we have
$$\begin{aligned}&F^{y}\left( \log \left( x+1\right) \right) \nonumber \\&\quad =\int _{(1-\sqrt{y})^2}^{(1+\sqrt{y})^2}\frac{\log \left( x+1\right) }{2\pi xy}\sqrt{(x-(1-\sqrt{y})^2)((1+\sqrt{y})^2-x)} dx\nonumber \\&\quad =-\frac{1}{4\pi i} \oint _{|z|=1} \frac{\log \left( |1+\sqrt{y}z|^2+1\right) (1-z^2)^2}{z^2(1+\sqrt{y}z)(z+\sqrt{y})}dz\nonumber \\&\quad =-\frac{1}{4\pi i}\left[ \oint _{|z|=1} \frac{\log (1+a_{1}(y)z)(1-z^2)^2}{z^2(1+\sqrt{y}z)(z+\sqrt{y})}dz \right. \nonumber \\&\left. \qquad +\oint _{|z|=1}\frac{\log (b_{1}(y)+\sqrt{y}z)(z+1)^2(z-1)^2}{(z+\sqrt{y})\left( z+\frac{1}{\sqrt{y}}\right) \sqrt{y}z^2}dz\right] \nonumber \\&\quad =-\frac{1}{2}\Bigg [\frac{2 a_{1}(y)}{\sqrt{y}}+\frac{1-y}{y}\log \left( 1-a_{1}(y)\sqrt{y}\right) \nonumber \\&\qquad +\frac{1-y}{y}\log \left( \frac{\sqrt{y}}{a_{1}(y)}-y\right) -\frac{y+1}{y}\log \left( \frac{\sqrt{y}}{a_{1}(y)}\right) \Bigg ], \end{aligned}$$
(17)
where in the left integral of the third equality \(z=0\) and \(z=-\sqrt{y}\) are poles of order one, in the right integral of the third equality \(z=-\sqrt{y}\) is a pole of order one and \(z=0\) is a pole of order two. Note that \(-1/a_{1}(y)\) and \(-b_{1}(y)/\sqrt{y}\) are not in the unit circle and hence are not isolated singular points.
(2) When \(p/n\rightarrow y\in [1,+\infty )\), for the M-P law \(F^{y}(1/(x+1))\), utilizing Proposition 2.10 in Yao et al. (2015) again, we have
$$\begin{aligned} F^{y}\left( \frac{1}{1+x}\right)&=\int _{(1-\sqrt{y})^2}^{(1+\sqrt{y})^2}\frac{\frac{1}{1+x}}{2\pi xy}\sqrt{(x-(1-\sqrt{y})^2)((1+\sqrt{y})^2-x)} dx+\left( 1-\frac{1}{y}\right) \\&=-\frac{1}{4\pi i} \oint _{|z|=1} \frac{(1-z^2)^2}{\left( 1+|1+\sqrt{y}z|^2\right) z^2(1+\sqrt{y}z)(z+\sqrt{y})}dz+\left( 1-\frac{1}{y}\right) . \end{aligned}$$
Notice the term \(1-1/y\), which is an additional point mass at zero. In the case of \(y\in [1,\infty )\), by setting \(z\left( 1+|1+\sqrt{y}z|^2\right) \)=\(\sqrt{y}z^2+(y+2)z+\sqrt{y}=0\), the two roots are
$$\begin{aligned} y_1=\frac{-(y+2)+\sqrt{y^2+4}}{2\sqrt{y}}\in (-1,0),~~~ y_2=\frac{-(y+2)-\sqrt{y^2+4}}{2\sqrt{y}}\notin (-1,0). \end{aligned}$$
It turns out that by Cauchy’s residue theorem
$$\begin{aligned}&F^{y}\left( \frac{1}{1+x}\right) \nonumber \\&\quad =-\frac{1}{4\pi i} \oint _{|z|=1} \frac{(1-z^2)^2}{y(z-y_1)(z-y_2)z(z+\frac{1}{\sqrt{y}})(z+\sqrt{y})}dz+\left( 1-\frac{1}{y}\right) \nonumber \\&\quad =-\frac{1}{2}\left[ \frac{1}{y}+\frac{1-y}{y^2(y_1+\frac{1}{\sqrt{y}})(y_2+\sqrt{y})}+\frac{(1-y_1^2)^2}{y(y_1-y_2)y_1\left( y_1+\frac{1}{\sqrt{y}}\right) \left( y_1+\sqrt{y}\right) }\right] \nonumber \\&\qquad +\left( 1-\frac{1}{y}\right) , \end{aligned}$$
(18)
where in the contour integral \(z=0\), \(z=-1/\sqrt{y}\) and \(z=y_1\) are poles of order one.
The final step is to compute the M-P law of \(\log (x+1)\), according to Proposition 2.10 in Yao et al. (2015) and Cauchy’s residue theorem, we can show that
$$\begin{aligned}&F^{y}\left( \log (x+1)\right) \nonumber \\&\quad =\int _{(1-\sqrt{y})^2}^{(1+\sqrt{y})^2}\frac{\log \left( x+1\right) }{2\pi xy}\sqrt{(x-(1-\sqrt{y})^2)((1+\sqrt{y})^2-x)} dx\nonumber \\&\quad =-\frac{1}{4\pi i}\left[ \oint _{|z|=1} \frac{\log (1+a_1(y)z)(1-z^2)^2}{z^2(1+\sqrt{y}z)(z+\sqrt{y})}dz \right. \nonumber \\&\left. \qquad +\oint _{|z|=1}\frac{\log (b_1(y)+\sqrt{y}z)(z+1)^2(z-1)^2}{(z+\sqrt{y})\left( z+\frac{1}{\sqrt{y}}\right) \sqrt{y}z^2}dz\right] \nonumber \\&\quad =-\frac{1}{2}\Bigg [\frac{2 a_1(y)}{\sqrt{y}}+\frac{y-1}{y}\log \left( 1-\frac{a_1(y)}{\sqrt{y}}\right) \nonumber \\&\qquad +\frac{y-1}{y}\log \left( \frac{\sqrt{y}}{a_1(y)}-1\right) -\frac{y+1}{y}\log \left( \frac{\sqrt{y}}{a_1(y)}\right) \Bigg ]. \end{aligned}$$
(19)
It is worthy noting that when \(y=1\), the same conclusion also holds by direct evaluation. The proof is completed. \(\square \)
1.2 Proof of Theorem 1
Proof
Rewrite T as
$$\begin{aligned} T=\frac{|{{\mathbf {S}}}_n+\frac{{\text {tr}}({{\mathbf {S}}}_n)}{p}{{\mathbf {I}}}_p|^{\frac{n}{2}}}{\left( {\text {tr}}({{\mathbf {S}}}_n)\right) ^{\frac{np}{2}}} =\frac{|{{\mathbf {S}}}_n+{{\mathbf {I}}}_p|^{\frac{n}{2}}}{\left( {\text {tr}}({{\mathbf {S}}}_n)\right) ^{\frac{np}{2}}}\left| {{\mathbf {I}}}_p +\left( \frac{{\text {tr}}\left( {{\mathbf {S}}}_n\right) }{p}-1\right) \left( {{\mathbf {S}}}_n+{{\mathbf {I}}}_p\right) ^{-1}\right| ^{\frac{n}{2}}. \end{aligned}$$
It follows that
$$\begin{aligned} -\frac{2}{n}\log T=-\sum _{i=1}^p\log \left( l_i+1\right) +p\log \left( \sum _{i=1}^p l_i\right) -\sum _{i=1}^p\log \left( 1+\frac{\sum \nolimits _{i=1}^p \frac{l_i}{p}-1}{l_i+1}\right) . \end{aligned}$$
Define \(a=(1-\sqrt{y})^2\) and \(b=(1+\sqrt{y})^2\). Let \({\mathcal {U}}\) be an open set of the complex plane \({\mathbb {C}}\) containing \([\lim \inf _n\lambda _{\min }({\varvec{\varSigma }}_p)I_{(0,1)}(y)(1-\sqrt{y})^2,\lim \sup _n\lambda _{\max }({\varvec{\varSigma }}_p)(1+\sqrt{y})^2]\), and \({\mathcal {M}}\) be the set of analytic functions \(f: {\mathcal {U}} \rightarrow {\mathbb {R}}\). For any \(f \in {\mathcal {A}}\), consider the centralized LSS with form
$$\begin{aligned} G_n(f)=p\int _{-\infty }^{+\infty } f(x) d\left( F^{{{\mathbf {S}}}_n}(x)-F^{y_n,H_p}(x)\right) , \end{aligned}$$
(20)
where \(F^{{{\mathbf {S}}}_n}\) is the ESD of \({{\mathbf {S}}}_n\) and \(F^{y_n,H_p}\) is the finite-horizon proxy of LSD \(F^{y,H}\). Under Assumption 1–3, by Bai and Silverstein (2004) and Pan and Zhou (2008), \(G_n(f)\) converges weakly to a Gaussian distribution. When \({\varvec{\varSigma }}_p={{\mathbf {I}}}_p\), for convenience, denote \(F^{{{\mathbf {S}}}_n}\) by \(F_n\) and represent \(F^{y_n,H_p}\) as \(F^{y_n}\), the standard M-P law with index \(y_n\). Define \(g(x)=x\), it has been established in Wang and Yao (2013) that as \(y\in (0,\infty )\), \(G_n(g)={\text {tr}}({{\mathbf {S}}}_n)-p\Rightarrow N(0,(\kappa +\beta )y)\) under \(H_0\), where \(\Rightarrow \) stands for convergence in distribution. As a consequence,
$$\begin{aligned} p\log \left( \sum _{i=1}^p l_i\right)= & {} p\log \left( 1+\frac{1}{p}\left( \sum _{i=1}^p l_i-p\right) \right) +p\log p\nonumber \\= & {} \sum _{i=1}^p l_i-p+p\log p+{\mathcal {O}}_p\left( \frac{1}{p}\right) . \end{aligned}$$
(21)
Further define \(f_1(x)=1/(1+x)\), \(f_2(x)=1/(1+x)^2\). In the light of the convergence of \(G_n(f_1)\) and \(G_n(f_2)\), we have
$$\begin{aligned} G_n(f_1)\frac{{\text {tr}}({{\mathbf {S}}}_n)-p}{p}&=p\left\{ \frac{1}{p}\sum _{i=1}^p\frac{1}{1+l_i}-F^{y_n}\left( \frac{1}{1+x}\right) \right\} \frac{{\text {tr}}({{\mathbf {S}}}_n)-p}{p}=o_p\left( 1\right) ,\\ G_n(f_2)\left( \frac{{\text {tr}}({{\mathbf {S}}}_n)-p}{p}\right) ^2&=p\left\{ \frac{1}{p}\sum _{i=1}^p\frac{1}{(1+l_i)^2}-F^{y_n}\left( \frac{1}{(1+x)^2}\right) \right\} \frac{({\text {tr}}({{\mathbf {S}}}_n)-p)^2}{p^2}=o_p\left( 1\right) . \end{aligned}$$
Accordingly, using the Taylor expansion on \(\log (1+x)\), we derive that
$$\begin{aligned} \sum _{i=1}^p\log \left( 1+\frac{\frac{{\text {tr}}({{\mathbf {S}}}_n)}{p}-1}{l_i+1}\right)= & {} \sum _{i=1}^p \frac{\frac{{\text {tr}}({{\mathbf {S}}}_n)}{p}-1}{l_i+1}+{\mathcal {O}}_p\left( \sum _{i=1}^p\frac{\left( \frac{{\text {tr}}({{\mathbf {S}}}_n)}{p}-1\right) ^2}{(l_i+1)^2}\right) \nonumber \\\doteq & {} \left( {\text {tr}}({{\mathbf {S}}}_n)-p\right) F^{y_n}\left( \frac{1}{1+x}\right) , \end{aligned}$$
(22)
where “\(\doteq \)” means asymptotic equivalence up to \(o_p(1)\). Combining (21) and (22), we obtain
$$\begin{aligned} -\frac{2}{n}\log T\doteq \left( 1-F^{y_n}\left( \frac{1}{1+x}\right) \right) \left( \sum _{i=1}^p l_i-p\right) -\sum _{i=1}^p \log \left( l_i+1\right) +p\log p. \end{aligned}$$
(23)
Let \(u(x)=\log (x+1)\), \(h=\sqrt{y}\), next we work with the form \(\sum _{i=1}^p l_i\) and \(\sum _{i=1}^p \log \left( l_i+1\right) \). Firstly, decompose them as
$$\begin{aligned} \sum _{i=1}^{p}l_i&=p\int g(x) d\left( F_n(x)-F^{y_n}(x)\right) +pF^{y_n}(g)\triangleq X_n(g)+p,\\ \sum _{i=1}^p \log \left( l_i+1\right)&=p\int u(x) d\left( F_n(x)-F^{y_n}(x)\right) +pF^{y_n}(u)\triangleq X_n(u)+pF^{y_n}(u). \end{aligned}$$
Then by virtue of Proposition A.1 in Wang and Yao (2013), one can show that
$$\begin{aligned} \left( \begin{array}{cc} X_n(g) \\ X_n(u) \\ \end{array} \right) \Longrightarrow N\left( \left( \begin{array}{cc} \mathrm{E}X_{g} \\ \mathrm{E}X_{u} \\ \end{array} \right) , \ \left( \begin{array}{cc} \mathop {\text {Cov}}(X_g,X_g) &{} \mathop {\text {Cov}}(X_g, X_u) \\ \mathop {\text {Cov}}(X_u,X_g) &{} \mathop {\text {Cov}}(X_u,X_u) \end{array} \right) \ \right) , \end{aligned}$$
where
$$\begin{aligned} \mathrm{E}X_g&=0,\nonumber \\ \mathrm{E}X_u&=(\kappa -1) I_1(u)+\beta I_2(u) , \end{aligned}$$
(24)
$$\begin{aligned} \mathop {\text {Cov}}(X_g,X_g)&=(\kappa +\beta )y,\nonumber \\ \mathop {\text {Cov}}(X_g,X_u)&=\kappa J_1(g,u)+\beta J_2(g,u) , \end{aligned}$$
(25)
$$\begin{aligned} \mathop {\text {Cov}}(X_u,X_u)&=\kappa J_1(u,u)+\beta J_2(u,u) , \end{aligned}$$
(26)
with
$$\begin{aligned} I_1(u)&=\lim _{r\downarrow 1} I_1(u,r)=\lim _{r\downarrow 1}\left[ \frac{1}{2\pi i}\oint _{|\xi |=1}u(|1+h\xi |^{2})\left[ \frac{\xi }{\xi ^{2}-r^{-2}}-\frac{1}{\xi }\right] d\xi \right] , \\ I_2(u)&=\frac{1}{2\pi i}\oint _{|\xi |=1}u(|1+h\xi |^{2})\frac{1}{\xi ^{3}}d\xi , \\ J_1(g,u)&=\lim _{r\downarrow 1} J_1(g,u,r)=\lim _{r\downarrow 1}\left[ -\frac{1}{4\pi ^{2}}\oint _{|\xi _{1}|=1}\oint _{|\xi _{2}|=1} \frac{g(|1+h\xi _{1}|^{2})u(|1+h\xi _{2}|^{2})}{(\xi _{1}-r\xi _{2})^{2}}d\xi _{1}d\xi _{2}\right] ,\\ J_2(g,u)&=-\frac{1}{4\pi ^{2}}\oint _{|\xi _{1}|=1}\frac{g(|1+h\xi _{1}|^{2})}{\xi _{1}^{2}}d\xi _{1}\oint _{|\xi _{2}|=1}\frac{u(|1+h\xi _{2}|^{2})}{\xi _{2}^{2}}d\xi _{2}, \end{aligned}$$
where r is larger than 1 but close to 1, \(J_1(u,u)\) and \(J_2(u,u)\) are the same to \(J_1(g,u)\) and \(J_2(g,u)\) with g replaced by u. The contours in \(J_1\) are non overlapping and all contours are oriented counterclockwise.
Given this result, the remainder of the proof is to derive explicit expressions of the limiting parameters. Recall that on the unit circle \(|z|=1\), \((1+\sqrt{y}z)(1+\sqrt{y}z^{-1})=1+\sqrt{y}z^{-1}+\sqrt{y}z+y\). Moreover, as \(y\in (0,1)\), it is straightforward to verify that
$$\begin{aligned} \log {\left( |1+\sqrt{y}z|^2+1\right) }= \log \left[ (1+a_{1}(y)z)(b_{1}(y)+\sqrt{y}z^{-1})\right] , \end{aligned}$$
where
$$\begin{aligned} a_{1}(y):=\frac{y+2-\sqrt{y^2+4}}{2\sqrt{y}}\in (0,1),~~~ b_{1}(y):=\frac{2y}{y+2-\sqrt{y^2+4}}=\frac{\sqrt{y}}{a_1(y)}\notin (0,1). \end{aligned}$$
For (24), by Cauchy’s residue theorem,
$$\begin{aligned} I_{2}(u)&=\frac{1}{2\pi i}\oint _{|z|=1}\frac{\log \left[ (1+a_{1}(y)z)(b_{1}(y)+\sqrt{y}z^{-1})\right] }{z^3} dz\\&=\frac{1}{2\pi i}\left[ \oint _{|z|=1}\frac{\log (1+a_{1}(y)z)}{z^3}dz+\oint _{|z|=1}\frac{\log (b_{1}(y)+\sqrt{y}z^{-1})}{z^3}dz\right] \\&=-\frac{1}{2}a_{1}^{2}(y)+\oint _{|\xi |=1} \log (b_{1}(y)+\sqrt{y}\xi )\xi d\xi \\&=-\frac{1}{2}a_{1}^{2}(y), \end{aligned}$$
where in the left integral of the second equality \(z=0\) is a pole of order two, \(-1/a_{1}(y)\) and \(-b_{1}(y)/\sqrt{y}\) are not in the unit circle and hence are not isolated singular points.
$$\begin{aligned} I_{1}(u,r)&=\frac{1}{2\pi i}\oint _{|\xi |=1}\log \left[ (1+a_{1}(y)\xi )(b_{1}(y)+\sqrt{y} \xi ^{-1})\right] \left[ \frac{\xi }{\xi ^2-r^{-2}}-\frac{1}{\xi }\right] d\xi \\&=\frac{1}{2\pi i}\Biggl [\oint _{|\xi |=1} \frac{\log (1+a_{1}(y)\xi )}{\xi ^{2}-r^{-2}}\cdot \xi d\xi +\oint _{|\xi |=1} \frac{\log (b_{1}(y)+\sqrt{y}{\xi }^{-1})}{\xi ^{2}-r^{-2}}\cdot \xi d\xi \\&\quad -\oint _{|\xi |=1}\frac{\log (b_{1}(y)+\sqrt{y}{\xi }^{-1})}{\xi } d\xi \Biggl ], \end{aligned}$$
where
$$\begin{aligned} \frac{1}{2\pi i}\oint _{|\xi |=1} \frac{\log (1+a_{1}(y)\xi )}{\xi ^{2}-r^{-2}}\cdot \xi d\xi&=\frac{1}{2}\log \left( 1+a_{1}(y)\frac{1}{r}\right) +\frac{1}{2}\log \left( 1-a_{1}(y)\frac{1}{r}\right) \\&=\frac{1}{2}\log \left( 1-\frac{a_{1}^{2}(y)}{r^2}\right) , \end{aligned}$$
and via the change of variable \(z=1/\xi ,\)
$$\begin{aligned} \frac{1}{2\pi i}\oint _{|\xi |=1} \frac{\log (b_{1}(y)+\sqrt{y}{\xi }^{-1})}{\xi ^{2}-r^{-2}}\cdot \xi d\xi&=-\frac{1}{2\pi i}\oint _{|z|=1}\frac{\log (b_{1}(y)+\sqrt{y}z)}{z\cdot r^{-2}(z^{2}-r^{2})} dz\\&=\log (b_{1}(y)),\\ \frac{1}{2\pi i}\oint _{|\xi |=1} \frac{\log (b_{1}(y)+\sqrt{y}\xi ^{-1})}{\xi }d\xi&=\frac{1}{2\pi i}\oint _{|z|=1}\frac{\log (b_{1}(y)+\sqrt{y}z)}{z}dz\\&=\log (b_{1}(y)). \end{aligned}$$
As a consequence,
$$\begin{aligned} I_{1}(u,r)&=\frac{1}{2}\log \left( 1-\frac{1}{r^2}a_{1}^2(y)\right) ,\\ \mathrm{E}X_u=(\kappa -1) I_1(u)+\beta I_2(u)&=(\kappa -1)\left( \frac{1}{2}\log (1-a_{1}^2(y))\right) +\beta \left( -\frac{1}{2}a^2_{1}(y)\right) . \end{aligned}$$
For (25), compute \(J_1(g,u,r)\) and \(J_2(g,u)\) as
$$\begin{aligned} J_1(g,u,r)&=-\frac{1}{4 \pi ^2} \oint _{|\xi _1|=1}\oint _{|\xi _2|=1} \frac{g\left( |1+h\xi _1|^2\right) \cdot u\left( |1+h \xi _2|^2\right) }{(\xi _1-r\xi _2)^2} d\xi _1 d\xi _2\\&=-\frac{1}{4 \pi ^2} \oint _{|\xi _2|=1} \left[ \oint _{|\xi _1|=1} \frac{(1+h\xi _1)(\xi _1+h)}{(\xi _1-r\xi _2)^2\xi _1} d\xi _1\right] \log \left( |1+h\xi _2|^2+1\right) \\&\qquad d\xi _2~~(|r\xi _2|>1)\\&=\frac{1}{2\pi i}\oint _{|\xi _2|=1}\frac{\log \left[ (1+a_{1}(y)\xi _2)(b_{1}(y)+\sqrt{y}\xi _2^{-1})\right] }{r^2\xi _2^2} \cdot h d\xi _2\\&=\frac{h}{r^2}\frac{1}{2\pi i} \oint _{|\xi _2|=1} \left[ \frac{\log (1+a_{1}(y)\xi _2)}{\xi _2^2}+\frac{\log (b_{1}(y)+\sqrt{y}\xi _2^{-1})}{\xi _2^2}\right] d\xi _2\\&=\frac{h}{r^2} a_{1}(y),\\ J_2(g,u)&=-\frac{1}{4\pi ^2}\oint _{|\xi _1|=1} \frac{|1+h\xi _1|^2}{\xi _1^2}d\xi _1 \\&\qquad \cdot \oint _{|\xi _2|=1}\frac{\log (1+a_{1}(y)\xi _2)+\log (b_{1}(y)+\sqrt{y}\xi _2^{-1})}{\xi _2^2} d\xi _2\\&=-\frac{1}{4 \pi ^2} \oint _{|\xi _1|=1} \frac{(1+h\xi _1)(\xi _1+h)}{\xi _1^3}d\xi _1 \cdot 2\pi i\cdot a_{1}(y)\\&=a_{1}(y)h. \end{aligned}$$
As a result,
$$\begin{aligned} \mathrm{Cov}(X_g,X_u)=\kappa \sqrt{y} a_{1}(y)+\beta \sqrt{y} a_{1}(y). \end{aligned}$$
For (26), \(\mathrm{Cov}(X_u,X_u)=\kappa \lim \limits _{r \downarrow 1}J_1(u,u,r)+\beta J_2(u,u)\),
$$\begin{aligned} J_1(u,u,r)&=-\frac{1}{4 \pi ^2}\oint _{|\xi _2|=1}\underbrace{\left[ \oint _{|\xi _1|=1}\frac{\log \left[ (1+a_{1}(y)\xi _1)(b_{1}(y)+\sqrt{y}\xi _1^{-1})\right] }{(\xi _1-r\xi _2)^2}d\xi _1\right] }_{(\text {i})}\\&\quad \cdot \log \left[ (1+a_{1}(y)\xi _2)(b_{1}(y)+\sqrt{y}\xi _2^{-1})\right] d\xi _2, \end{aligned}$$
Compute \((\text {i})\) as
$$\begin{aligned} (\text {i})&=\oint _{|\xi _1|=1}\frac{\log (1+a_{1}(y)\xi _1)}{(\xi _1-r\xi _2)^2}d\xi _1+\oint _{|z|=1} \frac{\log (b_{1}(y)+\sqrt{y}z)}{\left( r\xi _2\right) ^2\left( \frac{1}{r\xi _2}-z\right) ^2}dz\\&= \frac{2 \pi i\sqrt{y}}{r\xi _2(r\xi _2\cdot b_1(y)+\sqrt{y})}. \end{aligned}$$
Therefore,
$$\begin{aligned} J_1(u,u,r)&=\frac{\sqrt{y}}{2\pi i}\oint _{|\xi _2|=1}\frac{\log (1+a_{1}(y)\xi _2)+\log (b_{1}(y)+\sqrt{y}\xi ^{-1}_2)}{r\xi _2\cdot (r\xi _2\cdot b_{1}(y)+\sqrt{y})}d\xi _2\\&=\frac{\sqrt{y}}{2\pi i}\oint _{|z|=1} \frac{\log (1+a_{1}(y)z)}{rz(rb_{1}(y)z+\sqrt{y})}dz+\frac{\sqrt{y}}{2\pi i}\oint _{|z|=1}\frac{\log (b_{1}(y)+\sqrt{y}z)}{r\frac{1}{z}(r\frac{1}{z}b_{1}(y)+\sqrt{y})z^2}dz\\&=-\frac{\log \left( 1-\frac{\sqrt{y}}{r}\frac{a_{1}(y)}{b_{1}(y)}\right) }{r}, \end{aligned}$$
where in the left integral of the second equation \(z=0\) is a removable isolated singularity and \(z=-\sqrt{y}/(r b_{1}(y))\) is a pole of order one. Since \(|-rb_{1}(y)/\sqrt{y}|>1\), the right integral is zero. On the other hand,
$$\begin{aligned} J_2(u,u)&=-\frac{1}{4\pi ^2}\oint _{|\xi _1|=1}\frac{\log (1+a_{1}(y)\xi _1)+\log (b_{1}(y)+\sqrt{y}\xi _1^{-1})}{\xi _1^2}d\xi _1\\&\quad \cdot \oint _{|\xi _2|=1}\frac{\log (1+a_{1}(y)\xi _2)+\log (b_{1}(y)+\sqrt{y}\xi _2^{-1})}{\xi _2^2}d\xi _2\\&=a_{1}^2(y). \end{aligned}$$
We see then that
$$\begin{aligned} \mathrm{Cov}(X_u,X_u) =\kappa \left( -\log \left( 1-\sqrt{y}\frac{a_{1}(y)}{b_{1}(y)}\right) \right) +\beta a^2_{1}(y). \end{aligned}$$
Collecting the above results yields
$$\begin{aligned} \begin{aligned} \mathrm{E}X_u&=(\kappa -1)\left( \frac{1}{2}\log (1-a_1^2(y))\right) +\beta \left( -\frac{1}{2}a^2_1(y)\right) ,\\ \mathrm{Cov}(X_g,X_g)&=(\kappa +\beta )y,\\ \mathrm{Cov}(X_g,X_u)&=\kappa \sqrt{y} a_1(y)+\beta \sqrt{y} a_1(y),\\ \mathrm{Cov}(X_u,X_u)&=\kappa \left( -\log \left( 1-\sqrt{y}\frac{a_1(y)}{b_1(y)}\right) \right) +\beta a^2_1(y). \end{aligned} \end{aligned}$$
(27)
Thus, by the delta method we deduce that
$$\begin{aligned} \left( 1-F^{y_n}\left( \frac{1}{1+x}\right) \right) \left( \sum _{i=1}^p l_i-p\right) -\sum _{i=1}^p \log \left( l_i+1\right) +p F^{y_n}\left( \log (x+1)\right) \Rightarrow N(\mu ,V), \end{aligned}$$
(28)
where
$$\begin{aligned} \mu&=-(\kappa -1)\left( \frac{1}{2}\log \left( 1-a_{1}^2(y)\right) \right) -\beta \left( -\frac{1}{2}a^2_{1}(y)\right) ,\\ V&=\left( 1-F^y\left( \frac{1}{1+x}\right) \right) ^2\left( \kappa +\beta \right) y-2\left( 1-F^y\left( \frac{1}{1+x}\right) \right) \left( \kappa \sqrt{y}a_{1}(y)+\beta \sqrt{y}a_{1}(y)\right) \\&\quad +\kappa \left( -\log \left( 1-a_1^2(y)\right) \right) +\beta a_1^2(y). \end{aligned}$$
Finally by (23) and (28) we get the desired result that
$$\begin{aligned} -\frac{2}{n}\log T+p F^{y_n}\left( \log (x+1)\right) -p\log p \Rightarrow N(\mu ,V), \end{aligned}$$
where \(F^{y_n}(\log (x+1))\) is given in (17) with y replaced by \(y_n\) and \(F^{y}(1/(1+x))\) is given in (16). The proof is complete. \(\square \)
1.3 Proof of theorem 2
Proof
The proof of Theorem 2 mirrors that of Theorem 1. But, the expression of M-P law and the determination of poles in contour integrals are different when \(y\in [1,+\infty )\). Firstly, an argument similar to the one used in Theorem 1 reveals that (23) remains valid as \(y\ge 1\), which is
$$\begin{aligned} -\frac{2}{n}\log T\doteq \left( 1-F^{y_n}\left( \frac{1}{1+x}\right) \right) \left( \sum _{i=1}^p l_i-p\right) -\sum _{i=1}^p \log \left( l_i+1\right) +p\log p. \end{aligned}$$
To establish the asymptotic normality of ELRT, it remains to derive the limiting parameters as shown in (24), (25) and (26). To save space, we omit the procedure and just present the results below.
$$\begin{aligned} \mathrm{E}X_u&=(\kappa -1)\left( \frac{1}{2}\log (1-a_1^2(y))\right) +\beta \left( -\frac{1}{2}a^2_1(y)\right) ,\\ \mathrm{Cov}(X_g,X_g)&=(\kappa +\beta )y,\\ \mathrm{Cov}(X_g,X_u)&=\kappa \sqrt{y} a_1(y)+\beta \sqrt{y} a_1(y),\\ \mathrm{Cov}(X_u,X_u)&=\kappa \left( -\log \left( 1-\sqrt{y}\frac{a_1(y)}{b_1(y)}\right) \right) +\beta a^2_1(y). \end{aligned}$$
Coincidentally, the limiting parameters possess the same forms as those in Theorem 1. As a consequence, by the delta method we draw the conclusion that
$$\begin{aligned} \left( 1-F^{y_n}\left( \frac{1}{1+x}\right) \right) \left( \sum _{i=1}^p l_i-p\right) -\sum _{i=1}^p \log \left( l_i+1\right) +p F^{y_n}\left( \log (x+1)\right) \Rightarrow N(\mu ,V), \end{aligned}$$
(29)
where
$$\begin{aligned} \mu&=-(\kappa -1)\left( \frac{1}{2}\log \left( 1-a_{1}^2(y)\right) \right) -\beta \left( -\frac{1}{2}a^2_{1}(y)\right) ,\\ V&=\left( 1-F^y\left( \frac{1}{1+x}\right) \right) ^2\left( \kappa +\beta \right) y-2\left( 1-F^y\left( \frac{1}{1+x}\right) \right) \left( \kappa \sqrt{y}a_{1}(y)+\beta \sqrt{y}a_{1}(y)\right) \\&\quad +\kappa \left( -\log \left( 1-a_1^2(y)\right) \right) +\beta a_1^2(y). \end{aligned}$$
Finally by (23) and (29) we obtain the desired result that
$$\begin{aligned} -\frac{2}{n}\log T+p F^{y_n}\left( \log (x+1)\right) -p\log p \Rightarrow N(\mu ,V), \end{aligned}$$
where \(F^{y_n}(\log (x+1))\) is given in (19) with y replaced by \(y_n\) and \(F^{y}(1/(1+x))\) is given in (18). The proof is complete. \(\square \)
1.4 Proof of Theorem 4
Proof
For the spiked population model (10), we know \({\text {tr}}\left( {\varvec{\varSigma }}_p\right) /p=(p-M+\sum _{i=1}^k n_i a_i)/p \rightarrow 1\), as \(p\rightarrow \infty \). Rewrite T as
$$\begin{aligned} T&=\frac{\left| {{\mathbf {S}}}_n+\frac{{\text {tr}}({{\mathbf {S}}}_n)}{p}{{\mathbf {I}}}_p\right| ^{\frac{n}{2}}}{\left( {\text {tr}}({{\mathbf {S}}}_n)\right) ^{\frac{np}{2}}} =\frac{\left| {{\mathbf {S}}}_n+\frac{{\text {tr}}\left( {\varvec{\varSigma }}_p\right) }{p}{{\mathbf {I}}}_p\right| ^{\frac{n}{2}}}{\left( {\text {tr}}({{\mathbf {S}}}_n)\right) ^{\frac{np}{2}}} \left| {{\mathbf {I}}}_p \right. \\&\left. \quad +\left( \frac{{\text {tr}}\left( {{\mathbf {S}}}_n\right) }{p}-\frac{{\text {tr}}\left( {\varvec{\varSigma }}_p\right) }{p}\right) \left( {{\mathbf {S}}}_n+\frac{{\text {tr}}\left( {\varvec{\varSigma }}_p\right) }{p}{{\mathbf {I}}}_p\right) ^{-1}\right| ^{\frac{n}{2}}, \end{aligned}$$
which implies
$$\begin{aligned} -\frac{2}{n}\log T&=-\sum _{i=1}^p \log \left( l_i+\frac{{\text {tr}}\left( {\varvec{\varSigma }}_p\right) }{p}\right) -\sum _{i=1}^p \log \left( 1+\frac{\frac{{\text {tr}}\left( {{\mathbf {S}}}_n\right) }{p}-\frac{{\text {tr}}\left( {\varvec{\varSigma }}_p\right) }{p}}{l_i+\frac{{\text {tr}}\left( {\varvec{\varSigma }}_p\right) }{p}}\right) \\&\quad +p\log \left( \sum _{i=1}^p l_i\right) . \end{aligned}$$
By Lemma 2 in Tian et al. (2015), for general diagonal alternatives satisfying Assumption 3, as \(y \in (0,\infty )\), \({\text {tr}}\left( {{\mathbf {S}}}_n\right) -{\text {tr}}\left( {\varvec{\varSigma }}_p\right) ={\mathcal {O}}_p(1)\). As a result,
$$\begin{aligned} \begin{aligned} p\log \left( \sum _{i=1}^p l_i\right)&=p\log \left( 1+\frac{\sum \limits _{i=1}^p l_i-{\text {tr}}\left( {\varvec{\varSigma }}_p\right) }{{\text {tr}}\left( {\varvec{\varSigma }}_p\right) }\right) +p\log \left( {\text {tr}}\left( {\varvec{\varSigma }}_p\right) \right) \\&\doteq \sum _{i=1}^p l_i-{\text {tr}}\left( {\varvec{\varSigma }}_p\right) +p\log \left( {\text {tr}}\left( {\varvec{\varSigma }}_p\right) \right) . \end{aligned} \end{aligned}$$
(30)
Set \({\text {tr}}\left( {\varvec{\varSigma }}_p\right) /p=1/b\), define \(f_3(x)=1/(x+1/b)\) and \(f_4(x)=1/(x+1/b)^2\). Reconsider the centralized LSS in (20), by Pan and Zhou (2008) we derive that
$$\begin{aligned}&G_n(f_3)\frac{{\text {tr}}({{\mathbf {S}}}_n)-{\text {tr}}({\varvec{\varSigma }}_p)}{p} \\&\quad =p\left\{ \frac{1}{p}\sum _{i=1}^p\frac{1}{l_i+\frac{1}{b}}-F^{y_n,H_p}\left( \frac{1}{x+\frac{1}{b}}\right) \right\} \frac{{\text {tr}}({{\mathbf {S}}}_n)-{\text {tr}}({\varvec{\varSigma }}_p)}{p}=o_p\left( 1\right) ,\\&G_n(f_4)\left( \frac{{\text {tr}}({{\mathbf {S}}}_n)-{\text {tr}}({\varvec{\varSigma }}_p)}{p}\right) ^2 \\&\quad =p\left\{ \frac{1}{p}\sum _{i=1}^p\frac{1}{(l_i+\frac{1}{b})^2}-F^{y_n,H_p}\left( \frac{1}{(x+\frac{1}{b})^2}\right) \right\} \frac{({\text {tr}}({{\mathbf {S}}}_n)-{\text {tr}}({\varvec{\varSigma }}_p))^2}{p^2}=o_p\left( 1\right) . \end{aligned}$$
Using the Taylor expansion on \(\log (1+x)\), we obtain
$$\begin{aligned} \begin{aligned} \sum _{i=1}^p\log \left( 1+\frac{\frac{{\text {tr}}\left( {{\mathbf {S}}}_n\right) }{p}-\frac{{\text {tr}}\left( {\varvec{\varSigma }}_p\right) }{p}}{l_i+\frac{{\text {tr}}\left( {\varvec{\varSigma }}_p\right) }{p}}\right)&=\sum _{i=1}^p \frac{\frac{{\text {tr}}\left( {{\mathbf {S}}}_n\right) }{p}-\frac{{\text {tr}}\left( {\varvec{\varSigma }}_p\right) }{p}}{l_i+\frac{{\text {tr}}\left( {\varvec{\varSigma }}_p\right) }{p}}+{\mathcal {O}}_p\left( \sum _{i=1}^p\frac{\left( \frac{{\text {tr}}\left( {{\mathbf {S}}}_n\right) }{p}-\frac{{\text {tr}}\left( {\varvec{\varSigma }}_p\right) }{p}\right) ^2}{\left( l_i+\frac{{\text {tr}}\left( {\varvec{\varSigma }}_p\right) }{p}\right) ^2}\right) \\&\doteq \left( {\text {tr}}({{\mathbf {S}}}_n)-{\text {tr}}({\varvec{\varSigma }}_p)\right) F^{y_n,H_p}\left( \frac{1}{x+\frac{{\text {tr}}({\varvec{\varSigma }}_p)}{p}}\right) . \end{aligned} \end{aligned}$$
(31)
In the same way one can show that
$$\begin{aligned} \sum _{i=1}^p \log \left( l_i+\frac{{\text {tr}}({\varvec{\varSigma }}_p)}{p}\right) \doteq \sum _{i=1}^p \log \left( l_i+1\right) +({\text {tr}}({\varvec{\varSigma }}_p)-p)F^{y_n,H_p}\left( \frac{1}{x+1}\right) . \end{aligned}$$
(32)
Putting (30), (31) and (32) together yields
$$\begin{aligned}&-\frac{2}{n}\log T \nonumber \\&\quad \doteq \left( 1-F^{y_n,H_p}\left( \frac{1}{x+\frac{{\text {tr}}\left( {\varvec{\varSigma }}_p\right) }{p}}\right) \right) \sum _{i=1}^p l_i-\sum _{i=1}^p \log \left( l_i+1\right) -\left( {\text {tr}}\left( {\varvec{\varSigma }}_p\right) -p\right) \nonumber \\&\quad \qquad \cdot F^{y_n,H_p}\left( \frac{1}{x+1}\right) +p\log \left( {\text {tr}}({\varvec{\varSigma }}_p)\right) +{\text {tr}}\left( {\varvec{\varSigma }}_p\right) F^{y_n,H_p}\left( \frac{1}{x+\frac{{\text {tr}}\left( {\varvec{\varSigma }}_p\right) }{p}}\right) -{\text {tr}}\left( {\varvec{\varSigma }}_p\right) . \end{aligned}$$
(33)
Next we proceed to investigate the asymptotic behavior of (33). Based on the discussion in the opening of Sect. 4, we can conclude that for the spiked population model (10),
$$\begin{aligned}&\left( \begin{array}{ll} \sum \nolimits _{i=1}^p l_i-pF^{y_n,H_p}(x) \\ \sum \nolimits _{i=1}^p\log (l_i+1)-pF^{y_n,H_p}\left( \log (x+1)\right) \\ \end{array} \right) \\&\quad \Longrightarrow N\left( \left( \begin{array}{cc} \mathrm{E}X_{g} \\ \mathrm{E}X_{u} \\ \end{array} \right) , \ \left( \begin{array}{cc} \mathop {\text {Cov}}(X_g,X_g) &{} \mathop {\text {Cov}}(X_g, X_u) \\ \mathop {\text {Cov}}(X_u,X_g) &{} \mathop {\text {Cov}}(X_u,X_u) \end{array} \right) \ \right) , \end{aligned}$$
where the six parameters in the normal distribution are identical to those in Theorem 1. By directing invoking the delta method, we are led to the conclusion that
$$\begin{aligned}&-\frac{2}{n}\log T+\left( {\text {tr}}({\varvec{\varSigma }}_p)-pF^{y_n,H_p}(x)\right) \left( 1-F^{y_n,H_p}\left( \frac{1}{x+\frac{{\text {tr}}\left( {\varvec{\varSigma }}_p\right) }{p}}\right) \right) +\left( {\text {tr}}({\varvec{\varSigma }}_p)-p\right) \nonumber \\&\qquad \cdot F^{y_n,H_p}\left( \frac{1}{x+1}\right) +p F^{y_n,H_p}\left( \log (x+1)\right) -p\log \left( {\text {tr}}\left( {\varvec{\varSigma }}_p\right) \right) \Rightarrow N(\mu ,V), \end{aligned}$$
(34)
where \(\mu \) and V are given in (28). To finish the proof, it suffices to derive the detailed expression of \(F^{y_n,H_p}(x)\), \(F^{y_n,H_p}(1/(x+1/b))\), \(F^{y_n,H_p}(1/(x+1))\) and \(F^{y_n,H_p}(\log (x+1))\) according to the asymptotic expansion in Theorem 3.
Firstly, it is known in Wang et al. (2014) that
$$\begin{aligned} F^{y_n,H_p}(x)=1+\frac{1}{p}\sum _{i=1}^k n_i a_i-\frac{M}{p}+{\mathcal {O}}\left( \frac{1}{n^2}\right) . \end{aligned}$$
(35)
For \(u(x)=\log \left( x+1\right) \), when \(0<y_n<1\), write
$$\begin{aligned} u\left( -\frac{1}{m}+\frac{y_n}{1+m}\right)&=\log \left( \frac{m^2+y_n m-1}{m(1+m)}\right) =\log \left( \frac{(m-m_1)(m-m_2)}{m(1+m)}\right) , \end{aligned}$$
where
$$\begin{aligned} m_1=\frac{-y_n+\sqrt{y_n^2+4}}{2},~~~ m_2=\frac{-y_n-\sqrt{y_n^2+4}}{2}. \end{aligned}$$
Set \({\widetilde{f}}(m)=m^2+y_n m-1\), it is simple to check that \({\widetilde{f}}\left( -1/\left( 1-\sqrt{y_n}\right) \right) >0\) and \({\widetilde{f}}(-1/(1+\sqrt{y_n}))<0\), indicating that \(m_2\) is included in the integral region of (11) and (12). Also notice that \(-1/a_i \in \left[ -1/(1-\sqrt{y_n}),-1/(1+\sqrt{y_n})\right] \) when \(i \in \{k_1+1, \ldots , k\}\). Therefore the potential poles are \(\{m=-1\},~\{m=-1/a_i,~i=k_1+1,\ldots , k\}\) and \(\{m=m_2\}\) when calculating the contour integrals in Theorem 3. Since \({\widetilde{f}}(-1)=-y_n<0\) and the \(M-k_1\) close spikes satisfy \(1<a_i\le 1+\sqrt{y_n}\) for \(i \in \{k_1+1, \ldots , k\}\), the three poles are distinct which eliminates the possibility of repeatedly calculating the poles.
$$\begin{aligned} (11)&=-\frac{1}{2\pi i p}\oint _{{\mathcal {C}}_{1}}\left( \log \left( \frac{m-m_1}{m }\right) +\log \left( \frac{m-m_2}{1+m}\right) \right) \left( \frac{M}{y_n m}-\sum _{i=1}^{k}\frac{n_i a_i^2 m}{(1+a_i m)^2}\right) dm\\&=\frac{1}{2 \pi i p}\oint _{{\mathcal {C}}_1} \log \left( \frac{m-m_1}{m}\right) \sum _{i=1}^k \frac{n_i a_i^2 m}{(1+a_i m)^2} dm-\frac{1}{2 \pi i p}\oint _{{\mathcal {C}}_1}\log \left( \frac{m-m_2}{1+m}\right) \frac{M}{m y_n}dm\\&\quad +\frac{1}{2\pi i p}\oint _{{\mathcal {C}}_1}\log \left( \frac{m-m_2}{1+m}\right) \sum _{i=1}^{k}\frac{n_i a_i^2 m}{(1+a_i m)^2} dm \triangleq A-B+C, \\ A&=\frac{1}{p}\sum _{i=k_1+1}^{k}n_i \cdot \frac{\partial \log \left( \frac{m-m_1}{m}\right) m}{\partial m}\Biggl |_{m=-\frac{1}{a_i}}=\sum _{i=k_1+1}^{k}\frac{n_i}{p}\left( \frac{-m_1 a_i}{m_1 a_i+1}+\log \left( 1+m_1 a_i\right) \right) . \end{aligned}$$
To calculate B, plot a new contour as given in Fig. 4. Besides the original contour \({\mathcal {C}}_1\), \(\widetilde{{\mathcal {C}}_1}\) is a circle with radius \(\epsilon \) and \({\mathcal {C}}_r\) is a circle with radius a sufficiently large number r. By taking the straight line segment joining \(m_2\) and \(-1\) as the branch cut, the complex valued function \(\log \left( (m-m_2)/(m+1)\right) \) is analytic in the new contour Fig. 4 with contour integral equal to zero. Note that the integral in \(l_1\) and \(l_2\) could cancel each other out. Furthermore, by Cauchy’s residue theorem and the parametrization \(m=r e^{i \theta }\) we get
$$\begin{aligned} \frac{1}{2 \pi i}\oint _{\widetilde{{\mathcal {C}}}_1}\log \left( \frac{m-m_2}{m+1}\right) \frac{1}{m}dm&=\log (-m_2),\\ \frac{1}{2 \pi i} \oint _{{\mathcal {C}}_r} \log \left( \frac{m-m_2}{m+1}\right) \frac{1}{m} dm&=-\frac{1}{2 \pi }\int _{0}^{2 \pi } \log \left( \frac{m-m_2}{m+1}\right) d\theta \xrightarrow {r\rightarrow \infty }0, \end{aligned}$$
which gives rise to
$$\begin{aligned} B=-\frac{M}{p y_n}\log (-m_2). \end{aligned}$$
As for C,
$$\begin{aligned} C&=\frac{1}{2 \pi i p} \oint _{{\mathcal {C}}_1} \log \left( \frac{m-m_2}{1+m}\right) \sum _{i=1}^{k} \frac{n_i m}{\left( m+\frac{1}{a_i}\right) ^2} dm\\&=\frac{1}{2 \pi i p} \oint _{{\mathcal {C}}_1} \log \left( \frac{m-m_2}{1+m}\right) \sum _{i=1}^{k_1} \frac{n_i m}{\left( m+\frac{1}{a_i}\right) ^2} dm \\&\quad + \frac{1}{2 \pi i p} \oint _{{\mathcal {C}}_1} \log \left( \frac{m-m_2}{1+m}\right) \sum _{i=k_1+1}^{k} \frac{n_i m}{\left( m+\frac{1}{a_i}\right) ^2} dm, \end{aligned}$$
Following the same strategy for analyzing B, consider the contour in Fig. 4 but suitably plot \(\widetilde{{\mathcal {C}}}_1\) to make it include all \(-1/a_i~(i=1,\ldots ,k_1)\) and plot \({\mathcal {C}}_1\) to make it include all \(-1/a_i~(i=k_1+1,\ldots ,k)\). This is theoretically possible because the first \(k_1\) spikes \(a_i\)’s are not included in \({\mathcal {C}}_1\). As a consequence, the right integral of C is zero. For the left side, its integral value on \({\mathcal {C}}_r\) tends to zero as \(r\rightarrow \infty \). By Cauchy’s residue theorem
$$\begin{aligned} C&=-\frac{1}{2 \pi i p} \oint _{\widetilde{{\mathcal {C}}}_1} \log \left( \frac{m-m_2}{1+m}\right) \sum _{i=1}^{k_1} \frac{n_i m}{\left( m+\frac{1}{a_i}\right) ^2} dm\\&=\sum _{i=1}^{k_1} \frac{n_i}{p} \log \left( \frac{1-a_i}{1+m_2 a_i}\right) -\sum _{i=1}^{k_1} \frac{n_i}{p} \frac{(1+m_2)a_i}{(1+m_2 a_i)(a_i-1)}. \end{aligned}$$
Summarizing A, B and C we have
$$\begin{aligned} (11)&=\sum _{i=k_1+1}^{k}\frac{n_i}{p}\left( \frac{-m_1 a_i}{m_1 a_i+1}+\log \left( 1+m_1 a_i\right) \right) +\sum _{i=1}^{k}\frac{n_i}{p}\frac{\log (-m_2)}{y_n}\\&\quad +\sum _{i=1}^{k_1}\frac{n_i}{p}\log \left( \frac{1-a_i}{1+m_2 a_i}\right) \\&\quad -\sum _{i=1}^{k_1}\frac{n_i}{p}\frac{(1+m_2)a_i}{(1+a_i m_2)(a_i-1)}. \end{aligned}$$
Next we deal with the term (12), by direct computation it can be shown that
$$\begin{aligned} (12)&=-\frac{1}{2 \pi i p}\oint _{{\mathcal {C}}_1}\frac{m(1+m)}{(m-m_1)(m-m_2)}\sum _{i=1}^{k}\left( \frac{n_i a_i}{1+a_i m}-\frac{n_i}{1+m}\right) \left( \frac{1}{m}-\frac{y_n m}{(1+m)^2}\right) dm\\&=-\frac{1}{2 \pi i p}\sum _{i=1}^{k}\oint _{{\mathcal {C}}_1}\frac{n_i a_i (1+m) }{(m-m_1)(m-m_2)(1+a_i m)}dm \\&\quad +\frac{1}{2 \pi i p}\sum _{i=1}^{k}\oint _{{\mathcal {C}}_1} \frac{a_i y_n n_i m^2}{(1+a_i m)(1+m)(m-m_1)(m-m_2)}dm\\&\quad +\frac{1}{2 \pi i p}\sum _{i=1}^{k}\oint _{{\mathcal {C}}_1}\frac{n_i }{(m-m_1)(m-m_2)}dm \\&\quad -\frac{1}{2 \pi i p}\sum _{i=1}^{k}\oint _{{\mathcal {C}}_1}\frac{n_i m^2 y_n}{(m-m_1)(m-m_2)(1+m)^2}dm\\&\triangleq D+E+F+G, \end{aligned}$$
where
$$\begin{aligned} D&=-\sum _{i=1}^{k}\frac{n_i}{p}\frac{(m_2+1) a_i}{(m_2-m_1)(1+a_i m_2)}+\frac{1}{p}\sum _{i=k_1+1}^{k}\frac{n_i a_i (1-a_i)}{(1+m_1 a_i)(1+m_2 a_i)},\\ E&=\frac{1}{p}\sum _{i=1}^{k}\frac{n_i a_i y_n}{(m_2-m_1)}\frac{m_2^2}{(1+a_i m_2)(1+m_2)}+\frac{1}{p}\sum _{i=1}^{k}\frac{n_i a_i y_n}{(1+m_1)(1+m_2)(1-a_i)}\\&\quad +\frac{1}{p}\sum _{i=k_1+1}^{k}\frac{n_i y_n a_i}{(1+m_1 a_i)(1+m_2 a_i)(a_i-1)},\\ F&=\frac{1}{p}\sum _{i=1}^{k}\frac{n_i }{m_2-m_1},\\ G&=-\frac{1}{p}\sum _{i=1}^{k}\frac{n_i m_2^2 y_n}{(m_2-m_1)(1+m_2)^2}-\frac{1}{p}\sum _{i=1}^{k}\frac{n_i y_n(y_n+2)}{(1+m_1)^2(1+m_2)^2}. \end{aligned}$$
Notice that \(m_1 m_2=-1\) and \(m_1+m_2=-y_n\), by which the sum \(D+E+F+G\) reduces to
$$\begin{aligned} (12)&=\sum _{i=1}^k \frac{n_i}{p}\frac{a_i y_n m_2^2-a_i(1+m_2)^2}{(m_2-m_1)(1+a_i m_2)(1+m_2)}+\sum _{i=1}^k \frac{n_i}{p}\frac{(1+m_2)^2-m_2^2 y_n}{(m_2-m_1)(1+m_2)^2}\\&\quad +\sum _{i=k_1+1}^k \frac{n_i}{p}\frac{a_i (y_n-(1-a_i)^2)}{(1+m_1 a_i)(1+m_2 a_i)(a_i-1)}+\sum _{i=1}^k \frac{n_i}{p}\frac{y_n(2a_i-2-y_n)}{(1+m_1)^2(1+m_2)^2(1-a_i)}. \end{aligned}$$
As for (13),
$$\begin{aligned} (13)&=\left( 1-\frac{M}{p}\right) F^{y_n}\left( \log \left( x+1\right) \right) +\frac{1}{p}\sum _{i=1}^{k_1} n_i \log \left( a_i+\frac{y_n a_i}{a_i-1}+1\right) +{\mathcal {O}}\left( \frac{1}{n^2}\right) ,\\&=\left( 1-\frac{M}{p}\right) F^{y_n}\left( \log \left( x+1\right) \right) +\frac{1}{p}\sum _{i=1}^{k_1} n_i \log \left( \frac{(1+m_1 a_i)(1+m_2 a_i)}{1-a_i}\right) +{\mathcal {O}}\left( \frac{1}{n^2}\right) , \end{aligned}$$
where \(F^{y_n}\left( \log \left( x+1\right) \right) \) has been figured out in Theorem 1. Combining the above results we derive that when \(y\in (0,1)\),
$$\begin{aligned}&F^{y_n,H_p}\left( \log \left( x+1\right) \right) \nonumber \\&\quad =\left( 1-\frac{M}{p}\right) F^{y_n}\left( \log \left( x+1\right) \right) +\sum _{i=1}^{k}\frac{n_i}{p}\log \left( 1+m_1 a_i\right) +\sum _{i=1}^{k} \frac{n_i}{p}\frac{\log (-m_2)}{y_n} \nonumber \\&\qquad +{\mathcal {O}}\left( \frac{1}{n^2}\right) \nonumber \\&\qquad +\sum _{i=1}^{k}\frac{n_i}{p}\frac{(1+m_2) a_i}{(1+m_2 a_i)(1-a_i)}+\sum _{i=1}^{k}\frac{n_i}{p}\frac{a_i y_n m_2^2-a_i(1+m_2)^2}{(m_2-m_1)(1+a_i m_2)(1+m_2)}\nonumber \\&\qquad +\sum _{i=1}^{k}\frac{n_i}{p} \frac{(1+m_2)^2-m_2^2y_n}{(m_2-m_1)(1+m_2)^2}+\sum _{i=1}^{k}\frac{n_i}{p}\frac{y_n(2a_i-2-y_n)}{(1+m_1)^2(1+m_2)^2(1-a_i)}. \end{aligned}$$
(36)
Via an analogous argument, we proceed to calculate \(F^{y_n,H_p}(1/(x+1/b))\). When \(0<y_n<1\), note that for \(f_3(x)=1/(x+1/b)\) we have
$$\begin{aligned} f_3\left( -\frac{1}{m}+\frac{y_n}{1+m}\right) =\frac{m(1+m)b}{(m-\widetilde{m_1})(m-\widetilde{m_2})}, \end{aligned}$$
where
$$\begin{aligned} \widetilde{m_1}&=\frac{-((y_n-1)b+1)+\sqrt{((y_n-1)b+1)^2+4b}}{2}>0,\\ \widetilde{m_2}&=\frac{-((y_n-1)b+1)-\sqrt{((y_n-1)b+1)^2+4b}}{2}\in \left[ \frac{-1}{1-\sqrt{y_n}},\frac{-1}{1+\sqrt{y_n}}\right] . \end{aligned}$$
Then it can be deduced that when calculating the contour integrals in Theorem 3, the potential poles are \(\{m=-1\},~\{m=-1/a_i,~i=k_1+1,\ldots , k\}\) and \(\{m=m_2\}\). For (11),
$$\begin{aligned} (11)&=-\frac{1}{2\pi i p}\oint _{{\mathcal {C}}_{1}}f_3\left( -\frac{1}{m}+\frac{y_n}{1+m}\right) \left( \frac{M}{y_n m}-\sum _{i=1}^{k}\frac{n_i a_i^2 m}{(1+a_i m)^2}\right) dm\\&=\frac{1}{2\pi i p}\oint _{{\mathcal {C}}_1}\frac{m(1+m)b}{(m-\widetilde{m_1})(m-\widetilde{m_2})}\sum _{i=1}^{k_1}\frac{n_i a_i^2 m}{(1+a_i m)^2}dm\\&\quad +\frac{1}{2\pi i p}\oint _{{\mathcal {C}}_1}\frac{m(1+m)b}{(m-\widetilde{m_1})(m-\widetilde{m_2})}\sum _{i=k_1+1}^{k}\frac{n_i a_i}{1+a_i m}dm\\&\quad -\frac{1}{2\pi i p}\oint _{{\mathcal {C}}_1}\frac{m(1+m)b}{(m-\widetilde{m_1})(m-\widetilde{m_2})}\sum _{i=k_1+1}^{k}\frac{n_i a_i}{(1+a_i m)^2}dm\\&\quad -\frac{1}{2\pi i p}\oint _{{\mathcal {C}}_1}\frac{m(1+m)b}{(m-\widetilde{m_1})(m-\widetilde{m_2})}\frac{M}{y_n m}dm\\&=\sum _{i=1}^{k_1} \frac{n_i}{p}\frac{\widetilde{m_2}^2(1+\widetilde{m_2})ba_i^2}{(\widetilde{m_2}-\widetilde{m_1})(1+a_i \widetilde{m_2})^2}+\sum _{i=k_1+1}^k \frac{n_i}{p}\frac{(1-a_i)b}{(1+\widetilde{m_1} a_i)(1+\widetilde{m_2} a_i)}\\&\quad +\sum _{i=k_1+1}^k \frac{n_i}{p}\frac{\widetilde{m_2}(1+\widetilde{m_2})ba_i}{(\widetilde{m_2}-\widetilde{m_1})(1+\widetilde{m_2} a_i)}\\&\quad -\sum _{i=k_1+1}^k \frac{n_i}{p}\frac{a_i \widetilde{m_2}(1+\widetilde{m_2})b}{(1+a_i \widetilde{m_2})^2(\widetilde{m_2}-\widetilde{m_1})}-\sum _{i=k_1+1}^k \frac{n_i}{p}\frac{b^2a_i (2a_i-a_i^2-1+y_n)}{(1+\widetilde{m_1 }a_i)^2(1+\widetilde{m_2} a_i)^2}\\&\quad -\sum _{i=1}^{k} \frac{n_i}{p}\frac{(1+\widetilde{m_2})b}{(\widetilde{m_2}-\widetilde{m_1})y_n}\\&=\sum _{i=1}^k \frac{n_i}{p} \frac{\widetilde{m_2}^2(1+\widetilde{m_2})ba_i^2}{(\widetilde{m_2}-\widetilde{m_1})(1+a_i \widetilde{m_2})^2}+\sum _{i=k_1+1}^k \frac{n_i}{p}\frac{(1-a_i)b}{(1+\widetilde{m_1} a_i)(1+\widetilde{m_2} a_i)}\\&\quad -\sum _{i=k_1+1}^k \frac{n_i}{p}\frac{(y_n-(a_i-1)^2)b^2a_i}{(1+\widetilde{m_1} a_i)^2(1+\widetilde{m_2} a_i)^2}\\&\quad -\sum _{i=1}^k \frac{n_i}{p}\frac{(1+\widetilde{m_2})b}{(\widetilde{m_2}-\widetilde{m_1})y_n}. \end{aligned}$$
Next, (12) can be calculated as
$$\begin{aligned} (12)&=\frac{1}{2 \pi i p}\oint _{{\mathcal {C}}_1}\frac{m^2(1+m)^2b^2}{(m-\widetilde{m_1})^2(m-\widetilde{m_2})^2}\sum _{i=1}^{k}\left( \frac{n_i a_i}{1+a_i m}-\frac{n_i}{1+m}\right) \\&\quad {\times }\left( \frac{1}{m}-\frac{y_n m}{(1+m)^2}\right) dm\\&\triangleq I_1+I_2+I_3+I_4, \end{aligned}$$
where
$$\begin{aligned} I_1&=\frac{1}{2\pi i p}\oint _{{\mathcal {C}}_1}\sum _{i=1}^k \frac{m(1+m)^2b^2}{(m-\widetilde{m_1})^2(m-\widetilde{m_2})^2}\frac{n_i a_i}{1+m a_i}dm\\&=-\sum _{i=k_1+1}^k \frac{n_i}{p}\frac{a_i(a_i-1)^2b^2}{(1+\widetilde{m_1} a_i)^2(1+\widetilde{m_2} a_i)^2} \\&\quad +\sum _{i=1}^k \frac{n_i}{p}\frac{b^2a_i(1+\widetilde{m_2})(\widetilde{m_1}+\widetilde{m_2}+3\widetilde{m_1}\widetilde{m_2}+(2a_i(\widetilde{m_1}+1)-1)\widetilde{m_2}^2)}{(\widetilde{m_1}-\widetilde{m_2})^3(1+a_i \widetilde{m_2})^2},\\ I_2&=-\frac{1}{2\pi i p}\oint _{{\mathcal {C}}_1}\sum _{i=1}^k \frac{m^3b^2n_i y_n a_i}{(m-\widetilde{m_1})^2(m-\widetilde{m_2})^2(1+ma_i)}dm\\&=\sum _{i=k_1+1}^k \frac{n_i}{p} \frac{b^2y_n a_i}{(1+\widetilde{m_1} a_i)^2(1+\widetilde{m_2} a_i)^2}-\sum _{i=1}^k \frac{n_i y_n b^2}{p}\frac{a_i \widetilde{m_2}^2 (-\widetilde{m_2}+3\widetilde{m_1}+2a_i \widetilde{m_1} \widetilde{m_2})}{(\widetilde{m_1}-\widetilde{m_2})^3(1+a_i \widetilde{m_2})^2},\\ I_3&=-\frac{1}{2\pi i p}\oint _{{\mathcal {C}}_1}\sum _{i=1}^k \frac{m(1+m)b^2n_i}{(m-\widetilde{m_1})^2(m-\widetilde{m_2})^2}dm=-\sum _{i=1}^k \frac{n_i b^2}{p}\frac{\widetilde{m_1}+\widetilde{m_2}+2\widetilde{m_1}\widetilde{m_2}}{(\widetilde{m_1}-\widetilde{m_2})^3},\\ I_4&=\frac{1}{2\pi i p}\oint _{{\mathcal {C}}_1}\sum _{i=1}^k \frac{m^3 b^2 y_n n_i}{(m-\widetilde{m_1})^2(m-\widetilde{m_2})^2(1+m)}dm\\&=-\sum _{i=1}^k \frac{n_i y_n b^2}{p}\frac{1}{(1+\widetilde{m_1})^2(1+\widetilde{m_2})^2}+\sum _{i=1}^k \frac{n_i b^2 y_n}{p}\frac{\widetilde{m_2}^2(\widetilde{m_2}-3\widetilde{m_1}-2\widetilde{m_1}\widetilde{m_2})}{(\widetilde{m_2}+1)^2(\widetilde{m_2}-\widetilde{m_1})^3}. \end{aligned}$$
Lastly, (13) can be found to be
$$\begin{aligned} (13)=\left( 1-\frac{M}{p}\right) F^{y_n}\left( \frac{1}{x+\frac{{\text {tr}}\left( {\varvec{\varSigma }}_p\right) }{p}}\right) +\frac{1}{p}\sum _{i=1}^{k_1}\frac{n_i(1-a_i)b}{(1+\widetilde{m_1}a_i)(1+\widetilde{m_2}a_i)}+{\mathcal {O}}\left( \frac{1}{n^2}\right) . \end{aligned}$$
By Proposition 2.10 in Yao et al. (2015), we can compute the M-P law \(F^{y_n}(1/(x+1/b))\) as
$$\begin{aligned} F^{y_n}\left( \frac{1}{x+\frac{1}{b}}\right)&=\int _{(1-\sqrt{y_n})^2}^{(1+\sqrt{y_n})^2}\frac{\frac{1}{x+\frac{1}{b}}}{2\pi x y_n}\sqrt{(x-(1-\sqrt{y_n})^2)((1+\sqrt{y_n})^2-x)}dx\\&=-\frac{1}{4\pi i}\oint _{|z|=1}\frac{\frac{1}{|1+\sqrt{y_n}z|^2+\frac{1}{b}}}{z^2(1+\sqrt{y_n}z)(z+\sqrt{y_n})}(1-z^2)^2dz. \end{aligned}$$
Solving \(z(|1+\sqrt{y_n}z|^2+1/b)=\sqrt{y_n}z^2+(y_n+1/b+1)z+\sqrt{y_n}=0\) yields
$$\begin{aligned} \widetilde{y_1}&=\frac{-\left( y_n+\frac{1}{b}+1\right) +\sqrt{(y_n+\frac{1}{b}+1)^2-4y_n}}{2\sqrt{y_n}}\in (-1,0),\\ \widetilde{y_2}&=\frac{-\left( y_n+\frac{1}{b}+1\right) -\sqrt{(y_n+\frac{1}{b}+1)^2-4y_n}}{2\sqrt{y_n}}\notin (-1,0). \end{aligned}$$
It turns out that
$$\begin{aligned} F^{y_n}\left( \frac{1}{x+\frac{1}{b}}\right)&=-\frac{1}{2}\left[ \frac{1}{y_n}+\frac{y_n-1}{y_n(\widetilde{y_1}+\sqrt{y_n})(\widetilde{y_2}+\sqrt{y_n})} \right. \nonumber \\&\left. \quad +\frac{(1-\widetilde{y_1}^2)^2}{y_n(\widetilde{y_1} -\widetilde{y_2})\widetilde{y_1}\left( \widetilde{y_1}+\frac{1}{\sqrt{y_n}}\right) (\widetilde{y_1}+\sqrt{y_n})}\right] . \end{aligned}$$
(37)
Combining (37) together with (11), (12) and (13) for \(f_3(x)\) as given above, we obtain the desired result that
$$\begin{aligned} \begin{aligned}&F^{y_n,H_p}\left( \frac{1}{x+\frac{1}{b}}\right) \\&\quad =\left( 1-\frac{M}{p}\right) F^{y_n}\left( \frac{1}{x+\frac{1}{b}}\right) +\frac{1}{p}\sum _{i=1}^k \frac{n_i b(1-a_i)}{p(1+\widetilde{m_1} a_i)(1+\widetilde{m_2} a_i)}+{\mathcal {O}}\left( \frac{1}{n^2}\right) \\&\qquad +\sum _{i=1}^k \frac{n_i}{p}\frac{\widetilde{m_2}^2(1+\widetilde{m_2})ba_i^2}{(\widetilde{m_2}-\widetilde{m_1})(1+a_i\widetilde{m_2})^2}-\sum _{i=1}^k\frac{n_i}{p}\frac{(1+\widetilde{m_2})b}{(\widetilde{m_2}-\widetilde{m_1})y_n}\\&\qquad +\sum _{i=1}^k \frac{n_i b^2}{p}\frac{a_i(1+\widetilde{m_2})((2a_i-1)\widetilde{m_2}^2+\widetilde{m_1}+\widetilde{m_2}+3\widetilde{m_1}\widetilde{m_2}+2a_i\widetilde{m_1}\widetilde{m_2}^2)}{(\widetilde{m_1}-\widetilde{m_2})^3(1+a_i\widetilde{m_2})^2}\\&\qquad -\sum _{i=1}^k \frac{n_i y_n b^2}{p}\frac{a_i \widetilde{m_2}^2(-\widetilde{m_2}+3\widetilde{m_1}+2a_i\widetilde{m_1}\widetilde{m_2})}{(\widetilde{m_1}-\widetilde{m_2})^3(1+a_i \widetilde{m_2})^2}-\sum _{i=1}^k \frac{n_i b^2}{p}\frac{\widetilde{m_1}+\widetilde{m_2}+2\widetilde{m_1}\widetilde{m_2}}{(\widetilde{m_1}-\widetilde{m_2})^3}\\&\qquad -\sum _{i=1}^k \frac{n_i b^2 y_n}{p}\frac{1}{(1+\widetilde{m_1})^2(1+\widetilde{m_2})^2}+\sum _{i=1}^k \frac{n_i b^2 y_n}{p}\frac{\widetilde{m_2}^2(\widetilde{m_2}-3\widetilde{m_1}-2\widetilde{m_1}\widetilde{m_2})}{(\widetilde{m_2}+1)^2(\widetilde{m_2}-\widetilde{m_1})^3}. \end{aligned} \end{aligned}$$
(38)
By replacing b with 1 in (38) we can get the asymptotic expansion of \(F^{y_n,H_p}(1/(x+1))\). By (35), we know that \({\text {tr}}({\varvec{\varSigma }}_p)-pF^{y_n,H_p}(x)={\mathcal {O}}(1/p)\). Thus the second term of (34) disappears when \(p\rightarrow \infty \). Furthermore, \({\text {tr}}({\varvec{\varSigma }}_p)-p=\sum _{i=1}^k n_i(a_i-1)\), which is a fixed constant. From (38) we can deduce that as \(n, p\rightarrow \infty \),
$$\begin{aligned} \left( {\text {tr}}({\varvec{\varSigma }}_p)-p\right) F^{y_n,H_p}\left( \frac{1}{x+1}\right) -\left( {\text {tr}}({\varvec{\varSigma }}_p)-p\right) F^{y_n}\left( \frac{1}{x+1}\right) \rightarrow 0, \end{aligned}$$
where \(F^{y_n}(1/(x+1))\) is given in (16) with y replaced by \(y_n\). As a result, (34) can reduce to
$$\begin{aligned}&-\frac{2}{n}\log T+\left( {\text {tr}}({\varvec{\varSigma }}_p)-p\right) F^{y_n}\left( \frac{1}{x+1}\right) +p F^{y_n,H_p}\left( \log (x+1)\right) \\&\qquad -p\log \left( {\text {tr}}\left( {\varvec{\varSigma }}_p\right) \right) \Rightarrow N(\mu ,V). \end{aligned}$$
The proof is complete. \(\square \)
1.5 Proof of Theorem 5
Proof
Note that when \(y\ge 1\) (34) remains valid since it is established under the condition that \(p/n \rightarrow y \in (0,\infty )\), although the form of \(F^{y_n,H_p}\) is different as \(y\in [1,\infty )\). Hence in the current context the main object is still to calculate \(F^{y_n,H_p}(x)\), \(F^{y_n,H_p}(1/(x+1/b))\), \(F^{y_n,H_p}(1/(x+1))\) and \(F^{y_n,H_p}(\log (x+1))\) according to Theorem 3.
Firstly, (35) is not changed when \(y\in [1,\infty )\). For \(F^{y_n,H_p}(\log (x+1))\), according to the proof of Theorem 2 in Wang et al. (2014), in order to make the integrand analytic in the new contour, \(m_2\) should be included in the region of the contour integral. See Fig.4 in Page 206 of Wang et al. (2014) and the corresponding analysis there for further details. Moreover, \(1<a_i\le 1+\sqrt{y_n}\) when \(i \in \{k_1+1, \ldots , k\}\). The computations \(F^{y_n,H_p}(\log (x+1))\) of are totally the same as those in Theorem 4. By the Cauchy’s residue theorem, we derive that
$$\begin{aligned} (11)&=\sum _{i=k_1+1}^{k}\frac{n_i}{p}\left( \frac{-m_1 a_i}{m_1 a_i+1}+\log \left( 1+m_1 a_i\right) \right) +\sum _{i=1}^{k}\frac{n_i}{p}\frac{\log (-m_2)}{y_n} \\&\quad +\sum _{i=1}^{k_1}\frac{n_i}{p}\log \left( \frac{1-a_i}{1+m_2 a_i}\right) -\sum _{i=1}^{k_1}\frac{n_i}{p}\frac{(1+m_2)a_i}{(1+a_i m_2)(a_i-1)}. \end{aligned}$$
Subsequently, by direct evaluation, (12) and (13) can be computed as
$$\begin{aligned} (12)&=\sum _{i=1}^k \frac{n_i}{p}\frac{a_i y_n m_2^2-a_i(1+m_2)^2}{(m_2-m_1)(1+a_i m_2)(1+m_2)}+\sum _{i=1}^k \frac{n_i}{p}\frac{(1+m_2)^2-m_2^2 y_n}{(m_2-m_1)(1+m_2)^2}\\&\quad +\sum _{i=k_1+1}^k \frac{n_i}{p}\frac{a_i (y_n-(1-a_i)^2)}{(1+m_1 a_i)(1+m_2 a_i)(a_i-1)}+\sum _{i=1}^k \frac{n_i}{p}\frac{y_n(2a_i-2-y_n)}{(1+m_1)^2(1+m_2)^2(1-a_i)}.\\ (13)&=\left( 1-\frac{M}{p}\right) F^{y_n}\left( \log \left( x+1\right) \right) +\frac{1}{p}\sum _{i=1}^{k_1} n_i \log \left( a_i+\frac{y_n a_i}{a_i-1}+1\right) +{\mathcal {O}}\left( \frac{1}{n^2}\right) ,\\&=\left( 1-\frac{M}{p}\right) F^{y_n}\left( \log \left( x+1\right) \right) +\frac{1}{p}\sum _{i=1}^{k_1} n_i \log \left( \frac{(1+m_1 a_i)(1+m_2 a_i)}{1-a_i}\right) \\&\quad +{\mathcal {O}}\left( \frac{1}{n^2}\right) , \end{aligned}$$
where \(F^{y_n}\left( \log \left( x+1\right) \right) \) is given in (9). As a result, when \(y\in [1,\infty )\),
$$\begin{aligned}&F^{y_n,H_p}\left( \log \left( x+1\right) \right) \nonumber \\&\quad =\left( 1-\frac{M}{p}\right) F^{y_n}\left( \log \left( x+1\right) \right) +\sum _{i=1}^{k}\frac{n_i}{p}\log \left( 1+m_1 a_i\right) \nonumber \\&\qquad +\sum _{i=1}^{k} \frac{n_i}{p}\frac{\log (-m_2)}{y_n}+{\mathcal {O}}\left( \frac{1}{n^2}\right) \nonumber \\&\qquad +\sum _{i=1}^{k}\frac{n_i}{p}\frac{(1+m_2) a_i}{(1+m_2 a_i)(1-a_i)}+\sum _{i=1}^{k}\frac{n_i}{p}\frac{a_i y_n m_2^2-a_i(1+m_2)^2}{(m_2-m_1)(1+a_i m_2)(1+m_2)}\nonumber \\&\qquad +\sum _{i=1}^{k}\frac{n_i}{p} \frac{(1+m_2)^2-m_2^2y_n}{(m_2-m_1)(1+m_2)^2}+\sum _{i=1}^{k}\frac{n_i}{p}\frac{y_n(2a_i-2-y_n)}{(1+m_1)^2(1+m_2)^2(1-a_i)}.\nonumber \\ \end{aligned}$$
(39)
For \(F^{y_n,H_p}\left( 1/(x+1/b)\right) \), proceeding as in the calculation in Theorem 4, we have
$$\begin{aligned} \begin{aligned}&F^{y_n,H_p}\left( \frac{1}{x+\frac{1}{b}}\right) \\&\quad =\left( 1-\frac{M}{p}\right) F^{y_n}\left( \frac{1}{x+\frac{1}{b}}\right) +\frac{1}{p}\sum _{i=1}^k \frac{n_i b(1-a_i)}{p(1+\widetilde{m_1} a_i)(1+\widetilde{m_2} a_i)}+{\mathcal {O}}\left( \frac{1}{n^2}\right) \\&\qquad +\sum _{i=1}^k \frac{n_i}{p}\frac{\widetilde{m_2}^2(1+\widetilde{m_2})ba_i^2}{(\widetilde{m_2}-\widetilde{m_1})(1+a_i\widetilde{m_2})^2}-\sum _{i=1}^k\frac{n_i}{p}\frac{(1+\widetilde{m_2})b}{(\widetilde{m_2}-\widetilde{m_1})y_n}\\&\qquad +\sum _{i=1}^k \frac{n_i b^2}{p}\frac{a_i(1+\widetilde{m_2})((2a_i-1)\widetilde{m_2}^2+\widetilde{m_1}+\widetilde{m_2}+3\widetilde{m_1}\widetilde{m_2}+2a_i\widetilde{m_1}\widetilde{m_2}^2)}{(\widetilde{m_1}-\widetilde{m_2})^3(1+a_i\widetilde{m_2})^2}\\&\qquad -\sum _{i=1}^k \frac{n_i y_n b^2}{p}\frac{a_i \widetilde{m_2}^2(-\widetilde{m_2}+3\widetilde{m_1}+2a_i\widetilde{m_1}\widetilde{m_2})}{(\widetilde{m_1}-\widetilde{m_2})^3(1+a_i \widetilde{m_2})^2}-\sum _{i=1}^k \frac{n_i b^2}{p}\frac{\widetilde{m_1}+\widetilde{m_2}+2\widetilde{m_1}\widetilde{m_2}}{(\widetilde{m_1}-\widetilde{m_2})^3}\\&\qquad -\sum _{i=1}^k \frac{n_i b^2 y_n}{p}\frac{1}{(1+\widetilde{m_1})^2(1+\widetilde{m_2})^2}+\sum _{i=1}^k \frac{n_i b^2 y_n}{p}\frac{\widetilde{m_2}^2(\widetilde{m_2}-3\widetilde{m_1}-2\widetilde{m_1}\widetilde{m_2})}{(\widetilde{m_2}+1)^2(\widetilde{m_2}-\widetilde{m_1})^3}, \end{aligned} \end{aligned}$$
(40)
where
$$\begin{aligned} F^{y_n}\left( \frac{1}{x+\frac{1}{b}}\right)&=-\frac{1}{2}\left[ \frac{1}{y_n}+\frac{(1-\widetilde{y_1})^2}{y_n(\widetilde{y_1}-\widetilde{y_2})\widetilde{y_1} \left( \widetilde{y_1}+\sqrt{y_n}\right) \left( \widetilde{y_1}+\frac{1}{\sqrt{y_n}}\right) } \right. \nonumber \\&\left. \quad +\frac{1-y_n}{y_n^2\left( \widetilde{y_1}+\frac{1}{\sqrt{y_n}}\right) \left( \widetilde{y_2}+\frac{1}{\sqrt{y_n}}\right) }\right] +\left( 1-\frac{1}{y_n}\right) b. \end{aligned}$$
(41)
By substituting 1 for b in (40), we can obtain the expression of \(F^{y_n,H_p}(1/(x+1))\). By (35), we know that \({\text {tr}}({\varvec{\varSigma }}_p)-pF^{y_n,H_p}(x)={\mathcal {O}}(1/p)\). Thus the second term of (34) disappears when \(p\rightarrow \infty \). Furthermore, \({\text {tr}}({\varvec{\varSigma }}_p)-p=\sum _{i=1}^k n_i(a_i-1)\), which is a fixed constant. From (40) we can deduce that as \(n, p\rightarrow \infty \),
$$\begin{aligned} \left( {\text {tr}}({\varvec{\varSigma }}_p)-p\right) F^{y_n,H_p}\left( \frac{1}{x+1}\right) -\left( {\text {tr}}({\varvec{\varSigma }}_p)-p\right) F^{y_n}\left( \frac{1}{x+1}\right) \rightarrow 0, \end{aligned}$$
where \(F^{y_n}(1/(x+1))\) is given in (18) with y replaced by \(y_n\). As a result, (34) can reduce to
$$\begin{aligned}&-\frac{2}{n}\log T+\left( {\text {tr}}({\varvec{\varSigma }}_p)-p\right) F^{y_n}\left( \frac{1}{x+1}\right) +p F^{y_n,H_p}\left( \log (x+1)\right) \\&\quad -p\log \left( {\text {tr}}\left( {\varvec{\varSigma }}_p\right) \right) \Rightarrow N(\mu ,V). \end{aligned}$$
The proof is complete. \(\square \)
1.6 Proof of Corollary 1
Proof
-
(1)
When \(p/n\rightarrow y \in (0,1)\), combining Theorems 1 and 4 yields
$$\begin{aligned} \beta (\alpha )&=1-\varPhi \left( \varPhi ^{-1}(1-\alpha ) \right. \\&\left. \quad -\frac{\lim \limits _{n\rightarrow \infty }\left( {\text {tr}}({\varvec{\varSigma }}_p)-p\right) F^{y_n}\left( \frac{1}{x+1}\right) -\lim \limits _{n\rightarrow \infty }p\log ({\text {tr}}({\varvec{\varSigma }}_p)/p)+\varDelta _1}{\sqrt{V}}\right) , \end{aligned}$$
where
$$\begin{aligned} \lim \limits _{n\rightarrow \infty }\left( {\text {tr}}({\varvec{\varSigma }}_p)-p\right) F^{y_n}\left( \frac{1}{x+1}\right)&=\sum _{i=1}^k n_i(a_i-1) F^{y}\left( \frac{1}{x+1}\right) ,\\ \lim \limits _{n\rightarrow \infty }p\log ({\text {tr}}({\varvec{\varSigma }}_p)/p)&=\lim \limits _{n\rightarrow \infty }p\log \left( 1-\frac{M}{p}+\frac{\sum _{i=1}^k n_i a_i}{p}\right) \\&=\sum _{i=1}^k n_i(a_i-1), \end{aligned}$$
and
$$\begin{aligned} \begin{aligned} \varDelta _1&=\lim _{p\rightarrow \infty } \left[ pF^{y_n,H_p}\left( \log (x+1)\right) -pF^{y_n}\left( \log (x+1)\right) \right] \\&=-MF^{y}\left( \log \left( x+1\right) \right) +\sum _{i=1}^{k}n_i\log \left( 1+{\widehat{m}}_1 a_i\right) +\sum _{i=1}^{k} \frac{n_i\log (-{\widehat{m}}_2)}{y}\\&\quad +\sum _{i=1}^{k}\frac{n_i(1+{\widehat{m}}_2) a_i}{(1+{\widehat{m}}_2 a_i)(1-a_i)}+\sum _{i=1}^{k}\frac{n_i a_i y {\widehat{m}}_2^2-n_i a_i(1+{\widehat{m}}_2)^2}{({\widehat{m}}_2-{\widehat{m}}_1)(1+a_i {\widehat{m}}_2)(1+{\widehat{m}}_2)}\\&\quad +\sum _{i=1}^{k} \frac{n_i(1+{\widehat{m}}_2)^2-n_i {\widehat{m}}_2^2y}{({\widehat{m}}_2-{\widehat{m}}_1)(1+{\widehat{m}}_2)^2}+\sum _{i=1}^{k}\frac{n_i y(2a_i-2-y)}{(1+{\widehat{m}}_1)^2(1+{\widehat{m}}_2)^2(1-a_i)}, \end{aligned} \end{aligned}$$
(42)
where \(F^{y}(1/(1+x))\) is given in (4), \(F^{y}\left( \log (x+1)\right) \) is given in (5), \({\widehat{m}}_1=(\sqrt{y^2+4}-y)/2\) and \({\widehat{m}}_2=(-y-\sqrt{y^2+4})/2\). As a consequence,
$$\begin{aligned} \beta (\alpha )=1-\varPhi \left( \varPhi ^{-1}(1-\alpha )-\frac{\sum \limits _{i=1}^k n_i(a_i-1)\left( 1-F^{y}\left( \frac{1}{x+1}\right) \right) +\varDelta _1}{\sqrt{V}}\right) . \end{aligned}$$
-
2)
When \(p/n\rightarrow y\in [1,+\infty )\), combining Theorem 2 and Theorem 5 yields
$$\begin{aligned} \beta (\alpha )&=1-\varPhi \left( \varPhi ^{-1}(1-\alpha ) \right. \\&\left. \quad -\frac{\lim \limits _{n\rightarrow \infty }\left( {\text {tr}}({\varvec{\varSigma }}_p)-p\right) F^{y_n}\left( \frac{1}{x+1}\right) -\lim \limits _{n\rightarrow \infty }p\log ({\text {tr}}({\varvec{\varSigma }}_p)/p)+\varDelta _2}{\sqrt{V}}\right) , \end{aligned}$$
where
$$\begin{aligned} \lim \limits _{n\rightarrow \infty }\left( {\text {tr}}({\varvec{\varSigma }}_p)-p\right) F^{y_n}\left( \frac{1}{x+1}\right)&=\sum _{i=1}^k n_i(a_i-1) F^{y}\left( \frac{1}{x+1}\right) ,\\ \lim \limits _{n\rightarrow \infty }p\log ({\text {tr}}({\varvec{\varSigma }}_p)/p)&=\lim \limits _{n\rightarrow \infty }p\log \left( 1-\frac{M}{p}+\frac{\sum _{i=1}^k n_i a_i}{p}\right) \\&=\sum _{i=1}^k n_i(a_i-1), \end{aligned}$$
and
$$\begin{aligned} \begin{aligned} \varDelta _2&=\lim _{p\rightarrow \infty } \left[ pF^{y_n,H_p}\left( \log (x+1)\right) -pF^{y_n}\left( \log (x+1)\right) \right] \\&=-MF^{y}\left( \log \left( x+1\right) \right) +\sum _{i=1}^{k}n_i\log \left( 1+{\widehat{m}}_1 a_i\right) +\sum _{i=1}^{k} \frac{n_i\log (-{\widehat{m}}_2)}{y}\\&\quad +\sum _{i=1}^{k}\frac{n_i(1+{\widehat{m}}_2) a_i}{(1+{\widehat{m}}_2 a_i)(1-a_i)}+\sum _{i=1}^{k}\frac{n_i a_i y {\widehat{m}}_2^2-n_i a_i(1+{\widehat{m}}_2)^2}{({\widehat{m}}_2-{\widehat{m}}_1)(1+a_i {\widehat{m}}_2)(1+{\widehat{m}}_2)}\\&\quad +\sum _{i=1}^{k} \frac{n_i(1+{\widehat{m}}_2)^2-n_i {\widehat{m}}_2^2y}{({\widehat{m}}_2-{\widehat{m}}_1)(1+{\widehat{m}}_2)^2}+\sum _{i=1}^{k}\frac{n_i y(2a_i-2-y)}{(1+{\widehat{m}}_1)^2(1+{\widehat{m}}_2)^2(1-a_i)}, \end{aligned} \end{aligned}$$
(43)
where \(F^{y}(1/(1+x))\) is given in (6), \(F^{y}\left( \log (x+1)\right) \) is given in (7), \({\widehat{m}}_1=(\sqrt{y^2+4}-y)/2\) and \({\widehat{m}}_2=(-y-\sqrt{y^2+4})/2\). As a consequence,
$$\begin{aligned} \beta (\alpha )=1-\varPhi \left( \varPhi ^{-1}(1-\alpha )-\frac{\sum \limits _{i=1}^k n_i(a_i-1)\left( 1-F^{y}\left( \frac{1}{x+1}\right) \right) +\varDelta _2}{\sqrt{V}}\right) . \end{aligned}$$
\(\square \)
