Subgroup specific incremental value of new markers for risk prediction

Abstract

In many clinical applications, understanding when measurement of new markers is necessary to provide added accuracy to existing prediction tools could lead to more cost effective disease management. Many statistical tools for evaluating the incremental value (IncV) of the novel markers over the routine clinical risk factors have been developed in recent years. However, most existing literature focuses primarily on global assessment. Since the IncVs of new markers often vary across subgroups, it would be of great interest to identify subgroups for which the new markers are most/least useful in improving risk prediction. In this paper we provide novel statistical procedures for systematically identifying potential traditional-marker based subgroups in whom it might be beneficial to apply a new model with measurements of both the novel and traditional markers. We consider various conditional time-dependent accuracy parameters for censored failure time outcome to assess the subgroup-specific IncVs. We provide non-parametric kernel-based estimation procedures to calculate the proposed parameters. Simultaneous interval estimation procedures are provided to account for sampling variation and adjust for multiple testing. Simulation studies suggest that our proposed procedures work well in finite samples. The proposed procedures are applied to the Framingham Offspring Study to examine the added value of an inflammation marker, C-reactive protein, on top of the traditional Framingham risk score for predicting 10-year risk of cardiovascular disease.

Appendix

1.1 Appendix A

Let $\mathbb{P }_n$ and $\mathbb{P }$ denote expectation with respect to (wrt) the empirical probability measure of $\{(T_{i},\,\varDelta _{i},\, X_{i},\, Z_{i}),\,i=1,\ldots ,n\}$ and the probability measure of $(T,\,\varDelta ,\,X,\,Z),$ respectively, and  $\mathbb{G }_{n}=\sqrt{n}(\mathbb{P }_n-\mathbb{P }).$ We use $\dot{\mathcal{F }}(x)$ to denote $d\mathcal F (x)/dx$ for any function $\mathcal F ,\,\simeq$ to denote equivalence up to $o_{p}(1),$ and $\lesssim$ to denote being bounded above up to a universal constant. Let $\beta _{0}$ and $\gamma _{0}$ denote the solution to $E[V_{i}\{Y_{i}^{\dag }-g_{1}(\beta ^{\prime }V_{i})\}]=0$ and $E[W_{i}\{Y_{i}^{\dag }-g_{2}(\gamma ^{\prime }W_{i})\}]=0,$ respectively. Let $\bar{ p}_{1i}=g_{1}(\beta _{0}^{\prime }V_{i}),$ and $\bar{ p}_{2i}=g_{2}(\gamma _{0}^{\prime }W_{i}).$ Let $\omega = \varDelta I(T\le t_{0})/G_{X,\,Z}(T)+I(T> t_{0})/G_{X,\,Z}(t_{0}),\,{\widehat{M}}_{i}(c)=I({\widehat{p}}_{2i}\ge c)$ and $\bar{M}_{i}(c)=I(\bar{ p}_{2i}\ge c).$ For $y=0,\,1,$ let $f_{y}(c;\,s)$ denote the conditional density of $\bar{ p}_{2i}$ given $Y^{\dag }_i = y$ and $\bar{ p}_{1i}=s$ and we assumed that $f_{y}(c;\,s)$ is continuous and bounded away from zero uniformly in $c$ and $s.$ This assumption implies that $\mathrm{{ROC}}(u;s)$ has continuous and bounded derivative $\dot{\mathrm{{ROC}}}(u;\,s) = \partial \mathrm{{ROC}}(u;\,s)/\partial u.$ We assume that $V$ and $W$ are bounded, and $\tau (y;\,s) = \partial pr[\phi \{\bar{ p}_{1}(X)\} \le s,\,Y^{\dag }= y]/\partial s,$ is continuously differentiable with bounded derivatives and bounded away from zero. Throughout, the bandwidths are assumed to be of order $n^{-\nu }$ with $\nu \in (1/5,\,1/2).$ For ease of presentation and without loss of generality, we assume that $h_1 = h_0,$ denoted by $h,$ and suppress $h$ from the notations. Without loss of generality, we assume that $\sup _{t,x,z}|n^{\frac{1}{2}}\{{\widehat{G}}_{X,Z}(t)-G_{X,Z}(t)\}|=O_p(1).$ When $C$ is assumed to be independent of both $T$ and $(X,\,Z),$ the simple Kaplan–Meier estimator satisfies this condition. When $C$ depends on $(X,\,Z),\,{\widehat{G}}_{X,Z}$ obtained under the Cox model also satisfies this condition provided that $W_c$ is bounded. The kernel function $K$ is assumed to be symmetric, smooth with a bounded support on $[-1,\,1]$ and we let $m_2 = \int K(x)^2dx.$

1.1.1 A.1 Asymptotic expansions for $\widehat{\mathcal{{S}}}_y(c;\,s)$

Uniform convergence rate for $\widehat{\mathcal{{S}}}_y(c;\,s)$ We first establish the following uniform convergence rate of $\widehat{\mathcal{{S}}}_y(c;\,s)= g\{{\widehat{a}}_y(c;\,s)\}$:

$$\begin{aligned} \sup _{s\in \mathcal{{I}}_{h},c}|\widehat{\mathcal{S }}_{y}(c;\,s) - \mathcal S _{y}(c;\,s)|= O_{p}\{(nh)^{-\frac{1}{2}}\log (n)\} = o_{p}(1). \end{aligned}$$

(6)

To this end, we note that for any given $c$ and $s$,

$$\begin{aligned} {\widehat{\varvec{\zeta }}}_{y}(c;\,s) = \left[\begin{matrix}{\widehat{\zeta }}_{a_{y}}(c;\,s) \\ {\widehat{\zeta }}_{b_{y}}(c;\,s)\end{matrix}\right] =\left[\begin{matrix}{\widehat{a}}_{y}(c;\,s)-a_{y}(c;\,s) \\ {\widehat{b}}_{y}(c;\,s)-b_{y}(c;\,s) \end{matrix} \right] \end{aligned}$$

is the solution to the estimating equation ${\widehat{\varvec{\Psi }}}_{y}(\varvec{\zeta }_{y},\,c,\,s)=0,$ where $\varvec{\zeta }_{y}=(\zeta _{a_{y}},\,\zeta _{b_{y}})^{\prime }$ and

$$\begin{aligned} {\widehat{\varvec{\Psi }}}_{y}\left(\varvec{\zeta }_{y};\,c,\,s\right)&= \left[\begin{array}{c}{\widehat{\varvec{\Psi }}}_{y1}\left(\varvec{\zeta }_{y},\,c,\,s\right) \\ {\widehat{\varvec{\Psi }}}_{y2}\left(\varvec{\zeta }_{y},\,c,\,s\right) \end{array}\right]\\&= n^{-1}\displaystyle \sum _{i:\, Y_{i}=y}\widehat{w}_{i}\left[\begin{array}{c}1 \\ h^{-1}\widehat{\mathcal{E }}_{i1}(s)\end{array}\right]K_{h}\left\{ \widehat{\mathcal{E }}_{i1}(s)\right\} \\&\times \left[{\widehat{M}}_{i}(c) - \mathcal G \left\{ \varvec{\zeta }_{y},\,c,\,s;\,\phi \left({\widehat{p}}_{1i}\right),\,h\right\} \right], \end{aligned}$$

$a_{y}(c;\,s) = g^{-1}\{ { \mathcal S }_{y}(c;\,s)\},\,b_{y}(c;\,s)=\partial g^{-1}\{\mathcal{S }_{y}(c;\,s)\}/\partial s$ and $\mathcal G (\varvec{\zeta }_{y},\,c,\,s;\, e,\,h)=g[a_{y}(c;\,s)+b_{y}(c;\,s)\{e-\phi (s)\} + \zeta _{a_{y}}+\zeta _{b_{y}}h^{-1}\{e-\phi (s) \}].$ We next establish the convergence rate for $\sup _{\varvec{\zeta }_{y},\,c,\,s}|{\widehat{\varvec{\Psi }}}_{y}(\varvec{\zeta }_{y};\,c,\,s) - \varvec{\Psi }_{y}(\varvec{\zeta }_{y};\,c,\,s)|,$ where

$$\begin{aligned} \varvec{\Psi }_{y}\left(\varvec{\zeta }_{y};\,c,\,s\right)&= \left[\begin{array}{c} \varPsi _{y1}(\varvec{\zeta }_{y},\, c,\,s) \\ \varPsi _{y2}(\varvec{\zeta }_{y};\,c,\,s)\end{array}\right] \\&= \tau (y;\,s) \left[\begin{array}{c}\mathcal S _{y}(c;\,s)-\int K(t)g\{a_{y}(c;\,s)+\zeta _{a_{y}}+\zeta _{b_{y}}t\}dt\\ -\int tK(t)g\{a_{y}(c;\,s)+\zeta _{a_{y}}+\zeta _{b_{y}}t\}dt\end{array}\right]. \end{aligned}$$

We first show that

$$\begin{aligned} \sup _{s\in \mathcal{{I}}_{h},c}\left|n^{-1}\sum _{i:Y_{i}=y} {\widehat{\omega }}_{i} K_{h}\left\{ \widehat{\mathcal{E }}_{i1}(s)\right\} {\widehat{M}}_{i}(c) - \tau (y;\,s)\mathcal{{S}}_{y}(c;\,s)\right| \end{aligned}$$

and

$$\begin{aligned}&\sup _{\varvec{\zeta }_{y},s\in \mathcal{{I}}_{h},c}\left|n^{-1}\sum _{i:Y_{i}=y} {\widehat{\omega }}_{i} K_{h}\left\{ \widehat{\mathcal{E }}_{i1}(s)\right\} \mathcal G \left\{ \varvec{\zeta }_{y},\,c,\,s;\, \phi \left({\widehat{p}}_{1i}\right)\!,\,h\right\} \right. \\&\quad \quad \left.- \tau (y;\,s)\int K(t)g\left\{ a_{y}(c;\,s)+\zeta _{a_{y}}+\zeta _{b_{y}}t\right\} dt\right| \end{aligned}$$

are both $O_{p}\{(nh)^{-\frac{1}{2}}\log (n)\}$ where $\mathcal{{I}}_{h}=[\phi ^{-1}(\rho _{l}+h),\,\phi ^{-1}(\rho _{u}-h)]$ and $[\rho _{l},\,\rho _{u}]$ is a subset of the support of $\phi \{g_{1}(\beta _{0}^{T}V)\}.$ To this end, we note that since $\sup _{u} |{\widehat{G}}_{X,Z}(u)-G_{X,Z}(u)|=O_{p}(n^{-\frac{1}{2}})$ and $|{\widehat{\beta }}-\beta _{0}|=O_{p}(n^{-\frac{1}{2}}),$

$$\begin{aligned}&\left|n^{-1}\sum _{i:\,Y_{i}=y}\left({\widehat{\omega }}_{i} - \omega _{i}\right)K_{h}\left\{ \widehat{\mathcal{E }}_{i1}(s)\right\} \mathcal G \left\{ \varvec{\zeta }_{y},\,c,\,s;\, \phi \left({\widehat{p}}_{1i}\right),\,h\right\} \right|\\&\quad \le n^{-1}\sum _{i:\,Y_{i}=y} |{\widehat{\omega }}_{i}-\omega _{i}|K_{h}\left\{ \widehat{\mathcal{E }}_{i1}(s)\right\} \\&\quad = O_{p}\left(n^{-\frac{1}{2}}\right). \end{aligned}$$

This implies that

$$\begin{aligned}&\left|n^{-1}\sum _{i:\,Y_{i}=y} {\widehat{\omega }}_{i} K_{h}\left\{ \widehat{\mathcal{E }}_{i1}(s)\right\} \mathcal G \left\{ \varvec{\zeta }_{y},\,c,\,s;\, \phi \left({\widehat{p}}_{1i}\right),\,h\right\} - \tau (y;\,s)\int K(t)g\left\{ a_{y}(c;\,s)+\zeta _{a_{y}}+\zeta _{b_{y}}t\right\} dt\right|\\&\quad \le \left|n^{-\frac{1}{2}}\int K_{h}\{e-\phi (s)\}\mathcal G \left\{ \varvec{\zeta }_{y},\,c,\,s;\, \phi \left({\widehat{p}}_{1i}\right),\,h\right\} d \mathbb{G }_{n}\left[\omega I\left\{ \phi \left({\widehat{p}}_{i1}\right)\le e\right\} - \omega I\left\{ \phi \left(\bar{ p}_{i1}\right)\le e\right\} \right]\right|\\&\qquad + \left|\int K_{h}\{e-\phi (s)\}\mathcal G \left\{ \varvec{\zeta }_{y},\,c,\,s;\, \phi \left({\widehat{p}}_{1i}\right),\,h\right\} d\mathbb{P }\left[\omega I\left\{ \phi \left(\bar{ p}_{i1}\right)\le e\right\} \right] - \tau (y;\,s)\int K(t)g\left\{ a_{y}(c;\,s)+\zeta _{a_{y}}+\zeta _{b_{y}}t\right\} dt\right|\\&\qquad + \left|n^{-\frac{1}{2}}\int K_{h}\{e-\phi (s)\}d\mathbb{P }\left[\omega \mathcal G \left\{ \varvec{\zeta }_{y},\, c,\,s;\,\phi \left({\widehat{p}}_{1i}\right),\,h\right\} I\left\{ \phi \left(\bar{ p}_{i1}\right)\le e\right\} \right] \right| +O_{p}\left(n^{-\frac{1}{2}}\right)\lesssim n^{-\frac{1}{2}}h^{-1}\Vert \mathbb{G }_{n}\Vert _\mathcal{H _{\delta }} + \left|n^{-\frac{1}{2}}\int K_{h}\{e-\phi (s)\}d\mathbb{P }\left[\omega \mathcal G \left\{ \varvec{\zeta }_{y},\,c,\,s;\,\phi \left({\widehat{p}}_{1i}\right)\!,\,h\right\} I\left\{ \phi \left(\bar{ p}_{i1}\right)\le e\right\} \right] \right| + O_{p}\left(n^{-\frac{1}{2}}+h^{2}\right)\!, \end{aligned}$$

where $\mathcal H _{\delta }=\{\omega I[\phi \{g_{1}(\beta ^{\prime }v)\}\le e]-\omega I[\phi \{g_{1}(\beta _{0}^{\prime }v)\}\le e]:\,|\beta -\beta _{0}|\le \delta ,\,e\}$ is a class of functions indexed by $\beta$ and $e.$ By the maximum inequality of Van der Vaart and Wellner (1996), we have

$$\begin{aligned} E\Vert \mathbb{G }_{n}\Vert _\mathcal{H _{\delta }} \lesssim \delta ^{\frac{1}{2}} \{|\log (\delta )|+|\log (h)|\}\left[1 + \frac{\delta ^{\frac{1}{2}}\{|\log (\delta )|+|\log (h)|\}}{\delta n^{\frac{1}{2}}}\right] \end{aligned}$$

Together with the fact that $|{\widehat{\beta }}-\beta _{0}|=O_{p}(n^{-\frac{1}{2}})$ from Uno et al. (2007), it implies that $n^{-\frac{1}{2}}h^{-1}\Vert \mathbb{G }_{n}\Vert _\mathcal{H _{\delta }} = O_{p}\{(nh)^{-\frac{1}{2}}(nh^{2})^{-\frac{1}{4}}\log (n)\}.$ In addition, with the standard arguments used in Bickel and Rosenblatt (1973), it can be shown that

$$\begin{aligned}&\left|n^{-\frac{1}{2}}\int K_{h}\{e-\phi (s)\}d\mathbb{P }\left[\omega \mathcal G \left\{ \varvec{\zeta }_{y},\,c,\,s;\,\phi \left({\widehat{p}}_{1i}\right)\!,\,h\right\} I\left\{ \phi \left(\bar{ p}_{i1}\right)\le e\right\} \right] \right|\\&\quad =O_{p}\left\{ (nh)^{-\frac{1}{2}}\log (n)\right\} . \end{aligned}$$

Therefore, for $h=n^{-\nu },\,1/5 <\nu < 1/2,$

$$\begin{aligned}&\sup _{\varvec{\zeta }_{y},s\in \mathcal{{I}}_{h},c}\left|n^{-1}\sum _{i:\,Y_{i}=y} {\widehat{\omega }}_{i} K_{h}\left\{ \widehat{\mathcal{E }}_{i1}(s)\right\} \mathcal G \left\{ \varvec{\zeta }_{y},\,c,\,s;\, \phi \left({\widehat{p}}_{1i}\right),\,h\right\} \right.\\&\quad \left. - \tau (y;\,s)\int K(t)g\left\{ a_{y}(c;\,s)+\zeta _{a_{y}}+\zeta _{b_{y}}t\right\} dt\right| \end{aligned}$$

is $O_{p}\{(nh)^{-\frac{1}{2}}\log (n)\}.$ Following with similar arguments as given above, coupled with the fact that $|{\widehat{\gamma }}-\gamma _{0}|=O_{p}(n^{-\frac{1}{2}}),$ we have

$$\begin{aligned} \sup _{s\in \mathcal{{I}}_{h},c \!\in \! [0,1]}\left|\!n\!^{-1}\!\sum _{i:\,Y_{i}\!=\!y} {\widehat{\omega }}_{i} K_{h}\left\{ \widehat{\mathcal{E }}_{i1}(s)\right\} {\widehat{M}}_{i}(c) \!-\! \tau (y;\,s)\mathcal{{S}}_{y}(c;\,s)\right|\!=\! O_{p}\left\{ (nh)^{-\frac{1}{2}}\log (n)\right\} \!. \end{aligned}$$

Thus, $\sup _{\varvec{\zeta }_{y},c,s} |{\widehat{\Psi }}_{y1}(\varvec{\zeta }_{y};c,s) - \varPsi _{y1}(\varvec{\zeta }_{y};c,s)|=O_{p}\{(nh)^{-\frac{1}{2}}\log (n)\}=o_{p}(1).$ It follows from the same arguments as given above that

$$\begin{aligned} \sup _{\varvec{\zeta }_{y},c,s}\left|{\widehat{\Psi }}_{y2}\left(\varvec{\zeta }_{y};\,c,\,s\right) - \varPsi _{y2}\left(\varvec{\zeta }_{y};\,c,\,s\right)\right|=O_{p}\left\{ (nh)^{-\frac{1}{2}}\log (n)+h\right\} =o_{p}(1). \end{aligned}$$

Therefore, $\sup _{\varvec{\zeta }_{y},c,s} |{\widehat{\varvec{\Psi }}}_{y}(\varvec{\zeta }_{y};\,c,\,s)-\varvec{\Psi }_{y}(\varvec{\zeta }_{y};\,c,\,s)|=o_{p}(1).$ In addition, we note that $\mathbf 0$ is the unique solution to the equation $\varvec{\Psi }_{y}(\varvec{\zeta }_{y};\,c,\,s) = 0$ wrt $\varvec{\zeta }_{y}.$ It suggests that $\sup _{s,c}|{\widehat{\varvec{\zeta }}}_{a_{y}}(c;\,s)|=O_{p}\{(nh)^{-\frac{1}{2}}\log (n)\}=o_{p}(1),$ which implies the consistency of $\mathcal S _{y}(c;\,s),$

$$\begin{aligned} \sup _{s\in \mathcal{{I}}_{h},c\in [0,1]}\left|\widehat{\mathcal{S }}_{y}(c;\,s) - \mathcal S _{y}(c;\,s)\right|=O_{p}\left\{ (nh)^{-\frac{1}{2}}\log (n)\right\} = o_{p}(1). \end{aligned}$$

Asymptotic expansion for $\widehat{\mathcal{{S}}}_y(c;\,s)$ Let ${\widehat{d}}_{y}(c;\,s)=\sqrt{nh}\{{\widehat{a}}_{y}(c;\,s)-a_{y}(c;\,s)\}.$ It follows from a Taylor series expansion and the convergence rate of $\varvec{\zeta }_y(c;\,s)$ that

$$\begin{aligned} {\widehat{d}}_{y}(c;\,s) = \frac{\sqrt{nh}\mathbb{P }_{n}({\widehat{\omega }}I(Y=y)K_{h}\{\widehat{\mathcal{E }}_{1}(s)\}[{\widehat{M}}(c) - \mathcal G ^{0}_{y}\{c,\,s;\,\phi ({\widehat{p}}_{1})\}])}{\tau \{y;\,\phi (s)\} \dot{g}\{a_{y}(c;\,s)\}} + o_p(1), \nonumber \\ \end{aligned}$$

(7)

where $\mathcal G ^{0}_{y}(c,\,s;\,e) = g[a_{y}(c;\,s)+b_{y}(c;\,s)\{e-\phi (s)\}].$ Furthermore, since $\sup _{t\le t_{0}} |{\widehat{G}}_{X,Z}(t)-G_{X,Z}(t)|=O_{p}(n^{-1/2}),$

$$\begin{aligned} {\widehat{d}}_{y}(c;\,s)=\frac{\sqrt{nh}\mathbb{P }_{n}(\omega I(Y=y)K_{h}\{\widehat{\mathcal{E }}_{1}(s)\}[{\widehat{M}}(c) - \mathcal G ^{0}_{y}\{c,\,s;\,\phi ({\widehat{p}}_{1})\}])}{\tau \{y;\,\phi (s)\} \dot{g}\{a_{y}(c;\,s)\}} + o_p(1). \end{aligned}$$

We next show that $\widehat{d}_{y}(c;\,s)$ is asymptotically equivalent to

$$\begin{aligned} \widetilde{d}_{y}(c;\,s) = \frac{\sqrt{nh}\mathbb{P }_n(\omega I(Y=y)K_{h}\{\bar{\mathcal{E }}_{1}(s)\}[\bar{M}(c) - \mathcal G _{y}^{0} \{c,\,s;\,\phi (\bar{ p}_{1})\}])}{\tau \{y;\,\phi (s)\} \dot{g}\{a_{y}(c;\,s)\}}, \end{aligned}$$

(8)

where $\bar{\mathcal{E }}_{1}(s) = \phi (\bar{ p}_{1})-\phi (s).$ From (8) and the fact that $\tau \{y;\,\phi (s)\}$ is bounded away from 0 uniformly in $s,$ we have

$$\begin{aligned}&\left|\widehat{d}_{y}(s)-\widetilde{d}_{y}(s)\right|\\&\quad \lesssim h^{\frac{1}{2}}\left|\int \! K_{h}\{e\!-\!\phi (s)\}d\mathbb{G }_{n}\left(\!I(Y\!=\! y)\omega \left[{\widehat{M}}(c)I\left\{ \phi \left({\widehat{p}}_{1}\right)\!\le \! e\right\} \!-\!\bar{M}(c) I\left\{ \phi \left(\bar{ p}_{1}\right)\le e\right\} \right]\,\right)\right|\\&\qquad +h^{\frac{1}{2}}\left|\!\int \!K_{h}\{e\!-\!\phi (s)\}\mathcal G _{y}(c,\,s;\,e)d\mathbb{G }_{n}\left(\!I(Y \!=\!y)\left[\omega I\left\{ \phi \left({\widehat{p}}_{1}\right)\!\le \! e\right\} \!-\!\omega I\left\{ \phi \left(\bar{ p}_{1}\right)\!\le \!e\right\} \right]\,\right)\right|\\&\qquad +\left|\!\sqrt{nh}\!\int \!K_{h}\{e\!-\!\phi (s)\} d\mathbb{P }\left(\!I(Y\!=\! y)\left[\omega {\widehat{M}}(c)I\left\{ \phi \left({\widehat{p}}_{1}\right)\!\le \! e\right\} \!-\!\omega \bar{M}(c)I\left\{ \phi \left(\bar{ p}_{1}\right)\!\le \!e\right\} \right]\,\right)\right|\\&\qquad + \left|\sqrt{nh}\int K_{h}\{e\!-\!\phi (s)\} d\mathbb{P }\left(I(Y\!=\! y)\left[\omega \mathcal G _{y}\left\{ c,\,s;\,\phi \left({\widehat{p}}_{1}\right)\right\} I\left\{ \phi \left({\widehat{p}}_{1}\right)\le e\right\} \right.\right. \right.\\&\qquad -\left. \left. \left. \omega \mathcal G _{y}\left\{ c,\,s;\,\phi \left(\bar{ p}_{1}\right)\right\} I\left\{ \phi \left(\bar{ p}_{1}\right)\le e\right\} \right]\right)\right|\\&\quad \lesssim h^{\frac{1}{2}}\left\Vert\mathbb{G }_{n}\right\Vert_\mathcal{F _{\delta }}\!+\! h^{\frac{1}{2}}\left\Vert\mathbb{G }_{n}\right\Vert_\mathcal{H _{\delta }} \!+\!O_{p}\left\{ (nh)^{1/2}\left|{\widehat{\beta }}-\beta _{0}\right|\!+\!\left|{\widehat{\gamma }}\!-\!\gamma _{0}\right|\!+\!h^{2}\right\} , \end{aligned}$$

where $\mathcal F _{\delta }\!=\!\{\omega I\{g_{2}(\gamma ^{\prime }w)\!\ge \! c\}I[\phi \{g_{1}(\beta ^{\prime }v)\}\le e]\!-\!\omega I\{g_{2}(\gamma _{0}^{\prime }w)\!\ge \! c\} I[\phi \{g_{1}(\beta _{0}^{\prime }v)\}\!\le \! e]:\,|\gamma -\gamma _{0}|+|\beta -\beta _{0}|\le \delta ,\,e\}$ is the class of functions indexed by $\gamma ,\,\beta$ and $e.$ By the maximum inequality of Van der Vaart and Wellner (1996) and the fact that $|{\widehat{\beta }}-\beta _{0}|+|{\widehat{\gamma }}-\gamma _{0}|=O_{p}(n^{-\frac{1}{2}})$ from Uno et al. (2007), we have $h^{\frac{1}{2}}\left\Vert\mathbb{G }_{n}\right\Vert_\mathcal{F _{\delta }} = O_{p}\{h^{-\frac{1}{2}}n^{-\frac{1}{4}}\log (n)\}$ and $h^{\frac{1}{2}}\left\Vert\mathbb{G }_{n}\right\Vert_\mathcal{H _{\delta }} = O_{p}\{h^{-\frac{1}{2}}n^{-\frac{1}{4}}\log (n)\}.$ It follows that $\sup _{s}|\widehat{d}_{y}(s)-\widetilde{d}_{y}(s)|=o_{p}(1).$ Then, by a delta method,

$$\begin{aligned}&\widehat{\mathcal{{W}}}_{\mathcal{{S}}_y}(c;\,s)=\sqrt{nh}\left\{ \widehat{\mathcal{S }}_{y}(c;\,s)- \mathcal S _{y}(c;\,s)\right\} \simeq \sqrt{nh}\ \mathbb{P }_n\left[K_{h}\left\{ \bar{\mathcal{E }}_{1}(s)\right\} \mathcal{{D}}_{\mathcal{{S}}_y}(c;\,s) \right]\qquad \end{aligned}$$

(9)

$$\begin{aligned}&\text{ where}\qquad \mathcal{{D}}_{\mathcal{{S}}_y}(c;\,s)= \tau \{y;\,\phi (s)\}^{-1}\omega I(Y=y)\left\{ \bar{M}(c)- \mathcal S _{y}(c;\,s)\right\} \qquad \qquad \end{aligned}$$

(10)

Using the same arguments as for establishing the uniform convergence rate of conditional Kaplan–Meier estimators (Dabrowska 1989; Du and Akritas 2002), we obtain (6). Furthermore, following similar arguments as given in Dabrowska (1987, 1997), we have $\widehat{\mathcal{{W}}}_{\mathcal{{S}}_y}(c;\,s)$ converges weakly to a Gaussian process in $c$ for all $s.$ Note that as for all kernel estimators, $\widehat{\mathcal{{W}}}_{\mathcal{{S}}_y}(c;\,s)$ does not converge as a process in $s.$

1.1.2 A.2 Uniform consistency of $\widehat{\text{ pAUC}}_{f}(s)$

Next we establish the uniform convergence rate for $\widehat{\mathrm{{ROC}}}(u;\,s).$ To this end, we write

$$\begin{aligned} \widehat{\text{ ROC}}(u;\,s) - \text{ ROC}(u;\,s) = {\widehat{\varepsilon }}_1(u;\,s)+{\widehat{\varepsilon }}_0(u;\,s), \end{aligned}$$

where ${\widehat{\varepsilon }}_1(u;\,s) = { \widehat{ \mathcal S }}_{1}\{ { \widehat{\mathcal{S }}}_{0}^{-1}(u;\,s);\,s\} -{ \mathcal S }_{1}\{ { \widehat{\mathcal{S }}}_{0}^{-1}(u;\,s);\,s\}$ and  ${\widehat{\varepsilon }}_0(u;\,s)\!=\! { \mathcal S }_{1}\{ { \widehat{\mathcal{S }}}_{0}^{-1} (u;\,s); \,s\}\!-\! \mathcal{S }_{1}\{ { \mathcal S }_{0}^{-1}(u;\,s);\,s\}.$ It follows from (6) that $\sup _{u;s}|{\widehat{\varepsilon }}_1(u;s)|\!\le \! \sup _{c;s}|\widehat{ { \mathcal S }}_{1} (c;\,s)\!-\!\mathcal{{S}}_1(c;\,s)|.$ Let $\widehat{\mathcal{I}}(u;\,s)\!=\!\mathcal{{S}}_0\{\widehat{\mathcal{{S}}}_0^{-1}(u;\,s);\,s\}.$ Then ${\widehat{\varepsilon }}_0(u;\,s)\!=\!\mathrm{{ROC}}\{\widehat{\mathcal{I}}(u;\,s);s\} - \mathrm{{ROC}}(u;\,s).$ Noting that $\sup _{u}|\widehat{\mathcal{I}}(u;\,s)\!-\!u|\!=\!\sup _u |\widehat{\mathcal{I}}(u;\,s)\!-\! \widehat{\mathcal{{S}}}_0\{\widehat{\mathcal{{S}}}_0^{-1}(u;\,s);\,s\}|\!+\!n^{-1}\!\le \!\sup _c |\mathcal{{S}}_0(c;\,s)\!-\!\widehat{\mathcal{{S}}}_0(c;\,s)|\!+\!n^{-1}\!=\!O_{p}\{(nh)^{-1/2}\log n\},$ we have ${\widehat{\varepsilon }}_0(u;\,s)=O_{p}\{(nh)^{-1/2}\log n\}$ by the continuity and boundedness of $\dot{\mathrm{{ROC}}}(u;\,s)$. Therefore,

$$\begin{aligned} \sup _{u,s}|\widehat{\text{ ROC}}(u;\,s) - \text{ ROC}(u;\,s)| = O_p\left\{ (nh)^{-1/2}\log n\right\} \end{aligned}$$

which implies

$$\begin{aligned}&\displaystyle \sup _{s \in \mathcal{{I}}_h}\left|\widehat{\mathrm{{pAUC}}}_f(s) - \mathrm{{pAUC}}_f(s)\right|\\&\qquad \lesssim \displaystyle \sup _{s \in \mathcal{{I}}_h} \int \limits _{0}^{f} \left|\widehat{\text{ ROC}}(u;\,s)-\text{ ROC}(u;\,s)\right| du =O_p\left\{ (nh)^{-\frac{1}{2}}\log n\right\} . \end{aligned}$$

and hence the uniform consistency of $\widehat{\mathrm{{pAUC}}}_f(s).$

1.1.3 A.3 Asymptotic distribution of $\widehat{\mathcal{{W}}}_{\mathrm{{\tiny pAUC}}_f}(s)$

To derive the asymptotic distribution for $\widehat{\mathcal{{W}}}_{\mathrm{{\tiny pAUC}}_f}(s),$ we first derive asymptotic expansions for $\widehat{\mathcal{{W}}}_{\mathrm{{\tiny ROC}}}(u;\,s)=\sqrt{nh}\{\widehat{\text{ ROC}}(u;\,s) - \text{ ROC}(u;\,s)\} = \sqrt{nh}\ {\widehat{\varepsilon }}_1(u;\,s) + \sqrt{nh}\ {\widehat{\varepsilon }}_0(u;\,s).$ From the weak convergence of $\widehat{\mathcal{{W}}}_{S_y}(c;\,s)$ in $c,$ the approximation in (9), and the consistency of $\widehat{\mathcal{{S}}}_0^{-1}(c;\,s)$ given in the Appendix A.2, we have

$$\begin{aligned} \sqrt{nh}\ {\widehat{\varepsilon }}_1(u;\,s)&\simeq \, \sqrt{nh}\left[{ \widehat{ \mathcal S }}_{1}\left\{ { \mathcal S }_{0}^{-1}(u;\,s);\,s\right\} - \mathrm{{ROC}}(u;\,s) \right]\\&\simeq \, \sqrt{nh}\mathbb{P }_n\left[K_{h}\left\{ { \bar{\mathcal{E }}}_{1}(s)\right\} \mathcal{{D}}_{\mathcal{{S}}_1}\left\{ { \mathcal S }_{0}^{-1}(u;\,s);\,s\right\} \right] \end{aligned}$$

On the other hand, from the uniform convergence of $\widehat{\mathcal{I}}_0(u;\,s) \rightarrow u$ and the weak convergence of $\widehat{\mathcal{{D}}}_0(c;\,s)$ in $c,$ we have

$$\begin{aligned} \sqrt{nh}\left\{ u -\widehat{\mathcal{I}}(u;\,s) \right\}&\simeq \, \sqrt{nh}\left[\widehat{\mathcal{I}}^{-1}\left\{ \widehat{\mathcal{I}}(u;\,s);\,s\right\} -\widehat{\mathcal{I}}(u;\,s) \right] \simeq \sqrt{nh}\left\{ \widehat{\mathcal{I}}^{-1}(u;\,s) -u \right\} \\&\simeq \, \sqrt{nh}\left[\widehat{\mathcal{{S}}}_0\{\mathcal{{S}}_0^{-1}(u;s);s\} - u\right] \end{aligned}$$

This, together with a Taylor series expansion and the expansion given (9), implies that

$$\begin{aligned} \sqrt{nh}\ {\widehat{\varepsilon }}_0(u;\,s) \simeq - \dot{\mathrm{{ROC}}}(u;\,s) \mathbb{P }_n\left[K_{h}\left\{ \bar{\mathcal{E }}_{1}(s)\right\} \mathcal{{D}}_{\mathcal{{S}}_0}\left\{ \mathcal S _{0}^{-1}(u;\,s);\,s\right\} \right] \end{aligned}$$

It follows that

$$\begin{aligned} \widehat{\mathcal{{W}}}_{\mathrm{{\tiny pAUC}}_f}(s)&\simeq&\sqrt{nh}\mathbb{P }_n\left[K_{h}\left\{ \bar{\mathcal{E }}_{1}(s)\right\} \mathcal{{D}}_{\mathrm{{\tiny pAUC}}_f}(s) \right] \end{aligned}$$

(11)

$$\begin{aligned} \text{ where} \quad \mathcal{{D}}_{\mathrm{{\tiny pAUC}}_f}(s)&\!=\!&\! \int \limits _0^f \!\left[ \mathcal{{D}}_{\mathcal{{S}}_1}\!\left\{ \mathcal{{S}}_0^{-1}(u;\,s);\,s \right\} \!-\! {\dot{\mathrm{{ROC}}}}(u;\,s)\mathcal{{D}}_{\mathcal{{S}}_0}\!\left\{ \mathcal{{S}}_0^{-1}(u;\,s);\,s \right\} \right] du .\nonumber \\ \end{aligned}$$

(12)

It then follows from a central limit theorem that for any fixed $s,\,\widehat{\mathcal{{W}}}_{\mathrm{{\tiny pAUC}}_f}(s)$ converges to a normal with mean 0 and variance

$$\begin{aligned} \sigma _{\mathrm{{\tiny pAUC}}_f}^{2}(s)\!=\!m_{2}\left[\tau \{1;\,\phi (s)\}\dot{F}_{\small \phi (\bar{ p}_{1})}(s)\right]^{-1} \sigma _{1}^{2}(s)\!+\!m_{2}\left[\tau \{0;\,\phi (s)\}\dot{F}_{\small \phi (\bar{ p}_{1})}(s)\right]^{-1}\sigma _{0}^{2}(s), \end{aligned}$$

where $\dot{F}_{\small \phi (\bar{ p}_{1})}(s)$ is the density function of $\phi (\bar{ p}_{1}),$

$$\begin{aligned} \sigma _{1}^{2}(s)\!&= \! \mathrm{{E}}\!\left.\left(\!G\left(T^{\dag }\right)^{-1}\!\left[\int \limits _{0}^{f} \bar{M}\left\{ { \mathcal S }_{0}^{-1}(u;\,s)\right\} du\!-\!\mathrm{{pAUC}}_{f}(s)\right]^{2} \right| \bar{ p}_{1}\!=\!s,\;Y^{\dag }\!=\!1\right)\!,\quad \text{ and} \\ \sigma _{0}^{2}(s) \!&= \! \mathrm{{E}}\!\left.\left(\!G\left(t_{0}\right)^{-1}\!\left[\int \limits _{0}^{f} \bar{M}\left\{ { \mathcal S }_{0}^{-1}(u;\,s)\right\} d\mathrm{{ROC}}(u;\,s)\!-\!\int \limits _{0}^{f}u d\mathrm{{ROC}}(u;\,s)\!\right]^{2}\right| \bar{ p}_{1}\!=\!s,\; Y^{\dag }\!=\!0\!\right)\!. \end{aligned}$$

1.1.4 A.4 Justification for the resampling methods

To justify the resampling method, we first note that $|\beta ^{*} - {\widehat{\beta }}| + |\gamma ^{*}-{\widehat{\gamma }}| + \sup _{t \le t_0}|G_{X,Z}^{*}(t)-{\widehat{G}}_{X,Z}(t)|=O_p(n^{-\frac{1}{2}}).$ It follows from similar arguments given in the Appendix A and Appendix 1 of Cai et al. (2010) that $\mathcal{{W}}^{*} _{S_y}(c;\,s) = \sqrt{nh}\{\mathcal{{S}}_y^{*}(c;\,s)-\widehat{\mathcal{{S}}}_y(c;\,s)\} \simeq n^{\frac{1}{2}}h^{-1/2}\sum _{i=1}^n\widehat{\mathcal{{D}}}_{\mathcal{{S}}_y i}(c;\,s)\xi _i,$ where $\widehat{\mathcal{{D}}}_{\mathcal{{S}}_y i}(c;\,s)$ is obtained by replacing all theoretical quantities in $\mathcal{{D}}_{\mathcal{{S}}_y}(c;\,s)$ given in (10) with the estimated counterparts for the $i$th subject. This, together with similar arguments as given above for the expansion of $\widehat{\mathcal{{W}}}_{\mathrm{{\tiny ROC}}}(u;\,s),$ implies that

$$\begin{aligned} \mathcal{{W}}^{*} _{\mathrm{{\tiny pAUC}}_f}(s)&= \int \limits _0^f \sqrt{nh}\{\mathrm{{ROC}}^*(u;\,s)-\widehat{\mathrm{{ROC}}}(u;\,s)\} du\\&\simeq&n^{-\frac{1}{2}}h^{-1/2}\sum _{i=1}^nK_{h}\left\{ \widehat{\mathcal{E }}_{1}(s)\right\} \widehat{\mathcal{{D}}}_{\mathrm{{\tiny pAUC}}_f}(s)\xi _i, \end{aligned}$$

where $\widehat{\mathcal{{D}}}_{\mathrm{{\tiny pAUC}}_f}(s) = \int _0^f [\widehat{\mathcal{{D}}}_{\mathcal{{S}}_1 i} \{ \widehat{\mathcal{{S}}}_0^{-1}(u;\,s);\,s \}-{\dot{\mathrm{{ROC}}}}(u;\,s)\widehat{\mathcal{{D}}}_{\mathcal{{S}}_0 i} \{ \widehat{\mathcal{{S}}}_0^{-1}(u;\,s);\,s \}] du.$ Conditional on the data, $\mathcal{{W}}^{*}_{\mathrm{{\tiny pAUC}}_f} (s)$ is approximately normally distributed with mean 0 and variance

$$\begin{aligned} {\widehat{\sigma }}_{\mathrm{{\tiny pAUC}}_f}^{2}(s) = h^{-1}\sum _{i=1}^nK_{h}\left\{ \widehat{\mathcal{E }}_{1}(s)\right\} ^2\widehat{\mathcal{{D}}}_{\mathrm{{\tiny pAUC}}_f}(s)^2. \end{aligned}$$

Using the consistency of the proposed estimators along with similar arguments as given above, it is not difficult to show that the above variance converges to $\sigma _{\mathrm{{\tiny pAUC}}_f}^{2}(s)$ as $n\rightarrow \infty .$ Therefore, the empirical distribution obtained from the perturbed sample can be used to approximate the distribution of $\widehat{\mathcal{{W}}}_{\mathrm{{\tiny pAUC}}_f}(s).$

We now show that after proper standardization, the supermum type statistics $\varGamma$ converges weakly. To this end, we first note that, similar arguments as given in the Appendix A can be used to show that $\sup _{s\in \mathcal{{I}}_h}|{\widehat{\sigma }}_{\mathrm{{\tiny pAUC}}_f}^{2}(s)- \sigma _{\mathrm{{\tiny pAUC}}_f}^{2}(s)|= o_{p}(n^{-\delta })$ and

$$\begin{aligned} \varGamma = \sup _{s \in \mathcal{{I}}_h} \left|\frac{\sqrt{nh}\mathbb{P }_n[K_{h}\{\bar{\mathcal{E }}_{1}(s)\} \mathcal{{D}}_{\mathrm{{\tiny pAUC}}_f}(s)]}{\sigma _{\mathrm{{\tiny pAUC}}_f}(s)}\right| + o_{p}(n^{-\delta }), \end{aligned}$$

for some small positive constant $\delta .$ Using similar arguments in Bickel and Rosenblatt (1973), we have

$$\begin{aligned} pr\left\{ a_{n}\left(\varGamma -d_{n}\right)<x\right\} \rightarrow e^{-2e^{-x}}, \end{aligned}$$

where $a_{n} = [2\log \{(\rho _{u}-\rho _{l})/h\}]^{\frac{1}{2}}$ and $d_{n}=a_{n}+a_{n}^{-1}\log \{\int \dot{K}(t)^{2}dt/(4m_{2}\pi )\}.$ Now to justify the resampling procedure for constructing the CI, we note that

$$\begin{aligned} \mathcal{{W}}^*_{\mathrm{{\tiny pAUC}}_f}(s) = n^{-\frac{1}{2}}h^{\frac{1}{2}}\sum _{i=1}^{n} K_{h}\left\{ \widehat{\mathcal{E }}_{1i}(s)\right\} \widehat{\mathcal{{D}}}_{\mathrm{{\tiny pAUC}}_f}(s) \left(\xi _{i}-1\right)+\varepsilon ^*(s) \end{aligned}$$

where $pr\{\sup _{s\in \varOmega (h)}|n^{\delta }\varepsilon ^*(s)|\ge e \mid \text{ data}\} \rightarrow 0$ in probability. Therefore,

$$\begin{aligned} \varGamma ^{*} = \sup _{s \in \mathcal{{I}}_{h}} \left| \frac{n^{-\frac{1}{2}}h^{\frac{1}{2}}\sum _{i = 1}^{n}K_{h}\{\widehat{\mathcal{E }}_{1i}(s)\} \widehat{\mathcal{{D}}}_{\mathrm{{\tiny pAUC}}_f}(s) (\xi _{i}-1) }{\sigma _{\mathrm{{\tiny pAUC}}_f}(s)}\right|+ \left|\varepsilon _{\mathrm{{\tiny sup}}}^*\right|. \end{aligned}$$

where $pr\{|n^{\delta }\varepsilon _{\mathrm{{\tiny sup}}}^*| \ge e | \text{ data}\} \rightarrow 0.$ It follows from similar arguments as given in Tian et al. (2005) and Zhao et al. (2010) that

$$\begin{aligned} \sup \left| pr\left\{ a_{n}\left(\varGamma ^{*}-d_{n}\right)<x|\text{ data}\right\} - e^{-2e^{-x}} \right| \rightarrow 0, \end{aligned}$$

in probability as $n \rightarrow \infty .$ Thus, the conditional distribution of $a_{n}(\varGamma ^{*}-d_{n})$ can be used to approximate the unconditional distribution of $a_{n}(\varGamma -d_{n}).$ When $h_0 = h_1,$ in general, the standardized $\varGamma$ does not converge to the extreme value distribution. However, when $h_0=h_1 = k \in (0,\,\infty ),$ the distribution of the suitable standardized version of $\varGamma$ still can be approximated by that of the corresponding standardized $\varGamma ^*$ conditional on the data (Gilbert et al. 2002).

1.2 Appendix B

1.2.1 B.1 Bandwidth selection for $\mathrm{{pAUC}}_{f}(s)$

The choice of the bandwidths $h_{0}$ and $h_{1}$ is important for making inference about $\mathcal S _{y}(c;\,s)$ and consequently $\text{ pAUC}_{f}(s).$ Here we propose a two-stage K-fold cross-validation procedure to obtain the optimal bandwidth for $\widehat{\mathcal{S }}_{0,h_{0}}^{-1}(u;\,s)$ and $\widehat{\mathcal{S }}_{1,h_{1}}(c;\,s)$ sequentially. Specifically, we randomly split the data into K disjoint subsets of about equal sizes denoted by $\{\mathcal{J }_{k},\, k=1,\ldots ,K\}.$ The two-stage procedure is described as follows:

• (I) Motivated by the fact that $\mathcal{{S}}_{0}^{-1}(u;\,s)$ is essentially the $(1-u)$-th quantile of the conditional distribution of $\bar{ p}_{2}(X,\,Z)$ given $Y^{\dag }=0$ and $\bar{ p}_{1}(X)=s,$ for each k, we use all the observations not in $\mathcal J _{k}$ to estimate $q_{0,1-u}(s)=\mathcal S _{0}^{-1}(u;\,s)$ by obtaining $\{{\widehat{\alpha }}_{0}(s;\,h),\,{\widehat{\alpha }}_{1}(s;\,h)\},$ the minimizer of

  $$\begin{aligned} \displaystyle \sum _{j \in \mathcal J _{l},l\ne k} I\left(Y_{j}=0\right)\widehat{w}_{j}K_{h}\left\{ \widehat{\mathcal{E }}_{1j}(s)\right\} \rho _{1-u}\left[{\widehat{p}}_{2j}-g\left\{ \alpha _{0} + \alpha _{1}\widehat{\mathcal{E }}_{1j}(s)\right\} \right] \end{aligned}$$

  wrt $(\alpha _{0},\,\alpha _{1}),$ where $\rho _{\tau }(e)$ is a check function defined as $\rho _{\tau }(e)=\tau e,$ if $e\ge 0;\,=(\tau -1)e,$ otherwise. Let $\widehat{q}_{0,1-u}^{(-k)}(s;\,h)=g\{{\widehat{\alpha }}_{0}(s;\,h)\}$ denote the resulting estimator of $q_{0,1-u}(s).$ With observations in $\mathcal J _{k},$ we obtain

  $$\begin{aligned} Err^{(q_0)}_{k}(h) = \displaystyle \sum _{i\in \mathcal J _{k}} \left(1-Y_i\right)\widehat{w}_{i}\int \limits _{0}^{f} \rho _{1-u}\left[{\widehat{p}}_{2i} - \widehat{q}_{0,1-u}^{(-k)}\left({\widehat{p}}_{1i};\,h\right)\right]du. \end{aligned}$$

  Then, we let $h_0^{\mathrm{{\tiny opt}}}=\arg \min _{h}{\sum _{k=1}^{K}Err^{(q_0)}_{k}(h)}.$

• (II) Next, to find an optimal $h_1$ for $\widehat{\mathcal{{S}}}_{1,h_1}(\cdot ;\,s),$ we choose an error function that directly relates to $\mathrm{{pAUC}}_f(s)= - \int _{\mathcal{{S}}_0^{-1}(f;\,s)}^{\infty }\mathcal{{S}}_1(c;\,s)d\mathcal{{S}}_0(c;\,s).$ Specifically, noting the fact that

  $$\begin{aligned} E\!\!\left(\,\,\!\left.\int \limits _{\!\mathcal{{S}}^{-1}_{0}(f;\,s)}^{\infty } \left[\!I\left\{ g_{2}\left(\gamma ^{\prime }W_i\!\right)\!\ge \! c\!\right\} \!-\!\mathcal{{S}}_1(c;\,s)\right] d \mathcal S _{0}(c;\,s) \right| Y^{\dag }_{i}\!=\!1,\; g_1\left(\beta ^{\prime }X_i\!\right)\!=s\!\right)\!\!=0, \end{aligned}$$

  we use the corresponding mean integrated squared error for $I\{g_{2}(\gamma ^{\prime }W_i)\ge c\} - \mathcal{{S}}_1(c;\,s)$ as the error function. For each $k,$ we use all the observations which are not in $\mathcal J _{k}$ to obtain the estimate of $\mathcal S _{1}(c;\,s),$ denoted by $\widehat{\mathcal{S }}_{1,h}^{(-k)}(c;\,s)$ via (4). Then, with the observations in $\mathcal J _{k},$ we calculate the prediction error

  $$\begin{aligned} Err^{(\mathcal{{S}}_1)}_{k}(h)&\!=&\displaystyle \!-\!\sum _{i \in \mathcal J _{k},Y_{i}\!=\!1} \widehat{w}_{i} \!\int \limits _{\widehat{\mathcal{{S}}}_{0,h_{0}}^{-1}(f;\,{\widehat{p}}_{1i})}^{\infty }\! \!\left\{ \!I\!\left(\!{\widehat{p}}_{2i}\!\ge \! c \!\right)\!-\! \widehat{\mathcal{S }}_{1,h}^{(-k)}\left(\!c;\,{\widehat{p}}_{1i}\!\right)\!\right\} ^{2}\! d \widehat{\mathcal{S }}_{0,h_{0}}\left(\!c;\,{\widehat{p}}_{1i}\right)\!. \end{aligned}$$

  We let $h_{1}^{\mathrm{{\tiny opt}}} = \arg \min _{h}{\sum _{k=1}^{K}Err_{k}^{(\mathcal{{S}}_1)}(h)}.$

Since the order of $h_y^{\mathrm{{\tiny opt}}}$ is expected to be $n^{-1/5}$ (Fan and Gijbels 1995), the bandwidth we use for estimation is $h_y = h_y^{\mathrm{{\tiny opt}}} \times n^{-d_0}$ with $0 < d_0 < 3/10$ such that $h_y = n^{-\nu }$ with $1/5 < \nu < 1/2.$ This ensures that the resulting functional estimator $\mathcal{{S}}_{y,h_y}(c;\,s)$ with the data-dependent smooth parameter has the above desirable large sample properties.

1.2.2 B.2 Bandwidth selection for $\mathrm{{IDI}}(s)$

Same as bandwidth selection for pAUC, we also propose a K-fold cross validation procedure to choose the optimal bandwidth $h_{1}$ for $\text{ IS}(s)=\int _{0}^{1}\mathcal{{S}}_{1}(c;\,s)dc$ and $h_{0}$ for $\text{ IP}(s)=\int _{0}^{1}\mathcal{{S}}_{0}(c;\,s)dc$ separately. The procedure is described as follows: we randomly split the data into K disjoint subsets of about equal sizes denoted by $\{ { \mathcal J }_{k},\;k=1,\ldots ,K\}.$ Motivated by the fact (3), for each $k$, we use all the observations not in $\mathcal J _{k}$ to estimate $\int _{0}^{1}\mathcal{{S}}_{y}(c,\,s)dc$ by obtaining $\{\widehat{\varphi }^{(y)}_{0}(s;\,h),\,\widehat{\varphi }_{1}^{(y)}(s;\,h)\}$ for $y=0,\,1,$ which is the solution to the estimating equation

$$\begin{aligned} \sum _{j\in \mathcal J _{l},l\ne k} I\left(Y_{j}=y\right){\widehat{\omega }}_{j}K_{h}\left\{ \widehat{\mathcal{E }}_{1j}(s)\right\} \left[{\widehat{p}}_{2j} - g\left\{ \varphi _{0}^{(y)}+\varphi _{1}^{(y)}\widehat{\mathcal{E }}_{1j}(s)\right\} \right]=0, \end{aligned}$$

wrt $(\varphi _{0}^{(y)},\,\varphi _{1}^{(y)}).$ Let $\widehat{\text{ IS}}^{(-k)}(s;\,h)=g\{\widehat{\varphi }_{0}^{(1)}(s;\,h)\}$ and $\widehat{\text{ IP}}^{(-k)}(s;\,h)=g\{\widehat{\varphi }_{0}^{(0)}(s;\,h)\}.$ With observations in $\mathcal J _{k},$ we obtain

$$\begin{aligned} Err_{k}^{(\text{ IS})}(h) = \sum _{i \in \mathcal J _{k}}Y_i {\widehat{\omega }}_{i}\left\{ {\widehat{p}}_{2i} - \widehat{\text{ IS}}^{(-k)}\left({\widehat{p}}_{1i};\,h\right)\right\} ^{2}, \end{aligned}$$

or

$$\begin{aligned} Err_{k}^{(\text{ IP})}(h) = \sum _{i \in \mathcal J _{k}}\left(1-Y_i\right) {\widehat{\omega }}_{i}\left\{ {\widehat{p}}_{2i} - \widehat{\text{ IP}}^{(-k)}\left({\widehat{p}}_{1i};\,h\right)\right\} ^{2}. \end{aligned}$$

Then, we let $h_{1}^{opt}\!=\!\arg \min _{h} \sum _{k=1}^{K}Err_{k}^{(\text{ IS})}(h)$ and $h_{0}^{opt}\!=\!\arg \min _{h} \sum _{k=1}^{K}Err_{k}^{(\text{ IP})}(h).$

1.3 Appendix C

R codes for application will be available from the corresponding author upon request.