Skip to main content

Group Sequential Methods for Comparing Correlated Receiver Operating Characteristic Curves

  • Conference paper
  • First Online:

Part of the book series: ICSA Book Series in Statistics ((ICSABSS))

Abstract

Receiver operating characteristic (ROC) curves are commonly used to measure the performance of diagnostic tests. The ROC curve can be estimated empirically without assuming the distributions of the underlying diagnostic test data. Comparison of the accuracy of two diagnostic tests using ROC curves from the two tests are often conducted using fixed sample designs. However, to address the ethics and efficiency concerns of clinical trial studies, there is a need to employ a group sequential design (GSD) and periodically monitor and analyze the accruing data. In this chapter, we incorporate group sequential methods into the design of comparative diagnostic study with respect to the ROC curves. First, we study the difference between sequential empirical ROC curves on the process level. Then we derive the asymptotic distribution theory for the difference between sequential empirical ROC curves and derive the asymptotic covariance structure for comparative ROC statistics. Relating the difference between empirical ROC curves to the Kiefer process, we also show these results can be used to conduct a GSD using standard software.

This is a preview of subscription content, log in via an institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  • Jennison C, Turnbull BW (2000) Group sequential methods with applications to clinical trials. Chapman and Hall, New York

    MATH  Google Scholar 

  • Karr AF (1993) Probability. Springer, New York

    Book  MATH  Google Scholar 

  • Kim K, Demets DL (1992) Sample size determination for group sequential clinical trials with immediate response. Stat Med 11(10):1391–1399

    Article  Google Scholar 

  • Koopmeiners JS, Feng Z (2011) Asymptotic properties of the sequential empirical ROC, PPV and NPV curves under case-control sampling. Ann Stat 39(6):3234–3261

    Article  MATH  MathSciNet  Google Scholar 

  • Lardinois D, Weder W, Hany TF, Kamel EM, Korom S, Seifert B, von Schulthess GK, Steinert HC (2003) Staging of non-small-cell lung cancer with integrated positron-emission tomography and computed tomography. N Engl J Med 348(25):2500–2507

    Article  Google Scholar 

  • Liu A, Wu C, Schisterman EF (2008) Nonparametric sequential evaluation of diagnostic biomarkers. Stat Med 27(10):1667–1678

    Article  MathSciNet  Google Scholar 

  • Mazumdar M, Liu A (2003) Group sequential design for comparative diagnostic accuracy studies. Stat Med 22(5):727–739

    Article  Google Scholar 

  • McNeil BJ, Adelstein SJ (1976) Determining the value of diagnostic and screening tests. J Nucl Med 17(6):439–448

    Google Scholar 

  • Pepe MS, Feng Z, Longton G, Koopmeiners J (2009) Conditional estimation of sensitivity and specificity from a phase 2 biomarker study allowing early termination for futility. Stat Med 28(5):762–779

    Article  MathSciNet  Google Scholar 

  • Pocock SJ (1977) Group sequential methods in the design and analysis of clinical trials. Biometrika 64(2):191–199

    Article  Google Scholar 

  • O’Brien PC, Fleming TR (1979) A multiple testing procedure for clinical trials. Biometrics 35(3):549–556

    Google Scholar 

  • Silvestri GA, Tanoue LT, Margolis ML, Barker J, Detterbeck F (2003) The noninvasive staging of non-small cell lung cancerthe guidelines. CHEST J 123(1_suppl):147S –156S

    Article  Google Scholar 

  • Sox HC, Stern S, Owens D, Abrams HL (1989) Assessment of diagnostic technology in health care: rationale, methods, problems, and directions. National Academies Press, Washington, DC

    Google Scholar 

  • Tang L, Liu A (2010) Sample size recalculation in sequential diagnostic trials. Biostatistics 11:151–163

    Article  MathSciNet  Google Scholar 

  • Tang L, Emerson SS, Zhou XH (2008) Nonparametric and semiparametric group sequential methods for comparing accuracy of diagnostic tests. Biometrics 64(4):1137–1145

    Article  MATH  MathSciNet  Google Scholar 

  • vander Vaart AW, Wellner J (1996) Weak convergence and empirical processes: with applications to statistics. Springer, New York

    Book  MATH  Google Scholar 

  • Zhou XH, Obuchowski NA, McClish DK (2011) Statistical methods in diagnostic medicine, vol 712. Wiley, Hoboken

    Book  MATH  Google Scholar 

  • Zweig MH, Campbell G (1993) Receiver-operating characteristic (ROC) plot: a fundamental evaluation tool in clinical medicine. Clin Chem 39:561–577

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Xuan Ye .

Editor information

Editors and Affiliations

Appendix

Appendix

Derivation of the elements a ij in Σ:

$$\begin{aligned} a_{11} & = {\rm Var}(K_{1,1}(\mathit{\rm ROC}_1(t),r_D))+ {\rm Var}\left(\lambda^{1/2}\frac{r_D}{r_{\bar D}}\left(\frac{f_{1,D}(S_{1,\bar D}^{-1}(t))}{f_{1, \bar D}(S_{1,\bar D}^{-1}(t))}\right)K_{1,2}(t,r_{\bar D})\right) & \\ & = r_D(\mathit{\rm ROC}_1(t)-\mathit{\rm ROC}_1^2(t))+\lambda \frac{r_D^2}{r_{\bar D}} \left( \frac{f_{1,D}(S_{1,\bar D}^{-1}(t))}{f_{1,\bar D}(S_{1,\bar D}^{-1}(t))} \right)^2(t-t^2),\end{aligned}$$
$$\begin{aligned} a_{12} & = {\rm Cov}\Big( n_D^{-1/2}[n_Dr_D](\hat S_{1,D,r_D}(\hat S_{1,\bar D,r_{\bar D}}^{-1}(t)) - S_{1,D}(\hat S_{1,\bar D,r_{\bar D}}^{-1}(t))), \\ & n_D^{-1/2}[n_Dr_D](\hat S_{2,D,r_D}(\hat S_{2,\bar D,r_{\bar D}}^{-1}(t)) - S_{2,D}(\hat S_{2,\bar D,r_{\bar D}}^{-1}(t)))\Big) \\ & + {\rm Cov} \left( n_D^{-1/2}[n_Dr_D](S_{1,D}(\hat S_{1,\bar D,r_{\bar D}}^{-1}(t)) - S_{1,D}(S_{1,\bar D}^{-1}(t))),\right.\\ & \left. n_D^{-1/2}[n_Dr_D](S_{2,D}(\hat S_{2,\bar D,r_{\bar D}}^{-1}(t)) - S_{2,D}(S_{2,\bar D}^{-1}(t))) \right) \\ &= {\rm Cov}\bigg( n_D^{-1/2} \sum_{i=1}^{[n_Dr_D]}\left( I(X_{1,D,i}>\hat S_{1,\bar D,r_{\bar D}}^{-1}(t))-S_{1,D}(\hat S_{1,\bar D,r_{\bar D}}^{-1}(t)) \right), \\ & \hspace{1.1cm} n_D^{-1/2} \sum_{i=1}^{[n_Dr_D]}\left( I(X_{2,D,i}>\hat S_{2,\bar D,r_{\bar D}}^{-1}(t))-S_{2,D}(\hat S_{2,\bar D,r_{\bar D}}^{-1}(t)) \right) \bigg) \\ & + {\rm Cov} \left( n_D^{-1/2}[n_Dr_D](S_{1,D}(\hat S_{1,\bar D,r_{\bar D}}^{-1}(t)) - S_{1,D}(S_{1,\bar D}^{-1}(t))),\right.\\ & \left. n_D^{-1/2}[n_Dr_D](S_{2,D}(\hat S_{2,\bar D,r_{\bar D}}^{-1}(t)) - S_{2,D}(S_{2,\bar D}^{-1}(t))) \right) \\ & = r_D(S_D(S_{1,\bar D}^{-1}(t),S_{2,\bar D}^{-1}(t))-\mathit{\rm ROC}_1(t)\mathit{\rm ROC}_2(t)) \\ & \hspace{0.5cm} +\lambda \frac{r_D^2}{r_{\bar D}} \frac{f_{1,D}(S_{1,\bar D}^{-1}(t))}{f_{1,\bar D}(S_{1,\bar D}^{-1}(t))} \frac{f_{2,D}(S_{2,\bar D}^{-1}(t))}{f_{2,\bar D}(S_{2,\bar D}^{-1}(t))} (S_{\bar D}(S_{1,\bar D}^{-1}(t),S_{2,\bar D}^{-1}(t)) -t^2).\end{aligned}$$

The last step is derived by applying the results of sequential empirical process, the compact differentiability of the inverse function and delta method in vander Vaart and Wellner (1996).

$$\begin{aligned} a_{13} &= {\rm Cov}\left(K_{1,1}(\mathit{\rm ROC}_1(t),r_D),K_{1,1}(\mathit{\rm ROC}_1(t),r_D^{\prime}) \right) \\ & + {\rm Cov}\left(\lambda^{1/2}\frac{r_D}{r_{\bar D}}\left(\frac{f_{1,D}(S_{1,\bar D}^{-1}(t))}{f_{1, \bar D}(S_{1,\bar D}^{-1}(t))}\right)K_{1,2}(t,r_{\bar D}),\right.\\ & \left.\lambda^{1/2}\frac{r_D^{\prime}}{r_{\bar D}^{\prime}}\left(\frac{f_{1,D}(S_{1,\bar D}^{-1}(t))}{f_{1, \bar D}(S_{1,\bar D}^{-1}(t))}\right)K_{1,2}(t,r_{\bar D}^{\prime}) \right)\\ & = (r_D\wedge r_D^{\prime})(\mathit{\rm ROC}_1(t)-\mathit{\rm ROC}_1^2(t)) \\ & + (r_{\bar D}\wedge r_{\bar D}^{\prime})\lambda \frac{r_D}{r_{\bar D}} \frac{r_D^{\prime}}{r_{\bar D}^{\prime}} \left( \frac{f_{1,D}(S_{1,\bar D}^{-1}(t))}{f_{1,\bar D}(S_{1,\bar D}^{-1}(t))} \right)^2 (t-t^2).\end{aligned}$$
$$\begin{aligned} a_{14} &= {\rm Cov}\Big( n_D^{-1/2}[n_Dr_D](\hat S_{1,D,r_D}(\hat S_{1,\bar D,r_{\bar D}}^{-1}(t)) - S_{1,D}(\hat S_{1,\bar D,r_{\bar D}}^{-1}(t))), \\ & \hspace{1.2cm} n_D^{-1/2}[n_Dr_D^{\prime}](\hat S_{2,D,r_D^{\prime}}(\hat S_{2,\bar D,r_{\bar D}^{\prime}}^{-1}(t)) - S_{2,D}(\hat S_{2,\bar D,r_{\bar D}^{\prime}}^{-1}(t))) \Big) \\ & + {\rm Cov} \Big( n_D^{-1/2}[n_Dr_D](S_{1,D}(\hat S_{1,\bar D,r_{\bar D}}^{-1}(t)) - S_{1,D}(S_{1,\bar D}^{-1}(t))),\\ & \hspace{1.2cm} n_D^{-1/2}[n_Dr_D^{\prime}](S_{2,D}(\hat S_{2,\bar D,r_{\bar D}^{\prime}}^{-1}(t)) - S_{2,D}(S_{2,\bar D}^{-1}(t))) \Big) \\ & = (r_D \wedge r_D^{\prime}) (S_D(S_{1,\bar D}^{-1}(t),S_{2,\bar D}^{-1}(t))-\mathit{\rm ROC}_1(t)\mathit{\rm ROC}_2(t)) \\ & \hspace{0.5cm} +(r_{\bar D} \wedge r_{\bar D}^{\prime})\lambda \frac{r_D}{r_{\bar D}} \frac{r_D^{\prime}}{r_{\bar D}^{\prime}} \frac{f_{1,D}(S_{1,\bar D}^{-1}(t))}{f_{1,\bar D}(S_{1,\bar D}^{-1}(t))} \frac{f_{2,D}(S_{2,\bar D}^{-1}(t))}{f_{2,\bar D}(S_{2,\bar D}^{-1}(t))} (S_{\bar D}(S_{1,\bar D}^{-1}(t),S_{2,\bar D}^{-1}(t)) -t^2).\end{aligned}$$

The last step is again derived by applying the results of sequential empirical process, the compact differentiability of the inverse function and delta method in vander Vaart and Wellner (1996). Similarly, we can get the following elements of the covariance matrix: \(a_{22}= r_D(\mathit{\rm ROC}_2(t)-\mathit{\rm ROC}_2^2(t))+\lambda \frac{r_D^2}{r_{\bar D}} \left( \frac{f_{2,D}(S_{2,\bar D}^{-1}(t))}{f_{2,\bar D}(S_{2,\bar D}^{-1}(t))} \right)^2(t-t^2),\) \(a_{23}=a_{14},\) \(a_{24}= (r_D\wedge r_D^{\prime})(\mathit{\rm ROC}_2(t)-\mathit{\rm ROC}_2^2(t)) + (r_{\bar D}\wedge r_{\bar D}^{\prime})\lambda \frac{r_D}{r_{\bar D}} \frac{r_D^{\prime}}{r_{\bar D}^{\prime}} \left( \frac{f_{2,D}(S_{2,\bar D}^{-1}(t))}{f_{2,\bar D}(S_{2,\bar D}^{-1}(t))} \right)^2 (t-t^2),\) \(a_{33}=r_D^{\prime}(\mathit{\rm ROC}_1(t)-\mathit{\rm ROC}_1^2(t))+\lambda \frac{r_D^{{\prime} 2}}{r_{\bar D}^{\prime}} \left( \frac{f_{1,D}(S_{1,\bar D}^{-1}(t))}{f_{1,\bar D}(S_{1,\bar D}^{-1}(t))} \right)^2(t-t^2),\)

$$\begin{aligned} a_{34} & = r_D^{\prime}(S_D(S_{1,\bar D}^{-1}(t),S_{2,\bar D}^{-1}(t))-\mathit{\rm ROC}_1(t)\mathit{\rm ROC}_2(t)) & \\ & +\lambda \frac{r_D^{{\prime}2}}{r_{\bar D}^{\prime}} \frac{f_{1,D}(S_{1,\bar D}^{-1}(t))}{f_{1,\bar D}(S_{1,\bar D}^{-1}(t))} \frac{f_{2,D}(S_{2,\bar D}^{-1}(t))}{f_{2,\bar D}(S_{2,\bar D}^{-1}(t))} (S_{\bar D}(S_{1,\bar D}^{-1}(t),S_{2,\bar D}^{-1}(t)) -t^2),\end{aligned}$$

\(a_{44}= r_D^{\prime}(\mathit{\rm ROC}_2(t)-\mathit{\rm ROC}_2^2(t))+\lambda \frac{r_D^{{\prime}2}}{r_{\bar D}^{\prime}} \left( \frac{f_{2,D}(S_{2,\bar D}^{-1}(t))}{f_{2,\bar D}(S_{2,\bar D}^{-1}(t))} \right)^2(t-t^2).\)

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Ye, X., Tang, L. (2015). Group Sequential Methods for Comparing Correlated Receiver Operating Characteristic Curves. In: Chen, Z., Liu, A., Qu, Y., Tang, L., Ting, N., Tsong, Y. (eds) Applied Statistics in Biomedicine and Clinical Trials Design. ICSA Book Series in Statistics. Springer, Cham. https://doi.org/10.1007/978-3-319-12694-4_6

Download citation

Publish with us

Policies and ethics