Time-Dependent AUC with Right-Censored Data: A Survey

  • Paul BlancheEmail author
  • Aurélien Latouche
  • Vivian Viallon
Conference paper
Part of the Lecture Notes in Statistics book series (LNS, volume 215)


The ROC curve and the corresponding AUC are popular tools for the evaluation of diagnostic tests. They have been recently extended to assess prognostic markers and predictive models. However, due to the many particularities of time-to-event outcomes, various definitions and estimators have been proposed in the literature. This review article aims at presenting the ones that accommodate to right-censoring, which is common when evaluating such prognostic markers.


False Positive Rate Empirical Distribution Function Primary Estimate Spline Basis Function Conditional Survival Function 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Akritas, M.: Nearest neighbor estimation of a bivariate distribution under random censoring. Ann. Stat. 22, 1299–1327 (1994)CrossRefMathSciNetzbMATHGoogle Scholar
  2. 2.
    Blanche, P., Dartigues, J.F., Jacqmin-Gadda, H.: Review and comparison of ROC curve estimators for a time-dependent outcome with marker-dependent censoring. Biometrical Journal 55(5), 687–704. doi: 10.1002/bimj.201200045 (2013)Google Scholar
  3. 3.
    Cai, T., Pepe, M.S., Zheng, Y., Lumley, T., Jenny, N.S.: The sensitivity and specificity of markers for event times. Biostatistics 7, 182–197 (2006)CrossRefzbMATHGoogle Scholar
  4. 4.
    Cai, T., Gerds, T.A., Zheng, Y., Chen, J.: Robust prediction of t-year survival with data from multiple studies. Biometrics 67, 436–444 (2011)CrossRefMathSciNetzbMATHGoogle Scholar
  5. 5.
    Chambless, L.E., Diao, G.: Estimation of time-dependent area under the ROC curve for long-term risk prediction. Stat. Med. 25, 3474–3486 (2006)CrossRefMathSciNetGoogle Scholar
  6. 6.
    Chambless, L.E., Cummiskey, C.P., Cui, G.: Several methods to assess improvement in risk prediction models: extension to survival analysis. Stat. Med. 30(1), 22–38 (2011)CrossRefMathSciNetGoogle Scholar
  7. 7.
    Chiang, C.T., Hung, H.: Nonparametric estimation for time-dependent AUC. J. Stat. Plan. Inference 140(5), 1162–1174 (2010)CrossRefMathSciNetzbMATHGoogle Scholar
  8. 8.
    Cortese, G., Andersen, P.K.: Competing risks and time-dependent covariates. Biom. J. 52(1), 138–158 (2010)MathSciNetzbMATHGoogle Scholar
  9. 9.
    DeLong, E.R., DeLong, D.M., Clarke-Pearson, D.L.: Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44(3), 837–845 (1988)CrossRefzbMATHGoogle Scholar
  10. 10.
    Demler, O.V., Pencina, M.J., D’Agostino, R.B. Sr.: Equivalence of improvement in area under ROC curve and linear discriminant analysis coefficient under assumption of normality. Stat. Med. 30(12), 1410–1418 (2011)MathSciNetGoogle Scholar
  11. 11.
    Dodd, L.E., Pepe, M.S.: Partial AUC estimation and regression. Biometrics 59(3), 614–623 (2003)CrossRefMathSciNetzbMATHGoogle Scholar
  12. 12.
    Etzioni, R., Pepe, M., Longton, G., Hu, C., Goodman, G.: Incorporating the time dimension in receiver operating characteristic curves: a case study of prostate cancer. Med. Decis. Mak. 19(3), 242–251 (1999)CrossRefGoogle Scholar
  13. 13.
    Foucher, Y., Giral, M., Soulillou, J.P., Daures, J.P.: Time-dependent ROC analysis for a three-class prognostic with application to kidney transplantation. Stat. Med. 29, 3079–3087 (2010)CrossRefMathSciNetGoogle Scholar
  14. 14.
    Gail, M.H., Pfeiffer, R.M.: On criteria for evaluating models of absolute risk. Biostatistics 6(2), 227–239 (2005)CrossRefzbMATHGoogle Scholar
  15. 15.
    Gu, W., Pepe, M.S.: Measures to summarize and compare the predictive capacity of markers. Int. J. Biostat. 5, 27–49 (2009)CrossRefMathSciNetGoogle Scholar
  16. 16.
    Heagerty, P.J., Pepe, M.S.: Semiparametric estimation of regression quantiles with application to standardizing weight for height and age in US children. J. R. Stat. Soc. C (Appl. Stat.) 48(4), 533–551 (1999)Google Scholar
  17. 17.
    Heagerty, P.J., Zheng, Y.: Survival model predictive accuracy and ROC curves. Biometrics 61, 92–105 (2005)CrossRefMathSciNetzbMATHGoogle Scholar
  18. 18.
    Heagerty, P.J., Lumley, T., Pepe, M.S.: Time-dependent ROC curves for censored survival data and a diagnostic marker. Biometrics 56, 337–344 (2000)CrossRefzbMATHGoogle Scholar
  19. 19.
    Hung, H., Chiang, C.T.: Estimation methods for time-dependent AUC models with survival data. Can. J. Stat. 38(1), 8–26 (2010)MathSciNetzbMATHGoogle Scholar
  20. 20.
    Hung, H., Chiang, C.T.: Nonparametric methodology for the time-dependent partial area under the ROC curve. J. Stat. Plan. Inference 141(12), 3829–3838 (2011)CrossRefMathSciNetzbMATHGoogle Scholar
  21. 21.
    Lee, M., Cronin, K.A., Gail, M.H., Feuer, E.J.: Predicting the absolute risk of dying from colorectal cancer and from other causes using population-based cancer registry data. Stat. Med. 31(5), 489–500 (2012)CrossRefMathSciNetGoogle Scholar
  22. 22.
    Li, J., Ma, S.: Time-dependent ROC analysis under diverse censoring patterns. Stat. Med. 30, 1266–1277 (2011)CrossRefMathSciNetGoogle Scholar
  23. 23.
    Pencina, M.J., D’Agostino, R.B. Sr., D’Agostino, R.B. Jr., Vasan, R.S.: Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat. Med. 27(2), 157–172 (2008)CrossRefMathSciNetGoogle Scholar
  24. 24.
    Pepe, M.S.: The Statistical Evaluation of Medical Tests for Classification and Prediction. Oxford University Press, Oxford (2004)zbMATHGoogle Scholar
  25. 25.
    Pepe, M.S., Zheng, Y., Jin, Y., Huang, Y., Parikh, C.R., Levy, W.C.: Evaluating the ROC performance of markers for future events. Lifetime Data Anal. 14, 86–113 (2008)CrossRefMathSciNetzbMATHGoogle Scholar
  26. 26.
    Pepe, M.S., Kerr, K.F., Longton, G., Wang, Z.: Testing for improvement in prediction model performance. Preprint available at (2011)
  27. 27.
    Rizopoulos, D.: Dynamic predictions and prospective accuracy in joint models for longitudinal and time-to-event data. Biometrics 67, 819–829 (2011)CrossRefMathSciNetzbMATHGoogle Scholar
  28. 28.
    Saha, P., Heagerty, P.J.: Time-dependent predictive accuracy in the presence of competing risks. Biometrics 66, 999–1011 (2010)CrossRefMathSciNetzbMATHGoogle Scholar
  29. 29.
    Saha-Chaudhuri, P., Heagerty, P.J.: Non-parametric estimation of a time-dependent predictive accuracy curve. Biostatistics 14, 42–59 (2013)CrossRefGoogle Scholar
  30. 30.
    Satten, G.A., Datta, S.: The Kaplan-Meier estimator as an inverse-probability-of-censoring weighted average. Am. Stat. 55(3), 207–210 (2001)CrossRefMathSciNetzbMATHGoogle Scholar
  31. 31.
    Slate, E.H., Turnbull, B.W.: Statistical models for longitudinal biomarkers of disease onset. Stat. Med. 19(4), 617–637 (2000)CrossRefGoogle Scholar
  32. 32.
    Song, X., Zhou, X.H.: A semi-parametric approach for the covariate specific ROC curve with survival outcome. Stat. Sin. 18, 947–965 (2008)MathSciNetzbMATHGoogle Scholar
  33. 33.
    Song, X., Zhou, X.H., Ma, S.: Nonparametric receiver operating characteristic-based evaluation for survival outcomes. Stat. Med. 31(23), 2660–2675 (2012)CrossRefMathSciNetGoogle Scholar
  34. 34.
    Subtil, F., Pouteil-Noble, C., Toussaint, S., Villar, E., Rabilloud, M.: A simple modeling-free method provides accurate estimates of sensitivity and specificity of longitudinal disease biomarkers. Methods Inf. Med. 48, 299–305 (2009)CrossRefGoogle Scholar
  35. 35.
    Tsiatis, A.A., Davidian, M.: Joint modeling of longitudinal and time-to-event data: an overview. Stat. Sin. 14(3), 809–834 (2004)MathSciNetzbMATHGoogle Scholar
  36. 36.
    Uno, H., Cai, T., Tian, L., Wei, L.J.: Evaluating prediction rules for t-year survivors with censored regression models. J. Am. Stat. Assoc. 102(478), 527–537 (2007)CrossRefMathSciNetzbMATHGoogle Scholar
  37. 37.
    Viallon, V., Latouche, A.: Discrimination measures for survival outcomes: connection between the AUC and the predictiveness curve. Biom. J. 53, 217–236 (2011)CrossRefMathSciNetzbMATHGoogle Scholar
  38. 38.
    Xu, R., O’Quigley, J.: Proportional hazards estimate of the conditional survival function. J. R. Stat. Soc. B (Stat. Methodol.) 62(4), 667–680 (2000)Google Scholar
  39. 39.
    Zheng, Y., Heagerty, P.J.: Semiparametric estimation of time-dependent ROC curves for longitudinal marker data. Biostatistics 5(4), 615–632 (2004)CrossRefzbMATHGoogle Scholar
  40. 40.
    Zheng, Y., Heagerty, P.J.: Partly conditional survival models for longitudinal data. Biometrics 61(2), 379–391 (2005)CrossRefMathSciNetzbMATHGoogle Scholar
  41. 41.
    Zheng, Y., Heagerty, P.J.: Prospective accuracy for longitudinal markers. Biometrics 63, 332–341 (2007)CrossRefMathSciNetzbMATHGoogle Scholar
  42. 42.
    Zheng, Y., Cai, T., Jin, Y., Feng, Z.: Evaluating prognostic accuracy of biomarkers under competing risk. Biometrics 68(2), 388–396 (2012)CrossRefMathSciNetzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Paul Blanche
    • 1
    Email author
  • Aurélien Latouche
    • 2
  • Vivian Viallon
    • 3
  1. 1.University of Bordeaux, ISPED and INSERM U897BordeauxFrance
  2. 2.Conservatoire national des arts et métiersParisFrance
  3. 3.UMRESTTE (Univ. Lyon 1 and IFSTTAR)BronFrance

Personalised recommendations