Skip to main content

Advertisement

SpringerLink
Log in

Measuring Surrogacy in Clinical Research

With an Application to Studying Surrogate Markers for HIV Treatment-as-Prevention

  • Published:
Statistics in Biosciences Aims and scope Submit manuscript

Abstract

In clinical research, validated surrogate markers are highly desirable in study design, monitoring, and analysis, as they do not only reduce the required sample size and follow-up duration, but also facilitate scientific discoveries. However, challenges exist to identify a reliable marker. One particular statistical challenge arises on how to measure and rank the surrogacy of potential markers quantitatively. We review the main statistical methods for evaluating surrogate markers. In addition, we suggest a new measure, the so-called population surrogacy fraction of treatment effect, or simply the \(\rho \)-measure, in the setting of clinical trials. The \(\rho \)-measure carries an appealing population impact interpretation and supplements the existing statistical measures of surrogacy by providing “absolute” information. We apply the new measure along with other prominent measures to the HIV Prevention Trial Network 052 Study, a landmark trial for HIV/AIDS treatment-as-prevention.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Alonso A, Geys H, Molenberghs G, Kenward MG, Vangeneugden T (2003) Validation of surrogate markers in multiple randomized clinical trials with repeated measurements. Biometr J 45(8):931–945

    MathSciNet  MATH  Google Scholar 

  2. Alonso A, Molenberghs G, Burzykowski T, Renard D, Geys H, Shkedy Z, Tibaldi F, Abrahantes JC, Buyse M (2004) Prentice’s approach and the meta-analytic paradigm: a reflection on the role of statistics in the evaluation of surrogate endpoints. Biometrics 60(3):724–728

    MathSciNet  MATH  Google Scholar 

  3. Alonso A, Molenberghs G, Geys H, Buyse M, Vangeneugden T (2006) A unifying approach for surrogate marker validation based on prentice’s criteria. Stat Med 25(2):205–221

    MathSciNet  Google Scholar 

  4. Baker SG (2005) A simple meta-analytic approach for using a binary surrogate endpoint to predict the effect of intervention on true endpoint. Biostatistics 7(1):58–70

    MATH  Google Scholar 

  5. Baker SG, Kramer BS (2003) A perfect correlate does not a surrogate make. BMC Med Res Methodol 3(1):605

    Google Scholar 

  6. Benjamin P, Zeestraten E, Lambert C, Ster IC, Williams OA, Lawrence AJ, Patel B, MacKinnon AD, Barrick TR, Markus HS (2016) Progression of mri markers in cerebral small vessel disease: sample size considerations for clinical trials. J Cereb Blood Flow Metab 36(1):228–240

    Google Scholar 

  7. Beyene J, Moineddin R (2005) Methods for confidence interval estimation of a ratio parameter with application to location quotients. BMC Med Res Methodol 5(1):32

    Google Scholar 

  8. Boissel JP, Collet JP, Moleur P, Haugh M (1992) Surrogate endpoints: a basis for a rational approach. Eur J Clin Pharmacol 43(3):235–244

    Google Scholar 

  9. Buhr KA (2012) Surrogate end points in secondary analyses of cardiovascular trials. Prog Cardiovasc Dis 54(4):343–350

    Google Scholar 

  10. Burzykowski T, Buyse M (2006) Surrogate threshold effect: an alternative measure for meta-analytic surrogate endpoint validation. Pharm Stat 5(3):173–186

    Google Scholar 

  11. Burzykowski T, Molenberghs G, Buyse M, Geys H, Renard D (2001) Validation of surrogate end points in multiple randomized clinical trials with failure time end points. J R Stat Soc 50(4):405–422

    MathSciNet  MATH  Google Scholar 

  12. Burzykowski T, Molenberghs G, Buyse M (2004) The validation of surrogate end points by using data from randomized clinical trials: a case-study in advanced colorectal cancer. J R Stat Soc 167(1):103–124

    MathSciNet  Google Scholar 

  13. Buyse M, Molenberghs G (1998) Criteria for the validation of surrogate endpoints in randomized experiments. Biometrics 54(3):1014

    MATH  Google Scholar 

  14. Buyse M, Molenberghs G, Burzykowski T, Renard D, Geys H (2000) The validation of surrogate endpoints in meta-analyses of randomized experiments. Biostatistics 1(1):49–67

    MATH  Google Scholar 

  15. Bycott PW, Taylor JMG (1998) An evaluation of a measure of the proportion of the treatment effect explained by a surrogate marker. Control Clin Trials 19(6):555–568

    Google Scholar 

  16. Cardiac Arrhythmia Suppression Trial (CAST) Investigators (1989) Preliminary report: effect of encainide and flecainide on mortality in a randomized trial of arrhythmia suppression after myocardial infarction. N Engl J Med 321:406–412

  17. Chakravarty A (2005) Regulatory aspects in using surrogate markers in clinical trials. In: The evaluation of surrogate endpoints, pp 13–51

  18. Chen C, Wang H, Snapinn SM (2003) Proportion of treatment effect (pte) explained by a surrogate marker. Stat Med 22(22):3449–3459

    Google Scholar 

  19. Chen H, Geng Z, Jia J (2007) Criteria for surrogate end points. J R Stat Soc 69(5):919–932

    MathSciNet  Google Scholar 

  20. Chen YQ, Young A, Brown ER, Chasela CS, Fiscus SA, Hoffman IF, Valentine M, Emel L, Taha TE, Goldenberg RL et al (2010) Population attributable fractions for late postnatal mother-to-child transmission of HIV-1 in Sub-Saharan Africa. J Acquir Immune Defic Syndr (1999) 54(3):311

    Google Scholar 

  21. Chen YQ, Masse B, Wang L, San-San O, Li X, Donnell D, McCauley M, Gamble T, Ribauldo HJ, Cohen MS et al (2012) Statistical considerations for the HPTN 052 study to evaluate the effectiveness of early versus delayed antiretroviral strategies to prevent the sexual transmission of HIV-1 in serodiscordant couples. Contemp Clin Trials 33(6):1280–1286

    Google Scholar 

  22. Ciani O, Buyse M, Drummond M, Rasi G, Saad ED, Taylor RS (2017) Time to review the role of surrogate end points in health policy: state of the art and the way forward. Value Health 20(3):487–495

    Google Scholar 

  23. Cohen MS, Chen YQ, McCauley M, Gamble T, Hosseinipour MC, Kumarasamy N, Hakim JG, Kumwenda J, Grinsztejn B, Pilotto JHS et al (2016) Antiretroviral therapy for the prevention of HIV-1 transmission. N Engl J Med 375(9):830–839

    Google Scholar 

  24. Colburn W, DeGruttola VG, DeMets DL, Downing GJ, Hoth DF, Oates JA, Peck CC, Schooley RT, Spilker BA, Woodcock J et al (2001) Biomarkers and surrogate endpoints: preferred definitions and conceptual framework. Clin Pharmacol Ther 69:89–95

    Google Scholar 

  25. Cook TD, DeMets DL (2007) Introduction to statistical methods for clinical trials. CRC Press, Boca Raton

    MATH  Google Scholar 

  26. Daniels MJ, Hughes MD (1997) Meta-analysis for the evaluation of potential surrogate markers. Stat Med 16(17):1965–82

    Google Scholar 

  27. De Gruttola V, Wulfsohn M, FischI MA, Tsiatis A (1993) Modeling the relationship between survival and CD4 lymphocytes in patients with aids and aids-related complex. J Acquir Immune Defic Syndr 6(4):359–365

    Google Scholar 

  28. De Gruttola V, Fleming T, Lin DY, Coombs R (1997) Perspective: validating surrogate markers—are we being naive? J Infect Dis 175(2):237–246

    Google Scholar 

  29. Downing NS, Aminawung JA, Shah ND, Krumholz HM, Ross JS (2014) Clinical trial evidence supporting FDA approval of novel therapeutic agents 2005–2012. Jama 311(4):368–377

    Google Scholar 

  30. Echt DS, Liebson PR, Mitchell LB, Peters RW, Obias-Manno D, Barker AH, Arensberg D, Baker A, Friedman L, Greene HL et al (1991) Mortality and morbidity in patients receiving encainide, flecainide, or placebo: the Cardiac Arrhythmia Suppression trial. N Engl J Med 324(12):781–788

    Google Scholar 

  31. Ellenberg SS, Hamilton JM (1989) Surrogate endpoints in clinical trials: cancer. Stat Med 8(4):405–413

    Google Scholar 

  32. Fieller EC (1940) The biological standardization of insulin. Suppl J R Stat Soc 7(1):1–64

    Google Scholar 

  33. Fleming TR (1994) Surrogate markers in AIDS and cancer trials. Stat Med 13(13–14):1423–1435

    Google Scholar 

  34. Fleming TR, DeMets DL (1996) Surrogate end points in clinical trials: are we being misled? Ann Intern Med 125(7):605–613

    Google Scholar 

  35. Fleming TR, Powers JH (2012) Biomarkers and surrogate endpoints in clinical trials. Stat Med 31(25):2973–2984

    MathSciNet  Google Scholar 

  36. Follmann D (2006) Augmented designs to assess immune response in vaccine trials. Biometrics 62(4):1161–1169

    MathSciNet  MATH  Google Scholar 

  37. Frangakis CE, Rubin DB (2002) Principal stratification in causal inference. Biometrics 58(1):21–29

    MathSciNet  MATH  Google Scholar 

  38. Freedman LS (2001) Confidence intervals and statistical power of the ‘validation’ ratio for surrogate or intermediate endpoints. J Stat Plan Inference 96(1):143–153

    MATH  Google Scholar 

  39. Freedman LS, Graubard BI, Schatzkin A (1992) Statistical validation of intermediate endpoints for chronic diseases. Stat Med 11(2):167–178

    Google Scholar 

  40. Gail MH, Pfeiffer R, Van Houwelingen HC, Carroll RJ (2000) On meta-analytic assessment of surrogate outcomes. Biostatistics 1(3):231–246

    MATH  Google Scholar 

  41. Gallin JI, Malech HL, CURNUTTE WJT, QUIE PG, JAFFE HS, EZKOWITZ RAB (1991) A controlled trial of interferon gamma to prevent infection in chronic granulomatous disease. N Engl J Med 324(8):509–516

    Google Scholar 

  42. Grimes DA, Schulz KF (2005) Surrogate end points in clinical research: hazardous to your health. Obstet Gynecol 105(5, Part 1):1114–1118

    Google Scholar 

  43. Hillis A, Seigel D (1989) Surrogate endpoints in clinical trials: ophthalmologic disorders. Stat Med 8(4):427–430

    Google Scholar 

  44. Huang Y, Gilbert PB, Wolfson J (2013) Design and estimation for evaluating principal surrogate markers in vaccine trials. Biometrics 69(2):301–309

    MathSciNet  MATH  Google Scholar 

  45. Hughes MD (2008) Practical issues arising in an exploratory analysis evaluating progression-free survival as a surrogate endpoint for overall survival in advanced colorectal cancer. Stat Methods Med Res 17(5):487–495

    MathSciNet  Google Scholar 

  46. Hughes MD, DeGruttola V, Welles SL (1995) Evaluating surrogate markers. J Acquir Immune Defic Syndr Hum Retrovirol 10:S1–8

    Google Scholar 

  47. Judea P (2010) An introduction to causal inference. Int J Biostat 6(2):1–62

    MathSciNet  Google Scholar 

  48. Katzenstein DA, Hammer SM, Hughes MD, Gundacker H, Jackson JB, Fiscus S, Rasheed S, Elbeik T, Reichman R, Japour A et al (1996) The relation of virologic and immunologic markers to clinical outcomes after nucleoside therapy in HIV-infected adults with 200 to 500 CD4 cells per cubic millimeter. N Engl J Med 335(15):1091–1098

    Google Scholar 

  49. Kent JT (1983) Information gain and a general measure of correlation. Biometrika 70(1):163–173

    MathSciNet  MATH  Google Scholar 

  50. Kobayashi F, Kuroki M (2014) A new proportion measure of the treatment effect captured by candidate surrogate endpoints. Stat Med 33(19):3338–3353

    MathSciNet  Google Scholar 

  51. Kobayashi F, Kuroki M (2015) Causal measures of the treatment effect captured by candidate surrogate endpoints. J Agric Biol Environ Stat 20(3):409–430

    MathSciNet  MATH  Google Scholar 

  52. Landau WM (1990) Clinical neuromythology IX—pyramid sale in the bucket shop: DATATOP bottoms out. Neurology 40(9):1337–1339

    Google Scholar 

  53. Levin ML (1953) The occurrence of lung cancer in man. Acta-Unio Int Contra Cancrum 9(3):531–541

    MathSciNet  Google Scholar 

  54. Li Z, Meredith MP, Hoseyni MS (2001) A method to assess the proportion of treatment effect explained by a surrogate endpoint. Stat Med 20(21):3175–3188

    Google Scholar 

  55. Li Y, Taylor JMG, Elliott MR (2010) A bayesian approach to surrogacy assessment using principal stratification in clinical trials. Biometrics 66(2):523–531

    MathSciNet  MATH  Google Scholar 

  56. Lin DY, Fischl MA, Schoenfeld DA (1993) Evaluating the role of CD4-lymphocyte counts as surrogate endpoints in human immunodeficiency virus clinical trials. Stat Med 12(9):835–842

    Google Scholar 

  57. Lin DYDY, Fleming TRTR, DeGruttola V, De Gruttola V (1997) Estimating the proportion of treatment effect explained by a surrogate marker. Stat Med 16(13):1515–1527

    Google Scholar 

  58. Lonn E (2001) The use of surrogate endpoints in clinical trials: focus on clinical trials in cardiovascular diseases. Pharmacoepidemiol Drug Saf 10(6):497–508

    Google Scholar 

  59. Mildvan D, Landay A, De Gruttola V, Machado SG, Kagan J (1997) An approach to the validation of markers for use in AIDS clinical trials. Clin Infect Dis 24(5):764–774

    Google Scholar 

  60. Mildvan D, Landay A, De Gruttola V, Machado SG, Kagan J (1997) An approach to the validation of markers for use in AIDS clinical trials. Clin Infect Dis 24(5):764–774

    Google Scholar 

  61. Molenberghs G, Geys H, Buyse M (2001) Evaluation of surrogate endpoints in randomized experiments with mixed discrete and continuous outcomes. Stat Med 20(20):3023–3038

    Google Scholar 

  62. Molenberghs G, Buyse M, Geys H, Renard D, Burzykowski T, Alonso A (2002) Statistical challenges in the evaluation of surrogate endpoints in randomized trials. Control Clin Trials 23(6):607–625

    MATH  Google Scholar 

  63. Murray JS, Elashoff MR, Iacono-Connors LC, Cvetkovich TA, Struble KA (1999) The use of plasma hiv rna as a study endpoint in efficacy trials of antiretroviral drugs. Aids 13(7):797–804

    Google Scholar 

  64. Pearl J (2001) Direct and indirect effects. In: Proceedings of the seventeenth conference on uncertainty in artificial intelligence. Morgan Kaufmann Publishers Inc, Burlington, pp 411–420

  65. Pearl J (2012) The causal mediation formula—a guide to the assessment of pathways and mechanisms. Prev Sci 13(4):426–436

    Google Scholar 

  66. Prentice RL (1989) Surrogate endpoints in clinical trials: definition and operational criteria. Stat Med 8(4):431–40

    MathSciNet  Google Scholar 

  67. Renard D, Geys H, Molenberghs G, Burzykowski T, Buyse M (2002) Validation of surrogate endpoints in multiple randomized clinical trials with discrete outcomes. Biometr J 44(8):921–935

    MathSciNet  MATH  Google Scholar 

  68. Renard D, Geys H, Molenberghs G, Burzykowski T, Buyse M, Vangeneugden T, Bijnens L (2010) Validation of a longitudinally measured surrogate marker for a time-to-event endpoint. J Appl Stat 30(2):235–247

    MathSciNet  MATH  Google Scholar 

  69. Retherford RD, Choe MK (2011) Statistical models for causal analysis. Wiley, New York

    MATH  Google Scholar 

  70. Robins JM, Greenland S (1992) Identifiability and exchangeability for direct and indirect effects. Epidemiology 3(2):143–155

    Google Scholar 

  71. Rubin DB (2004) Direct and indirect causal effects via potential outcomes. Scand J Stat 31(2):161–170

    MathSciNet  MATH  Google Scholar 

  72. Schatzkin A, Freedman LS, Schiffman MH, Dawsey SM (1990) Validation of intermediate end points in cancer research. J Natl Cancer Inst 82(22):1746–1752

    Google Scholar 

  73. Temple RJ (1995) A regulatory authority’s opinion about surrogate endpoints. Clin Meas Drug Eval 1–22

  74. Temple R (1999) Are surrogate markers adequate to assess cardiovascular disease drugs? JAMA 282(8):790–795

    Google Scholar 

  75. Tsiatis AA, Degruttola V, Wulfsohn MS (1995) Modeling the relationship of survival to longitudinal data measured with error. Applications to survival and CD4 counts in patients with AIDS. J Am Stat Assoc 90(429):27–37

    MATH  Google Scholar 

  76. VanderWeele TJ (2013) Surrogate measures and consistent surrogates. Biometrics 69(3):561–565

    MathSciNet  MATH  Google Scholar 

  77. VanderWeele TJ (2016) Mediation analysis: a practitioner’s guide. Annu Rev Public Health 37:17–32

    Google Scholar 

  78. Wang Y, Taylor JMG (2002) A measure of the proportion of treatment effect explained by a surrogate marker. Biometrics 58(4):803–12

    MathSciNet  MATH  Google Scholar 

  79. Wittes J, Lakatos E, Probstfield J (1989) Surrogate endpoints in clinical trials: cardiovascular diseases. Stat Med 8(4):415–425

    Google Scholar 

  80. Writing Group for the Women’s Health Initiative Investigators et al (2002) Risks and benefits of estrogen plus progestin in healthy postmenopausal women: principal results from the women’s health initiative randomized controlled trial. Jama, 288(3):321–333

  81. Yongming Q, Case M (2006) Quantifying the indirect treatment effect via surrogate markers. Stat Med 25(2):223–231

    MathSciNet  Google Scholar 

  82. Yongming Q, Case M (2007) Quantifying the effect of the surrogate marker by information gain. Biometrics 63(3):958–960

    MathSciNet  MATH  Google Scholar 

  83. Zografos K (1998) On a measure of dependence based on fisher’s information matrix. Commun Stat-TheoryMethods 27(7):1715–1728

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

We thank the participants and investigators of the HPTN 052 study for data sharing. Research reported in this publication was supported by the NIH (R01 MH105857 and R01 AI121259).

Author information

Authors and Affiliations

  1. University of Washington, Seattle, WA, 98195, USA

    Rui Zhuang

  2. Vaccine and Infectious Disease Division and Public Health Sciences Division, Fred Hutchinson Cancer Research Center, Seattle, Washington, 98109, USA

    Ying Qing Chen

Authors
  1. Rui Zhuang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ying Qing Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rui Zhuang.

Appendix: Connection of the \(\rho \)-Measure with the Proportion of Treatment Effect Explained

Appendix: Connection of the \(\rho \)-Measure with the Proportion of Treatment Effect Explained

In the following, we show that both the PTE \(\pi \)-measure and the F-measure can be expressed by the \(\rho \)-measure. The \(\pi \)-measure in terms of relative risk in (6) can be written as

$$\begin{aligned} \pi = \frac{P(T=1 \mid Z=1)/P(T=1 \mid Z=0)-RR_{s}}{P(T=1 \mid Z=1)/P(T=1 \mid Z=0)}. \end{aligned}$$

Multiplying \(P(T=1 \mid Z=0)\) in both the numerator and denominator gives

$$\begin{aligned} \pi = \frac{P(T=1 \mid Z=1)-P(T=1 \mid Z=0) RR_{s}}{P(T=1 \mid Z=1)}. \end{aligned}$$
(7)

Under model (4) and (5), \(RR_{s} = P(T=1\mid Z=1, S=s)/P(T=1 \mid Z=0, S=s)\) and is independent of s. We can write

$$\begin{aligned} P(T=1 \mid Z=0) RR_{s}= & {} \int _{\varOmega _S} P(T=1 \mid Z=0, S=s) dP(s\mid Z=0) \cdot RR_{s} \nonumber \\= & {} \int _{\varOmega _S} P(T=1 \mid Z=0, S=s) \nonumber \\&\cdot \frac{P(T=1\mid Z=1, S=s)}{P(T=1 \mid Z=0, S=s)}dP(s\mid Z=0) \nonumber \\= & {} \int _{\varOmega _S} P(T=1 \mid Z=1, S=s)dP(s\mid Z=0). \end{aligned}$$
(8)

Plugging (8) into (7) reveals

$$\begin{aligned} \pi =\frac{P(T=1 \mid Z=1)-\int _{\varOmega _S} P(T=1 \mid Z=1, S=s)dP(s \mid Z=0)}{P(T=1 \mid Z=1)} = \rho . \end{aligned}$$

To show \(\rho \)-measure in terms of the F-measure,

$$\begin{aligned} \rho&= \frac{P(T=1 \mid Z=1)-\int _{\varOmega _S} P(T=1 \mid Z=1, S=s)dP(s\mid Z=0)}{P(T=1 \mid Z=1)-P(T=1 \mid Z=0)}\\&\quad \times \frac{P(T=1 \mid Z=1)-P(T=1 \mid Z=0)}{P(T=1 \mid Z=1)}= F \times \frac{RR-1}{RR}. \end{aligned}$$

Thus \(\rho =F\times (RR-1)/RR\).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhuang, R., Chen, Y.Q. Measuring Surrogacy in Clinical Research. Stat Biosci 12, 295–323 (2020). https://doi.org/10.1007/s12561-019-09244-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12561-019-09244-4

Keywords

Search

Navigation