Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Multiplicity Adjustment and Sample Size Calculation in Clinical Trials with Multiple Endpoints: An Industry Survey of Current Practices in Japan



Two issues in clinical trials with multiple endpoints were surveyed: (1) the terminology of multiple endpoints, the relationship between rare events and endpoints, and the differences in multiplicity adjustment between regions, and (2) the current practice on multiplicity adjustment and sample size calculation. This article summarizes the results of the survey on the second issue.


Eligible trials for this survey fulfilled the following conditions: (1) confirmatory phase 3 trial; (2) use of multiple primary endpoints, co-primary endpoints, key secondary endpoint(s) or composite endpoint(s); (3) inclusion of Japanese participants; and (4) protocols created in 2010 or later. The survey was conducted at member companies of the Japan Pharmaceutical Manufacturers Association from October 2017 to November 2017.


Useable responses were obtained from 78 trials in 13 companies based in Japan and 9 companies based in other countries. The Bonferroni procedure was mostly used in clinical trials with multiple primary endpoints, while multiple testing procedures that consider a hierarchy of endpoints or a structure of hypotheses were used in clinical trials with key secondary endpoint(s). In sample size calculation, we can consider the probability of study success, such as the probability of statistical significance in at least one comparison of primary endpoints; however, other probabilities were also considered. This survey reveals that multiplicity adjustment and the correlation of endpoints were not always considered in sample size calculation.


In clinical trials with multiple endpoints, clinical importance was considered when determining multiple testing procedures. Challenges remain with the definition of power, the consideration of multiple testing procedures and the correlation between endpoints in sample size calculation.

This is a preview of subscription content, log in to check access.

Figure 1.


  1. 1.

    Maher-Edwards G, Watson C, Ascher J, et al. Two randomized controlled trials of SB742457 in mild-to-moderate Alzheimer’s disease. Alzheimer’s Dementia. 2015;1:23–36.

  2. 2.

    Committee for Medicinal Products for Human Use. Guideline on the clinical investigation of medicines for the treatment of Alzheimer’s disease. Accessed 19 Feb 2019.

  3. 3.

    U.S. Food and Drug Administration. Multiple endpoints in clinical trials: draft guidance for industry. Accessed 19 Feb 2019.

  4. 4.

    Hodi FS, Chiarion-Sileni V, Gonzalez R, et al. Nivolumab plus ipilimumab or nivolumab alone versus ipilimumab alone in advanced melanoma (CheckMate 067): 4-year outcomes of a multicentre, randomised, phase 3 trial. Lancet Oncol. 2018;19:1480–92.

  5. 5.

    Nakagawa K, Yang JH, Park K, et al. 481TiP Checkmate 722: a phase 3 trial of nivolumab with chemotherapy or ipilimumab vs chemotherapy in epidermal growth factor receptor (EGFR)-mutation, T790M-negative stage IV or recurrent non-small cell lung cancer (NSCLC) after EGFR tyrosine kinase inhibitor (TKI) therapy. Ann Oncol. 2016.

  6. 6.

    Sharma P, Escudier B, McDermott DF, et al. 3LBA CheckMate 025: a randomized, open-label, phase III study of nivolumab (NIVO) versus everolimus (EVE) in advanced renal cell carcinoma (RCC). Eur J Cancer 2015;51:S708.

  7. 7.

    O’Donoghue ML, Braunwald E, White HD, et al. Effect of darapladib on major coronary events after an acute coronary syndrome: the SOLID-TIMI 52 randomized clinical trial. JAMA. 2014;312:1006–15.

  8. 8.

    Senn S, Bretz F. Power and sample size when multiple endpoints are considered. Pharm Stat. 2007;6:161–70.

  9. 9.

    International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH harmonised tripartite guideline: Statistical principles for clinical trials-E9. Accessed 19 Feb 2019.

  10. 10.

    Committee for Human Medicinal Products. Guideline on multiplicity issues in clinical trials. Accessed 19 Feb 2019.

  11. 11.

    Snapinn S. Some remaining challenges regarding multiple endpoints in clinical trials. Stat Med. 2017;36:4441–5.

  12. 12.

    Vickerstaff V, Ambler G, King M, et al. Are multiple primary outcomes analysed appropriately in randomised controlled trials? A review. Contemp Clin Trials. 2015;45:8–12.

  13. 13.

    Moye LA. P-value interpretation and alpha allocation in clinical trials. Ann Epidemiol. 1998;8:351–7.

  14. 14.

    Westfall PH, Krishen A. Optimally weighted, fixed sequence and gatekeeper multiple testing procedures. J Stat Plan Inference. 2001;99:25–41.

  15. 15.

    Wiens BL. A fixed sequence Bonferroni procedure for testing multiple endpoints. Pharm Stat. 2003;2:211–5.

  16. 16.

    Dmitrienko A, Tamhane AC, Bretz F. Multiple Testing Problems in Pharmaceutical Statistics. Boca Raton: CRC Press; 2009.

  17. 17.

    Bretz F, Maurer W, Brannath W, et al. A graphical approach to sequentially rejective multiple test procedures. Stat Med. 2009;28:586–604.

  18. 18.

    O’Brien PC. Procedures for comparing samples with multiple endpoints. Biometrics. 1984;40:1079–87.

  19. 19.

    Chuang-Stein C, Li J. Changes are still needed on multiple co-primary endpoints. Stat Med. 2017;36:4427–36.

  20. 20.

    Odekerken VJ, van Laar T, Staal MJ, et al. Subthalamic nucleus versus globus pallidus bilateral deep brain stimulation for advanced Parkinson’s disease (NSTAPS study): a randomised controlled trial. Lancet Neurol. 2013;12:37–44.

  21. 21.

    Sakamaki K, Yoshida S, Morita Y, et al. Challenges on multiple endpoints in clinical trials: an industry survey in Japan. Ther Innov Regul Sci. (in press).

  22. 22.

    Dmitrienko A, Paux G, Brechenmacher T. Power calculations in clinical trials with complex clinical objectives. J Jpn Soc Comput Stat. 2015;28:15–50.

  23. 23.

    U.S. Food and Drug Administration. Guidance for industry: clinical trial endpoints for the approval of cancer drugs and biologics. Accessed 19 Feb 2019.

  24. 24.

    Thaçi D, Humeniuk J, Frambach Y, et al. Secukinumab in psoriasis: randomized, controlled phase 3 trial results assessing the potential to improve treatment response in partial responders (STATURE). Br J Dermatol. 2015;173:777–87.

  25. 25.

    Ye Y, Li A, Liu L, et al. A group sequential Holm procedure with multiple primary endpoints. Stat Med. 2013;32:1112–24.

  26. 26.

    Sozu T, Sugimoto T, Hamasaki T, et al. Sample Size Determination in Clinical Trials with Multiple Endpoints. New York: Springer; 2015.

Download references


The authors would like to thank the JPMA members who responded to this survey.


The first author’s work was supported in part by Grants-in-Aid for Early-Career Scientists (19K19383) from the Japan Society for the Promotion of Science (JSPS).

Author information

Correspondence to Kentaro Sakamaki.

Ethics declarations

Conflict of interest

The authors declare that there are no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Sakamaki, K., Morita, Y., Iba, K. et al. Multiplicity Adjustment and Sample Size Calculation in Clinical Trials with Multiple Endpoints: An Industry Survey of Current Practices in Japan. Ther Innov Regul Sci (2020).

Download citation


  • Multiple endpoints
  • Multiplicity adjustment
  • Sample size calculation
  • Survey of current practices