Type I Error Rates from Mixed Effects Model Repeated Measures Versus Fixed Effects Anova with Missing Values Imputed Via Last Observation Carried Forward

  • Craig H. Mallinckrodt
  • W. Scott Clark
  • Stacy R. David


Treatment effects are often evaluated by comparing change over time in outcome measures. However, valid analyses of longitudinal data can be problematic when subjects discontinue (dropout) prior to completing the trial. This study compared the Type I error rates from a likelihood-based repeated measures analysis (MMRM) to a fixed-effects analysis of variance where missing values were imputed using the last observation carried forward approach (LOCF). Comparisons were made in 32 scenarios, with 3000 simulated data sets per scenario. The null hypothesis of no difference between treatments in mean change from baseline to endpoint was true in all data sets. Subject dropout was introduced to generate ignorable and nonignorable missingness.

Pooled across all scenarios, the Type I error rates for MMRM and LOCF were 5.85% and 10.36%, respectively. Type I error rates in the 32 scenarios ranged from 5.03% to 7.17% for MMRM, and from 4.43% to 36.30% for LOCF. In 19 of the 32 scenarios, MMRM yielded a Type I error rate that was at least 1.00% closer to the expected rate of 5.00% than the corresponding rate from LOCF.

Greater inflation of Type I error in LOCF resulted from greater bias in estimates of mean change from baseline to endpoint and unduly small standard errors that were a consequence of failing to account for the uncertainty of imputation. The superior control of Type I error by MMRM suggested that MMRM should replace LOCF as the default primary analysis for longitudinal clinical trials where dropout bias may exist.

Key Words

Missing data Mixed-effects models Dropout bias LOCF 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Milliken GA, Johnson DE. “The Analysis of Messy Data.” In Vol. I. Designed Experiments. New York, NY: Chapman & Hall; 1993:326.Google Scholar
  2. 2.
    Gibbons RD, Hedeker D, Elkin I, Waternaux C, Kraemer HC, Greenhouse JB, Shea T, Imber S, Stotsky SM, Watkins JT. Some conceptual and statistical issues in analysis of longitudinal psychiatric data. Arch Gen Psych. 1993;50:739–750.CrossRefGoogle Scholar
  3. 3.
    Laird NM. Missing data in longitudinal studies. Stat Med. 1988;7:305–315.CrossRefGoogle Scholar
  4. 4.
    Little R, Rubin D. Statistical Analysis with Missing Data. New York, NY: John Wiley and Sons: 1987.Google Scholar
  5. 5.
    Lavori PW. Clinical trials in psychiatry: Should protocol deviation censor patient data? Neuropsycho-pharmacol. 1992;6(1):39–48.Google Scholar
  6. 6.
    Verbeke G, Molenberghs G. Linear Mixed Models for Longitudinal Data. New York, NY: Springer; 2000.Google Scholar
  7. 7.
    Littell RC, Milliken GA, Stroup WW, Wolfinger RD. The SAS System for Mixed Models. Cary, NC: SAS Institute Inc; 1996; Chap. 1–10.Google Scholar
  8. 8.
    Cnaan A, Laird NM, Slasor P. Using the general linear mixed model to analyze unbalanced repeated measures and longitudinal data. Stat Med. 1997; 16: 2349–2380.CrossRefGoogle Scholar
  9. 9.
    Lavori PW, Dawson R, Shera D. A multiple imputation strategy for clinical trials with truncation of patient data. Stat Med. 1995;14:1913–1925.CrossRefGoogle Scholar
  10. 10.
    Siddiqui O, Ali MW. A comparison of the random-effects pattern mixture model with last-observation-carried-forward (LOCF) analysis in longitudinal clinical trials with dropouts. J Biopharm Stat. 1998;8(4):545–563.CrossRefGoogle Scholar
  11. 11.
    Heyting A, Tolboom JTBM, Essers JGA. Statistical handling of dropouts in longitudinal clinical trials. Stat Med. 1992;11:2043–2061.CrossRefGoogle Scholar
  12. 12.
    Mallinckrodt CH, Clark WS, David SR. Accounting for dropout bias using mixed-effects models. J Bio-Pharm Stat. Forthcoming.Google Scholar
  13. 13.
    Kay SR, Opler LA, Fiszbein A. Positive and Negative Syndrome Scale (PANSS) Manual. North Tonawanda, NY: Multi-Health Systems; 1986.Google Scholar
  14. 14.
    Tollefson GD, Beasley CM, Jr., Tran P, Street JS, Krueger JA, Tamura RN, Graffeo KA, Thieme ME. Olanzapine versus Haloperidol in the treatment of schizophrenia, schizoaffective, and schizophreniform disorders: Results of an international collaborative trial. Am J Psychiatry. 1997;154(4):457–465.CrossRefGoogle Scholar
  15. 15.
    Van Vleck LD, Gregory KE. Multiple trait restricted maximum likelihood for simulated measures of ovulation rate with underlying multivariate normal distributions. J Anim Sci. 1992;70:57.CrossRefGoogle Scholar
  16. 16.
    Mallinckrodt CH. The Effects of Animal Model Approximations and Data Problems on the Reliability of Genetic Evaluations. Ph.D. Dissertation. Fort Collins, CO: Colorado State University; 1993.Google Scholar
  17. 17.
    Littell RC, Milliken GA, Stroup WW, Wolfinger RD. The SAS System for Mixed Models. Cary, NC: SAS Institute Inc; 1996;499.Google Scholar
  18. 18.
    Littell RC, Milliken GA, Stroup WW, Wolfinger RD. The SAS System for Mixed Models. Cary, NC: SAS Institute Inc; 1996;38.Google Scholar
  19. 19.
    Rubin D, Shenker N. Multiple imputations in health care databases: An overview and some applications. Stat Med. 1991;10:585–598.CrossRefGoogle Scholar
  20. 20.
    Little R, Yau L. Intent-to-treat analysis for longitudinal studies with drop-outs. Biometrics. 1996;52: 1324–1333.CrossRefGoogle Scholar

Copyright information

© Drug Information Association, Inc 2001

Authors and Affiliations

  • Craig H. Mallinckrodt
    • 1
  • W. Scott Clark
    • 1
  • Stacy R. David
    • 1
  1. 1.Lilly Corporate Center, Department MCYXDEli Lilly & Co.IndianapolisUSA

Personalised recommendations