Type I Error Rates from Mixed Effects Model Repeated Measures Versus Fixed Effects Anova with Missing Values Imputed Via Last Observation Carried Forward
Treatment effects are often evaluated by comparing change over time in outcome measures. However, valid analyses of longitudinal data can be problematic when subjects discontinue (dropout) prior to completing the trial. This study compared the Type I error rates from a likelihood-based repeated measures analysis (MMRM) to a fixed-effects analysis of variance where missing values were imputed using the last observation carried forward approach (LOCF). Comparisons were made in 32 scenarios, with 3000 simulated data sets per scenario. The null hypothesis of no difference between treatments in mean change from baseline to endpoint was true in all data sets. Subject dropout was introduced to generate ignorable and nonignorable missingness.
Pooled across all scenarios, the Type I error rates for MMRM and LOCF were 5.85% and 10.36%, respectively. Type I error rates in the 32 scenarios ranged from 5.03% to 7.17% for MMRM, and from 4.43% to 36.30% for LOCF. In 19 of the 32 scenarios, MMRM yielded a Type I error rate that was at least 1.00% closer to the expected rate of 5.00% than the corresponding rate from LOCF.
Greater inflation of Type I error in LOCF resulted from greater bias in estimates of mean change from baseline to endpoint and unduly small standard errors that were a consequence of failing to account for the uncertainty of imputation. The superior control of Type I error by MMRM suggested that MMRM should replace LOCF as the default primary analysis for longitudinal clinical trials where dropout bias may exist.
Key WordsMissing data Mixed-effects models Dropout bias LOCF
Unable to display preview. Download preview PDF.
- 1.Milliken GA, Johnson DE. “The Analysis of Messy Data.” In Vol. I. Designed Experiments. New York, NY: Chapman & Hall; 1993:326.Google Scholar
- 4.Little R, Rubin D. Statistical Analysis with Missing Data. New York, NY: John Wiley and Sons: 1987.Google Scholar
- 5.Lavori PW. Clinical trials in psychiatry: Should protocol deviation censor patient data? Neuropsycho-pharmacol. 1992;6(1):39–48.Google Scholar
- 6.Verbeke G, Molenberghs G. Linear Mixed Models for Longitudinal Data. New York, NY: Springer; 2000.Google Scholar
- 7.Littell RC, Milliken GA, Stroup WW, Wolfinger RD. The SAS System for Mixed Models. Cary, NC: SAS Institute Inc; 1996; Chap. 1–10.Google Scholar
- 12.Mallinckrodt CH, Clark WS, David SR. Accounting for dropout bias using mixed-effects models. J Bio-Pharm Stat. Forthcoming.Google Scholar
- 13.Kay SR, Opler LA, Fiszbein A. Positive and Negative Syndrome Scale (PANSS) Manual. North Tonawanda, NY: Multi-Health Systems; 1986.Google Scholar
- 14.Tollefson GD, Beasley CM, Jr., Tran P, Street JS, Krueger JA, Tamura RN, Graffeo KA, Thieme ME. Olanzapine versus Haloperidol in the treatment of schizophrenia, schizoaffective, and schizophreniform disorders: Results of an international collaborative trial. Am J Psychiatry. 1997;154(4):457–465.CrossRefGoogle Scholar
- 16.Mallinckrodt CH. The Effects of Animal Model Approximations and Data Problems on the Reliability of Genetic Evaluations. Ph.D. Dissertation. Fort Collins, CO: Colorado State University; 1993.Google Scholar
- 17.Littell RC, Milliken GA, Stroup WW, Wolfinger RD. The SAS System for Mixed Models. Cary, NC: SAS Institute Inc; 1996;499.Google Scholar
- 18.Littell RC, Milliken GA, Stroup WW, Wolfinger RD. The SAS System for Mixed Models. Cary, NC: SAS Institute Inc; 1996;38.Google Scholar