Skip to main content
Log in

Type I Error Rates and Parameter Bias in Multivariate Behavioral Genetic Models

Behavior Genetics Aims and scope Submit manuscript

Cite this article


For many multivariate twin models, the numerical Type I error rates are lower than theoretically expected rates using a likelihood ratio test (LRT), which implies that the significance threshold for statistical hypothesis tests is more conservative than most twin researchers realize. This makes the numerical Type II error rates higher than theoretically expected. Furthermore, the discrepancy between the observed and expected error rates increases as more variables are included in the analysis and can have profound implications for hypothesis testing and statistical inference. In two simulation studies, we examine the Type I error rates for the Cholesky decomposition and Correlated Factors models. Both show markedly lower than nominal Type I error rates under the null hypothesis, a discrepancy that increases with the number of variables in the model. In addition, we observe slightly biased parameter estimates for the Cholesky decomposition and Correlated Factors models. By contrast, if the variance–covariance matrices for variance components are estimated directly (without constraints), the numerical Type I error rates are consistent with theoretical expectations and there is no bias in the parameter estimates regardless of the number of variables analyzed. We call this the direct symmetric approach. It appears that each model-implied boundary, whether explicit or implicit, increases the discrepancy between the numerical and theoretical Type I error rates by truncating the sampling distributions of the variance components and inducing bias in the parameters. The direct symmetric approach has several advantages over other multivariate twin models as it corrects the Type I error rate and parameter bias issues, is easy to implement in current software, and has fewer optimization problems. Implications for past and future research, and potential limitations associated with direct estimation of genetic and environmental covariance matrices are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5


  1. We repeated the second simulation study to examine whether the observed results were specific to the likelihood ratio test or whether they could generalize to other hypotheses testing techniques such as the Wald Test. The Wald test results were effectively equivalent with the LRT results above, making their presentation here unnecessary.


  • Boker SM, Neale MC, Maes HH, Wilde MJ, Spiegel M, Brick TR, Estabrook R, Bates TC, Mehta P, von Oertzen T, Gore RJ, Hunter MD, Hackett DC, Karch J, Brandmaier A, Pritikin JM, Zahery M, Kirkpatrick RM, Wang Y, Driver C, Johnson SG, Kraft D, Wilhelm S, Manjunath BG (2017) OpenMx 2.7.17-23 User Guide.

  • Bulik-Sullivan BK (2015) Relationship between LD score and haseman-Elston, bioRxiv.

    Article  Google Scholar 

  • Carey G (2005) Cholesky problems. Behav Genet 35(5):653–665

    Article  PubMed  Google Scholar 

  • Coventry WL, Keller MC (2005) Estimating the extent of parameter bias in the classical twin design: a comparison of parameter estimates from extended twin-family and classical twin designs. Twin Res Hum Genet 8(3):214–223

    Article  PubMed  Google Scholar 

  • Dominicus A, Skrondal A, Gjessing HK, Pedersen NL, Palmgren J (2006) Likelihood ratio tests in behavioral genetics: problems and solutions. Behav Genet 36(2):331–340

    Article  PubMed  Google Scholar 

  • Falconer DS (1960) Introduction to quantitative genetics. Oliver and Boyd, London

    Google Scholar 

  • Haldane JBS (1996) The negative heritability of neonatal jaundice. Ann Hum Genet 60:3–5

    Article  PubMed  Google Scholar 

  • Martin NG, Eaves LJ (1977) The genetical analysis of covariance structure. Heredity 38(1):79–95

    Article  PubMed  Google Scholar 

  • Neale MC, Cardon LR (1992) Methodology for genetic studies of twins and families. Kluwer Academic Publishers BV, Dordrecht

    Book  Google Scholar 

  • Neale MC, Fulker DW (1984) Heritability of item responses on the eysenck personality questionnaire. Personal Individ Differ 7:771–779

    Article  Google Scholar 

  • Neale MC, Røysamb E, Jacobson K (2006) Multivariate genetic analysis of sex limitation and G E interaction. Twin Res Hum Genet 9(4):481–489

    Article  PubMed  PubMed Central  Google Scholar 

  • Neale MC, Hunter MD, Pritikin JN, Zahery M, Brick TR, Kickpatrick RM, Estabrook R, Bates TC, Maes HH, Boker SM (2016) OpenMx 2.0: Extended structural equation and statistical modeling. Psychometrika 80(2):535–549.

    Article  Google Scholar 

  • Newman HH, Freeman FN, Holzinger KJ (1937) Twins: a study of heredity and environment. The University of Chicago Press, Chicago, IL

    Google Scholar 

  • Pritikin JN, Rappaport LM, Neale MC (2017) Likelihood-based confidence intervals for a parameter with an upper or lower bound. Struct Equ Modeling 24(3):395–401

    Article  PubMed  PubMed Central  Google Scholar 

  • Purcell S (2002) Variance components models for gene-environment interaction in twin analysis. Twin Res 5(6):554–571

    Article  PubMed  Google Scholar 

  • R Core Team (2017) R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing.

  • Steiger JH (1980) Tests for comparing elements of a correlation matrix. Psychol Bull 87(2):245–251

    Article  Google Scholar 

  • Steinsaltz D, Dahl A, Wachter KW (2017) On negative heritability and negative estimates of heritability. bioRxiv.

    Article  Google Scholar 

  • Sullivan PF, Eaves LJ (2002) Evaluation of analyses of univariate discrete twin data. Behav Genet 32(3):221–227

    Article  PubMed  Google Scholar 

  • Venables WN, Ripley BD (2002) Modern applied statistics with S, 4th edn. Springer: New York

    Book  Google Scholar 

  • Visscher PM (2006) A note on the asymptotic distribution of likelihood ratio tests to test variance components. Twin Res Hum Genet 9(4):490–495

    Article  PubMed  Google Scholar 

  • Wu H, Neale MC (2012) Adjusted confidence intervals for a bounded parameter. Behav Genet 42(6):886–898

    Article  PubMed  PubMed Central  Google Scholar 

  • Zheng J, Erzurumluoglu AM, Elsworth BB, Kemp JP, Howe L, Haycock PC, Hemani G, Tansey K, Laurin C, Early Genetics and Lifecourse Epidemiology (EAGLE), Eczema Consortium, Warrington NM, Finucane HK, Price AK, Bulik-Sullivan BK, Anttila V, Paternoster L, Gaunt TR, Evans DM, Neale BM (2016) LD Hub: a centralized database and web interface to perform LD score regression that maximizes the potential of summary level GWAS data for SNP heritability and genetic correlation analysis. Bioinformatics 33(2):272–279

    Article  PubMed  PubMed Central  Google Scholar 

Download references


An earlier draft of this paper was circulated to the faculty of the 2018 International Workshop on Statistical Genetic Methods for Human Complex Traits in Boulder, Colorado and was presented at the 48th meeting of the Behavioral Genetics Association in Boston Mass., June 20 to June 23, 2018. We would like to thank the workshop faculty and students, conference attendees for their suggestions to improve the paper.


This study was supported by NIDA Grants R01DA-018673 and R25DA-26119.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Brad Verhulst.

Ethics declarations

Conflict of interest

Brad Verhulst, Elizabeth Prom-Wormley, Matthew C Keller, Sarah Medland, and Michael C. Neale declare that they have no conflict of interest.

Informed consent

For this type of study formal consent is not required.

Statement of human and animal rights

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Edited by Stacey Cherny.

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Verhulst, B., Prom-Wormley, E., Keller, M. et al. Type I Error Rates and Parameter Bias in Multivariate Behavioral Genetic Models. Behav Genet 49, 99–111 (2019).

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: