Skip to main content
SpringerLink
Log in

How Can General Intelligence Composites Most Accurately Index Psychometric g and What Might Be Good Enough?

  • Published:
Contemporary School Psychology Aims and scope Submit manuscript

Abstract

Intelligence tests produce composite scores that are interpreted as indexes of psychometric g. Like all measures, general intelligence composites are not pure representations of their intended construct, so it is important to evaluate the score characteristics that affect accuracy in measurement. In this study, we identified three characteristics of general intelligence composite scores that vary across intelligence tests, including the number, the g loadings, and the heterogeneity of contributing subtests. We created 77 composite scores to test the influence of these characteristics in measuring psychometric g. Internal consistency reliability coefficients and g loadings were calculated for the composites. General intelligence composites most accurately index psychometric using numerous highly g-loaded subtests. Considering confidence intervals, composites stemming from four subtests produced scores as highly g loaded as those composites that stem from additional subtests. Discussion focuses on what methods should be use to optimally measure psychometric g and how standards in constructing composites should balance psychometric and practical considerations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. For a more detailed discussion about the relations between g loadings and the cognitive complexity of intelligence tests subtests, see McGrew (2015).

  2. The analyses within this study do not rely on absolute differences between scores but rather covariation between and among subtests. As such, age of the norming samples associated with these subtests is probably inconsequential to this study.

  3. We refer to reliability coefficients in general but recognize that there has been variation in how these coefficients were obtained. Subtest reliabilities yielded from analysis of norming data typically stem (a) from internal consistency reliability analysis for power tests and (b) from test–retest reliability analysis for speed tests.

  4. The correlation between the second-order general factors employed in this study was .97. Floyd et al. (2013) reported likelihood ratio tests indicating that this correlation was not statistically significantly different from 1.00; thus, the two second-order general factors were effectively perfectly correlated.

  5. For details about the relations between stratified alpha values and model-based reliability estimates similar to omega hierarchical, see Gignac et al. (2018).

References

  • American Association on Intellectual and Developmental Disabilities. (2010). Mental retardation: definition, classification, and systems of supports (11th ed.). Washington, DC: Author.

    Google Scholar 

  • American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.

    Google Scholar 

  • American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: Author.

    Google Scholar 

  • Baraldi, A. N., & Enders, C. K. (2010). An introduction to modern missing data analyses. Journal of School Psychology, 48, 5–37.

    PubMed  Google Scholar 

  • Benson, N., Kranzler, J. H., & Floyd, R. G. (2018). Exploratory and confirmatory factor analysis of the Universal Nonverbal Intelligence Test-Second Edition: testing dimensionality and invariance across age, gender, race, and ethnicity. Assessment. https://doi.org/10.1177/1073191118786584.

  • Benson, N., Floyd, R. G., Kranzler, J. H., Eckert, T. L., Fefer, S. A., & Morgan, G. B. (2019). Test use and assessment practices of school psychologists in the United States: findings from the 2017 National Survey. Journal of School Psychology, 29–48. doi: https://doi.org/10.1016/j.jsp.2018.12.004.

    PubMed  Google Scholar 

  • Bracken, B. A., & McCallum, R. S. (2016). Universal Nonverbal Intelligence Test (Second ed.). Austin: Pro-Ed.

    Google Scholar 

  • Canivez, G. L. (2011). Hierarchical structure of the Cognitive Assessment System: variance partitions from the Schmid-Leiman (1957) procedure. School Psychology Quarterly, 26, 305–317.

    Google Scholar 

  • Canivez, G. L. (2013). Psychometric versus actuarial interpretation of intelligence and related aptitude batteries. In D. H. Saklofske, C. R. Reynolds, & V. L. Schwean (Eds.), The Oxford handbook of child psychological assessment (pp. 84–112). Oxford: Oxford University Press.

    Google Scholar 

  • Carroll, J. B. (1993). Human cognitive abilities: a survey of factor-analytic studies. New York: Cambridge University Press.

    Google Scholar 

  • Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). Mahwah: Erlbaum.

    Google Scholar 

  • Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334.

    Google Scholar 

  • Cronbach, L. J., Schönemann, P., & McKie, D. (1965). Alpha coefficient for stratified-parallel tests. Educational and Psychological Measurement, 25, 291–312.

    Google Scholar 

  • Dombrowski, S. C., Canivez, G. L., & Watkins, M. W. (2018). Factor structure of the 10 WISC-V primary subtests across four standardization age groups. Contemporary School Psychology, 22, 90–104.

    Google Scholar 

  • Elliott, C. (2007). Differential Ability Scales (Second ed.). San Antonio: Psychological Corporation.

    Google Scholar 

  • Farmer, R. L. (2015). Building a better IQ: G loadings of IQs experimentally controlled for subtest number, heterogeneity, G loading saturation, and weighting. The University of Memphis. Available from ProQuest Dissertations & Theses Global. (1728126312).

  • Farmer, R. L., Floyd, R. G., Reynolds, M. R., & Kranzler, J. (2014). IQs are very strong but imperfect indicators of psychometric g: results from conjoint confirmatory factor analysis. Psychology in the Schools, 51, 801–813.

    Google Scholar 

  • Floyd, R. G., Clark, M. H., & Shadish, W. R. (2008). The exchangeability of IQs: implications for professional psychology. Professional Psychology: Research and Practice, 39, 414–423.

    Google Scholar 

  • Floyd, R. G., McGrew, K. S., Barry, A., Rafael, F. A., & Rogers, J. (2009). General and specific effects on Cattell–Horn–Carroll broad ability composites: analysis of the Woodcock–Johnson III Normative Update CHC factor clusters across development. School Psychology Review, 38, 249–265.

    Google Scholar 

  • Floyd, R. G., Reynolds, M. R., Farmer, R. L., & Kranzler, J. H. (2013). Are the general factors from different child and adolescent intelligence tests the same? Results from a five-sample, six-test analysis. School Psychology Review, 42, 383–401.

    Google Scholar 

  • Floyd, R. G., Farmer, R. L., Schneider, W. J., & McGrew, K. S. (in press). Theories and measurement of intelligence. In L. M. Glidden (Ed.), APA handbook of intellectual and developmental disabilities. Washington, DC: American Psychological Association.

  • Gignac, G. E., & Watkins, M. W. (2013). Bifactor modeling and the estimation of model-based reliability in the WAIS-IV. Multivariate Behavioral Research, 48, 639–662.

    PubMed  Google Scholar 

  • Gignac, G. E., Kovacs, K., & Reynolds, M. R. (2018). Backward and forward serial recall across modalities: an individual differences perspective. Personality and Individual Differences, 121, 147–151.

    Google Scholar 

  • Gottfredson, L. S. (1998). The general intelligence factor. Scientific American Presents-Exploring Intelligence, 9, 24–29.

    Google Scholar 

  • Gustafsson, J.-E. (2002). Measurement from a hierarchical point of view. In H. I. Braun, D. N. Jackson, & D. E. Wiley (Eds.), The role of constructs in psychological and educational measurement (pp. 73–95). Mahwah: Erlbaum.

    Google Scholar 

  • Homack, S. R., & Reynolds, C. R. (2007). Essentials of assessment with brief intelligence tests. New York: Wiley.

    Google Scholar 

  • Humphreys, L. G. (1979). The construct of general intelligence. Intelligence, 3, 105–120.

    Google Scholar 

  • Individuals with Disabilities Education Act (2004). Pub. L. No. 108–446.

  • Jensen, A. R. (1998). The g factor. Westport: Praeger Publisher.

    Google Scholar 

  • Johnson, W., te Nijenhuis, J., & Bouchard, T. J. Jr. (2008). Still just 1 g: consistent results from five test batteries. Intelligence, 36(1), 81–95.

    Google Scholar 

  • Keith, T. Z., & Reynolds, M. R. (2010). Cattell-Horn-Carroll abilities and cognitive tests: what we’ve learned from 20 years of research. Psychology in the Schools, 47, 635–650.

    Google Scholar 

  • Keith, T. Z., Kranzler, J. H., & Flanagan, D. P. (2001). What does the Cognitive Assessment System (CAS) measure? Conjoint confirmatory factor analysis of the CAS and the Woodcock–Johnson Tests of Cognitive Ability (3rd edition). School Psychology Review, 30, 89–119.

    Google Scholar 

  • Keith, T. Z., Fine, J. G., Reynolds, M. R., Taub, G. E., & Kranzler, J. H. (2006). Higher order, multisample, confirmatory factor analysis of the Wechsler Intelligence Scale for Children—Fourth Edition: what does it measure? School Psychology Review, 35, 108–127.

    Google Scholar 

  • Keith, T. Z., Low, J. A., Reynolds, M. R., Patel, P. G., & Ridley, K. P. (2010). Higher-order factor structure of the Differential Ability Scales—II: consistency across ages 4 to 17. Psychology in the Schools, 47, 676–697.

    Google Scholar 

  • Kranzler, J. H., & Floyd, R. G. (2013). Assessing intelligence in children and adolescents: a practical guide. New York: Guilford Press.

    Google Scholar 

  • McDonald, R. P. (1999). Test theory: a unified treatment. Mahwah: Erlbaum.

    Google Scholar 

  • McGrew, K. (2015). Intellectual functioning. In E. Polloway (Ed.), The death penalty and intellectual disability (pp. 85–111). Washington, DC: American Association on Intellectual and Developmental Disabilities.

    Google Scholar 

  • McGrew, K. S., & Flanagan, D. P. (1998). The intelligence test desk reference (ITDR): Gf-Gc cross-battery assessment. Boston: Allyn & Bacon.

    Google Scholar 

  • Meyer, E. M., & Reynolds, M. R. (2017). Scores in space: multidimensional scaling of the WISC-V. Journal of Psychoeducational Assessment, 36, 562–575.

    Google Scholar 

  • Muthén, L. K., & Muthén, B. O. (1998–2012). Mplus user’s guide (7th edn.). Los Angeles: Muthén & Muthén.

  • Naglieri, J. A., & Das, J. P. (1997). Cognitive Assessment System. Itasca: Riverside.

    Google Scholar 

  • Naglieri, J. A., Das, J. P., & Goldstein, S. (2014). Cognitive Assessment System, Second Edition. Itasca: Riverside.

    Google Scholar 

  • Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York: McGraw-Hill.

    Google Scholar 

  • Price, L. R. (2016). Psychometric methods: theory into practice. New York: Guilford Press.

    Google Scholar 

  • Reynolds, M. R. (2013). Interpreting the g loadings of intelligence test composite scores in light of Spearman’s law of diminishing returns. School Psychology Quarterly, 28, 63–76.

    PubMed  Google Scholar 

  • Reynolds, C. R., & Kamphaus, R. W. (2015). Reynolds Intellectual Assessment Scales (Second ed.). Lutz: Psychological Assessment Resources.

    Google Scholar 

  • Reynolds, M. R., & Keith, T. Z. (2017). Multi-group and hierarchical confirmatory factor analysis of the Wechsler Intelligence Scale for Children—Fifth Edition: what does it measure? Intelligence, 62, 31–47.

    Google Scholar 

  • Reynolds, C. R., & Livingston, R. B. (2014). A psychometric primer for school psychologists. In P. L. Harrison & A. Thomas (Eds.), Best practices in school psychology (6th ed., pp. 281–300). Bethesda: National Association of School Psychologists.

    Google Scholar 

  • Reynolds, M. R., Floyd, R. G., & Niileksela, C. R. (2013). How well is psychometric g indexed by global composites? Evidence from three popular intelligence tests. Psychological Assessment, 25, 1314–1321.

    PubMed  Google Scholar 

  • Reynolds, M. R., Hajovsky, D. B., Pacel, J. R., & Niileksela, C. R. (2015). What does the Shipley-2 measure for children and adolescents? Integrated and conjoint confirmatory factor analysis with the WISC-IV. Assessment, 23, 23–41.

    PubMed  Google Scholar 

  • Rushton, J. P., Brainerd, C. J., & Pressley, M. (1983). Behavioral development and construct validity: the principle of aggregation. Psychological Bulletin, 94, 18–38.

    Google Scholar 

  • Salthouse, T. A. (2014). Evaluating the correspondence of different cognitive batteries. Assessment, 21, 131–142.

    PubMed  Google Scholar 

  • Sattler, J. M. (2018). Assessment of children: cognitive foundations (6th ed.). La Mesa: Author.

    Google Scholar 

  • Schmidt, F. L., & Hunter, J. (2004). General mental ability in the world of work: occupational attainment and job performance. Journal of Personality and Social Psychology, 86, 162–173.

    PubMed  Google Scholar 

  • Schneider, W. J. (2013). What if we took our models seriously? Estimating latent scores in individuals. Journal of Psychoeducational Assessment ,31(2), 186–201.

    Google Scholar 

  • Schneider, W. J., & McGrew, K. S. (2018). The Cattell-Horn-Carroll theory of cognitive abilities. In D. P. Flanagan & E. M. McDonough (Eds.), Contemporary intellectual assessment: theories, tests, and issues (4th ed., pp. 73–162). New York: Guilford.

    Google Scholar 

  • Schrank, F. A., McGrew, K. S., & Mather, N. (2014). Woodcock–Johnson IV Tests of Cognitive Abilities. Rolling Meadows: Riverside.

    Google Scholar 

  • Shipley, W. C., Gruber, C. P., Martin, T. A., & Klein, A. M. (2009). Shipley-2. Los Angeles: Western Psychological Services.

    Google Scholar 

  • Sternberg, R. J., Grigorenko, E. L., & Bundy, D. A. (2001). The predictive value of IQ. Merrill-Palmer Quarterly, 47, 1–41.

    Google Scholar 

  • te Nijenhuis, J., & van der Flier, H. (2005). Immigrant-majority group differences on work-related measures: the case for cognitive complexity. Personality and Individual Differences, 38, 1213–1221.

    Google Scholar 

  • Valerius, S., & Sparfeldt, J. R. (2014). Consistent g- as well as consistent verbal-, numerical- and figural-factors in nested factor models? Confirmatory factor analyses using three test batteries. Intelligence, 44, 120–133.

    Google Scholar 

  • Wechsler, D. (2003). The Wechsler Intelligence Scale for Children, Fourth Edition. San Antonio: Psychological Corporation.

    Google Scholar 

  • Wechsler, D. (2008). The Wechsler Adult Intelligence Scale, Fourth Edition. San Antonio: Psychological Corporation.

    Google Scholar 

  • Wechsler, D. (2011). The Wechsler Abbreviated Scale of Intelligence, Second Edition. San Antonio: Psychological Corporation.

    Google Scholar 

  • Wechsler, D. (2014). Wechsler Intelligence Scale for Children, Fifth Edition. San Antonio: Psychological Corporation.

    Google Scholar 

  • Woodcock, R. W., McGrew, K. S., & Mather, N. (2001). Woodcock–Johnson III Tests of Cognitive Abilities. Itasca: Riverside Publishing.

    Google Scholar 

  • World Health Organization, Division of Mental Health and Prevention of Substance Abuse. (2010). ICD-10 guide for mental retardation. Geneva: World Health Organization.

    Google Scholar 

  • Zinbarg, R. E., Yovel, I., Revelle, W., & McDonald, R. P. (2006). Estimating generalizability to a latent variable common to all of a scale’s indicators: a comparison of estimators for ωH. Applied Psychological Measurement, 30, 121–144.

    Google Scholar 

Download references

Acknowledgements

This research was completed as a partial requirement for the first author’s receipt of a doctoral degree in school psychology at The University of Memphis. We thank the Woodcock–Munoz Foundation, Richard Woodcock, Fredrick Schrank, and Kevin McGrew for providing data from the WJ III validity studies.

Author information

Authors and Affiliations

  1. Oklahoma State University, Stillwater, OK, USA

    Ryan L. Farmer

  2. The University of Memphis, Memphis, TN, USA

    Randy G. Floyd & Kristoffer S. Berlin

  3. University of Kansas, Lawrence, KS, USA

    Matthew R. Reynolds

  4. The University of Tennessee Health Sciences Center, Memphis, TN, USA

    Kristoffer S. Berlin

Authors
  1. Ryan L. Farmer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Randy G. Floyd
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Matthew R. Reynolds
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kristoffer S. Berlin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ryan L. Farmer.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical Approval

The article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Farmer, R.L., Floyd, R.G., Reynolds, M.R. et al. How Can General Intelligence Composites Most Accurately Index Psychometric g and What Might Be Good Enough?. Contemp School Psychol 24, 52–67 (2020). https://doi.org/10.1007/s40688-019-00244-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40688-019-00244-1

Keywords

Search

Navigation