Abstract
Researchers and clinicians in neuropsychology are interested in measuring constructs that cannot be directly observed, such as cognitive ability, executive function, or depression. Modern measurement theory (MMT) provides a framework for understanding and measuring these constructs (also called latent variables) through the use of test batteries and other assessments that provide observable responses that are theoretically related to the underlying construct of interest. MMT subsumes a number of statistical methods, including factor analysis and item response theory (IRT), that are focused on using observable responses to model these unobservable constructs. These methods are able to provide information on how test batteries or individual items relate to each other and the latent variable across varying levels of the construct of interest (e.g., mild ADHD and severe ADHD). Finally, we review several applications of IRT to the measurement of executive function in children to highlight the unique information IRT provides when evaluating the relationship of observed responses to unobservable constructs. In summary, this paper seeks to provide an accessible overview of latent variable models within MMT (with a focus on IRT) that could improve measurement in neuropsychology, if more widely used, and offer readers the foundations of a knowledge base to become more critical consumers of measurement-related research in pediatric neuropsychology.
References
Ackerman, T. A., Gierl, M. J., & Walker, C. M. (2003). Using multidimensional item response theory to evaluate educational and psychological tests. Educational Measurement: Issues and Practice, 22, 37–51. https://doi.org/10.1111/j.1745-3992.2003.tb00136.x
Byrne, B. M. (2005). Factor analytic models: Viewing the structure of an assessment instrument from three perspectives. Journal of personality assessment, 85(1), 17–32.
Cai, L., & Houts, C. R. (2021). Longitudinal analysis of patient reported outcomes in clinical trials: Applications of multilevel and multidimensional item response theory. Psychometrika, 86(3), 754–777.
Cordier, R., Munro, N., Wilkes-Gillan, S., Speyer, R., Parsons, L., & Joosten, A. (2019). Applying item response theory (IRT) modeling to an observational measure of childhood pragmatics: The Pragmatics Observations Measure -2. Frontiers in Psychology, 10, 408. https://doi.org/10.3389/fpsyg.2019.00408
Cronbach, L. J. (1947). Test “reliability”: Its meaning and determination. Psychometrika, 12(1), 1–16.
Cudeck, R. (2000). Exploratory factor analysis. In H. E. A. Tinsley & S. D. Brown (Eds.), Handbook of applied multivariate statistics and mathematical modeling (pp. 265–296). Academic Press. https://doi.org/10.1016/B978-012691360-6/50011-2
De Champlain, A. F. (2010). A primer on classical test theory and item response theory for assessments in medical education. Medical Education, 44, 109–117.
Edwards, M. C. (2009). An introduction to item response theory using the need for cognition scale. Social and Personality Psychology Compass, 3(4), 507–529.
Edwards, M. C., & Houts, C. R. (2017). Conceptual overview of computerized adaptive testing. Value and Outcomes Spotlight, 3, 11–12.
Embretson, S. E. (1996). The new rules of measurement. Psychological Assessment, 8(4), 341–349.
Gibbons, R. D., & Hedeker, D. (1992). Full-information item bifactor analysis. Psychometrika, 57, 423–436.
Guilford, J. P. (1956). The structure of intellect. Psychological Bulletin, 53(4), 267–293. https://doi.org/10.1037/h0040755
Howard, M. C. (2016). A review of exploratory factor analysis decision and overview of current practices: What are we doing and how can we improve? International Journal of Human-Computer Interaction, 32(1), 51–62.
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. https://doi.org/10.1080/10705519909540118
Karr, J. E., Areshenkoff, C. N., Rast, P., Hofer, S. M., Iverson, G. L., & Garcia-Barrera, M. A. (2018). The unity and diversity of executive functions: A systematic review and re-analysis of latent variable studies. Psychological bulletin, 144(11), 1147.
Langer, M. M., Hill, C. D., Thissen, D., Burwinkle, T. M., Varni, J. W., & DeWalt, D. A. (2008). Item response theory detected differential functioning between healthy and ill children in quality of life metrics. Journal of Clinical Epidemiology, 61(3), 268–276.
Lecavalier, L., Gadow, K. D., Devincent, C. J., Houts, C. R., & Edwards, M. C. (2011). Validity of DSM-IV syndromes in preschoolers with autism spectrum disorders. Autism, 15(5), 527–543.
MacCallum, R. C. (2007). Factor analysis. In R. E. Millsap & A. Maydeu-Olivares (Eds.), The Sage Handbook of Quantitative Methods in Psychology (pp. 123–147). Sage.
Naglieri, J. A., Goldstein, S. (2014) Using the Comprehensive Executive Function Inventory (CEFI) to assess executive function: From theory to application. In: Goldstein S., Naglieri J. (eds) Handbook of Executive Functioning. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-8106-5_14
Nunnally, J. C. (1978). Psychometric Theory (2nd ed.). McGraw-Hill.
Reeve, B. B. (n.d.). An introduction to modern measurement theory. National Cancer Institute. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.207.3244&rep=rep1&type=pdf. Accessed 29 Nov 2021.
Ryan, R. M., Fauth, R. C., & Brooks-Gunn, J. (2006). Childhood Poverty: Implications for School Readiness and Early Childhood Education. In B. Spodek & O. N. Saracho (Eds.), Handbook of research on the education of young children (pp. 323–346). Lawrence Erlbaum Associates Publishers.
Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometric Monograph Supplement, 34(4, Pt. 2), 100.
Spearman, C. (1904). ‘General intelligence’, objectively determined and measured. The American Journal of Psychology, 15(2), 201–293. https://doi.org/10.2307/1412107
Teresi, J. A., Ocepek-Welikson, K., Kleinman, M., Ramirez, M., & Kim, G. (2016). Psychometric properties and performance of the patient reported outcomes measurement information system ® (PROMIS®) depression short forms in ethnically diverse groups. Psychological Test and Assessment Modeling, 58(1), 141–181.
Thissen, D. & Orlando, M., (2001). Item response theory for items scores in two categories. In D. Thissen & HJ. Wainer (Eds). Test Scoring (pp. 73–140). Mahwah, NJ: Lawrence Erlbaum Associates
Thissen, D., Steinberg, L., & Wainer, H. (1993). Detection of differential item functioning using the parameters of item response models. In P. W. Holland & H. Wainer (Eds.), Differential Item Functioning (pp. 67–113). Lawrence Erlbaum Associates, Inc.
Thurstone, L. L. (1938). Primary mental abilities. Psychometric Monographs, 1, ix + 121.
Wainer, H., Dorans, N. J., Green, B. F., Steinberg, L., Flaugher, R., Mislevy, R. J., & Thissen, D. (2000). Computerized adaptive testing: A primer (2nd edition). Lawrence Erlbaum Associates, Inc
Welsh, J. A., Nix, R. L., Blair, C., Bierman, K. L., & Nelson, K. E. (2010). The development of cognitive skills and gains in academic school readiness for children from low-income families. Journal of Educational Psychology, 102(1), 43–53.
Widaman, K. F. (2012). Exploratory factor analysis and confirmatory factor analysis. In H. Cooper, P. M. Camic, D. L. Long, A. T. Panter, D. Rindskopf, & K. J. Sher (Eds.), APA handbook of research methods in psychology, Vol. 3. Data analysis and research publication (pp. 361–389). American Psychological Association. https://doi.org/10.1037/13621-018
Willoughby, M. T., Blair, C. B., Wirth, R. J., Greenberg, M., & The Family Life Project Investigators. (2010). The measurement of executive function at age 3 years: Psychometric properties and criterion validity of a new battery of tasks. Psychological Assessment, 22(2), 306–317.
Willoughby, M. T., Blair, C. B., Wirth, R. J., Greenberg, M., & The Family Life Project Investigators. (2012). The measurement of executive function at age 5: Psychometric properties and relationship to academic achievement. Psychological Assessment, 24(1), 226–239.
Willoughby, M. T., Wirth, R. J., & Blair, C. B. (2011). Contributions of modern measurement theory to measuring executive function in early childhood: An empirical demonstration. Journal of Experimental Child Psychology, 108, 414–435.
Wirth, R. J., & Edwards, M. C. (2007). Item factor analysis: Current approaches and future directions. Psychological Measurement, 67, 73–87.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing Interests
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Houts, C.R., Savord, A. & Wirth, R.J. Overview of Modern Measurement Theory and Examples of Its Use to Measure Execution Function in Children. J Pediatr Neuropsychol 8, 1–14 (2022). https://doi.org/10.1007/s40817-021-00117-7
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40817-021-00117-7