Skip to main content

Advertisement

Log in

Overview of Modern Measurement Theory and Examples of Its Use to Measure Execution Function in Children

Journal of Pediatric Neuropsychology Aims and scope Submit manuscript

Abstract

Researchers and clinicians in neuropsychology are interested in measuring constructs that cannot be directly observed, such as cognitive ability, executive function, or depression. Modern measurement theory (MMT) provides a framework for understanding and measuring these constructs (also called latent variables) through the use of test batteries and other assessments that provide observable responses that are theoretically related to the underlying construct of interest. MMT subsumes a number of statistical methods, including factor analysis and item response theory (IRT), that are focused on using observable responses to model these unobservable constructs. These methods are able to provide information on how test batteries or individual items relate to each other and the latent variable across varying levels of the construct of interest (e.g., mild ADHD and severe ADHD). Finally, we review several applications of IRT to the measurement of executive function in children to highlight the unique information IRT provides when evaluating the relationship of observed responses to unobservable constructs. In summary, this paper seeks to provide an accessible overview of latent variable models within MMT (with a focus on IRT) that could improve measurement in neuropsychology, if more widely used, and offer readers the foundations of a knowledge base to become more critical consumers of measurement-related research in pediatric neuropsychology.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

reproduced from Willoughby et al. (2010). b TRF reproduced from Willoughby et al. (2012)

References

  • Ackerman, T. A., Gierl, M. J., & Walker, C. M. (2003). Using multidimensional item response theory to evaluate educational and psychological tests. Educational Measurement: Issues and Practice, 22, 37–51. https://doi.org/10.1111/j.1745-3992.2003.tb00136.x

    Article  Google Scholar 

  • Byrne, B. M. (2005). Factor analytic models: Viewing the structure of an assessment instrument from three perspectives. Journal of personality assessment, 85(1), 17–32.

    Article  PubMed  Google Scholar 

  • Cai, L., & Houts, C. R. (2021). Longitudinal analysis of patient reported outcomes in clinical trials: Applications of multilevel and multidimensional item response theory. Psychometrika, 86(3), 754–777.

    Article  Google Scholar 

  • Cordier, R., Munro, N., Wilkes-Gillan, S., Speyer, R., Parsons, L., & Joosten, A. (2019). Applying item response theory (IRT) modeling to an observational measure of childhood pragmatics: The Pragmatics Observations Measure -2. Frontiers in Psychology, 10, 408. https://doi.org/10.3389/fpsyg.2019.00408

    Article  PubMed  PubMed Central  Google Scholar 

  • Cronbach, L. J. (1947). Test “reliability”: Its meaning and determination. Psychometrika, 12(1), 1–16.

    Article  PubMed  Google Scholar 

  • Cudeck, R. (2000). Exploratory factor analysis. In H. E. A. Tinsley & S. D. Brown (Eds.), Handbook of applied multivariate statistics and mathematical modeling (pp. 265–296). Academic Press. https://doi.org/10.1016/B978-012691360-6/50011-2

  • De Champlain, A. F. (2010). A primer on classical test theory and item response theory for assessments in medical education. Medical Education, 44, 109–117.

    Article  Google Scholar 

  • Edwards, M. C. (2009). An introduction to item response theory using the need for cognition scale. Social and Personality Psychology Compass, 3(4), 507–529.

    Article  Google Scholar 

  • Edwards, M. C., & Houts, C. R. (2017). Conceptual overview of computerized adaptive testing. Value and Outcomes Spotlight, 3, 11–12.

    Google Scholar 

  • Embretson, S. E. (1996). The new rules of measurement. Psychological Assessment, 8(4), 341–349.

    Article  Google Scholar 

  • Gibbons, R. D., & Hedeker, D. (1992). Full-information item bifactor analysis. Psychometrika, 57, 423–436.

    Article  Google Scholar 

  • Guilford, J. P. (1956). The structure of intellect. Psychological Bulletin, 53(4), 267–293. https://doi.org/10.1037/h0040755

    Article  PubMed  Google Scholar 

  • Howard, M. C. (2016). A review of exploratory factor analysis decision and overview of current practices: What are we doing and how can we improve? International Journal of Human-Computer Interaction, 32(1), 51–62.

    Article  Google Scholar 

  • Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. https://doi.org/10.1080/10705519909540118

    Article  Google Scholar 

  • Karr, J. E., Areshenkoff, C. N., Rast, P., Hofer, S. M., Iverson, G. L., & Garcia-Barrera, M. A. (2018). The unity and diversity of executive functions: A systematic review and re-analysis of latent variable studies. Psychological bulletin, 144(11), 1147.

  • Langer, M. M., Hill, C. D., Thissen, D., Burwinkle, T. M., Varni, J. W., & DeWalt, D. A. (2008). Item response theory detected differential functioning between healthy and ill children in quality of life metrics. Journal of Clinical Epidemiology, 61(3), 268–276.

    Article  Google Scholar 

  • Lecavalier, L., Gadow, K. D., Devincent, C. J., Houts, C. R., & Edwards, M. C. (2011). Validity of DSM-IV syndromes in preschoolers with autism spectrum disorders. Autism, 15(5), 527–543.

    Article  Google Scholar 

  • MacCallum, R. C. (2007). Factor analysis. In R. E. Millsap & A. Maydeu-Olivares (Eds.), The Sage Handbook of Quantitative Methods in Psychology (pp. 123–147). Sage.

    Google Scholar 

  • Naglieri, J. A., Goldstein, S. (2014) Using the Comprehensive Executive Function Inventory (CEFI) to assess executive function: From theory to application. In: Goldstein S., Naglieri J. (eds) Handbook of Executive Functioning. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-8106-5_14

  • Nunnally, J. C. (1978). Psychometric Theory (2nd ed.). McGraw-Hill.

    Google Scholar 

  • Reeve, B. B. (n.d.). An introduction to modern measurement theory. National Cancer Institute. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.207.3244&rep=rep1&type=pdf. Accessed 29 Nov 2021.

  • Ryan, R. M., Fauth, R. C., & Brooks-Gunn, J. (2006). Childhood Poverty: Implications for School Readiness and Early Childhood Education. In B. Spodek & O. N. Saracho (Eds.), Handbook of research on the education of young children (pp. 323–346). Lawrence Erlbaum Associates Publishers.

    Google Scholar 

  • Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometric Monograph Supplement, 34(4, Pt. 2), 100.

  • Spearman, C. (1904). ‘General intelligence’, objectively determined and measured. The American Journal of Psychology, 15(2), 201–293. https://doi.org/10.2307/1412107

    Article  Google Scholar 

  • Teresi, J. A., Ocepek-Welikson, K., Kleinman, M., Ramirez, M., & Kim, G. (2016). Psychometric properties and performance of the patient reported outcomes measurement information system ® (PROMIS®) depression short forms in ethnically diverse groups. Psychological Test and Assessment Modeling, 58(1), 141–181.

    PubMed  PubMed Central  Google Scholar 

  • Thissen, D. & Orlando, M., (2001). Item response theory for items scores in two categories. In D. Thissen & HJ. Wainer (Eds). Test Scoring (pp. 73–140). Mahwah, NJ: Lawrence Erlbaum Associates

  • Thissen, D., Steinberg, L., & Wainer, H. (1993). Detection of differential item functioning using the parameters of item response models. In P. W. Holland & H. Wainer (Eds.), Differential Item Functioning (pp. 67–113). Lawrence Erlbaum Associates, Inc.

  • Thurstone, L. L. (1938). Primary mental abilities. Psychometric Monographs, 1, ix + 121.

  • Wainer, H., Dorans, N. J., Green, B. F., Steinberg, L., Flaugher, R., Mislevy, R. J., & Thissen, D. (2000). Computerized adaptive testing: A primer (2nd edition). Lawrence Erlbaum Associates, Inc

  • Welsh, J. A., Nix, R. L., Blair, C., Bierman, K. L., & Nelson, K. E. (2010). The development of cognitive skills and gains in academic school readiness for children from low-income families. Journal of Educational Psychology, 102(1), 43–53.

    Article  Google Scholar 

  • Widaman, K. F. (2012). Exploratory factor analysis and confirmatory factor analysis. In H. Cooper, P. M. Camic, D. L. Long, A. T. Panter, D. Rindskopf, & K. J. Sher (Eds.), APA handbook of research methods in psychology, Vol. 3. Data analysis and research publication (pp. 361–389). American Psychological Association. https://doi.org/10.1037/13621-018

  • Willoughby, M. T., Blair, C. B., Wirth, R. J., Greenberg, M., & The Family Life Project Investigators. (2010). The measurement of executive function at age 3 years: Psychometric properties and criterion validity of a new battery of tasks. Psychological Assessment, 22(2), 306–317.

    Article  Google Scholar 

  • Willoughby, M. T., Blair, C. B., Wirth, R. J., Greenberg, M., & The Family Life Project Investigators. (2012). The measurement of executive function at age 5: Psychometric properties and relationship to academic achievement. Psychological Assessment, 24(1), 226–239.

    Article  Google Scholar 

  • Willoughby, M. T., Wirth, R. J., & Blair, C. B. (2011). Contributions of modern measurement theory to measuring executive function in early childhood: An empirical demonstration. Journal of Experimental Child Psychology, 108, 414–435.

    Article  Google Scholar 

  • Wirth, R. J., & Edwards, M. C. (2007). Item factor analysis: Current approaches and future directions. Psychological Measurement, 67, 73–87.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Carrie R. Houts.

Ethics declarations

Competing Interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Houts, C.R., Savord, A. & Wirth, R.J. Overview of Modern Measurement Theory and Examples of Its Use to Measure Execution Function in Children. J Pediatr Neuropsychol 8, 1–14 (2022). https://doi.org/10.1007/s40817-021-00117-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40817-021-00117-7

Keywords

Navigation