Advertisement

Validation of a matrix reasoning task for mobile devices

  • Anja Pahor
  • Trevor Stavropoulos
  • Susanne M. Jaeggi
  • Aaron R. Seitz
Article

Abstract

Many cognitive tasks have been adapted for tablet-based testing, but tests to assess nonverbal reasoning ability, as measured by matrix-type problems that are suited to repeated testing, have yet to be adapted for and validated on mobile platforms. Drawing on previous research, we developed the University of California Matrix Reasoning Task (UCMRT)—a short, user-friendly measure of abstract problem solving with three alternate forms that works on tablets and other mobile devices and that is targeted at a high-ability population frequently used in the literature (i.e., college students). To test the psychometric properties of UCMRT, a large sample of healthy young adults completed parallel forms of the test, and a subsample also completed Raven’s Advanced Progressive Matrices and a math test; furthermore, we collected college records of academic ability and achievement. These data show that UCMRT is reliable and has adequate convergent and external validity. UCMRT is self-administrable, freely available for researchers, facilitates repeated testing of fluid intelligence, and resolves numerous limitations of existing matrix tests.

Keywords

UCMRT Reasoning Fluid intelligence Matrix problems Validity Mobile 

Supplementary material

13428_2018_1152_MOESM1_ESM.pdf (1014 kb)
ESM 1 (PDF 1013 kb)

References

  1. Ackerman, P. L., & Kanfer, R. (2009). Test length and cognitive fatigue: An empirical examination of effects on performance and test-taker reactions. Journal of Experimental Psychology. Applied, 15, 163–181.  https://doi.org/10.1037/a0015719 CrossRefPubMedGoogle Scholar
  2. Arthur, W, & Day, D. V. (1994). Development of a short form for the Raven Advanced Progressive Matrices test. Educational and Psychological Measurement, 54, 394–403.  https://doi.org/10.1177/0013164494054002013 CrossRefGoogle Scholar
  3. Arthur, W., & Woehr, D. J. (1993). A confirmatory factor analytic study examining the dimensionality of the raven’s advanced progressive matrices. Educational and Psychological Measurement, 53, 471–478.  https://doi.org/10.1177/0013164493053002016 CrossRefGoogle Scholar
  4. Bors, D. A., & Stokes, T. L. (1998). Raven’s Advanced Progressive Matrices: Norms for first-year university students and the development of a short form. Educational and Psychological Measurement, 58, 382–398.  https://doi.org/10.1177/0013164498058003002 CrossRefGoogle Scholar
  5. Cattell, R. B. (1963). Theory of fluid and crystallized intelligence: A critical experiment. Journal of Educational Psychology, 54, 1–22.  https://doi.org/10.1037/h0046743 CrossRefGoogle Scholar
  6. Clark, C. M., Lawlor-Savage, L., & Goghari, V. M. (2017). Working memory training in healthy young adults: Support for the null from a randomized comparison to active and passive control groups. Plos One, 12, e0177707.  https://doi.org/10.1371/journal.pone.0177707 CrossRefPubMedPubMedCentralGoogle Scholar
  7. Colom, R., Quiroga, M. Á., Shih, P. C., Martínez, K., Burgaleta, M., Martínez-Molina, A., . . . Ramírez, I. (2010). Improvement in working memory is not related to increased intelligence scores. Intelligence, 38, 497–505.  https://doi.org/10.1016/j.intell.2010.06.008 CrossRefGoogle Scholar
  8. Colom, R., Román, F. J., Abad, F. J., Shih, P. C., Privado, J., Froufe, M., . . . Jaeggi, S. M. (2013). Adaptive n-back training does not improve fluid intelligence at the construct level: Gains on individual tests suggest that training may enhance visuospatial processing. Intelligence, 41, 712–727.  https://doi.org/10.1016/j.intell.2013.09.002 CrossRefGoogle Scholar
  9. Condon, D. M., & Revelle, W. (2014). The international cognitive ability resource: Development and initial validation of a public-domain measure. Intelligence, 43, 52–64.  https://doi.org/10.1016/j.intell.2014.01.004 CrossRefGoogle Scholar
  10. Coyle, T. R., & Pillow, D. R. (2008). SAT and ACT predict college GPA after removing g. Intelligence, 36, 719–729.  https://doi.org/10.1016/j.intell.2008.05.001 CrossRefGoogle Scholar
  11. Frearson, W., & Eysenck, H. J. (1986). Intelligence, reaction time (RT) and a new “odd-man-out” RT paradigm. Personality and Individual Differences, 7, 807–817.  https://doi.org/10.1016/0191-8869(86)90079-6 CrossRefGoogle Scholar
  12. Freund, P. A., & Holling, H. (2011). How to get really smart: Modeling retest and training effects in ability testing using computer-generated figural matrix items. Intelligence, 39, 233–243.  https://doi.org/10.1016/j.intell.2011.02.009 CrossRefGoogle Scholar
  13. Goff, M., & Ackerman, P. L. (1992). Personality-intelligence relations: Assessment of typical intellectual engagement. Journal of Educational Psychology, 84, 537–552.  https://doi.org/10.1037/0022-0663.84.4.537 CrossRefGoogle Scholar
  14. Hamel, R., & Schmittmann, V. D. (2006). The 20-minute version as a predictor of the Raven Advanced Progressive Matrices test. Educational and Psychological Measurement, 66, 1039–1046.  https://doi.org/10.1177/0013164406288169 CrossRefGoogle Scholar
  15. Hogrefe, A. B., Studer-Luethi, B., Kodzhabashev, S., & Perrig, W. J. (2017). Mechanisms underlying N-back training: Response consistency during training influences training outcome. Journal of Cognitive Enhancement, 1, 406–418.  https://doi.org/10.1007/s41465-017-0042-3 CrossRefGoogle Scholar
  16. Hossiep, R., Turck, D., & Hasella, M. (1999). Bochumer Matrizentest. BOMAT advanced.Google Scholar
  17. ICAR Catalogue. (2017). Version 1.0, 06 I 17. Retrieved August 19, 2018, from https://icar-project.com/ICAR_Catalogue.pdf
  18. Jaeggi, S. M., Buschkuehl, M., Shah, P., & Jonides, J. (2014). The role of individual differences in cognitive training and transfer. Memory & Cognition, 42, 464–480.  https://doi.org/10.3758/s13421-013-0364-z CrossRefGoogle Scholar
  19. Jaeggi, S. M., Studer-Luethi, B., Buschkuehl, M., Su, Y.-F., Jonides, J., & Perrig, W. J. (2010). The relationship between n-back performance and matrix reasoning—Implications for training and transfer. Intelligence, 38, 625–635.  https://doi.org/10.1016/j.intell.2010.09.001 CrossRefGoogle Scholar
  20. JASP Team. (2018). JASP (Version 0.9.0.1) [Computer software]. Retrieved from https://jasp-stats.org/download/
  21. Koenig, K. A., Frey, M. C., & Detterman, D. K. (2008). ACT and general cognitive ability. Intelligence, 36, 153–160.  https://doi.org/10.1016/j.intell.2007.03.005 CrossRefGoogle Scholar
  22. Koretz, D., Yu, C., Mbekeani, P. P., Langi, M., Dhaliwal, T., & Braslow, D. (2016). Predicting freshman grade point average from college admissions test scores and state high school test scores. AERA Open, 2, 233285841667060.  https://doi.org/10.1177/2332858416670601 CrossRefGoogle Scholar
  23. Lee, M. D., & Wagenmakers, E.-J. (2013). Bayesian cognitive modeling: A practical course. Cambridge, UK: Cambridge University Press.  https://doi.org/10.1017/CBO9781139087759 CrossRefGoogle Scholar
  24. Matzen, L. E., Benz, Z. O., Dixon, K. R., Posey, J., Kroger, J. K., & Speed, A. E. (2010). Recreating Raven’s: Software for systematically generating large numbers of Raven-like matrix problems with normed properties. Behavior Research Methods, 42, 525–541.  https://doi.org/10.3758/BRM.42.2.525 CrossRefPubMedGoogle Scholar
  25. Raven, J. C. (1938). Progressive matrices: A perceptual test of intelligence. London: H.K. Lewis.Google Scholar
  26. Raven, J. C., Court, J. H., & Raven, J. (1998). Manual for Raven’s Progressive Matrices and Vocabulary Scales: Section 4. Advanced Progressive Matrices, Sets I & II. Oxford, UK: Oxford Psychologists Press.Google Scholar
  27. Redick, T. S., Shipstead, Z., Harrison, T. L., Hicks, K. L., Fried, D. E., Hambrick, D. Z., . . . Engle, R. W. (2013). No evidence of intelligence improvement after working memory training: A randomized, placebo-controlled study. Journal of Experimental Psychology: General, 142, 359–379.  https://doi.org/10.1037/a0029082 CrossRefGoogle Scholar
  28. Rohde, T. E., & Thompson, L. A. (2007). Predicting academic achievement with cognitive ability. Intelligence, 35, 83–92.  https://doi.org/10.1016/j.intell.2006.05.004 CrossRefGoogle Scholar
  29. Salthouse, T. A. (1993). Influence of working memory on adult age differences in matrix reasoning. British Journal of Psychology, 84(Part 2), 171–199.CrossRefGoogle Scholar
  30. Sefcek, J. A., Miller, G. F., & Figueredo, A. J. (2016). Development and validation of an 18-item medium form of the Ravens Advanced Progressive Matrices. SAGE Open, 6, 215824401665191.  https://doi.org/10.1177/2158244016651915 CrossRefGoogle Scholar
  31. Stough, C., Camfield, D., Kure, C., Tarasuik, J., Downey, L., Lloyd, J., . . . Reynolds, J. (2011). Improving general intelligence with a nutrient-based pharmacological intervention. Intelligence, 39, 100–107.  https://doi.org/10.1016/j.intell.2011.01.003 CrossRefGoogle Scholar
  32. Unsworth, N., & Engle, R. (2005). Working memory capacity and fluid abilities: Examining the correlation between Operation Span and Raven. Intelligence, 33, 67–81.  https://doi.org/10.1016/j.intell.2004.08.003 CrossRefGoogle Scholar
  33. Unsworth, N., Heitz, R. P., Schrock, J. C., & Engle, R. W. (2005). An automated version of the operation span task. Behavior Research Methods, 37, 498–505.  https://doi.org/10.3758/BF03192720 CrossRefPubMedGoogle Scholar
  34. Unsworth, N., Redick, T. S., Lakey, C. E., & Young, D. L. (2010). Lapses in sustained attention and their relation to executive control and fluid abilities: An individual differences investigation. Intelligence, 38, 111–122.  https://doi.org/10.1016/j.intell.2009.08.002 CrossRefGoogle Scholar
  35. Wagenmakers, E.-J., Love, J., Marsman, M., Jamil, T., Ly, A., Verhagen, J., . . . Morey, R. D. (2018). Bayesian inference for psychology: Part II. Example applications with JASP. Psychonomic Bulletin & Review, 25, 58–76.  https://doi.org/10.3758/s13423-017-1323-7 CrossRefGoogle Scholar
  36. Westrick, P. A., Le, H., Robbins, S. B., Radunzel, J. M. R., & Schmidt, F. L. (2015). College Performance and retention: A meta-analysis of the predictive validities of ACT® scores, high school grades, and SES. Educational Assessment, 20, 23–45.  https://doi.org/10.1080/10627197.2015.997614 CrossRefGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2018

Authors and Affiliations

  • Anja Pahor
    • 1
  • Trevor Stavropoulos
    • 1
  • Susanne M. Jaeggi
    • 2
  • Aaron R. Seitz
    • 1
  1. 1.University of CaliforniaRiversideUSA
  2. 2.University of CaliforniaIrvineUSA

Personalised recommendations