Advertisement

Investigation of adolescents’ developmental stages in deductive reasoning: An application of a specialized confirmatory mixture IRT approach

  • Minjeong JeonEmail author
  • Karen Draney
  • Mark Wilson
  • Yinghao Sun
Article

Abstract

In this paper, we propose a specialized confirmatory mixture IRT model to analyze complex cognitive assessment data that is designed to evaluate adolescents’ developmental stages in deductive reasoning. The model is specified for the following purposes: (1) to measure multiple deductive reasoning traits, (2) to identify adolescents’ differential developmental stages based on their ability levels in the multiple dimensions, (3) to quantify the differences in dimension-specific performance between developmental stages, and (4) to examine the difficulty levels of test design factors. A Bayesian estimation of the model is described. The overall goodness-of-fit of the model is assessed as well as its parameter recovery to validate the application of the model to the data.

Keywords

Deductive reasoning Developmental stages Piaget’s theory Mixture IRT Multi-dimensionality Item properties Saltus model 

Notes

Supplementary material

13428_2019_1221_MOESM1_ESM.pdf (39 kb)
(PDF 39.3 KB)

References

  1. Bond, T. G. (1995a). Piaget and measurement II: Empirical validation of the Piagetian model. Archives de Psychologie, 63, 155–185.Google Scholar
  2. Bond, T. G. (1995b). Piaget and measurement I: The Twain really do meet. Archives de Psychologie, 63, 71–87.Google Scholar
  3. Bond, T. G., & Bunting, E. M. (1995). Piaget and measurement III: Reassessing the methode clinique. Archives de Psychologie, 63, 231–255.Google Scholar
  4. Boughton, K. A., & Yamamoto, K. (2007). A HYBRID model for test speededness. In D. von, & C. Carstensen (Eds.) Multivariate and mixture distribution Rasch models (pp. 147–156). New York: Springer.Google Scholar
  5. Chazan, D. (1993). High school geometry students’ justification for their views of empirical evidence and mathematical proof. Educational Studies in Mathematics, 24, 359–387.CrossRefGoogle Scholar
  6. Cho, S. -J., Cohen, A., Kim, S. -H., & Bottge, B. (2010). Latent transition analysis with a mixture IRT measurement model. Applied Psychological Measurement, 34, 583–604.CrossRefGoogle Scholar
  7. Cho, S. -J., Cohen, A. S., & Bottge, B. A. (2013). Detecting intervention effects using a multilevel latent transition analysis with a mixture IRT model. Psychometrika, 78, 576–600.CrossRefGoogle Scholar
  8. Cho, S. -J., Cohen, A. S., & Kim, S. -H. (2014). A mixture group bi-factor model for binary responses. Structural Equation Modeling: A Multidisciplinary Journal, 21, 375–39.CrossRefGoogle Scholar
  9. Choi, I. -H., & Wilson, M. (2015). Multidimensional classification of examinees using the mixture random weights linear logistic test model. Educational and Psychological Measurement, 75, 78–101.CrossRefGoogle Scholar
  10. Commons, M. L., Trudeau, E. J., Stein, S. A., Richards, F. A., & Krause, S. R. (1998). Hierarchical complexity of tasks shows the existence of developmental stages. Developmental Review, 18, 237–278.CrossRefGoogle Scholar
  11. De Boeck, P., Cho, S. -J., & Wilson, M. (2011). Explanatory secondary dimension modelling of latent DIF. Applied Psychological Measurement, 35, 583–603.CrossRefGoogle Scholar
  12. De Boeck, P., & Wilson, M. (2004) Explanatory item response models: A generalized linear and nonlinear approach. New York: Springer.CrossRefGoogle Scholar
  13. De Jong, M. G., & Steenkamp, J. -B. E. M. (2010). Finite mixture multilevel multidimensional ordinal IRT models for large-scale cross-cultural research. Psychometrika, 75, 3–32.  https://doi.org/10.1007/s11336-009-9134-z CrossRefGoogle Scholar
  14. Demetriou, A., & Efklides, A. (1989). The persons’ conception of the structures of developing intellect: Early adolescence to middle age. Genetic, Social, and General Psychology Monographs, 115(371), 423.Google Scholar
  15. Demetriou, A., & Efklides, A. (1994). Structure, development, and dynamics of mind: A meta-Piagetian theory. In A. Demetriou, & A. Efklides (Eds.) Intelligence, mind, and reasoning: Structure and development. Amsterdam, The Netherlands: North-Holland (pp. 75–109).Google Scholar
  16. Draney, K. (2007). The Saltus model applied to proportional reasoning data. Journal of Applied Measurement, 8, 438–455.Google Scholar
  17. Draney, K., & Wilson, M. (2008). A LLTM approach to the examination of teachers’ ratings of classroom assessment tasks. Psychology Science, 50, 417–432.Google Scholar
  18. Draney, K., Wilson, M., Gluck, J., & Spiel, C. (2007). Mixture models in a developmental context. In R. Hancock, & K. M. Samuelson (Eds.) Latent variable mixture models (p. 199-216) Charlotte, NC: Information Age.Google Scholar
  19. Finch, W. H., & Finch, M. E. H. (2013). Investigation of specific learning disability and testing accommodations based differential item functioning using a multilevel multidimensional mixture item response theory model. Educational and Psychological Measurement, 73, 973–993.CrossRefGoogle Scholar
  20. Fischer, G. (1973). Linear logistic test model as an instrument in educational research. Acta Psychologica, 37, 359–374.CrossRefGoogle Scholar
  21. Fischer, K. W., Hand, H. H., & Russel, S. (1984). The development of abstractions in adolescence and adulthood. In M. L. Commons, F. A. Richards, & C. Armon (Eds.) Beyond formal operations: Late adolescent and adult cognitive development (pp. 43–73). New York: Praeger.Google Scholar
  22. Gelman, A., Carlin, J., Stern, H., & Rubin, D. (2004) Bayesian data analysis. New York: Chapman and Hall.Google Scholar
  23. Gelman, A., & Rubin, D. B. (1992). Inference from iterative simulation using multiple sequences. Statistical Science, 7, 457–472.  https://doi.org/10.1214/ss/1177011136 CrossRefGoogle Scholar
  24. Geweke, J. (1992). Evaluating the accuracy of sampling-based approaches to calculating posterior moments. In J. M. Bernardo, J. O. Berger, A. P. Dawiv, & A. F. M. Smith (Eds.) Bayesian statistics (pp. 169–193). Oxford: Clarendon Press.Google Scholar
  25. Huang, H. -Y. (2016). Mixture IRT model with a higher-order structure for latent traits. Educational and Psychological Measurement, 77, 275–304.CrossRefGoogle Scholar
  26. Janssen, R., Schepers, J., & Peres, D. (2004). Models with item and item group predictors. In P. D. Boeck, & M. Wilson (Eds.) Explanatory item response models (pp. 198–212). New York: Springer.Google Scholar
  27. Jeon, M. (2018). A constrained confirmatory mixture IRT model: Extensions and estimation of the Saltus model using mplus. The Quantitative Methods for Psychology, 14, 120–136.CrossRefGoogle Scholar
  28. Jeon, M., Draney, K., & Wilson, M. (2015). Multidimensional Saltus linear logistic test model for modeling children’s cognitive development. In R. Millsap, D. Bolt, L. van der Ark, & W. -C. Wang (Eds.) Quantitative psychology research: Proceedings of the 78th annual meeting of the Psychometric Society (p. 73). New York: Springer.Google Scholar
  29. Jin, K. -Y., Chen, H. -F., & Wang, W.-C. (in press). Mixture item response models for inattentive responding behavior. Organizational Research Methods.Google Scholar
  30. Li, F., Cohen, A., Kim, S.-H., & Cho, S.-J. (2009). Model selection methods for mixture dichotomous IRT models. Applied Psychological Measurement, 33, 353–373.  https://doi.org/10.1177/0146621608326422.CrossRefGoogle Scholar
  31. Markovits, H., Fleury, M. -L., Quinn, S., & Venet, M. (1998). The development of conditional reasoning and the structure of semantic memory. Child Development, 69, 742–755.  https://doi.org/10.2307/1132201 CrossRefGoogle Scholar
  32. Mislevy, R. J., & Verhelst, N. (1990). Modeling item responses when different subjects employ different solution strategies. Psychometrika, 55, 195–215.  https://doi.org/10.1007/BF02295283 CrossRefGoogle Scholar
  33. Mislevy, R. J., & Wilson, M. (1996). Marginal maximum likelihood estimation for a psychometric model of discontinuous development. Psychometrika, 61, 41–71.  https://doi.org/10.1007/BF02296958.CrossRefGoogle Scholar
  34. Molenaar, D., Oberski, D., Vermunt, J., & De Boeck, P. (2016). Hidden Markov IRT models for responses and response times. Multivariate Behavioral Research, 51, 606–626.CrossRefGoogle Scholar
  35. Neimark, E. D. (1975). Longitudinal development of formal operations thought. Genetic Psychology Monographs, 91, 171–225.Google Scholar
  36. Overton, W. F. (1985). Scientific methodologies and the competence- moderator performance issue. In E. D. Neimark, R. de Lisi, & J. L. Newman (Eds.) Moderators of competence (pp. 15–41). Hillsdale: Erlbaum.Google Scholar
  37. Piaget, J. (1971) Biology and knowledge. Chicago: University of Chicago Press.Google Scholar
  38. Plummer, M. (2003). JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling. In Proceedings of the 3rd international workshop on distributed statistical computing (DSC 2003), Vienna, Austria.Google Scholar
  39. Plummer, M. (2011). JAGS version 3.1.0 user manual. (http://sourceforge.net/projects/mcmc-jags/files/)
  40. Rips, L. J. (1994). The psychology of proof, MIT Press, Cambridge.Google Scholar
  41. Roberge, J. J., & Mason, E. J. (1978). Effects of negation on adolescents’ class and conditional reasoning abilities. The Journal of General Psychology, 98, 187–195.  https://doi.org/10.1080/00221309.1978.9920872.CrossRefGoogle Scholar
  42. Schnipke, D. L., & Scrams, D. J. (1997). Modeling item response times with a two-state mixture model: A new method of measuring speededness. Journal of Educational Measurement, 34, 213–232.CrossRefGoogle Scholar
  43. Sinharay, S., Johnson, M. S., & Stern, H. S. (2006). Posterior predictive assessment of item response theory models. Applied Psychological Measurement, 30, 198–321.  https://doi.org/10.1177/0146621605285517 .CrossRefGoogle Scholar
  44. Spiel, C., Gittler, G., Sirsch, U., & Gluck, J. (1997). Application of the Rasch model for testing Piaget’s theory of cognitive development. In R. Langeheine, & J. Rost (Eds.) Applications of latent trait and latent class models in the social sciences (pp. 111–117). Germany: Waxmann.Google Scholar
  45. Spiel, C., & Gluck, J. (2008). A model based test of competence profile and competence level in deductive reasoning. In J. Hartig, E. Klieme, & D. Leutner (Eds.) Assessment of competencies in educational contexts: State of the art and future prospects (pp. 41–60). Gottingen: Hogrefe.Google Scholar
  46. Spiel, C., Gluck, J., & Gossler, H. (2001). Stability and change of unidimensionality: The sample case of deductive reasoning. Journal of Adolescent Research, 16, 150–168.  https://doi.org/10.1177/0743558401162003.CrossRefGoogle Scholar
  47. Staudenmeyer, H., & Bourne, L. E. (1972). Learning to interpret conditional sentences: A developmental study. Psychology, 13, 616–623.Google Scholar
  48. Tijmstra, J., Bolsinova, M. A., & Jeon, M. (in press). Generalized mixture IRT models with different item-response structures: A case study using Likert-scale data. Behavior Research Methods.Google Scholar
  49. van Hiele, P. M. (1986) Structure and insight: A theory of mathematics education. Orlando: Academic Press.Google Scholar
  50. von Davier, M., Xu, X., & Carstensen, C. H. (2011). Measuring growth in a longitudinal large-scale assessment with a general latent variable model. Psychometrika, 76, 318–336.CrossRefGoogle Scholar
  51. Wildman, T. M., & Fletcher, H. J. (1977). Developmental increases and decreases in solutions of conditional syllogism problems. Developmental Psychology, 13, 630–636.CrossRefGoogle Scholar
  52. Wilson, M. R. (1989). Saltus: a psychometric model of discontinuity in cognitive development. Psychological Bulletin, 105, 276–289.  https://doi.org/10.1037/0033-2909.105.2.276 CrossRefGoogle Scholar
  53. Yamamoto, K., & Everson, H. (1997). Modeling the effects of test length and test time on parameter estimation using the HYBRID model. In J. Rost, & R. Langeheine (Eds.) Applications of latent trait and latent class models in the social sciences (pp. 89–98). New York: Waxmann Verlag GmbH.Google Scholar

Copyright information

© The Psychonomic Society, Inc. 2019

Authors and Affiliations

  • Minjeong Jeon
    • 1
    Email author
  • Karen Draney
    • 2
  • Mark Wilson
    • 2
  • Yinghao Sun
    • 3
  1. 1.Department of EducationUniversity of California, Los AngelesLos AngelesUSA
  2. 2.University of CaliforniaBerkeleyUSA
  3. 3.Ohio State UniversityColumbusUSA

Personalised recommendations