Advertisement

Psychometrika

, Volume 80, Issue 1, pp 205–235 | Cite as

Bayesian Estimation of Multinomial Processing Tree Models with Heterogeneity in Participants and Items

  • Dora MatzkeEmail author
  • Conor V. Dolan
  • William H. Batchelder
  • Eric-Jan Wagenmakers
Article

Abstract

Multinomial processing tree (MPT) models are theoretically motivated stochastic models for the analysis of categorical data. Here we focus on a crossed-random effects extension of the Bayesian latent-trait pair-clustering MPT model. Our approach assumes that participant and item effects combine additively on the probit scale and postulates (multivariate) normal distributions for the random effects. We provide a WinBUGS implementation of the crossed-random effects pair-clustering model and an application to novel experimental data. The present approach may be adapted to handle other MPT models.

Key words

multinomial processing tree model parameter heterogeneity crossed-random effects model hierarchical Bayesian modeling 

Notes

Acknowledgements

We thank Helen Steingroever for her help with the data collection and Ute Bayen for her considerable effort in providing us with data, which we were unfortunately unable to analyze.

References

  1. Ashby, F.G., Maddox, W.T., & Lee, W.W. (1994). On the dangers of averaging across subjects when using multidimensional scaling or the similarity-choice model. Psychological Science, 144–151. Google Scholar
  2. Baayen, R.H. (2008). Analyzing linguistic data: a practical introduction to statistics using R. Cambridge: Cambridge University Press. CrossRefGoogle Scholar
  3. Batchelder, W.H. (1975). Individual differences and the all-or-none vs incremental learning controversy. Journal of Mathematical Psychology, 12, 53–74. CrossRefGoogle Scholar
  4. Batchelder, W.H. (1998). Multinomial processing tree models and psychological assessment. Psychological Assessment, 10, 331–344. CrossRefGoogle Scholar
  5. Batchelder, W.H. (2009). Cognitive psychometrics: using multinomial processing tree models as measurement tools. In S.E. Embretson (Ed.), Measuring psychological constructs: Advances in model based measurement (pp. 71–93). Washington: American Psychological Association. Google Scholar
  6. Batchelder, W.H., & Crowther, C.S. (1997). Multinomial processing tree models of factorial categorization. Journal of Mathematical Psychology, 41, 45–55. CrossRefGoogle Scholar
  7. Batchelder, W.H., & Riefer, D.M. (1980). Separation of storage and retrieval factors in free recall of clusterable pairs. Psychological Review, 87, 375–397. CrossRefGoogle Scholar
  8. Batchelder, W.H., & Riefer, D.M. (1986). The statistical analysis of a model for storage and retrieval processes in human memory. British Journal of Mathematical & Statistical Psychology, 39, 129–149. CrossRefGoogle Scholar
  9. Batchelder, W.H., & Riefer, D.M. (1990). Multinomial processing models of source monitoring. Psychological Review, 97, 548–564. CrossRefGoogle Scholar
  10. Batchelder, W.H., & Riefer, D.M. (1999). Theoretical and empirical review of multinomial process tree modeling. Psychonomic Bulletin & Review, 6, 57–86. CrossRefGoogle Scholar
  11. Batchelder, W.H., & Riefer, D.M. (2007). Using multinomial processing tree models to measure cognitive deficits in clinical populations. In R. Neufeld (Ed.), Advances in clinical cognitive science: formal modeling of processes and symptoms (pp. 19–50). Washington: American Psychological Association. CrossRefGoogle Scholar
  12. Bröder, A., Herwig, A., Teipel, S., & Fast, K. (2008). Different storage and retrieval deficits in normal aging and mild cognitive impairment: a multinomial modeling analysis. Psychology and Aging, 23, 353–365. CrossRefPubMedGoogle Scholar
  13. Brooks, S.B., & Gelman, A. (1998). General methods for monitoring convergence of iterative simulations. Journal of Computational and Graphical Statistics, 7, 434–455. Google Scholar
  14. Clark, H.H. (1973). The language-as-fixed-effect fallacy: a critique of language statistics in psychological research. Journal of Verbal Learning and Verbal Behavior, 12, 335–359. CrossRefGoogle Scholar
  15. Curran, T., & Hintzman, D.L. (1995). Violations of the independence assumption in process dissociation. Journal of Experimental Psychology. Learning, Memory, and Cognition, 21, 531–547. CrossRefPubMedGoogle Scholar
  16. De Boeck, P. (2008). Random item IRT models. Psychometrika, 73, 533–559. CrossRefGoogle Scholar
  17. De Boeck, P., & Partchev, I. (2012). IRTTrees: tree-based item response models of the GLMM family. Journal of Statistical Software, 48, 1–28. Google Scholar
  18. DeCarlo, L.T. (2002). Signal detection theory with finite mixture distributions: theoretical developments with applications to recognition memory. Psychological Review, 109, 710–721. CrossRefPubMedGoogle Scholar
  19. Deese, J. (1960). Frequency of usage and number of words in free recall: the role of association. Psychological Reports, 337–344. Google Scholar
  20. DeLosh, E.L., & McDaniel, M.A. (1996). The role of order information in free recall: application to the word–frequency effect. Journal of Experimental Psychology. Learning, Memory, and Cognition, 22, 1136–1146. CrossRefGoogle Scholar
  21. Duncan, C.P. (1974). Retrieval of low-frequency words from mixed lists. Bulletin of the Psychonomic Society, 4, 137–138. CrossRefGoogle Scholar
  22. Erdfelder, E., Auer, T.S., Hilbig, B.E., Aßfalg, A., Moshagen, M., & Nadarevic, L. (2009). Multinomial processing tree models. Zeitschrift für Psychologie, 217, 108–124. CrossRefGoogle Scholar
  23. Estes, W.K. (1956). The problem of inference from curves based on group data. Psychological Bulletin, 53, 134–140. CrossRefPubMedGoogle Scholar
  24. Farrell, S., & Ludwig, C.J.H. (2008). Bayesian and maximum likelihood estimation of hierarchical response time models. Psychonomic Bulletin & Review, 15, 1209–1217. CrossRefGoogle Scholar
  25. Fischer, G.H., & Molenaar, I.W. (1995). Rasch models: foundations, recent developments, and applications. New York: Springer. CrossRefGoogle Scholar
  26. Gamerman, D., & Lopes, H.F. (2006). Markov chain Monte Carlo: stochastic simulation for Bayesian inference. Boca Raton: Chapman & Hal/CRC. Google Scholar
  27. Gelman, A., Carlin, J.B., Stern, H.S., & Rubin, D.B. (2003). Bayesian data analysis. Boca Raton: Chapman & Hall. Google Scholar
  28. Gelman, A., & Hill, J. (2007). Data analysis using regression and multilevel/hierarchical models. Cambridge: Cambridge University Press. Google Scholar
  29. Gelman, A., Meng, X., & Stern, H. (1996). Posterior predictive assessment of model fitness via realized discrepancies. Statistica Sinica, 6, 733–807. Google Scholar
  30. Gelman, A., & Rubin, D.B. (1992). Inference from iterative simulation using multiple sequences. Statistical Science, 7, 457–472. CrossRefGoogle Scholar
  31. Gilks, W.R., Richardson, S., & Spiegelhalter, D.J. (1996). Markov chain Monte Carlo in practice. Boca Raton: Chapman & Hall/CRC. Google Scholar
  32. Gill, J. (2002). Bayesian methods: a social and behavioral sciences approach. New York: Chapman & Hall. Google Scholar
  33. Golz, D., & Erdfelder, E. (2004). Effekte von L-Dopa auf die Speicherung und den Abruf verbaler Informationen bei Schlaganfallpatienten [Effects of L-Dopa on storage and retrieval of verbal information in stroke patients]. Zeitschrift für Neuropsychologie, 15, 275–286. CrossRefGoogle Scholar
  34. Gregg, V.H. (1976). Word frequency, recognition and recall. In J. Brown (Ed.), Recall and recognition (pp. 183–216). London: Wiley. Google Scholar
  35. Hall, J.F. (1954). Learning as a function of word-frequency. The American Journal of Psychology, 138–140. Google Scholar
  36. Heathcote, A., Brown, S., & Mewhort, D.J.K. (2000). The power law repealed: the case for an exponential law of practice. Psychonomic Bulletin & Review, 7, 185–207. CrossRefGoogle Scholar
  37. Hintze, J.L., & Nelson, R.D. (1998). Violin plots: a box plot-density trace synergism. American Statistician, 52, 181–184. Google Scholar
  38. Hintzman, D.L. (1980). Simpson’s paradox and the analysis of memory retrieval. Psychological Review, 87, 398–410. CrossRefGoogle Scholar
  39. Hintzman, D.L. (1993). On variability, Simpson’s paradox, and the relation between recognition and recall: reply to Tulving and Flexser. Psychological Review, 100, 143–148. CrossRefPubMedGoogle Scholar
  40. Hu, X., & Batchelder, W.H. (1994). The statistical analysis of general processing tree models with the EM algorithm. Psychometrika, 59, 21–47. CrossRefGoogle Scholar
  41. Hu, X., & Phillips, G.A. (1999). GPT.EXE: a powerful tool for the visualization and analysis of general processing tree models. Behavior Research Methods, 31, 220–234. CrossRefGoogle Scholar
  42. Karabatsos, G., & Batchelder, W.H. (2003). Markov chain estimation for test theory without an answer key. Psychometrika, 68, 373–389. CrossRefGoogle Scholar
  43. Kass, R.E., & Raftery, A.E. (1995). Bayes factors. Journal of the American Statistical Association, 90, 773–795. CrossRefGoogle Scholar
  44. Klauer, K.C. (2006). Hierarchical multinomial processing tree models: a latent-class approach. Psychometrika, 71, 7–31. CrossRefGoogle Scholar
  45. Klauer, K.C. (2010). Hierarchical multinomial processing tree models: a latent-trait approach. Psychometrika, 75, 70–98. CrossRefGoogle Scholar
  46. Kruschke, J.K. (2010). Doing Bayesian data analysis: a tutorial introduction with R and BUGS. Burlington: Academic Press. Google Scholar
  47. Lee, M.D. (2008). Three case studies in the Bayesian analysis of cognitive models. Psychonomic Bulletin & Review, 15, 1–15. CrossRefGoogle Scholar
  48. Lee, M.D. (2011). How cognitive modeling can benefit from hierarchical Bayesian models. Journal of Mathematical Psychology, 55, 1–7. CrossRefGoogle Scholar
  49. Lee, M.D., & Newell, B.R. (2011). Using hierarchical Bayesian methods to examine the tools of decision-making. Judgment and Decision Making, 6, 832–842. Google Scholar
  50. Lee, M. D., & Wagenmakers, E.J. (in press). Bayesian modeling for cognitive science: a practical course. Cambridge: Cambridge University Press. Google Scholar
  51. Lee, M.D., & Webb, M.R. (2005). Modeling individual differences in cognition. Psychonomic Bulletin & Review, 12, 605–621. CrossRefGoogle Scholar
  52. Lord, F.M., & Novick, M.R. (1986). Statistical theories of mental test scores. Reading: Addison-Wesley. Google Scholar
  53. Lunn, D., Jackson, C., Best, N., Thomas, A., & Spiegelhalter, D. (2012). The BUGS book: a practical introduction to Bayesian analysis. Boca Raton: CRC Press/Chapman and Hall. Google Scholar
  54. Lunn, D., Spiegelhalter, D., Thomas, A., & Best, N. (2009). The BUGS project: Evolution, critique and future directions. Statistics in Medicine, 28, 3049–3067. CrossRefPubMedGoogle Scholar
  55. Lunn, D., Thomas, A., Best, N., & Spiegelhalter, D. (2000). WinBUGS—a Bayesian modelling framework: concepts, structure, and extensibility. Statistics and Computing, 10, 325–337. CrossRefGoogle Scholar
  56. Masson, M.E.J. (2011). A tutorial on a practical Bayesian alternative to null-hypothesis significance testing. Behavior Research Methods, 43, 679–690. CrossRefPubMedGoogle Scholar
  57. Merritt, P.S., DeLosh, E.L., & McDaniel, M.A. (2006). Effects of word frequency on individual-item and serial order retention: tests of the order-encoding view. Memory & Cognition, 34, 1615–1627. CrossRefGoogle Scholar
  58. Moshagen, M. (2010). MultiTree: a computer program for the analysis of multinomial processing tree models. Behavior Research Methods, 42, 42–54. CrossRefPubMedGoogle Scholar
  59. Navarro, D.J., Griffiths, T.L., Steyvers, M., & Lee, M.D. (2006). Modeling individual differences using Dirichlet processes. Journal of Mathematical Psychology, 50, 101–122. CrossRefGoogle Scholar
  60. Nilsson, H., Rieskamp, J., & Wagenmakers, E.J. (2011). Hierarchical Bayesian parameter estimation for cumulative prospect theory. Journal of Mathematical Psychology, 55, 84–93. CrossRefGoogle Scholar
  61. Plummer, M. (2003). JAGS: a program for analysis of Bayesian graphical models using Gibbs sampling [Computer software manual]. Retrieved from http://citeseer.ist.psu.edu/plummer03jags.html.
  62. Postman, L. (1970). Effects of word frequency on acquisition and retention under conditions of free-recall learning. The Quarterly Journal of Experimental Psychology, 22, 185–195. CrossRefGoogle Scholar
  63. Pratte, M.S., & Rouder, J.N. (2011). Hierarchical single-and dual-process models of recognition memory. Journal of Mathematical Psychology, 55, 36–46. CrossRefGoogle Scholar
  64. Purdy, B.P., & Batchelder, W.H. (2009). A context-free language for binary multinomial processing tree models. Journal of Mathematical Psychology, 53, 547–561. CrossRefGoogle Scholar
  65. Raftery, A.E. (1995). Bayesian model selection in social research. Sociological Methodology, 25, 111–164. CrossRefGoogle Scholar
  66. Raftery, A.E. (1999). Bayes factors and BIC. Sociological Methods & Research, 27, 411–417. CrossRefGoogle Scholar
  67. Riefer, D.M., & Batchelder, W.H. (1988). Multinomial modeling and the measurement of cognitive processes. Psychological Review, 95, 318–339. CrossRefGoogle Scholar
  68. Riefer, D.M., & Batchelder, W.H. (1991). Statistical inference for multinomial processing tree models. In J.P. Doignon & J.C.G. Falmagne (Eds.), Mathematical psychology: current developments (pp. 313–335). New York: Springer. CrossRefGoogle Scholar
  69. Riefer, D.M., Knapp, B.R., Batchelder, W.H., Bamber, D., & Manifold, V. (2002). Cognitive psychometrics: assessing storage and retrieval deficits in special populations with multinomial processing tree models. Psychological Assessment, 14, 184–200. CrossRefPubMedGoogle Scholar
  70. Rouder, J.N., & Lu, J. (2005). An introduction to Bayesian hierarchical models with an application in the theory of signal detection. Psychonomic Bulletin & Review, 12, 573–604. CrossRefGoogle Scholar
  71. Rouder, J.N., Lu, J., Morey, R.D., Sun, D., & Speckman, P.L. (2008). A hierarchical process–dissociation model. Journal of Experimental Psychology. General, 137, 370–389. CrossRefPubMedGoogle Scholar
  72. Rouder, J.N., Lu, J., Sun, D., Speckman, P., Morey, R., & Naveh-Benjamin, M. (2007). Signal detection models with random participant and item effects. Psychometrika, 72, 621–642. CrossRefGoogle Scholar
  73. Rouder, J.N., Sun, D., Speckman, P.L., Lu, J., & Zhou, D. (2003). A hierarchical Bayesian statistical framework for response time distributions. Psychometrika, 68, 589–606. CrossRefGoogle Scholar
  74. Schmittmann, V., Dolan, C., Raijmakers, M., & Batchelder, W.H. (2010). Parameter identification in multinomial processing tree models. Behavior Research Methods, 42, 836–846. CrossRefPubMedGoogle Scholar
  75. Sheu, C., & O’Curry, S.L. (1998). Simulation-based Bayesian inference using BUGS. Behavior Research Methods, 30, 232–237. CrossRefGoogle Scholar
  76. Shiffrin, R.M., Lee, M.D., Kim, W., & Wagenmakers, E.J. (2008). A survey of model evaluation approaches with a tutorial on hierarchical Bayesian methods. Cognitive Science, 32, 1248–1284. CrossRefPubMedGoogle Scholar
  77. Smith, J.B., & Batchelder, W.H. (2008). Assessing individual differences in categorical data. Psychonomic Bulletin & Review, 15, 713–731. CrossRefGoogle Scholar
  78. Smith, J.B., & Batchelder, W.H. (2010). Beta-MPT: multinomial processing tree models for addressing individual differences. Journal of Mathematical Psychology, 54, 167–183. CrossRefGoogle Scholar
  79. Spiegelhalter, D.J., Thomas, A., Best, N.G., Gilks, W.R., & Lunn, D. (2003). BUGS: Bayesian inference using Gibbs sampling [Computer software manual]. Retrieved from http://www.mrc-bsu.cam.ac.uk/bugs/.
  80. Stahl, C., & Klauer, K.C. (2007). HMMTree: a computer program for latent-class hierarchical multinomial processing tree models. Behavior Research Methods, 39, 267–273. CrossRefPubMedGoogle Scholar
  81. Stan Development Team (2012). Stan modeling language [Computer software manual]. Retrieved from http://mc-stan.org/.
  82. Sumby, W.H. (1963). Word frequency and serial position effects. Journal of Verbal Learning and Verbal Behavior, 1, 443–450. CrossRefGoogle Scholar
  83. Wagenmakers, E.J. (2007). A practical solution to the pervasive problems of p values. Psychonomic Bulletin & Review, 14, 779–804. CrossRefGoogle Scholar
  84. Wickelmaier, F. (2011). Mpt: multinomial processing tree (MPT) models [Computer software manual]. Retrieved from http://cran.r-project.org/web/packages/mpt/index.html.

Copyright information

© The Psychometric Society 2013

Authors and Affiliations

  • Dora Matzke
    • 1
    Email author
  • Conor V. Dolan
  • William H. Batchelder
  • Eric-Jan Wagenmakers
  1. 1.Department of PsychologyUniversity of AmsterdamAmsterdamThe Netherlands

Personalised recommendations