Skip to main content
SpringerLink
Log in

Bayesian Estimation of Multinomial Processing Tree Models with Heterogeneity in Participants and Items

  • Published:
Psychometrika Aims and scope Submit manuscript

Abstract

Multinomial processing tree (MPT) models are theoretically motivated stochastic models for the analysis of categorical data. Here we focus on a crossed-random effects extension of the Bayesian latent-trait pair-clustering MPT model. Our approach assumes that participant and item effects combine additively on the probit scale and postulates (multivariate) normal distributions for the random effects. We provide a WinBUGS implementation of the crossed-random effects pair-clustering model and an application to novel experimental data. The present approach may be adapted to handle other MPT models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1.
Figure 2.
Figure 3.
Figure 4.
Figure 5.
Figure 6.
Figure 7.
Figure 8.

Similar content being viewed by others

References

  • Ashby, F.G., Maddox, W.T., & Lee, W.W. (1994). On the dangers of averaging across subjects when using multidimensional scaling or the similarity-choice model. Psychological Science, 144–151.

  • Baayen, R.H. (2008). Analyzing linguistic data: a practical introduction to statistics using R. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  • Batchelder, W.H. (1975). Individual differences and the all-or-none vs incremental learning controversy. Journal of Mathematical Psychology, 12, 53–74.

    Article  Google Scholar 

  • Batchelder, W.H. (1998). Multinomial processing tree models and psychological assessment. Psychological Assessment, 10, 331–344.

    Article  Google Scholar 

  • Batchelder, W.H. (2009). Cognitive psychometrics: using multinomial processing tree models as measurement tools. In S.E. Embretson (Ed.), Measuring psychological constructs: Advances in model based measurement (pp. 71–93). Washington: American Psychological Association.

    Google Scholar 

  • Batchelder, W.H., & Crowther, C.S. (1997). Multinomial processing tree models of factorial categorization. Journal of Mathematical Psychology, 41, 45–55.

    Article  Google Scholar 

  • Batchelder, W.H., & Riefer, D.M. (1980). Separation of storage and retrieval factors in free recall of clusterable pairs. Psychological Review, 87, 375–397.

    Article  Google Scholar 

  • Batchelder, W.H., & Riefer, D.M. (1986). The statistical analysis of a model for storage and retrieval processes in human memory. British Journal of Mathematical & Statistical Psychology, 39, 129–149.

    Article  Google Scholar 

  • Batchelder, W.H., & Riefer, D.M. (1990). Multinomial processing models of source monitoring. Psychological Review, 97, 548–564.

    Article  Google Scholar 

  • Batchelder, W.H., & Riefer, D.M. (1999). Theoretical and empirical review of multinomial process tree modeling. Psychonomic Bulletin & Review, 6, 57–86.

    Article  Google Scholar 

  • Batchelder, W.H., & Riefer, D.M. (2007). Using multinomial processing tree models to measure cognitive deficits in clinical populations. In R. Neufeld (Ed.), Advances in clinical cognitive science: formal modeling of processes and symptoms (pp. 19–50). Washington: American Psychological Association.

    Chapter  Google Scholar 

  • Bröder, A., Herwig, A., Teipel, S., & Fast, K. (2008). Different storage and retrieval deficits in normal aging and mild cognitive impairment: a multinomial modeling analysis. Psychology and Aging, 23, 353–365.

    Article  PubMed  Google Scholar 

  • Brooks, S.B., & Gelman, A. (1998). General methods for monitoring convergence of iterative simulations. Journal of Computational and Graphical Statistics, 7, 434–455.

    Google Scholar 

  • Clark, H.H. (1973). The language-as-fixed-effect fallacy: a critique of language statistics in psychological research. Journal of Verbal Learning and Verbal Behavior, 12, 335–359.

    Article  Google Scholar 

  • Curran, T., & Hintzman, D.L. (1995). Violations of the independence assumption in process dissociation. Journal of Experimental Psychology. Learning, Memory, and Cognition, 21, 531–547.

    Article  PubMed  Google Scholar 

  • De Boeck, P. (2008). Random item IRT models. Psychometrika, 73, 533–559.

    Article  Google Scholar 

  • De Boeck, P., & Partchev, I. (2012). IRTTrees: tree-based item response models of the GLMM family. Journal of Statistical Software, 48, 1–28.

    Google Scholar 

  • DeCarlo, L.T. (2002). Signal detection theory with finite mixture distributions: theoretical developments with applications to recognition memory. Psychological Review, 109, 710–721.

    Article  PubMed  Google Scholar 

  • Deese, J. (1960). Frequency of usage and number of words in free recall: the role of association. Psychological Reports, 337–344.

  • DeLosh, E.L., & McDaniel, M.A. (1996). The role of order information in free recall: application to the word–frequency effect. Journal of Experimental Psychology. Learning, Memory, and Cognition, 22, 1136–1146.

    Article  Google Scholar 

  • Duncan, C.P. (1974). Retrieval of low-frequency words from mixed lists. Bulletin of the Psychonomic Society, 4, 137–138.

    Article  Google Scholar 

  • Erdfelder, E., Auer, T.S., Hilbig, B.E., Aßfalg, A., Moshagen, M., & Nadarevic, L. (2009). Multinomial processing tree models. Zeitschrift für Psychologie, 217, 108–124.

    Article  Google Scholar 

  • Estes, W.K. (1956). The problem of inference from curves based on group data. Psychological Bulletin, 53, 134–140.

    Article  PubMed  Google Scholar 

  • Farrell, S., & Ludwig, C.J.H. (2008). Bayesian and maximum likelihood estimation of hierarchical response time models. Psychonomic Bulletin & Review, 15, 1209–1217.

    Article  Google Scholar 

  • Fischer, G.H., & Molenaar, I.W. (1995). Rasch models: foundations, recent developments, and applications. New York: Springer.

    Book  Google Scholar 

  • Gamerman, D., & Lopes, H.F. (2006). Markov chain Monte Carlo: stochastic simulation for Bayesian inference. Boca Raton: Chapman & Hal/CRC.

    Google Scholar 

  • Gelman, A., Carlin, J.B., Stern, H.S., & Rubin, D.B. (2003). Bayesian data analysis. Boca Raton: Chapman & Hall.

    Google Scholar 

  • Gelman, A., & Hill, J. (2007). Data analysis using regression and multilevel/hierarchical models. Cambridge: Cambridge University Press.

    Google Scholar 

  • Gelman, A., Meng, X., & Stern, H. (1996). Posterior predictive assessment of model fitness via realized discrepancies. Statistica Sinica, 6, 733–807.

    Google Scholar 

  • Gelman, A., & Rubin, D.B. (1992). Inference from iterative simulation using multiple sequences. Statistical Science, 7, 457–472.

    Article  Google Scholar 

  • Gilks, W.R., Richardson, S., & Spiegelhalter, D.J. (1996). Markov chain Monte Carlo in practice. Boca Raton: Chapman & Hall/CRC.

    Google Scholar 

  • Gill, J. (2002). Bayesian methods: a social and behavioral sciences approach. New York: Chapman & Hall.

    Google Scholar 

  • Golz, D., & Erdfelder, E. (2004). Effekte von L-Dopa auf die Speicherung und den Abruf verbaler Informationen bei Schlaganfallpatienten [Effects of L-Dopa on storage and retrieval of verbal information in stroke patients]. Zeitschrift für Neuropsychologie, 15, 275–286.

    Article  Google Scholar 

  • Gregg, V.H. (1976). Word frequency, recognition and recall. In J. Brown (Ed.), Recall and recognition (pp. 183–216). London: Wiley.

    Google Scholar 

  • Hall, J.F. (1954). Learning as a function of word-frequency. The American Journal of Psychology, 138–140.

  • Heathcote, A., Brown, S., & Mewhort, D.J.K. (2000). The power law repealed: the case for an exponential law of practice. Psychonomic Bulletin & Review, 7, 185–207.

    Article  Google Scholar 

  • Hintze, J.L., & Nelson, R.D. (1998). Violin plots: a box plot-density trace synergism. American Statistician, 52, 181–184.

    Google Scholar 

  • Hintzman, D.L. (1980). Simpson’s paradox and the analysis of memory retrieval. Psychological Review, 87, 398–410.

    Article  Google Scholar 

  • Hintzman, D.L. (1993). On variability, Simpson’s paradox, and the relation between recognition and recall: reply to Tulving and Flexser. Psychological Review, 100, 143–148.

    Article  PubMed  Google Scholar 

  • Hu, X., & Batchelder, W.H. (1994). The statistical analysis of general processing tree models with the EM algorithm. Psychometrika, 59, 21–47.

    Article  Google Scholar 

  • Hu, X., & Phillips, G.A. (1999). GPT.EXE: a powerful tool for the visualization and analysis of general processing tree models. Behavior Research Methods, 31, 220–234.

    Article  Google Scholar 

  • Karabatsos, G., & Batchelder, W.H. (2003). Markov chain estimation for test theory without an answer key. Psychometrika, 68, 373–389.

    Article  Google Scholar 

  • Kass, R.E., & Raftery, A.E. (1995). Bayes factors. Journal of the American Statistical Association, 90, 773–795.

    Article  Google Scholar 

  • Klauer, K.C. (2006). Hierarchical multinomial processing tree models: a latent-class approach. Psychometrika, 71, 7–31.

    Article  Google Scholar 

  • Klauer, K.C. (2010). Hierarchical multinomial processing tree models: a latent-trait approach. Psychometrika, 75, 70–98.

    Article  Google Scholar 

  • Kruschke, J.K. (2010). Doing Bayesian data analysis: a tutorial introduction with R and BUGS. Burlington: Academic Press.

    Google Scholar 

  • Lee, M.D. (2008). Three case studies in the Bayesian analysis of cognitive models. Psychonomic Bulletin & Review, 15, 1–15.

    Article  Google Scholar 

  • Lee, M.D. (2011). How cognitive modeling can benefit from hierarchical Bayesian models. Journal of Mathematical Psychology, 55, 1–7.

    Article  Google Scholar 

  • Lee, M.D., & Newell, B.R. (2011). Using hierarchical Bayesian methods to examine the tools of decision-making. Judgment and Decision Making, 6, 832–842.

    Google Scholar 

  • Lee, M. D., & Wagenmakers, E.J. (in press). Bayesian modeling for cognitive science: a practical course. Cambridge: Cambridge University Press.

  • Lee, M.D., & Webb, M.R. (2005). Modeling individual differences in cognition. Psychonomic Bulletin & Review, 12, 605–621.

    Article  Google Scholar 

  • Lord, F.M., & Novick, M.R. (1986). Statistical theories of mental test scores. Reading: Addison-Wesley.

    Google Scholar 

  • Lunn, D., Jackson, C., Best, N., Thomas, A., & Spiegelhalter, D. (2012). The BUGS book: a practical introduction to Bayesian analysis. Boca Raton: CRC Press/Chapman and Hall.

    Google Scholar 

  • Lunn, D., Spiegelhalter, D., Thomas, A., & Best, N. (2009). The BUGS project: Evolution, critique and future directions. Statistics in Medicine, 28, 3049–3067.

    Article  PubMed  Google Scholar 

  • Lunn, D., Thomas, A., Best, N., & Spiegelhalter, D. (2000). WinBUGS—a Bayesian modelling framework: concepts, structure, and extensibility. Statistics and Computing, 10, 325–337.

    Article  Google Scholar 

  • Masson, M.E.J. (2011). A tutorial on a practical Bayesian alternative to null-hypothesis significance testing. Behavior Research Methods, 43, 679–690.

    Article  PubMed  Google Scholar 

  • Merritt, P.S., DeLosh, E.L., & McDaniel, M.A. (2006). Effects of word frequency on individual-item and serial order retention: tests of the order-encoding view. Memory & Cognition, 34, 1615–1627.

    Article  Google Scholar 

  • Moshagen, M. (2010). MultiTree: a computer program for the analysis of multinomial processing tree models. Behavior Research Methods, 42, 42–54.

    Article  PubMed  Google Scholar 

  • Navarro, D.J., Griffiths, T.L., Steyvers, M., & Lee, M.D. (2006). Modeling individual differences using Dirichlet processes. Journal of Mathematical Psychology, 50, 101–122.

    Article  Google Scholar 

  • Nilsson, H., Rieskamp, J., & Wagenmakers, E.J. (2011). Hierarchical Bayesian parameter estimation for cumulative prospect theory. Journal of Mathematical Psychology, 55, 84–93.

    Article  Google Scholar 

  • Plummer, M. (2003). JAGS: a program for analysis of Bayesian graphical models using Gibbs sampling [Computer software manual]. Retrieved from http://citeseer.ist.psu.edu/plummer03jags.html.

  • Postman, L. (1970). Effects of word frequency on acquisition and retention under conditions of free-recall learning. The Quarterly Journal of Experimental Psychology, 22, 185–195.

    Article  Google Scholar 

  • Pratte, M.S., & Rouder, J.N. (2011). Hierarchical single-and dual-process models of recognition memory. Journal of Mathematical Psychology, 55, 36–46.

    Article  Google Scholar 

  • Purdy, B.P., & Batchelder, W.H. (2009). A context-free language for binary multinomial processing tree models. Journal of Mathematical Psychology, 53, 547–561.

    Article  Google Scholar 

  • Raftery, A.E. (1995). Bayesian model selection in social research. Sociological Methodology, 25, 111–164.

    Article  Google Scholar 

  • Raftery, A.E. (1999). Bayes factors and BIC. Sociological Methods & Research, 27, 411–417.

    Article  Google Scholar 

  • Riefer, D.M., & Batchelder, W.H. (1988). Multinomial modeling and the measurement of cognitive processes. Psychological Review, 95, 318–339.

    Article  Google Scholar 

  • Riefer, D.M., & Batchelder, W.H. (1991). Statistical inference for multinomial processing tree models. In J.P. Doignon & J.C.G. Falmagne (Eds.), Mathematical psychology: current developments (pp. 313–335). New York: Springer.

    Chapter  Google Scholar 

  • Riefer, D.M., Knapp, B.R., Batchelder, W.H., Bamber, D., & Manifold, V. (2002). Cognitive psychometrics: assessing storage and retrieval deficits in special populations with multinomial processing tree models. Psychological Assessment, 14, 184–200.

    Article  PubMed  Google Scholar 

  • Rouder, J.N., & Lu, J. (2005). An introduction to Bayesian hierarchical models with an application in the theory of signal detection. Psychonomic Bulletin & Review, 12, 573–604.

    Article  Google Scholar 

  • Rouder, J.N., Lu, J., Morey, R.D., Sun, D., & Speckman, P.L. (2008). A hierarchical process–dissociation model. Journal of Experimental Psychology. General, 137, 370–389.

    Article  PubMed  Google Scholar 

  • Rouder, J.N., Lu, J., Sun, D., Speckman, P., Morey, R., & Naveh-Benjamin, M. (2007). Signal detection models with random participant and item effects. Psychometrika, 72, 621–642.

    Article  Google Scholar 

  • Rouder, J.N., Sun, D., Speckman, P.L., Lu, J., & Zhou, D. (2003). A hierarchical Bayesian statistical framework for response time distributions. Psychometrika, 68, 589–606.

    Article  Google Scholar 

  • Schmittmann, V., Dolan, C., Raijmakers, M., & Batchelder, W.H. (2010). Parameter identification in multinomial processing tree models. Behavior Research Methods, 42, 836–846.

    Article  PubMed  Google Scholar 

  • Sheu, C., & O’Curry, S.L. (1998). Simulation-based Bayesian inference using BUGS. Behavior Research Methods, 30, 232–237.

    Article  Google Scholar 

  • Shiffrin, R.M., Lee, M.D., Kim, W., & Wagenmakers, E.J. (2008). A survey of model evaluation approaches with a tutorial on hierarchical Bayesian methods. Cognitive Science, 32, 1248–1284.

    Article  PubMed  Google Scholar 

  • Smith, J.B., & Batchelder, W.H. (2008). Assessing individual differences in categorical data. Psychonomic Bulletin & Review, 15, 713–731.

    Article  Google Scholar 

  • Smith, J.B., & Batchelder, W.H. (2010). Beta-MPT: multinomial processing tree models for addressing individual differences. Journal of Mathematical Psychology, 54, 167–183.

    Article  Google Scholar 

  • Spiegelhalter, D.J., Thomas, A., Best, N.G., Gilks, W.R., & Lunn, D. (2003). BUGS: Bayesian inference using Gibbs sampling [Computer software manual]. Retrieved from http://www.mrc-bsu.cam.ac.uk/bugs/.

  • Stahl, C., & Klauer, K.C. (2007). HMMTree: a computer program for latent-class hierarchical multinomial processing tree models. Behavior Research Methods, 39, 267–273.

    Article  PubMed  Google Scholar 

  • Stan Development Team (2012). Stan modeling language [Computer software manual]. Retrieved from http://mc-stan.org/.

  • Sumby, W.H. (1963). Word frequency and serial position effects. Journal of Verbal Learning and Verbal Behavior, 1, 443–450.

    Article  Google Scholar 

  • Wagenmakers, E.J. (2007). A practical solution to the pervasive problems of p values. Psychonomic Bulletin & Review, 14, 779–804.

    Article  Google Scholar 

  • Wickelmaier, F. (2011). Mpt: multinomial processing tree (MPT) models [Computer software manual]. Retrieved from http://cran.r-project.org/web/packages/mpt/index.html.

Download references

Acknowledgements

We thank Helen Steingroever for her help with the data collection and Ute Bayen for her considerable effort in providing us with data, which we were unfortunately unable to analyze.

Author information

Authors and Affiliations

  1. Department of Psychology, University of Amsterdam, Weesperplein 4, 1018 XA, Amsterdam, The Netherlands

    Dora Matzke

Authors
  1. Dora Matzke
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Conor V. Dolan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. William H. Batchelder
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Eric-Jan Wagenmakers
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dora Matzke.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Matzke, D., Dolan, C.V., Batchelder, W.H. et al. Bayesian Estimation of Multinomial Processing Tree Models with Heterogeneity in Participants and Items. Psychometrika 80, 205–235 (2015). https://doi.org/10.1007/s11336-013-9374-9

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11336-013-9374-9

Key words

Search

Navigation