Skip to main content

Hierarchical Approximate Bayesian Computation

Psychometrika volume 79, pages 185–209 (2014)Cite this article

Abstract

Approximate Bayesian computation (ABC) is a powerful technique for estimating the posterior distribution of a model’s parameters. It is especially important when the model to be fit has no explicit likelihood function, which happens for computational (or simulation-based) models such as those that are popular in cognitive neuroscience and other areas in psychology. However, ABC is usually applied only to models with few parameters. Extending ABC to hierarchical models has been difficult because high-dimensional hierarchical models add computational complexity that conventional ABC cannot accommodate. In this paper, we summarize some current approaches for performing hierarchical ABC and introduce a new algorithm called Gibbs ABC. This new algorithm incorporates well-known Bayesian techniques to improve the accuracy and efficiency of the ABC approach for estimation of hierarchical models. We then use the Gibbs ABC algorithm to estimate the parameters of two models of signal detection, one with and one without a tractable likelihood function.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

Figure 1.
Figure 2.
Figure 3.
Figure 4.
Figure 5.
Figure 6.
Figure 7.
Figure 8.
Figure 9.

Notes

  1. The Wald distribution describes the behavior of the first-passage time distribution of a single boundary diffusion process.

  2. One can also use a kernel to weigh the fitness of the proposals ξ ∗ and \(\theta^{*}_{j}\).

  3. Note that the posterior distributions of θ and ξ will still exist despite a misspecified model. The goal is to estimate the shapes of those posteriors by generating data that is close to the observed data as measured by ρ(X j ,Y j ).

  4. We investigated a range of priors and determined that the choice of priors, if reasonably variable, had little effect on the final estimated posterior. The priors that we selected permit a range of values for d and b that reflect those that are reported in the perceptual and memory literature (Rouder & Lu, 2005; Lee, 2008).

References

  • Bazin, E., Dawson, K.J., & Beaumont, M.A. (2010). Likelihood-free inference of population structure and local adaptation in a Bayesian hierarchical model. Genetics, 185, 587–602.

    PubMed Central  PubMed  Article  Google Scholar 

  • Beaumont, M.A. (2010). Approximate Bayesian computation in evolution and ecology. Annual Review of Ecology, Evolution, and Systematics, 41, 379–406.

    Article  Google Scholar 

  • Beaumont, M.A., Cornuet, J.-M., Marin, J.-M., & Robert, C.P. (2009). Adaptive approximate Bayesian computation. Biometrika, asp052, 1–8.

    Google Scholar 

  • Benjamin, A.S., Diaz, M., & Wee, S. (2009). Signal detection with criterior noise: applications to recognition memory. Psychological Review, 116, 84–115.

    PubMed Central  PubMed  Article  Google Scholar 

  • Brown, S., & Heathcote, A. (2005). A ballistic model of choice response time. Psychological Review, 112, 117–128.

    PubMed  Article  Google Scholar 

  • Brown, S., & Heathcote, A. (2008). The simplest complete model of choice reaction time: linear ballistic accumulation. Cognitive Psychology, 57, 153–178.

    PubMed  Article  Google Scholar 

  • Brown, S., & Steyvers, M. (2005). The dynamics of experimentally induced criterion shifts. Journal of Experimental Psychology. Learning, Memory, and Cognition, 31, 587–599.

    PubMed  Article  Google Scholar 

  • Christensen, R., Johnson, W., Branscum, A., & Hanson, T.E. (2011). Bayesian ideas and data analysis: an introduction for scientists and statisticians. Boca Raton: CRC Press, Taylor and Francis Group.

    Google Scholar 

  • DeCarlo, L.T. (2012). On a signal detection approach to m-alternative forced choice with bias, with maximum likelihood and Bayesian approaches to estimation. Journal of Mathematical Psychology, 56, 196–207.

    Article  Google Scholar 

  • Dennis, S., & Humphreys, M.S. (2001). A context noise model of episodic word recognition. Psychological Review, 108, 452–478.

    PubMed  Article  Google Scholar 

  • Dorfman, D.D., & Alf, E. (1969). Maximum-likelihood estimation of parameters of signal-detection theory and determination of confidence intervals—rating-method data. Journal of Mathematical Psychology, 6, 487–496.

    Article  Google Scholar 

  • Efron, B. (1986). Why isn’t everyone a Bayesian? American Statistician, 40, 1–11.

    Google Scholar 

  • Egan, J.P. (1958). Recognition memory and the operating characteristic (Tech. Rep. AFCRC-TN-58-51). Hearing and Communication Laboratory, Indiana University, Bloomington, Indiana.

  • Erev, I. (1998). Signal detection by human observers: a cutoff reinforcement learning model of categoriation decisions under uncertainty. Psychological Review, 105, 280–298.

    PubMed  Article  Google Scholar 

  • Excoffer, L., Estoup, A., & Cornuet, J.-M. (2005). Bayesian analysis of an admixture model with mutations and arbitrarily linked markers. Genetics, 169, 1727–1738.

    Article  Google Scholar 

  • Fearnhead, P., & Prangle, D. (2012). Constructing summary statistics for approximate Bayesian computation: semi-automatic approximate Bayesian computation. Journal of the Royal Statistical Society. Series B, 74, 419–474.

    Article  Google Scholar 

  • Gelman, A., Carlin, J.B., Stern, H.S., & Rubin, D.B. (2004). Bayesian data analysis. New York: Chapman and Hall.

    Google Scholar 

  • Gilks, W.R., & Wild, P. (1992). Adaptive rejection sampling for Gibbs sampling. Applied Statistics, 41, 337–348.

    Article  Google Scholar 

  • Green, D.M., & Swets, J.A. (1966). Signal detection theory and psychophysics. New York: Wiley.

    Google Scholar 

  • Hickerson, M.J., & Meyer, C. (2008). Testing comparative phylogeographic models of marine vicariance and dispersal using a hierarchical Bayesian approach. BMC Evolutionary Biology, 8, 322.

    PubMed Central  PubMed  Article  Google Scholar 

  • Hickerson, M.J., Stahl, E.A., & Lessios, H.A. (2006). Test for simultaneous divergence using approximate Bayesian computation. Evolution, 60, 2435–2453.

    PubMed  Article  Google Scholar 

  • Jilk, D.J., Lebiere, C., O’Reilly, R.C., & Anderson, J.R. (2008). SAL: an explicitly pluralistic cognitive architecture. Journal of Experimental and Theoretical Artificial Intelligence, 20, 197–218.

    Article  Google Scholar 

  • Kubovy, M., & Healy, A.F. (1977). The decision rule in probabilistic categorization: what it is and how it is learned. Journal of Experimental Psychology. General, 106, 427–446.

    Article  Google Scholar 

  • Lee, M.D. (2008). Three case studies in the Bayesian analysis of cognitive models. Psychonomic Bulletin & Review, 15, 1–15.

    Article  Google Scholar 

  • Lee, M. D., & Wagenmakers, E.-J. (2012). A course in Bayesian graphical modeling for cognitive science. Available from http://www.ejwagenmakers.com/BayesCourse/BayesBookWeb.pdf; last downloaded January 1, 2012.

  • Lunn, D., Thomas, A., Best, N., & Spiegelhalter, D. (2000). WinBUGS—a Bayesian modelling framework: concepts, structure and extensibility. Statistics and Computing, 10, 325–337.

    Article  Google Scholar 

  • Macmillan, N.A., & Creelman, C.D. (2005). Detection theory: a user’s guide. Mahwah: Lawrence Erlbaum Associates.

    Google Scholar 

  • Malmberg, K.J., Zeelenberg, R., & Shiffrin, R. (2004). Turning up the noise or turning down the volume? On the nature of the impairment of episodic recognition memory by Midazolam. Journal of Experimental Psychology. Learning, Memory, and Cognition, 30, 540–549.

    PubMed  Article  Google Scholar 

  • Matzke, D., & Wagenmakers, E.-J. (2009). Psychological interpretation of the ex-Gaussian and shifted Wald parameters: a diffusion model analysis. Psychonomic Bulletin & Review, 16, 798–817.

    Article  Google Scholar 

  • Mazurek, M.E., Roitman, J.D., Ditterich, J., & Shadlen, M.N. (2003). A role for neural integrators in perceptual decision making. Cerebral Cortex, 13, 1257–1269.

    PubMed  Article  Google Scholar 

  • McElree, B., & Dosher, B.A. (1993). Serial retrieval processes in the recovery of order information. Journal of Experimental Psychology. General, 122, 291–315.

    Article  Google Scholar 

  • Mueller, S.T., & Weidemann, C.T. (2008). Decision noise: an explanation for observed violations of signal detection theory. Psychonomic Bulletin & Review, 15, 465–494.

    Article  Google Scholar 

  • Myung, I.J. (2003). Tutorial on maximum likelihood estimation. Journal of Mathematical Psychology, 47, 90–100.

    Article  Google Scholar 

  • Nelder, J.A., & Mead, R. (1965). A simplex method for function minimization. Computer Journal, 7, 308–313.

    Article  Google Scholar 

  • Nosofsky, R.M., Little, D.R., Donkin, C., & Fific, M. (2011). Short-term memory scanning viewed as exemplar-based categorization. Psychological Review, 118, 280–315.

    PubMed Central  PubMed  Article  Google Scholar 

  • Nosofsky, R.M., & Palmeri, T. (1997). Comparing exemplar-retrieval and decision-bound models of speeded perceptual classification. Perception and Psychophysics, 59, 1027–1048.

    PubMed  Article  Google Scholar 

  • O’Reilly, R., & Frank, M. (2006). Making working memory work: a computational model of learning in the prefrontal cortex and basal ganglia. Neural Computation, 18, 283–328.

    PubMed  Article  Google Scholar 

  • Pritchard, J.K., Seielstad, M.T., Perez-Lezaun, A., & Feldman, M.W. (1999). Population growth of human Y chromosomes: a study of Y chromosome microsatellites. Molecular Biology and Evolution, 16, 1791–1798.

    PubMed  Article  Google Scholar 

  • Ratcliff, R. (1978). A theory of memory retrieval. Psychological Review, 85, 59–108.

    Article  Google Scholar 

  • Ratcliff, R., & Rouder, J.N. (1998). Modeling response times for two-choice decisions. Psychological Science, 9, 347–356.

    Article  Google Scholar 

  • Ratcliff, R., & Smith, P.L. (2004). A comparison of sequential sampling models for two-choice reaction time. Psychological Review, 111, 333–367.

    PubMed Central  PubMed  Article  Google Scholar 

  • Ratcliff, R., & Starns, J. (2009). Modeling confidence and response time in recognition memory. Psychological Review, 116, 59–83.

    PubMed Central  PubMed  Article  Google Scholar 

  • Robert, C.P., & Casella, G. (2004). Monte Carlo statistical methods. New York: Springer.

    Book  Google Scholar 

  • Rouder, J.N., & Lu, J. (2005). An introduction to Bayesian hierarchical models with an application in the theory of signal detection. Psychonomic Bulletin & Review, 12, 573–604.

    Article  Google Scholar 

  • Rouder, J.N., Lu, J., Speckman, P., Sun, D., & Jiang, Y. (2005). A hierarchical model for estimating response time distributions. Psychonomic Bulletin & Review, 12, 195–223.

    Article  Google Scholar 

  • Rouder, J.N., Sun, D., Speckman, P., Lu, J., & Zhou, D. (2003). A hierarchical Bayesian statistical framework for response time distributions. Psychometrika, 68, 589–606.

    Article  Google Scholar 

  • Shiffrin, R.M., Lee, M.D., Kim, W., & Wagenmakers, E.-J. (2008). A survey of model evaluation approaches with a tutorial on hierarchical Bayesian methods. Cognitive Science, 32, 1248–1284.

    PubMed  Article  Google Scholar 

  • Shiffrin, R.M., & Steyvers, M. (1997). A model for recognition memory: REM—retrieving effectively from memory. Psychonomic Bulletin & Review, 4, 145–166.

    Article  Google Scholar 

  • Sisson, S., Fan, Y., & Tanaka, M.M. (2007). Sequential Monte Carlo without likelihoods. Proceedings of the National Academy of Sciences of the United States of America, 104, 1760–1765.

    PubMed Central  PubMed  Article  Google Scholar 

  • Sousa, V.C., Fritz, M., Beaumont, M.A., & Chikhi, L. (2009). Approximate Bayesian computation without summary statistics: the case of admixture. Genetics, 181, 1507–1519.

    PubMed Central  PubMed  Article  Google Scholar 

  • Toni, T., Welch, D., Strelkowa, N., Ipsen, A., & Stumpf, M.P. (2009). Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems. Journal of the Royal Society Interface, 6, 187–202.

    PubMed Central  Article  Google Scholar 

  • Treisman, M., & Williams, T. (1984). A theory of criterion setting with an application to sequential dependencies. Psychological Review, 91, 68–111.

    Article  Google Scholar 

  • Tsetsos, K., Usher, M., & McClelland, J.L. (2011). Testing multi-alternative decision models with non-stationary evidence. Frontiers in Neuroscience, 5, 1–18.

    Article  Google Scholar 

  • Turner, B.M., Dennis, S., & Van Zandt, T. (2013). Likelihood-free Bayesian analysis of memory models. Psychological Review, 120, 667–678.

    PubMed  Article  Google Scholar 

  • Turner, B.M., & Sederberg, P.B. (2012). Approximate Bayesian computation with differential evolution. Journal of Mathematical Psychology, 56, 375–385.

    Article  Google Scholar 

  • Turner, B.M., & Van Zandt, T. (2012). A tutorial on approximate Bayesian computation. Journal of Mathematical Psychology, 56, 69–85.

    Article  Google Scholar 

  • Turner, B.M., Van Zandt, T., & Brown, S.D. (2011). A dynamic, stimulus-driven model of signal detection. Psychological Review, 118, 583–613.

    PubMed  Article  Google Scholar 

  • Usher, M., & McClelland, J.L. (2001). On the time course of perceptual choice: the leaky competing accumulator model. Psychological Review, 108, 550–592.

    PubMed  Article  Google Scholar 

  • Van Zandt, T. (2000). How to fit a response time distribution. Psychonomic Bulletin & Review, 7, 424–465.

    Article  Google Scholar 

  • Van Zandt, T., Colonius, H., & Proctor, R.W. (2000). A comparison of two response time models applied to perceptual matching. Psychonomic Bulletin & Review, 7, 208–256.

    Article  Google Scholar 

  • Van Zandt, T., & Jones, M.R. (2011). Stimulus rhythm and choice performance. Unpublished manuscript.

  • Vandekerckhove, J., Tuerlinckx, F., & Lee, M.D. (2011). Hierarchical diffusion models for two-choice response time. Psychological Methods, 16, 44–62.

    PubMed  Article  Google Scholar 

  • Wald, A. (1947). Sequential analysis. New York: Wiley.

    Google Scholar 

  • Wilkinson, R.D. (2011). Approximate Bayesian computation (ABC) gives exact results under the assumption of model error. Manuscript submitted for publication.

Download references

Author information

Authors and Affiliations

  1. Stanford University, Stanford, USA

    Brandon M. Turner

  2. The Ohio State University, Columbus, USA

    Trisha Van Zandt

Authors
  1. Brandon M. Turner
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Trisha Van Zandt
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Brandon M. Turner.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Turner, B.M., Van Zandt, T. Hierarchical Approximate Bayesian Computation. Psychometrika 79, 185–209 (2014). https://doi.org/10.1007/s11336-013-9381-x

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11336-013-9381-x

Key words

  • approximate Bayesian computation
  • hierarchical Bayesian estimation
  • signal detection theory
  • dynamic signal detection