Category effects on stimulus estimation: Shifting and skewed frequency distributions—A reexamination

Theoretical Review

Abstract

Duffy, Huttenlocher, Hedges, and Crawford (Psychonomic Bulletin & Review, 17(2), 224–230, 2010) report on experiments where participants estimate the lengths of lines. These studies were designed to test the category adjustment model (CAM), a Bayesian model of judgments. The authors report that their analysis provides evidence consistent with CAM: that there is a bias toward the running mean and not recent stimuli. We reexamine their data. First, we attempt to replicate their analysis, and we obtain different results. Second, we conduct a different statistical analysis. We find significant recency effects, and we identify several specifications where the running mean is not significantly related to judgment. Third, we conduct tests of auxiliary predictions of CAM. We do not find evidence that the bias toward the mean increases with exposure to the distribution. We also do not find that responses longer than the maximum of the distribution or shorter than the minimum become less likely with greater exposure to the distribution. Fourth, we produce a simulated dataset that is consistent with key features of CAM, and our methods correctly identify it as consistent with CAM. We conclude that the Duffy et al. (2010) dataset is not consistent with CAM. We also discuss how conventions in psychology do not sufficiently reduce the likelihood of these mistakes in future research. We hope that the methods that we employ will be used to evaluate other datasets.

Keywords

Judgment Memory Category adjustment model Central tendency bias Recency effects Bayesian judgments 

Notes

Acknowledgements

We thank Roberto Barbera, I-Ming Chiu, L. Elizabeth Crawford, Johanna Hertel, Rosemarie Nagel, and Adam Sanjurjo for helpful comments. This project was supported by Rutgers University Research Council Grant #202297. John Smith thanks Biblioteca de Catalunya.

Supplementary material

13423_2017_1392_MOESM1_ESM.pdf (64 kb)
ESM 1 (PDF 63 kb)
13423_2017_1392_MOESM2_ESM.csv (642 kb)
ESM 2 (CSV 642 kb)
13423_2017_1392_MOESM3_ESM.csv (959 kb)
ESM 3 (CSV 959 kb)
13423_2017_1392_MOESM4_ESM.txt (77 kb)
ESM 4 (TXT 77 kb)

References

  1. Allred, S., Crawford, L. E., Duffy, S., & Smith, J. (2016). Working memory and spatial judgments: Cognitive load increases the central tendency bias. Psychonomic Bulletin & Review, 23(6), 1825–1831.CrossRefGoogle Scholar
  2. Barth, H., Lesser, E., Taggart, J., & Slusser, E. (2015). Spatial estimation: A non-Bayesian alternative. Developmental Science, 18(5), 853–862.CrossRefPubMedGoogle Scholar
  3. Blackwell, D., & Dubins, L. (1962). Merging of opinions with increasing information. Annals of Mathematical Statistics, 33, 882–886.CrossRefGoogle Scholar
  4. Bowers, J. S., & Davis, C. J. (2012a). Bayesian just-so stories in psychology and neuroscience. Psychological Bulletin, 138(3), 389–414.CrossRefPubMedGoogle Scholar
  5. Bowers, J. S., & Davis, C. J. (2012b). Is that what Bayesians believe? Reply to Griffiths, Chater, Norris, and Pouget (2012). Psychological Bulletin, 138(3), 423–426.CrossRefPubMedGoogle Scholar
  6. Cassey, P., Hawkins, G. E., Donkin, C., & Brown, S. D. (2016). Using alien coins to test whether simple inference is Bayesian. Journal of Experimental Psychology: Learning, Memory, and Cognition, 42(3), 497–503.PubMedGoogle Scholar
  7. Chater, N., Goodman, N., Griffiths, T. L., Kemp, C., Oaksford, M., & Tenenbaum, J. B. (2011). The imaginary fundamentalists: The unshocking truth about Bayesian cognitive science. Behavioral and Brain Sciences, 34(4), 194–196.CrossRefGoogle Scholar
  8. Chater, N., Tenenbaum, J. B., & Yuille, A. (2006). Probabilistic models of cognition: Conceptual foundations. Trends in Cognitive Sciences, 10(7), 287–291.CrossRefPubMedGoogle Scholar
  9. Choplin, J. M., & Hummel, J. E. (2002). Magnitude comparisons distort mental representations of magnitude. Journal of Experimental Psychology: General, 131(2), 270–286.CrossRefGoogle Scholar
  10. Corneille, O., Huart, J., Becquart, E., & Brédart, S. (2004). When memory shifts toward more typical category exemplars: Accentuation effects in the recollection of ethnically ambiguous faces. Journal of Personality and Social Psychology, 86(2), 236–250.CrossRefPubMedGoogle Scholar
  11. DeCarlo, L. T., & Cross, D. V. (1990). Sequential effects in magnitude scaling: Models and theory. Journal of Experimental Psychology: General, 119(4), 375–396.CrossRefGoogle Scholar
  12. Duffy, S., Huttenlocher, J., Hedges, L. V., & Crawford, L. E. (2010). Category effects on stimulus estimation: Shifting and skewed frequency distributions. Psychonomic Bulletin & Review, 17, 224–230.CrossRefGoogle Scholar
  13. Elqayam, S., & Evans, J. S. B. (2011). Subtracting “ought” from “is”: Descriptivism versus normativism in the study of human thinking. Behavioral and Brain Sciences, 34(5), 233–248.CrossRefPubMedGoogle Scholar
  14. Estes, W. K. (1956). The problem of inference from curves based on group data. Psychological Bulletin, 53(2), 134–140.CrossRefPubMedGoogle Scholar
  15. Feldman, N. H., Griffiths, T. L., & Morgan, J. L. (2009). The influence of categories on perception: Explaining the perceptual magnet effect as optimal statistical inference. Psychological Review, 116(4), 752–782.CrossRefPubMedPubMedCentralGoogle Scholar
  16. Goldstone, R. (1994). Influences of categorization on perceptual discrimination. Journal of Experimental Psychology: General, 23, 178–200.CrossRefGoogle Scholar
  17. Goodman, N. D., Frank, M. C., Griffiths, T. L., Tenenbaum, J. B., Battaglia, P. W., & Hamrick, J. B. (2015). Relevant and robust: A response to Marcus and Davis (2013). Psychological Science, 26(4), 539–541.CrossRefPubMedGoogle Scholar
  18. Griffiths, T. L., Chater, N., Norris, D., & Pouget, A. (2012). How the Bayesians got their beliefs (and what those beliefs actually are): Comment on Bowers and Davis (2012). Psychological Bulletin, 138(3), 415–422.CrossRefPubMedGoogle Scholar
  19. Griffiths, T. L., & Tenenbaum, J. B. (2006). Optimal predictions in everyday cognition. Psychological Science, 17(9), 767–773.CrossRefPubMedGoogle Scholar
  20. Hahn, U. (2014). The Bayesian boom: Good thing or bad? Frontiers in Psychology, 5, 765.CrossRefPubMedPubMedCentralGoogle Scholar
  21. Hemmer, P., & Steyvers, M. (2009a). Integrating episodic memories and prior knowledge at multiple levels of abstraction. Psychonomic Bulletin & Review, 16(1), 80–87.CrossRefGoogle Scholar
  22. Hemmer, P., & Steyvers, M. (2009b). A Bayesian account of reconstructive memory. Topics in Cognitive Science, 1, 189–202.CrossRefPubMedGoogle Scholar
  23. Hemmer, P., Tauber, S., & Steyvers, M. (2015). Moving beyond qualitative evaluations of Bayesian models of cognition. Psychonomic Bulletin & Review, 22(3), 614–628.CrossRefGoogle Scholar
  24. Hertwig, R., Pachur, T., & Kurzenhäuser, S. (2005). Judgments of risk frequencies: Tests of possible cognitive mechanisms. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31(4), 621–642.PubMedGoogle Scholar
  25. Hollingworth, H. L. (1910). The central tendency of judgment. The Journal of Philosophy, Psychology and Scientific Methods, 7(17), 461–469.CrossRefGoogle Scholar
  26. Huttenlocher, J., Hedges, L. V., & Vevea, J. L. (2000). Why do categories affect stimulus judgment? Journal of Experimental Psychology: General, 129, 220–241.CrossRefGoogle Scholar
  27. John, L. K., Loewenstein, G., & Prelec, D. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23(5), 524–532.CrossRefPubMedGoogle Scholar
  28. Jones, M., Curran, T., Mozer, M. C., & Wilder, M. H. (2013). Sequential effects in response time reveal learning mechanisms and event representations. Psychological Review, 120(3), 628–666.CrossRefPubMedGoogle Scholar
  29. Jones, M., & Love, B. C. (2011a). Bayesian fundamentalism or enlightenment? On the explanatory status and theoretical contributions of Bayesian models of cognition. Behavioral and Brain Sciences, 34(4), 169–188.CrossRefPubMedGoogle Scholar
  30. Jones, M., & Love, B. C. (2011b). Pinning down the theoretical commitments of Bayesian cognitive models. Behavioral and Brain Sciences, 34(4), 215–231.CrossRefGoogle Scholar
  31. Lewandowsky, S., Griffiths, T. L., & Kalish, M. L. (2009). The wisdom of individuals: Exploring people’s knowledge about everyday events using iterated learning. Cognitive Science, 33(6), 969–998.CrossRefPubMedGoogle Scholar
  32. Loken, E., & Gelman, A. (2017). Measurement error and the replication crisis. Science, 355(6325), 584–585.CrossRefPubMedGoogle Scholar
  33. Marcus, G. F., & Davis, E. (2013). How robust are probabilistic models of higher-level cognition? Psychological Science, 24(12), 2351–2360.CrossRefPubMedGoogle Scholar
  34. Marcus, G. F., & Davis, E. (2015). Still searching for principles: A response to Goodman et al. (2015). Psychological Science, 26(4), 542–544.CrossRefPubMedGoogle Scholar
  35. Moore, D. A., & Healy, P. J. (2008). The trouble with overconfidence. Psychological Review, 115(2), 502–517.CrossRefPubMedGoogle Scholar
  36. Mozer, M. C., Pashler, H., & Homaei, H. (2008). Optimal predictions in everyday cognition: The wisdom of individuals or crowds? Cognitive Science, 32(7), 1133–1147.CrossRefPubMedGoogle Scholar
  37. Norris, D., & McQueen, J. M. (2008). Shortlist B: A Bayesian model of continuous speech recognition. Psychological Review, 115(2), 357–395.CrossRefPubMedGoogle Scholar
  38. Pashler, H., & Harris, C. R. (2012). Is the replicability crisis overblown? Three arguments examined. Perspectives on Psychological Science, 7(6), 531–536.CrossRefPubMedGoogle Scholar
  39. Perfors, A., Tenenbaum, J. B., Griffiths, T. L., & Xu, F. (2011). A tutorial introduction to Bayesian models of cognitive development. Cognition, 120(3), 302–321.CrossRefPubMedGoogle Scholar
  40. Petzold, P., & Haubensak, G. (2004). The influence of category membership of stimuli on sequential effects in magnitude judgment. Perception & Psychophysics, 66(4), 665–678.CrossRefGoogle Scholar
  41. Petzschner, F. H., Glasauer, S., & Stephan, K. E. (2015). A Bayesian perspective on magnitude estimation. Trends in Cognitive Sciences, 19(5), 285–293.CrossRefPubMedGoogle Scholar
  42. Sailor, K. M., & Antoine, M. (2005). Is memory for stimulus magnitude Bayesian? Memory & Cognition, 33, 840–851.CrossRefGoogle Scholar
  43. Sampson, R. J., & Raudenbush, S. W. (2004). Seeing disorder: Neighborhood stigma and the social construction of “broken windows”. Social Psychology Quarterly, 67(4), 319–342.CrossRefGoogle Scholar
  44. Savage, L. J. (1954). The foundations of statistics. New York: Wiley.Google Scholar
  45. Siegler, R. S. (1987). The perils of averaging data over strategies: An example from children’s addition. Journal of Experimental Psychology: General, 116(3), 250–264.CrossRefGoogle Scholar
  46. Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366.CrossRefPubMedGoogle Scholar
  47. Spencer, J. P., & Hund, A. M. (2002). Prototypes and particulars: Geometric and experience-dependent spatial categories. Journal of Experimental Psychology: General, 131(1), 16–37.CrossRefGoogle Scholar
  48. Steegen, S., Tuerlinckx, F., Gelman, A., & Vanpaemel, W. (2016). Increasing transparency through a multiverse analysis. Perspectives on Psychological Science, 11(5), 702–712.CrossRefPubMedGoogle Scholar
  49. Stewart, N., Brown, G. D., & Chater, N. (2002). Sequence effects in categorization of simple perceptual stimuli. Journal of Experimental Psychology: Learning, Memory, and Cognition, 28(1), 3–11.PubMedGoogle Scholar
  50. Tauber, S., Navarro, D. J., Perfors, A., & Steyvers, M. (2017). Bayesian models of cognition revisited: Setting optimality aside and letting data drive psychological theory. Psychological Review, 124(4), 410–441.CrossRefPubMedGoogle Scholar
  51. Tenenbaum, J. B., Griffiths, T. L., & Kemp, C. (2006). Theory-based Bayesian models of inductive learning and reasoning. Trends in Cognitive Sciences, 10(7), 309–318.CrossRefPubMedGoogle Scholar
  52. Wagenmakers, E. J., Wetzels, R., Borsboom, D., & Maas, H. L. J. (2011). Why psychologists must change the way they analyze their data: The case of psi. Journal of Personality and Social Psychology, 100(3), 426–432.CrossRefPubMedGoogle Scholar
  53. Wicherts, J. M., Veldkamp, C. L., Augusteijn, H. E., Bakker, M., van Aert, R. C., & Van Assen, M. A. (2016). Degrees of freedom in planning, running, analyzing, and reporting psychological studies: A checklist to avoid p-hacking. Frontiers in Psychology, 7, 1832.CrossRefPubMedPubMedCentralGoogle Scholar
  54. Wilder, M., Jones, M., & Mozer, M. C. (2009). Sequential effects reflect parallel learning of multiple environmental regularities. Advances in Neural Information Processing Systems, 22, 2053–2061.Google Scholar
  55. Xu, J., & Griffiths, T. L. (2010). A rational analysis of the effects of memory biases on serial reproduction. Cognitive Psychology, 60(2), 107–126.CrossRefPubMedGoogle Scholar
  56. Yu, A. J., & Cohen, J. D. (2009). Sequential effects: Superstition or rational behavior? Advances in Neural Information Processing Systems, 21, 1873–1880.Google Scholar

Copyright information

© Psychonomic Society, Inc. 2017

Authors and Affiliations

  1. 1.Department of PsychologyRutgers University-CamdenCamdenUSA
  2. 2.Department of EconomicsRutgers University-CamdenCamdenUSA

Personalised recommendations