Behavior Research Methods

, Volume 51, Issue 4, pp 1477–1484 | Cite as

Decision contamination in the wild: Sequential dependencies in online review ratings

  • David W. VinsonEmail author
  • Rick Dale
  • Michael N. Jones


Current judgments are systematically biased by prior judgments. Such biases occur in ways that seem to reflect the cognitive system’s ability to adapt to statistical regularities within the environment. These cognitive sequential dependencies have primarily been evaluated in carefully controlled laboratory experiments. In this study, we used these well-known laboratory findings to guide our analysis of two datasets, consisting of over 2.2 million business review ratings from Yelp and 4.2 million movie and television review ratings from Amazon. We explored how within-reviewer ratings are influenced by previous ratings. Our findings suggest a contrast effect: Current ratings are systematically biased away from prior ratings, and the magnitude of this bias decays over several reviews. This work is couched within a broader program that aims to use well-established laboratory findings to guide our understanding of patterns in naturally occurring and large-scale behavioral data.


Sequential dependence Decision making Data mining Online reviews Big data Cognitive principles 


Author note

This work was funded by NSF BCS-1056744 to M.N.J. D.W.V. was supported by an IBM PhD fellowship.


  1. Bock, K., & Griffin, Z. M. (2000). The persistence of structural priming: Transient activation or implicit learning? Journal of Experimental Psychology: General, 129, 177–192. CrossRefGoogle Scholar
  2. Cantallops, A. S., & Salvi, F. (2014). New consumer behavior: A review of research on eWOM and hotels. International Journal of Hospitality Management, 36, 41–51. CrossRefGoogle Scholar
  3. Dellarocas, C., & Narayan, R. (2006). A statistical measure of a population’s propensity to engage in post-purchase online word-of-mouth. Statistical Science, 21, 277–285.CrossRefGoogle Scholar
  4. Dixon, P, McAnsh, S., & Read, L. (2012). Repetition effects in grasping. Canadian Journal of Experimental Psychology, 66, 1–17.CrossRefGoogle Scholar
  5. Donkin, C., Rae, B., Heathcote, A., & Brown, S. D. (2015). Why is accurately labelling simple magnitudes so hard? A past, present and future look at simple perceptual judgment. In J. R. Busemeyer, Z. Wang, J. T. Townsend, & A. Eidels (Eds.), Oxford handbook of computational and mathematical psychology (pp. 121–141). Oxford, UK: Oxford University Press.Google Scholar
  6. Doshi, A., Tran, C., Wilder, M. H., Mozer, M. C., & Trivedi, M. M. (2012). Sequential dependencies in driving. Cognitive Science, 36, 948–963.CrossRefGoogle Scholar
  7. Freyd, J. J., & Finke, R. A. (1984). Representational momentum. Journal of Experimental Psychology: Learning, Memory, and Cognition, 10, 126–132. Google Scholar
  8. Furnham, A. (1986). The robustness of the recency effect: Studies using legal evidence. Journal of General Psychology, 113, 351–357.CrossRefGoogle Scholar
  9. Garner, W. R. (1953). An informational analysis of absolute judgments of loudness. Journal of Experimental Psychology, 46, 373–380. CrossRefGoogle Scholar
  10. Holland, M. K., & Lockhead, G. R. (1968). Sequential effects in absolute judgments of loudness. Perception & Psychophysics, 3, 409–414. CrossRefGoogle Scholar
  11. Hsu, S.-M., & Yang, L.-X. (2013). Sequential effects in facial expression categorization. Emotion, 13, 573–586. CrossRefGoogle Scholar
  12. Hu, N., Zhang, J., & Pavlou, P. A. (2009). Overcoming the J-shaped distribution of product reviews. Communications of the ACM, 52, 144–147. CrossRefGoogle Scholar
  13. Jesteadt, W., Luce, R. D., & Green, D. M. (1977). Sequential effects in judgments of loudness. Journal of Experimental Psychology: Human Perception and Performance, 3, 92–104. Google Scholar
  14. Jones, M. N. (2017). Big data in cognitive science. New York, NY: Taylor & Francis.Google Scholar
  15. Kristjánsson, Á. (2006). Simultaneous priming along multiple feature dimensions in a visual search task. Vision Research, 46, 2554–2570.CrossRefGoogle Scholar
  16. Laming, D. (1984). The relativity of “absolute” judgements. British Journal of Mathematical and Statistical Psychology, 37, 152–183.CrossRefGoogle Scholar
  17. Liberman, A., Fischer, J., & Whitney, D. (2014). Serial dependence in the perception of faces. Current Biology, 24, 2569–2574. CrossRefGoogle Scholar
  18. Luca, M. (2011, September 16). Reviews, reputation, and revenue: The case of (Harvard Business School NOM Unit Working Paper, 12-016). Cambridge, MA: Harvard University, School of Business.
  19. Mozer, M. C., Pashler, H., Wilder, M., Lindsey, R. V., Jones, M. C., & Jones, M. N. (2010). Decontaminating human judgments by removing sequential dependencies. In J. Laffterty, C. K. I. Williams, J. Shawe-Taylor, R. S. Zemel, & A. Culota (Eds.), Advances in neural information processing systems 23 (pp. 1705–1713). La Jolla, CA: NIPS Foundation.Google Scholar
  20. Mudambi, S. M., & Schuff, D. (2010). What makes a helpful online review? A study of customer reviews on Amazon. com. MIS Quarterly, 34, 185–200.CrossRefGoogle Scholar
  21. Mumma, G. H., & Wilson, S. B. (1995). Procedural debiasing of primacy/anchoring effects in clinical-like judgments. Journal of Clinical Psychology, 51, 841–853.CrossRefGoogle Scholar
  22. Olivola, C. Y., & Sagara, N. (2009). Distributions of observed death tolls govern sensitivity to human fatalities. Proceedings of the National Academy of Sciences, 106, 22151–22156.CrossRefGoogle Scholar
  23. Parducci, A. (1968). The relativism of absolute judgments. Scientific American, 219, 84–90. CrossRefGoogle Scholar
  24. Qian, T., & Aslin, R. N. (2014). Learning bundles of stimuli renders stimulus order as a cue, not a confound. Proceedings of the National Academy of Sciences, 111, 14400–14405.CrossRefGoogle Scholar
  25. Stewart, N., Brown, G. D. A., & Chater, N. (2002). Sequence effects in categorization of simple perceptual stimuli. Journal of Experimental Psychology: Learning, Memory, and Cognition, 28, 3–11. Google Scholar
  26. Stewart, N., Brown, G. D. A., & Chater, N. (2005). Absolute identification by relative judgment. Psychological Review, 112, 881–911. CrossRefGoogle Scholar
  27. Ward, L. M., & Lockhead, G. R. (1971). Response system processes in absolute judgment. Perception & Psychophysics, 9, 73–78. CrossRefGoogle Scholar
  28. Wilder, M., Jones, M., & Mozer, M. C. (2010). Sequential effects reflect parallel learning of multiple environmental regularities. In Y. Bengio, D. Schuurmans, J. Lafferty, C. K. I. Williams, & A. Culotta (Eds.), Advances in neural information processing systems 22 (pp. 2053–2061). La Jolla, CA: NIPS Foundation.Google Scholar
  29. Yu, A. J., & Cohen, J. D. (2009). Sequential effects: Superstition or rational behavior? In Advances in neural information processing systems 21 (pp. 1873–1880). La Jolla, CA: NIPS Foundation.Google Scholar
  30. Zotov, V., Jones, M. N., & Mewhort, D. J. (2011). Contrast and assimilation in categorization and exemplar production. Attention, Perception, & Psychophysics, 73, 621–639. CrossRefGoogle Scholar

Copyright information

© The Psychonomic Society, Inc. 2018

Authors and Affiliations

  • David W. Vinson
    • 1
    Email author
  • Rick Dale
    • 2
  • Michael N. Jones
    • 3
  1. 1.University of CaliforniaMercedUSA
  2. 2.University of CaliforniaLos AngelesUSA
  3. 3.Indiana UniversityBloomingtonUSA

Personalised recommendations