Decision contamination in the wild: Sequential dependencies in online review ratings
Current judgments are systematically biased by prior judgments. Such biases occur in ways that seem to reflect the cognitive system’s ability to adapt to statistical regularities within the environment. These cognitive sequential dependencies have primarily been evaluated in carefully controlled laboratory experiments. In this study, we used these well-known laboratory findings to guide our analysis of two datasets, consisting of over 2.2 million business review ratings from Yelp and 4.2 million movie and television review ratings from Amazon. We explored how within-reviewer ratings are influenced by previous ratings. Our findings suggest a contrast effect: Current ratings are systematically biased away from prior ratings, and the magnitude of this bias decays over several reviews. This work is couched within a broader program that aims to use well-established laboratory findings to guide our understanding of patterns in naturally occurring and large-scale behavioral data.
KeywordsSequential dependence Decision making Data mining Online reviews Big data Cognitive principles
This work was funded by NSF BCS-1056744 to M.N.J. D.W.V. was supported by an IBM PhD fellowship.
- Donkin, C., Rae, B., Heathcote, A., & Brown, S. D. (2015). Why is accurately labelling simple magnitudes so hard? A past, present and future look at simple perceptual judgment. In J. R. Busemeyer, Z. Wang, J. T. Townsend, & A. Eidels (Eds.), Oxford handbook of computational and mathematical psychology (pp. 121–141). Oxford, UK: Oxford University Press.Google Scholar
- Jones, M. N. (2017). Big data in cognitive science. New York, NY: Taylor & Francis.Google Scholar
- Luca, M. (2011, September 16). Reviews, reputation, and revenue: The case of Yelp.com (Harvard Business School NOM Unit Working Paper, 12-016). Cambridge, MA: Harvard University, School of Business.
- Mozer, M. C., Pashler, H., Wilder, M., Lindsey, R. V., Jones, M. C., & Jones, M. N. (2010). Decontaminating human judgments by removing sequential dependencies. In J. Laffterty, C. K. I. Williams, J. Shawe-Taylor, R. S. Zemel, & A. Culota (Eds.), Advances in neural information processing systems 23 (pp. 1705–1713). La Jolla, CA: NIPS Foundation.Google Scholar
- Wilder, M., Jones, M., & Mozer, M. C. (2010). Sequential effects reflect parallel learning of multiple environmental regularities. In Y. Bengio, D. Schuurmans, J. Lafferty, C. K. I. Williams, & A. Culotta (Eds.), Advances in neural information processing systems 22 (pp. 2053–2061). La Jolla, CA: NIPS Foundation.Google Scholar
- Yu, A. J., & Cohen, J. D. (2009). Sequential effects: Superstition or rational behavior? In Advances in neural information processing systems 21 (pp. 1873–1880). La Jolla, CA: NIPS Foundation.Google Scholar