Behavior Research Methods

, Volume 51, Issue 4, pp 1565–1585 | Cite as

Think of the consequences: A decade of discourse about same-sex marriage

  • Babak HemmatianEmail author
  • Sabina J. Sloman
  • Uriel Cohen Priva
  • Steven A. Sloman


Approaching issues through the lens of nonnegotiable values increases the perceived intractability of debate (Baron & Spranca in Organizational Behavior and Human Decision Processes, 70, 1–16, 1997), while focusing on the concrete consequences of policies instead results in the moderation of extreme opinions (Fernbach, Rogers, Fox, & Sloman in Psychological Science, 24, 939–946, 2013) and a greater likelihood of conflict resolution (Baron & Leshner in Journal of Experimental Psychology: Applied, 6, 183–194, 2000). Using comments on the popular social media platform Reddit from January 2006 until September 2017, we showed how changes in the framing of same-sex marriage in public discourse relate to changes in public opinion. We used a topic model to show that the contributions of certain protected-values-based topics to the debate (religious arguments and freedom of opinion) increased prior to the emergence of a public consensus in support of same-sex marriage (Gallup, 2017), and declined afterward. In contrast, the discussion of certain consequentialist topics (the impact of politicians’ stance and same-sex marriage as a matter of policy) showed the opposite pattern. Our results reinforce the meaningfulness of protected values and consequentialism as relevant dimensions for describing public discourse and highlight the usefulness of unsupervised machine-learning methods in tackling questions about social attitude change.


Protected values Consequentialism Same-sex marriage Latent Dirichlet allocation Reddit 


Author note

This article greatly benefited from discussion with Robert Thorstad and members of the Sloman Lab at Brown University. We thank Elinor Amit, Linda Covington, David Sherman, Leila Sloman, Semir Tatlidil, An Vo, and Luana Pessanha de Mattos for their help with the data gathering. Earlier versions of the results included in this article were presented at the Heterodox Psychology Workshop held at Chapman University in Orange, California, USA, in August 2018, and at the 39th Annual Conference of the Society for Judgment and Decision-Making in New Orleans, Louisiana, USA, in November 2018. This publication was made possible through a grant from the Intellectual Humility in Public Discourse Project at the University of Connecticut and the John Templeton Foundation.

Supplementary material

13428_2019_1215_MOESM1_ESM.docx (730 kb)
ESM 1 (DOCX 730 kb)


  1. Atran, S., Axelrod, R., & Davis, R. (2007). Sacred barriers to conflict resolution. Science, 317, 1039–1040.CrossRefPubMedGoogle Scholar
  2. Baron, J., & Leshner, S. (2000). How serious are expressions of protected values? Journal of Experimental Psychology: Applied, 6, 183–194.PubMedGoogle Scholar
  3. Baron, J., & Spranca, M. (1997). Protected values. Organizational Behavior and Human Decision Processes, 70, 1–16.CrossRefGoogle Scholar
  4. Bird, S., Klein, E., & Loper, E. (2009). Natural language processing with Python. Sebastopol, CA: O’Reilly Media.Google Scholar
  5. Bishop, C. M. (2006). Pattern recognition and machine learning. Secaucus, NJ, USA: Springer.Google Scholar
  6. Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent Dirichlet allocation. Journal of Machine Learning Research, 3, 993–1022.Google Scholar
  7. Brewer, P. R. (2003). The shifting foundations of public opinion about gay rights. Journal of Politics, 65, 1208–1220.CrossRefGoogle Scholar
  8. Chang, J., Boyd-Graber, J., Gerrish, S., Wang, C., & Blei, D. (2009). Reading tea leaves: How humans interpret topic models. In Advances in neural information processing systems (Vol. 22, pp. 288–296). New York, NY: Curran Associates.Google Scholar
  9. Cohen Priva, U., & Austerweil, J. L. (2015). Analyzing the history of cognition using topic models. Cognition, 135, 4–9. CrossRefPubMedGoogle Scholar
  10. Dehghani, M., Iliev, R., Sachdeva, S., Atran, S., Ginges, J., & Medin, D. (2009). Emerging sacred values: Iran’s nuclear program. Judgment and Decision making, 4, 930–933.Google Scholar
  11. Duggan, M., & Smith, A. (2013). 6% of online adults are Reddit users. Pew Internet & American Life Project, 3, 1–10.Google Scholar
  12. Esmaeili, B., Huang, H., Wallace, B. C., & van de Meent, J. W. (2019). Structured representations for reviews: Aspect-based variational hidden factor models. arXiv preprint. arXiv:1812.05035Google Scholar
  13. Fellbaum, C. (1998). WordNet: An electronic lexical database. Cambridge, MA: MIT Press.CrossRefGoogle Scholar
  14. Fernbach, P. M., Min, L., & Sloman, S. A. (2018). Values-based and consequence-based policy attitudes (Working article).Google Scholar
  15. Fernbach, P. M., Rogers, T., Fox, C., & Sloman, S. A. (2013). Political extremism is supported by an illusion of understanding. Psychological Science, 24, 939–946.CrossRefPubMedGoogle Scholar
  16. Gallup. (2017). US support for gay marriage edges to new high. Retrieved from
  17. Griffiths, T. L., Steyvers, M., & Tenenbaum, J. B. (2007). Topics in semantic representation. Psychological Review, 114, 211–244. CrossRefPubMedGoogle Scholar
  18. Hardwig, J. (1985). Epistemic dependence. Journal of Philosophy, 82, 335–349.CrossRefGoogle Scholar
  19. Hoffman, M. D., Blei, D. M., & Bach, F. (2010). Online learning for Latent Dirichlet Allocation. In J. D. Lafferty, C. K. I. Williams, J. Shawe-Taylor, R. S. Zemel, & A. Culotta (Eds.), Advances in neural information processing systems (Vol. 23, pp. 856–864). New York, NY: Curran Associates.Google Scholar
  20. Kant, I. (1797). The metaphysics of morals. Cambridge, UK: Cambridge University Press.Google Scholar
  21. Lakoff, G. (2004). Don’t think of an elephant! Know your values and frame the debate. White River Junction, VT: Chelsea Green.Google Scholar
  22. Manning, C. D., Manning, C. D., & Schütze, H. (1999). Foundations of statistical natural language processing. Cambridge, MA, USA: MIT Press.Google Scholar
  23. Mimno, D., Wallach, H. M., Talley, E., Leenders, M., & McCallum, A. (2011). Optimizing semantic coherence in topic models. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (pp. 262–272). Stroudsburg, PA: Association for Computational Linguistics.Google Scholar
  24. Newport, F., & Dugan, A. (2017). Partisan differences growing on a number of issues (Editorial). Retrieved from
  25. Nugroho, R., Zhong, Y., Yang, J., Paris, C., & Nepal, S. (2015). Matrix inter-joint factorization—A new approach for topic derivation in twitter. In 2015 IEEE International Congress on Big Data (pp. 79–86). Piscataway, NJ: IEEE Press.CrossRefGoogle Scholar
  26. Pew Research Center. (2017, June). Changing attitudes on gay marriage. Retrieved from
  27. Řehůřek, R., & Sojka, P., (2010). Software framework for topic modelling with large corpora. In Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks (pp. 45–50). Retrieved from
  28. Rozenblit, L., & Keil, F. (2002). The misunderstood limits of folk science: An illusion of explanatory depth. Cognitive Science, 26, 521–562.CrossRefPubMedGoogle Scholar
  29. Sloman, S., & Fernbach, P. (2017). The knowledge illusion: Why we never think alone. New York, NY: Riverhead Books.Google Scholar
  30. Stevens, K., Kegelmeyer, P., Andrzejewski, D., & Buttler, D. (2012). Exploring topic coherence over many models and many topics. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (pp. 952–961). Stroudsburg, PA: Association for Computational Linguistics.Google Scholar
  31. Tan, C., Niculae, V., Danescu-Niculescu-Mizil, C., & Lee, L. (2016). Winning arguments: Interaction dynamics and persuasion strategies in good-faith online discussions. In Proceedings of the 25th International Conference on World Wide Web (pp. 613–624). Geneva, Switzerland: International World Wide Web Conferences Steering Committee.CrossRefGoogle Scholar
  32. Tanner, C., Medin, D. L., & Iliev, R. (2008). Influence of deontological versus consequentialist orientations on act choices and framing effects: When principles are more important than consequences. European Journal of Social Psychology, 38, 757–769.CrossRefGoogle Scholar
  33. Tetlock, P. E. (2003). Thinking the unthinkable: Sacred values and taboo cognitions. Trends in Cognitive Sciences, 7, 320–324. CrossRefPubMedGoogle Scholar
  34. Thompson, W. H. W., Wojtowicz, Z., & DeDeo, S. (2018, December). Levy flights of the collective imagination. Retrieved December 28, 2018, from
  35. Zhao, W., Chen, J. J., Perkins, R., Liu, Z., Ge, W., Ding, Y., & Zou, W. (2015). A heuristic approach to determine an appropriate number of topics in topic modeling. BMC Bioinformatics, 16, S8.CrossRefPubMedGoogle Scholar

Copyright information

© The Psychonomic Society, Inc. 2019

Authors and Affiliations

  1. 1.Department of Cognitive, Linguistic and Psychological SciencesBrown UniversityProvidenceUSA
  2. 2.Department of Social and Decision SciencesCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations