A brief assessment tool for investigating facets of moral judgment from realistic vignettes

  • Michael Kruepke
  • Erin K. Molloy
  • Konrad Bresin
  • Aron K. Barbey
  • Edelyn Verona


Humans make moral judgments every day, and research demonstrates that these evaluations are based on a host of related event features (e.g., harm, legality). In order to acquire systematic data on how moral judgments are made, our assessments need to be expanded to include real-life, ecologically valid stimuli that take into account the numerous event features that are known to influence moral judgment. To facilitate this, Knutson et al. (in Social Cognitive and Affective Neuroscience, 5(4), 378–384, 2010) developed vignettes based on real-life episodic memories rated concurrently on key moral features; however, the method is time intensive (~1.4–3.4 h) and the stimuli and ratings require further validation and characterization. The present study addresses these limitations by: (i) validating three short subsets of these vignettes (39 per subset) that are time-efficient (10–25 min per subset) yet representative of the ratings and factor structure of the full set, (ii) norming ratings of moral features in a larger sample (total N = 661, each subset N = ~220 vs. Knutson et al. N = 30), (iii) examining the generalizability of the original factor structure by replicating it in a larger sample across vignette subsets, sex, and political ideology, and (iv) using latent profile analysis to empirically characterize vignette groupings based on event feature ratings profiles and vignette content. This study therefore provides researchers with a core battery of well-characterized and realistic vignettes, concurrently rated on key moral features that can be administered in a brief, time-efficient manner to advance research on the nature of moral judgment.


Moral judgment Vignette Moral Assessment Sex 


Author Note

EKM was supported by the NSF IGERT Fellowship, Grant No. 0903622.

We would like to thank the reviewers for their comments and assistance in improving the manuscript.


  1. Arts, W., & Gelissen, J. (2001). Welfare states, solidarity and justice principles: Does the type really matter? Acta Sociologica, 44(4), 283–299.CrossRefGoogle Scholar
  2. Bentler, P. M., & Chou, C. H. (1987). Practical issues in structural modeling. Sociological Methods & Research, 16, 78–117.CrossRefGoogle Scholar
  3. Blasi, A. (1980). Bridging moral cognition and moral action: A critical review of the literature. Psychological Bulletin, 88(1), 1.CrossRefGoogle Scholar
  4. Bzdok, D., Schilbach, L., Vogeley, K., Schneider, K., Laird, A. R., Langner, R., & Eickhoff, S. B. (2012). Parsing the neural correlates of moral cognition: ALE meta-analysis on morality, theory of mind, and empathy. Brain Structure and Function, 217(4), 783–796.CrossRefPubMedPubMedCentralGoogle Scholar
  5. Carver, R. P. (1990). Reading rate: A review of research and theory. San Diego, CA: Academic Press.Google Scholar
  6. Church, A. T., & Burke, P. J. (1994). Exploratory and confirmatory tests of the Big Five and Tellegen’s three-and-four-dimensional models. Journal of Personality and Social Psychology, 66, 93–114.CrossRefPubMedGoogle Scholar
  7. Clark, S. L., Muthén, B., Kaprio, J., D’Onofrio, B. M., Viken, R., & Rose, R. J. (2013). Models and strategies for factor mixture analysis: An example concerning the structure underlying psychological disorders. Structural Equation Modeling: A Multidisciplinary Journal, 20(4), 681–703. doi: 10.1080/10705511.2013.824786 CrossRefGoogle Scholar
  8. Clifford, S., Iyengar, V., Cabeza, R., & Sinnott-Armstrong, W. (2015). Moral foundations vignettes: A standardized stimulus database of scenarios based on moral foundations theory. Behavior Research Methods, 47(4), 1178–1198.CrossRefPubMedPubMedCentralGoogle Scholar
  9. Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155–159.CrossRefPubMedGoogle Scholar
  10. Collaboration, O. S. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716–aac4716. doi: 10.1126/science.aac4716
  11. Cresswell, M., & Karimova, Z. (2010). Self-Harm and Medicine’s Moral Code: A Historical Perspective, 1950–2000. Ethical Human Psychology and Psychiatry, 12(2), 158–175.CrossRefGoogle Scholar
  12. Cumming, G. (2013). Understanding the new statistics: Effect sizes, confidence intervals, and meta-analysis. New York, NY: Routledge.Google Scholar
  13. Cushman, F., Sheketoff, R., Wharton, S., & Carey, S. (2013). The development of intent-based moral judgment. Cognition, 127(1), 6–21.Google Scholar
  14. Davis, M. H. (1983). Measuring individual differences in empathy: Evidence for a multidimensional approach. Journal of Personality and Social Psychology, 44, 113–126.CrossRefGoogle Scholar
  15. Escobedo, J. R. (2009). Investigating Moral Events: Characterization and Structure of Autobiographical Moral Memories. Unpublished Dissertation. Pasadena, California: California Institute of Technology.Google Scholar
  16. FeldmanHall, O., Mobbs, D., Evans, D., Hiscox, L., Navrady, L., & Dalgleish, T. (2012). What we say and what we do: The relationship between real and hypothetical moral choices. Cognition, 123(3), 434–441.CrossRefPubMedPubMedCentralGoogle Scholar
  17. Ferrando, P. J., & Lorenzo, U. (2000). Unrestricted versus restricted factor analysis of multidimensional test items: Some aspects of the problem and some suggestions. Psicológica, 21, 301–323.Google Scholar
  18. Flesch, R. (1948). A new readability yardstick. Journal of Applied Psychology, 32, 221–233.CrossRefPubMedGoogle Scholar
  19. Galen, L. W. (2012). Does religious belief promote prosociality? A critical examination. Psychological Bulletin, 138(5), 876.CrossRefPubMedGoogle Scholar
  20. Gilligan, C. (1982). In a different voice. Harvard University Press.Google Scholar
  21. Gold, N., Pulford, B. D., & Colman, A. M. (2014). The outlandish, the realistic, and the real: Contextual manipulation and agent role effects in trolley problems. Frontiers in Psychology 5, 35. doi: 10.3389/fpsyg.2014.00035
  22. Graham, J., & Haidt, J. (2010). Beyond beliefs: Religions bind individuals into moral communities. Personality and Social Psychology Review, 14(1), 140–150.CrossRefPubMedGoogle Scholar
  23. Graham, J., Haidt, J., & Nosek, B. A. (2009). Liberals and conservatives rely on different sets of moral foundations. Journal of Personality and Social Psychology, 96(5), 1029–1046.CrossRefPubMedGoogle Scholar
  24. Graham, J., Nosek, B. A., Haidt, J., Iyer, R., Koleva, S., & Ditto, P. H. (2011). Mapping the moral domain. Journal of Personality and Social Psychology, 101(2), 366.Google Scholar
  25. Greenberg, E., Dunleavy, E., and Kutner, M. (2007). Literacy Behind Bars: Results From the 2003 National Assessment of Adult Literacy Prison Survey (NCES 2007-473). U.S. Department of Education. Washington, DC: National Center for Education StatisticsGoogle Scholar
  26. Greene, J. D., Cushman, F. a., Stewart, L. E., Lowenberg, K., Nystrom, L. E., & Cohen, J. D. (2009). Pushing moral buttons: The interaction between personal force and intention in moral judgment. Cognition, 111(3), 364–371. doi: 10.1016/j.cognition.2009.02.001 CrossRefPubMedGoogle Scholar
  27. Greene, J., & J, H. (2002). How (and where) does moral judgement work? Trends in Cognitive Sciences, 6(12), 517–523.CrossRefGoogle Scholar
  28. Greene, J. D., Sommerville, R. B., Nystrom, L. E., Darley, J. M., & Cohen, J. D. (2001). An fMRI investigation of emotional engagement in moral judgment. Science (New York, N.Y.), 293(5537), 2105–2108. doi: 10.1126/science.1062872
  29. Haidt, J. (2001) The emotional dog and its rational tail: A social intuitionist approach to moral judgment. Psychological Review, 108(4), 814–834.Google Scholar
  30. Haidt, J. (2007). The new synthesis in moral psychology. Science, 316, 998–1002.CrossRefPubMedGoogle Scholar
  31. Herzog, A. R., & Bachman, J. G. (1981). Effects of questionnaire length on response quality. The Public Opinion Quarterly, 45(4), 549–559. doi: 10.1086/268687
  32. Huang, J. L., Curran, P. G., Keeney, J., Poposki, E. M., & DeShon, R. P. (2012). Detecting and deterring insufficient effort responding to surveys. Journal of Business and Psychology, 27(1), 99–114. doi: 10.1007/s10869-011-9231-8 CrossRefGoogle Scholar
  33. Ince, R. A. A., Giordano, B. L., Kayser, C., Rousselet, G. A., Gross, J., & Schyns, P. G. (2016). A statistical framework for neuroimaging data analysis based on mutual information estimated via a Gaussian copula. Human Brain Mapping. doi: 10.1002/hbm.23471 PubMedPubMedCentralGoogle Scholar
  34. Jaffee, S., & Hyde, J. S. (2000). Gender differences in moral orientation: A meta-analysis. Psychological Bulletin, 126(5), 703–726. doi: 10.1037/0033-2909.126.5.703 CrossRefPubMedGoogle Scholar
  35. Kahane, G. (2015). Sidetracked by trolleys: Why sacrificial moral dilemmas tell us little (or nothing) about utilitarian judgment. Social Neuroscience, 00(00), 1–10.Google Scholar
  36. Kaiser, H. F. (1958). The varimax criterion for analytic rotation in factor analysis. Psychometrika, 23(3), 187–200.CrossRefGoogle Scholar
  37. Kapogiannis, D., Barbey, A. K., Su, M., Zamboni, G., Krueger, F., & Grafman, J. (2009). Cognitive and neural foundations of religious belief. Proceedings of the National Academy of Sciences, 106(12), 4876–4881.CrossRefGoogle Scholar
  38. Knutson, K. M., Krueger, F., Koenigs, M., Hawley, A., Escobedo, J. R., Vasudeva, V., Adolphs, R., Grafman, J. (2010). Behavioral norms for condensed moral vignettes. Social Cognitive and Affective Neuroscience, 5(4), 378–384. doi: 10.1093/scan/nsq005 CrossRefPubMedPubMedCentralGoogle Scholar
  39. Koster-Hale, J., Saxe, R., Dungan, J., & Young, L. L. (2013). Decoding moral judgments from neural representations of intentions. Proceedings of the National Academy of Sciences of the United States of America, 110(14), 5648–53. doi: 10.1073/pnas.120799211 CrossRefPubMedPubMedCentralGoogle Scholar
  40. Kotsiantis, S., & Kanellopoulos, D. (2006). Discretization Techniques: A recent survey. GESTS International Transactions on Computer Science and Engineering, 32(1), 47–58.Google Scholar
  41. Lorenzo-Seva, U., & Ten Berge, J. M. (2006). Tucker's congruence coefficient as a meaningful index of factor similarity. Methodology, 2(2), 57–64.CrossRefGoogle Scholar
  42. MATLAB (2014). Natick, Massachusetts. The MathWorks Inc.Google Scholar
  43. McCrae, R. R., Zonderman, A. B., Costa, P. T., Bond, M. H., & Pauonen, S. V. (1996). Evaluating replicability of factors in the revised NEO personality inventory: Confirmatory factor analysis versus Procrustes rotation. Journal of Personality and Social Psychology, 70, 552–566.CrossRefGoogle Scholar
  44. Moll, J., de Oliveira-Souza, R., Bramati, I. E., & Grafman, J. (2002). Functional networks in emotional moral and nonmoral social judgments. Neuro Image, 16(3 Pt 1), 696–703.PubMedGoogle Scholar
  45. Moll, J., de Oliveira-Souza, R., Eslinger, P. J., Bramati, I. E., Mourão-Miranda, J., Andreiuolo, P. A., & Pessoa, L. (2002). The neural correlates of moral sensitivity: A functional magnetic resonance imaging investigation of basic and moral emotions. The Journal of Neuroscience: The Official Journal of the Society for Neuroscience, 22(7), 2730–2736.Google Scholar
  46. Molloy, E. K. and Kruepke, M. D., (2017). Selecting representative subsets of vignettes for investigating multiple facets of moral judgement: Documentation and MATLAB Code. GitHub repository, https://github.com/ekmolloy/select-vignette-subsets
  47. Muthén, L. K., & Muthén, B. O. (2008). Mplus (Version 5.1). Los Angeles, CA: Muthén & Muthén.Google Scholar
  48. Nylund, K. L., Asparouhov, T., & Muthén, B. O. (2007). Deciding on the number of classes in latent class analysis and growth mixture modeling: A Monte Carlo simulation study. Structural Equation Modeling: A Multidisciplinary Journal, 14(4), 535–569.CrossRefGoogle Scholar
  49. Palan, K. M., Areni, C. S., & Kiecker, P. (1999). Reexamining masculinity, femininity, and gender identity scales. Marketing Letters, 10(4), 357–371.CrossRefGoogle Scholar
  50. Phillips, J., Luguri, J. B., & Knobe, J. (2015). Unifying morality’s influence on non-moral judgments: The relevance of alternative possibilities. Cognition, 145, 30–42. doi: 10.1016/j.cognition.2015.08.001 CrossRefPubMedGoogle Scholar
  51. R Core Team (2014). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0, URL http://www.R-project.org/
  52. Revelle, W. (2015) psych: Procedures for Personality and Psychological Research, Northwestern University, Evanston, Illinois, USA, https://CRAN.R-project.org/package=psych Version = 1.5.8.
  53. Reynolds, S. J., & Ceranic, T. L. (2007). The effects of moral judgment and moral identity on moral behavior: An empirical examination of the moral individual. The Journal of Applied Psychology, 92(6), 1610–1624. doi: 10.1037/0021-9010.92.6.1610 CrossRefPubMedGoogle Scholar
  54. Rusbult, C. E., & Van Lange, P. A. M. (1996). Interdependence processes. In E. T. Higgins & A. W. Kruglanski (Eds.), Social psychology: Handbook of basic principles (pp. 564–596). New York, NY: Guilford Press.Google Scholar
  55. Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6(2), 461–464.CrossRefGoogle Scholar
  56. Shavelson, R. J., Webb, N. M., & Rowley, G. L. (1989). Generalizability theory. American Psychologist, 44(6), 922.CrossRefGoogle Scholar
  57. Shweder, R., Much, N., Mahapatra, M., & Park, L. (1997). Divinity and the “Big Three” Explanations of Suffering. Morality and Health, 119.Google Scholar
  58. Simpson, A., & Laham, S. M. (2015). Individual differences in relational construal are associated with variability in moral judgment. Personality and Individual Differences, 74, 49–54.CrossRefGoogle Scholar
  59. Strohminger, N., & Nichols, S. (2014). The essential moral self. Cognition, 131(1), 159–171. doi: 10.1016/j.cognition.2013.12.005 CrossRefPubMedGoogle Scholar
  60. Thoma, S. (1994). Moral judgments and moral action. Moral development in the professions: Psychology and applied ethics, 199–21.Google Scholar
  61. Ugazio, G., Lamm, C., & Singer, T. (2012). The role of emotions for moral judgments depends on the type of emotion and moral scenario. Emotion, 12(3), 579.CrossRefPubMedGoogle Scholar
  62. Vranka, M. A., & Bahník, Š. (2016). Is the Emotional Dog Blind to Its Choices? Experimental Psychology, 63(3), 180–188.CrossRefPubMedGoogle Scholar
  63. Young, L., Bechara, A., Tranel, D., Damasio, H., Hauser, M., & Damasio, A. (2010). Damage to ventromedial prefrontal cortex impairs judgment of harmful intent. Neuron, 65(6), 845–851.Google Scholar

Copyright information

© Psychonomic Society, Inc. 2017

Authors and Affiliations

  • Michael Kruepke
    • 1
  • Erin K. Molloy
    • 2
  • Konrad Bresin
    • 1
  • Aron K. Barbey
    • 3
  • Edelyn Verona
    • 4
  1. 1.Psychology DepartmentUniversity of Illinois at Urbana-Champaign, Urbana-ChampaignChampaignUSA
  2. 2.Computer Science DepartmentUniversity of Illinois at Urbana-ChampaignUrbana-ChampaignUSA
  3. 3.Beckman Institute for Advanced Science and TechnologyUniversity of Illinois at Urbana-ChampaignUrbana-ChampaignUSA
  4. 4.Psychology DepartmentUniversity of South FloridaTampaUSA

Personalised recommendations