Abstract
As colleges and universities seek to invigorate ethics education, they need methods to identify where and describe how ethics is already present across their curricula. Meeting this need is complicated by the fact that much ethics education occurs in courses not explicitly focused on ethics or morality. In this paper, we review recent methodological advances before presenting a new Ethics Course Identification Tool (ECIT) that combines application of an expert-derived weighted dictionary and natural language processing methods to identify ethics-related courses based on their titles and course catalog descriptions, even when the terms “ethic” or “moral” are not present. Two studies, the second a pre-registered replication, revealed considerable interrater reliability among experts in ethics education regarding the ethical relevance of courses. Critically, both studies revealed strong correlations between expert judgments and ECIT scores. This empirical evidence points to a shared understanding of ethics education among experts, and it supports the valid use of the ECIT to rapidly and reliably identify ethics-related courses. Based on these findings, we propose that the ECIT can be used both to advance research on trends in ethics education and to help target interventions to improve ethics education at colleges and universities.
This is a preview of subscription content, access via your institution.

Data Availability
Link to OSF data: https://osf.io/25jvd/?view_only=f26d1844bc924fa08de18b7ebb1e45e7
References
American Association of Colleges and Universities. (2022, August 8). Essential learning outcomes. AAC&U. Retrieved September 19, 2022, from https://www.aacu.org/trending-topics/essential-learning-outcomes
Araque, O., Gatti, L., & Kalimeri, K. (2020). MoralStrength: Exploiting a moral lexicon and embedding similarity for moral foundations prediction. Knowledge-based Systems, 191, 105184.
Bankhead, M., Choi, Y. J., Patil, Y., & Thoma, S. J. (2022). Individual and School Correlates of DIT-2 Scores Using a Multilevel Modeling and Data Mining Analysis. Applied Sciences, 12(9), 4573.
Barnwell, D. (1989). ‘Naïve’native speakers and judgements of oral proficiency in Spanish. Language Testing, 6(2), 152–163.
Bebeau, M. J., & Monson, V. E. (2008). Guided by theory, grounded in evidence: A way forward for professional ethics education. In Handbook of Moral and Character Education (pp. 573-598). Routledge.
Beever, J., Kuebler, S. M., & Collins, J. (2021). Where ethics is taught: An institutional epidemiology. International Journal of Ethics Education, 6(2), 215–238.
Berry, J. W. (1989). Imposed etics—emics—derived etics: The operationalization of a compelling idea. International Journal of Psychology, 24(6), 721–735.
Boyd, R. L., & Schwartz, H. A. (2021). Natural language analysis and the psychology of verbal behavior: The past, present, and future states of the field. Journal of Language and Social Psychology, 40(1), 21–41.
Christensen, L. J., Peirce, E., Hartman, L. P., Hoffman, W. M., & Carrier, J. (2007). Ethics, CSR, and sustainability education in the Financial Times top 50 global business schools: Baseline data and future research directions. Journal of Business Ethics, 73(4), 347–368.
Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6(4), 284–290.
Colby, A., Ehrlich, T., Beaumont E., & Stephens, J. (2003). Educating citizens: Preparing America’s undergraduates for lives of moral responsibility. JosseyBass.
Drechsler Sharp, M., Komives, S. R., & Fincher, J. (2011). Learning outcomes in academic disciplines: Identifying common ground. Journal of Student Affairs Research and Practice, 48(4), 481–504.
Elliott, D. (2007). Ethics in the First Person: A Guide to Teaching and Learning Practical Ethics. Rowman & Littlefield.
Elliott, D., & June, K. (2018). The Evolution of Ethics Education 1980–2015. In Ethics Across the Curriculum—Pedagogical Perspectives (pp. 11-37). Springer, Cham.
Finelli, C. J., Holsapple, M. A., Ra, E., Bielby, R. M., Burt, B. A., Carpenter, D. D., Harding, T. S., & Sutkus, J. A. (2012). An assessment of engineering students’ curricular and co-curricular experiences and their ethical development. Journal of Engineering Education, 101(3), 469–494.
Fischman, W., & Gardner, H. (2022). The real world of college: What higher education is and what it can be. MIT Press.
Garten, J., Hoover, J., Johnson, K. M., Boghrati, R., Iskiwitch, C., & Dehghani, M. (2018). Dictionaries and distributions: Combining expert knowledge and large scale textual data content analysis. Behavior Research Methods, 50(1), 344–361.
Glanzer, P. L., & Ream, T. C. (2008). Educating different types of citizens: Identity, tradition, and moral education. Journal of College and Character, 9(4), 1–14.
Griffith, S. M., Domenech Rodriguez, M. M., & Anderson, A. J. (2014). Graduate ethics education: A content analysis of syllabi. Training and Education in Professional Psychology, 8(4), 248–252.
Hamer, R. M. (1991). A SAS macro for computing intraclass correlation coefficients. Virginia Commonwealth University.
Hoover, J., Johnson, K., Boghrati, R., Graham, J., & Dehghani, M. (2018). Moral framing and charitable donation: Integrating exploratory social media analyses and confirmatory experimentation. Collabra: Psychology, 4(1).
Hoover, J., Portillo-Wightman, G., Yeh, L., Havaldar, S., Davani, A. M., Lin, Y., ... & Dehghani, M. (2020). Moral foundations twitter corpus: A collection of 35k tweets annotated for moral sentiment. Social Psychological and Personality Science, 11(8), 1057-1071.
Jarvis, S. (2017). Grounding lexical diversity in human judgments. Language Testing, 34(4), 537–553.
Kidd, D., Miner, J., Schein, M., Blauw, M., & Allen, D. (2020). Ethics across the curriculum: Detecting and describing emergent trends in ethics education. Studies in Educational Evaluation, 67, 100914.
Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163.
Lambeth, E. B., Christians, C. G., Fleming, K., & Lee, S. T. (2004). Media ethics teaching in century 21: Progress, problems, and challenges. Journalism & Mass Communication Educator, 59(3), 239–358.
Matchett, N. J. (2008). Ethics across the curriculum. New Directions for Higher Education, 142, 25–38.
May, D. R., Luth, M. T., & Schwoerer, C. E. (2014). The influence of business ethics education on moral efficacy, moral meaningfulness, and moral courage: A quasi-experimental study. Journal of Business Ethics, 124(1), 67–80.
Mayhew, M. J., Rockenbach, A. N., Bowman, N. A., Seifert, T. A., & Wolniak, G. C. (2016). How college affects students: 21st century evidence that higher education works (Vol. 3). John Wiley & Sons.
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. In Yoshua Bengio et al., editors, 1st International Conference on Learning Representations, ICLR 2013, Scottsdale, Arizona, USA, May 2- 4, 2013, Workshop Track Proceedings.
Mills, A., Sanders, A. K., & Hussain, S. S. (2019). Fitting it all in? A census of undergraduate ethics and leadership courses in accredited US journalism and mass communication programs. Journalism & Mass Communication Educator, 74(3), 265–275.
Neill, M. S. (2017). Ethics education in public relations: Differences between stand-alone ethics courses and an integrated approach. Journal of Media Ethics, 32(2), 118–131.
Nosek, B. A., & Lindsay, D. S. (2018). Preregistration becoming the norm in psychological science. Observer, 31(3), 19–21.
Pike, K. L. (2015). Language in relation to a unified theory of the structure of human behavior (Vol. 24). Walter de Gruyter GmbH & co KG.
Sahatjian, Z., MacDougall, A. E., & McIntosh, T. (2022). Benefits beyond service: The facilitative effects of service-learning pedagogy on nuanced reflection and ethical sensemaking. Academy of Management Learning & Education, 21(1), 7–34.
Schouten, G. (2022). The case for egalitarian consciousness raising in higher education. Philosophical Studies, 1-24.
Triandis, H. C., Malpass, R. S., & Davidson, A. R. (1973). Psychology and culture. Annual Review of Psychology, 24(1), 355–378.
Van’t Veer, A. E., & Giner-Sorolla, R. (2016). Pre-registration in social psychology—A discussion and suggested template. Journal of Experimental Social Psychology, 67, 2–12.
Wang, L. C., & Calvano, L. (2015). Is business ethics education effective? An analysis of gender, personal ethical perspectives, and moral judgment. Journal of Business Ethics, 126(4), 591–602.
Weston, A. (2006). A Practical Companion to Ethics (3rd ed.). Oxford University Press.
You, D., Ruiz, A., & Warchal, J. (2018). The presentation of ethics in undergraduate psychology syllabi: A review. Teaching of Psychology, 45(4), 346–350.
Zou, G. Y. (2012). Sample size formulas for estimating intraclass correlation coefficients with precision and assurance. Statistics in Medicine, 31(29), 3972–3981.
Acknowledgements
We are grateful to Ka Ya Lee and Drew Chambers for their careful review of the manuscript and for their valuable insight. We also extend our thanks to Cara Biasucci and Deni Elliott for assistance in recruiting participants. Their contributions were invaluable to the success of this project. This work was generously supported by the Edmond J. Safra Philanthropic Foundation and Eugene P. Beard.
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Conflict of Interests
We have no conflicts of interest to disclose.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Ongis, M., Kidd, D. & Miner, J. Ethics Incognito: Detecting Ethically Relevant Courses Across Curricula in Higher Education. J Acad Ethics (2023). https://doi.org/10.1007/s10805-023-09482-8
Accepted:
Published:
DOI: https://doi.org/10.1007/s10805-023-09482-8