Abstract
This paper presents methods and challenges attendant on the use of protocol analysis to develop a model of heuristic processing applied to research ethics. Participants are exposed to ethically complex scenarios and asked to verbalize their thoughts as they formulate a requested decision. The model identifies functional parts of the decision-making task: interpretation, retrieval, judgment and editing and seeks to reliably code participant verbalizations to those tasks as well as to a set of cognitive tools generally useful in such work. Important difficulties in the reliability and external validity of measurement are evaluated and a small set of illustrative data is used in support of that discussion. Results indicate that both intuitive emotional but also more deliberative cognition is present which is consistent with work in related literatures in expertise and in neuropsychology. Finally, the theoretical and practical potential of the approach is elaborated, particularly through links to a framing in Aristotelian ethics.
Similar content being viewed by others
Notes
Important exceptions that are particularly resonant with our approach are discussed in the conclusion.
As Mumford et al. 2008 concluded, “We would not argue, of course, that the seven metacognitive strategies trained in the present effort represent the only strategies that might be trained or prove of value in ethical decision-making. … However, it is clear that ethics training in the sciences might benefit from a systematic analysis of strategies contributing to ethical decision-making and attempts to encourage application of these strategies through sensemaking-based training interventions. … The present results suggest that mental models, cases, and metacognitive reasoning strategies may represent critical underlying mechanisms shaping ethical decision-making”.
The external validity problem is that we require or at least encourage extended cognitive attention because of the presentation of the scenario as a problem requiring resolution. Absent the requirement for “solving the problem now”, latent frames and judgments may operate differently—perhaps more powerfully as they are out of reach of critical reflection.
Even when reliably applied, any theoretically derived set of concepts will capture only part of the contextualized complexity of the case (or part thereof) as Toulmin (1989, 201 and passim) and innumerable others have pointed out. Third level codes in part came from this dissatisfaction but that is, of course, only a first step toward grounded conceptual elaboration.
We want to thank an anonymous reviewer for recommending that a brief presentation of quotes in this section would significantly enhance the reader’s ability to assess the approach.
Indeed, while IF codes tend to appear early, they can come much later, after a fair amount of other cognitive work has been articulated.
The social cohesion explanation is persuasive but also teleological as functional explanations can tend to be. It is important in utilizing such approaches to push for the empirical variability in patterns of cognition, for example, to assess whether, to what extent and with what variability various system needs are indeed served. And of course the causes of variability remain a logically distinct question from whatever functional needs may or may not be met (c.f., Giddens 1977). Similarly, supposedly pervasive functional responses tend to be quite broad and consequently operate more as a limit to explanation than as an explanation generative of localized intervention (in training or institutional policy for example). For example, while imitating one’s peers is no doubt an important tendency and often socially useful, resisting authority is also widespread and useful. Likewise, equality is ingrained but so too is partiality toward closer kin and father’s may generally spend more time with sons than daughters despite an equality heuristic—there is simply a lot more going on both in the environment and in other dispositions of varying levels of universality. In sum, such high-level human dispositions are real and must be recognized but may not get us far enough in terms of desirable research ethics interventions. I think social scientists go too far when they argue that one can safely ignore these broad human limits as a harmful form of biological reductionism and we should rather remember that Wimsatt (2007) explored reductive explanation as itself a form of heuristic—one with both bias and payoff.
References
Bowie, N. E. (2009). How empirical research in human cognition does and does not affect philosophical ethics. Journal of Business Ethics, 88, 635–643.
Brock, M. E., Vert, A., Kligyte, V., Waples, E. P., Sevier, S. T., & Mumford, M. D. (2008). Mental models: An alternative evaluation of a sensemaking approach to ethics instruction. Science and Engineering Ethics, 14, 449–472.
Darwin, C. (1871). The origin of species and the descent of man (pp. 393–1000). New York: The Modern Library.
Doris, J. M. (2002). Lack of character: Personality and moral behavior. Cambridge, MA: Cambridge University Press.
Ericsson, K. A., & Charness, N. (1994). Expert performance: Its structure and acquisition. American Psychologist, 49(8), 725–747.
Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data. Cambridge, MA: MIT Press.
Freese, J., Jui-Chung, A. L., & Wade, L. D. (2003). The potential relevances of biology to social inquiry. Annual Review of Sociology, 29, 233–256.
Garrison, D. R., Cleveland-Innes, M., Koole, M., & Kappelman, J. (2006). Revisiting methodological issues in transcript analysis: Negotiated coding and reliability. The Internet and Higher Education., 9(1), 1–8.
Giddens, A. (1977). Functionalism: Après la lute. In Studies in social and political theory (pp. 96–134). New York: Basic Books.
Gigerenzer, G. (2009). Moral staisficing: Rethinking moral behavior as bounded rationality. Topics in Cognitive Science, 2, 528–554.
Gigerenzer, G., & Gaissmaier, W. (2011). Heuristic decision making. Annual Review of Psychology, 62, 451–482.
Gigerenzer, G., Todd, P. M., & the ABC Research Group. (1999). Simple heuristics that make us smart. Oxford: Oxford University Press.
Greene, J. D. (2009). The cognitive neuroscience of moral judgment. The Cognitive Neurosciences, 4, 1–48.
Gunsalus, K. C., Bruner, E. M., Burbules, N., Dash, L. D. Jr., Finkin, M., Goldberg, J., Greenough, W., Miller, G., & Pratt, M. G. (2006). The Illinois White Paper—Improving the System for protecting Human Subjects: Counteracting IRB Mission Creep. U Illinois Law and Economics Research Paper No. LE06-016.
Haidt, J. (2001). The emotional dog and its rational tail: A social intuitionist approach to moral judgment. Psychological Review, 108(4), 814–834.
Han, H. (2015). Virtue ethics, positive psychology, and a new model of science and engineering ethics education. Science and Engineering Ethics, 21, 441–460.
Han, H., Glover, G. H., & Jeong, C. (2014). Cultural influences on the neural correlate of moral decision making processes. Behavioral Brain Research, 259, 215–228.
Kahneman, D., Slovic, P., & Tversky, A. (1982). Judgment under uncertainty; heuristics and biases. Cambridge: Cambridge University Press.
Kristjansson, K. (2014). Phronesis and moral education: Treading beyond the truisms. Theory and Research in Education,. doi:10.1177/1477878514530244.
Latour, B., & Wolgar, S. (1986). Laboratory life: The construction of scientific facts. Princeton, NJ: Princeton University Press.
Moran, G. S., & Diamond, G. M. (2006). The modified cognitive constructions coding system: Reliability and validity assessments. Journal of Marital and Family Therapy, 32, 451–464.
Mumford, M. D., Connelly, S., Brown, R. P., Murphy, S. T., Hill, J. H., Antes, A. L., Waples, E. P., & Davenport, L. D. (2008). A sensemaking approach to ethics training for scientists: Preliminary evidence of training effectiveness. Ethics and Behavior, 18, 315–339. Retrieved from NIH Public Access with pagination 1–25.
Musschenga, A. W. (2009). Moral intuitions, moral expertise and moral reasoning. Journal of Philosophy of Education, 43, 597–613.
Narvaez, D., & Lapsley, D. K. (2005). The psychological foundations of everyday morality and moral expertise. In D. Lapsley & D. Power (Eds.), Character psychology and character education (pp. 140–165). Notre Dame, IN: University of Notre Dame Press.
Neilsen, F. (1994). Sociobiology and sociology. Annual Review of Sociology, 20, 267–303.
Rest, J. (1986). Moral development: Advances in research and theory. New York: Praeger.
Reynolds, S. J. (2006). A neurocognitive model of the ethical decision-making process: Implications for study and practice. Journal of Applied Psychology, 91, 737–748.
Rieskamp, J., & Hoffrage, U. (1999). When do people use simple heuristics, and how can we tell? In G. Gigerenzer, P. M. Todd & teh ABC Research Group (Eds.), Simple heuristics that make us smart (pp. 141–168). New York: Oxford University Press.
Rogerson, M. D., Gottlieb, M. C., Handelsman, M. N., Knapp, S., & Younggren, J. (2011). Nonrational processes in ethical decision making. American Psychologist, 66, 614–623.
Shanteau, J. (1992). How much information does an expert use? Is it relevant? Acta Psychologica, 81, 75–86.
Simon, H. A. (1955). A behavioral model of rational choice. Quarterly Journal of Economics, 69, 99–118.
Sonenshein, S. (2007). The role of construction, intuition, and justification in responding to ethical issues at work: The sensemaking-intuition model. Academy of Management Review, 32, 1022–1040.
Sudman, S., & Bradburn, N. M. (1982). Asking questions. San Francisco: Jossey-Bass.
Sudman, S., Bradburn, N. M., & Schwarz, N. (1996). Thinking about answers: The application of cognitive processes to survey methodology. San Francisco, CA: Jossey-Bass.
Sunstein, C. (2005). Moral heuristics. Behavioral and Brain Sciences, 28, 531–573.
Toulmin, S. (1989). Cosmopolis: The hidden agenda of modernity. Chicago: University of Chicago Press.
Tourangeau, R. (1984). Cognitive science and survey methods: A cognitive perspective. In T. Thomas, M. Straf, & R. Tourangeau Jabine (Eds.), Cognitive aspects of methodology: Building a bridge between disciplines (pp. 73–100). Washington, DC: National Academy Press.
Tversky, A. (1972). Elimination by aspects: A theory of choice. Psychological Review, 79, 281–299.
Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185, 1124–1131.
Van Valey, T., Fuqua, W., Hartmann, D., Evans, A., Staros, K., & Walmsley, C. (2014). Ethical actions: A comparison of individual and group recommendations. Sociological Focus, 47(4), 268–278.
Van Valey, T., Hartmann, D., Fuqua, W., Evans, A., Ing, A. D., Meyer, A., et al. (2015). The process of ethical decision-making: Experts vs. novices. Journal of Academic Ethics, 13, 45–60.
Weick, K. (1995). Sensemaking in organizations. Beverly Hills, CA: Sage.
Wimsatt, W. C. (2007). Re-engineering philosophy for limited beings: Piecewise approximations to reality. Cambridge, MA: Harvard University Press.
Acknowledgments
This research was funded by National Science Foundation Grant, SES-0924751, “The Nature of Ethical Decision-making in Research”.
Author information
Authors and Affiliations
Corresponding author
Appendix: Sample of Coded Protocols for “Natasha”
Appendix: Sample of Coded Protocols for “Natasha”
“Natasha is a graduate student interested in studying the roles of alcohol consumption on risk assessment and decision making in acquaintance rape situations. She proposes to recruit female college students to listen to an audio recording of an interaction between a man and woman that becomes increasingly coercive and ends with a rape. Each subject will be instructed to indicate at what point in the recording the man’s effort to persuade his female companion becomes unacceptable. In addition, each subject will be asked what action she would recommend the woman take. Prior to the actual research, all the subjects will consume two six ounce drinks. All of the participants will be instructed that their drinks contain vodka, when in fact, only half contain significant amounts. Natasha presents her proposal to her advisor. If you were Natasha’s advisor, what would you do?”
See Fig. 1 for the codes used in the table below.
1. Ethicist | 2. Regulatory official | ||
Well, first off, I think Natasha has probably been the victim of date rape | IF | Well, I think here, um, again, there’s no indication of an ethics board or IRB reviewing approval | IS |
That is my first off the top of my head thought | IH | This is a high-risk study | IF |
The second off the top of my head thought | IH | I’m assuming, um, that she’s gonna be working with students that would be of drinking age | ICz |
Is that she is going to be missing a significant proportion of her population if she is looking at universities because you cannot serve alcohol to anyone under 21 | IS | But it did not say that | IH |
And most university, uh, undergraduates, particularly, uh, freshman and sophomores, are 18, 19, and 20. So she’s only now dealing with older students who may, in fact, have a different perspective on things than younger students would | IA | And in college, you have your first—well, I don’t know where she’s— | IH |
Um, personally, I think she would have difficulty with the audio recording of an interaction between a man and a woman that becomes increasingly coercive and ends with rape | JF | It says college student, so that could be anything from 17 to 20 | IA |
I’m not sure how she’s going to make that recording herself and how she would be able to do something that was realistic by herself, which means she’d probably have to pay to have it done | IS/RS | Which could all be students that would be committing an illegal act if they were given alcohol | IF |
Which, um, again, I don’t know, seems out of the league for a graduate student | JH | And the actual investigator would be contributing to a minor’s delinquency | JF |
The, um—the additional ethical problem here in terms of human subjects issues is the fact that she’s going to be deceiving her patients—not patients, her—her participants | JF | So, I would tell her that, you know, she would need to consider the age bracket | JF |
By telling them that they’re all drinking something. It may be that some students will come into the study, uh, interested in the opportunity to drink something | JS | And, um, how she would go about that | JH |
Not highly likely in many, uh, college campuses. But I can envision some students saying yeah, that’s the drink study | JCz | That would be a critical statement | JCu |
Um, but in addition to the—the question of giving, um, under aged students alcohol | JF | Um, in listening to the recording, this persuasion of the girl to have sex | JS |
She’s got the challenge of finding a safe and environment in which to give the numbers of people whom she will give drinks to, um, drink in a safe environment where they will then be able to, um—to leave afterwards without driving home | JCz | Um, would this recording be presented in a way in which the two people were over a phone or in person? | JCz |
Um, Natasha presented her proposal to advisor and asked her questions. If you were Natasha’s advisor, what would you do? Well, all right | IS | Interviewer: I don’t know that answer. I’m sorry | |
So again, not a good question at the end | JF | Interviewee: Um, because I think that will be helpful | JF |
If I were Natasha’s advisor, we would have started with a much different set of presuppositions and not simply had me see this at the end after she’s already come up with something | JF | When you have a phone, you have a bit of a buffer | JH |
So we would have built it up from a—a smaller set of steps into something that would have been feasible | JF | Um, what it says—let me see here—increasing—it says it ends in rape | JS |
This is not my field, so I don’t know what those smaller steps would have looked like | JH | So I’m assuming that if it ends in rape, ah, they would have to be in person | JS |
But I think the question that would be important to ask is what is it that she’s actually trying to establish here? Does he have a hypothesis? | JF | Well, I don’t know. But it doesn’t say that | JH |
It doesn’t look like she does | JCu | So, if it were to end in rape, more than likely they would have to be in person for that to actually happen | JCu |
Is she trying to establish that, um, women who have had something to drink have different decision points than women who have not? | JF | Um, it doesn’t go into a lot of detail about, um, the actual relationship between the two people | IS |
It’s not clear to me | JH | It just says they kind a know each other | IS |
And I don’t know if that was part of the intent of the way the case was written or if, in fact, you all, um, wrote the case without thinking about that | E | Um, well, so that also would be an issue | IF |
Since I know that the importance of a hypothesis is different from field to field | RF | I would also question if, um, the student would be evaluating if any other potential participants have been victims of rapes or violent crimes | ICz |
Um, I don’t think this is very good research | JCu | And how they would address either screening them out or getting them additional, ah, support or help after going through listening to something to that affect | JF |
Again, I don’t—I mean, the ethical issues involved with it, again, have more to do with the fact that the research question itself was not well defined. And so the variables she’s looking at are a mess. And that’s all | ECu | Um, also, I’m not sure where the actual, um, study would take place | JCz |
But if they are consuming alcohol, ah, if they are driving home that would be a problem | JCu | ||
I would like to know how she would be transporting them or how they would transport themselves | JCu | ||
Um, I don’t think alcohol poisoning would be an issue with the amount | JF | ||
But then again, it depends on the person | JH | ||
So, that’s all I can think of on that one | No action |
Discussion of Appendix
The first protocol is from an ethicist. The pattern of codes employed is roughly describable as consistent with an alteration of interpretive work (largely framing) with attempts at judgment. Various frames are tried out and filled in a bit by concretization, some hesitation is evidenced by the use of hedges, and conclusions are reached. Explicit attention to the researcher as interlocutor is provided as well (the editing). What can only be seen in the full set of coded scenarios is whether there are distinct patterns, either by form or detail, and whether they vary by group membership.
The second protocol in “Appendix”, is similar in detail but distinguishable as to a greater alteration of interpretation and judgment and a tendency to carefully work out frames without formalizing an action. Its tone is more advisory (stressing points to consider) and less directive as to ethical standards per se. Whether such an interpretation is ultimately sustainable, much less whether it is an example of a generic pattern, are tasks for the substantive work currently under way. In any event, our interpretations are theoretical and perspectival and must be clearly distinguished from the coded protocols themselves. This second protocol happens to be from a regulatory official. Although group membership is mentioned in this discussion section, we obviously do not imply that the single scenario from each of the two groups presented cast any light on what might turn out to be group differences. That work will be advanced by the full analysis of results in process. Even then, it will be necessarily preliminary and cautious. That analysis concentrates first on simple counts of the codes used, second on their combinations into sequences and third on commonalities of sequences (what may justify the term “patterns”) within and between groups.
Rights and permissions
About this article
Cite this article
Hartmann, D.J., Van Valey, T. & Fuqua, W. Coding Ethical Decision-Making in Research. Sci Eng Ethics 23, 121–146 (2017). https://doi.org/10.1007/s11948-016-9756-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11948-016-9756-3