1 Introduction

Public and academic discourse about the ethics of artificial intelligence (AI), machine learning (ML), and data science (DS) has to date primarily focused on algorithms and the companies deploying them. This discourse typically centers on harms arising from algorithms used in hiring decisions, mortgage approvals, and prison sentences [1,2,3]. These harms also extend to misinformation and psychological distress associated with social media use [4]. In response, organizations and governments across the globe have issued ethics codes meant to guide more responsible development of automated systems [5, 6].

Yet, evidence for the effectiveness of these guidelines is slim. McNamara et al. [7], for example, found that ethics guidelines have had no effect on the behavior of technology professionals. Meanwhile, Mittelstadt [8] has argued that principles-based ethics codes are too vague to be practical and they treat complex normative questions as if they have technological answers. More recently, Munn [9] issued a strong criticism of prevailing AI ethics guidance, calling principles-based ethics codes “meaningless” and “toothless”.

There may be several reasons why attempts to bring ethics principles into AI development have failed. Two are relevant for this paper. First, developers of AI/ML/DS systems cannot simply agree to ethics principles in concept. They must know how to apply them in practice, and in practice developers are faced with questions more complex than abstract principles can address. Second, the conversation about AI/ML/DS ethics is largely happening external to the developer community. What is lacking is a robust conversation with developers about the ethics of being a developer. To be clear, there have been studies that engage directly with developers, including those conducted by Veale et al. [10], Holstein et al. [11], and Sanderson et al. [12]. But the goal of these studies was how to achieve more ethical algorithms, not necessarily more ethical developers. The research presented in this paper is the first known effort at intentionally engaging with the developer community about their own ethical agency and how they perceive the ethics of their occupation.

The paper proceeds in five parts. We begin by describing our study design, including the novel conceptual model we developed for situating developer ethical agency and the methodology we employed to recruit and interview developers. The second section details the results of these interviews, organized into three themes: ethics in the occupational ecosystem, ethical agency, and the characteristics of an ethical developer. In the third section, we discuss the implications of these findings. The fourth section will cover the limitations of this research, and the final section will make recommendations for engaging more fully with developers about ethics in the discipline, including how to surface the wisdom that exists in this community.

1.1 Study design and methodology

By interviewing developers across industries and geographies, we seek simply to describe how developers conceive of their ethical agency. We applied a pragmatist’s approach to this research in that we did not assert a precise definition of “moral” or “ethical.” It is important in this phase to understand how developers see themselves, and whether they believe they have enough agency to do the “right thing,” whatever that might look like to them.

1.2 Conceptual model of possible ethics sources

In Fig. 1, we present our novel conceptual model for the potential sources of ethics in the developer community. The sources of ethics in the model are summarized below. We must emphasize that these sources are not mutually exclusive and can overlap and reinforce one another. The model simply hypothesizes a spectrum of sources developers might draw from (in any combination) as they make decisions about AI/ML/DS systems.

Fig. 1
figure 1

Conceptual model of possible ethics sources

Full size image

Briefly, the potential sources of ethics in the model include the technical, in which developers might claim that AI/ML/DS practice is (or should be) guided only by technical rules, not moral ones. Their ethics might also derive from the individual developers’ own values and virtues. Another potential source for ethics is the organizing principle native to the industry in which the developer works. For example, a developer might look to the overarching aim of health if working in healthcare or to the value of equal protection under the law if working in criminal justice. Finally, a source of ethical guidance may be the norms and rules of the occupation itself, such as open source or the oft quoted ideal to “move fast and break things.”

1.3 Recruitment and interview structure

Recruitment was limited to developers actively involved in the design, development, and maintenance of AI/ML/DS systems. Potential participants were identified via LinkedIn using the key words: artificial intelligence, machine learning, and/or data science. Each potential candidate’s profile was then closely reviewed to confirm they were indeed developing AI/ML/DS systems (as opposed to working in business development or marketing). This meant looking for experience with natural language processing, recommender systems, search/information retrieval, vision/multimodal sensing, chatbots, conversational AI, adaptive tutoring and mentoring, decision support, robotics, predictive analytics, deep learning, or artificial neural networks. The initial set of potential participants were contacted via LinkedIn messaging or a personal email address if one was publicly available. At the end of each interview participants were asked if they could recommend other practitioners for this research. The study thus also included snowball recruitment.

Selection criteria aimed to achieve a balance in gender, ethnicity, industry, and geography. The decision to recruit worldwide reflects the reality of how practitioners in this space work. Global demand for AI/ML/DS expertise often means practitioners are recruited across borders. Such recruitment is facilitated by practitioners’ shared knowledge of technical languages (Python, R, and Java are in common use worldwide), statistical methods (linear or logistic regression, for example), and modeling tools (like TensorFlow or PyTorch). Developers also engage in borderless interactions via sites like GitHub, Stack Overflow, Hugging Face, and Kaggle to share and review code, answer one another’s questions, and compete to develop more efficient systems.

As noted in previous research by Veale et al. [10] and Holstein et al. [11], this population can be hesitant to participate in research about ethics in their field out of fear of negative media attention or reprisal from an employer. We thus took great care to avoid exposing participants to reputational or employment risks. In addition to guaranteeing anonymity, participants were asked to speak only for themselves (not on behalf of, or as a representative of, any employer, past or present). We did not ask about proprietary matters and further requested that participants not discuss specific projects, deployments, or companies. All text-based communication was conducted through LinkedIn, personal email, or encryption-based messaging. Unless the participant asserted otherwise, interviews were scheduled during non-work hours to avoid any association with their official employment duties. These protocols were intended to reduce as much as possible the risk of reprisal from an employer.

The semi-structured interview guide was designed to give participants broad space to expound on their experiences and beliefs. We were mindful to avoid designing questions that would result in overfitting responses into the sources in our conceptual model. As such, the interview guide focused quite generally on occupational ethics awareness, duties, and virtues. The conceptual model thus only established semantic cues to listen for during the interviews. Not all questions were asked to each participant and depending on the actual (and unexpected) ideas provided by the participant, additional follow-up questions were asked.

All interviews were conducted by a single researcher (Griffin) between February 2022 and October 2022. Unless the participant declined, interviews were recorded for the sole purpose of obtaining an accurate transcript. For the two instances in which participants declined to be recorded, answers were logged manually. Transcripts were anonymized and saved under a random number plus a generic job description (e.g., 7534_MachineLearning). Within 24–48 h after an interview, the transcript was anonymized and the original recording was permanently deleted. To further assure participants that they would remain anonymous, we also guaranteed that the full transcripts would not be made publicly available. The study received ethics approval from the university’s research ethics committee.

2 Findings

A total of forty (40) developers were interviewed for this study. Of these, ten (10) were recruited via four separate snowball referrals. The final number reflects a data saturation point wherein no new insights or themes arose after thirty-five (35) interviews. Table 1 provides a demographic summary of the participants. Because participants were guaranteed anonymity, no further demographic detail will be provided.

Table 1 Participant demographics
Full size table

Transcript analysis revealed more than twenty (20) themes, of which the three (3) most prevalent are described in this paper: ethics in the occupational ecosystem, ethical agency, and the characteristics of an ethical developer.

2.1 Ethics in the occupational ecosystem

The interviews revealed an ethical ecosystem characterized by three overlapping features: the personal aims and intentions of developers, the ethics of the occupation, and the ethical influence of the tools they use. These characteristics represent a different, but partially overlapping set than the four we hypothesized in our conceptual model (see Fig. 1). We will take up the conceptual model again in the discussion section.

2.1.1 Personal aims and intentions

Developers see themselves and (most of) their occupational cohort as well intentioned. They spoke of having individual goals like helping people, making the world a better place, advancing science, or having a positive impact on the industry in which they work. A few had more nuanced personal aims, which they adopted after gaining more experience in the field. For example, participants 2182_MachineLearning and 7433_DeepLearning said they wanted to “curb the hype” and “limit harm,” respectively.

Participants universally acknowledged the harms of some AI/ML/DS deployments, but they asserted that the cause was a lack of knowledge or experience, not intentional malfeasance. For example:

  • “It’s important to not ascribe the intent of they’re trying to be bad and exclude people.”—4807_ArtificialIntelligence

  • “We don’t consider ourselves to be nefarious agents. It’s not that we have the pinnacle of high ethical standards, but we are also not evil geniuses sitting in our lair trying to figure out ways to hurt people.”—2353_MachineLearning

  • “I’ve never met anyone nefarious in my work. It’s just, people have different experiences and different tendencies when it comes to how they think the models will behave in the real world.”—7671_ArtificialIntelligence

Notably, in all these cases, participants spoke in terms of what (or who) they were not rather than what (or who) they were.

As optimistic as developers are about their personal intentions, they also recognize that the incentives in the marketplace can be at odds with those intentions. The allure of high salaries and enticements to indulge in personal brand management offset the otherwise favorable opinions practitioners have about their own intentions. For example:

  • “I think most people are in here for the money data scientists can make over other professions.”—2032_DataScience

  • “You hear about people bouncing between companies [every two to four years] so they can get more compensation.”—4807_ArtificialIntelligence

  • “People have become so obsessed about their personal brand…to a point where it becomes harder to review their code, because the moment you mention something, they feel you are attacking them personally.”—6862_MachineLearning

2.1.2 Occupational morality and organizing principle

Almost half of participants (17) claimed that the occupation of AI/ML/DS development was neither ethical nor unethical. This neutrality did not arise from anything intrinsic to the practice, however. As illustrated in the following verbatims, the neutrality arose by virtue of the field being new and therefore unexamined:

  • “There’s nothing ethical or non-ethical about this profession. There is a hype and a marketing around these jobs, and you have to sell these technologies…I don’t necessarily think it’s a matter of ethics. It’s just not a requirement to think about these things when there’s all this excitement.”—2182_MachineLearning

  • “I don’t think there’s a general ethical question even. We don’t think of ourselves as unethical, but only because we’re not wondering if what we’re doing is ethical or not.”—7015_ArtificialIntelligence

  • “I think the field is too young to have a determination if it is ethical or not yet. It’s a field that got big incredibly fast.”—5119_Robotics

As was the case with developer intentions, participants also acknowledged the occupation-wide effects of misaligned incentives. “AI and machine learning has traditionally been people like me who are just sort of dreamers coming in,” said participant 5119_Robotics, “but now there’s the money aspect and you’re starting to get the people who just want to get paid.”

When asked if there was an organizing principle or a higher good toward which the occupation aims (in the same way that “health” is the ostensible aim of medical practitioners), participants gave a wide range of answers. These answers included time on platform, efficiency, optimization, and, more cynically, making money. Although there was no clear consensus, the most common answer was that the aim of practitioners would change based on the industry in which they worked (n = 10). If developers were working in healthcare, they would be aiming for health. If they changed jobs to work in a law firm, their aim would shift to justice. If they worked for a weapons manufacturer, their aim might simply be accuracy.

Notably, one participant, 4807_ArtificialIntelligence, did articulate a vision for the occupation, saying, “The ethos of the whole field has to shift toward: Our field exists to help people. Our job…is to organize information and understand flows of information logic in a way that makes the world better for people.” This participant went on to say, “If you start framing it more in that way, and less like, ‘You write that code, you make that money,’ then maybe you’ll start seeing more ethical behavior intrinsically come out of it.”

2.1.3 Tool neutrality

While the interview guide did not include questions about the morality of the technology that developers use and deploy, enough participants made statements supporting the value-neutrality thesis that we include it here. The value-neutrality thesis posits that technology is neither good nor bad on its own; it is morally neutral [13]. Moreover, because many of the developers interviewed for this project perceive these technologies as neutral, or as “just tools,” the onus for ethics was situated outside the occupation. Ethics was instead located in the use case, and with the users:

  • “The algorithm ultimately is a tool that is optimizing for something. So it cannot be that the algorithm is fair or unfair. It’s about the data that’s going to come in and the use case.”—8511_DataScience

  • “It’s just a tool and it comes with a lot of limitations. It’s a bit binary at the moment. Either you have full trust, or you have zero trust. You can’t really reach a level where people can see this as a tool, and they can use it when they feel it’s right and then not use it when they hit the limitation.”—7433_DeepLearning

  • “The technology is agnostic. It’s only how you apply it that makes it bad…it’s up to you to understand that data has no bias. It’s only the thought that we attribute to an abstract concept that makes it biased. So, it’s actually a human bias, it’s not an AI bias.”—41821_MachineLearning

Practitioners’ belief in the neutrality of technology was strong conceptually but was shakier in practice. When participants spoke about their models, they often used language that admitted the design, construction, and deployment of their systems was value laden. Participant 3286_MachineLearning, for example, distinguished between the developer’s incentives and the machine’s incentives by saying, “It’s never as simple as pushing a button or writing a line of code that says: don’t be evil. We don’t really understand how these systems work [and] it’s difficult to build systems with good incentives when there is a profit motive. Even if you do try to optimize for something that’s positive, you’re often going to have unintended consequences.” Similarly, participant 6862_MachineLearning said, “I do see and acknowledge the fact that…if it’s not well governed, the technology can be used in some malicious ways. It’s not necessarily the technology itself, but the motive behind the design and the usage of the technology.”

2.2 Ethical agency

There may be multiple ways in which the ethical agency of developers can be revealed. For this study, we focused on two. First, whether developers believed they could technically achieve what is being ethically asked of them. Second, whether they could intervene in a system they are working on to investigate a potential harm. Over the course of the interviews, a third window into ethical agency emerged, which we are calling “veiled agency.” Each of these is described below.

2.2.1 Technical feasibility of ethical demands

The ethical demands placed on AI/ML/DS developers currently focus on principles like fairness and explainability. As shown in Table 2, responses to whether it was technically possible to code these principles into an algorithm were mixed.

Table 2 Technical feasibility of ethical demands
Full size table

Ten (10) said it was not possible, seven (7) said it was possible, and three (3) said they did not know. Two (2) were not asked the question either because of time demands or because the conversation took a different direction. The remaining seventeen (17) said that some principles (like explainability) may be easier to achieve than others (like fairness), but in either case a concrete definition would be required. Developers need to assign a metric or a number to the principle, and then iterate it. The only way to do that is to have a clear definition of the principle. The difficulty, of course, is arriving at that definition in a global environment where automated systems are deployed in broad, pluralistic, and complex human communities.

2.2.2 Authority to intervene

Our proxy for the level of (actual) ethical agency developers have was to first ask whether they could pause a project or pull a module from production so a potential harm could be investigated. Second, whether they thought all developers “like them” or “in their role” ought to be able to do this.

As shown in Fig. 2, more developers responded that they did have this authority (n = 17) than did not (n = 7). Thirteen (13) responded they were empowered to escalate the issue to their managers, but not to act directly. Three (3) were not asked the question. When it comes to whether developers ought to have this authority, sixteen (16) said yes, one (1) said no, and eighteen (18) said that they ought to be empowered to escalate the matter but not to directly make the decision themselves. Five (5) participants were not asked the question.

Fig. 2
figure 2

Authority to Intervene in a Project to Investigate a Potential Harm

Full size image

In three (3) cases, developers said they would intervene to prevent harm even if they did not have the authority to do so. And another seven (7) pointed to circumstances in which developers had successfully halted a project for ethical reasons. For example:

  • “There are definitely circumstances…where developers of particular products have [said], ‘I think that this is harmful in ways that regulation has not caught up with, and at this point we’re just going to pull it until we figure it out.’ And I think almost always there’s been kudos for people who have made that decision.”—9202_ResearchScience

It is also important to note that developers believe their authority to intervene is tied to their positionality. They asserted that a junior developer should be able to raise the issue, but not pull the module from production, whereas if a senior developer or the developer who built the model discovers something, it would be their responsibility to act. Likewise, developers were aligned that they have more authority to intervene while a system is in development, but much less so once a system is deployed. They understood that the harms arising from pulling a module from production should be collaboratively discussed.

2.2.3 Veiled agency

As participants spoke more extemporaneously about the challenges they face, a picture emerged of ethical agency “veiled” as technical agency. Developers would sometimes list the myriad technical choices they make in the design, training, and deployment of automated systems. Occasionally, they would acknowledge they were also navigating ethical territories. For example:

  • “You find this metric over here is maximizing [for] safety and this one [for] fairness. Which one is more important? So, you find yourself thinking, okay, well, I want this amount of fairness, but this amount of safety, and so I’m going to balance them out around here.” 38410_DataScience

Another participant, 8511_DataScience, was much more explicit about the pipeline of decisions developers must make, often on their own. This participant pointed to choices about data collection and transformation, about what data to keep or remove, which models and predictions to use, and setting the optimization target. In some cases, even the design and functionality of the tool is left to the developer to decide. This comports with the perspective offered by participant 13479_MachineLearning, who said, “We do everything. We clean the data, we use algorithms to create models, then we operationalize the models.”

The perils of this veiled agency are not lost on the developers who recognize that big, society-wide decisions may be resting with them:

  • “We want [the product] to be fair, we want everyone to benefit. We know there are tradeoffs, and no government is going to say, yeah, we want you to do this thing that privileges this subgroup of people even if it harms this other subgroup of people. So, then, these hard choices happen on the ground. People make them. AI developers make them.”—4970_ResearchScience.

  • “There’s a huge danger of the technical person having all of the information and all the power and [who] can effectively sway decisions because they uniquely possess that understanding.”—6342_MachineLearning.

2.3 The characteristics of an ethical developer

What, then, does an ethical developer look like? What habits or virtues would stand out as commendable? One way to gauge this is to ask about moral exemplars. As Zagzebski [14] points out in her exemplarist moral theory, we learn to be ethical in the same way we learn other behaviors: by imitation. And we most often imitate the people whose behavior we think is admirable. When asked to name an exemplar, participants identified Edward Snowden, Moxie Marlinspike, Phillip Rogaway, Rich Sutton, Timnit Gebru, Margaret Mitchell, Emily Bender, Eric Topol, and Nassim Taleb. A few participants also named their current or past managers, whose names are not shared here to protect the identities of the participants.

The virtue most commonly attributed to these exemplars is conspicuous courage. Developers see these exemplars as vocal defenders of societal goods who call out unethical behavior and are not afraid to risk their own position to defend what they believe to be right. A typical response along this axis: “These are people who are in a good position, and [are] willing to risk their position to advance ethical goals. I find it inspiring.”—9927_MachineLearning.

Outside of the conspicuous courage ideal, there were other habits and virtues that participants respected. One such habit was “pride in craft” noted by participant 5119_Robotics, who clarified, “You should know what your code is for; it should be usable, maintainable, and well-documented.” Other participants also acknowledged the significance of technical hygiene, such as participant 44377_DataScience, who said, “If I get code from someone that’s well documented and very accessible, that would be the first hint that this person is thinking about people outside of themselves, and that they might be thinking about more ethical things.” Other habits identified include having a solid grasp on the fundamentals of the science and on the provenance of the data set. One participant summed up this sentiment thusly:

“[There are] a lot of idols in machine learning research, but they tend to be more on the theoretical part. They’re developing the tools, they’re not the ones exemplifying how to use the tools. [The habits I would look for] are questioning everything and supplying a high level of rigor in understanding the data, where you’re getting it, what it’s representing. And then the evaluation process for your model, if it’s representative, if it’s fair. And then also foresight into how, if I put this out in the real world and it impacted a lot of people, what would happen.”—7671_ArtificialIntelligence

3 Discussion

When we apply the insights from the transcripts to the conceptual model, it appears on the surface that developers situate their ethical agency in the technical category of Fig. 1. They conceive of themselves as well-intentioned technicians who work with morally neutral tools in a morally neutral occupation. Yet, the transcripts reveal significant gaps between how developers conceive of themselves and the reality of their work lives. Figure 3 illustrates these gaps.

Fig. 3
figure 3

Gaps developers navigate between perception and experience

Full size image

Let us begin with their intentions. Developers may indeed harbor good intentions, but they are not hired for their intentions, nor are they incentivized to pursue them. They are hired for their technical expertise and incentivized to pursue their own (and their employer’s) financial and reputational interests. Indeed, one participant said their employer would have to incentivize them to prioritize ethics over other metrics, like publications or speed-to-market. This developer said that without that incentive, taking the time to consider ethics would be “holding me back”—1855_Robotics.

This is not to say that the developers interviewed for this study are naïve. Many participants pointed to industries or projects they would avoid for ethical reasons. And they are aware that it would be trivially easy to change employers if their personal ethics were at risk of violation. Nonetheless, they lean heavily on their positive intentions to balance the perverse incentives that drive demand for their expertise. Participant 4970_ResearchScience, for example, said, “I work in a big tech company and all big tech companies are evil at some level. It’s hard to reconcile the personal with [the company]. What I try to do is make sure my work is good, and I can stand by it, and I can defend it.”

The second gap between developer perception and the reality of their work concerns the tools themselves. The repeated refrain that the algorithm is “just a tool” and it “depends on how you use it” implies that many developers believe the technologies they use, develop, and deploy are morally neutral—neither good nor bad. But a closer reading of participants’ word choices suggests that developers may more precisely consider the algorithms to be morally dormant until they get used. We see this in statements like “the tech is malleable” by participant 7150_MachineLearning, or “it’s only how you apply it that makes it bad” from participant 41821_MachineLearning. Through these statements, developers do acknowledge that these systems are capable of taking on a moral valence. That is, developers believe the technology can become bad or good when it is in use, but it does not (and cannot) exert a moral influence on human life when dormant.

Yet, reality is very different. Algorithmic systems do not arise out of a morally neutral ether, get used in only morally binary ways, and then return to the neutral ether once the code finishes running. Automated systems are created by humans (who are not morally neutral) for specific ends (which are also not morally neutral) and have ongoing active and passive impacts (which are rarely morally neutral) on human life. This dynamic is often referred to as “affordances” in philosophy and user design circles [15, 16]. This is the idea that technology, whether in use or not, is always morally active in that its mere presence changes what people think they can or cannot do. Although none of the participants explicitly mentioned the concept of affordances in our discussions, one participant came close, saying:

“All practitioners have to be aware of what they’re playing with. It’s not just about the data, it’s not just about models. It’s about decision making. It’s about replacing human decision making. I think people don’t see it as if they’re playing with something risky. It’s very exciting [and] it can be very rewarding in the sense that it can drive really good results. But you have to understand how those metrics are weighted in favor of particular incentives. And whether these are good.”—38410_DataScience

The third gap is the belief that the occupation of AI/ML/DS development is also morally neutral. This gap may be attributed to the fact that the discipline is so young (with the predominance of its growth occurring in the last 20 years) that practitioners have not yet deeply considered the ethics of the occupation they all share. As a consequence, some developers worry that their occupational ethics are being co-opted. Participant 6862_MachineLearning, for example, said that “the engineering field has not paid much attention to the development of AI, to the point where it has now been taken over by disciplines where there’s not much regulation.”

Indeed, this field did arise out of engineering, and many of the participants in this study could trace their path to AI/ML/DS through mechanical, chemical, or electrical engineering. Yet, AI/ML/DS development does not share the same safety demands that engineering does, nor its practice restrictions. AI/ML/DS developers are not in danger of losing their license to practice if they fail to communicate a safety issue, for example. Even worse, the sheer numbers of developers it sometimes takes to build and deploy complicated models means that few of them have visibility into the full technical stack.

What we are left with are ethical chameleonsFootnote 1—highly skilled practitioners whose ethics are largely adapted from whatever industry employs them, and a moral bright line that shifts from practitioner to practitioner. Their ethical agency is likewise variable. They are empowered to raise concerns (and believe they ought to do so), but they rarely see the myriad technical decisions they make as ethical ones. That is, they do not often realize just how much ethical agency they have.

4 Limitations

This paper presents the first known effort to center AI/ML/DS developers’ ethical agency in the conversation about AI ethics. As such, it comes with important limitations. First, the sample size of 40 for a global study is small, even if saturation suggests the insights gained are representative. Nevertheless, additional research in each geographic region may reveal how ethical decision making happens in different cultural contexts, and how it shapes what it means to be an ethical developer. Second, this research relied on participants who were willing to talk about ethics within their occupation. The results may therefore be subject to selection bias, with developers unusually concerned about the ethical aspects of their work being overrepresented. It is worth pointing out, however, that if our findings are biased in this manner, our conclusions would have likely been more pessimistic than they already are had the pool of interviewees included a majority of developers not at all interested in the ethical aspects of their work. We still think that the three themes identified here—the ethical ecosystem, ethical agency, and the characteristics of an ethical developer—may serve as a starting point for the deeper conversations that need to happen within the occupation.

5 Conclusion

Ethics in practice is communal. This is as true in AI/ML/DS development as it is in any other human endeavor. There are many people and institutions that contribute to the design, development, deployment, and maintenance of automated systems. All of them need to have strong expectations for ethics. This study is focused on one cohort in the larger system. We selected this cohort, because little attention has been paid to developers’ ethical agency, and because if we are to succeed in the project of ethical AI, we need ethical developers.

We, therefore, make three recommendations. First, developers would be well served to engage in occupation-wide discussions about what it means to be an ethical developer, not just what it means to develop ethical algorithms. These conversations can help surface the wisdom in this community. For example, how might a developer know if a system was developed with ethics in mind should they inherit that system from another developer? As an occupation-wide expectation, what do current developers owe to the developers who come after them?

Second, the importance of an ethics-first corporate culture cannot be overstated. Businesses must actively foster an environment in which developers can raise concerns or ask questions without fear of reprisal. This may not only include anonymous reporting mechanisms, but also ethics assessments conducted throughout the lifecycle of automated systems. These assessments can help expose developers to questions and issues they might not have considered otherwise. Businesses should also seek to make subject matter experts available (psychologists, sociologists, ethicists, historians, community members, etc.) who can engage in dialogue and help pressure test the assumptions that underly the creation of automated systems. We recognize that not every company will have the resources to bring this kind of expertise on board. Further research may help surface collaborative and equitable structures that can make this expertise more broadly available.

Third, academic institutions should integrate ethics education into the comprehensive course work of computer science majors, from the bachelor to the doctoral level. This education should be just as robust as the statistics, coding, and modeling courses they take. Given the impact automated systems have (and will likely continue to have) on human societies across the globe, critical thinking in ethics must become a standard expectation for all graduates.

For ethics in AI/ML/DS to be sustainable, it needs to be anchored in the practice itself. Yet, practitioners are grappling with morally troublesome gaps between who they believe themselves to be and what they are doing. It may be tempting for them to continue as ethical chameleons, but this position has limits. Even if developers think of themselves as guided only by technical rules, this research reveals that personal morality still influences their sense of moral action and that they are engaged in ethical decision-making while they are developing automated systems. If we may extend the metaphor, ethical chameleons can blend into the environment, but they cannot disappear.