Every craftsman likes his own product more than it would like him if it acquired a soul.

- Aristotle, Nicomachean Ethics 1167b35.

Machines are coming to resemble human beings more each day. Karl Marx, living in a world of steam engines and power looms, noticed that machines were created in the first place because they resemble human beings. Following in the tradition of Adam Smith and Charles Babbage, he observed that machines are initially engineered to perform functions that otherwise would be performed by human workers.Footnote 1 Machines decrease the cost of production and therefore increase profits for those who own the machines and what the machines produce. As a result, in the market, a human being competes not only with other human beings, but also with machines. Marx makes the point in the machine chapter in Capital, the longest of vol. 1: “The contest between the capitalist and the wage labourer dates back to the very origin of capital. […] But only since the introduction of machinery has the workman fought against the instrument of labor itself, the material embodiment of capital” (MECW 35, 430).Footnote 2 And later, more to the point, he says, “The instrument of labour, when it takes the form of a machine, immediately becomes a competitor of the workman himself” (MECW 35, 433). The resemblance between human beings and machines is understood through their overlapping capacities for production. After all, Karel Čapek’s use of the Czech word robota in R.U.R., among the first uses of the term, is meant to carry the sense of forced or serf labor.

Marx found technological development at the heart of capitalism’s drive and, ultimately, its undoing.Footnote 3 The economic logic pushed to its limits leads to the prospect of full automation: a world in which all labor required to meet human needs is superseded and performed by machines; technologies have fully substituted, not merely augmented, human labor.Footnote 4 What should we make of this prospect? How does the economic context in which machines currently exist and have their being affect how we view and categorize the technologies? What does the increasing sophistication of machines say about the human beings they are coming to resemble?

I explore the possible future of automation by considering a specific point of resemblance between human beings and machines: intelligence. The power loom represents a moment of technological development, but a power loom cannot solve novel problems or track the random movement of objects in its environment. It does not have common sense. I am concerned instead with machine intelligence (MI).Footnote 5 Many well-funded organizations and well-educated people are working to produce one, and their techniques, priorities, and goals cover a wide range. For that reason—along with the fact that a human level MI does not currently exist—it is difficult to say what an MI would look like with any certainty or specificity. A fixture of literature on intelligent technology is that many critical terms are amorphous. The lack of consensus supplies motivation for my discussion. For reasons that will become more evident as I proceed, what I mean by ‘MI’ will remain broad. However, my initial discussion will not concern what is called ‘weak’ or ‘narrow’ Artificial Intelligence (AI)—namely, intelligent technologies designed to have a specific, limited task. Rather, I take an MI to be a piece of technology that possesses general intelligence, meaning, following Bostrom, that it possesses “common sense, and an effective ability to learn, reason, and plan to meet complex information-processing challenges across a wide range of natural and abstract domains” (Bostrom, 2014, 3).Footnote 6 An MI is an AGI. It also has human level intelligence or above. It is an intelligence that either matches or exceeds the cognitive performance of humans in virtually all domains of interest.Footnote 7 So it might be a superintelligence (ASI).

The specifics of the definition are less important than the comparative procedure it assumes. The definition references an accepted set of normal human capacities or abilities, standards of success in performing tasks, and a set of relevant domains. How might the features of an MI account be, in Marx’s sense, demystified and historically situated?

Sections I-III explore (1) the Marxist concepts of alienation and reification in the act of producing MIFootnote 8, and (2) the consequences an existing MI might have on the conception of a potential future in which machines have freed humans from the need to do labor. My aim is not to provide an exhaustive interpretive account or make concrete predictions about the shape MI development will take. I instead set out tools for a Marx inspired analysis of a certain type of MI. Section IV utilizes the framework to discuss possible futures of automation, including those with other types of MI. Ultimately, the prospect of full automation sheds light on current debates about the moral status of machines.

1 Alienation through Reification

The concept of alienation (Entäusserung) or estrangement (Entfremdung) appears mainly in Marx’s work from the 1840 and 1850s. The idea is broad and, in terms of MI production, fruitful. For Marx, alienation is the structure of production in capitalism. It generally signals the separation of things that should be found together—a unity that has been split into isolated parts. As Marx says in the “Notes on James Mill,” the capitalist world is a “caricature of […] real community” because of alienation (MECW III, 217). The real community is characterized in part by a unifying sociality, a consciousness of species-being (Gattungswesen), that determines the productive activity of its members. Under capitalism, production is conceived as the activity of private competitive individuals who conceive of themselves as separate fundamental units of society. Because survival is dependent on employment and private property, the high stakes of competition in the market leads to social antagonism and isolation (Mészáros, 1970, 81-2).

The machine is an object of alienation for the worker in several ways. First, workers are alienated from the act of producing machines. The labor consists of repetitive tasks that have little connection to the overall design or function of the product. For instance, a computer might be designed by people in one part of the world then produced and assembled by countless people across other parts of the world. Only in an abstract sense are the workers producing a machine together. More immediately, they are soldering, boxing, or operating other machines. Second, workers are alienated from other workers. When I buy a computer, no one I interact with (assuming I interact with anyone) was actually involved in making it. And the people who made it have little to no interaction with each other. Further, for Marx, because machines are also participants in the labor market, alienation from other workers is a phenomenon that includes alienation from machines. Workers and machines are producing machines that increasingly eliminate the need for human labor. Humans produce machines and then encounter their products in alien form, as both products and competitors.

In early Marx, there are cryptic remarks about how labor involves externalization of the worker. The concept of reification or objectification (Verdinglichung) is not particularly prominent in Marx (especially in his later writing) but is there nonetheless. It aids in understanding alienation. He says in the 1844 manuscripts, “The product of labour is labour which has been embodied in an object, which has become material: it is the objectification of labor.” The product is not simply the arrangement of parts of nature, but labor is contained in the product. He says two paragraphs later,

The worker puts his life into the object; but now his life no longer belongs to him, but to the object. Hence, the greater this activity, the more the worker lacks objects. Whatever the product of his labour is, he is not. Therefore, the greater this product, the less is he himself. The alienation of the worker in his product means not only that his labour becomes an object, an external existence, but that it exists outside him, independently, as something alien to him, and that it becomes a power on its own confronting him. It means that the life which he has conferred on the object confronts him as something hostile and alien. (MECW 3, 272)

There are two ways to read the passage. First, in producing or creating an object, a worker expends a certain amount of effort. The effort is directed towards the object. Accordingly, the object is a repository of labor that the worker cannot get back. It is permanently expended, formed into and represented by the object. Second, there is a more metaphysical reading according to which labor is the objectification of some part of the essence, being, or spirit of the worker. Labor is a form of self-actualization.Footnote 9 The object stands as a testament that a human being produced it. The products of our labor represent a part of ourselves—our desires, preferences, and self-estimation. Consider why it is that we often want others to recognize and appreciate what we have made. For Marx, this pride is increasingly stripped away under capitalism. Labor becomes nothing more than the means for producing objects that are not our own. Alienation describes the phenomenon of having an object that reifies and contains a part of ourselves taken away and owned by someone else.

Setting aside its plausibility as a general account of a production, the metaphysical reading makes sense as a frame for talking about machines. Because machines are made to perform a task that before was performed by a human being, in producing a machine, there is an unmistakable sense in which the machine is the objectification of its human producer. It’s ratio essendi is the productive potential it shares with humans. Both transform materials and energy into commodities for capitalists. In the case of MI, the product is intentionally and explicitly meant to include core aspects of its designer. It is the externalization of intelligence, the capacity for generalized production. The machine embodies the fundamental creative part of the worker that makes reification possible. It is not merely the result of alienated productive activity, but it is an objectification of the very productive activity that produced it. More precisely, it is the objectification of what the human producer, within the current economic scheme, takes their generalized productive power to be. The MI is an embodied form of the designer’s conception of their own intelligence.

The point can be expressed as the coalescence of distinct forms of alienation. First, according to Marx, workers face technological alienation: with the introduction of machines, workers view themselves in mechanical terms and grapple with the precarity of their jobs. When workers produce a machine, it is taken away from them and then it takes the human workers’ place in the market, resulting in “technological unemployment.”Footnote 10 Second, workers are alienated from the product: the commodity is not only hailed as the source of wealth, but it also competes with, and increasingly outcompetes, humans in the labor market. Finally, we are alienated from ourselves. This form has several senses. For instance, we are alienated from each other: we view and value others most immediately through their capacity as workers. It is more salient to me that the person on the phone is a customer service representative than that they are a human being. In the experience, I feel an antagonism towards them, not a unity with them. Further, I am alienated from myself. I am no longer in touch with my creative social productive potential. I develop skills that I can sell or find ways to sell skills I have. I view and value myself most immediately as a worker.

MI exhibits all three in a heightened and systematic form. The worker encounters something foreign, with respect to its origin and substrate, but also uncannily familiar, with respect to its capacities and general role in the market. If the MI were comfortably in the category of the power loom or steam engine, the economic status of the human worker would not be threatened so deeply. While the textile workers in 19th century England worried about their individual jobs in their specific sector, they could in principle learn skills that made them more economically valuable than a loom. (They should just learn to code!) With an MI, this possibility is in principle closed off. The machine embodies the adaptability that characterizes the human worker. The capacity to develop marketable skills has been thoroughly automated. No relevant domains of interest remain only for the human. Because of this, the economic status of the human worker is thoroughly threatened, leaving us to wonder whether a reliable barrier between humans and machines has been breached.Footnote 11

To summarize, although all machines can be viewed broadly as externalizations of the productive capacity, there is a difference in the scope of the capacities. The power loom is limited in what it can do. It is restricted to a specific domain and requires a human operator. A weak or narrow AI is similarly restricted. However, an MI, as I am using the term, by definition overcomes these restrictions. It is the externalization not of any specific productive capacity, but of what is taken to be a more basic, essential feature of the human worker: the capacity to produce across all relevant domains. The worker accordingly becomes alienated from a central, perhaps essential, human ability in a new way. When human workers encounter the MI in alien form, they are also looking at a thorough and upgraded reflection of themselves.

2 Liberation through Alienation

Alienation presents a categorization puzzle. Either, on the one hand, the MI is simply another product of alienated labor. It is a glorified power loom, a product on the shelf. If so, there is nothing qualitatively special about MI. Its creation does not represent a conceptually significant moment in technological development. Or, on the other, the MI, in virtue of its generalized productive capacity, is the same as a human being in capitalism. It is another laborer, an “artificial proletarian,” (2019, 135) as Dyer-Witheford et al. say. If so, perhaps surprisingly, again there is nothing special. The MI may represent a significant moment in technological development, but the significance consists in the fact that we simply place the MI in an established category alongside humans.

To which category does it belong? What, if anything, is special about an MI? Viewed from the other side, what, if anything, is special about human beings?

A potential key is an entity’s capacity for unalienated labor. While there is a distinction between carbon-based producers and silicon-based producers, both belong to the same category under capitalism. Viewed solely through economic eyes, the material substrate of the producer does not matter.Footnote 12 But there is also a distinction between (1) producers who can be members of the “real community,” the proletariat that can be liberated, and (2) those that cannot. The power loom cannot be liberated, at least in the same respect as the human worker. It may be “put to emancipatory use,” but it does not stand among its fellow proletarians in the fight (2019, 147). The question is where MI belongs.

It is tempting to propose a resolution by saying that when the MI exists, it will be a person, an entity that belongs in the moral community.Footnote 13 A person has moral standing or status. Use of these terms differs, but the general thought is that the MI, so described, seems to possess whatever traits or qualities mark entities (mostly obviously human beings) as deserving of respect, recognition, consideration, etc. I intentionally left the moral status of the MI unresolved and will continue to do so. Based on my definitions, an MI is a piece of technology with human intelligence. The MI may seem sufficiently like a “normal” human being that we should welcome it as a comrade in the struggle. “Workers of the world” is a group that would include the MI.Footnote 14 Machines and humans are in the same category, morally speaking.

This proposal may be fine as far as it goes, and I will return to it below, but it skips over critical issues—namely, the creative design choices that take place prior to the existence of an MI person. The nature of the MI is the result of choices made by its designer. The choices are alienated productive activity. The MI is wholly the outcome of alienated labor in a non-arbitrary way. It is not an entity with a pre-existing or contingent nature that is thrust into an economic scheme. The MI is a native child of capitalism.Footnote 15 It is the objectification of the designer’s self-conception, a conception that is of an alienated self; the MI is the alienated embodiment of the designer’s alienated self. It might still make sense, and even be true, to say that it is a person, but more work is required to see how something with such a nature would also have the capacity for liberation.

Let us bring these points out further. The designer’s activity is straightforward commodity production. They are externalizing a capacity that can be used to generate profit. In the case of an unintelligent machine, the laborer externalizes a capacity that is conceived as a tool for creating wealth. So the machine is alienated from the laborer because the capacity is represented in the machine as inherently oriented towards profit. The machine cannot be liberated. However, in the case of an MI, the designer is externalizing the very capacity that makes the designer distinct from the machines that came before. The MI can perform all the relevant tasks that a human can. So the designer sets out to make an efficient producer, not a person. The designer is attempting to create another commodity, but it is a commodity defined by its fundamental resemblance to the designer qua intelligent entity. Would the MI then be capable of liberation?

To answer, we should consider the motivation, explicit or not, behind producing MI. In most cases, the productive activity is carried out to rid themselves of their alienated selves. We embark on the project of creating MI so that we can eliminate human drudgery. If we, so the thinking goes, can externalize the parts of ourselves that are compelled to do unwelcome wage-labor, we will no longer be alienated from ourselves. The parts of ourselves that do the unwelcome labor are now distinct, autonomous entities. Somewhat prosaically, we can become unalienated by alienating ourselves from our alienated selves.

Instead of asking whether a designer can produce a person, a comrade, through alienated labor, we ask whether a designer can work to produce a commodity, a sophisticated piece of technology, that ends up, as a secondary effect, being a comrade. Here we encounter a tension. In Marxist spirit, I call it a contradiction. Marx, especially in Capital, is concerned with finding contradictions immanent in capitalism. My concern is a contradiction found in the union of certain technophilic communist visions and the reality of capitalistic technological development oriented towards MI as I have described it. If the possibility of liberation depends on the existence of MI, i.e. a machine that embodies human intelligence, then the MI must be incapable of its own liberation. In that case, there must be core features of human beings that are not reified in the process of creating an MI. Otherwise, the possibility of human liberation would be threatened. However, if the MI embodies its human producers more completely, and intelligence is or can be a core human feature that is distorted under capitalism, then the MI is a fellow comrade in the struggle for liberation. MI is meant to be radically similar to its designer and yet also a commodity, incapable of unalienated labor. Is the externalized alienated self of the designer a person, something that should be liberated? If so, to create MI, one must fail to create MI. If it truly is an MI, it cannot come about through alienated labor.

3 Communism or Technofeudalism

To prepare for a discussion of automation, let us consider the same topic from the perspective of communism, as nebulous as that perspective is. Because there is a common experience of exploitation, fellow wage-laborers, despite being alienated, have a certain camaraderie with each other. This relationship is the beginning of a real human community. We feel an urge towards unity that is thwarted by the current mode of production. Without this glimpse of shared humanity, there would be no hope for the supersession of capitalism. In the creation of MI, we attempt to create something that is meant to bear the burden of labor but that also is comfortably outside the human community, something with which we feel no (genuine) camaraderie.Footnote 16 We also explicitly do the opposite: we create something that shares our intellectual capacities, the very capacities we hope to utilize in a free way in the real community. As we become more united, our forms of technology become more like us. Yet the viability of the revolution requires them to be unlike us.

Here we find a fresh look at the contradiction. Human laborers and machines are interchangeable under capitalism. But under communism, when humans are their real selves, they will no longer belong to the wage-laborer category. In other words, although they are in the same category under capitalism, there is something in human beings, but not in machines, that becomes distorted or warped by capitalism. Human labor in capitalism is a perversion of its real form. In a real community, the work—which might well be used to create the same object as under capitalism—is free and creative.Footnote 17 However, in producing MI, we strive to make a machine whose nature is, by necessity, pure alienated labor. It is not as if its creative drive is perverted under a particular economic scheme. MI’s nature and reason for existence is the result of market forces in a non-arbitrary way. It, according to the primary intentions of its inventors, cannot be liberated. In fact, we are assuming that our own freedom is dependent on the labor of the machine.

Despite its origins, if the MI is capable of liberation, the viability of a revolution might be built on the slave-labor of advanced technology. We enter into technofeudalism organized on the basis of material substrate—carbon bourgeoisie and silicon proletariat.Footnote 18 Technological alienation would remain, though in a different guise. If we do not wish to face this problem, we face another. We are making MI in a way that is, on a Marxist view, supposed to make a post-labor economy possible. The MI by hypothesis is similar to human beings in the domain of intellectual capacity. If they are similar, they might be our comrades, in which case, to force them to provide the labor that allows for liberation would be to fall back into the same horrors we are fighting against. So we are incentivized to make them unlike human beings in a specific way. If we do that, we might no longer be talking about MI.

My suggestion is inspired by the concern that calling a machine a person, or even openness to adjusting the concepts of personhood or rights in light of sophisticated technologies, risks ignoring the importance of the values that guide productive activity.Footnote 19 This is frequently overlooked in current conversations about ethics and MI/AI. To use Marx’s phrase, they tend to take place “only in the misty realm of philosophical phantasy” (MECW, IV). We should not jump to conversations about the moral status of a machine without considering the morality of the productive activity itself. A stork does not drop machines on our stoop, as if they are created ex nihilo (though sometimes it feels that way). Here we encounter challenging questions about a possible economic, and thereby alienated and alienating, ideology that informs philosophical concepts of personhood or moral standing. This is not to say that the concepts are entirely a product of material economic factors and contain no ethical insight. I have been assuming (perhaps unwisely) that our moral values and self-conceptions are not, or need not be, mere epiphenomena of material conditions or capitalist ideology. This assumption comes to the fore in the next section. Questions about the moral status or personhood of a machine lead to an examination of how the machine is designed and created, what and whose labor it embodies. As Floridi (2017, 4) says, “the debate is not about robots but about us.” More accurately, there can be no debate about machines that is not also, at the same time, a debate about us.

4 The Four Possible Futures of Automation

With the practical contradiction in mind, I shift to consider how it manifests in the broader issues of automation and the moral status of machines. My interest remains the productive activity that results in an MI person and whether all required labor can be performed by such an MI. Could all jobs be automated? And how might an answer to this question interact with the potential moral status of machines? Are there possible futures in which we do not face the contradiction I have been exploring?

Since, in the short term, automation represents the threat of unemployment and, in the longer term, full automation represents the threat of full unemployment, many people feel an urge to defend the special status of human beings. If there is to be durable job security, we must identify some distinctive trait or set of traits that cannot be automated. Candidates might be creativity or imagination, a spirit or a soul. We seek to draw a relevant line between humans and machines. In the economic context, the question is whether such traits are required to do the jobs that supply human needs. Even if there is something special that will always distinguish humans from machines, it might not save jobs from the threat of automation.

On this framing, there are two different questions:

  1. 1.

    Will all special human traits be replicated in technology? Is there a bright line between humans and machines or will the line dim to darkness?Footnote 20 The question is about whether an MI will in fact exist. If it cannot, it follows that it will not. However, if it can, it does not follow that it will. I consider these permutations below. Notice also that the analysis is not committed to any claims about what exactly is special about humans, or even the claim that there is something special.

  2. 2.

    Is what is special about humans essential to perform all required labor? The term ‘required labor’ refers to labor that produces everything humans (and other members of the human/moral community) need to live at or above a certain threshold of comfort. For technologies to perform all the labor required for humans to live reasonably stable, desirable, and sustainable lives, must we create them to possess the special human trait(s)? Does the bright line between humans and machines matter practically, materially, or economically? Where the threshold should be set is not an issue I consider here, though the prospect of full automation is often associated with utopian thinking.

Both are yes/no questions. They therefore yield four possible versions of the future of automation.Footnote 21

4.1 Yes/Yes

In the first possible future, for all required labor to be done, the (previously) special human spirit is essential. But because there is a bonafide MI that resembles humans in every way that currently matters to us, full automation seems possible.

Those who dream of a post-work utopia might call the scenario terrific news. We have machines to do any required job. Free time for humans abounds. But it is also part of the scenario that there is no significant distinction between humans and machines. When we take stock of the candidate special traits, they tend to be the aspects that make humans morally valuable—what makes us more than machines.Footnote 22 But now machines possess those traits too. So offloading all labor onto machines would be wrong in the same way that human forced labor is wrong. The desire to put an end to human labor would fail, perhaps by definition, since we could not draw any relevant line between humans and machines. Viewed from this perspective, full automation is not possible. Calling the labor in this future fully automated is equivalent to saying that labor today is fully automated.

The failure to end human labor is another way of stating the contradiction outlined above. Assuming, as we do in this future, that the utopian future of full automation necessarily utilizes an MI, the utopia would devolve into an oppressive economic scheme. Class conflict would map onto the human/machine divide. Moral consistency demands a return to the status quo under capitalism: humans and machines working side by side under the same economic category. The dream of free time dies.

As a speculative point, if human free time were contingent on the labor of machine equals, it would be tempting to rationalize new conceptions of personhood or redefine what is special about humans. It is in the material interests of the human overclass to invent a way to reestablish the bright line between humans and machines. The conceptualizations would conveniently define a moral problem out of existence or prevent people from noticing it. The oppression then appears like the natural order of things. The Great Chain of Being persists. For many thinkers, this is a simplified description of a phenomenon found at every turn in the history of colonialism. Systems of oppression with respect to race, gender, species, class, and potentially material substrate are built on rationalizations that align with the advantages of the dominant groups.Footnote 23

4.2 Yes/No

One version of this possible future looks the same as the previous one. The MI exists, and although the MI is not essential for the performance of all required labor, humans still use them to achieve full automation. Here the moral issues are more overt but fundamentally the same as the ones considered above. The fact that the special human spirit is not essential to the labor indicates that the labor is drudgery. But the point remains that we are compelling MI, our moral equal, to do it. For the reasons outlined in the previous future, humans look especially monstrous.

However, a more pleasant future also falls under this classification. Because neither humans nor MI are essential to required labor, it is possible to achieve an automation regime that leaves us all free. The MI might help create such a system and then enjoy free time alongside us. Or humans might engineer the system of full automation and then spend time creating MI as a separate leisure project. Once the MI exists, it would be accepted among humans as one of our own. In both cases, the machines are rightful members of the real community. Once full automation is up and running, they have no more part of it than humans do.

This future avoids the practical contradiction considered above by denying that required labor depends on intelligence or the special human traits. More precisely, ex hypothesi it is possible to meet the community’s needs without workers who exhibit special human traits. Instead of speculating about what a complex automation system consisting of unintelligent technology or weak AI would look like, one might wonder how plausible a ‘no’ to the second guiding question is. How many special human traits are there? Are none of them essential to perform the required labor? Because these questions are tricky, futures initially placed in this classification might, on closer inspection, fit better in the previous one. Or it might be difficult to determine where they fit. Consider a future in which the work is drudgery but still (usually? occasionally?) needs a human touch. The motivation would be strong to give the work to an MI, which raises the moral questions again. Alternatively, we might create a machine that exhibits the minimum human traits to do the labor. In that case, part of what is special about humans is still found in the machine. We therefore cannot confidently place it outside the moral community. Even when a machine is a partial or quasi-MI, there is a categorization puzzle similar to the one described above. Contemplating a possible future of automation also encourages reflection on how capacities for labor inform our self-conceptions.

4.3 No/Yes

In this possible future, (1) we do not replicate what is special about humans, and (2) required labor needs the human touch. In other words, the present extends indefinitely into the future. Here we are confronted with what is, to many, a dispiriting conclusion: there is no hope for the end of human labor. Full automation is impossible.

Some people view technology as the key to a utopian future of free time. We could have a world in which humans are still active, productive, and creative but not compelled to sell their labor power. Although this would not be possible for everyone, many utopian visions remain compatible with future #3.Footnote 24 We might change how we view work and take solace in the fact that the required work taps into the distinctive human spirit. Plus, in the future, a wide range of work remains eligible for automation. Narrow AI will continue to change the form that labor takes. We might approach full automation asymptotically.Footnote 25 Eventually, while automation is not strictly speaking full, for all practical purposes it would be.Footnote 26

The contradiction described above does not emerge in this future. However, the more pervasive the automation, the more challenging it is to imagine capitalism continuing to exist in any currently recognizable form. A Marxist is likely to think that the internal tensions within capitalism become unsustainable long before a full MI arrives on the scene. This future is compatible with an economy in which some of what is special about humans is required to do some of the required labor. There may be pervasive technological unemployment and a need for only a handful of human workers. If well being still depends on owning capital or selling one’s labor, the immiseration would be stunning. A functioning economy and society in this future would need to be radically restructured. The contradiction inherent in capitalism, according to the Marxist, can become explicit without an MI.

While the contradiction I described presents itself most overtly when an MI enables full automation, a more general version of the contradiction exists in automation regimes that displace a sufficient amount of labor. In other regimes (up to ‘practically full’ automation) that make the current form of capitalism unsustainable, humans still depend on technologies that resemble them, potentially in ways that raise questions about the morality of the regime.

4.4 No/No

Finally, it is possible that we are special and that we still could automate all required labor. Compared to the previous futures, this might seem like the ideal arrangement. In virtue of the first ‘no’, we avoid many moral problems.

The second ‘no’ opens up the possibility of full automation without any need for human involvement. Again setting aside the plausibility, the good news is accompanied by another dispiriting conclusion: either (1) your job is not required, (2) what is special about human beings is not an essential part of your job, or (3) both. Many people find this an apt description of life in the market. We often get the sense that what we are doing at our jobs is not about meeting any genuine human needs. On the contrary, some industries seem to profit from harming people. Or we might get paid to do a job that would not exist in a better world. Many of us feel most human when we are not at work—when we are lounging, eating, and socializing (those distinctively human activities, of course).Footnote 27 Many of us describe our work as ‘mindless’ or ‘soul-deadening’, as something that stifles creativity and imagination. The fact that we feel like carbon-based machines at work foretells more of the same. The resemblance between humans and machines does not supply an ideal to which technological development can aspire, but rather the gradual loss of a distinctive human essence.

Another version of this future is worth exploring. The fact that we do not replicate in technology what is special about humans does not mean that we cannot. If we knew that all required labor could be automated without creating technologies that resemble humans in every way that currently matters, maybe we should avoid creating such machines.Footnote 28 Full automation in this future does not depend on MI. Because the moral problem that generates the contradiction only arises when the MI is among us, it seems prudent to withhold the special human spirit from our technologies. If we could create them different enough, we might be free of the moral hazards. We do not need to rely on an ethically enlightened human race accepting the MIs into the fold.Footnote 29

Setting aside whether an industry-wide prohibition like this is practically feasible, an array of moral questions remain. As I mentioned in the previous section, the fact that a machine lacks some of what is special about humans does not place it entirely outside the moral community. It may still be a person, close enough to a person, or in some sense a partial person.Footnote 30 Nonhuman animals are a parallel case. One need not view other animals as the same or equal to humans to believe that it is morally wrong to exploit them. They should be given at least some moral consideration, some place in the community. Denying technologies special human traits does not avoid all moral hazards. It merely avoids the simplest and most obvious one. Forcing the machine to work might still be unacceptable, even if it is not an MI.Footnote 31 More challenging problems exist in cases of technologies that only partially resemble human beings. We face a moral uncanny valley.

A possible future in which we replicate in technology what is special about humans also presents a question that is rarely considered in debates about the moral status of machines. The question about values that guide productive activity can be directed towards the machines themselves, as (eventual) moral patients. For instance, if we can create a machine that shares in what is special about humans, but we deny the machine those traits and instead make it unclear what moral status the machine has, do we do something wrong to the machines? What are our responsibilities as creators to our creations? Should we create the best? What parallels are there between producing machines and the ethics of enhancement? We might, for instance, create a machine that is unaware of its status as an uncompensated worker. In a sense, this describes all currently existing technology. However, it is not as if we currently have the power to create machines sophisticated enough to be aware of their status but simply opt not to utilize it.Footnote 32 When we have the power, how should we think about using it and whether to use it at all?

5 Conclusion

The four possible futures provide insight into how the two guiding questions are connected. There is a tendency to frame discussions about technological development in terms of an attempt to replicate important human traits, like general intelligence. But it must not be forgotten that the development also takes place, directly or indirectly, on the back of the drive for profit, not a disinterested exploration of technological potential for its own sake (Dyer-Witheford et al., 2019, 3). Within productive activity, our special traits are viewed most immediately as commodities. Companies seek to replicate them in technologies because it serves a financial interest. In the eyes of the market, where the creation of machines takes place, the human capacity for labor is paramount. My discussion has resisted treating intelligence or other important human traits as mystical, ahistorical abstractions.

But as Adam Smith noticed, there is another side to automation. It also makes economic sense to turn labor into the sort of activity that can more easily be done by a machine.Footnote 33 This, as is familiar to plenty of workers, makes a human more into a machine. How we view labor morphs into how we view ourselves. And the bright line starts to dim in another way. The conclusion is that, at least while discussing technology, it is difficult and perhaps impossible to think about “special human traits” in a way that does not cast them as tools for economic advantage.Footnote 34 We find evidence for this in the fact that many people struggle to imagine what they would do if it were no longer possible for them to sell their labor. It is earning a wage that is significant to them, not imaginative and distinctive creation.

These reflections reposition questions about the moral status of machines. By understanding machines as economic entities, and the process that produces them as an economic and perhaps alienated process, we reckon with possible economic values that inform concepts like moral standing, personhood, or ‘special human traits’ like intelligence. My utopian hope is that with a better understanding of ourselves, we and our technologies are equipped to make better creative choices. Whether the choices can avoid contradictions depends on which specter of automation haunts us.