1 Introduction

Until recently, robots were mainly used in factories for automating production processes. In the 1970s, the appearance of factory robots led to much debate on their influence on employment. Mass unemployment was feared. Although this did not come to pass, robots have radically changed the way work is done in countless factories. This article focuses on how the use of robotics outside the factory will change our lives over the coming decades.

New robotics no longer concerns only factory applications, but also the use of robotics in a more complex and unstructured outside world, that is, the automation of numerous human activities, such as caring for the sick, driving a car, making love, and killing people. New robotics, therefore, literally concerns automation from love to war. The military sector and the car industry are particularly strong drivers behind the development of this new information technology. In fact they have always been so. The car industry took the lead with the introduction of the industrial robot as well as with the robotisation of cars. The military, especially in the United States, stood at the forefront of artificial intelligence development, and now artificial intelligence is driven by computers and the Internet. More precisely, robotics makes use of the existing ICT infrastructure and also implies a continued technological evolution of these networks. Through robotics, the Internet has gained, as it were, ‘senses and hands and feet’. The new robot is thus not usually a self-sufficient system. In order to understand the possibilities and impossibilities of the new robotics, it is therefore important to realise that robots are usually supported by a network of information technologies, such as the Internet, and thus are often presented as networked robots.

New robotics is driven by two long-term engineering ambitions. Firstly, there is the engineering dream of building machines that can move and act autonomously in complex and unstructured environments. Secondly, there is the dream of building machines that are capable of social behaviour and have the capacity for moral decision making. The notion that this may be technologically possible within a few decades is referred to as the ‘strong AI’ view (AI: artificial intelligence). It is highly doubtful that this will indeed happen. At the same time, the ‘strong AI’ view prevails in the media and is highly influential in the formulation and public financing of IT research. It is beyond dispute that this technology will strongly influence the various practices researched. This also puts many societally and politically sensitive issues on the political and public agenda. For example, according to Peter Singer, the robotisation of the army is ‘the biggest revolution within the armed forces since the atom bomb’ [82]. The robotisation of cars, too, appears to have begun causing large technological and cultural changes in the field of mobility. Netherlands Organisation for Applied Scientific Research (TNO) describes the introduction of car robots as a “gradual revolutionary development” [94]. Through robots, the police may enjoy an expansion of the current range of applications for surveillance technologies. Home automation and robotics make tele-care possible and will radically change health care practice over the coming years. Finally, we point to the fact that over the past years, ‘simple’ robotics technologies have given the entertainment industry a new face: think of Wii or Kinect. We will continue to be presented with such technological gadgets in the coming period.

New robotics offers numerous possibilities for making human life more pleasant, but it also raises countless difficult societal and ethical issues. The debate on the application of robotics to distant battlegrounds is very current, while the application of care robots is just appearing on the horizon. Prompted by the arrival of new robotics, the Rathenau Instituut in 2011 and 2012 investigated the social meaning of robotics for the years to come in Europe and the US by studying robotics developments in five application domains: the home, health care, traffic, the police, and the army [70]. For this study, a comprehensive literature review was carried out with the goal of selecting the most relevant articles on the robots of the five application domains and the related ethics. For each application domain, we present the main ethical issues and the most relevant findings obtained from the literature, with the focus on the following three central questions:

  1. 1.

    What is possible right now in terms of new robotic technologies, and what is expected to become possible in the medium and long term?

  2. 2.

    What ethical questions does the new robotics raise in the shorter and longer terms?

  3. 3.

    What regulatory issues are raised by these ethical issues? In other words, what points should be publicly discussed or put on the agenda by politicians and policymakers?

Based on the results of our literature review, this article firstly addresses the above questions in the following five sections relating to, respectively, the home, health care, traffic, the police, and the army. After that, this review provides the first overview of studies that investigate the ethical aspects of new robotics based on some key characteristics of new robotics that are discussed in Sect. 7. We will end with an epilogue.

2 Home Robot

In this section we will discuss two types of home robots: the functional household robot and the entertainment robot. In relation to entertainment robots, we have made a distinction between the social interaction robot and the physical interaction robot such as the sex robot.

2.1 Household Robots

In relation to household robots, we see a huge gap between the high expectations concerning multifunctional, general-purpose robots that can completely take over housework and the actual performance of the currently available robots, and robots that we expect in the coming years. In 1964, Medith Wooldridge Thring [92] predicted that by around 1984 a robot would be developed that would take over most household tasks and that the vast majority of housewives would want to be entirely relieved of the daily work in the household, such as cleaning the bathroom, scrubbing floors, cleaning the oven, doing laundry, washing dishes, dusting and sweeping, and making beds. Thring theorised that an investment of US$5 million would be sufficient for developing such a household robot within ten years. Despite a multitude of investments, the multifunctional home robot is still not within reach. During the last ten years, the first robots have made their entry into the household, but they are all ‘one trick ponies’ or monomaniacal: specialised machines that can only perform one task. According to Bill Gates [40]: ‘[w]e may be on the verge of a new era, when the PC will get up from the desktop and allow us to see, hear, touch and manipulate objects in places where we are not physically present.’

It is unlikely that households will start using in droves the monomaniacal simple cleaning robots such as vacuum cleaner robots, robot lawn mowers, and robots that clean windows with a chamois leather. These robots can only perform parts of the household tasks, and they also force the user to adapt and streamline part of their environment. The study by Sung et al. [88] showed that almost all users of a robotic vacuum cleaner made changes to the organisation of their home and their home furniture. The more tidy and less furnished the household is, the easier it is to make use of that robot vacuum cleaner. This process of rationalising the environment so that the robot vacuum cleaner can do its job better is known as ‘roombarization’ [89], referring to the vacuum cleaner robot Roomba. Typical modifications are moving or hiding cables and cords, removing deep-pile carpet, removing lightweight objects from the floor, and moving furniture. An inhibiting factor for the rise of the commercial vacuum cleaner robot probably lies in this need for a structured environment. The history of technology research shows that the interest in new devices quickly decreases when existing practices require too many changes [66].

The expectation that the new generation of robots will operate in more unstructured environments will not work in the household. This is not only a matter of time and technological innovation and development, but it is also an issue that comes up against fundamental limitations. Housework turns out to be less simple than expected. Closer inspection shows that many situations in which a household task must be performed do require a lot of common-sense decisions, for which no fixed algorithms exist. The degree of difficulty is shown by research from the University of California at Berkeley, which aims to develop a robot that is able to fold laundry. Eventually, a robot was developed that took 25 minutes to fold one towel [62].

Bill Gates’ prediction of “a robot in every home by 2015”, in our opinion, is highly unlikely to happen. We expect that this cannot be realised in the short or the medium term. Many technical challenges must be overcome before the home robot can convince the public that it can take over a variety of household chores efficiently.

2.2 Amusement Robots

2.2.1 Expectations

It seems that entertainment robots do meet expectations and social needs. Compared to the household robot, expectations concerning the entertainment robot are much less pre-defined. The goals are just communicating, playing, and relaxing. The need is not set, but arises in the interaction. We see an age-old dream come true: devices that resemble humans or animals, and can interact with us. Examples are the dog AIBO (a robot companion shaped like a dog), the fluffy cuddly toy Furby, the funny My Keepon (a little yellow dancing robot that can dance to the rhythm of music) and the sex robot: all four invite us to play out social and physical interaction. People become attached to the robot and attribute human features to it. This is called ‘anthropomorphism’, i.e. attributing human traits and behaviours to non-human subjects. People even assign robots a psychological and a moral status, which we previously only attributed to living humans [59]. Research shows that young children are much more attached to toy robots than to dolls or teddy bears, and even consider them as friends [90].

Nevertheless, we certainly cannot speak of a success story. The social interaction robots that are currently available are very limited in their social interaction and are very predictable, so many consumers will not remain fascinated for long eventually. This motivates researchers to proceed in order to reach a more efficient and effective interaction [12, 29, 45]. There is a lack of knowledge about the mechanisms that encourage communication between humans and robots, how behaviour occurs between humans and robots, and even how the interaction between people actually works. Such knowledge is critical to the design of the social robot, because its success depends on successful interaction [13]. This research discipline of human–robot interaction is still in its infancy. At this time—and probably within the next ten years—we should therefore consider commercially available social interaction robots like Furby and My Keepon as fads and gadgets whose lustre soon fades rather than as kinds of family friends. How the sex robot will develop is still unknown, but the sex industry and some robot technologists see a great future for this robot and consider the sex robot to be a driving force behind the development of social robots and human–robot interaction research (see, for example, [51]).

In order to let robots interact with humans in a successful manner, many obstacles must be overcome, especially to develop a social robot which has many of the social intelligence properties as defined by Fong et al. [38]: it can express and observe feelings, is able to communicate via a high-level dialogue, has the ability to learn social skills, the ability to maintain social relationships, the ability to provide natural cues such as looks and gestures, and has (or simulates) a certain personality and character. It will take decades before a social robot has matured enough to incorporate these properties, but modern technology will make it increasingly possible to interact with robots in a refined manner. This will turn out to be a very gradual process.

2.2.2 Ethical and Regulatory Issues

2.2.2.1 Emotional Development The entertainment robot is modelled on the principle of anthropomorphism. Since users are strongly inclined to anthropomorphism, robots quickly generate feelings [28]. This raises all sorts of social and ethical questions, particularly the question of what influence entertainment robots have on the development of children and on our human relationships. Sharkey and Sharkey [79] especially question nanny robots for children, as they think it will damage their emotional and social development and will lead to bonding problems in children. Sparrow [83] has already expressed worries about toy robots remaining as ‘simulacra’ for true social interaction. Emotions expressed by robots promise an emotional connection, which they can never give, for emotions displayed by robots are indeed merely imitations, and therein lies the danger, according to Sparrow, “imitation is likely to involve the real ethical danger that we will mistake our creations for what they are not” [83] (p. 317).

2.2.2.2 De-socialisation For Turkle [96], the advance of the robot for social purposes is worrying, and she fears that people will lose their social skills and become even lonelier. She is concerned that children will get used to perfect friendships with perfectly programmed positive robots, so they will not learn to deal with real-life people with all their complexities, problems, and bad habits. These ideas remain speculations, because there has been only limited research on the actual effects of the impact of social robots on children and adults [91]. In addition, Turkle sees the sex robot as a symbol of a great danger, namely that the robot’s influence stops us from being willing to exert the necessary effort required for regular human relations: “Dependence on a robot presents itself as risk free. But when one becomes accustomed to ‘companionship’ without demands, life with people may seem overwhelming. Dependence on a person is risky—because it makes us the subject of rejection—but it also opens us to deeply knowing another.” She states that the use of sex robots leads to de-socialisation.

According to Evans [36], a real friendship between robots and humans is impossible, since friendship is conditional. Intimate friendship, therefore, is a kind of paradox: on the one hand, we want a friend to be reliable and not to let us down, but when we receive complete devotion we lose interest. In addition, Evans argues that we can only really care about a robot when the robot can actually suffer. If robots cannot experience pain, we will just consider them to be objects. This raises further ethical questions, such as whether we should develop robots that can suffer and whether we should grant rights to robots (see [53]).

Little research has been done on socialisation and de-socialisation, but it is important that we think about boundaries; where and when do social robots have a positive socialising effect and where do we expect de-socialisation?

2.2.2.3 Sex with Robots The possibility of having sex with robots may reduce the incidence of cheating on a partner and adultery. There is still the question of whether having robot sex would be considered as being unfaithful, or whether robot sex would become just as humdrum and innocent as the use of a vibrator nowadays [56]. These robots could satisfy people’s desire for illicit sexual practices, such as paedophilia, and could be used in therapy to remedy the underlying problem. According to Levy [52], the sex robot, the ultimate symbol of the automation of lust, may in the future contribute to solving the general problem of sex slavery and the trafficking of women, issues which are currently ignored by many politicians. The sex robot could even be an alternative to a human prostitute. Research in this field is still lacking. Ian Yeoman, a futurologist in the tourism and leisure industry, predicts that the prostitution robot will be introduced [58]. In response to this prospect, Amanda Kloer [49] even sees robots as perfect prostitutes, but she inserts a note:

In a way, robots would be the perfect prostitutes. They have no shame, feel no pain, and have no emotional or physical fall-out from the trauma which prostitution often causes. As machines, they can’t be victims of human trafficking. It would certainly end the prostitution/human trafficking debate. But despite all the arguments I can think of for this being a good idea, I’ve gotta admit it creeps me out a little bit. Have we devalued sex so much that is doesn’t even matter if what we have sex with isn’t human? Has the commercial sex industry made sex so mechanical that it will inevitably become ... mechanical?

Related to the adult-sized sex robot is the issue of sex with child–robots and the associated issue of whether child–robot sex should be punishable. The questions that now arise are whether this child–robot sex contributes to a subculture that promotes sexual abuse of children, or whether it reduces the sexual abuse of children. No country’s legislation establish that sex with childrobots is a criminal offence. National legislators (or, for example, the European Commission) will have to create a legal framework if this behaviour is to be prohibited.

3 Care Robot

3.1 Expectations

A staffing shortage—due to future ageing—is often invoked as an argument for deploying robotics in long-term care. An ageing population is defined as a population that has an increase in the number of persons aged 65 and over compared with the rest of the population. According to the European Commission [35], the proportion of those aged 65 and over is projected to rise from 17 % in 2010 to 30 % in 2060. Moreover, it is expected that people will be living longer: life expectancy at birth is projected to increase from 76.6 years in 2010 to 84.6 in 2060 for males, and from 82.5 to 89.1 for females. One out of ten people aged 65 and over will be octogenarians or older. The growth of the very oldest group will put pressure on care services and will result in an increase in the demand for various services for the elderly: (1) assisting the elderly and/or their caregivers in daily tasks; (2) helping to monitor their behaviour and health; and (3) providing companionship [79]. Care-robot developers have high expectations: in the future, care robots will take the workload away from caregivers. However, the argument that robots can solve staff shortages in health care has no basis in hard evidence. Instead of replacing labour, the deployment of care robots rather leads to a shift and redistribution of responsibilities and tasks and forms new kinds of care [67].

During the next 10 years, care support robots may not widely enter the field of care. The use of care robots must be viewed primarily from the perspective of current development and deployment of home automation (domotics). These smart technologies, which at present are being incorporated widely into our environment, are the prelude to a future home with robots. Domotics allows the tele-monitoring of people; it offers the possibility of using a TV or computer screen at home in order to be able to easily talk with health care professionals [1]. Also, medical data such as blood glucose levels or electrocardiogram (ECG) can be uploaded to the doctor or hospital. We expect that the possibilities of automation will continuously expand and will become supportive of robotic technologies. However, in addition to technological challenges, the challenge is to make domotics and robotics applications cost-effective. This often necessitates many years of innovative trajectories of research, and, especially in the field of long-term care, innovation processes are usually difficult to finance [15].

Despite this difficult process, domotics will increasingly become part of health care practice because of the current trend of decentralisation and the fact that telecommunications technologies will allow people to receive more remote care at home. This may lengthen the period during which senior citizens will be able to live independently at home.

In the long term, robots will enter the field of care. A development such as the Japanese lifting robot, RIBA II, has already brought the use of care robots one step closer. This robot supports human carers in lifting their clients. Apart from robots that support caregivers, there are already robots that allow people to live independently at home for longer, such as Kompai, a robot that gives support to ’dependent’ persons in their private home, which has been developed by the French company Robosoft. This robot has a touch screen and is directly connected to the Internet, thereby enabling contact via videoconferences between a resident and a doctor. This robot on wheels that has no arms can recognise human speech and can talk. The robot understands commands and can perform actions, such as leaving the room on command, playing music or creating a shopping list. Kompai is also able to determine its location within the house and will return independently to its docking station when its batteries need recharging. At the moment, the robot is unable to express human emotions, but Robosoft expects this functionality will be added to Kompai in the future. Robosoft has launched Kompai as a basic robot platform, enabling technicians to continue tinkering so that in time new applications can be developed and added.Footnote 1 Kompai is now deployed and will be further developed within several European projects, e.g. in the Seventh Framework project Mobiserv (An Integrated Intelligent Home Environment for the Provision of Health, Nutrition and Mobility Services to the Elderly). This project examines how innovative technologies help to support the elderly in a user-friendly way so that they can live independently for as long as possible.Footnote 2

3.2 Ethical Issues and Regulatory Issues

3.2.1 Care

The use of robotic technologies in care puts forward the question of to what extent the current and future health care system will have space to give actual care. Care implies concern about the welfare of people, entering into a relationship with them, dealing with their discomforts, and finding a balance between what is good for that person and whatever it is that they are asking for. Robots seem to be the epitome of effective and efficient care: the ultimate rationalisation of a concept that perhaps cannot be captured in sensors, figures, and data. The use of care robots requires a vision of care practice, and the discussion should be about what exactly we mean by ‘care’, taking into consideration aspects such as reciprocity, empathy, and warmth, and the role taken up by technology. In 2006, Robert and Linda Sparrow [85] noted their thoughts about the use of robots in care for senior citizens. Their message is utterly clear: the use of care robots is unethical. They emphasise that robots are unable to provide the care, companionship, or affection needed by ageing people. They see the use of robots by the elderly as an expression of a profound lack of respect for senior citizens. Other ethicists do not immediately reject the care robot, as they see opportunities for care robots, if used in certain conditions and with particular qualifications (e.g. [11, 19, 23]).

3.2.2 Fine-Tuning

Developers should take into consideration the wishes and needs of caregivers as well as those of care recipients in their design process [103]. Technicians are therefore required to make a ‘value sensitive design’, a design which also takes into account the wishes and needs of different groups of users—caregivers as well as care recipients. Both of these users should be involved as early as possible in the design process [106]. The process also requires the use of tele-technologies and home automation for fine-tuning and getting clear coordination with other stakeholders such as general practitioners, hospitals, nursing homes, home health agencies, insurance companies, and family members. This takes us to the point made by Van Oost and Reed [105]: when reflecting on deploying care robots one must not focus only on those persons directly involved—the entire socio-technical context must also be examined.

3.2.3 Privacy

In the short term, ethical issues play a role in home automation. Registering and monitoring the behaviour of care recipients raises privacy issues. Exactly what information is collected on the people being tele-monitored? What does that data say about the daily activities within the household? Who has access to the data that is collected? How long will the data be stored? Are the care recipients aware of the fact that information is being collected about them? Is it justified to deploy these technologies and data-gathering methods when some people, for example because they have dementia, are unaware of the presence of such technologies? These questions about privacy should be taken into consideration by developers and politicians when they are advocating the deployment of both home automation and robotics. According to Borenstein and Pearson [11], the degree of control that the care recipient has over the information collected is important. When a person has actual control over the information collected, this enhances the autonomy of that person. This requires developers, right from the beginning of the design process, to consider the consequences for privacy of their robotic technologies. The challenge is to strike a proper balance between the protection of privacy and the need to keep living at home independently.

3.2.4 Human Dignity

Another important drawback put forward by ethicists is the feared reduction in human contact. Care recipients will no longer have direct contact with human caretakers; they will only have contact with devices or will have remote contact, mediated by technology. The increasing use of robots therefore raises social issues relating to the human dignity of the care recipient. The manner in which robots are deployed proves to be a crucial point. When robots are used to replace the caregiver, there is a risk that care becomes dehumanised [107]. Sharkey and Sharkey [79] also point to the danger of the objectification of care for senior citizens by using care robots. When robots take over tasks like feeding and lifting, the recipients may consider themselves as objects. They foresee the possibility that senior citizens may develop the idea that they have less control over their lives if they receive care from care robots compared to just receiving care from human caregivers. The risk of paternalism comes into play here in terms of the extent to which the robot may enforce actions. The ethical objection of the ‘objectification of the patient’ is consistent with the idea that robots cannot provide ‘real’ care. Human contact is often seen as crucial for the provision of good care [19]. According to Sparrow and Sparrow [85], robots are unable to meet the emotional and social needs that senior citizens have in relation to almost all aspects of caring.

The robot-as-companion technology raises controversial images: lonely elderly people who only have contact with robot animals or humanoid robots. The ethical concerns about the pet robot focus on the reduction of human contact that such technology brings about and the possibility of deceiving, for example, patients with dementia [11, 19, 79, 85, 95]. Sparrow and Sparrow [85] describe care robots for the elderly as ’simulacra’ that replace real social interaction, just like the nanny robot for children (see Sect. 2). In addition, they wonder about the degree to which a robot can provide entertainment for longer periods; their expectation is that once the novelty of the robot has worn off, it ends up being idle after being thrown out in a corner. Borenstein and Pearson [11] are more positive about the deployment of robots; they believe that although robots cannot provide real friendship, the deployment of a companion robot, such as the seal robot Paro, may relieve feelings of loneliness and isolation.

The question underlying all of this is how much right has a care recipient to receive real human contact? Or, to put it more bluntly, how many minutes of real human contact is a care recipient entitled to receive each day? It is important to observe the choice of the care recipient. Some people might prefer a human caregiver, while others may prefer a support robot, depending on which one gives them a greater sense of self-esteem. Robots can thus be used to make people more independent (e.g. by assisting people when showering or going to the toilet) or to motivate them to go out more often to keep up their social contacts. The more control the care recipient has over the robot, the less likely he or she is to feel objectified by the care robot. Thus, the use of robotics should be tailor-made and should not lose sight of the needs of care recipients. We agree with the advice of Sharkey and Sharkey [79], who argue that in the use of robotics for health care a balance must always be sought between increasing the autonomy and quality of life of older people—by allowing them to remain at home longer—and protecting the individual rights of people and their physical and mental well-being.

3.2.5 Competences of Caregivers

In the medium term, the increasing use of care robots puts demands on the professional skills of caregivers [100]. The use of robotic technologies creates a new care practice in which the caregivers get a new role, and their duties and responsibilities will shift [3, 67]. Indeed, working with a lifting robot requires specific skills of caregivers: knowing how to steer the robot and to predict potential failures. Providing care at a distance requires that caregivers are able to diagnose and tele-monitor people via a computer or TV screen and are able to reassure a patient. New skills are also expected of the care receiver. The care receiver should be able to deal with tele-conferencing and with forwarding data messages to a doctor. Obviously, this requires that care professionals have the ability to instruct patients about the technology and to familiarise them with its use. Dealing with robotic technology therefore opens a new chapter in the training of caregivers, so that caregivers may easily cope with it and anticipate the possibilities and limitations of robotic technologies (see also [80]. The entire deployment of care technologies should be re-examined in the context of both practice and policy.

4 Robot Car

4.1 Expectations

The robotisation of the car is in full swing. Advanced Driver Assistance Systems (ADAS) support the driver, but do not yet allow fully automated driving in traffic. The application of driver assistance systems is rapidly developing and is fully stimulated by industry, research institutions, and governments. There are high expectations of these systems regarding safety effects. The available driver assistance systems are probably only harbingers of a major development that will lead to a progressive automation of the driving task. This trend can now be observed. Systems that in principle only advised or warned, as in alerting the driver if they were speeding or unintentionally veering off the roadway, are being further developed into systems that actually intervene, causing the car to return to the correct lane when the driver unintentionally leaves the roadway. In addition, car manufacturers especially compete with each other in terms of comfort and safety, because there is not much more that can be done to improve the quality of cars [43]. Intelligence therefore becomes the unique selling point for a new car.

Science also points to the potential of cooperative systems—in conjunction with traffic management—that operate through connected navigation. Nowadays, much research is being carried out and many of these research projects are funded by the European Union, such as Safespot (2006–2010),Footnote 3 CO-OPerative SystEms for Intelligent Road Safety (COOPERS, 2006–2010),Footnote 4 Cooperative Vehicle-Infrastructure Systems (CVIS, 2006–2010),Footnote 5 and Safe Road Trains for the Environment (SATRE 2009–2013).Footnote 6 The first pilot projects have now been realised, and it is expected that these cooperative systems will lead to less congestion and better use of the road network. The European Commission [32] will propose short-term technical specifications that are required to exchange data and information from vehicle to vehicle (V2V) and vehicle to infrastructure (V2I). This proposed standardisation should result in a push for the further implementation of these systems. ‘Train driving’ (i.e. cars that follow close to each other and exchange information about their speed, position, and acceleration) requires a cooperative driving electronics standardisation so that different car brands can click into the ’train’ plan. Despite the expected positive contribution of cooperative systems—in contrast with driver assistance systems—little research has been done on the safety and possible side effects of cooperative systems. It will take some years before V2V and V2I communication is safe and reliable enough to be used in cooperative driving.

The development of cooperative systems will contribute to the further implementation of autonomous driving. The autonomous driving may have to be applied on motorways with cooperative adaptive Cruise control (ACC), for which V2V communication will be necessary. The infrastructure will not need to change much, because drivers can already get all information about local traffic regulations, traffic congestion, roadworks, and the like via the navigation system or any other in-car information source. Perhaps roadside systems could be placed on the road to guide autonomous driving, especially at motorway slip roads. This semi-autonomous driving allows autonomous driving of the car on certain roads with non-complex traffic situations, such as motorways, but not in places with more complex traffic situations, such as in a city. Scientists consider that this will be realistic by about 2020 [108]; see also the SATRE project). The expected result of this semi-autonomous driving is that cars will become more fuel-efficient, road safety on highways will increase, and traffic congestion will be partly mitigated, especially in shock wave traffic congestion. During autonomous driving, the driver can read a book, use the Internet, or have breakfast, and so on.

The autonomous car was first promised in 1939 by Bel Geddes during the Futurama exhibition he designed for General Motors for New York World’s Fair. With Futurama, Geddes speculated about what society would look like in future. In his book Magic Motorways (1940), he writes, “[t]hese cars of 1960 and the highways on which they drive will have in them devices which will correct the faults of human beings as drivers. They will prevent the driver from committing errors.” In 1958, General Motors’ engineers demonstrated the first ‘autonomous car’. This car was autonomously driven over a stretch of highway by way of magnets attached to the car and wiring in the roadway—also called ‘automatic highways’. In a press release, General Motors proudly announced the result [109]: “An automatically guided automobile cruised along a one-mile check road at the General Motors Technical Center today, steered by an electric cable beneath the concrete surface. It was the first demonstration of its kind with a full-size passenger car, indicating the possibility of a built-in guidance system for tomorrow’s highways. ... The car rolled along the two-lane road and negotiated the check banked turn-around loops at either end without the driver’s hands on the steering wheel.” In 1974, a group of forty-six researchers predicted that these automatic highways would become a reality between 2000 and 2020 [97].

In 2010, it was announced that Google would undertake research on autonomous vehicles. Meanwhile, the company has driven autonomous cars (six Toyota Prius and one Audi TT) for thousands of test kilometres on the Californian public roads. Legal fines were prevented by positioning the drivers’ hands just over the steering wheel, ready to intervene in case of problems. Early in 2011, Google started to lobby in the US state of Nevada about adjusting road traffic regulations. According to Google, autonomously driven vehicles should be legalised and the ban on text messaging from moving autonomous cars should be lifted. Meanwhile, the states of California, Nevada, and Florida are setting ground rules for self-driving cars on the roads [18]. According to research leader Sebastian Thrun, Google hopes that the development will ultimately contribute to better traffic flow and a reduction in the number of accidents. He estimates that the annual number of 1.2 million casualties in the entire world can be reduced by half by the use of the autonomous car [93].

Since 2011, an autonomous car has also been driven in Berlin; it is called Made in Germany and is the successor to the Spirit of Berlin, which participated in the DARPA Urban Challenge in 2007. The car, a modified Volkswagen Passat, is the result of the Nomos car project, subsidised by the German government and implemented by the Artificial Intelligence Group of the Free University of Berlin.Footnote 7 The developers have been awarded a licence to carry out car tests on the roads in the states of Berlin and Brandenburg. The next goal of the developers is to drive the car across Europe. A notable development of this project is that you can order the car with your smartphone. The developers demonstrate a clear vision of the future. The idea is that cars should vanish from the road when they are not driving. In their view, future cars should remain in central car parks until an order call is made. As soon as the call is received, the car, a driverless taxi, sets off for the customer’s location and then picks up the customer and takes them to a destination specified by the customer by smartphone. During the ride, the car’s system can decide whether it picks up other customers it encounters with a matching destination on the planned route. According to the researchers, in a city like Berlin, given the tie-in with existing public transport, private car use can still be efficient with a mere 10 % of the number of cars that now run daily in the city. Hence, the researchers see this development as a trend for ‘greener’ cars.

4.2 Ethical and Regulatory Issues

4.2.1 Acceptance

In several European research projects, research is being carried out into the acceptance of the robotic control of the car—and in particular the acceptance of driver support and cooperative systems, such as in the projects European Field Operational Test on Active Safety Systems (EuroFOT)Footnote 8 and Adaptive Integrated Driver-vehicle InterfacE (AIDE).Footnote 9 It focuses on two questions: (1) how do motorists feel about technology taking over the driving task and (2) will motorists accept interference from these systems? In principle, drivers are hesitant when systems taking over driving tasks, because they often sense initial discomfort in a machine-dominated environment. However, according to a recent Cisco report on the consumer experience within the automotive industry, 57 % of global consumers trust autonomous cars.Footnote 10 Moreover, acceptance grows as motorists have driven them and have come to trust the systems [104]. The RESPONSE project showed that for a successful market introduction of driver assistance systems, the focus should be on convincing the public that the systems are effective and safe [26]. In addition, drivers want to have the ability to personally intervene and to turn off the system.

4.2.2 Skilling Versus de-Skilling

The fact that many drivers come to rely on driver assistance systems makes them less alert. In addition, these systems can lead to de-skilling, so that driving ability may deteriorate. This can lead to dangerous situations at times when the (semi-autonomous) car does not respond autonomously and control should be taken over by a driver who has become less road savvy [27]. Consequently, driver assistance systems require new driver skills. It is important that attention is paid to this, and a possible solution could be that driving with driver assistance systems becomes a mandatory part of the driving licence.

4.2.3 The Better Driver

Autonomous cars by Google and the Free University of Berlin make the driver redundant. Many researchers see the autonomous car as a method of preventing traffic accidents, for conscious or unconscious human error is involved in almost all traffic accidents. Several studies show that more than 90 % of all accidents occur due to human error and that only 5–10 % are the result of deficiencies in the vehicle or the driving environment (see, for example, [14, 25]. Autonomous vehicles have continuous complete attention and focus, keep within the speed limit, do not get drunk, and abstain from aggressive behaviour, and so on. In addition, humans are no match for the technology when it comes to reaction time and alertness, both in routine situations and in critical situations [101]. But before the human factor can be switched off in traffic, the autonomous vehicle must be thoroughly tested in the actual dynamic traffic before safely functioning on the road. This could take many years; predictions range from five to thirty years. In the development process, a good step forward would be fitting autonomous cars with V2V and V2I systems, allowing multiple systems to monitor traffic situations at the same time.

4.2.4 Safety

The greatest benefit of these systems, sought by the European Commission [33] in particular, is in traffic safety. The Commission aims to halve the total number of road deaths in the European Union by 2020 as compared to 2010. This is a very ambitious goal, which in our opinion can only be achieved by rigorous measures such as mandating a number of driver assistance systems. The European Commission [34] has already made the Anti-lock braking system, electronic stability control, and eCall (a warning system that automatically alerts emergency services in case of an accident) compulsory. The next systems that could qualify for such an obligation are the ACC system (adaptive Cruise control system) and the forward collision warning. Cars equipped with both systems could potentially affect up to 5.7 % of the accidents causing injury on motorways. The European Commission seeks to achieve the strategic goal of halving the number of road casualties by stronger enforcement of traffic regulations. Speed is also a basic risk factor in traffic and has a major impact on the number of traffic accidents [4]. This could easily be addressed by using a far-reaching variant of the intelligent speed assistance (ISA) system so that drivers cannot violate the speed limit. It is not expected that this variant will be implemented easily, because public acceptance of such an intrusive system is quite low [60]. The car is considered by many as an ‘icon of freedom’, and an intrusive system restricts the freedom of the motorist. The question is whether a moral duty exists to mitigate this freedom in the interest of road safety. The European Commission could have an important role here by making this variant of the ISA system compulsory. Although this would result in a lot of resistance from both drivers and political parties, it would be an effective measure.

4.2.5 Security

Cooperative systems have to deal with the security of the information and communication network. Cooperative driving, for example, necessitates both hardware for communication and a link to the engine management system so that the vehicle can control its own speed. A disadvantage is that the system is fragile and the car could become the victim of hacking attempts. American researchers at Center for Automotive Embedded Systems Security (CAESS) have shown that it is possible to hijack and take over full control of the car [17]. In theory, malicious people could take over a motorway junction, completely disrupting traffic or causing injury. The European research project Preserve (Preparing Secure Vehicle-to-X Communication Systems), started in 2011, deals with the development and testing of a security system.Footnote 11 Securing data is complicated because cryptology increases the necessary information flow but at the same time restricts the available bandwidth.

4.2.6 Privacy

The most pressing problems are going to arise in the short term in the field of privacy. Increasing the enforcement of road traffic rules can easily be accomplished via V2I systems that can monitor a driver’s behaviour, allowing the owner to be fined automatically for violations of traffic rules. In addition, insurance firms may introduce premiums for drivers who drive safely and use monitoring. This is still considered an invasion of privacy, but the question is whether in the long term traffic safety will beat privacy. Therefore, the remaining question is whether politicians can keep their promise that the eCall system will remain ‘dormant’ and that this system will not be used as an electronic data recorder for tracking criminals or for fining drivers who violate traffic regulations [30]. This danger is real, as shown by CCTV cameras in Amsterdam that were only supposed to be keeping an eye on polluting trucks, and were installed for monitoring only those, but that were subsequently also used for other purposes, such as whether owners of number plates had an unpaid fine [55].

4.2.7 Autonomous Car

Fully autonomous driving will not be a realistic picture before 2020, even though this is predicted by both General Motors and Google. However, given the developments in the field of car robotics, it seems inevitable that the autonomous car will become commonplace. A more likely estimate is that these systems will function by around 2030. The launch will probably take place via taxi systems, as outlined by the German researchers in the Auto Nomos project, and it will be possible to call an autonomous car on a mobile phone and it will be waiting for its passenger at stations and theatres, and the like (see also [14]).

The social impact of the introduction of the autonomous car could be very significant. Visions of the future of the autonomous car now lead to different scenarios, sometimes even diametrically opposed ones (see, for example, [7]). Therefore, policymakers and politicians must anticipate these possible scenarios. What exactly will the implications be for public transport, car ownership, road safety, and road use, et cetera? This should be investigated for each scenario, so that policymakers and politicians can design the road of the future and discourage undesirable developments at an early stage.

Yet it is high time that various parties (governments, industry, research institutes, and interest groups) considered the technical and legal aspects of cooperative and autonomous driving. The autonomous car will force regulators to rethink the car and driver relationship, and will possibly place more emphasis on the regulation of the car than of the driver. For instance, instead of certifying that a driver can pass a road test, the state might certify that a car can pass a test, upending the traditional drivers’ licensing system. Questions also arise about liability for accidents, since the technology that makes an autonomous car is an after-market system. So if it hits something, does the fault lie with the manufacturer or the technology company that modified the car? These aspects require time to resolve and, after all, will needlessly slow the introduction of car robotisation by a few years if no moves are made now.

5 Police Robot

5.1 Expectations

Within the global expansion in robotics, the police domain is an important application. This aspect is largely fuelled by developments in the field of military robots. In the field of police robots, the USA and Japan are making clear headway compared to Europe. We may conclude that the application of robotics within the police domain is still in an experimental, exploratory phase. Two applications are central: carrying out surveillance and disarming explosives. In most countries, the police have a number of ground and airborne robots outfitted with smart cameras. Over the past decade, a large increase in smart cameras has been observed in public areas, and this increase has been reinforced, especially since 2001, by a higher priority being given to investigation and law enforcement by the police. The ground and airborne police robots are mobile unmanned systems with limited autonomy that can be deployed for a specific task, and they are tele-operated [61]). Robots can be particularly useful for the police when they use their authority to gain access to any location. A robot can, for example, be used to bring objects to or pick up objects in a so-called hot area that the police cannot enter in order to observe situations that police officers cannot see. In this way, robots are strengthening the core missions of enforcement and investigation. They also increase the safety of the police officers, who can thus avoid dangerous places. For example, in the US, the remotely controlled so-called V-A1 robots are already deployed, especially in the state of Virginia [39]. They are equipped with cameras, chemical detection equipment, and a mechanical arm to grab objects. These robots enable agent operators to assess dangerous situations from a distance without running risks themselves.

With regard to another police function, providing assistance to citizens, the use of robots is further away in the future. We may call it a prelude to the social robot, because for them to succeed on the street, the quality of their direct interaction with citizens will be crucial. In the long term, robots could come into service in police work, for example in the form of humanoids, operating visibly and having contact with the public. One can imagine a robot traffic cop, a robot as part of riot control, or as a police officer on the street, just patrolling, providing a service, and keeping its eyes open in the street or in a shopping mall. For the moment, these types of applications in robot technology still face major challenges, given the complexity of the social and physical space in the public domain [10]. In addition, it is of great importance that police robots are accepted and obeyed by people who are panicky or violent. According to Salvini et al. [71], violence against police robots may constitute an obstacle to the deployment of these robots, because they somehow encourage young people to act aggressively towards these robots.

In Japan, we see experiments with robots in the street; they are called ‘urban surveillance robots’ or ‘urban robots’, and they can take over some of the responsibilities of a police officer, such as identifying criminal or suspicious behaviour and providing a service to the public. The Reborg-Q was tested in shopping malls, airports, and hotels. It is a behemoth of 100 kg, is 1.50 m long, and is equipped with cameras, a touchscreen, and an artificial voice. It is a tele-operated robot that can also supervise pre-programmed routes independently. The Reborg-Q can identify the faces of unwanted visitors when it is on patrol and respond by alerting human guards.Footnote 12

5.2 Ethical and Regulatory Issues

5.2.1 Privacy Versus Safety

A tricky issue with robots and intelligent cameras is the violation of privacy [75]. It is possible that in the short term the government will monitor our daily activities 24 h a day for the sake of safety. This creates tension between ensuring privacy and ensuring security. The very essence of the rule of law is that there should be a balance between protecting the public by the government and protecting it from the government. Without privacy protection, the government is a potential threat to the rights of its own citizens. At the same time, the government loses its legitimacy when it cannot guarantee safety for its citizens from outside threats from other parties. Therefore, it is important that it is clear when data may be collected, what data may be collected and stored, and for what purpose these robots and intelligent cameras may gather data, and that a clear distinction remains between the monitored public space and private life [72]. Moreover, the risk of the manipulation of sound and image recordings and the risk that data might end up in the hands of the wrong people should also be factored in.

5.2.2 Skilling Versus de-Skilling

The increasing deployment of police robots means that police officers must acquire new skills. The operation of tele-operated police robots and the performance of police actions using robotic technology both need different operational and strategic requirements from the police personnel. The downside is that a loss of essential police skills may occur—skills acquired through extensive training and experience—as a result of getting used to the deployment of police robots, after which police officers would be less able to intervene in serious problems that cannot be solved using robots [44].

5.2.3 Deployment

A legal complication regarding the deployment of airborne robots for police purposes is that it is not yet clear how these robots can be deployed in accordance with existing laws and regulations [111]. From the perspective of the British police, the national Aviation Acts and regulations can be seen as obstacles in relation to the use of air robots for specific purposes [54]. For any type of aircraft there are laws and regulations to ensure safe traffic in the airspace. In this respect, one may question whether airborne robots seamlessly fit into existing laws and regulations. Does this technological innovation require a new, not yet existing category within the Aviation Acts and regulations? In other words, which restrictions of a national legal framework apply regarding, for example, the deployment of UAVs over a festival or a fire in some dunes?

Moreover, deployment reliability is an issue. This means that the robots must not pose any danger to civilians and that certain safety rules must be met. A failure of an airborne robot that hovers above a festival crowd could result in disastrous consequences. Extensive security measures must therefore be taken before making the decision to deploy these police robots.

In the short term, research will be needed on the laws and regulations relating to the deployment of police robots, especially the airborne robots, so that the applications of robots for police purposes can be adapted. It is important that safety in the air and on the ground is not compromised. To this end, governments could make a contribution by establishing safety standards for police robots.

5.2.4 Armed Police Robots

Armed police robots raise important ethical questions. In the short and medium term, it is not expected that there will be armed autonomous police robots, but there are ongoing experiments with armed tele-operated police robots. In the USA, for example, there are concrete plans for tele-operated police robots equipped with a taser. They can, among other tasks, be used for crowd control. At the moment, when this police robot with a taser attacks a ’suspect’, it creates a new, dangerous situation that is different from the situation in which an agent arrests a suspect using a Taser. Neil Davison, an expert in the field of non-lethal weapons at the University of Bradford, says: “The victim would have to receive shocks for longer, or repeatedly, to give police time to reach the scene and restrain them, which carries greater risk to their health” [64].

The emergence of armed police robots requires a political stance. This political position should be based on the desirability and legality of deploying robots for police tasks, in comparison with non-robotic alternatives, for public safety, and on the implications of these robots for the exercise of police violence.

6 Military Robot

6.1 Expectations

During the invasion of Iraq in 2003, no use was made of robots, as conventional weapons were thought to yield enough ‘shock and awe’. However, the thousands of American soldiers and Iraqi civilians killed reduced popular support for the invasion and made the deployment of military robots desirable. By the end of 2008, there were 12,000 ground robots operating in Iraq, mostly being used to defuse roadside bombs, and 7,000 reconnaissance planes or drones were deployed [82]. The robot is therefore a technological development that has a great influence on contemporary military operations, and this is seen as a new military revolution. New robotics applications are constantly sought these days and are developed in order to perform dull, dangerous, and dirty jobs and to improve situational awareness, but also in order to kill targets.

During the last decade, advances have been made in the development of the armed military robot. From 2009, more ground-based pilots—or cubicle warriors—have been trained to use armed unmanned aircraft than have been trained as fighter pilots [102]. The expectation is that unmanned aircraft will increasingly replace manned aircraft, and in the medium term will even make manned aircraft obsolete. To this end, further technological developments are required, such as the development of self-protection systems for unmanned systems, so that they become less vulnerable, and the development of sense and avoid systems, so they can be safely controlled in civilian airspace. In the short term, we do not expect the introduction of armed ground robots on the battlefield. These have already been developed, but are deployed with little success in conflict zones.

A trend we are observing in military robotics development is a shift ‘from in-the-loop to on-the-loop to out-the loop’ [78]. We have seen that cubicle warriors are increasingly being assigned monitoring tasks rather than having a supervisory role. The next step would be for the cubicle warrior to become unnecessary and for the military robot to function autonomously. The autonomous robot is high on the US military agenda, and the US Air Force [98] assumes that by around 2050 it will be possible to fully deploy autonomous unmanned combat aerial vehicles (UCAV). Given current developments and investment in military robotics technology, this US Air Force prediction seems not to be utopian but a real image of the future. The wish to promote autonomous robots is mainly driven by the fact that tele-guided robots are more expensive, firstly because the production cost of tele-guided robots are higher and secondly because these robots incur personnel costs as they need human flight support. One of the main goals of the Future Combat Systems programme, therefore, is to deploy military robots as ‘force multipliers’ so that one military operator can run a multiple large-scale robot attack [76]. To this end, robots are programmed to cooperate in swarms so they can run coordinated missions. In 2003, the Americans deployed the first test with 120 small reconnaissance planes in a mutually coordinated flight [50]. This swarm technology is developing rapidly, and will probably become military practice in a few years’ time. This is a future in which the automation of death will become reality.

6.2 Ethical and Regulatory Issues

6.2.1 The Better Soldier

Using armed military robots can also allow greater risks to be taken than with manned systems. During the Kosovo war, for the sake of the pilots’ safety, NATO aircraft flew no lower than 15,000 feet so that hostile fire could not touch them. During one specific air raid from this elevation, NATO aircraft bombed a convoy of buses filled with refugees—although they thought they were hitting Serbian tanks [9]. These tragic mistakes could be prevented by deploying unmanned aircraft that are able to fly lower, because they are equipped with advanced sensors and cameras, allowing the operator to better identify the target. Furthermore, by using military robots, operators are in a better position to consider their decisions. These robots could be used in very dangerous military operations, including home-to-home searches in cities where the situation is unclear. In such a situation, a soldier has to determine within a fraction of a second who is a combatant and who is a citizen and neutralise those persons who form an immediate threat—before they open fire themselves. A military robot is able to enter a building without endangering soldiers or civilians. An operator will open fire on someone only when that person has shot at the robot first. According to Strawser [86], it is morally reprehensible to command a soldier to run the risk of fatal injury if that task that could be carried out by a military robot. In circumstances like this, Strawser holds that the use of armed military robots is ethically mandatory because of the ‘principle of unnecessary risk’.

Arkin [6] suggests that in the future autonomous armed military robots will be able to take better rational and ethical decisions than regular soldiers, and will thus prevent acts of revenge and torture (see also [87]. At war, soldiers are exposed to enormous stress and all of its consequences. This is evident from a report that includes staggering figures on the morale of soldiers during the military operation Iraqi Freedom [65]. For example, the report found that less than half of the soldiers believed that citizens should be treated with respect, that a third felt that the torture of civilians should be allowed to save a colleague, and that 10 % indicated that they had abused Iraqi civilians. Furthermore, less than half said they would betray a colleague who had behaved unethically, 12 % thought that Iraqi citizens can be regarded as rebels, and less than 10 % said that they would report an incident in which their unit had failed to adhere to fighting orders. The report also shows that after the loss of a fellow soldier, feelings of anger and revenge double the number of abuses of civilians, and that emotions can cloud soldiers’ judgement during a war.

6.2.2 Abuse and Proliferation

The first signs of an international arms race in relation to military robotics technology are already visible. All over the world, significant amounts of money are being invested in the development of armed military robots. This is happening in countries such as in the United States, Britain, Canada, China, South Korea, Russia, Israel, and Singapore. Proliferation to other countries, e.g. by transfer of robotics technology, materials, and knowledge, is almost inevitable. This is because, unlike with other weapons systems, the research and development of armed military robots is fairly transparent and accessible. Furthermore, robotics technology is relatively easy to copy and the necessary equipment for armed military robots can easily be bought and is not too expensive [82].

In addition, much of the robotics technology is in fact open source technology and is a so-called dual-use technology; thus, it is a technology that in future will potentially be geared towards applications in both the military and the civilian market. One threat is that in future certain commercial robotic devices, which can be bought on the open market, could be transformed relatively easily into robot weapons.

It also likely that unstable countries and terrorist organisations will deploy armed military robots. Singer [81] fears that armed military robots will become the ultimate weapon for ethnic rebels, fundamentalists, and terrorists. Noel Sharkey [77] also predicts that soon a robot will replace a suicide bomber. International regulations on the use of armed military robots will not solve this problem, as terrorists and insurgents disregard international humanitarian law. From this perspective, the American researcher and former fighter pilot Mary Cummings [22] outlines a doomsday scenario of a small, unmanned aircraft loaded with a biological weapon being flown into any sports stadium by terrorists.

An important tool to curb the proliferation of armed military robots is obviously controlling production of these robots by global arms control treaties. A major problem with this, however, is that major powers such as the US and China are not parties to these treaties. In addition, legislation in the UN framework is needed in the field of the export of armed military robots to combat the illicit trafficking of armed military robots and to set up licences for traders in armed military robotics technology.

6.2.3 Hacking

Another danger is that military robots will be hacked or may become infected with a computer virus. In October 2011, US Predators and Reapers were infected by a computer virus [74]. This virus did nothing but keylogging: it forwarded all the commands to and from these drones and sent it elsewhere. This particular incident was not serious, but the danger of such viruses could become immense. Through hacking, others could take over unmanned combat aircraft, and viruses can disrupt the robots in such a way that they become uncontrollable, and they could then be hijacked by people acting illegally. This is a real and present danger: in 2009, the Iraqis intercepted drone video feeds using US$26 off-the-shelf software [42].

6.2.4 Targeted Killing

On 30 September 2011, in a northern province of Yemen, Anwar al-Awlaki, an American citizen and a senior figure in Al Qaeda in the Arabian Peninsula, was killed by an American drone, as was a second American citizen, of Pakistani origin, whom the drone operators did not realise was present [20]. So Anwar-al-Awlaki was killed with the President’s intent but without Anwar-al-Awlaki having been charged formally with a crime or convicted at trial. Khan [48] argues that indiscriminate killing of suspected terrorists by drone attacks in Pakistan cannot be justified on moral grounds, because these attacks do not discriminate between terrorists and innocent women, children, and the elderly. This tactical move of using the drones is counterproductive and is “unwittingly helping terrorists”. He states that the US’s international counterterrorism efforts can only be successful by devising a clear strategy: adopting transparent, legitimate procedures with the help of Pakistan to bring the culprits to book and to achieve long-term results. More scholars [2, 16, 110] hold the opinion that the United States probably carries out illegal targeted killings in Pakistan, Yemen, Somalia, and elsewhere. The government must be held to account when it carries out such killings in violation of the Constitution and international law.

6.2.5 Loss of Public Support

Although armed military robots, due their precision robotics, are much more accurate in hitting their target, their use may eventually lead to more victims being killed, because they will be deployed much faster and more frequently, even in civilian areas. The sharp increase in air strikes by unmanned aircraft could ultimately lead to more casualties than before. According to estimates made by the New America Foundation [63], the US air strikes in Pakistan using UAVs increased from nine air attacks between 2004 and 2007, resulting in 100 victims, including nine civilians, to 118 air raids in 2010, with 800 casualties, of whom 46 were civilians. The Bureau of Investigate Journalism (TBIJ) estimates that in Pakistan more than 750 civilians were killed by drones between 2004 and 2013.Footnote 13 In addition, the use of armed unmanned systems is often considered as a cowardly act by locals, and every civilian victim of such an automated device will be used by insurgents for propaganda. All this leads to a loss of psychological support among the local population, despite the fact that support is an essential tool for providing a positive contribution, for example in stabilising a conflict (see, for example, [48]).

According to a recent report of Human Rights Watch on civilian casualties in Afghanistan, taking “tactical measures to reduce civilian deaths may at times put combatants at greater risk”, yet they are prerequisites for maintaining the support of the local population [46] (p. 5), which on turn is something that the success of the mission in Afghanistan depends on. Clearly, a mounting civilian death toll is something that might very well strengthen the resentment against the West and might make recruitment easier for both the insurgency and the terrorist groups that the coalition troops are trying to fight. For example, Ghosh and Thompson [41] describe how, in Waziristan, the region in Pakistan afflicted by a lot of drone attacks, the use of unmanned aircraft is certainly seen as dishonourable and cowardly, which does not contribute to ‘the winning of the hearts and minds’. Just before he was killed by a UCAV, Baitullah Mehsud, the Pashtun commander of the Pakistani Taliban, even claimed that each drone attack “brings him three or four suicide bombers” [41], mainly found among the families of the drones’ victims.

6.2.6 Responsibility and Autonomy

Military robots may in the future have ethical constraints built into their design—a so-called ethical governor—which suppresses unethical lethal behaviour. Sponsored by the US Army, Arkin [5] has carried out research to create a mathematical decision mechanism consisting of prohibitions and obligations derived directly from the laws of war. The idea is that future military robots might give a warning if orders, according to their ethical governor, are illegal or unethical. For example, a military robot might advise a human operator not to fire because the diagnosis of the camera images tells it that the operator is about to attack non-combatants. An argument for rejecting this approach to what constitutes an ethical design is that ethical governors may form a ‘moral buffer’ between human operators and their actions, allowing them to tell themselves that the military robot took the decision. According to Cummings [21] (p. 30), “[t]hese moral buffers, in effect, allow people to ethically distance themselves from their actions and diminish a sense of accountability and responsibility.” A consequence of this is that humans might then simply show a type of behaviour that was desired by the designers of the technology instead of explicitly choosing to act this way, and thus might over-rely on military robots (the “automation bias”). This can lead to a dangerous situation; because the technology is “imperfectly reliable”, the human operator must catch the instances when some aspect of the technology fails.

According to several authors (for example, [8, 37, 78, 84], the assumption about and/or the allocation of responsibility is a fundamental condition of fighting a just war. Ethical governors might blur the line between non-autonomous and autonomous systems, as the decision of a human operator is not the result of deliberation but is mainly determined or even enforced by a military robot. In other words, human operators do not have sufficient freedom to make independent decisions, which makes the attribution of responsibility difficult. The moralising of the military robot can deprive the human operator of controlling the situation; his future role will be restricted to monitoring. The value of ‘keeping the man in-the-loop’ will then be eroded, and will be replaced by ‘keeping the man on-the-loop’. This can have consequences for the question of responsibility. Detert et al. [24] have argued that people who believe that they have little personal control in certain situations—such as those who monitor, i.e. who are on-the-loop—are more likely to go along with rules, decisions, and situations even if they are unethical or have harmful effects. This implies that it would be more difficult to hold a human operator reasonably responsible for his decisions, since it is not really the operator that takes the decisions, but a military robot system [69].

The idea that this might become real follows from a report from The Intercept by Glenn Greenward in which he states that the US military and the CIA often rely on data from the National Security Agency’s electronic surveillance programmes for targeted drone strikes and killings. According to a former drone operator, the NSA often identifies targets based on controversial metadata analysis and mobile phone-tracking technologies: “Rather than confirming a target’s identity with operatives or information on the ground, the CIA or the US military then orders a strike based on the activity and location of the mobile phone a person is believed to be using.” In fact, no human intelligence is used to identify the target, so the technology “geolocates” the SIM card or handset of a suspected terrorist’s mobile phone, allowing the operator to push the button in order to kill the person using the device [73]. Furthermore, the drone operator said that while the technology has led to the deaths of terrorists and others involved in attacks against US forces, innocent people have “absolutely” been killed because of the NSA’s reliance on the surveillance methods.

An important question regarding autonomous armed robots is whether they are able to meet the principles of proportionality and discrimination. Compliance with these principles often requires empathy and situational awareness. A tele-operated military robot can be helpful for an operator because of its highly sophisticated sensors, but it does not seem feasible that in the next decade military robots will possess the ability to empathise and exercise common sense. Some scientists wonder if this will be possible at all because of the dynamic and complex battle environments in which these autonomous robots will have to operate (see, for example, [76]).

For the development of military robotics technology, a broad international debate is required about the responsibilities of governments, industry, the scientific community, lawyers, non-governmental organisations, and other stakeholders. Such a debate has not been realised because of the rapid development of military robotics so far. A start was made with a debate during an informal meeting of experts at the United Nations in Geneva in May 2014 on lethal autonomous weapons systems [99]. The necessity of a broad international debate is shown by the contemporary technological developments in military robotics, which cannot always be qualified as ethical. The deployment of armed military robots affects the entire world, and it is therefore important that all stakeholders with a variety of interests and views enter into a mutual debate (see also [57]). The starting point for this debate must be the development of common legal and ethical principles for the responsible deployment of armed military robots.

7 Conclusion

In this section we summarise our findings (see Table 1). Our summary is based on some key characteristics of new robotics that evoke various social and ethical issues: (1) short-, medium- and long-term trends in the field of robot technology, (2) social gains of robotisation, (3) robots as information technology, (4) the lifelike appearance of robots, (5) the degree of autonomy of robots, (6) robotic systems as dehumanising systems, and (7) governance issues relating to new robotics.

Table 1 Summary of the various social and ethical issues related to new robotics
Full size table

7.1 Technology Trends

Both in Europe and the United States, the goal of developing robotics for the domestic environment, care, traffic, police, and the army is embraced by policymakers and industry as a new research and societal goal. There is an aim for technology to enable an increasing amount of autonomous moral and social actions. Thus, a radical development path unfolds, namely, the modelling, digitisation, and automation of human behaviour, decisions, and actions. This development is at least partially legitimated by speculative socio-technical imaginaries, such as multifunctional, autonomous, and socially and morally intelligent robots.

In the short and medium term, developments in the field of new robotics are mainly characterised by terms such as ‘man in-the-loop’ and ‘man on-the-loop’, which indicate that robotic systems are increasingly advising human operators on which action must be taken. Firstly, this signifies the digitisation of various previously low-technology fields, such as the sex industry and elderly care. For example, in the coming decade a combined breakthrough of home automation and tele-care is expected, which will place the caretaker and patient in a technological loop. The experimentation with care robots must be seen from this perspective. Secondly, in high-technology practices, such as the automotive industry and the military, we see a shift from ‘man in-the-loop’ to ‘man on-the-loop’ and even ‘man out-of-the loop’. It is not unlikely, for example, that autonomous cars will gradually become common by around 2030.

7.2 Expected Social Gains

Robotisation presents a way of rationalising social practices and reducing their dependence on people (cf. [68]). Rationalisation can have many benefits: higher efficiency, less mistakes, cheaper products, and a higher quality of services, et cetera. Rational systems aim for greater control over the uncertainties of life, in particular over people, who constitute a major source of uncertainty in social life. A way of limiting the dependence on people is to use machines to help them or even to replace them with machines. As Ritzer [68] (p. 105) argues: “With the coming of robots we have reached the ultimate stage in the replacement of humans with nonhuman technology.”

The development and use of robotic systems is often legitimated by the fact that they will take over “dirty, dull, and dangerous” work. Some claim that the ‘principle of unnecessary risk’ leads to an ethical obligation to apply robotics in certain situations. Strawser [86] believes it is morally unacceptable to give a soldier an order that may lead him to suffer lethal injuries if a military robot could perform this same task. This principle of unnecessary risk is also applicable to driving cars and sex robots. Given the many degrading circumstances in prostitution, should the presence of a reasonable technological alternative not lead to an ethical obligation to replace human prostitutes by sex robots? And if robots can be better drivers that cause far less severe traffic accidents, aren’t we obliged to gradually replace the human driver by technology?

7.3 Robots as Information Technology

New robotics also brings up many ethical, legal, and social issues. Some of the issues are related to the fact that robotic systems are information systems. That means that social issues, such as privacy, informed consent, cyber security, hacking, and data ownership also play a role in robotics. Because much of the robotics technology is open source, this makes it easier for the technology to proliferate and for terrorist organisations to abuse it. The fact that within new robotics great attention is given to improving the interface between machines and humans brings new questions with it, especially in the area of privacy. The vision of affective computing, for example, can only be realised if the robot is allowed to measure and store data about our facial expressions.

7.4 Lifelike Appearance of Robots

The lifelike appearance of robots may raise various issues. To improve the interaction between humans and robots, robotics explicitly makes use of the ability of man to anthropomorphise technology. This raises questions about the limits within which this psychological phenomenon may be used. Some fear that a widespread future use of socially intelligent nanny robots may negatively influence the emotional and social development of children. Others warn of the possibility that persuasive social robots may influence or fake people, and may even try to deceive them. The possibility of building child-like robots raises the question of whether child–robot sex should be punishable.

7.5 Degree of Autonomy of Robots

Another characteristic of robotics which raises many issues concerns the degree of autonomy of the robot or, more precisely, the degree of control that is delegated by the user to the machine. The safety of tele-operated or autonomous robotic systems is an important topic. Tele-operation and semi-automation of tasks, whether in the field of care, driving, the police, or the army, require new skills of caregivers, drivers, police officers, and soldiers. When a controller delegates various tasks to the robot, this immediately raises questions in the field of responsibility and liability. The shift towards ‘man on- the-loop’ raises the question of to what extent the user still receives enough information to make well-informed decisions. The future option of taking people completely out of the loop raises the question of what kind of decisions and actions we want to leave to a robotic machine? Do we want machines to autonomously make decisions about killing people or raising children?

7.6 Robot Systems as Dehumanising Systems

A central, almost existential, worry that is linked to the development of new robotics is related to the notion of dehumanisation. This happens when robotisation as rationalisation overshoots its mark and lead to socio-technical systems that may become anti-human or even destructive of human beings. With regard to social robotics, there is a fear that people will gradually grow accustomed to pleasant, customised relationships with artificial companions and will lose interest in dealing with complex, real people. With respect to care robots, some fear that the use of robots will ultimately lead to a reduction of human contact between the patient and the caregiver and an objectification of the patient. The mechanisation of human encounters is at the core of this debate, regardless of whether it relates to sex or the act of killing.

7.7 Governance of New Robotics

In this article several large-scale socio-technical transitions were described, in particular the shift towards domotics and tele-care, smart mobility, and robotic warfare. These important transitions need to be guided by widespread public and political debate, and efforts should be made to regulate all kinds of social and ethical issues that have been identified in this article. In particular, with regard to the (proliferation of the) use of armed military robots, a broad international debate is needed. We also spotted some more specific issues that need attention: how can society deal with armed police robots, how can one deal with the issue of sex with child–robots, can sex robots be an alternative to human prostitutes, and what is the influence of social interaction technology on our social capital?

8 Epilogue

The introduction of new robotics is paired with an enormous human challenge. Making use of opportunities and dealing with their social, legal, and ethical aspects calls for human wisdom. Trust in our technological capabilities is an important part of this. But let us hope that trust in technology will not get the upper hand. Trust in humans and the acceptance of human abilities, but also human failures, ought to be the driving force behind our actions. Like no other technology, new robotics is inspired by the physical and cognitive abilities of humans. This technology aims to copy and subsequently improve human characteristics and capacities. It appears to be a race between machines and us. Bill Joy [47] is afraid of this race, because he fears that humans will ultimately be worse off. His miserable worst-case scenario describes “a future that does not need us”. The ultimate goal of new robotics should not be to create an autonomous and socially and morally capable machine. This is an engineer’s dream. Such a vision should not be the leading principle. Robotics is not about building the perfect machine, but about supporting the well-being of humans.

Our exploratory study shows that social practices often possess a balance between ‘hard’ and ‘soft’ tasks. The police take care of law enforcement and criminal investigation, and also offer support to citizens. The war must be won, but so also must the ‘hearts and minds’ of people. Care is about ‘taking care’ (washing and feeding) and ‘caring for’ through a kind word or a good conversation. We enjoy making love, but we especially want to give and receive love. Robotics can play a role in the functional side of social practices. Hereby, we must watch out that the focus on technology does not erase the ‘soft’ human side of the practice in question. Such a trap can easily lead to appalling practices: inhumane care, a repressive police force, a hardening of our sex culture, and cruel war crimes.

A central question is what decisive position people should take in the control hierarchy. The European Robotics Technology Platform looks to robots to play mainly a supporting role: “Robots should support, but not replace, human caretakers or teachers and should not imitate human form or behaviour” [31] (p. 9). Robotics does not exist for itself, but for society. Robotics ought to support humankind, not overshadow it. This begins with the realisation that new robotics offers numerous opportunities for improving the lives of people, but also that there is sometimes no space for robots. A robot exists that can play the trumpet very well. And yet it would be disgraceful if, for example, the daily performance of the Last Post in Ieper (Belgium), in memory of victims of the First World War, were to be outsourced to a robot. Furthermore, we must watch out that trust in technology does not lead to technological paternalism. Even if, in the very distant future, there are robots that are better at raising our children than we are, we must still do it ourselves. An important aspect of this is the notion of personal responsibility and the human right to make autonomous decisions and mistakes. Even if a robot can do something better than a human can, it could still be better that the human continues to do it less well.