1 Introduction

Rapid developments in the areas of sensor and computer technology are expected to lead toward radical changes in the automotive domain. Throughout the twentieth century, driving has been an activity conducted by human drivers. However, in the twenty-first century, developments in automotive technology may relieve people from the need to drive themselves. Instead, the vehicle is forecasted to become a robot that performs most or all aspects of the driving task on its own. Apart from the obvious industrial interests in developing the technology, having to do with gaining or maintaining market share and competitive advantage, the development is motivated by societal and user benefits including safety, convenience/productivity, comfort, sustainability, efficiency (throughput), and mobility for all. Concerning safety, it is clear that present-day traffic has a safety issue. In 2017, more than 25,000 people were killed in traffic on EU roads [1]. Furthermore, it has been argued that most traffic accidents are due to human error (estimates are between 70 and 90%.) Among other things, in situations that require swift action, human drivers suffer from slow reactions, which may be further delayed if the driver does not expect the event or is distracted. Therefore, it is argued that, if driving is automated and the human driver is taken out-of-the-loop, driving can be safer and could eventually result in zero or near-zero fatalities. The current paper aims to argue that such automated driving will create characteristics that do not fully satisfy the users’ needs and that it would be preferable to develop technology that allow users to influence the behavior of the system.

Automating all aspects of the driving task is an enormous challenge, extending over years or even decades. The SAE has coined a taxonomy of levels of automation characterizing the transition. Level 0 (for taxonomic completeness) consists of manual driving. In Levels 1 and 2, one or more driving tasks are automated, such as (adaptive) cruise control and lane keeping (lateral control), but the driver needs to monitor the behavior of the vehicle and the surrounding traffic at all times in order to regain control in case of a situation that the system cannot handle. In Level 3, conditional automation, the system can handle specific driving situations itself, such as normal conditions on the highway. Furthermore, the system is able to identify situations that it cannot handle, in which case the system requests the driver to regain control. Therefore, as long as the vehicle is in automated mode, the driver does not need to monitor the behavior of the vehicle and the surrounding traffic and can engage in non-driving-related activities (NDRA) [2]. Level 4, high driving automation, is similar to level 3, except that the system is able to handle situations where the driver does not respond to the system’s request to regain control (e.g., if the driver has suffered a heart attack), by maneuvering the vehicle safely to the emergency lane, bringing it to a standstill and notifying an emergency crew. Finally, level 5, full automation, to be achieved in some—more or less distant—future, denotes the case where the vehicle can handle all driving situations: No monitoring is required, there are no situations that require the driver to take over control, and no qualified driver is required, so that the vehicle may even drive around without occupants (e.g., to drive to a parking lot after delivering the user to the office). Based on safety considerations, level 5 may be considered the holy grail of driving automation. It is reasoned that, if the human is taken out of the loop, human error can no longer be the cause of accidents, so that 70%–90% of present-day accidents can be avoided. This vision is concretized through concept vehicles that no longer contain traditional controls like the steering wheel and pedals.

There are good reasons to question this vision, however. Most notably, the behavior of the system that is preferable from a system perspective may not always be preferable from a user perspective. One example has been discussed by Lin [3]: Noting that it is prohibited to cross an uninterrupted line, Lin discussed the case of a rock lying on the road that requires the vehicle to cross the uninterrupted middle line in order to pass the rock. The system may need human assistance to tell it that it is okay to cross the line, as the system itself would act by the rule that it is forbidden to cross the line. More generally, it is argued that, even at level 5, automated vehicles will act by the rule, and this may not always be a good strategy, so that the technology should be developed such that either it can solve such situations by itself or that it can leave room for human intervention.

To discuss what such intervention might look like, Michon’s taxonomy for levels of control involved in the driving task is applied [4]. Strategic control concerns aspects of the driving task that relate to the overall purpose of driving: Where do people want to go (destination), what time do people want to arrive (desired time of arrival), what kind of route do people want to take (scenic route, fastest route, optimized for NDRA, etc.). Tactical control concerns aspects of the driving task that relate to deciding the maneuvers to implement the overall purpose: taking turns, overtaking, choosing the proper lane, lane changes, etc. Operational control concerns the planning and execution of concrete actions that implement the maneuvers decided at the tactical level and that are executed by manipulating the traditional controls (steering wheel, pedals).

Now, it may be readily assumed that even at level 5 automation the user is still in charge of decisions at the strategic level: choosing the destination, the type of route (scenic vs. fastest), and the desired time of arrival. However, it may already be questioned if the authority is absolute. If the desired time of arrival implies violating the speed limit, the authority of the user may be constrained by the system, which complies with and acts by the rule. At the operational level, it may readily be assumed that control may be left to the system. Most likely, it is preferable to avoid burdening the user with the task of actually manipulating the traditional controls. At the tactical level, the situation is less straightforward. It may be assumed that, under normal conditions, the decision about which tactical maneuvers to conduct might stay with the vehicle, but that, in specific situations, the user might want to tell the system to execute (or consider) a certain maneuver, e.g., to take over another vehicle.

In this paper, it is investigated whether the development of automated driving systems should aim toward taking the driver completely out of the loop, or whether situations can be identified where users might want to stay involved in the decision-making process. The analysis will focus primarily on tactical maneuvers. After having presented related work in the next section, a conceptual analysis of relevant use cases is presented to evaluate the desirability of user involvement in the decision process in the age of automated driving.

2 Related Work

The idea of shared control and responsibility between humans and automated systems has been proposed before. Parasuraman et al. [5] promoted human-centered automation and proposed a hierarchy of different levels of human involvement in the decision process, ranging from choosing between different options proposed by the system to vetoing a decision of the system or not being involved at all. In the automotive area, relevant work has been done by Flemish and colleagues [6,7,8,9]. This work is mostly focused on investigating how to complement human and system ability and to ascertain graceful transitions. A theoretical framework for shared control is proposed in Refs [10,11,12,13], linking it to the more general topic of human–system collaboration, but the question of what shared control might concretely mean for vehicle automation is not treated in much detail. Herrmann and Schmidt [14, 15] discuss the so-called intervention interfaces that allow users to influence and alter the behavior of highly automated systems, but the only automotive application that they discuss is automated parking.

Much previously mentioned work aims to investigate how humans and automation may complement each other and compensate for each other’s limitations. Furthermore, many studies focus on shared control at the level of operational control, whereas for the current purpose it is more relevant to investigate shared control at the level of tactical control. The question of whether people want to influence the behavior of an automated system at all was investigated by Habibovic et al. [16]. They developed three different interfaces allowing participants in a study in a driving simulator to influence tactical maneuvers of the system such as overtaking and parking, and found that indeed participants wanted to have the opportunity to influence the system’s behavior. The participants did not engage in NDRA, however, so that the ecological validity may be limited.

A different approach to taking users’ preferences into consideration when shaping the behavior of an automated vehicle is through a learning approach, where the system learns the preferences of the user by observing the human driver [17,18,19]. However, in the first place, it may be questioned whether this approach is able to deal with situationally conditioned variations in driver preferences. In addition, findings from research indicate that the preferred driving style of an automated driving system may not always coincide with the driver’s preferred driving style. Yusof et al. [20] found that both drivers with an assertive driving style and drivers with a defensive driving style preferred a more defensive driving style for automated vehicles. As one of the participants put it: “I would not like to be driven by someone who drives the way I drive”.

Finally, Wada et al. [21] discussed shared control, in which the driver is kept in the loop, as a way to prevent loss of driver skill. The division of labor is based on the task difficulty.

3 Approach

In the following, a number of use cases are analyzed that denote situations where the user may want to influence the behavior of the system. For each use case, the current situation (manual driving), where the driver deals with the situation in a certain way, and the future situation, where the automated driving system behaves in a different way, are described. The latter is based on the assumption that automated driving systems prioritize safety and therefore will drive in a defensive way and act by the rule (see, e.g., [22, 23]). In each case, it is speculated that the user might wish to influence the behavior of the automated driving system. From these observations, speculative requirements may be derived for how the technology should be developed to meet user needs. In Sect. 5, implications are considered for how the human machine interface might support such shared control. Finally, issues raised by the proposed concept of shared control are discussed and topics for further research are identified.

The use cases were selected on the basis of traffic observations and discussions with colleagues and domain experts. Although the inventory of use cases is not claimed to be exhaustive, it is intended to cover a representative set of relevant cases. Furthermore, as the current technological developments toward automated driving systems head toward out-of-the-loop automation, it suffices for the current argument if there are a few cases where out-of-the-loop automation will not lead to satisfactory outcomes.

This discussion of use cases and future options is seen as the first step toward understanding intervention-based interaction and opportunities for shared control in automated cars.

4 Use Cases

4.1 Use Case A: Influencing the Route Taken by the Vehicle

Manual driving: Sometimes, a passenger in a vehicle, either privately or in a taxi, notices that the driver is taking a certain route, while he/she knows the area and knows that there is a shortcut. In this case, the passenger may suggest the driver to take the shortcut. Similar cases are where the passenger knows the area and knows that the selected route is less attractive, either because of the traffic situation (e.g., very busy at that time of day) or because of the boring scenery. Again, the passenger may suggest an alternative route to the driver.

Automated driving: It may be assumed that even with fully automated driving (Level 5), if the vehicle has no user controls for tactical and operational control, there is no way to influence the route taken while on the road.

Requirement: It should be possible to influence the planned route and tell the system to take a different route.

4.2 Use Case B: Influencing the Acceleration for Takeover Maneuvers

Manual driving: When the driver is driving on the highway and approaching slow traffic (like trucks), the normal action is to overtake. However, when the left lane is busy, it is not always possible to overtake immediately. In that case, the driver needs to slow down and adjust the speed to that of the slow traffic ahead and to change lanes when there is a gap in the left lane. Once there is a gap, the driver changes lanes and accelerates quickly in order not to hinder upcoming traffic.

Automated driving: If it is assumed that the acceleration behavior of an automated vehicle copies the behavior of an adaptive cruise control (ACC) system, the vehicle might show undesirable behavior in this situation. An issue with ACC is that, when the vehicle has slowed down, the vehicle accelerates rather slowly. Thus, when changing to the left lane, the vehicle will hinder the upcoming traffic. A second aspect of this situation is that an automated vehicle may be hesitant to use a gap between vehicles in the left lane to change lanes, because it considers the gap size unsafe. In case of manual driving, the driver, overlooking the situation, may decide that the gap size considered unsafe by the automated system is still manageable.

Requirement: It should be possible to help the ACC and tell the vehicle to accelerate more strongly. Also, the user might want to tell the hesitant system to initiate a lane change.

4.3 Use Case C: Overtaking Slow Traffic Before the Highway Exit

Manual driving: The driver needs to take the approaching highway exit and prepares for the exiting maneuver by sorting to the right lane. However, he/she notices that there is slow traffic in the right lane (e.g., two trucks in a row). The driver may judge that the distance to the exit is still long enough to take over the trucks before the exit and decides to take over.

Automated driving: It may be assumed that an automated vehicle is more defensive and stays in the right lane.

Requirement: It should be possible to tell the system to initiate a takeover maneuver (provided the system can accelerate properly, as stipulated in the previous use case).

4.4 Use Case D: Speeding Up to Meet the Desired Arrival Time (e.g., After Having Suffered a Delay Through a Traffic Jam)

Manual driving: It may happen that the driver chooses a departure time such that, under normal traffic conditions, he/she arrives at the destination in due time for an appointment. However, due to a traffic jam the driver has a small delay. In this case, the driver may want to speed up after getting out of the traffic jam to compensate for the lost time and still make it to the appointment in time—even if this implies having to violate the speed limit. A similar case is when drivers are driving to the airport to deliver someone who needs to catch a flight and are delayed by dense traffic.

Automated driving: It may be assumed that the automated system will act by the rule and not violate the speed limit.

Requirement: It should be possible to tell the system to violate the speed limit (assuming that adequate performance of the automated vehicle is still possible in the resulting speed range).

4.5 Use Case E: Violating the Speed Limit (e.g., to Take Your Pregnant Wife to the Hospital)

Manual driving: In certain situations, like when taking his pregnant wife who is about to give birth to the hospital, the driver may not care much about the speed limit, but instead cares about taking his wife to the hospital as quickly as possible by driving at high speed. Since it is highly likely that the driver in this case is stressed, it would pose an excellent example for automating the driving task, especially since the combination of a stressed driver and relatively high speed appears prone to accidents.

Automated driving: Like in use case D, it may be assumed that the automated system will not violate the speed limit. However, if the system does not allow the user to adjust the speed, the alternative is to turn the automation off, which does not seem a sensible action for obvious reasons. Note: It has been suggested that an automated vehicle may need to be designed such that it will be able to violate the speed limit by 10 mph to adjust to the surrounding traffic [23], but that may not cover for this use case.

Requirement: It should be possible to tell the system to violate the speed limit (assuming that safe performance of the automated vehicle is still possible in the resulting speed range).

4.6 Use Case F: Crossing a Busy Pedestrian Crossing

Manual driving: This use case concerns a situation where there is an uncontrolled pedestrian crossing, i.e., there is a pedestrian crossing but there are no traffic lights, neither for the pedestrians nor for the approaching traffic. In such cases, the pedestrians always have right of way at the pedestrian crossing. Some of these pedestrian crossings are very busy at certain times of the day, so that the approaching traffic is queuing up. In such cases, it is not uncommon for drivers to use small gaps in the pedestrian flow to push themselves through the flow, even if it hinders the free passage of newly arriving pedestrians. Or, even more strongly, drivers may create a gap by pushing themselves slowly and gently into the pedestrian flow, hoping that the pedestrians will understand.

Automated driving: A fully automated vehicle will likely act in a defensive way and act by the rules, and not exploit small gaps, let alone create gaps by gently pushing into the pedestrian flow. This may result in waiting times of several minutes.

Requirement: It should be possible for the user to tell the system to exploit a small gap, or slowly and gently push into the pedestrian flow, even if the pedestrians have right of way. Of course, this may also require vehicle–pedestrian communication through an external HMI.

4.7 Use Case G: Adjusting the Crossing Behavior at an Intersection

Manual driving: Certain intersections may result in deadlock cases. One example is shown in Fig. 1. Here, car A needs to give right of way to car B, car B needs to give right of way to car C, and car C needs to give right of way to car A again. Here, it needs to be negotiated who is going first. Either a rule is applied that vehicles that go straight ahead have right of way over vehicles that make a turn, in which case B would be the first to go. Or the situation may be resolved by individual differences in assertiveness, in which case the driver with lowest assertiveness (or most courtesy) will be the last to go. Similar cases concern 4-way stops, if four vehicles arrive at the same time and busy intersections, where one wants to enter or cross the main road from a secondary road and has to give right of way to traffic on the main road.

Fig. 1
figure 1

Deadlock situation at an intersection

Full size image

Automated driving: Since fully automated vehicles will show defensive behavior, an automated vehicle will most likely wait until other, manually driven vehicles have moved, or, in the case of the busy intersection, until there is a gap of sufficient length. Note: It has already been suggested that automated vehicles may have to be designed such that they are more assertive [22].

Requirement: It should be possible to tell the vehicle to start moving.

4.8 Use Case H: Adjusting the Driving Style of the Vehicle (e.g., to Accelerate more Quickly at the Traffic Light, Changing the Braking Behavior or the Cornering Behavior)

Manual driving: Surveys indicate that the majority of drivers are careful/calm drivers [24]. Still, quite a few drivers exhibit a driving style that includes behavior such as speedy acceleration after a traffic light turns green; maintaining speed when approaching a traffic light and then braking abruptly; not or minimally reducing speed when approaching a side street based on knowledge about the statistical likelihood of traffic events (whether traffic will enter the road from a side street). Furthermore, the driving style may be conditional on the situation.

Automated driving: It may be assumed that automated vehicles will adopt a rather defensive driving style in order to maximize safety. While such a defensive driving style may be acceptable to certain drivers (like the proverbial 70-year-old lady), it may not be acceptable to all drivers.

Requirement: It should be possible to adjust the driving style to one’s own preferences. Note: As was already mentioned above, the driver’s preferred driving style may not be the preferred driving style for a passenger [20], so this requirement needs to be treated with caution.

4.9 Use Case I: Allowing a Vehicle Coming from the Opposite Direction to Enter a Drive or Parking Lot, or Allowing a Vehicle Leaving a Drive or Parking Lot to Enter the Street

Manual driving: This use case is illustrated in Fig. 2. Car A, coming from the opposite direction, needs to turn into a drive or parking lot. The vehicles behind car A are going straight ahead, but need to wait until car A has cleared the lane. The cars in the right lane have been waiting for the traffic light and start moving once the traffic light turns green. If there is a large queue behind car B, car B may decide not to start driving when car C has moved forward but instead to wait and give car A the opportunity to enter the drive, so that the traffic from the opposite direction can move on. Alternatively, car A may want to leave the drive or parking lot to enter the street. In this case, the decision to give right of way to the vehicle that wants to enter the road is not motivated by the consideration to facilitate the throughput, but only to be courteous and reduce the waiting time for the car that is waiting to leave the drive.

Fig. 2
figure 2

Vehicle A waiting to enter a parking lot or drive. R/L: Right and Left lane

Full size image

Automated driving: Like in use case C, a fully automated vehicle will likely act by the rules and in this case not yield to the vehicle waiting in the other lane, therewith reducing the overall throughput.

Requirement: It should be possible to tell the vehicle to wait and let another vehicle enter or leave a drive before starting to move.

4.10 Use Case J: Giving a Pedestrian Right of Way Where There Is No Pedestrian Crossing

Manual driving: In a normal street where there is no pedestrian crossing, vehicles have right of way over pedestrians who are waiting on the sidewalk to cross the street. However, it may be observed that sometimes drivers decelerate or stop completely and invite the pedestrian by means of a gesture to cross the street, showing politeness or courtesy. Motivations for this behavior may be that it is raining and the pedestrian is getting wet while the driver is sitting comfortably dry in his/her vehicle, or because the driver wants to be courteous to an elderly person [25].

Automated driving: Like in use cases C and I, a fully automated vehicle will likely act by the rules and in this case not yield to the pedestrian, so that the user may experience the vehicle as a blunt agent.

Requirement: It should be possible to tell the vehicle to slow down or stop, and yield to the pedestrian, satisfying the user’s desire for courtesy.

4.11 Use Case K: Giving a Bicyclist Right of Way When She/He Has No Right of Way

Manual driving: In the Netherlands, at an intersection of two main roads, bicyclists who come from the right have right of way over vehicles coming from the left. However, if the bicyclist comes from the left, the vehicle coming from the right has right of way. Now, it may be observed that drivers coming from the right sometimes give right of way to bicyclists coming from the left. Motivations may be similar to the ones in the previous use case, for instance, that it is raining or that the bicyclist is an elderly person [25, 26], or because it takes less physical effort for a driver to brake and wait and start driving again than it takes for the bicyclist to brake, stand still and start cycling again.

Automated driving: Like in use cases C, I and J, a fully automated vehicle will likely act by the rules and in this case not yield to the bicyclist, so that the user may experience the vehicle as a blunt agent.

Requirement: It should be possible to tell the vehicle to wait and yield to the bicyclist, satisfying the user’s desire for courtesy.

5 Implications for HMI

To investigate the implications for the HMI, first we summarize the requirements emerging from the use cases in Table 1.

Table 1 Requirements emerging from the use case analysis
Full size table

Use cases A, D and E involve influencing the behavior of the system at the strategic level, while use cases B, C, F–K concern influencing the system at the tactical level. Furthermore, use cases B, C and H involve influencing different aspects of the driving style of the vehicle, and the same may be said about many of the other use cases, e.g., use cases G and F involve intervening to avoid long waiting times. Finally, requirements D, E, F, I, J and K involve cases where the user tells the vehicle to deviate from the rules, either to meet the desired time of arrival (D, E), to avoid long waiting times (F), to be courteous and optimize the throughput (I), or just to be courteous (J, K).

With respect to the use cases that involve deviating from the rules, a relevant insight was proposed in Ref [27]. Building on the Dreyfus and Dreyfus model for skill acquisition and skilled behavior, Benner [27] proposed that, at the level of expertise, people do no longer just rely on rules, but are able to go beyond the rules and act as the situation requires, based on a holistic understanding of the situation. In other words, always acting by the rules is not characteristic of expert behavior. From this reasoning, it follows that fully automated vehicles will most likely act like competent or proficient agents (levels 3 and 4, respectively, of the Dreyfus and Dreyfus taxonomy), but not like experts. On the other hand, decision making in traffic requires expert understanding and action—if not in general, certainly for the user cases considered above. This expert understanding and behavior needs to be supplied by the user.

The question, then, is when and how the user should be enabled to influence the behavior of the vehicle and intervene in its normal way of acting. Here, a distinction is made between user influence through customization and ad hoc intervention. In the case of customization, the user may influence the behavior of the system by adjusting settings determining the system’s behavior throughout a full journey or for all future rides. In the case of ad hoc interventions, the behavior of the vehicle is influenced on the fly for individual situations.

Customization: Aspects of the driving style may be adjusted by the user even before departure. For example, the user may adjust the settings such that the vehicle accelerates more strongly when the traffic light turns to green or when executing a takeover maneuver. The HMI could offer these as separate options, or it could offer more generalized settings like Defensive/Relaxed versus Assertive/Energetic. When in Assertive/Energetic mode, the vehicle will accelerate more vehemently, brake more abruptly, and show more assertive behavior at intersections (the question of how the user would know what concrete behavior the different driving styles entail certainly needs attention but will be left aside for the moment). Alternatively, the user might choose a Courteous/Polite option to give right of way to pedestrians and bicyclists when they have no right of way.

Ad hoc intervention: It seems unlikely that all requirements can be covered through (generalized) settings, the more so because many decisions about how to behave are dependent on the situation. For instance, it is unlikely that all pedestrians and bicyclists should be given right of way when they don’t have right of way. Therefore, it is likely that ad hoc intervention will be needed in addition to, or instead of customization. In order to further analyze what this might entail, the following scenario is considered. “Anne is driving home from the office in her fully automated vehicle. Since she is in a good mood and is not in a hurry, she has set the vehicle to Relaxed mode. She is listening to music and browsing through a magazine. The rain ticks gently on the roof of the vehicle, making her feel cozy.” In the scenario, if the vehicle is in automated mode, the user may be involved in a NDRA and have low awareness of the traffic situation. Now, if there is a pedestrian who wants to cross the street where there is no pedestrian crossing, Anne is likely to miss the opportunity, as she is absorbed in the magazine, even if, when driving manually, she would yield to the poor pedestrian who is getting wet. In this case, the system needs to identify the opportunity for yielding to the pedestrian and notify the user. Obviously, there is a window of opportunity within which the user needs to tell the system how to behave, otherwise the opportunity has passed.

For other use cases, the window of opportunity may be less critical. For instance, the vehicle in use case F is waiting for a busy pedestrian crossing. Even if the system does not inform the user, she may notice herself that the vehicle has been standing still longer than normal and look up from the NDRA. But also in these cases, it might be desirable if the system informs the user about the situation and asks the user how it should behave. Thus, in addition to the requirements emerging from the use cases, indicating which behavior the user should be able to influence, a requirement is added that the system should be able to identify relevant situations, to inform the user that there is an opportunity for intervention, and to ask what should be done, often under time-critical conditions (since otherwise the window of opportunity has closed again).

While terms such as Inform and Ask appear to favor speech-based interaction, hence, more generally, a conversational interaction style for the HMI, the fact that the situations may be time-critical argues against a speech interface. Conversations take time and are good for interaction that is not time-critical, while screen-based interactions enable the user to quickly see what is going on. Thus, the most promising direction appears to be a multimodal interface that employs speech-based interaction when time is not critical and screen-based interaction supplemented by non-speech audio when time is critical. Obviously, the balance between speech- and screen-based interactions depends on the width of the window of opportunity. For use case C (taking over slow traffic at the highway before the exit), the window of opportunity might be wider than for use case K (yielding to a bicyclist). Clearly, further research is needed here.

6 Conclusion and Discussion

Developments in automated driving are heading toward fully automated driving, where the driver is out-of-the decision-making loop. However, the analysis of relevant use cases leads to the conclusion that it might be preferable to consider an outcome of the development process whereby users/occupants have the opportunity to influence the decision process. While fully automated driving, where the user is completely out-of-the-loop, is advocated by industry and promoted by the argument that it will banish fatalities or, more generally, accidents, it is believed that a technology that is not designed to fit the needs and preferences of the users may be rejected by the customers. Furthermore, implications for the human machine interface concerning how the involvement of the user might be realized are outlined, within which a distinction is made between customization (influencing the behavior of the vehicle by means of adjustable settings) and ad hoc influencing, where the user is involved in the decision process on the fly. Once the desirability of shared control has been plausibly argued, it needs to be investigated through experiments whether this is indeed what users want. Once that has been established, the HMI needs to be designed and evaluated.

Obviously, the outcome of this analysis raises a number of questions and issues. In the first place, the potential conflict between the preferences of the individual users and the societal goal of reducing the number of fatalities is a clear case of a social dilemma. A likely way in which the dilemma may be resolved by policy makers is that the preferences of the users, to the extent that they are in conflict with the societal goals, have to give in. Individual customers might then decide not to acquire or use the technology, therewith impeding the achievement of the societal goals. It should be noted, however, that the ability of the user to influence the outcome of the decision process does not automatically lead to a conflict between the preferences of the users and the societal goals and to less safe traffic. For one part, the outcome of the decision process should lead to behavior that is still within the capabilities of the system to handle the situation, because the operational control always remains with the system. For instance, in use case C, where the user tells the system to overtake slow traffic in order to arrive at the exit of the highway sooner, the system should still check whether it is safe to change lanes. For another part, allowing users to influence the behavior of the automated driving system might be the wisest option, if the alternative is that users turn the automation off, like in use case E (taking your pregnant wife to the hospital).

Engineers may claim that allowing users to influence the decision-making process may lead to unpredictability of the behavior of automated vehicles, so that algorithmic control is impeded. However, as stated above, technology that restricts the users in adjusting it to their needs, is unlikely to be successful. Furthermore, as was also stated above, the outcome should always be within the capabilities of the system to handle the situation.

Another issue is that it is by no means clear that users might want to influence the behavior of the vehicle, certainly if it implies ad hoc intervention. Surveys indicate that users are eager to engage in NDRA such as texting (as they are doing already now with manual driving), and it should be established whether users mind to be interrupted from their NDRA by notifications about opportunities for influencing the behavior of the vehicle. This is clearly a topic for further investigation, and it might turn out that there are individual differences and that the acceptance of such interruptions is also dependent on the situation. Furthermore, there might also be cultural differences. Some of the use cases involve courteous behavior, and there may be traffic cultures that do not value courteous behavior very much, so that in these cases the concerned use cases would not apply.

Also, all use cases are based on scenarios where there is a single user who wants to influence the behavior of the vehicle. While indeed many trips involve a single occupant, what about multiple occupants?

Furthermore, there are legal and insurance issues. Car manufacturers are working out the legal and insurance implications of fully automated driving, based on the assumption that, if the vehicle is driven by algorithms, the developers of the algorithms will be held accountable in case of mistakes. They might argue that, if the user is given the opportunity to influence the behavior of the vehicle, at least some of the responsibility should be transferred to the user. It can be easily seen that this may lead to heated discussions about which decisions by whom caused an incident.

Finally, one might argue that the discussion about whether users should be given the opportunity to influence the behavior of the vehicle is very much bound to the present temporal context. It might be that the discussion mainly stems from the fact that, at present, drivers have the opportunity to behave according to their preferences and that a future generation who is not familiar with driving as it exists at present might just take the technology as is. However, even for new technologies that have come into the market, like personal computers and mobile phones, users have experienced the desire to tune the technology to their needs, and there is little reason to think that it might be different for vehicle automation.

Many of the points that have been mentioned in the previous paragraphs require further research and insights. For some of the points, empirical research may be conducted to acquire the necessary insights. For others, it appears more a matter of “time will tell”. Rather than summarizing the points and discussing which points belong in which category, two additional topics should be mentioned that require further research and can be addressed by empirical methods. In the first place, the division of labor between customization and ad hoc intervention will have a profound effect on the user experience, and simulator studies and on-road studies may be conducted to investigate this division of labor. Which decisions may be influenced through generalized settings and which decisions may be influenced through ad hoc intervention? In the second place, it needs to be investigated how often there might be an opportunity or need to influence the behavior of the vehicle. Again, on-road studies may be conducted to collect data concerning this point. Thus, while the current analysis provides plausible arguments for the desirability of shared control, empirical research needs to be conducted to investigate how many opportunities there are for shared control, whether shared control is indeed desirable, and if so, how it should be implemented concretely.