Modeling driver-vehicle interaction in automated driving

In automated vehicles, the collaboration of human drivers and automated systems plays a decisive role in road safety, driver comfort, and acceptance of automated vehicles. A successful interaction requires a precise interpretation and investigation of all influencing factors such as driver state, system state, and surroundings (e.g., traffic, weather). This contribution discusses the detailed structure of the driver-vehicle interaction, which takes into account the driving situation and the driver state to improve driver performance. The interaction rules are derived from a controller that is fed by the driver state within a loop. The regulation of the driver state continues until the target state is reached or the criticality of the situation is resolved. In addition, a driver model is proposed that represents the driver’s decision-making process during the interaction between driver and vehicle and during the transition of driving tasks. The model includes the sensory perception process, decision-making, and motor response. The decision-making process during the interaction deals with the cognitive and emotional states of the driver. Based on the proposed driver-vehicle interaction loop and the driver model, an experiment with 38 participants is performed in a driving simulator to investigate (1) if both emotional and cognitive states become active during the decision-making process and (2) what the temporal sequence of the processes is. Finally, the evidence gathered from the experiment is analyzed. The results are consistent with the suggested driver model in terms of the cognitive and emotional state of the driver during the mode change from automated system to the human driver.

1 Related works Parasuraman [46] presents a model for the levels of humanmachine interaction that employs a human-centered perspective.The proposed model defines automation in four distinct classes based on the simple model of human information processing that includes four stages of sensory processing, perception, decision making, and response selection.It follows that automation can be applied to information acquisition, in which data are collected from the environment; information analysis, which involves extracting features of the input data; decision and action selection, where next actions are recommended to the driver; and action implementation, with the automated system responding directly to the driving situation.The use of a human-centered model for the interaction levels facilitates the design and diagnosis of the driver-vehicle interaction concept.

Driver-vehicle interaction
In automated vehicles, driver-vehicle interaction (DVI) is not merely limited to interface design but is responsible for information processing and transition in dynamic, complex situations.The H-metaphor [20] is a proposed interpretation of DVI.Inspired by horse riding, the H-metaphor resembles the driver to the rider and the automated vehicle to the horse.In this simulation, the automated vehicle is assumed to interact appropriately with the environment, be predictable, exhibit situationally appropriate behavior, have a multimodal interface, and assist humans.Although the H-metaphor is a simplification of the DVI, it is limited to SAE Level 2 [45] and it is challenging to generalize it to all driving scenarios.
Marberger et al. [36] propose a holistic model for the transition process in SAE Level 3 [45] from automated driving to manual driving and assign several phases to the transition process: automated mode with AD compatible driver state, takeover mode with the transition of driver state, a post-transition mode where the driver intervenes and stabilizes the control of the vehicle.The driver state transition means the reorientation of the driver state from non-driving related task (NDRT) or any other non-attentive state to a wakeful attentive driver state.The driver intervention [9] refers to the deactivation of the automated mode by the driver, which can be issued in distinguished ways depending on the system design.The control stabilization interval is an additional time window required by the driver to gain the driving precision and to increase the control performance to the average driving performance of the individuals.
A general approach to DVI should cover all levels of automation and interaction and address situational and automa-tion-related failures.Four of the main failures de-tected in the human-machine relationship [24] are loss of expertise as a consequence of assistance systems, complacency or overreliance of automation, trust and confidence built on user experience, and loss of adaptability to the environment caused by the human-out-of-loop phenomenon.Hoc [24] introduces human-machine cooperation where each agent (driver or vehicle) has a goal and can interfere with the other agent in a way that it can manage the interference by cooperating in planning and action.Four requirements for efficient cooperation are [64] mutual predictability of driver and automated system, directability of actions, shared situation representation with mutual intention, and calibrated reliance on automation to avoid overand under-trust.

Driver model in interaction concept
The collaboration of human and technology requires precise product design based on the psychological and physiological principles of the user.Since the development of driver assistance systems and automated vehicles, DVI has become a focus in the design process.One of the aims of the DVI is to keep the driver in-the-loop when necessary and to transfer the driving task step by step from the automated system to the human driver [19].Flemisch et al. [19] provide general guidance for the design of human-machineinterface (HMI) to form a suitable mental model of the user over the automated system and emphasizes the necessity of verifying the driver's activity level before the task transition request.The driver state assessment component monitors the driver directly through cameras and indirectly by recording driver performance and detects driver inattention due to driver distraction and drowsiness [52].Even though these two elements are crucial variables, identifying the driver state requires more aspects to cover the complex structure of the human being.Three of the existing HMIs are mentioned below, all of which aim to increase the driver's mode awareness.
The first HMI is Continental's automated assistance in roadworks and congestion (ARC), which has a visual modality in the instrument cluster and center console to inform the driver about the level of automation, and haptic feedback on the accelerator pedal to indicate to the driver when the current velocity exceeds the maximum speed.The second HMI is Volvo Technology's automatic queue assistance (AQuA), which has three levels of automation: manual driving, longitudinal assistance system, and automated driving.AQuA is limited to 30 km h −1 and indicates the level of automation and the extent to which the driver is supported.The third HMI is the temporary autopilot (TAP) [47,48] of Volkswagen.TAP has three modes similar to AQuA, but it is designed for higher speeds of up to 130 km h −1 .

K
By transition of driving tasks in SAE Level 3 of automation, the driver state can be divided into three categories: sensory state, motor state, and cognitive state, which are evaluated under a specific arousal level and motivational condition of drivers [36].Even though a driver model is not explicitly specified in this study, the assessment of the current driver state and the target state for the driver are mentioned as essentials for modeling the transition process.Furthermore, the concept of driver availability [36] is proposed as a temporal quantity that identifies at each time step whenever the driver has sufficient time budget for overtaking or not.Driver availability can be influenced by three main factors that affect driver state.First, the NDRT, which the drivers choose to perform during automated driving, has an impact on their sensory state [41].Depending on the modality of activity, the driver's visual perception performance may change.An auditory task concentrates the driver's gaze on the middle of the road [63] and a visual task redirects the driver's gaze from the driving scene to the NDRT.Second, the driver's characteristics, such as experience [31], cognitive capacity [28], and risk tolerance [42], personalize each driver's intervention performance.Third, the way the takeover request (TOR) [22] is presented also affects driver performance.Sensory latency, perceived urgency [43], and the time required to maintain situation awareness depend on the TOR design.The transition process starts with automated driving (AD), where the driver has an AD compatible driver state [36].
In the project "personalized, adaptive cooperative systems for highly automated cars (PAKoS)" collaboration between the human driver and the automated system is planned through driver monitoring, activity estimation, design of the HMI, and transition control [18].The driver state is defined as the body pose of the drivers [37], which is observed by RGB-and depth-cameras during all driving modes, from manual to automated driving.The recognition of driver activity includes information about driver alertness [67], which plays a crucial role in road safety.Activity detection can also help to increase driver comfort by implementing various control signals such as music or light.However, the mental state of the driver cannot be fully detected by behavioral measurements.Communication between the human and the vehicle also benefits from the detection of driver gestures.Furthermore, the prediction of the driver's next action can prevent hazardous situations caused by driver errors.Therefore, the gathered camera data is processed with distinguishe-d algorithms to classify driver activity [8,51,58,62,65].Then, the results from interior 3D models and convolutional neural network-based models are compared.To integrate driver characteristics into the interaction process, a user profile and subprofiles [18] are introduced, which are the key part of a mobile phone application.The architecture of the user profile comprises three levels: Persona, which is the personal information of the driver; user needs, which explain the driver's preferences; product applications, which represent the user requirements and the manufacturer-dependent application parameters.To include specific configurations that are defined separately by the driver for specific situations, such as family trips, the subprofiles are added to the mobile phone application, as well.The transition of the driving task from an automated system to a human driver is identified in two phases.The first phase is the preparation of the driver [17,49,50] where the driver is informed about the intention of the automated vehicle in the second phase of the transition.This process is realized by a haptic seat, visual aides on the head-up display (HUD), and auditory announcements [18].The second phase [34] is supporting the driver to overtake control of the vehicle.In this phase, a game theory approach [59] is utilized to realize collaborative driving based on haptic shared control.The interaction is based on a differential game between the human driver, the automated system, and the vehicle.
Manstetten et al. [35] restrict the driver state to two variables, distraction and sleepiness.Distraction is measured by eye-tracking and facial features.Assessment of sleepiness is simply done by measuring the PERCLOS [66] of drivers which is a measure of eyelid openness.Monitoring these quantified driver state variables, a driver model is defined which detects the driver's inattention through filtering, feature extraction, and distinguished classification methods.In addition, the classifier receives the criticality of the driving situation from an environment model as well.Furthermore, the data that the HMI presents to the driver is a further input of the driver model to achieve a classified driver state.The detected driver state is then fed into a designed Attention and Activity Assistance system (AAA) [32].Depending on the input signal, the AAA makes decisions, sends messages to the other components and interacts with the driver.The AAA is able to detect distraction, prevent monotony, recommend breaks or route adjustments, and detect and prevent sleepiness.The present contribution gives a comprehensive DVI model in automated driving by equipping the feedback control structure [10] with a driver model.In the next section, the fundamental aspects are explained.In Sect. 3 the structure of the feedback control for the DVI concept is examined in detail.The proposed driver model is described in Sect. 4. Sect. 5 illustrates an experiment performed in a driving simulator.Subsequently, the results obtained from the experiment are mapped to the proposed driver model to discuss the conformity of the model.Finally, in Sect.6 the limitations of the experiment are explained and possible next steps are named.

Fundamental issues
The review of the available literature points to the demand for a general framework for DVI, in which all influencing parameters are considered simultaneously and in realtime.A comprehensive model for DVI can lead to a unique structure that is useful in all driving situations and for all automation levels.Besides, the interaction should be personalized for each driver to take into account the individual differences.The present contribution is based on the interaction method proposed by [10], which suggests an online feedback control as a method for DVI.This structure takes into account the driver state and the situation criticality, and adapts the TOR in real-time according to these factors by receiving online feedback from them.Additionally, the proposed DVI offers the possibility to define a driver model and integrate it into the interaction procedure.
The interaction is a decision task for drivers.The decision can result in an action, such as overtaking driving task, or it can only lead to a change in the driver state.Previously, decision-making was considered as a cognitive process involving thinking, computation, and problem-solving.Recently, however, several theories support the importance of emotions in decision-making [54].The degree of pleasure in the emotion influences the chosen strategy of information processing in terms of top-down processing tending to preexisting knowledge structures when the mood is happy and bottom-up processing with high attention to current details when the mood is sad [57].Lerner and Keltner [33] discuss that information processing and decision making can be influenced by the degree of appraisal and the tendency underlying the emotion in different emotional states with the same degree of pleasure.This contribution hypothesizes that the driver-vehicle interaction involves both emotional and cognitive decision-making processes, which should be considered in parallel.The popular models of cognition and emotion are briefly discussed in this section.

Cognition architecture
Cognitive architectures refer to the structure of human mind.One of the architectures for cognitive modeling is adaptive control of thought-rational (ACT-R) [2].The main feature of ACT-R is that it assumes all components of the brain as a unified single agent [1].Furthermore, the implementation of this architecture on real-world problems is possible.ACT-R consists of several cooperating modules, each dedicated to a specific function.The exact number of modules is not specified, but the main modules related to the driving context are depicted in the Fig. 1, adapted from [1].The declarative module retrieves information from the memory.The perceptual modules (e.g., visual, auditory, haptic) collect data from the field.Body Fig. 1 ACT-R architecture adapted from [1] motions are supervised by the manual module.The intentional module conducts functions toward the illustrated goal.Each module has a buffer as a communicator with other modules, which stores a chunk of information from the corresponding module.A central production system is connected to the buffers.The production system organizes all modules based on the information represented in the buffers and generates the next behavior that updates the manual buffer.
Another computational architecture of cognition, which also includes motivation and emotion as well as their interaction, is the PSI theory [13].According to PSI, the agent adapts to the situation and acts in a goal-directed manner.Cognition is modeled using quads, which are a combination of five neurons, one central neuron, and four neighboring neurons.Each of the neurons in the quads is responsible for different parts of the cognitive process, such as searching and backward scanning.

Emotional models
Two main categories of emotion theory are discrete emotion theories [26,60], and core affect/constructionist theories, such as Russell's circumplex model [55] and PAD [38].The first group of theories assumes limited discrete basic emotions for humans, which build all emotional experiences of the human being.The basic emotions are the same for everybody and can vary in intensity [61], however, each theory suggests different basic emotions [15,26,61].The second group of theories suggests that human emotion can move in a two-or three-dimensional space.These theories consider emotion to be a continuous value.Arousal and pleasure are two of the main dimensions.Motivation tendency, attention, or dominance are possible candidates for the third dimension.
In the PSI theory, emotions are part of information processing to adapt to current needs.Emotions are defined in PSI by the three parameters resolution level, selection threshold and activation as well as behavioral tendencies and help in memory, planning and action processes [13].

Driver-vehicle interaction loop
Dargahi Nobari et al. [10] propose a controller with a feedback loop as a comprehensive structure for DVI.The controller aims at regulating the driver state by exposing suitable stimuli.The suggested structure considers the driving situation by utilizing a quantified scale for situation criticality and adjusts the intensity and modality of the stimuli in real-time concerning the driver state [11] and situation criticality [10].The feedback loop (Fig. 2) is made up of the automated system, a target state estimator, a controller, an interface, a human driver, and sensory equipment.The controller is the main part of the loop that defines the control law for regulating the driver.The interfaces and sensors are integrated to enable communication between the automated system and the human driver.In the feedback loop, the driver obtains information about the automated system from inside the vehicle and also information about the traffic condition from outside the vehicle.This information can contain useful data about the driving situation that can help the driver with the driving task or decision making, and it also contains useless data that causes in disturbances and distracts the driver from the driving scene.Based on the perceived information the driver state may vary.Therefore, the system should always monitor the driver state and detect the changes.These variations can cause the driver to be in an unfitting driver state while driving or during collaboration with the assistance system.Thus, a driver state controller is integrated into the interaction loop that is supposed to control the driver state.The controller also takes the driving situation into account.To achieve this goal, it requires a measure of situation criticality.Depending on the situation criticality and the driver state, and based on a driver model, the controller decides on proper stimuli to present to the driver in order to improve driver performance [11] or simply to bring the driver back into-the-loop and increase the driver's awareness.

Controller
The feedback controller is supposed to regulate the driver state to achieve the desired state (target state) so that driving becomes safer and the driver has comfortable driving experience.The controller is responsible for considering the situation and selecting the proper interaction strategy.As input, the controller obtains the driver state, the target state, the automation level, and the situation criticality.The automation level identifies the distribution of tasks between the human driver and the automated system.This distribution determines to what extent the driver should be attentive or aware of the situation.Then, a target state must be determined for the driver.The target state identifies the most suitable driver state, which is reached by a minimal change in the current driver state and leads to the safest driving behavior.The target state thus is estimated based on the current driver state and the driving situation.For example, if an obstacle is ahead of the ego-vehicle, the driver should first look at the obstacle to be aware of the hazard, or if a jeopardizing event occurs behind the ego-vehicle, the gaze direction of the driver should first change towards the rearview mirror to perceive the information about the hazard.The same applies to the motor state, where the drivers should react by steering or braking depending on the driving situation.The target state should be determined individually for each driver.As an example, the average heart rate (HR) value differs between the test subjects, so the optimal HR value of the drivers during the driving task would be different for each driver.Or the optimal arousal level for the normal driving task without any critical situation is a medium level, but its precise value is not identical for the drivers.By comparing the driver state and the target state, the controller can estimate the necessary intensity, modality, and time of TOR to achieve the best driver performance in the situation.After determining a target state for the driver, a proper communication signal or stimuli should be selected.The stimuli generation process is based on a control law defined for the controller, which ranges from a simple state controller to an optimal control mechanism [53].Using a state feedback controller, the driver state is compared with the target state; if they are identical, then stimuli are not required.The modality and intensity of the stimuli depend on the situation criticality, the driver state, and the automation level.Implementing an optimal control strategy creates the opportunity to generate stimuli based on defined objectives.This goal requires a driver model that represents the relationship between TOR features and driver performance.An optimal controller requires an objective function as well, which includes all desired goals such as improving the driver state and increasing comfort in the shortest possible time.The output (u) of the controller Fig. 2 Feedback control loop for regulation of driver state determines the intensity, modality, and exposure time of stimuli.

Interface
In an automated driving context, the common methods of communication with the driver are through visual presentations such as warnings on displays [5,68] and changes in interior lighting [23], auditory signals such as voice or alarm tones [5,68], and haptic impulses like a vibration in the driver seat and the steering wheel or resistive force on the pedals [5,68].These stimuli are presented to the driver and their intensity (e.g., volume, frequency) is continuously adapted based on the input u.

Driver
In this contribution, a driver is described by the driver state.The driver state consists of sensory, motor, cognitive, and emotional states.The sensory state defines what a person can perceive from the environment at present.The motor state is the degree to which a person reacts to the environment through physical movements.The driver's ability to mentally process data and perceive and interpret sensory stimuli is called the cognitive state.And the emotional state reflects the feelings caused by internal or external stimuli.During the interaction, the driver is exposed to communication signals or TOR generated by the automated system that are synthetic and manipulable as well as disturbances originated from the surroundings, as non-manipulable influence.All signals that are perceived by the driver can influence the driver state.

Sensor
The interaction is composed of the data flow from the driver to the system and vice versa.The automated system receives the data from the driver by means of build-in sensors in the vehicle and processes the collected data to estimate the driver state.The sensory state is conventionally measured by eye-trackers that follow the gaze behavior [3].Besides, the noise of the environment or the volume of the onboard sound player can be evaluated as driver's auditory availability.The measurement of the motor state is commonly based on camera or accelerometers [7].The assessment of the cognitive state is only possible indirectly through physiological data gathered from the heart and brain (e.g., electroencephalography activities, pupil diameter (PD), HR, respiration rate) [30].The emotional state can be estimated from physiological data, behavioral and facial cues, and subjective ratings on the basis of the psychological physiology [25].
The sensory data is most of the time accompanied by measurement noise.If the noise amplitude is not negligible, a filter should also be applied to the data before estimating the driver state.

Proposed driver model
As mentioned in the previous section, the controller requires a driver model to generate stimuli based on the driver's characteristics.The driver model should have the basic and irreducible state variables that are sufficient to describe the driver behavior.The model is a representation of the human driver for the vehicle so that the vehicle can interpret and predict human behavior.The structure of the driver model proposed in this contribution is based on the psychological functions of the human brain.The inputs to the model are the driver state and intensity of the stimuli that are exhibited to the driver at present.The output of the model estimates the performance of the driver in terms of reaction time and reaction type, i.e. how fast the driver reacts to the stimuli and whether the driver reacts by braking or steering.Interaction is a decision-making task for the drivers where they choose not to react or to react with different functions.
As shown in the Fig. 3, the driver model consists of the driver's sensory perception, decision-making procedure, and motor reaction, which interact with each other.The logical sequence of the decision-making process starts with sensory perception.Then a decision is made based on the emotional and cognitive processes, and finally, an action is selected.

Sensory perception
The drivers, first, sense the incoming data from surrounding.During the driving task, the main sensory data involves K Fig. 3 Driver model in critical driving situations visual, auditory, and haptic stimuli.Therefore, the visual state such as gaze direction, haptic state such as sensing vibrations, and the auditory state such as whether the drivers are listening to music or not, or the presence of ambient auditory noise should be considered as a sensory state.

Decision-making procedure
After new data has been perceived, the decision-making process starts.The decision can result in an action, such as overtaking the driving task, or it can only lead to a change in the driver state.When making a decision, the cognitive process and the emotional process can both be active.These two processes work in parallel and exchange information.If an action is necessary, the decision is made based on the processed information.Otherwise, the procedure remains at the risk processing level.This contribution examines the temporal sequence of drivers' emotional and cognitive activities and their response in takeover situations.
To model the cognitive decision-making process the ACT-R [2] is adapted since it can handle dynamic, realworld situations.Moreover, multitasking in the ACT-R architecture has already been demonstrated for drivers.Furthermore, the integrated ACT-R models have predictive power that makes them suitable for the DVI loop.In the context of automated driving, the perceptional modules can be modeled by the attention of the driver to the driving scene.The mental workload of the driver caused by NDRT or any driving-irrelevant activity influences the declarative and procedural memories of the driver.Therefore, the mental workload of the driver should be investigated as well.The intentional module described in the ACT-R architecture represents the motivation and intention of the driver during the driving task.According to ACT-R and several other cognitive architectures [1,27,44], the minimum time required for cognition is 50 ms.That means, the reactions faster than this amount are not decided by the cognitive process but as a reflection.
The emotional process is specified according to the threedimensional emotional state model (PAD: pleasure, arousal, and dominance) [38] that defines each emotion as a combination of arousal, pleasure, and dominance.The ideal amount for the best driving performance is defined with positive pleasure and medium arousal [6].

Motor reaction
After the decision-making process, if an action is required, the motor state of the driver is involved.In manual driving, drivers usually steer, press pedals, or communicate with other traffic members explicitly and implicitly.In automated driving, these actions are reduced to steering and pressing pedals in critical driving situations when the automated system asks for takeover.So, the position of hands and feet, and the activity of the driver during automated driving should be investigated to have precise knowledge of the motor state of the driver.
According to the proposed driver model, the activity of the driver's emotional and cognitive states increases whenever the driver perceives a critical situation until the driver makes a decision.Then the activity level decreases again before the next decision situation.The next section describes an experiment that examines the driver's emotional and cognitive state during the driving mode change, where the driver has to react to a critical driving situation.This contribution hypothesizes that 1. the activity level of both emotion and cognition of the drivers increases during the takeover situation, 2. the increase in activation levels occurs before the onset of the response as a result of the decision-making process.

Experiment in a driving simulator
To assess the proposed driver model, the data gathered from an experiment in a static driving simulator (Fig. 4) is examined.The utilized simulator has three screens that provide

Procedure
All participants filled out a consent form at the beginning and got informed about the experiment goals and functionality of the driving simulator and all of the measuring sensors.To familiarize themselves with the driving simulator and the virtual driving environment, the subjects were asked to drive for twenty minutes in the simulator in both manual and automated modes.After feeling comfortable and convenient with the driving simulator, the participants drove seven driving scenarios with permuted order.In the next subsections, the data obtained from one of these scenarios is presented.The driving scenario started in manual driving mode and the participants were immediately asked to change to the automated mode.The implemented automation was SAE Level 2 [45] where the system was controlling the vehicle in the lateral and longitudinal direction, however, the human driver should all the time monitor the driving situa-Fig.5 Critical driving situation in SAE Level 2: sudden deactivation of the automated mode as a result of a pedestrian crossing the road about 42 m before a possible accident with 3 s time budget tion and react to possible hazardous situations.The participants were beforehand informed about the performance of the automated system and that they are responsible to the driving during the whole scenario.Additionally, they were instructed to monitor the driving scene carefully.The scenario took place in dawn time where the lack of sunlight had limited the sight distance.The drivers could turn on the headlights of the vehicle, however, to keep the situation the same for all participants, the limitation on sight distance was configured to remain the same even with lights on.After activation of the automated mode, the vehicle was driven for about 3 min on one-lane streets of a city with a speed limit of 50 km h −1 .Occasional pedestrians on sidewalks (2 pedestrian km −1 ) and vehicles on the opposite lane (1 vehicle km −1 ) were simulated to increase the acceptance of the driving scene.During the automated driving, a hazardous situation occurred where a pedestrian on the sidewalk suddenly turned into the street to cross the road (Fig. 5).Facing this situation the automated mode turned off without any warning.The driver was able to see the pedestrian on the sidewalk from 5 s before the mode change, and afterward, he could recognize the mode change by a short beep and changes in the appearance of the dashboard (Fig. 6).At this moment the participants had to continue driving and had a time budget of about 3 s to avoid an accident with the pedestrian.All of the participants react to the critical situation after the mode change.After going through this situation the drivers continue to drive manually for a short time until the end of the scenario.

Data collection
The sample included 38 university students (14 females, 24 males) with a valid driver's license who aged from 18 to 32 (mean = 22.92, standard deviation = 3.20).Trials of 5 participants were removed from the data analysis as a result of technical problems and motion sickness.
To determine HRV, the RR-intervals (the intervals between successive heartbeats) are extracted from BVP.In the literature, various methods for determining HRV are presented.Here the RMSSD method based on 10 second intervals is calculated using the equation where RR m are the sequential RR-intervals and n is the number of RR-intervals in 10 s.To be able to compare the HRV of the subjects with each other, the normal value is computed.Since the calculated HRV is based on RR-intervals, the normalization follows [56] HRV norm = HRV RMSSD RR ; where RR is the average of RR-intervals.Finally, to standardize all physiological signals, their z-score [29] is determined by where is the mean of the variable x and is the standard deviation of x.

Results and discussion
The task-related cognitive workload on the driver is reflected in the PD of the driver [21].Therefore, the driver's cognitive state is assessed with pupil dilation.Engonopulos et al. [16] state that the driving difficulty has significant effect on pupil dilation of right eye.Therefore, the data gathered from right eye is considered in this study.An increase in the driver's cognitive workload leads to higher PD and vice versa.Since changes in ambient light influence PD, the index of pupillary activity (IPA) [14] is also computed that is almost resistant to luminosity changes.The IPA estimates the rate of change in PD so that introduces a measure that is comparable between individuals.HRV and EDA are mentioned as measures of the driver's emotional state [4,40].HRV indicates emotional arousal and increases with the emotional regulation of an individual.A decrease in HRV shows an elevation in the driver's emotions.Another index for emotion is EDA that raises with increasing emo-tional activity.The reaction of EDA to emotional changes has a delay of 1 to 5 s.
As the effect of decision making on drivers' emotional and cognitive activity levels is investigated, drivers had to be calm and not cognitively engaged before the critical situation.Therefore, the experiment was conducted in SAE Level 2, in which drivers were not allowed to perform NDRT and were only required to observe the driving situation.At the same time, the situation was kept stress-free.Although the experiment was conducted in the same way for all of the participants, it was not guaranteed that the drivers were completely relaxed/unconcerned and the initial state of the subjects was identical at the beginning of the designed critical situation.Therefore, when analyzing the results, the participants are divided into two groups according to their initial HRV 5 s before the mode change, and the dynamics of their HRV during the transition of the driving task.The participants in group 1 have a relatively high HRV at 5 s before the mode change followed by decrements, which can be interpreted as calm drivers who trust the automated system.However, participants in group 2 have a lower HRV at 5 s before the mode change and later an incremental rate, which shows that they were already excited before the critical situation or did not trust the automated system.Fig. 7 is an example of the variation in physiological measurements of drivers in both groups during the driving scenario, with a time span of 10 s including the mode change.The x-axis shows the time in seconds in which the mode change time is set to zero.The negative and positive amounts show the time before and after the mode change, respectively.In this diagram, under the assumption that the ambient light is almost constant throughout the whole scenario, the z-score of the PD of the driver is depicted as a measure of the driver's cognitive state.The applied automated system was limited to SAE Level 2 and the driver was instructed to be attentive to the driving scene.The driver could see the pedestrian on the sidewalk from 5 s before the mode change.Fig. 7a shows data of a driver from group 1. 4 s before the mode change the HRV of the driver suddenly decreased and remained low.Sequentially PD of the driver started to grow 1 s before the mode change and before taking action and remained high.The early changes in physiological data indicate the start of the decision-making process before the first action and during the driving process by the driver.The increase in EDA also confirms the increase in emotional activity.Furthermore, the decrease in HRV occurred before the increase in PD, which may indicate that emotional decision-making is a faster process than cognitive.
In Fig. 7b, the subject initially has low HRV, which means that the driver is already agitated and actively making decisions, so seeing the pedestrian did not cause any The observations from both samples agreed with the proposed driver model.Fig. 8 presents the statistical results of all participants separately in two groups.The thin lines are the data of the participants and the thick lines are the mean value from all participants of the group.To get a comprehensive overview of the statistical data, the distribution of the changes of the physiological values are depicted in Fig. 9 using the kernel distribution estimation (KDE).Again, the zero point of the time axis is set to the mode change time.
HRV For drivers in group 1, HRV decreases from a higher initial value (Fig. 8a), and the negative slope persists until 1 to 2 s after the mode change, where the slope becomes positive.In contrast, the HRV of drivers in group 2 has a low initial value (Fig. 8b).Within the first 5 s, HRV remains constant, and around the time of the mode change, the value of HRV increases.
EDA The EDA value for both groups is almost the same at the initial and throughout the critical situation, however, the final EDA value (5 s after the mode change) of group 1 is lower than that of group 2 (Fig. 8c and d.Fig. 9a shows the distribution of the first increase in EDA due to the pedestrian for both groups.According to the KDE diagram, the increase in EDA for group 2 occurs before the time of the mode change, showing that group 2 already has higher emotional activity before this event. Pupil The mean value of drivers' PD is almost identical in both groups (Fig. 8e and f), except that in group 1, PD increases before the mode change when drivers process the situation, whereas for drivers in group 2, the noticeable change in PD occurs after the mode change when drivers continue to drive the vehicle manually (Fig. 9b).Fig. 10 additionally shows the IPA of drivers with 1 s time step.For each time step, the mean value and the standard deviation of the IPA among drivers are shown.The behavior of IPA differs slightly from PD, which can be explained by the unavoidable light pollution of the screens.Again, the IPA of drivers approves that the peak of drivers' cognitive activity in group 1 happens before the mode change and in group 2 after the mode change.The peak in the graph of group 1 manifests a decision-making process without further actions, while the crossing pedestrian is seen, and the peak in the graph of group 2 is caused by the decision made on the drivers' reaction to the situation.
Consistent with the first hypothesis drivers in both groups exhibit elevated cognitive and emotional activity, but with different patterns.For group 1, emotional and cognitive activities increase before the mode change.For group 2, emotional activity was already higher initially (5 s before the mode change).Cognitive activity is also almost high before the mode change and has a peak about 1 s after the mode change (Fig. 10b).According to the second hypothesis, the increase in the activation level of emotional and cognitive states is expected to occur before the onset of the mode change, which is due to decision-making.This result is clearly seen in the physiological data of group 1. How-K ever, for group 2, it is difficult to determine the exact time of activation for emotional and cognitive states because they are already quite active.
The results of both groups on the temporal sequence of emotional and cognitive activities of the drivers show that emotional activation occurs before cognitive activation.One reason for this may be the nature of these processes.Mohr et al. [39] indicate that the emotional processing in risky decisions is a fast procedure that roughly estimates the negative outcome of the situation and prepares the body for the reaction, however, cognitive processing computes the probability of negative and positive outcomes and estimates the riskiness of the situation.

Summary and future work
In this contribution, first, a comprehensive framework for DVI is discussed.Then, a qualitative driver model in the context of automated driving is proposed to model the decision-making process during the interaction between driver and vehicle and during the transition of driving tasks.Finally, an experiment is designed and conducted to investigate the defined hypotheses about the driver model.
In the discussed DVI framework, the driver can always be informed about the intentions of the automated vehicle through interfaces, and the vehicle can predict the driver's response based on a driver model, thus achieving mutual predictability.In the proposed interaction, both agents are able to assess each other's actions and states (directability).The automated system constantly compares the driver's state with the target state and exposes the driver to stimuli when distracted.On the other hand, the driver is constantly aware of the state of the automated system and can intervene at any time.The inclusion of a situation criticality block in the system also guarantees a common situation representation.In addition, monitoring the driver through sensors will prevent driver over-trust in the automated system by, for example, alerting distracted drivers when they are expected to monitor the situation or sleeping drivers when they are supposed to be awake.The integrated controller, in turn, improves emotions such as under-trust by providing a comfortable interior environment.
One of the biggest challenges in traffic psychology is to find a reliable interpretation of physiological measures.Most of the variables are highly correlated, and this correlation complicates the analysis [12].Furthermore, statistical significance is more likely to be a false positive than a true positive [12].
The next step of this study is to consider alternative cognitive and emotional architectures for the driver model.In addition, the experiment should be repeated with a larger number of participants with a variety of characteristics and backgrounds.Furthermore, the definition of the mathemat-ical representation of the driver model and the integration of the mathematical model into the controller to complete the control loop is required.

Fig. 6
Fig. 6 Change in appearance of dashboard from automated mode to manual mode

Fig. 7
Fig. 7 Physiological data of two sample drivers from 5 s before to 5 s after the mode change: z-score of pupil dilation from right eye, z-score of HRV, and z-score of EDA. a Physiological data from one sample driver of group 1, b Physiological data from one sample driver of group 2

Fig. 8 Fig. 9 Fig. 10
Fig.8 Physiological data gathered from participants during the mode change (cont.).a HRV z of all participants of group 1, b HRV z of all participants of group 2, c EDA z of all participants of group 1, d EDA z of all participants of group 2, e PD z from right eye of all participants of group 1, f PD z from right eye of all participants of group 2