Abstract
This paper reports on a study which investigated the effects of adding vehicle vibrations on the driving simulator’s validity in terms of the stabilization performance. For this reason a motion-based simulator with three degrees of freedom was upgraded to present a realistic perception of engine and road surface induced vehicle vibrations which improves the overall immersion of the simulator. The study showed that representing vertical dynamics to the participants improves the lateral stabilization performance at nearly constant longitudinal stabilization performance and self-reported simulation sickness scores. This improvement leads partially to absolute validity for the stabilization performance. However the study also shows that the presentation of vehicle vibrations leads to an increase in self-reported audible and temporal mental demand. To improve the self-reported mental workload further examinations are necessary.
You have full access to this open access chapter, Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
A multitude of virtual methods, amongst which driving simulators, are increasingly used in the automotive product development process. An important driver for this is the demand for shorter and more cost-efficient development processes [1, 2]. Driving simulators offer the opportunity to carry out driving tests in a safe, reproducible and controlled environment [3]. The testing scope for driving simulators includes human-machine interactions, such as driver distraction studies for operating concepts, assessment of assistance functions and automated driving functions [4] while also involving aspects like suspension set-ups utilizing hardware-in-the-loop simulations [5, 6].
Despite the continuous improvement, driving in a simulator has been and always will be merely a simulated reproduction of reality. Due to an increasing number of optimizations and a strong dependency between individual components of the driving simulation, a holistic evaluation of further developments in the driving simulation is necessary. With a larger number of applications and an ever advancing technical development, the need to prove the validity of driving simulator tests has been increasing. This requires methods to analyse the differences between simulators and to quantify the effects of improvements in simulator configuration. With the method reported in this paper the influence of the reproduced vertical dynamics on behavioural validity will be examined. The National Highway Traffic Safety Administration (NHTSA) recommends testing on driving simulators especially for driver distraction related studies [7]. Resulting from the need to evaluate, for example, new touch screen control elements, a suitable method for presenting vertical dynamics is necessary. For the implementation of vehicle vibrations, the simulation software was upgraded and a static simulator was equipped with four linear actuators (Fig. 2). The four actuators allowing a simulator motion with three degrees of freedom (3-DOF: heave, pitch and roll).
2 Driving Simulator Validity
Driving simulator validity is an active research topic [1, 8, 9] and an important requirement for transferring development tasks from the road into driving simulator laboratories [10]. Depending on the desired focus of the research, different types of simulator validity are used. According to Blana et al. [11, 12] the term of driving simulator validity can be divided into:
-
Behavioural validity describes the correspondence between driving behaviour in the simulator and on-road.
-
Physical validity measures the extent to which a driving simulator can reproduce the visual and vestibular cues while driving an actual vehicle.
-
Absolute validity is defined as a quantitative criterion which is achieved if the numerical values of these performance differences are about equal or the same.
-
Relative validity is achieved if the difference is in the same direction and relative size of the effect of the measure as in reality.
To examine the validity of the simulator in this study the definition of behavioural validity is used. In this case the behavioural concept refers to the driving performance and the self-reported mental workload.
3 Background and Research Question
The research in [13] provided some evidence that adding a 3-DOF motion system to a static simulator can improve the lane keeping performance and decrease the self-chosen vehicle speed on country roads. Another study showed that adding vibrations to a 3-DOF motion system can alleviate the symptoms of motion sickness [14]. Neither study, however, investigated the influence on behavioural validity. Another approach focused on improving the realism of a driving simulator by presenting vehicle vibrations to the participants [15] and validating the influences on the self-chosen speed [16], but the findings were based on a advanced driving simulator with 6-DOF.
The current paper aims to add to existing knowledge by providing a detailed analysis of the influence of vertical dynamics on behavioural validity on highway conditions in a cost-efficient mid-size 3-DOF simulator.
Reproducing vertical dynamics leads to vehicle vibrations, which are perceived by the driver. On the one hand the presence of vehicle vibrations can be interpreted as an additional cue for the driver and results in an increased immersion. On the other hand the resulting vehicle movements can be interpreted as a disturbance input for the driver-vehicle-environment control loop. To determine the influence of the vehicle vibrations on the driving performance the following research question will be answered:
How does the reproduction of vehicle vibrations on a driving simulator influence the stabilization performance, the-self reported mental driver workload and the self-reported simulation sickness?
4 Methodology
To examine the research questions two empirical studies were performed. In May 2018 a real driving study on an highway near Munich was conducted with 30 participants. This study will be referred to here as the On-Road Study. Furthermore, a driving simulator study in one of the BMW Group’s driving simulation facilities with 41 participants was conducted in August 2018.
4.1 Apparatus
On-Road Study. Figure 1 shows the experimental vehicle used for the on-road study. The used car was a 4-door sedan BMW 5-Series with 248 horse power. For data acquisition the internal vehicle bus was recorded with 20 Hz using a data logging system. At the bottom left on Fig. 1 the experimenter interface is shown. The interface was displayed on a \(7^{\prime \prime }\) display which can be positioned near the passenger or rear seat. The right-hand side of Fig. 1 shows the driver position. The experimental car was equipped with a car radio to perform the selected driving tasks (see Sect. 4.4). The modifications of the experimental vehicle were implemented such that the general appearance for the participants, apart from the radio, is not different from a series-production vehicle.
Simulator. The driving simulator used in this validation study is shown on Fig. 2. The vehicle mockup consisted of the front section of a BMW 5-Series chassis and 5-Series cockpit elements, which was mounted on a electromotive 3-DOF platform. The platform was actuated using 4 D-Box actuators (D-BOX TECHNOLOGIES INC., Gen II Actuators \(6^{\prime \prime }\)). The vehicle dynamic model described the same vehicle type which was used for the on-road study. All typical control elements such as direction-indicator control, brake and acceleration pedal and gear selector lever were fully functional. The steering torque feedbacks came from an electric actuator (Sensodrive GmbH, SENSO-Wheel SD-LC) that delivers a nominal torque of \(7.5\,\mathrm {Nm}\). For the visualisation of the virtual environment five video projectors were used which enable a \(220 \mathrm {^\circ }\) horizontal and \(45 \mathrm {^\circ }\) vertical field of view. The views seen in the exterior and interior rear view mirrors were provided by three LCD screens.
4.2 Vehicle Vibrations
The main aspect of this study was to examine the influence of vehicle vibrations on driving simulator validity.Vehicle vibrations can be classified according to Heissing et al. [17] in three categories. Vibrations with frequencies from 0–25 Hz are perceived by tactile sense and are categorized as tangible. Frequencies higher than 100 Hz are mainly noticed as audible noise and hence categorized as audible. Between these frequency bands exists a so called transition zone. Phenomena in this transition zone can be perceived by both, the acoustic and tactile senses and are therefore both audible and tangible. To present these effects to the participants three types of actuators were used:
-
3-DOF motion-system (D-BOX TECHNOLOGIES INC., Gen II Actuators \(6^{\prime \prime }\)) for frequencies up to 30 Hz, i.e. the tangible range
-
Structure-borne sound converter (SONIC IMMERSION TECHNOLOGIES, IBEAM VT-200 Transducer) for frequencies from 30–125 Hz, i.e. reproducing roughly the transition zone
-
Four channel sound system mainly for frequencies over 100 Hz up to 20 kHz
The structure-borne sound converter was located under the driver seat to represent phenomena like road roughness, road bumps and engine vibrations. Vehicle vibrations with frequencies up to 100 Hz are mainly caused by tyre-road interactions [18]. For an adequate representation of these effects a realistic simulation of road surfaces is necessary.
Therefore the road surface for the simulated road was derived from laser-measured data of the highway on which on-road study was performed. The measured road excitations are provided to the driving simulation by the OpenCRG file format [19] with a resolution of 10Â \(\times \)Â 10Â mm. To obtain the resulting vehicle accelerations and angular rates, a non-linear double-track model is used for the vehicle dynamics simulation. To present the vehicle vibrations to the participants the calculated movements are transferred to the motion-system and the structure-borne converter.
It should be noted that this implementation also allows for the reproduction of steering torque effects caused by road excitations from road damages, such as grooves, potholes and cracks. Furthermore, it has to be taken into account that in the real world road damages are not only perceived as vestibular or tactile cues but also by visually cues. Therefore the existing road damages were also reproduced graphically (see Fig. 3).
4.3 Participants
In the on-road study 30 BMW employees (28 men and 2 women), with at least six years of driving experience and a minimal annual mileage of \(5000 \; \mathrm {km}\), took part. The age of the participants ranged from 25 to 50 (mean \((\mu ) = 36.13\) years, standard deviation \((\sigma ) = 8.28\)).
The simulator study was conducted with 41 BMW employees (34 men and 7 women), with at least four years of driving experience and an minimal annual mileage of \(5000 \; \mathrm {km}\). The age of the participants ranged from 20 to 59 \(\mathrm {(\mu = 37.12 \ years, \sigma = 12.08)}\). 31 of 41 participants had prior experience in a driving simulator. The remaining 10 participants took part for the first time in a driving simulator study.
All participants were included on a voluntary basis. The participants received written instructions and provided their informed consent before the experiment commenced. They were informed about that they could abort the experiment at any time in case of discomfort.
4.4 Study Design
Independent Variables. The driving simulation study was designed as a within-subject design with the two independent variables ‘Flat’ and ‘Ride’ combined with the real driving study ‘Real’.
-
Real: On-road study which serves as benchmark for a realistic stabilization performance.
-
Flat: Only roll- and pitch angles induced by vehicle accelerations were presented to the participants by the motion system.
-
Ride: This condition extends the ‘Flat’ condition by representing vehicle vibrations due to road-excitations by the motion system and a structure-borne sound converter. See also Sect. 4.2 for more information.
The driving task consisted of a highway ride in which the so-called radio tuning task was carried out according to the specifications of AAM [20]. In the on-road study participants were instructed to follow a lead vehicle at a distance of \(50 \ \mathrm {m}\). The lead vehicle, driven by on of the experimenters, drove in the right lane at a speed of \(92 \ \mathrm {km/h}\). The written instruction for the participants was to focus on lane keeping and following the lead vehicle with a constant distance. This task will be referred to here as the baseline condition. In order to allow an analysis of relative effects (i.e., relative validity) the above described driving task was also conducted while the driver was instructed to simultaneously perform a secondary task. The secondary task was the so-called radio tuning task (radio). In the radio tuning task, the participants were asked to select a given frequency on a radio, which was installed according to the AAM [20]. The radio which was used for the radio tuning task is shown on the right-hand side in Fig. 1.
The drive to the highway took about 20 min. During this time the participants could familiarize themselves with the car. At the beginning of the highway drive the participants practiced the baseline task for about 5–10 min. The radio task was not practiced before.
After the on-road study the radio was installed into the simulator mock-up. The test procedure in the simulator study was the same as in the real study. Before the test cases have been started the participants drove along urban, rural and highway roads for around 10Â min to get familiar with the environment. The lead vehicle on the highway was controlled automatically by the simulation software. The highway section where the on-road study was conducted, was reproduced for the simulation including the laser-measured road surface.
To calculate the performance indicators six repetitions of the radio tuning task and the baseline task with a duration of 60 s were performed. The order in which the participants performed the two tasks was randomized.
Figure 4 summarizes the experimental design and provides an overview of the independent and dependent variables of the experiment. In the next section the selected dependent variables will be described.
Dependent Variables. The above described scenario was used for the examination of the stabilization performance, whereby the stabilization task can be divided into lateral and longitudinal stabilization [21]. The lateral stabilization performance was measured with the standard deviation of lane position (SDLP). To calculate the SDLP the distance between the right-hand front wheel and the side stripes was chosen. The longitudinal stabilization performance was measured with the standard deviation of head-way (SDHW) [22, 23], in this case the head-way was defined as the distance between the front and rear bumper of the experimental and the lead vehicle.
In addition the self-reported driver mental workload was recorded with the driver activity load index (DALI).
The DALI is a revised version of the NASA-TLX, adapted to the driving task [24] and consist seven workload dimensions:
-
effort of attention
-
visual demand
-
auditory demand
-
temporal demand
-
interference
-
situational stress
-
total score (consisting of the average of the other dimensions)
Each dimension has been valuated by the participants with a numeric rating scale from 0 (low demand) to 5 (high demand) after each of the three repetitions. Afterwards the average of both submitted ratings was calculated.
Simulator sickness is a common issue in the context of driving simulation and a major obstacle to the use of driving simulators for research, training and driver assessment purposes [25, 26]. Therefore the self-reported simulation sickness was quantified by Kennedy’s SSQ Simulator Sickness Questionnaire (SSQ) [27] to analyse the effect of vehicle vibrations on the participant’s sense of well-being. After data acquisition it is possible to provide simulator sickness scores classified in three symptoms referred to as nausea (N), oculomotor disturbance (O), disorientation (D). Additionally a total score (TS) can be derived from the three symptoms sub scales.
5 Results
To test whether the data meet the requirements for a parametric independent analysis of variance (ANOVA), which are that the residuals are normally distributed and have homogeneity of variances [28], the Lilliefors and Levene test were performed [29]. If the conditions are not fulfilled the significance test was performed using the non-parametric Kruskal-Wallis-Test [30]. For the post-hoc tests the p-values were corrected with the least significant difference method [31].
5.1 Lateral Stabilization Performance
Figure 5 shows the results (mean and 95% confidence interval) for the lateral stabilization performance quantified by the SDLP. On the ‘real’ condition the participants produced the lowest mean for the SDLP followed by the ‘ride’ and ‘flat’ condition. The statistical tests reveals significant differences for the SDLP during the baseline and radio task (baseline: \(F(2) = 6.30,\, p < .01\), radio: \(F(2) = 4.83,\, p = .01\)). Regarding to the post-hoc tests the results for the condition ‘ride’ on both tasks do not differ significantly from the on-road study (baseline: \(p>.30\), radio: \(p>.08\)). These results indicates that the addition of vehicle vibrations are required to obtain absolute validity, i.e., no significant difference with the real condition.
5.2 Longitudinal Stabilization Performance
Figure 6 illustrates the results for the longitudinal stabilization performance quantified by the SDHW. For the baseline task the means of all three conditions are similar. For the radio task the mean for the ‘real’ condition is the lowest followed by the ‘ride’ and ‘flat’ condition. The ANOVA for the SDHW on baseline condition shows no significant differences (\(H(2) = 1.16,\, p > .55\)). The AMOVA test for the radio task shows significant differences between the three test conditions (\(H(2) = 8.08,\, p = .018\)). The post-hoc contrasts, however, show no significant differences between the simulator and on-road conditions (real-flat: \(p=.058\), real-ride: \(p=.804\)).
5.3 Mental Driver Workload
The mental workload quantified by the DALI total score is shown in Fig. 7). A statistical analysis showed a significant effect of task but no significant difference between the conditions real, ride and flat. For closer investigation the six DALI sub scales were also analysed (see Fig. 8) which showed that the auditory demand for the baseline task for the ‘ride’ condition is significantly higher (\(H(2) = 6.31,\, p = .043\)). Furthermore the temporal demand is perceived higher for the ‘ride’ condition during the baseline task (\(H(2) = 6.15,\, p = .046\)). Regarding the radio task, the descriptive analysis shows also higher values for the auditory and temporal demand but the differences are not statistically significant.
5.4 Simulator Sickness
To analyse the effect of vertical dynamics on simulator sickness a Wilcoxon rank sum test for two independent samples was performed. Although the ‘ride’ condition provides an additional vertical movement to the participants, regarding the SSQ sub-scales no significant differences were found when comparing the ‘flat’ and ‘ride’ condition (see Fig. 9).
6 Discussion and Outlook
The results of this approach underline the importance of vehicle vibrations on lateral stabilization performance. Absolute validity could be achieved after presenting vehicle vibrations for the SDLP in both conditions. These results show that the presented additional cues provide useful information for the lateral stabilization task. For the head-way performance (SDHW) absolute validity is reached for all simulator conditions. This means that presenting road excitations to the subject has no negative effect on longitudinal stabilization performance. It will be assumed that the perceived visual cues are sufficient to perform the longitudinal stabilization task.
Although the DALI total score shows no significant differences between the two test conditions - flat and ride - the descriptive analysis indicates an increasing overall mental workload for the ride condition. This increase seems to be caused by a higher temporal and audible demand.
Possible explanations for the higher audible demand could be that the cues, provided by the vehicle vibration system in the ride condition, did not present the participants a realistic experience or caused additional noise that did not aid the drivers in their task. Another explanation may be that the study was conducted with experienced driving simulator drivers which were not familiar with the newly presented cues in this experiment. The possible correlation between the increased audible demand and temporal demand is to be investigated.
A 3-DOF motion system in combination with vehicle vibrations delivers a cost-efficient solution to improve the lateral stabilization performance without influencing the longitudinal stabilization performance. An negative effect regarding to the SSQ could not be found. A possible next step is to investigate the specific reasons for the higher demanded mental workload while driving with simulated vehicle vibrations.
To analyse the effect of vehicle vibrations on motion simulators with more than 3-DOF the study should be repeated in classical motion simulators with a hexapod motion base. Furthermore this approach will be extended by a country road scenario to evaluate the implementation of vehicle vibrations, with more behavioural aspects. In this future study additional behavioural aspects, such as a subjective evaluation of the realism of the driving scenario and perceived driving speed, will be included.
References
Zöller, I.M.: Analyse des Einflusses ausgewälter Gestaltungsparameter einer Fahrsimulation auf die Fahrerverhaltensvalidität. Dissertation, Technische Universtät Darmstadt, Darmstadt (2015)
Hassan, B., Gausemeier, J.: A design framework for developing a reconfigurable driving simulator. IARIA Int. J. Adv. Syst. Meas. 8, 1–17 (2015)
de Winter, J.C., van Leeuwen, P.M., Happee, R.: Advantages and disadvantages of driving simulators: a discussion. In: Spink, A., Grieco, F., Krips, O., Loijens, L., Noldus, L., Zimmerman, P. (eds.) Proceedings of Measuring Behavior 2012, pp. 47–50 (2012)
Winner, H., ed.: Handbuch Fahrerassistenzsysteme: Grundlagen, Komponenten und Systeme für aktive Sicherheit und Komfort; mit 45 Tabellen. 1. aufl. edn. ATZ-MTZ-Fachbuch. Vieweg + Teubner, Wiesbaden (2009)
Fioccardi, A., Dusini, L.: How to reduce development time: Winter session at the driving simulator (2017)
Moscatelli, S.: Virtual simulation hil & sil: Using driving simulator and hardware in the loop in calibration of magneto-rheological damper controller (2017)
NHTSA: Visual-manual NHTSA driver distraction guidelines for in-vehicle electronic devices (2012)
Klüver, M.: Can we trust driving simulator studies? the behavioral validity of the daimler AG driving simulators. Dissertation, Johannes Gutenberg Universität Mainz, Mainz (2016)
Reich, D., et al.: Gestaltungsaspekte immersiver Fahrsimulationsumgebungen. Dissertation, Fraunhofer-Institut für Produktionsanlagen und Konstruktionstechnik and Fraunhofer IRB-Verlag (2017)
Harms, L.: Driving performance on a real road and in a driving simulator: results of a validation study. VTI särtryck 1996(267), 19–26 (1996)
Blana, E.: Driving simulator validation studies: A literature review (1997)
Hoskins, A.H., El-Gindy, M.: Technical report: literature survey on driving simulator validation studies. Int. Journey Heavy Veh. Syst. 13(3), 241–252 (2006)
Parduzi, A., Bezikofer, F., Comulada-Simpson, E., Marker, S.: Comparable evaluation of a 3DOF mid-size simulator concept. In: Kemeny, A., Colombet, F., Mérienne, F., Espié, S. (eds.) Proceedings of the DSC 2018 Europe Conference, pp. 207–208 (2018)
Guillaume, L., Kemeny, A., Paillot, D., Colombet, F.: A simulation sickness study on a driving simulator equipped with a vibration platform. In: Kemeny, A., Colombet, F., Mérienne, F., Espié, S. (eds.) Proceedings of the DSC 2018 Europe Conference, pp. 19–22 (2018)
Bolling, A., et al.: Improving the realism in the VTI driving simulator. Shake final report (2012)
Ahlström, C., Bolling, A., Sörensen, G., Eriksson, O., Andersson, A.: Validating speed and road surface realism in VTI driving simulator III (2012)
Heißing, B., Ersoy, M., Gies, S.: Fahrwerkhandbuch: Grundlagen, Fahrdynamik, Komponenten, Systeme, Mechatronik, Perspektiven. Vieweg+Teubner Verlag/Springer Fachmedien Wiesbaden GmbH Wiesbaden, Wiesbaden (2011). https://doi.org/10.1007/978-3-8348-8168-7. 3., überarbeitete und erweiterter auflage edn
Thomaier, M.: Optimierung der NVH-Eigenschaften von Pkw-Fahrwerkstrukturen mittels Active-Vibration-Control. Dissertation, Technische Universtät Darmstadt, Darmstadt, 16 June 2008
Dupuis, M.: OpenCRG: managing the road surface ahead. http://www.opencrg.org/
Alliance of Automobil Manufactures: Statement of principles, criteria and verification procedures on driver interactions with advanced in-vehicle information and communication systems: Including 2006 updated sections (2006)
Bubb, H., Bengler, K., Grünen, R.E., Vollrath, M.: Automobilergonomie. A. Springer, Wiesbaden (2015). https://doi.org/10.1007/978-3-8348-2297-0
Östlund, J., Engström Johan, Keinath, A., Horst Dorit et. al.: Driving performance assessment: methods and metrics: aide adaptive integrated driver-vehicle interface (2005)
Papantoniou, P., Papadimitriou, E., Yannis, G.: Review of driving performance parameters critical for distracted driving research. Transp. Res. Procedia 25, 1796–1805 (2017). World Conference on Transport Research - WCTR 2016 Shanghai. 10–15 July 2016
Pauzié, A.: A method to assess the driver mental workload: the driving activity load index (DALI). IET Intell. Transp. Syst. 2(4), 315 (2008)
Helland, A., Lydersen, S., Lervag, L.E., Jenssen, G.D.: Driving simulator sickness: impact on driving performance, influence of blood alcohol concentration, and effect of repeated simulator exposures. Accid. Anal. Prev. 94, 180–187 (2016)
Neukum, A., Heidi, G.: Kinetose in der fahrsimulation: Simulation von einsatzfahrten im auftrag des präsidiums der bayerischen bereitschaftspolizei, abschlussbericht - teil ii
Kennedy, R.S., Stanney, K.M., Harm, D.L.: Configural scoring of simulator sickness, cybersickness and space: Adaptation syndrome: Similarities and differences? https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20100033371.pdf
Rasch, B., Friese, M., Hofmann, W., Naumann, E.: Quantitative Methoden 2. Springer-Lehrbuch. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-43548-9
Bortz, J., Schuster, C.: Statistik für Human- und Sozialwissenschaftler. Springer-Lehrbuch. Springer, Heidelberg, 201 (2010). https://doi.org/10.1007/978-3-642-12770-0. 7, vollständig überarbeitete und erweiterte auflage edn
Du Prel, J.B., Röhrig, B., Hommel, G., Blettner, M.: Choosing statistical tests: part 12 of a series on evaluation of scientific publications. Deutsches Arzteblatt international 107(19), 343–348 (2010)
Salkind, N.J.: Encyclopedia of Research Design. Sage, Thousand Oaks (2010)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Parduzi, A., Venrooij, J., Marker, S. (2019). A Method to Assess the Effect of Vertical Dynamics on Driving Performance in Driving Simulators: A Behavioural Validation Study. In: Krömker, H. (eds) HCI in Mobility, Transport, and Automotive Systems. HCII 2019. Lecture Notes in Computer Science(), vol 11596. Springer, Cham. https://doi.org/10.1007/978-3-030-22666-4_13
Download citation
DOI: https://doi.org/10.1007/978-3-030-22666-4_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-22665-7
Online ISBN: 978-3-030-22666-4
eBook Packages: Computer ScienceComputer Science (R0)