Abstract
In this article, we use self-reported driving behaviors from a written questionnaire to assess the measurement validity of data derived from a driving simulation. The issue of validity concerns the extent to which measures from the experimental context map onto constructs of interest. Following a description of the experimental methods and setting, an argument for the face validity of the data is advanced. Convergent validity was assessed by regressing behaviors observed in the driving simulator on self-reported measures of driving behaviors. Significant relationships were found across six measures: accidents, speeding, velocity, passing, weaving between traffic, and behavior at stop signs. Concurrent validity was evaluated with an analysis of simulator accident involvement and attention deficit hyperactivity disorder status. Discriminant validity was assessed using a multitrait-multimethod matrix of simulator and questionnaire data. We concluded that although the relationship between self-reported behaviors and observed responses in the simulator falls short of perfect correspondence, the data collected from the driving simulator are valid measures of the behaviors of interest.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Allen, R. W., Park, G., Rosenthal, T. J., & Aponso, B. (July, 2004).A process for developing scenarios for driving simulations. Paper presented at the IMAGE 2004 Conference, Scottsdale, AZ.
Alm, H., &Nilsson, L. (1995). The effects of a mobile telephone task on driver behaviour in a car following situation.Accident Analysis & Prevention,27, 707–715.
Barkley, R. A., Guevremont, D. C., Anastopoulos, A. D., DuPaul, G. J., &Shelton, T. L. (1993). Driving-related risks and outcomes of attention deficit hyperactivity disorder in adolescents and young adults: A 3- to 5-year follow-up survey.Pediatrics,92, 212–218.
Ben-Yaacov, A., Maltz, M., &Shinar, D. (2002). Effects of an invehicle collision avoidance warning system on short- and long-term driving performance.Human Factors,44, 335–342.
Berg, D., Raugi, G. R., Gladstone, H. B., Berkley, J., Weghorst, S., Ganter, M., &Turkiyyah, G. (2001). Virtual reality simulators for dermatologic surgery: Measuring their validity as a teaching tool.Dermatologic Surgery,27, 370–374.
Biederman, J., &Faraone, S. V. (2004). The Massachusetts General Hospital studies of gender influences on attention-deficit/hyperactivity disorder in youth and relatives.Psychiatric Clinics of North America,27, 225–232.
Blana, E., &Golias, J. (2002). Differences between vehicle lateral displacment on the road and in a fixed-base simulator.Human Factors,44, 303–313.
Boyle, J. M., Dienstfrey, S. J., &Sothoron, A. (1998).National survey of speeding and other unsafe driving actions (Rep. No. DOT HS 808 749). U.S. Department of Transportation, National Highway Traffic Safety Administration, Washington, DC.
Brown, I. D., Tickner, A. H., &Simmonds, D. C. (1969). Interference between concurrent tasks of driving and telephoning.Journal of Applied Psychology,53, 419–424.
Brunnauer, A., Laux, G., Geiger, E., &Möller, H. J. (2004). The impact of antipsychotics on psychomotor performance with regards to car driving skills.Journal of Clinical Psychopharmacology,24, 155–160.
Campbell, D. T., &Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix.Psychological Bulletin,56, 81–105.
Campbell, D. T., &Stanley, J. C. (1963).Experimental and quasiexperimental designs for research. Boston: Houghton Mifflin.
Carr, D. B., Duchek, J., &Morris, J. C. (2000). Characteristics of motor vehicle crashes of drivers with dementia of the Alzheimer type.Journal of the American Geriatric Society,48, 100–102.
Cox, D. J., Humphrey, J. W., Merkel, R. L., Penberthy, J. K., &Kovatchev, B. (2004). Controlled-release methylphenidate improves attention during on-road driving by adolescents with attention-deficit/hyperactivity disorder.Journal of the American Board of Family Practice,17, 235–239.
Cox, D. J., Merkel, R. L., Penberthy, J. K., Kovatchev, B., &Hankin, C. S. (2004). Impact of methylphenidate delivery profiles on driving performance of adolescents with attention-deficit/hyperactivity disorder: A pilot study.Journal of the American Academy of Child & Adolescent Psychiatry,43, 269–275.
Cox, D. J., Penberthy, J. K., Zrebic, J., Weinger, K., Aikens, J. E., Frier, B., et al. (2003). Diabetes and driving mishaps.Diabetes Care,26, 2329–2334.
Dawson, S. L., &Kaufman, J. A. (1998). The imperative for medical simulation.Proceedings of the IEEE,86, 479–483.
Decina, L. E., Gish, K. W., Staplin, L., &Kirchner, A. H. (1996).Feasibility of new simulation technology to train novice drivers (Pub. No. DTNH22-95-C-05104). Washington, DC: National Highway Traffic Safety Administration.
Döring, B. (1990). A simulation study for analysing pilot’s rule-based behavior.Proceedings of the computer aided system design and simulation conference (p. 1–19). France: Neuilly Sur Seine.
Godley, S. T., Triggs, T. J., &Fildes, B. N. (2002). Driving simulator validation for speed research.Accident Analysis & Prevention,34, 589–600.
Hahn, R. W., &Tetlock, P. C. (1999, October).The economics of regulating cellular phones in vehicles (Working Paper, 99-9). AEI-Brookings Joint Center for Regulatory Studies, Washington, DC.
Harms, L. (1996). Driving performance on a real road and in a driving simulator: Results of a validation study. In A. G. Gale, I. D. Brown, C. M. Haslegrave, & S. P. Taylor (Eds.),Vision in vehicles V (pp. 19–26). Amsterdam: Elsevier.
Hawkins, R., MacKrell Gaglione, M., LaDuca, T., Leung, C., Sample, L., Gliva-McConvey, G., et al. (2004). Assessment of patient management skills and clinical skills of practising doctors using computer-based case simulations and standardised patients.Medical Education,38, 958–968.
Hoffman, J. D., Lee, J. D., Brown, T. L., &McGehee, D. V. (2002). Comparison of driver braking responses in a high-fidelity simulator and on a test track.Transportation Research Record,1803, 59–65.
Holland, C. A., Handley, S., &Feetam, C. (2004).Older drivers, illness and medication (Road Safety Res. Rep. No. 39). Department for Transport: London.
Judd, C. M., &McClelland, G. H. (1998). Measurement. In D. T. Gilbert, S. T. Fiske, & G. Lindzey (Eds.),Handbook of social psychology:Vol. 1 (4th ed., pp. 180–232). New York: McGraw-Hill.
Kaptein, N. A., Theeuwes, J., &van der Horst, R. (1996). Driving simulator validity: Some considerations.Transportation Research Record,1550, 30–36.
Kidder, L. H., &Judd, C. M. (1986). Research methods in social relations. New York: Holt, Rinehart & Winston.
Klee, H., Bauer, C., Radwan, E., &Al-Deek, H. (1999). Preliminary validation of driving simulator based on forward speed.Transportation Research Record,1689, 33–39.
Lagarde, E., Chastang, J. F., Gueguen, A., Coeuret-Pellicer, M., Chiron, M., &Lafont, S. (2004). Emotional stress and traffic accidents: The impact of separation and divorce.Epid emiology,15, 762–766.
Landau, F., Laur, M., Hein, C. M., Srinivasan, R., &Jovanis, P. P. (1994).A simulator evaluation of five in-vehicle navigation aids (Publication No. UCD-ITS-RR-94-19). Institute of Transportation Studies, University of California, Davis.
Lawton, R., Parker, D., Manstead, A. S. R., &Stradling, S. G. (1997). The role of affect in predicting social behaviors: The case of road traffic violations.Journal of Applied Social Psychology,27, 1258–1276.
Lee, H. C., Cameron, D., &Lee, A. H. (2003). Assessing the driving performance of older adult drivers: On-road versus simulated driving.Accident Analysis & Prevention,35, 797–803.
Lee, J. D. (2004).Simulator fidelity: How low can you go? Retrieved November 1, 2004, from www.uiowa.edu/neuroerg/Simulator%20Users%20Group/HFES_SimPanel.htm.
McGaw, D., &Watson, G. (1976).Political and social inquiry. New York: Wiley.
McGwin, G., Jr.,Sims, R. V., Pulley, L., &Roseman, J. M. (2000). Relations among chronic medical conditions, medications, and automobile crashes in the elderly: A population-based case-control study.American Journal of Epidemiology,152, 424–431.
McKnight, A. J., &McKnight, A. S. (1993). The effect of cellular phone use upon driver attention.Accident Analysis & Prevention,25, 259–265.
Motor Trend (1999).1999 import car of the year: Volkswagen New Beetle. Retrieved September 13, 2004, from www.motortrend.com/oftheyear/car/112_9902_icoy/value.html.
Murphy, K., &Barkley, R. A. (1996). Prevalence of DSM-IV symptoms of attention deficit hyperactivity disorder in adult licensed drivers: Implications for clinical diagnosis.Journal of Attention Disorders,1, 147–161.
National Transportation Safety Board (2004).Special investigation highway report: Medical oversight of noncommercial drivers (NTSB No. SIR-04/01). Washington, DC: Author.
Nunes, L., &Recarte, M. (2002). Cognitive demands of hands-freephone conversation while driving.Transportation Research Part F: Traffic Psychology & Behaviour,5, 133–144.
Parker, D., Reason, J. T., Manstead, A. S. R., &Stradling, S. G. (1995). Driving errors, driving violations and accident involvement.Ergonomics,38, 1036–1048.
Ranney, T. A., Heydinger, G., Watson, G., Salaani, K., Mazzae, E. N., &Grygier, P. (2002).Investigation of driver reactions to tread separation scenarios in the National Advanced Driving Simulator (NADS) (Rep. No. DOT HS 809 523). Washington, DC: National Highway Traffic Safety Administration.
Reed, M. P., &Green, P. A. (1999). Comparison of driver performance on-road and in a low-cost simulator using a concurrent telephone dialing task.Ergonomics,42, 1015–1037.
Rimmö, P. A., &Hakamies-Blomqvist, L. (2002). Older drivers’ aberrant behaviour, impaired activity, and health as reasons for selfimposed driving limitations.Transportation Research Part F: Traffic Psychology & Behaviour,5, 47–62.
Rizzo, M., Reinach, S., McGehee, D., &Dawson, J. (1997). Simulated car crashes and crash predictors in drivers with Alzheimer disease.Archives of Neurology,54, 545–551.
Rolfe, J., &Hampson, B. P. (2003). Flight simulation-Viability versus liability issues of accuracy, data and validation.Aeronautical Journal,107, 631–635.
Rosenstone, S. J., Hansen, J. M., &Kinder, D. R. (1986). Measuring change in personal economic well-being.Public Opinion Quarterly,50, 176–192.
Sarathy, S., &Higman, J. (1994). Development and validation of the OH-58D Kiowa Warrior High Fidelity Flight Simulation Model.Proceedings of the annual forum of the American Helicopter Society,2, 905–914.
Sodhi, M., Reimer, B., &Llamazares, I. (2002). Glance analysis of driver eye movements to evaluate distraction.Behavior Research Methods, Instruments, & Computers,34, 529–538.
Stanton, N. A., Young, M. S., Walker, G. H., Turner, H., &Randle, S. (2001). Automating the driver’s control tasks.International Journal of Cognitive Ergonomics,5, 221–236.
Stein, A. C., Allen, R. W., & Parseghian, Z. (1992, July).The use of low-cost interactive driving simulation to detect impaired drivers. Paper presented at the IMAGE VI Conference, Scottsdale, AZ.
Stradling, S. G., &Meadows, M. L. (2000). Highway code and aggressive violations in UK drivers. InProceedings of the Global Web Conference on Aggressive Driving Issues (pp. 1–14). Ontario: Ministry of Transportation of Ontario.
Stutts, J. C., &Wilkins, J. W. (2003). On-road driving evaluations: A potential tool for helping older adults drive safely longer.Journal of Safety Research,34, 431–439.
Törnros, J. (1998). Driving behavior in a real and a simulated road tunnel—a validation study.Accident Analysis & Prevention,30, 497–503.
Weiss, G., Hechtman, L., Perlman, T., Hopkins, J., &Wener, A. (1979). Hyperactives as young adults: A controlled prospective tenyear follow-up of 75 children.Archives of General Psychiatry,36, 675–681.
Zuin, D., Ortiz, H., Boromei, D., &Lopez, O. L. (2002). Motor vehicle crashes and abnormal driving behaviours in patients with dementia in Mendoza, Argentina.European Journal of Neurology,9, 29–34.
Author information
Authors and Affiliations
Corresponding author
Additional information
This project was supported by the Johnson & Johnson Center for Pediatric Psychopathology Research at the Massachusetts General Hospital and the U.S. Department of Transportation’s New England University Center at the Massachusetts Institute of Technology.
Rights and permissions
About this article
Cite this article
Reimer, B., D’Ambrosio, L.A., Coughlin, J.F. et al. Using self-reported data to assess the validity of driving simulation data. Behavior Research Methods 38, 314–324 (2006). https://doi.org/10.3758/BF03192783
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/BF03192783