Learning Optimal Control Strategies from Interactions with a PADAS

  • Fabio Tango
  • Raghav Aras
  • Olivier Pietquin
Conference paper


This paper addresses the problem to find an optimal warning and intervention strategy for a partially autonomous driver’s assistance system. Here, an optimal strategy is regarded as the one minimizing the risk of collision with an obstacle ahead, while keeping the number of warnings and interventions as low as possible, in order to support the driver and avoid distraction or annoyance. A novel approach to this problem is proposed, based on the solution of a sequential decision making problem.


Automotive environment Optimal warning and intervention strategy Partially Autonomous Driver Assistance System Markovian Decision Processes 



The research leading to these results has received funding from the European Commission Seventh Framework Programme (FP7/2007-2013) under grant agreement no. FP7–218552, Project ISi-PADAS (Integrated Human Modelling and Simulation to support Human Error Risk Analysis of Partially Autonomous Driver Assistance Systems). The authors would like to specially thank the ISi-PADAS consortium that has supported the development of this research.


  1. 1.
    Alsen J, et al (2010). Integrated human modelling and simulation to support human error risk analysis of partially autonomous driver assistance systems: the ISI-PADAS project. Transport Research Arena Europe, BrusselsGoogle Scholar
  2. 2.
    Puterman ML (1994) Markov decision processes: discrete stochastic dynamic programming. Wiley, New YorkzbMATHGoogle Scholar
  3. 3.
    Bradtke SJ, Barto AG (1996). Linear least-squares algorithms for temporal difference learning. J Mach Learn Res 22(2):33–57Google Scholar
  4. 4.
    Sutton R, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, CambridgeGoogle Scholar
  5. 5.
    Muhrer E, Vollrath M (2009) Final report for task 1.22–results from accident analysis. Technical report, ISI-PADAS projectGoogle Scholar
  6. 6.
    Vollrath M et al (2006) Ableitung von anforderungen an ein fahrerassistenzsystem aus sicht der verkehrssicherheit, berichte der bundesanstalt für strassenwesen, fahrzeugtechnik, heft F 60. Wirtschaftsverlag NW, Bremerhaven, in German languageGoogle Scholar
  7. 7.
    Briest S, Vollrath M (2006) In welchen situationen machen fahrer welche fehler? Ableitung von anforderungen an fahrerassistenzsysteme durch, in-depth-unfallanalysen. In VDI (ed), Integrierte sicherheit und fahrerassistenzsysteme. VDI, Wolfsburg, pp 449–463, in German languageGoogle Scholar
  8. 8.
    Kiefer RJ, Salinger J, Ference JJ (2005) Status of NHTSA’s rear-end crash prevention research program, National Highway Traffic and Safety Administration, paper number 05-0282Google Scholar
  9. 9.
    Lee JD et al (2002) Collision warning timing, driver distraction and driver response to imminent rear-end collisions in a high-fidelity driving simulator. Hum Factors J 44:314–334CrossRefGoogle Scholar
  10. 10.
    Boyan J (1999) Technical update: LSTD. J Mach Learn Res 49(2-3):233–246 Google Scholar
  11. 11.
    Atkeson CG, Santamaria JC (1997) A comparison of direct and model-based reinforcement learning. In the proceedings of the international conference on robotics and automation (ICRA)Google Scholar
  12. 12.
    Lagoudakis M, Parr R (2003) Least squares policy iteration. J AI Res 4:1107–1149Google Scholar

Copyright information

© Springer-Verlag Italia Srl 2011

Authors and Affiliations

  1. 1.CRF–Centro Ricerche FiatOrbassano TOItaly
  2. 2.SUPELEC, 2 rue Edouard Bélin, Metz TechnopoleMetzFrance

Personalised recommendations