Applying External Guidance Commands to Deep Reinforcement Learning for Autonomous Driving

Abstract

End-to-end deep reinforcement learning [1] algorithms used in autonomous car field and trained on lane-keeping task achieve good results in roads that don’t require decision making but cannot deal with situations where getting driving direction is mandatory like choosing to turn left or right in an upcoming crossroads, deciding when to leave a traffic circle or toward which path/destination to go. In this paper we introduce a new Deep Reinforcement Learning model that enable to integrate guidance commands at test time as a complementary input that indicate the right direction, that we call Deep Reinforcement Learning with guidance (DRLG), we apply the DRLG architecture on two algorithms, the asynchronous advantage actor-critic A3C and the Deep Deterministic Policy Gradient algorithm DDPG. For the training and experimentations of the new model, we adopt the CARLA virtual environment, a High-fidelity realistic driving simulator as a testbed since leading driving tests in the real world turns out to be neither safe nor affordable in term of materials and requirements. The results of testing show that DDPG and A3C with Guidance (DDPGG and A3CG) models succeed on their driving task through roads/roundabouts, by being appropriately responsive to the external commands, which allow to the autonomous car to follow the indicated route and take the right turns.