Chapter

Advances in Artificial Intelligence – SBIA 2004

Volume 3171 of the series Lecture Notes in Computer Science pp 245-254

Heuristically Accelerated Q–Learning: A New Approach to Speed Up Reinforcement Learning

  • Reinaldo A. C. BianchiAffiliated withLaboratório de Técnicas Inteligentes, Escola Politécnica da Universidade de São PauloCentro Universitário da FEI
  • , Carlos H. C. RibeiroAffiliated withInstituto Tecnológico de Aeronáutica
  • , Anna H. R. CostaAffiliated withLaboratório de Técnicas Inteligentes, Escola Politécnica da Universidade de São Paulo

* Final gross prices may vary according to local VAT.

Get Access

Abstract

This work presents a new algorithm, called Heuristically Accelerated Q–Learning (HAQL), that allows the use of heuristics to speed up the well-known Reinforcement Learning algorithm Q–learning. A heuristic function \(\mathcal{H}\) that influences the choice of the actions characterizes the HAQL algorithm. The heuristic function is strongly associated with the policy: it indicates that an action must be taken instead of another. This work also proposes an automatic method for the extraction of the heuristic function \(\mathcal{H}\) from the learning process, called Heuristic from Exploration. Finally, experimental results shows that even a very simple heuristic results in a significant enhancement of performance of the reinforcement learning algorithm.

Keywords

Reinforcement Learning Cognitive Robotics