Compound Reinforcement Learning: Theory and an Application to Finance

  • Tohgoroh Matsui
  • Takashi Goto
  • Kiyoshi Izumi
  • Yu Chen
Conference paper

DOI: 10.1007/978-3-642-29946-9_31

Part of the Lecture Notes in Computer Science book series (LNCS, volume 7188)
Cite this paper as:
Matsui T., Goto T., Izumi K., Chen Y. (2012) Compound Reinforcement Learning: Theory and an Application to Finance. In: Sanner S., Hutter M. (eds) Recent Advances in Reinforcement Learning. EWRL 2011. Lecture Notes in Computer Science, vol 7188. Springer, Berlin, Heidelberg

Abstract

This paper describes compound reinforcement learning (RL) that is an extended RL based on the compound return. Compound RL maximizes the logarithm of expected double-exponentially discounted compound return in return-based Markov decision processes (MDPs). The contributions of this paper are (1) Theoretical description of compound RL that is an extended RL framework for maximizing the compound return in a return-based MDP and (2) Experimental results in an illustrative example and an application to finance.

Keywords

Reinforcement learning compound return value functions finance 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Tohgoroh Matsui
    • 1
  • Takashi Goto
    • 2
  • Kiyoshi Izumi
    • 3
    • 4
  • Yu Chen
    • 3
  1. 1.Chubu UniversityKasugaiJapan
  2. 2.Bank of Tokyo-Mitsubishi UFJ, Ltd.TokyoJapan
  3. 3.The University of TokyoTokyoJapan
  4. 4.JST PRESTOTokyoJapan

Personalised recommendations