Reinforced Learning in Market Games

Abstract

Financial markets investors are involved in many games — they must interact with other agents to achieve their goals. Among them are those directly connected with their activity on markets but one cannot neglect other aspects that influence human decisions and their performance as investors. Distinguishing all subgames is usually beyond hope and resource consuming. In this paper we study how investors facing many different games, gather information and form their decision despite being unaware of the complete structure of the game. To this end we apply reinforcement learning methods to the Information Theory Model of Markets (ITMM). Following Mengel, we can try to distinguish a class Γ of games and possible actions (strategies) $a_{m_i }^i$ for i-th agent. Any agent divides the whole class of games into analogy subclasses she/he thinks are analogous and therefore adopts the same strategy for a given subclass. The criteria for partitioning are based on profit and costs analysis. The analogy classes and strategies are updated at various stages through the process of learning. We will study the asymptotic behavior of the process and attempt to identify its crucial stages, e.g., existence of possible fixed points or optimal strategies. Although we focus more on the instrumental aspects of agents behaviors, various algorithm can be put forward and used for automatic investment. This line of research can be continued in various directions.