Competitive Markov Decision Processes

  • Jerzy Filar
  • Koos Vrieze

Table of contents

  1. Front Matter
    Pages i-xii
  2. Introduction

    1. Jerzy Filar, Koos Vrieze
      Pages 1-6
  3. Mathematical Programming Perspective

    1. Front Matter
      Pages 7-7
    2. Jerzy Filar, Koos Vrieze
      Pages 9-84
    3. Jerzy Filar, Koos Vrieze
      Pages 85-151
  4. Existence, Structure and Applications

    1. Front Matter
      Pages 153-153
    2. Jerzy Filar, Koos Vrieze
      Pages 155-234
    3. Jerzy Filar, Koos Vrieze
      Pages 235-300
    4. Jerzy Filar, Koos Vrieze
      Pages 301-341
  5. Back Matter
    Pages 343-393

About this book


This book is intended as a text covering the central concepts and techniques of Competitive Markov Decision Processes. It is an attempt to present a rig­ orous treatment that combines two significant research topics: Stochastic Games and Markov Decision Processes, which have been studied exten­ sively, and at times quite independently, by mathematicians, operations researchers, engineers, and economists. Since Markov decision processes can be viewed as a special noncompeti­ tive case of stochastic games, we introduce the new terminology Competi­ tive Markov Decision Processes that emphasizes the importance of the link between these two topics and of the properties of the underlying Markov processes. The book is designed to be used either in a classroom or for self-study by a mathematically mature reader. In the Introduction (Chapter 1) we outline a number of advanced undergraduate and graduate courses for which this book could usefully serve as a text. A characteristic feature of competitive Markov decision processes - and one that inspired our long-standing interest - is that they can serve as an "orchestra" containing the "instruments" of much of modern applied (and at times even pure) mathematics. They constitute a topic where the instruments of linear algebra, applied probability, mathematical program­ ming, analysis, and even algebraic geometry can be "played" sometimes solo and sometimes in harmony to produce either beautifully simple or equally beautiful, but baroque, melodies, that is, theorems.


Markov Markov chain Markov decision process algorithm algorithms linear algebra linear optimization mathematical programming modeling nonlinear optimization optimization programming

Authors and affiliations

  • Jerzy Filar
    • 1
  • Koos Vrieze
    • 2
  1. 1.School of MathematicsUniversity of South AustraliaAdelaideAustralia
  2. 2.Department of MathematicsUniversity of LimburgMaastrichtThe Netherlands

Bibliographic information