Data Mining and Knowledge Discovery

, Volume 31, Issue 6, pp 1735–1757 | Cite as

A Markov Game model for valuing actions, locations, and team performance in ice hockey

  • Oliver SchulteEmail author
  • Mahmoud Khademi
  • Sajjad Gholami
  • Zeyu Zhao
  • Mehrsan Javan
  • Philippe Desaulniers
Part of the following topical collections:
  1. Sports Analytics


We apply the Markov Game formalism to develop a context-aware approach to valuing player actions, locations, and team performance in ice hockey. The Markov Game formalism uses machine learning and AI techniques to incorporate context and look-ahead. Dynamic programming is applied to learn value functions that quantify the impact of actions on goal scoring. Learning is based on a massive new dataset, from SportLogiq, that contains over 1.3M events in the National Hockey League. The SportLogiq data include the location of an action, which has previously been unavailable in hockey analytics. We give examples showing how the model assigns context and location aware values to a large set of 13 action types. Team performance can be assessed as the aggregate value of actions performed by the team’s players, or the aggregate value of states reached by the team. Model validation shows that the total team action and state value both provide a strong indicator predictor of team success, as measured by the team’s average goal ratio.


Markov Game Sports analytics National Hockey League Q-learning 



This work was supported by an Engage Grant from the National Sciences and Engineering Council of Canada. We are grateful for constructive discussions in SFU’s Sports Analytics Research Group.


  1. Cervone D, D’Amour A, Bornn L, Goldsberry K (2014) Pointwise: predicting points and valuing decisions in real time with NBA optical tracking data. In: 8th Annual MIT sloan sports analytics conference, February, vol 28Google Scholar
  2. Frey BJ, Dueck D (2007) Clustering by passing messages between data points. Science 315(5814):972–976MathSciNetCrossRefzbMATHGoogle Scholar
  3. Gramacy R, Jensen S, Taddy M (2013) Estimating player contribution in hockey with regularized logistic regression. J Quant Anal Sports 9:97–111Google Scholar
  4. Gunawardana A, Meek C, Xu P (2011) A model for temporal dependencies in event streams. In: Advances in neural information processing systems, pp 1962–1970Google Scholar
  5. Krzywicki K (2005) Shot quality model: a logistic regression approach to assessing NHL shots on goal.
  6. Levesque H, Pirri F, Reiter R (1998) Foundations for the situation calculus. Linköping Electron Artic Comput Inf Sci 3(18):1–18Google Scholar
  7. Littman ML (1994) Markov games as a framework for multi-agent reinforcement learning. In: Proceedings of the eleventh international conference on machine learning, vol 157, pp 157–163Google Scholar
  8. Macdonald B (2011) An improved adjusted plus-minus statistic for NHL players. In: MIT sloan sports analytics conference 2011Google Scholar
  9. National Hockey League (2014) National hockey league official rules 2014–2015Google Scholar
  10. Parikh AP, Gunawardana A, Meek C (2012) Conjoint modeling of temporal dependencies in event streams. In: UAI Bayesian modelling applications workshopGoogle Scholar
  11. Pettigrew S (2015) Assessing the offensive productivity of NHL players using in-game win probabilities. In: 9th annual MIT sloan sports analytics conferenceGoogle Scholar
  12. Routley K, Schulte O (2015a) A Markov game model for valuing player actions in ice hockey. In: Uncertainty in artificial intelligence (UAI). pp 782–791Google Scholar
  13. Routley K, Schulte O, Zhao Z (2015) Sports data mining. Springer, New York. Q-learning for the NHL.
  14. Russell S, Norvig P (2010) Artificial intelligence: a modern approach. Prentice Hall, Upper Saddle River, NJzbMATHGoogle Scholar
  15. Schuckers M, Curro J (2013) Total hockey rating (THOR): a comprehensive statistical rating of national hockey league forwards and defensemen based upon all on-ice events. In: 7th annual MIT sloan sports analytics conferenceGoogle Scholar
  16. Schumaker RP, Solieman OK, Chen H (2010) Sports data mining. Springer, New YorkGoogle Scholar
  17. Schwartz A (1993) A reinforcement learning method for maximizing undiscounted rewards. In: Proceedings of the tenth international conference on machine learning, vol 298, pp 298–305Google Scholar
  18. Spagnola N (2013) The complete plus-minus: a case study of the columbus blue jackets. Master’s thesis, University of South CarolinaGoogle Scholar
  19. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, CambridgeGoogle Scholar
  20. Tegen E (2015) The best predictor for future performance is expected goals.
  21. Thomas A, Ventura S, Jensen S, Ma S (2013) Competing process hazard function models for player ratings in ice hockey. Ann Appl Stat 7(3):1497–1524MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© The Author(s) 2017

Authors and Affiliations

  1. 1.School of Computing ScienceSimon Fraser UniversityVancouver-BurnabyCanada
  2. 2.SportLoqigMontrealCanada

Personalised recommendations