A Markov Game model for valuing actions, locations, and team performance in ice hockey


We apply the Markov Game formalism to develop a context-aware approach to valuing player actions, locations, and team performance in ice hockey. The Markov Game formalism uses machine learning and AI techniques to incorporate context and look-ahead. Dynamic programming is applied to learn value functions that quantify the impact of actions on goal scoring. Learning is based on a massive new dataset, from SportLogiq, that contains over 1.3M events in the National Hockey League. The SportLogiq data include the location of an action, which has previously been unavailable in hockey analytics. We give examples showing how the model assigns context and location aware values to a large set of 13 action types. Team performance can be assessed as the aggregate value of actions performed by the team’s players, or the aggregate value of states reached by the team. Model validation shows that the total team action and state value both provide a strong indicator predictor of team success, as measured by the team’s average goal ratio.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11


  1. 1.

    Pulling the goalie can also result in a skater manpower advantage.


  1. Cervone D, D’Amour A, Bornn L, Goldsberry K (2014) Pointwise: predicting points and valuing decisions in real time with NBA optical tracking data. In: 8th Annual MIT sloan sports analytics conference, February, vol 28

  2. Frey BJ, Dueck D (2007) Clustering by passing messages between data points. Science 315(5814):972–976

    MathSciNet  Article  MATH  Google Scholar 

  3. Gramacy R, Jensen S, Taddy M (2013) Estimating player contribution in hockey with regularized logistic regression. J Quant Anal Sports 9:97–111

    Google Scholar 

  4. Gunawardana A, Meek C, Xu P (2011) A model for temporal dependencies in event streams. In: Advances in neural information processing systems, pp 1962–1970

  5. Hockey Graphs (2015) https://hockey-graphs.com/2015/10/01/expected-goals-are-a-better-predictor-of-future-scoring-than-corsi-goals/

  6. Krzywicki K (2005) Shot quality model: a logistic regression approach to assessing NHL shots on goal. http://www.hockeyanalytics.com/Research_files/Shot_Quality_Krzywicki.pdf

  7. Levesque H, Pirri F, Reiter R (1998) Foundations for the situation calculus. Linköping Electron Artic Comput Inf Sci 3(18):1–18

  8. Littman ML (1994) Markov games as a framework for multi-agent reinforcement learning. In: Proceedings of the eleventh international conference on machine learning, vol 157, pp 157–163

  9. Macdonald B (2011) An improved adjusted plus-minus statistic for NHL players. In: MIT sloan sports analytics conference 2011

  10. National Hockey League (2014) National hockey league official rules 2014–2015

  11. Parikh AP, Gunawardana A, Meek C (2012) Conjoint modeling of temporal dependencies in event streams. In: UAI Bayesian modelling applications workshop

  12. Pettigrew S (2015) Assessing the offensive productivity of NHL players using in-game win probabilities. In: 9th annual MIT sloan sports analytics conference

  13. Routley K, Schulte O (2015a) A Markov game model for valuing player actions in ice hockey. In: Uncertainty in artificial intelligence (UAI). pp 782–791

  14. Routley K, Schulte O, Zhao Z (2015) Sports data mining. Springer, New York. Q-learning for the NHL. http://www.cs.sfu.ca/~oschulte/sports/

  15. Russell S, Norvig P (2010) Artificial intelligence: a modern approach. Prentice Hall, Upper Saddle River, NJ

    Google Scholar 

  16. Schuckers M, Curro J (2013) Total hockey rating (THOR): a comprehensive statistical rating of national hockey league forwards and defensemen based upon all on-ice events. In: 7th annual MIT sloan sports analytics conference

  17. Schumaker RP, Solieman OK, Chen H (2010) Sports data mining. Springer, New York

  18. Schwartz A (1993) A reinforcement learning method for maximizing undiscounted rewards. In: Proceedings of the tenth international conference on machine learning, vol 298, pp 298–305

  19. Spagnola N (2013) The complete plus-minus: a case study of the columbus blue jackets. Master’s thesis, University of South Carolina

  20. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, Cambridge

    Google Scholar 

  21. Tegen E (2015) The best predictor for future performance is expected goals. http://11tegen11.net/2015/01/05/the-best-predictor-for-future-performance-is-expected-goals/

  22. Thomas A, Ventura S, Jensen S, Ma S (2013) Competing process hazard function models for player ratings in ice hockey. Ann Appl Stat 7(3):1497–1524

    MathSciNet  Article  MATH  Google Scholar 

  23. Tulsky E (2013) Zone entries. http://nhlnumbers.com/2013/2/20/zone-entries-and-the-sloan-sports-analytics-conference

Download references


This work was supported by an Engage Grant from the National Sciences and Engineering Council of Canada. We are grateful for constructive discussions in SFU’s Sports Analytics Research Group.

Author information



Corresponding author

Correspondence to Oliver Schulte.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Schulte, O., Khademi, M., Gholami, S. et al. A Markov Game model for valuing actions, locations, and team performance in ice hockey. Data Min Knowl Disc 31, 1735–1757 (2017). https://doi.org/10.1007/s10618-017-0496-z

Download citation


  • Markov Game
  • Sports analytics
  • National Hockey League
  • Q-learning