Causal Analytics and Risk Analytics

  • Louis Anthony Cox Jr.
  • Douglas A. Popken
  • Richard X. Sun
Part of the International Series in Operations Research & Management Science book series (ISOR, volume 270)


Countless books and articles on data science and analytics discuss descriptive analytics, predictive analytics, and prescriptive analytics. An additional analytics area that is much less discussed links this world of analytics, with its statistical model-based descriptions and predictions, to the world of practical decisions in which actions have consequences that decision-makers, and perhaps other stake-holders, care about, and about which they are often uncertain. This is the area of causal analytics. How causal analytics relates to other analytics areas and how its methods can be used to predict what to expect next, explain past outcomes and observations, prescribe what to do next to improve future outcomes, and evaluate how well past or current policies accomplish their intended goals—for whom, and under what conditions—are the main topics of this book.


  1. Abdoos M, Mozayani N, Bazzan ALC (2014) Hierarchical control of traffic signals using Q-learning with tile coding. Appl Intell 40(2):201–213CrossRefGoogle Scholar
  2. Agrawal S, Goyal N (2012) Analysis of Thompson sampling for the multi-armed bandit problem. J Mach Learn Res 23(39):39.1–39.26. Scholar
  3. Akhavan-Tabatabaei R, Sánchez DM, Yeung TG (2017) A Markov decision process model for cervical cancer screening policies in Colombia. Med Decis Mak 37(2):196–211. Scholar
  4. Amacher GS, Ollikainen M, Koskela E (2009) Economics of forest resources. The MIT Press, Cambridge, MAGoogle Scholar
  5. Amato C, Oliehoeck (2015) Scalable planning and learning for multiagent POMDPs. In: Proceedings of the twenty-ninth AAAI conference on artificial intelligence: 1995–2002.
  6. Amaran S, Sahinidis NV, Bikram S, Bury SJ (2016) Simulation optimization: a review of algorithms and applications. Ann Oper Res 240(1):351–380CrossRefGoogle Scholar
  7. Ashcroft M (2013) Performing decision-theoretic inference in Bayesian network ensemble models. In: Jaeger M, Nielsen TD, Viappiani P (eds) Twelfth scandinavian conference on artificial intelligence, vol 257, pp 25–34Google Scholar
  8. Auer P, Cesa-Bianchi N, Fischer P (2002) Finite-time analysis of the multiarmed bandit problem. Mach Learn 47(2/3):235–256. Scholar
  9. Averbeck BB (2015) Theory of choice in bandit, information sampling and foraging tasks. PLoS Comput Biol 11(3):e1004164. Scholar
  10. Bala MV, Mauskopf JA (2006) Optimal assignment of treatments to health states using a Markov decision model: an introduction to basic concepts. PharmacoEconomics 24(4):345–354CrossRefGoogle Scholar
  11. Bareinboim E, Pearl J (2013) Causal transportability with limited experiments. In: Proceedings of the 27th AAAI conference on artificial intelligence, pp 95–101.
  12. Beck JL, Zuev KM (2017) Rare event simulation. In: Ghanem R, Higdon D, Owhadi H (eds) Handbook of uncertainty quantification. Springer, New York. Scholar
  13. Bennett CC, Hauser K (2013) Artificial intelligence framework for simulating clinical decision-making: a Markov decision process approach. Artif Intell Med 57(1):9–19. Scholar
  14. Bertsekas DM, Shreve SE (1996) Stochastic optimal control: the discrete-time case. Athena Scientific, Belmont, MAGoogle Scholar
  15. Bier VM, Cox LA Jr (2017) Coping with uncertainty in adversarial risk models. In: Abbas A, Tambe M, von Winterfeldt D (eds) Improving homeland security decisions. Cambridge University Press, New YorkGoogle Scholar
  16. Browne CB, Powley E, Whitehouse D, Lucas SM, Cowling PI, Rohlfshagen P, Taverner S, Pere D, Samothrakis S, Colton S (2012) A survey of Monte Carlo tree search methods. IEEE Trans Comput Intell AI Games 4(1):1–43CrossRefGoogle Scholar
  17. Box GEP (1957) Evolutionary Operation: a method for increasing industrial productivity. J R Stat Soc Ser C Appl Stat 6(2):81–101. Scholar
  18. Cami A, Wallstrom GL, Hogan WR (2009) Measuring the effect of commuting on the performance of the bayesian aerosol release detector. BMC Med Inform Decis Mak 9(Suppl 1):S7CrossRefGoogle Scholar
  19. Campbell DT, Stanley JC (1963) Experimental and quasi-experimental designs for research. Houghton Mifflin Company, Boston, MAGoogle Scholar
  20. Cao Q, Buskens E, Feenstra T, Jaarsma T, Hillege H, Postmus D (2016) Continuous-time semi-Markov models in health economic decision making: an illustrative example in heart failure disease management. Med Decis Mak 36(1):59–71. Scholar
  21. Cartwright N (2003) Two theorems on invariance and causality. Philos Sci 70:203–224. Scholar
  22. Chakraborty M, Chua KYP, Das S, Juba B (2017) Coordinated versus decentralized exploration in multi-agent multi-armed bandits. In: Proceedings of the twenty-sixth international joint conference on artificial intelligence (IJCAI), Melbourne, Australia, pp 164–170.
  23. Choi E, Schuetz A, Stewart WF, Sun J (2017) Using recurrent neural network models for early detection of heart failure onset. J Am Med Inform Assoc 24(2):361–370. Scholar
  24. Clancy L, Goodman P, Sinclair H, Dockery DW (2002) Effect of air-pollution control on death rates in Dublin, Ireland: an intervention study. Lancet 360(9341):1210–1214CrossRefGoogle Scholar
  25. Cox LA Jr (2008) What’s wrong with risk matrices? Risk Anal 28(2):497–512CrossRefGoogle Scholar
  26. Dayer MJ, Jones S, Prendergast B, Baddour LM, Lockhart PB, Thornhill MH (2015) Incidence of infective endocarditis in England, 2000–13: a secular trend, interrupted time-series analysis. Lancet 385(9974):1219–1228. Scholar
  27. Dockery DW, Rich DQ, Goodman PG, Clancy L, Ohman-Strickland P, George P, Kotlov T, HEI Health Review Committee (2013) Effect of air pollution control on mortality and hospital admissions in Ireland. Res Rep Health Eff Inst 176:3–109Google Scholar
  28. Dorfman R (1969) An economic interpretation of optimal control theory. Am Econ Rev 59(5):817–831Google Scholar
  29. Doucet A, Johansen AM (2009) A tutorial on particle filtering and smoothing: fifteen years later. In: Crisan D, Rozovsky B (eds) The oxford handbook of nonlinear filtering. Oxford University Press, OxfordGoogle Scholar
  30. Dupac V (1965) A dynamic stochastic approximation method. Ann Math Statist 36(6):1695–1702CrossRefGoogle Scholar
  31. Feng X, Shekhar A, Yang F, Hebner RE, Bauer P (2017) Comparison of hierarchical control and distributed control for microgrid. Elect Power Comp Syst 45(10):1043–1056. Scholar
  32. Fu MC (2016) AlphaGo and Monte Carlo tree search: the simulation optimization perspective. In: Proceedings of the winter simulation conference (WSC), 11–14 December 2016. IEEE, Washington, DC.
  33. Fu MC (ed) (2015) Handbook of simulation optimization. Springer, New York. Scholar
  34. Ganger M, Duryea E, Hu W (2016) Double Sarsa and double expected Sarsa with shallow and deep learning. J Data Analy Inf Process 4:159–176. Scholar
  35. Gasparrini A, Gorini G, Barchielli A (2009) On the relationship between smoking bans and incidence of acute myocardial infarction. Eur J Epidemiol 24(10):597–602CrossRefGoogle Scholar
  36. Gilmour S, Degenhardt L, Hall W, Day C (2006) Using intervention time series analyses to assess the effects of imperfectly identifiable natural events: a general method and example. BMC Med Res Methodol 6:16CrossRefGoogle Scholar
  37. Goehler A, Geisler BP, Manne JM, Jahn B, Conrads-Frank A, Schnell-Inderst P, Gazelle GS, Siebert U (2011) Decision-analytic models to simulate health outcomes and costs in heart failure: a systematic review. PharmacoEconomics 29(9):753–769CrossRefGoogle Scholar
  38. Gómez, V, Thijssen, S, Symington, A, Hailes, S, Kappen, HJ (2016) Real-time stochastic optimal control for multi-agent quadrotor systems. In: Proceedings of the 26th international conference on automated planning and scheduling (ICAPS’16), June 12–17. AAAI Press, London, UK.
  39. Grundmann O (2014) The current state of bioterrorist attack surveillance and preparedness in the US. Risk Manag Healthc Policy 7:177–187CrossRefGoogle Scholar
  40. Heinze-Deml C, Peters J, Meinshausen N (2017) Invariant causal prediction for nonlinear models.
  41. Ho Y-C, Chu K-C (1972) Team decision theory and information structures in optimal control problems—part I. IEEE Trans Autom Control 17(1):15–22CrossRefGoogle Scholar
  42. Höfler M (2005) Causal inference based on counterfactuals. BMC Med Res Methodol 5:28CrossRefGoogle Scholar
  43. Hoover KD (2014) Reductionism in economics: causality and intentionality in the microfoundations of macroeconomics. CHOPE Working Paper, No. 2014–03.
  44. Ilic MD, Liu S (1996) Hierarchical power systems control: its value in achanging industry. Springer, HeidelbergCrossRefGoogle Scholar
  45. James NA, Matteson DS (2014) ecp: an r package for nonparametric multiple change point analysis of multivariate data. J Stat Softw 62(7):1–25CrossRefGoogle Scholar
  46. Kaelbling LP, Littman ML, Moore AW (1996) Reinforcement learning: a survey. J Artif Intell Res Arch 4(1):237–285. Scholar
  47. Kalai A, Vempala S (2005) Efficient algorithms for online decision problems. J Comput Syst Sci 71:291–307. Scholar
  48. Kale DC, Che Z, Bahadori MT, Li W, Liu Y, Wetzel R (2015) Causal phenotype discovery via deep networks. AMIA Annu Symp Proc 2015:677–686Google Scholar
  49. Kamien KI, Schwartz NL (2012) Dynamic optimization: the calculus of variations and optimal control in economics and management, 2nd edn. Dover Publications, Mineola, NYGoogle Scholar
  50. Katt S, Oliehoek FA, Amato C (2017) Learning in POMDPs with monte carlo tree search. In: Proceedings of the 34th international conference on machine learning, Sydney, Australia, PMLR 70.
  51. Koller D, Parr R (1999) Computing factored value functions for policies in structured MDPs. In: Proceedings of the sixteenth international joint conference on artificial intelligence (IJCAI99), Stockholm, Sweden, July 31–August 6. Morgan Kaufmann, San Francisco, CAGoogle Scholar
  52. Koller D, Milch B (2003) Multi-agent influence diagrams for representing and solving games. Games Econ Behav 45(1):181–221. Scholar
  53. Król A, Saint-Pierre P. (2015) SemiMarkov: an R package for parametric estimation in multi-state semi-markov models. J Stat Softw 66(5).
  54. Lange K, Chi EC, Zhou H (2014) A brief survey of modern optimization for statisticians. Int Stat Rev 82(1):46–70CrossRefGoogle Scholar
  55. Lee S, Honavar V (2013) m-transportability: transportability of a causal effect from multiple environments. In: Proceedings of the twenty-seventh AAAI conference on artificial intelligence.
  56. Liao L, Ahn H (2016) Combining deep learning and survival analysis for asset health management. Int J Prognost Health Manag 7:7. Scholar
  57. Lopiano KK, Smith RL, Young SS (2015) Air quality and acute deaths in California, 2000–2012.
  58. Luce RD, Raiffa H (1957) Games and decisions: introduction and critical survey. Wiley, New YorkGoogle Scholar
  59. Marschak J, Radner R (1972) Economic theory of teams. Cowles Foundation for Research in Economics at Yale University, Monograph 22. Yale University Press, New HavenGoogle Scholar
  60. Morgan MG, Henrion M (1990) Chapter 10 of uncertainty: a guide to dealing with uncertainty in quantitative risk and policy analysis. Cambridge University Press, New York, reprinted in 1998.
  61. Myerson RB (1991) Game theory: analysis of conflict. Harvard University Press, Cambridge, MAGoogle Scholar
  62. Ortega PA, Braun DA (2014) Generalized Thompson sampling for sequential decision-making and causal inference. Complex Adapt Syst Model 2:2. Scholar
  63. Osborne MJ (2004) An introduction to game theory. Oxford University PressGoogle Scholar
  64. Patsopoulos NA (2011) A pragmatic view on pragmatic trials. Dialogues Clin Neurosci 13(2):217–224Google Scholar
  65. Pearl J (2009) Causal inference in statistics: an overview. Stat Surv 3:96–146. Scholar
  66. Pepels T, Cazenave T, Winands MHM, Lanctot M (2014) Minimizing simple and cumulative regret in Monte-Carlo tree search. In: Cazenave T, Winands MHM, Björnsson Y (eds) Computer Games. CGW 2014. Communications in Computer and Information Science, vol 504. Springer, Cham. Scholar
  67. Peters J, Bühlmann P, Meinshausen N (2016) Causal inference using invariant prediction: identification and confidence intervals. J R Stat Soc Ser B 78(5):947–1012. Scholar
  68. Ross GJ (2015) Parametric and nonparametric sequential change detection in R: the cpm package. J Stat Softw 66(3)Google Scholar
  69. Ross S, Pineau J, Brahim C, Kreitmann P (2011) Bayesian approach for learning and planning in partially observable markov decision processes. J Mach Learn Res 12(2):1729–1770Google Scholar
  70. Shachter RD, Bhattacharjya D (2010) Solving influence diagrams: exact algorithms. In: Cochran J et al (eds) Wiley encyclopedia of operations research and management science. Wiley, New York. Scholar
  71. Shackleton M, Sødal S (2010) Harvesting and recovery decisions under uncertainty. J Econ Dyn Control 34(12):2533–2546. Scholar
  72. Shan G, Pineau J, Kaplow R (2013) A survey of point-based POMDP solvers. Auton Agent Multi-Agent Syst 27(1):1–51. Scholar
  73. Shen Y, Cooper GF (2010) A new prior for Bayesian anomaly detection: application to biosurveillance. Methods Inf Med 49(1):44–53Google Scholar
  74. Shoham Y, Leyton-Brown K (2009) Multiagent systems: ALgorithimic, game-theoretic, and logical foundations. Cambridge University Press.
  75. Silver D, Veness J (2010) Monte-Carlo planning in large POMDPs. Advances in Neural Information Processing Systems 23 (NIPS)Google Scholar
  76. Simon HA, Iwasaki Y (1988) Causal ordering, comparative statics, and near decomposability. J Econ 39:149–173. Scholar
  77. Simpson KN, Strassburger A, Jones WJ, Dietz B, Rajagopalan R (2009) Comparison of Markov model and discrete-event simulation techniques for HIV. PharmacoEconomics 27(2):159–165. Scholar
  78. Sutton RS, Barto AG, Williams RJ (1992) Reinforcement learning is direct adaptive control. IEEE Control Syst 12(2):19–22. Accessed 9 Oct 17
  79. Taghipour S, Caudrelier LN, Miller AB, Harvey B (2017) Using simulation to model and validate invasive breast cancer progression in women in the study and control groups of the Canadian National Breast Screening Studies I and II. Med Decis Mak 37(2):212–223. Scholar
  80. Tartakovsky A, Nikiforov I, Basseville M (2014) Sequential analysis: hypothesis testing and changepoint detection. Chapman and Hall/CRC, Boca RatonGoogle Scholar
  81. Thomas LC (2003) Games, theory and applications. Dover Publications, Mineola, NYGoogle Scholar
  82. Tolpin D, Shimony S (2012) MCTS based on simple regret. In: Proc. Assoc. Adv. Artif. Intell. pp 570–576.
  83. Tsoukalas A, Albertson T, Tagkopoulos I (2015) From data to optimal decision making: a data-driven, probabilistic machine learning approach to decision support for patients with sepsis. JMIR Med Inform 3(1):e11. Scholar
  84. Van Seijen H, Van Hasselt H, Whiteson S, Wiering M (2009) A theoretical and empirical analysis of expected Sarsa. In: 2009 I.E. symposium on adaptive dynamic programming and reinforcement learning, Nashville, 30 March–2 April 2009, pp 177–184.
  85. Wilson RB (1968) The theory of syndicates. Econometrica 36(1):119–132CrossRefGoogle Scholar
  86. Yüksel S, Saldi N (2017) Convex analysis in decentralized stochastic control, strategic measures, and optimal solutions. SIAM J Control Optim 55(1):1–28CrossRefGoogle Scholar
  87. White H, Sabarwei S (2014) Quasi-experimental design and methods. UNICEF Office of Research. Methodological Briefs Impact Evaluation No. 8. UNICEF Office of Research—Innocenti, Florence, Italy.
  88. Zarchan P, Musoff H (2015) Fundamentals of kalman filtering: a practical approach, 4th edn. American Institute of Aeronautics and Astronautics, Reston, VACrossRefGoogle Scholar
  89. Che Z, Purushotham S, Khemani R, Liu Y (2016) Interpretable deep models for ICU outcome prediction. AMIA Annu Symp Proc 2016:371–380Google Scholar
  90. Zigler CM, Dominici F (2014) Point: clarifying policy evidence with potential-outcomes thinking—beyond exposure-response estimation in air pollution epidemiology. Am J Epidemiol 180(12):1133–1140CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Louis Anthony Cox Jr.
    • 1
  • Douglas A. Popken
    • 2
  • Richard X. Sun
    • 3
  1. 1.Cox AssociatesDenverUSA
  2. 2.Cox AssociatesLittletonUSA
  3. 3.Cox AssociatesEast BrunswickUSA

Personalised recommendations