Abstract
This paper addresses the critical challenge of detecting financial crises in their early stages given their profound economic and societal consequences. It investigates the efficacy of machine learning models by comparing the standard econometric model of Logistic Regression with k-nearest Neighbours, Random Forest, Extremely Randomised Trees, Support Vector Machine, and artificial Neural Network models in a cross-validation experiment. The study utilises financial crisis observations between 1870 and 2020 from the MacroHistory database and up to 14 early warning indicators from different theoretical backgrounds. The results demonstrate that advanced machine learning models, particularly Random Forest and Extremely Randomised Trees, outperform Logistic Regression, as measured by the area under the receiver operating characteristic curve (AUROC). This finding holds across different crisis data sources, early warning indicator sets, and model specifications. Furthermore, this study emphasises the importance of model explainability for both researchers and policymakers by employing Accumulated Local Effect (ALE) plots to unravel the complex predictive processes of advanced machine learning models and provide deeper insight into the underlying crisis dynamics. Nonlinear connections to financial crises were identified for several early warning indicators, which are difficult to capture by linear models. Several U-shaped relationships were observed, highlighting the disruptive role of strong economic changes for financial stability. The results emphasise the potential of advanced machine learning methods to limit the societal impact of financial crises by enabling earlier intervention by policymakers and public institutions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Availability of data and code
The source code of this paper is publicly available.
Notes
The time periods of both World Wars (1914–1918, 1939–1945) will be excluded from the analysis.
The covered countries are Australia, Belgium, Canada, Denmark, Finland, France, Germany, Ireland, Italy, Japan, Netherlands, Norway, Portugal, Spain, Sweden, Switzerland, United Kingdom and the USA.
For this paper, the term “machine learning” refers exclusively to supervised machine learning, as opposed to unsupervised and reinforcement learning.
References
Alessi L, Savona R (2021) Machine learning for financial stability. In: Data Science for Economics and Finance: Methodologies and Applications, Springer International Publishing Cham, pp 65–87. https://doi.org/10.1007/978-3-030-66891-4_4
Apley DW, Zhu J (2020) Visualizing the effects of predictor variables in black box supervised learning models. J R Stat Soc 82(4). https://doi.org/10.1111/rssb.12377
Argitis G, Evans T, Michell J, Toporowski J (2014) Finance and crisis: Marxian, institutionalist and circuitist approaches. IPE Working Papers 2014(45)
Arlot S, Celisse A (2010) A survey of cross-validation procedures for model selection. Stat Surv 4:40–79. https://doi.org/10.1214/09-SS054
Axtell R, Farmer JD (forthcoming) Agent-based modeling in economics and finance: past, present, and future. J Econ Lit
Baker D (2008) The housing bubble and the financial crisis. Real-World Econ Rev 46(20):72–81
Begg I (2009) Regulation and supervision of financial intermediaries in the eu: the aftermath of the financial crisis. J Common Mark Stud 47(5):1107–1128. https://doi.org/10.1111/j.1468-5965.2009.02037.x
Bergmeir C, Benítez JM (2012) On the use of cross-validation for time series predictor evaluation. Inf Sci 191:192–213. https://doi.org/10.1016/j.ins.2011.12.028
Beutel J, List S, von Schweinitz G (2019) Does machine learning help us predict banking crises? J Financ Stab 45. https://doi.org/10.1016/j.jfs.2019.100693
Beutel J, List S, von Schweinitz G (2018) An evaluation of early warning models for systemic banking crises: does machine learning improve predictions? Deut Bundesbank Discus Pap 2018(48). https://doi.org/10.2139/ssrn.3312608
Blanton RG, Blanton SL, Peksen D (2018) The gendered consequences of financial crises: a cross-national analysis. Polit Gend 15:941–970. https://doi.org/10.1017/s1743923x18000545
Block F (2015) A neo-polanyian theory of economic crises. Am J Econ Sociol 74(2):361–378. https://doi.org/10.1111/ajes.12095
Bluwstein K, Buckmann M, Joseph A, Kapadia S, Şimşek Özgür (2023) Credit growth, the yield curve and financial crisis prediction: evidence from a machine learning approach. J Int Econ 145. https://doi.org/10.1016/j.jinteco.2023.103773
Bookstaber R (2017) Agent-based models for financial crises. Ann Rev Financ Econ. https://doi.org/10.1146/annurev-financial-110716-032556
Breiman L (2001) Random forests. Mach Learn 45:5–32. https://doi.org/10.1023/A:1010933404324
Brunnermeier MK, Oehmke M (2013) Bubbles, financial crises, and systemic risk. Handbook of the Economics of Finance 2:1221–1288. https://doi.org/10.1016/B978-0-44-459406-8.00018-4
Bussiere M (2006) Fratzscher M (2006) Towards a new early warning system of financial crises. J Intern Money Financ 25:953–973. https://doi.org/10.1016/j.jimonfin.2006.07.007
Casabianca EJ, Catalano M, Forni L, Giarda E, Passeri S (2019) An early warning system for banking crises: from regression-based analysis to machine learning techniques. Marco Fanno Working Papers, 235
Chen S, Svirydzenka K (2021) Financial cycles - early warning indicators of banking crises? IMF Working Paper. https://doi.org/10.5089/9781513582306.001
Cover TM, Hart PE (1967) Nearest neighbor pattern classification. IEEE Trans Inf Theory 13(1):21–27. https://doi.org/10.1109/TIT.1967.1053964
Crotty J (2009) Structural causes of the global financial crisis: a critical assessment of the ’new financial architecture. Camb J Econ 33(4):563–580. https://doi.org/10.1093/cje/bep023
Cutler A, Cutler DR, Stevens JR (2012) Random forests. In: Ensemble machine learning, Springer, pp 157–175. https://doi.org/10.1007/978-1-4419-9326-7_5
Davanzati GF, Pacella A (2014) Thorstein veblen on credit and economic crises. Camb J Econ 38(5):1043–1061. https://doi.org/10.1093/cje/beu002
De Ville B (2013) Decision trees. Wiley Interdiscip Rev Comput Stat 5(6):448–455. https://doi.org/10.1002/wics.1278
Detzer D, Herr H (2014) Theories of financial crises - an overview. IPE Working Papers, 2014(32)
Döpke J, Fritsche U, Pierdzioch C (2017) Predicting recessions with boosted regression trees. Int J Forecast 33(4):745–759. https://doi.org/10.1016/j.ijforecast.2017.02.003
Drehmann M, Borio C, Tsatsaronis K (2011) Anchoring countercyclical capital buffers: the role of credit aggregates. Int J Cent Bank 7(4):189–240
Fawcett T (2006) An introduction to roc analysis. Pattern Recogn Lett 27(8):861–874. https://doi.org/10.1016/j.patrec.2005.10.010
Feurer M, Hutter F (2019) Hyperparameter optimization. In: Hutter F, Kotthoff L, Vanschoren J (eds) Automated Machine Learning: Methods, Systems, Challanges., Springer, pp 3–33. https://doi.org/10.1007/978-3-030-05318-5_1
Géron A (2017) Hands-On Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques for Building Intelligent Systems. O’Reilly UK Ltd
Geurts P, Ernst D, Wehenkel L (2006) Extremely randomized trees. Mach Learn 63(1):3–42. https://doi.org/10.1007/s10994-006-6226-1
Gogas P, Papadimitriou T, Sofianos E (2022) Forecasting unemployment in the euro area with machine learning. J Forecast 41(3):551–566. https://doi.org/10.1002/for.2824
Greenwood R, Hanson SG, Shleifer A, Sørensen JA (2022) Predictable financial crises. J Financ 77(2):863–921. https://doi.org/10.1111/jofi.13105
Guenther N, Schonlau M (2016) Support vector machines. Stata J 16(4):917–937. https://doi.org/10.1177/1536867X1601600407
Hardy DC, Pazarbasioglu C (1999) Determinants and leading indicators of banking crises: further evidence. IMF Staff Pap 46(3). https://doi.org/10.5089/9781451973686.024
Hastie T, Tibshirani R, Friedman J (2009) The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd edn. Springer. https://doi.org/10.1007/978-0-387-84858-7
Hawkins DM (2004) The problem of overfitting. Chem Inf Comput Sci 44. https://doi.org/10.1021/ci0342472
Heijdra BJ (2009) Foundations of Modern Macroeconomics, 2nd edn. Oxford University Press
Hermansen M, Röhn O (2017) Economic resilience: the usefulness of early warning indicators in oecd countries. OECD J Econ Stud. https://doi.org/10.1787/eco_studies-2016-5jg2ppjrd6r3
Holopainen M, Sarlin P (2017) Towards robust early-warning models: a horse race, ensembles and model uncertainty. Quant Finan 17(12). https://doi.org/10.1080/14697688.2017.1357972
James G, Witten D, Hastie T, Tibshirani R (2021) An Introduction to Statistical Learning, 2nd edn. Springer. https://doi.org/10.1007/978-1-4614-7138-7
Kamath U, Liu J (2021) Explainable Artificial Intelligence: an Introduction to Interpretable Machine Learning. Springer Cham. https://doi.org/10.1007/978-3-030-83356-5
Keen S (2012) Growth theory. In: The Elgar Companion to Post Keynesian Economics, Edward Elgar, pp 271–277
Laeven L, Valencia F (2018) Systemic banking crises revisited. IMF Working Paper 2018(206). https://doi.org/10.2139/ssrn.3267233
Lawrence M, Homer-Dixon T, Janzwood S, Rockström J, Renn O, Donges J (2024) Global polycrisis: The causal mechanisms of crisis entanglement. Glob Sustain 7. https://doi.org/10.1017/sus.2024.1
LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444. https://doi.org/10.1038/nature14539
Li L, Doroslovački M, Loew M (2020) Approximating the gradient of cross-entropy loss function. IEEE Access 8. https://doi.org/10.1109/ACCESS.2020.3001531
Liu L, Chen C, Wang B (2020) Predicting financial crises with machine learning methods. J Forecast 41(5):871–910. https://doi.org/10.1002/for.2840
Lo-Duca M, Koban A, Basten M, Bengtsson E, Klaus B, Kusmierczyk P, Lang JH, Detken C, Peltonen T (2017) A new database for financial crises in european countries. ECB Occasional Paper Series 194
Lucas RE (1977) Understanding business cycles. Carnegie-Rochester Conference Series on Public Policy, pp 7–29. https://doi.org/10.1016/0167-2231(77)90002-1
Lucas RE (1972) Expectations and the neutrality of money. J Econ Theory 4:103–124. https://doi.org/10.1016/0022-0531(72)90142-1
Mammone A, Turchi M, Cristianini N (2009) Support vector machines. WIREs. Comput Stat 1(3):283–289. https://doi.org/10.1002/wics.49
Minsky HP (1977) The financial instability hypothesis: an interpretation of keynes and an alternative to “standard” theory. Neb J Econ Bus 16(1):5–16
Minsky HP (1992) The financial instability hypothesis. Levy Economics Institute Working Paper, 74
Mitchell T (1997) Machine Learning. McGraw-Hill
Molnar C (2022) Interpretable Machine Learning, 2nd edn. https://christophm.github.io/interpretable-ml-book. Accessed 20 Aug 2023
Morgan J, Negru I (2012) The austrian perspective on the global financial crisis: a critique. Econ Iss 17(2)
Murdoch WJ, Singh C, Kumbier K, Abbasi-Asl R, Yu B (2019) Definitions, methods, and applications in interpretable machine learning. Proc Natl Acad Sci 116(44). https://doi.org/10.1073/pnas.1900654116
Òscar Jordá, Schularick M, Taylor AM (2017) Macrofinancial history and the new business cycle facts. In: Eichenbaum M, Parker JA (eds) NBER Macroeconomics Annual 2016, vol 32. https://doi.org/10.1086/690241
Òscar Jordá, Schularick M, Taylor AM (2021) Jst financial crisis chronology
Ozden E, Guleryuz D (2022) Optimized machine learning algorithms for investigation the relationship between economic development and human capital. Comput Econ. https://doi.org/10.1007/s10614-021-10194-7
Pauly LW (2009) The old and the new politics of international financial stability. J Common Mark Stud 47(5):955–975. https://doi.org/10.1111/j.1468-5965.2009.02030.x
Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: Machine learning in Python. J Mach Learn Res 12:2825–2830
Quinlan J (1990) Probabilistic decision trees. In: Kodratoff Y, Michalski RS (eds) Machine Learning: An Artificial Intelligence Approach, Morgan Kaufmann, pp 140–152. https://doi.org/10.1016/B978-0-08-051055-2.50011-0
Reinhart CM, Rogoff KS (2008) Is the 2007 u.s. sub-prime financial crisis so different? an international historical comparison. Am Econ Rev 98(2):339–344. https://doi.org/10.1257/aer.98.2.339
Reinhart CM, Rogoff KS (2009) The aftermath of financial crises. Am Econ Rev 99(2). https://doi.org/10.1257/aer.99.2.466
Reinhart CM (2022) From health crisis to financial distress. IMF Econ Rev 4–31. https://doi.org/10.1057/s41308-021-00152-6
Rewilak J (2018) The impact of financial crises on the poor. J Int Dev 30:3–19. https://doi.org/10.1002/jid.3334
Rumelhart DE, McClelland JL (1988) Learning internal representations by error propagation. Readings in Cognitive Science pp 399–421. https://doi.org/10.1016/B978-1-4832-1446-7.50035-2
Shalev-Shwartz S, Ben-David S (2014) Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press. https://doi.org/10.1017/CBO9781107298019
Sherman H (2001) The business cycle theory of wesley mitchell. J Econ Issues 35(1):85–97. https://doi.org/10.1080/00213624.2001.11506341
Shiller RJ (2002) Bubbles, human judgment, and expert opinion. Financ Anal J. https://doi.org/10.2469/faj.v58.n3.2535
Tölö E (2020) Predicting systemic financial crises with recurrent neural networks. J Financ Stab 49. https://doi.org/10.1016/j.jfs.2020.100746
Tooze A (2018) Crashed: How a Decade of financial crises changed the world. Penguin Books
Tversky A, Kahneman D (1979) Judgment under uncertainty: Heuristics and biases. Science 185(4157). https://doi.org/10.1126/science.185.4157.1124
Verbeek M (2004) A Guide to Modern Econometrics. John Wiley & Sons
von Hayek FA (1975) Profits, interest and investment and other essays on the theory of industrial fluctuations, 2nd edn. Kelley Publishers, Augustus M
von Mises L (1954) The Theory of Money and Credit, 2nd edn. Yale University Press
Vrontos SD, Galakis J, Vrontos ID (2021) Modeling and predicting u.s. recessions using machine learning techniques. Int J Forecast 37(2):647–671. https://doi.org/10.1016/j.ijforecast.2020.08.005
Welch J (2011) The financial crisis in the european union: an impact assessment and response critique. Eur J Risk Regul 2(4):481–490. https://doi.org/10.1017/S1867299X00001550
Wicksell K (1898) Interest and Prices. Sentry Press
Wolpert DH (1996) The lack of a priori distinctions between learning algorithms. Neural Comput 8(7):1341–1390. https://doi.org/10.1162/neco.1996.8.7.1341
Yang L, Shami A (2020) On hyperparameter optimization of machine learning algorithms: theory and practice. Neurocomputing 415:295–316. https://doi.org/10.1016/j.neucom.2020.07.061
Zhang P (1993) Model selection via multifold cross-validation. Ann Stat 21:299–311. https://doi.org/10.1214/aos/1176349027
Zhao L, Rasoulinezhad E, Sarker T, Taghizadeh-Hesary F (2023) Effects of covid-19 on global financial markets: evidence from qualitative research for developed and developing economies. Eur J Dev Res. https://doi.org/10.1057/s41287-021-00494-x
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
The author declares no competing interests.
Appendix
Appendix
Observations of financial crises from 16 OECD countries between 1970 and 2020 from Laeven and Valencia (2018)
Observations of financial crises from 13 OECD countries between 1870 and 2020 from the ESRB (Lo-Duca et al. 2017)
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Reimann, C. Predicting financial crises: an evaluation of machine learning algorithms and model explainability for early warning systems. Rev Evol Polit Econ (2024). https://doi.org/10.1007/s43253-024-00114-4
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s43253-024-00114-4