Abstract
Credit risk models are validated to check that they produce unbiased, “high-quality” estimates of credit risk. Credit risk models follow different rating philosophies, ranging from point-in-time (PIT) models that reflect all currently available information to through-the-cycle (TTC) models whose credit risk estimates are independent of cyclical changes in macroeconomic conditions. TTC models have been favoured in particular from a macro-prudential perspective because they produce more stable capital requirements for banks over the cycle, thus avoiding pro-cyclicality. This paper investigates different ways to validate TTC credit rating systems, including possibilities to separate the validation of a TTC system into the validation of its PIT component and the validation of its adjustment for the cycle. We conclude that the validation of TTC models is significantly more difficult than the validation of PIT models, which may make the regulatory promotion of TTC models questionable. We argue that the regulatory requirement of PIT models combined with a more extensive use of the counter-cyclical capital buffer as a macro-prudential policy tool could be a potentially superior alternative to address pro-cyclicality.
The views expressed in this paper are those of the authors and do not necessarily reflect the views of the European Central Bank, Oesterreichische Nationalbank or the Eurosystem.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
The Basel framework uses credit conversion factors to convert off-balance sheet exposures such as credit lines and other facilities into risk-adjusted on-balance sheet equivalents.
- 2.
Other sources of differences include the definition of default, the calculation of default rates, the partial use of IRB models, the LGD calibration and a number of other factors (see EBA 2013a).
- 3.
The fundamental challenge to validate the TTC models highlighted in this paper is not affected by the public nature of credit rating agencies’ ratings in contrast to banks’ non-public IRB PDs. However, incentive effects and reputation mechanisms may differ between rating agencies and banks.
- 4.
For example, Drehmann et al. (2012) and ECB (2015a) highlight important divergences between financial cycles (i.e. in credit volumes and a broad set of asset prices, in particular credit and property prices) and cycles in real economic activity at the shorter business cycle frequencies. Drehmann et al. (2012) see the standard business-cycle length between 6 quarters to 8 years; the financial cycle can last between 10 and 20 years. The CRR refers only to the “business cycle” in its IRB part. Instead, when discussing various reporting obligations of the European Commission, the EBA and other organisations regarding cyclicality, the CRR refers to an “economic cycle”.
- 5.
Notably, the CRR foresees in Art. 181 and 182 that banks use estimates that are “appropriate for an economic downturn” for LGDs and credit conversion factors, respectively, under the advanced IRB approach.
- 6.
The stressed pool PD reflects “stressed” macroeconomic conditions irrespective of the current state of the economy.
- 7.
EBA (2013a, p. 26) itself defines PIT ratings as an “assessment of the borrower’s ability to discharge his obligations over a relatively short horizon (e.g. a year), and so can vary considerably over the cycle. The TTC approach focuses on a longer horizon, abstracting in principle from current cyclical conditions. TTC ratings are therefore inherently more stable and less cyclical than PIT ratings.”
- 8.
Back-testing can be complemented by “benchmarking” the PDs of one model against the PDs of other, usually external, credit risk models. Benchmarking assumes that the validator knows the quality of the benchmark model. This can be statistically tested only by back-testing the benchmark model. Hence, all the arguments raised below regarding the challenges to validate TTC models apply at least for this benchmark model, independent of the additional challenges that are usually associated with benchmarking.
- 9.
An example of default correlation potentially independent of systematic risk is the joint default of several companies belonging to the same group. Such default correlation based on non-systematic risk should be irrelevant for the back-testing of credit risk models if the sample size is large enough. The formula to determine risk-weighted assets under the IRB approach reflects some degree of asset correlation and thus default correlation.
- 10.
It is important to note that the TTC nature of Standard and Poor’s ratings has been challenged in the literature, see, e.g., Amato and Furfine (2004). As a consequence, the difference between PIT PDs at the peak/bottom of the cycle and a “true” TTC PD would be even greater than based on Standard and Poor’s data. Of course, the difference also further increases in the tail of the default rate distribution, i.e. for greater quantiles.
- 11.
According to ESMA (2015), “ESMA has observed that the majority of the credit rating agencies find assessing the predictive power of their methodologies challenging. In certain cases, credit rating agencies state that their ratings are based on an ordinal rather than a cardinal ranking which limits the extent to which internal expectations are relevant to the validation of the predictive power of a methodology, given the volatility of these expectations across the economic cycle.” Notably, ESMA (2015) argues that credit rating agencies should overcome this challenge given that credit ratings are used not only for the appropriate rank ordering, e.g. for regulatory purposes in the context of the standardised approach for banks’ or insurance firms’ capital requirements according to Basel III and Solvency II. In ESMA’s (2015) view, it would raise standards in the industry if CRAs consistently use a minimum standard of statistical measures in demonstrating the predictive power of their methodologies.
- 12.
Both statistics are linearly related according to the formula AR=2A−1, were A refers to the area under the ROC curve, i.e. the ROC measure.
- 13.
In this case, a test statistic and confidence intervals are available in Engelmann et al. (2003), for example.
- 14.
Additional necessary assumptions are, for example, that rating grades are sufficiently granular to avoid differences in the pooling of debtors in rating grades (a sufficient condition is that the PDs are “continuous” in the sense that they are not bucketed into rating grades at all).
- 15.
Statistical software such as R or SAS offer many of these tests (see, e.g., the overview of statistical measures used for Basel II reports generated by SAS available at http://support.sas.com/documentation/cdl/en/mdsug/65072/HTML/default/viewer.htm#n194xndt3b3y1pn1ufc0mqbsmht4.htm).
- 16.
Furthermore, it is easier to apply statistical methods to portfolios with large sample sizes and large number of defaults such as retail portfolios or portfolios of small- and medium-sized enterprises than low-default portfolios such as portfolios of sovereigns or financial institutions.
- 17.
Art. 179 of the CRR requires that “Where institutions use different estimates for the calculation of risk weights and for internal purposes, it shall be documented and be reasonable.” Other qualitative aspects that supervisors consider include the model design, the data quality and availability and governance aspects such as the independence of the rating process (see, e.g., Deutsche Bundesbank 2003).
- 18.
This argument does not apply in the extreme and completely unrealistic case of perfect discriminatory power. In this case, the rating system could perfectly separate all defaulting and non-defaulting debtors, so that the respective PD levels (0% and 100%) would also be known.
- 19.
According to EBA (2013b), other measures include the lower sensitivity of risk weights to changes in PDs in lower credit quality ranges (more relevant in downturns), the focus on unexpected losses for capital requirements (which may even decrease in downturns as more losses become “expected”), and the possible use of Pillar 2 (see Sect. 3.3). These measures were already part of the 2006 Capital Requirements Directive, i.e. Basel II.
- 20.
Emphasis added by the author.
- 21.
This issue is raised in the part of EBA (2015) that deals with issues beyond current EBA regulatory developments based on the CRR/CRD IV package or from the recommendations of the Report on comparability and pro-cyclicality of capital requirements. The current EBA regulatory developments focus on the definition of default, risk estimates (in particular PD, LGD, conversion factors), treatment of defaulted assets, scope of the application of the IRB approach, internal risk management processes and credit risk mitigation. EBA (2015, p. 48) notes that the “BCBS is also undertaking a review of the regulatory capital framework, in particular it is evaluating options for improving consistency and comparability of regulatory capital requirements as well as alternative approaches that reduce the reliance on internal models maintaining adequate risk sensitivity. The results of work of the BCBS will subsequently be implemented through the European legislative process.”
- 22.
Ernst & Young (2014) provide further details on the ECL approach. For example, the ECL approach distinguishes between (i) the 12-month ECL which applies to all items (from initial recognition) as long as there is no significant deterioration in credit quality, and the (ii) life-time ECLs which apply when a “significant increase in credit risk” has occurred.
- 23.
According to PRA (2015, pp. 14–15), the PRA considers the variable scalar approach acceptable provided:
-
(i)
banks meet four principles aiming at ensuring that the considerable conceptual and technical challenges of the approach are properly overcome and the adjustments are carried out in an appropriate way; these four principles are (1) Both the initial calculation of and subsequent changes to the scalar must be able to take account of changes in default risk that are not purely related to the changes in the cycle. (2) A firm must be able to accurately measure the long run default risk of its portfolio even if there were no changes in the business written. (3) A firm must use a data series of appropriate length in order to establish the long run default risk. (4) A firm must be able to demonstrate the appropriateness of the scaling factor being used across a portfolio.
-
(ii)
stress testing includes a stress test covering the downturn scenarios outlined by the PRA, based on the PDs of the underlying PIT rating system, in addition to the stress test based on the parameters used in the Pillar 1 capital calculation (i.e. the portfolio level average long-run default rates);
-
(iii)
and banks are able to understand and articulate up-front how the scaling factor would vary over time in order to achieve the intended effect.
Financial Services Authority (2009b) provides some further details on the approach banks have to follow for the acceptance of their variable scalar approach.
-
(i)
- 24.
The variable scalar approach can, of course, comply with the use test in the wider sense of the CRR which explicitly allows for reasonably justified deviations from the internal use in Art. 179.
- 25.
So far, according to BCBS (2010), the primary aim of the counter-cyclical capital buffer regime is to use a buffer of capital to achieve the broader macro-prudential goal of protecting the banking sector from periods of excess aggregate credit growth that have often been associated with the build up of system-wide risk. It may also help to lean against the build-up phase of the cycle in the first place.
- 26.
According to BCBS (2011), “national authorities can implement a range of additional macro-prudential tools, including a buffer in excess of 2.5% for banks in their jurisdiction, if this is deemed appropriate in their national context. However, the international reciprocity provisions set out in this regime treat the maximum counter-cyclical buffer as 2.5%.” The BCBS provides information on its website about the domestic implementation of current and future counter-cyclical capital buffers in its member jurisdictions (see http://www.bis.org/bcbs/ccyb/index.htm).
- 27.
- 28.
Such incentives, if present, could also be mitigated by other already available sector-specific macro-prudential policy tools such as add-ons on internal risk weights. The use of alternative macro-prudential tools to address sector-specific developments is already foreseen in principle 5 of the BCBS (2010) guidance for national authorities operating the counter-cyclical capital buffer. See also https://www.esrb.europa.eu/mppa/html/index.en.html for a list of recent macro-prudential policy actions in Europe and EBA (2016) for a critical assessment of the 5% add-on to the risk weights for retail exposures secured by Belgian residential immovable property for Belgian Internal Ratings Based (IRB) banks. EBA (2016) indicates a preference for addressing the concerns about the risk weights of IRB banks via Pillar 1 and 2 (SREP). The Belgian competent authority, however, sees no justification to directly interfere in the IRB models on the basis of the CRR, as the models have no generalised problem, but simply lack data on any major property crisis in Belgium.
- 29.
In the European Union, the Regulation establishing the Single Supervisory Mechanism allocates the primary responsibility for macro-prudential policy tools to the national designated authority. The ECB can apply higher capital requirements than the national designated authority for those instruments that are included in the EU legal texts (i.e. CRD IV/CRR); see also ECB (2013).
- 30.
Initially, the Basel framework required banks to hold capital for the sum of expected and unexpected loss minus actually built provisions. Since Basel II, capital is only required to cover for unexpected loss and banks have to demonstrate that they build adequate provisions against expected loss (see BCBS 2005b).
- 31.
Tier 1 capital (going-concern capital) includes common equity (shares, retained earnings) and additional tier 1 capital; tier 2 capital (gone-concern capital) includes sub-ordinated debt, convertible securities and certain loan loss provisions. BCBS (2011) and the CRR provide the detailed definitions.
References
Actuaries Institute. (2012). Basel II retail modelling approaches. Accessed May 11, 2016, from http://actuaries.asn.au/Library/Events/Insights/2012/Presentation-BaselIIRetailModellingApproaches.pdf
Aguais, S. D., Lawrence, R. F., King, M., Lennon, M. L., & Lordkipanidze, B. (2007). Designing and implementing a Basel II compliant PIT–TTC ratings framework. In M. K. Ong (Ed.), The Basel handbook: A guide for financial practitioners (2nd ed., pp. 267–297). London: Risk Books.
Amato, J. D., & Furfine, C. D. (2004). Are credit ratings procyclical? Journal of Banking and Finance, 28(11), 2641–2677.
Barclays. (2015). Pillar 3 Report 2014. Accessed May 11, 2016, from https://www.home.barclays/content/dam/barclayspublic/docs/InvestorRelations/AnnualReports/AR2014/Barclays_Pillar_3_Disclosures_2014.pdf
Basel Committee on Banking Supervision (BCBS). (2000). Principles for the management of credit risk. Accessed May 11, 2016, from http://www.bis.org/publ/bcbs75.pdf
Basel Committee on Banking Supervision (BCBS). (2005a). Studies on the validation of internal rating systems (Working Paper No. 14). Accessed May 11, 2016, from http://www.bis.org/publ/bcbs_wp14.pdf
Basel Committee on Banking Supervision (BCBS). (2005b). An explanatory note on the Basel II IRB risk weight functions. Accessed May 11, 2016, from http://www.bis.org/bcbs/irbriskweight.pdf
Basel Committee on Banking Supervision (BCBS). (2010). Guidance for national authorities operating the counter-cyclical capital buffer. Accessed May 11, 2016, from http://www.bis.org/publ/bcbs187.pdf
Basel Committee on Banking Supervision (BCBS). (2011). Basel III: A global regulatory framework for more resilient banks and banking systems. Accessed May 11, 2016, from http://www.bis.org/publ/bcbs189.pdf
Basel Committee on Banking Supervision (BCBS). (2013). Regulatory Consistency Assessment Programme (RCAP) Analysis of risk-weighted assets for credit risk in the banking book. Accessed May 11, 2016, from http://www.bis.org/publ/bcbs256.pdf
Basel Committee on Banking Supervision (BCBS). (2014). Reducing excessive variability in banks’ regulatory capital ratios – A report to the G20. Accessed May 11, 2016, from http://www.bis.org/bcbs/publ/d298.pdf
Basel Committee on Banking Supervision (BCBS). (2015). Second consultative document: Revisions to the standardised approach. Accessed May 11, 2016, from https://www.bis.org/bcbs/publ/d347.pdf
Blochwitz, S., Hamerle, A., Hohl, S., Rauhmeier, R., & Rösch, D. (2005). Myth and reality of discriminatory power for rating systems. Wilmott Magazine, 2–6.
Catarineu-Rabell, E., Jackson, P., & Tsomocos, D. (2005). Procyclicality and the New Basel Accord – Banks choice of loan rating system. Economic Theory, 26, 537–557.
Committee of European Banking Supervisors (CEBS). (2006). Guidelines on the implementation, validation and assessment of Advanced Measurement (AMA) and Internal Ratings Based (IRB) Approaches (GL10). Accessed May 11, 2016, from https://www.eba.europa.eu/documents/10180/16094/GL10.pdf
Committee of European Banking Supervisors (CEBS). (2009). Position paper on a counter-cyclical capital buffer, 17 July 2009. Accessed May 11, 2016, from https://www.eba.europa.eu/documents/10180/16166/CEBS+position+paper+on+a+countercyclical+capital+buffer.pdf
Coppens, F., González, F., & Winkler, G. (2007). The performance of credit rating systems in the assessment of collateral used in Eurosystem monetary policy operations (ECB Occasional Paper Series 65).
Coppens, F., Mayer, M., Millischer, L., Resch, F., Sauer, S., & Schulze, K. (2016). Advances in multivariate back-testing for credit risk underestimation (ECB Working Paper No 1885).
Danielsson, J., Embrechts, P., Goodhart, C., Keating, C., Muennich, F., Renault, O., & Shin, H. S. (2001). An academic response to Basel II (Special Paper 130). Financial Markets Group, London School of Economics.
Deutsche Bundesbank. (2003, September). Approaches to the validation of internal rating systems. Monthly Bulletin, 59–71.
Drehmann, M., Borio, C., & Tsatsaronis, K. (2012). BIS Working Paper No. 380.
Engelmann, B., Hayden, E., & Tasche, D. (2003). Testing for rating accuracy. Risk, 16, 82–86.
Ernst & Young. (2014). Impairment of financial instruments under IFRS 9. Accessed May 11, 2016, from http://www.ey.com/Publication/vwLUAssets/Applying_IFRS:_Impairment_of_financial_instruments_under_IFRS_9/$FILE/Apply-FI-Dec2014.pdf
European Banking Authority (EBA). (2013a). Summary report on the comparability and pro-cyclicality of capital requirements under the Internal Ratings Based Approach in accordance with Article 502 of the Capital Requirements Regulation. Accessed May 11, 2016, from https://www.eba.europa.eu/documents/10180/15947/20131217+Summary+report+on+comparability+and+pro-cyclicality+of+the+IRB+Approach.pdf
European Banking Authority (EBA). (2013b). Report on the pro-cyclicality of capital requirements under the Internal Ratings Based Approach. Accessed May 11, 2016, from https://www.eba.europa.eu/documents/10180/15947/20131217+Report+on+the+pro-cyclicality+of+capital+requirements+under+the+IRB+Approach.pdf
European Banking Authority (EBA). (2014). Guidelines on common procedures and methodologies for the supervisory review and evaluation process (SREP). Accessed May 11, 2016, from http://www.eba.europa.eu/documents/10180/935249/EBA-GL-2014-13+%28Guidelines+on+SREP+methodologies+and+processes%29.pdf/4b842c7e-3294-4947-94cd-ad7f94405d66
European Banking Authority (EBA). (2015). Discussion paper – Future of the IRB approach. Accessed May 11, 2016, from https://www.eba.europa.eu/documents/10180/1003460/EBA-DP-2015-01+DP+on+the+future+of+IRB+approach.pdf
European Banking Authority (EBA). (2016). Opinion of the European Banking Authority on measures in accordance with Article 458 Regulation (EU) (No 575/2013). Accessed May 11, 2016, from http://www.eba.europa.eu/documents/10180/1359456/EBA-Op-2016-03+EBA+Opinion+on+measures+in+accordance+with+Art+458
European Central Bank (ECB). (2013, May). Exploring the nexus between macro-prudential policies and monetary policy measures. Special Feature A in ECB Financial Stability Review.
European Central Bank (ECB). (2015a, November). Relationship between business and financial cycles. Box 2 in ECB Financial Stability Review.
European Central Bank (ECB). (2015b, November). Quantifying the policy mix in a monetary union with national macroprudential policies. Special Feature D in ECB Financial Stability Review.
European Securities and Markets Authority (ESMA). (2015, November 17). Discussion paper: On the validation and review of Credit Rating Agencies’ methodologies (ESMA/2015/1735). Accessed May 11, 2016, from https://www.esma.europa.eu/sites/default/files/library/2015-1735_discussion_paper_on_validation_final.pdf
Financial Services Authority. (2009a, March). The Turner review: A regulatory response to the global banking crisis.
Financial Services Authority. (2009b, February). Variable scalar approaches to estimating through the cycle PDs. Accessed May 11, 2016, from http://www.fsa.gov.uk/pubs/international/variable_scalars.pdf
Financial Stability Board. (2009). Report of the financial stability forum on addressing procyclicality in the financial system. Accessed May 11, 2016, from http://www.fsb.org/wp-content/uploads/r_0904a.pdf
Goodhart, C. A. E., Sunirand, P., & Tsomocos, D. (2005). A risk assessment model for banks. Annals of Finance, 1, 197–224.
Gordy, M. B., & Howells, B. (2006). Procyclicality in Basel II: Can we treat the disease without killing the patient? Journal of Financial Intermediation, 15, 395–417.
Hamilton, D. T., Sun, Z., & Ding, M. (2011). Through-the-cycle EDF credit measures. Moody’s Analytics.
Hanson, S., Kashyap, A., & Stein, J. (2011). A macroprudential approach to financial regulation. Journal of Economic Perspectives, 25(1), 3–28.
Kashyap, A., & Stein, J. (2004). Cyclical implications of the Basel II Capital Standards. Federal Reserve Bank of Chicago Economic Perspective, 1st Quarter, 18–31.
Kiff, J., Kisser, M., & Schumacher, L. (2013). Rating through-the-cycle: What does the concept imply for rating stability and accuracy? (IMF Working Paper WP/13/64).
Le Leslé, V., & Avramova, S. (2012). Revisiting risk-weighted assets – “Why do RWAs differ across countries and what can be done about it?” (IMF Working Paper 2012/90).
Lin, L. (2012). Rating systems and procyclicality: An evaluation in a DSGE framework. Paper presented at the meeting of the American Economic Association in 2013.
Lingo, M., & Winkler, G. (2008). Discriminatory power – An obsolete validation criterion? Journal of Risk Model Validation, 4(1), 4–24.
Mayer, M., Resch, F., & Sauer, S. (2016). Point-in-time vs. through-the-cycle credit measures. mimeo.
Merton, R. C. (1974). On the pricing of corporate debt: The risk structure of interest rates. Journal of Finance, 29, 449–470.
Miu, P., & Ozdemir, B. (2005). Practical and theoretical challenges in validating Basel parameters: Key learnings from the experience of a Canadian bank. Jounal of Credit Risk, 1(4), 98–136.
Pederzoli, C., Torricelli, C., & Tsomocos, D. P. (2010). Rating systems, procyclicality and Basel II: An evaluation in a general equilibrium framework. Annals of Finance, 6, 33–49.
Prudential Regulation Authority (PRA). (2015). Internal ratings based (IRB) approaches. Supervisory Statement SS11/13. Accessed May 11, 2016, from http://www.bankofengland.co.uk/pra/Documents/publications/ss/2015/ss1113update.pdf
Prudential Regulation Authority (PRA). (2016). Residential mortgage risk weights (Consultation paper CP29/16). Accessed May 11, 2016, from http://www.bankofengland.co.uk/pra/Documents/publications/cp/2016/cp2916.pdf
Repullo, R., Saurina, J., & Trucharte, C. (2010). Mitigating the procyclicality of Basel II. Economic Policy, 64, 659–702.
Repullo, R., & Suarez, J. (2013). The procyclical effects of bank capital regulation. Review of Financial Studies, 26, 452–490.
Acknowledgements
We would like to thank José Filipe Abreu, Frank Heinemann, Christoffer Kok and Nikolas Sauter for helpful comments.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Annex: Economic Foundations of the IRB Approach
Annex: Economic Foundations of the IRB Approach
BCBS (2005b) explains that the IRB approach has been based on the assumption that financial institutions view expected losses (EL) as a cost component of doing business, and manage them by a number of means, including through the pricing of credit exposures and through provisioning (see BCBS 2005b). Expected loss as a share of the exposure at default is EL = PD × LGD.
Capital has a loss-absorbing function as it is supposed to cover the risks of unexpected losses (UL).Footnote 30 The likelihood that losses will exceed the sum of EL and UL equals the hatched area under the right hand side of the curve in Fig. 2. This threshold, i.e. the sum of EL and UL, is called the value at risk (VaR). If capital is set according to the gap between EL and VaR, and if EL is covered by provisions or revenues, then the likelihood that the bank will remain solvent over a 1-year horizon is equal to the confidence level of the VaR. The fixed supervisory confidence level under Basel II and III for minimum regulatory capital requirements is 99.9%, i.e. a bank is expected to suffer losses that exceed its level of tier 1 and tier 2 capitalFootnote 31 on average once in a 1000 years.
The relationship between expected and unexpected losses in the IRB approach (BCBS 2005b, p. 3)
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this chapter
Cite this chapter
Mayer, M., Sauer, S. (2017). Are Through-the-Cycle Credit Risk Models a Beneficial Macro-Prudential Policy Tool?. In: Heinemann, F., Klüh, U., Watzka, S. (eds) Monetary Policy, Financial Crises, and the Macroeconomy. Springer, Cham. https://doi.org/10.1007/978-3-319-56261-2_10
Download citation
DOI: https://doi.org/10.1007/978-3-319-56261-2_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-56260-5
Online ISBN: 978-3-319-56261-2
eBook Packages: Economics and FinanceEconomics and Finance (R0)