Regulatory Developments in Risk Management: Restoring Confidence in Internal Models

Abstract

The paper deals with the question of how to restore lost confidence in the results of internal models (especially market risk models). This is an important prerequisite for continuing to use these models as a basis for calculating risk-sensitive prudential capital requirements. The authors argue that restoring confidence is feasible. Contributions to this end will be made both by the reform of regulatory requirements under Basel 2.5 and the Trading Book Review and by refinements of these models by the banks themselves. By contrast, capital requirements calculated on the basis of a leverage ratio and prudential standardised approaches will not be sufficient, even from a regulatory perspective, owing to their substantial weaknesses. Specific proposals include standardising models with a view to reducing complexity and enhancing comparability, significantly improving model validation and increasing transparency as to how model results are determined, also over time. The article reflects the personal views of the authors.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Since 1997 (“Basel 1.5”), banks in Germany have been allowed to calculate their capital requirements for the trading book using internal value-at-risk (VaR) models that have passed a comprehensive and stringent supervisory vetting and approval process. Basel II and Basel III saw the introduction of further internal models complementing the standardised approaches already available—take, for example, the internal ratings-based (IRB) approach for credit risk under Basel II and the advanced credit valuation adjustment (CVA) approach for counterparty risk under Basel III. During the financial crisis, particular criticism was directed at internal market risk models, the design of which supervisors largely left to the banks themselves. This article therefore confines itself to examining these models, which are a good starting point for explaining and commenting on the current debate. Much of the following applies to other types of internal models as well.

Banks and supervisors learned many lessons from the sometimes unsatisfactory performance of VaR models in the crisis—one of the root causes of the loss of confidence by investors in model results. This led, at bank level, to a range of improvements in methodology, and also to the realisation that not all products and portfolios lend themselves to internal modelling. At supervisory level, Basel 2.5 ushered in an initial reform with rules that were much better at capturing extreme risks (tail risks) and that increased capital requirements at least threefold. Work on a fundamental trading book review (Basel 3.5), which will bring further methodological improvements to regulatory requirements, is also underway.

Nevertheless, models are still criticised as being

• too error-prone,

• suitable only for use in “fair-weather” conditions,

• too variable in their results when analysing identical risks,

• insufficiently transparent for investors and

• manipulated by banks, with the tacit acceptance of supervisors, with the aim of reducing their capital requirements.

As a result, the credibility of model results and thus their suitability for use as a basis for calculating capital requirements have been challenged. This culminated in, for example, the following statement by the academic advisory board at the German Ministry for Economic Affairs: “Behind these flaws (in risk modelling)¹ lie fundamental problems that call into question the system of model-based capital regulation as a whole.”² It therefore makes good sense to explore the suitability of possible alternatives. The authors nevertheless conclude that model-based capital charges should be retained. But extensive efforts are needed to restore confidence in model results.

2 Loss of Confidence in Internal Models—How Did It Happen?

2.1 An Example from the First Years of the Crisis

The market disruption which accompanied the start of the financial crisis in the second half of 2007 took the form in banks’ trading units of sharply falling prices with a corresponding impact on their daily P&Ls after a prolonged phase of low volatility. Uncertainty grew rapidly about the accuracy of estimated probabilities of default, default correlations of the underlying loans and the scale of loss in the event of default, and thus also about the probabilities of default and recovery rates of the securitisation instruments. This in turn caused spreads to widen, volatility to increase and market liquidity for securitisation products to dry up. A major exacerbating factor was that many market participants responded in the same way (“flight to simplicity”, “flight to quality”). Later on, there were also jump events such as downgrades. Calibrating the above parameters proved especially problematic since there was often a lack of historical default or market data. Unlike in the period before the crisis, even AAA-rated senior or super senior tranches of securitisation instruments, which only start to absorb loss much later than their riskier counterparts, suffered considerably in value as the protective cushion of more junior tranches melted away, necessitating substantial write-downs.³

The performance of internal market-risk models was not always satisfactory, especially in the second half of 2007 and in the “Lehman year” of 2008. In this period, a number of banks found that the daily loss limits forecast by their models were sometimes significantly exceeded (backtesting outliers).⁴ The performance results of Deutsche Bank, for instance, show that losses on some sub-portfolios were evidently serious enough to have an impact on the overall performance of the bank’s trading unit. This demonstrates the extremely strong market disruption which can follow an external shock. When backtesting a model’s performance, the current clean P&L—P&L$_t$—is compared with the previous day’s VaR forecast VaR$_{t - 1}$.⁵ At a confidence level of 99 %, an average of two to three outliers a year may be anticipated over the long term (representing 1 % of 250–260 trading days a year). In the years between 2007 and 2013, Deutsche Bank had 12, 35, 1, 2, 3, 2 and 2 outliers.⁶ Although the models’ performance for 2007 and 2008 looks bad at first sight, the question nevertheless arises as to whether or not these outliers are really the models’ “fault”, so to speak. By their very nature, models can only do what they have been designed to do: “If you’re in trouble, don’t blame your model.” To function properly, the models needed liquid markets, adequate historical market data and total coverage of all market risks, particularly migration and default risk. These prerequisites were not always met by markets and banks. Anyone using a model has to be aware of its limitations and exercise caution when working with its results.

Even Germany’s Federal Financial Supervisory Authority BaFin pointed out that, given the extreme combination of circumstances on the market in connection with the financial crisis, the figures do not automatically lead to the conclusion that the predicative quality of the models is inadequate.⁷ The example could indicate that, since 2009, the bank has been successful in eliminating its models weaknesses, at least at the highest portfolio level. It should nevertheless be borne in mind that market phases analysed after 2008 were sometimes quieter and that there has also been some reduction in risk. The increasing shift in the nature of the financial crisis from 2010 towards a crisis concerning the creditworthiness of peripheral European countries, which created new market disruption, is most certainly reflected at the highest level of the backtesting time series. Particularly large losses were incurred in March and May 2010, which only in May 2010 led to the two outliers realised that year. These outliers may be explained by the fears brewing at the time about the situation of the PIIGS states. Possibly, the scale of the corresponding trading activities was such that any problems with the models for these sub-portfolios made themselves felt at the highest portfolio level. The weaknesses outlined below were, by the banks own testimony, identified and rapidly addressed.⁸ As mentioned above, two to three outliers per year represent the number to be expected and are not sufficient, in themselves, to call the quality of modelling into question.

The flaws banks identified in their models following the outbreak of the crisis revealed that a variety of areas needed work and improvement. These improvements have since been carried out. Some examples of model weaknesses which banks have now resolved are⁹:

• 1. No coverage of default-risk driven “jump events”, such as rating changes and issuer defaults. At the outbreak of the crisis, models often failed to cover the growing amount of default risk in the trading book. The introduction of IRC models¹⁰ to cover migration and default risk helped to overcome this.

• 2. Insufficient coverage of market liquidity risk. It was often not possible to liquidate or hedge positions within the ten-day holding period assumed under Basel 1.5. This led to risks being underestimated. Basel 2.5 takes account of market liquidity risk explicitly and in a differentiated way, at least for IRC models. Full coverage will be achieved under Basel 3.5.

• 3. Slow response to external shocks (outlier clustering). The introduction of stress VaR under Basel 2.5 went a long way towards eliminating the problem of underestimating risks in benign market conditions. Historical market data for “normal VaR” are now adjusted daily, while monthly or quarterly adjustments were the norm before the crisis.

• 4. Insufficient consideration of the risk factors involved in securitisation. As a result, models designed for securitisation portfolios may no longer be used to calculate capital charges (with the exception of the correlation trading portfolio). Even before the rule change, some banks had already decided themselves to stop using these models.

• 5. Flawed proxy approaches. Prior to the crisis, it was often possible to assign a newly introduced product to an existing one and assume the market risk would behave in the same way. During the crisis, this assumption proved to be flawed.¹¹ The supervisory treatment of such approaches is now much more restrictive.

• 6. The approximation of changes in the price of financial instruments cannot accommodate large price movements (delta-gamma approximations). Full revaluation of instruments is now standard practice.

• 7. No and/or flawed scaling to longer time horizons. Scaling practices of this kind, such as square-root-of-time scaling, are now subject to prudential requirements to ensure their suitability.

These problems were the basis of the review of market risk rules under Basel 2.5 and, as described above, were able to be eliminated both by banks themselves and by new supervisory requirements.¹² Despite this large-scale and appropriate response, distrust of internal model results and their use for prudential purposes persisted, leading to further fundamental discussions.¹³

2.2 Divergence of Model Results

This continuing distrust at the most senior level of the Basel Committee¹⁴ led to the commissioning of the Standards Implementation Group for Market Risk (SIG-TB) to compare the results generated by the internal models of various banks when applied to the same hypothetical trading portfolios (hypothetical portfolio exercise). A major point of criticism has always been that internal model results are too variable even if the risks involved are the same. In January 2013, the SIG-TB published its analysis.¹⁵ The following factors were identified as the key drivers of variation:

• The legal framework: some of the banks in the sample did not have to apply Basel 2.5. This means the US banks, for instance, supplied data from models that had neither been implemented nor approved. Analysis showed that some of these banks had significantly overestimated risk, though this did not, in practice, translate into higher capital requirements.

• National supervisory rules for calculating capital requirements: differences were noted, for example, in the multipliers set by supervisors for converting model results into capital requirements. In addition, some supervisors had already imposed restrictions on the type of model that could be used and/or set specific capital add-ons.

• Legitimate modelling decisions taken by the banks: among the most important of these was the choice of model (spread-based, transition matrix-based) in the absence of a market standard for modelling rating migration and default risk (IRC models). Different assumptions about default correlations also led to different results. In VaR and stressed VaR models, major factors were the length of data histories (at least one year, no maximum limit), the weighting system, the aggregation of asset classes and of general and specific market risk, and the decision whether to scale a one-day VaR up to ten days or estimate a 10-day VaR directly. The choice of stress period for the stressed VaR also played an important role.¹⁶

In summary, the differences noted were the result of legitimate decisions taken by banks with the approval of supervisors and of variations between supervisory approval procedures. There is no evidence to suggest manipulation with the aim of reducing capital requirements. Differences can also be explained by variations in the applicable legal framework and in the market phase on which the study was based. An issue related to the market phase is the length of the observation period used. Observation periods of differing lengths will have an impact if, for instance, the volatility of market data has changed from high (during a period over one year ago) to low (last year). In this example, a bank using a one-year data history will not capture the phase of higher volatility. This volatility will, by contrast, most certainly be captured by any bank using a longer data history (with the extent also depending on the weighting system applied to historical data).

It is also important to note that the study was based on a hypothetical portfolio approach at the lowest portfolio level and not on real portfolios. The study does not address the inherent weakness of this method. One major weakness is that the test portfolios used do not reflect portfolio structures in the real world. Portfolios for which banks calculate VaR are normally located at a far higher level in the portfolio “tree” and are consequently more diversified. If the portfolios analysed had been more realistic, variations would probably have been significantly less marked.¹⁷

Even if the variation between results can be readily explained and cannot be “blamed” on the banks, it is nevertheless difficult to communicate differences of, for instance, around 13–29 million euros in the results for portfolio 25, the most highly aggregated portfolio.¹⁸ Efforts are most certainly needed to reduce the amount of variation by means of further standardisation, even if complete alignment would not make good sense (see Sect. 4.2). At first sight, the differences could also be interpreted as a quantitative measure of the uncertainty surrounding model results and thus as an expression of model risk. Section 4.7 will explore to what extent this is a reasonable analysis and whether banks should try to capitalise model risk themselves as things stand. As the next section shows, dispensing with internal models for prudential purposes would not, by contrast, be the correct response.

3 Alternatives to Internal Models

3.1 Overview

Given the difficulties associated with modelling and the variation in results, it is legitimate to ask whether model-based, risk-sensitive capital charges should be dropped altogether. Such a step would, moreover, significantly simplify regulation. But it could also be asked whether it would not make more sense to address the undoubted weaknesses of internal models by means of the reforms already in place or in the pipeline without “throwing the baby out with the bath water”, i.e. should we not try to learn from past mistakes instead of just giving up. These questions can best be answered systematically by examining to what extent the existing regulatory proposals could, together or on their own, replace model-based capital charges. There are essentially two alternatives under discussion:

• dropping risk-sensitive capital charges and introducing a leverage ratio as the sole “risk metric”;

• regulatory standardised approaches: applying risk-sensitive capital charges while abandoning model-based ones.

3.2 The Leverage Ratio

An exclusively applicable, binding leverage ratio—defined as the ratio of tier 1 capital to total assets including off-balance-sheet and derivative positions¹⁹—is only a logical response if it must be assumed that neither banks nor supervisors are capable of measuring the risks involved in banking. Advocates of this approach talk of the “illusion of the measurability of risk.”²⁰ They argue that we are in a situation of “uncertainty”, not “risk”. Uncertainty in decision theory is characterised by two things: neither are all conceivable results known, nor is it possible to assign probabilities to the results or estimate a probability density function. In this case, it would not, for example, be possible to calculate a VaR defined as a quantile of a portfolio loss distribution. This is only possible under “risk”.

The concepts of “uncertainty” and “risk” are, however, abstract, theoretical extremes, while the various situations observed in reality usually lie somewhere in between. The answer to the question of whether it is more appropriate to assume a risk situation or an uncertainty situation is determined above all by the availability of the data needed for the model estimate (such as market data or historical default data). If, in addition, the risk factors associated with the financial instruments are known and taken into account, and if the potential changes in the value of a trading portfolio can be satisfactorily measured, (quality of the stochastic model, no normal loss distribution as a rule), determining a VaR of portfolio losses is likely to be appropriate.²¹ This may be assumed for the vast majority of trading portfolios. Should this nevertheless not be the case, regulatory standardised approaches, which normally require less data to be available, could then be used. Reviewing and adjusting models is a never-ending task for banks. The model risks which undoubtedly exist (e.g. estimation errors) are also a focus of supervisors’ attention. An awareness of the limits of a model and of such model risks does not, however, make the use of models obsolete.²² Although modelling by its very nature always involve simplification of reality, quantitative and qualitative model validation is crucial. Supervisors set and enforce stringent rules for such validation.²³

Advocates of the “uncertainty approach” propose a so-called heuristic as a “rule of thumb” and as a risk metric, at least for supervisors. Leverage ratios with widely differing minimum levels have been suggested as a heuristic for ensuring the solvency of banks. The levels called for range from 3 to 30 %.²⁴ As is generally recognised, it is not possible to infer a specific minimum level from theory.

The question of whether a leverage ratio is actually a suitable heuristic for ensuring solvency has not been satisfactorily answered, however. Empirical studies to determine to what extent the leverage ratio is a statistical, univariate risk factor that can distinguish between banks that survive and those that fail come to different conclusions.²⁵ Often, no such distinguishing ability can be demonstrated. This may have an economic explanation since the leverage ratio, as a vertical metric on the liabilities side of the balance sheet, cannot act as a horizontal metric of a bank’s risk-bearing capacity by means of which sources of loss (causes of insolvency), which are mainly to be found on the assets side of the bank’s balance sheet, are compared with a loss-absorbing indicator (capital). This can, by contrast, be accomplished by ratios such as the “core tier 1” or “tier 1” capital ratio. If, moreover, a leverage ratio were a measure capable of predicting the insolvency of certain types of banks, it would probably swiftly cease to be a good measure once it became a binding target (Goodhart’s Law).

What is more, the leverage ratio has a very long—and already widely discussed—list of drawbacks.²⁶ These are the points of most relevance here:

• Perverse incentives and the potential for arbitrage: there are strong incentives to make business models more risky. Because assets are measured on a non-risk-weighted basis, an AAA investment, for instance, ties up just as much capital as does a B investment.

• A leverage ratio is by no means “model free”: highly complex valuation models or even simulation approaches are sometimes needed to measure derivatives on a marked-to-market basis, for example. In a broader context, this is more or less true for all balance-sheet valuations. So even a leverage ratio cannot claim to be the simple, robust rule that proponents of a heuristic approach are looking for.²⁷

• It makes it impossible to compare capital adequacy across banks. The adequacy of a bank’s capital resources cannot be assessed without measuring the associated risks.

For these and other reasons not mentioned here, the international banking community continues to reject the leverage ratio as a sole indicator and as a binding limit. At most, it may make sense to monitor changes in a bank’s leverage ratio, but not its absolute level; this is the approach of the German Banking Act at present.²⁸ Supervisors have widely differing views on the leverage ratio. Even Haldane/Madouros (Bank of England) by no means call in their famous “The dog and the frisbee” speech for a leverage ratio on its own or a minimum leverage ratio set at such a high level that risk-based requirements are overridden and therefore indirectly rendered obsolete (leverage ratio as a frontstop instead of the Basel backstop). Owing to the massive perverse incentives which they too have noted, they talk instead of placing leverage ratios on an equal footing with capital ratios.²⁹

3.3 Regulatory Standardised Approaches

Standardised approaches, i.e. approaches which spell out in detail how to calculate capital requirements on the basis of prudential algorithms (“supervisory models”), will always be needed for smaller banks which cannot or do not wish to opt for internal models. But larger banks need standardised approaches too—as a fallback solution if their internal models are or become unsuitable for all or for certain portfolios. Having said that, a standardised approach alone is by no means sufficient for larger banks; the reasons are as follows³⁰:

• It is invariably true of a standardised approach that “one size does not fit all banks”. Since a standardised approach is not tailored to an individual bank’s portfolio structure, it cannot measure certain risks (such as certain basis risks) or can only do so very inaccurately. It is normally much less risk-sensitive than an internal model.

• A related problem is that a standardised approach usually works only with comparatively simple portfolios. This results in risk being overstated or understated.

• It normally fails to capture diversification or hedging effects satisfactorily.

• Standardised approaches can thus be more dangerous than internal models because it is often easy to “game the system”. Trading revenue, for instance, can be generated seemingly without risk, enabling trading units to inflate risk-adjusted earnings.³¹

• If internal models are no longer used, supervisors will also have to dispense with banks’ risk-management expertise.

• Standardised approaches are simple models. But as all proposals for standardised approaches to date have shown, supervisors are by no means better at constructing models than are the banks themselves.

A further alternative would be scenario-based approaches, which are often relatively similar to models, such as those which may currently be used for calculating capital charges for options under the standardised approach to market risk (scenario matrix approach). This alternative, though definitely worth considering, is not being discussed at present and will therefore be only briefly explored in this article. Scenario approaches may be regarded as a kind of “halfway house” between risk-sensitive standardised approaches and internal models. If they are prescribed as a regulatory standardised approach, they may also demonstrate the weaknesses of standardised approaches described above. The key criteria for evaluating such approaches are the scenario generation technique and the process/algorithm used for calculating valuation adjustments on the basis of the scenarios. An especially critical question is to what extent the (tail) loss risk of the instruments and portfolios concerned can be captured. At one end of the spectrum are approaches that merely differentiate between a few scenarios (e.g. base case and adverse case) and make no attempt to estimated a loss distribution. At the other extreme are internal models which simulate such a large number of scenarios that it is possible to estimate a loss distribution on the basis of which a parameter such as VaR or expected shortfall can be calculated. Another important question is whether or not the scenario generation takes account of stressed environmental conditions.

To sum up, standardised approaches usually have considerable failings when it comes to measuring risk, especially the risk associated with large-scale, complex trading activities. On their own, they are not an adequate basis on which to determine appropriate capital requirements.³² So it may be concluded at this point that, together or separately, a leverage ratio and standardised approaches are inappropriate and insufficient from a supervisory perspective. Internal models must remain the first choice. Nevertheless, confidence in internal models needs to be significantly strengthened.

4 Ways of Restoring Confidence

4.1 Overview

The first, important step should be to standardise supervisory approval processes to eliminate this major source of variation. A single set of approval and review standards should be developed for application worldwide. A globally consistent procedure needs to be enforced for granting and withdrawing permission to use models. With activities of this kind, supervisors themselves could make a significant contribution to restoring confidence.³³

A number of further proposals are also under discussion at present. Together, they have the potential to go a long way towards winning back trust:

• a. Reducing the variation in model results through standardisation (Sect. 4.2).

• b. Enhancing transparency (Sect. 4.3).

• c. Highlighting the positive developments as a result of the trading book review (Sect. 4.4).

• d. Strengthening the use test concept (Sect. 4.5).

• e. A comprehensive approach to model validation (Sect. 4.6).

• f. Quantification and capitalisation of model risk (Sect. 4.7).

• g. Voluntary commitment by banks to a code of “model ethics” (Sect. 4.8).

• h. Other approaches (Sect. 4.9).

4.2 Reducing the Variation in Model Results Through Standardisation

First of all, however, it is important to be aware of the dangers of excessive standardisation³⁴:

• Standardised models can pose a threat to financial stability because they encourage all banks to react in the same way (herd behaviour). Model diversity is a desirable phenomenon from a prudential point of view since it generates less procyclicality.

• Standardised models would frequently be unsuitable for internal use at larger banks, which would consequently need to develop alternative models for internal risk management purposes. As a result, the regulatory model would be maintained purely for prudential purposes (in violation of the use test; see below). This would encourage strategies aimed at reducing capital requirements since the results of this model would not have to, and could not, be used internally.

• It is therefore in the nature of models that a certain amount of variation will inevitably exist.

Nonetheless, it is most certainly possible to standardise models in a way which will reduce their complexity and improve the comparability of their results but will not compromise their suitability for internal use. Here are a few suggestions³⁵:

• Develop a market standard for IRC models to avoid variation as a result of differences in the choice of model (proposed standard established by supervisors: see Trading Book Review).

• Reduce the amount of flexibility in how historical data are used. For the standard VaR, one year should be not just the minimum but both the minimum and maximum period. This may well affect different banks in different ways, sometimes increasing capital requirements and sometimes reducing them.

• Standardise the stress period for stressed VaR. The period should be set by supervisors instead of being selected by banks. True, this means the stress period would no longer be optimally suited to the individual portfolio in question. But as the study by the Basel Committee’s SIB-TB has shown, similar periods may, as a result of the financial crisis, be considered relevant at the highest portfolio level—namely the second half of 2008 (including Lehman insolvency) and the first half of 2009.³⁶

4.3 Enhancing Transparency

Much could also be done to improve transparency. Banks could disclose their modelling methodologies in greater detail, and explain—for example—why changes made to their models have resulted in reduced capital charges. Transparency of this kind will significantly benefit informed experts and analysts. These experts will then be faced with the difficult challenge of preparing their analyses in such a way as to be accessible to the general public. The public at large cannot be expected to be the primary addressees of a bank’s disclosures. Someone without specialist knowledge is unlikely to be able to understand a risk report, for instance. Nor is it the task of banks to write their reports in a manner that makes such specialist knowledge unnecessary. This is, however, by no means an argument against improving transparency.

The work of the Enhanced Disclosure Task Force (EDTF) is also a welcome contribution³⁷ and some banks have already implemented its recommendations in their trading units voluntarily. The slide from the Deutsche Bank’s presentation for analysts on 31 January 2013 is just one illustration.³⁸ This explains, in particular, the changes in market-risk-related RWAs (mRWA flow), i.e. it is made clear what brought about the reduction in capital requirements in the trading area. The reasons include reduced multipliers (for converting model results into capital requirements) on the back of significantly better review results, approval of models (IRB approach, IMM) for some additional products and the consideration of additional netting agreements and collateral in calculations of capital requirements.

Another possible means of improving transparency would be to disclose the history of individual positions with a certain time lag. Serious discussion is nevertheless called for to determine at what point the additional cost of transparency incurred by banks would exceed the additional benefit for stakeholders. From an economic perspective, this may be regarded as a transparency ceiling.

4.4 Highlighting the Positive Developments as a Result of the Trading Book Review

The Basel Committee is currently working on a fundamental review of how capital requirements should be calculated for trading book exposures.³⁹ It has taken criticism of the existing regime on board and proposes to reduce the leeway granted to banks in the design of their internal models. Without going into the Committee’s extensive analysis in detail, here are some key elements of relevance to the questions examined in this article:

• Expected shortfall is to be introduced as a new risk metric calibrated to a period of market stress. The intention is to switch to a coherent measure of risk which can take better account of tail risk.⁴⁰ The reference to a stress period is intended to address the issue of “fair-weather models” (the problem facing the turkey in Taleb’s “The Black Swan”).

• A so-called desk approach is to be introduced for granting and withdrawing approval for models. In the future, model approval is to be decided on a case-by-case basis at trading desk level. This will enable portfolios which are illiquid and/or cannot easily be modelled to be excluded from the model’s scope.

• Model validation will take place at desk level and become even more stringent through backtesting and a new P&L attribution process. This will significantly improve the validation process. At the same time, it will have the effect of raising the barriers to obtaining supervisory approval of internal models.

• All banks using models will also have to calculate requirements using the standardised approach. Supervisors take the view that the standardised approach can serve as a floor, or even a benchmark, for internal models (the level of the floor has not yet been announced). This may provide a further safety mechanism to avoid underestimating risk, even if the standardised approach does not always produce sound results (see above).

4.5 Strengthening the Use Test Concept

Up to now, approval of internal models has been dependent, among other things, on supervisors being convinced that the model is really used for internal risk management purposes. Banks consequently have to demonstrate that the model they have submitted for supervisory approval is their main internal risk management tool. Basically, they have to prove that the internal model used to manage risk is largely identical to the model used to calculate capital charges (use test). The rationale behind this sensible supervisory requirement is that the quality of these risk measurement systems can best be ensured over time if the internal use of the model results is an absolute prerequisite of supervisory approval. As a result of the use test, the bank’s own interests are linked to the quality of the model. The design of the model should on no account be driven purely by prudential requirements. Moreover, the reply to the question of how model results are used for internal risk management purposes shows what shape the bank’s “risk culture” is in.

The use test concept has been undermined, however, by a development towards more prudentially driven models which began under Basel 2.5 and is even more pronounced under Basel 3.5. This trend should be reversed. At a minimum, the core of the model should be usable internally—that is to say be consistent with the bank’s strategies for measuring risk. Conservative adjustments can then be made outside the core.

4.6 A Comprehensive Approach to Model Validation

It should be borne in mind that conventional backtesting methods cannot be performed on IRC models. Instead, the EBA has issued special guidelines based on indirect methods such as stress tests, sensitivity and scenario analyses.⁴¹ A distinction therefore needs to be made between “normal” market risk models and IRC models. Though validation standards already exist for IRC models, they can by no means be described as comprehensive.

For normal market risk models, a comprehensive approach going beyond purely quantitative backtesting and the P&L attribution process could be supported by banks themselves. Proposals to this effect are already on the table at the Federal Financial Supervisory Authority (BaFin).⁴² It would be worth examining whether the minimum requirements for the IRB approach could make an additional contribution. These minimum requirements already pursue a comprehensive quantitative and qualitative approach to validation, though it may not be possible to apply a number of problems needing to be resolved to the area of market risk.⁴³

4.7 Quantification and Capitalisation of Model Risk

A further approach might be to quantify and capitalise model risk either in the form of a capital surcharge on model results under pillar 1 or as an additional risk category under pillar 2.

It would be worthwhile discussing the idea of using the diverging result interval of the hypothetical portfolio exercise (see Sect. 2.2) as a quantitative basis for individual capital surcharges. This may be regarded as prudential benchmarking.⁴⁴ The portfolios tested in this exercise do not, however, correspond to banks’ real individual portfolios, which makes them a questionable basis for individual capital surcharges. As explained above in Sect. 2.2, moreover, it cannot be concluded that the differences are largely due to model weaknesses. The question of how to derive the differences actually due to model risk from the observed “gross” differences is yet to be clarified and will probably be fraught with difficulties. What is more, model risk is not reflected solely in the differences in model results (see below on the nature of model risk, which also covers the inappropriate use of model results, for example, which can result in flawed management decisions).

This raises the question as to whether it may be better to address model risk under pillar 2. If model risk is assumed to arise, first, when statistical models are not used properly and, second, from an inevitable uncertainty surrounding key features of models, then it is likely to be encountered above all in the areas of

• design (model assumptions concerning the distribution of market risk parameters or portfolio losses, for example),

• implementation (e.g. the approximation assumptions necessary for IT purposes),

• internal processes (e.g. complete and accurate coverage of positions, capture of market data, valuation models at instrument level [see below]) and IT systems used by banks to estimate risk, and

• model use.⁴⁵

The authors take the view that solving the question of how to quantify model risk for the purpose of calculating capital charges is a process very much in its infancy and that it is consequently too soon for regulatory action in this field. As in other areas, risk-sensitive capital requirements should be sought; one-size-fits-all approaches, like that called for by the Liikanen Group, should not be pursued because they usually end up setting perverse incentives.

This point notwithstanding, there are already rigid capital requirements for trading activities under pillar 1 which address model risk, namely in the area of prudent valuation. These require valuation adjustments to be calculated on accounting measurements of fair value instruments (additional valuation adjustments, AVAs) and deducted from CET1 capital. This creates a capital buffer to cover model risk associated with valuation models at instrument level (see above).⁴⁶ Valuation risk arising from the existence of competing valuation models and from model calibration is addressed by the EBA standard. Deductions for market price uncertainty (Article 8 of the EBA RTS) can also be interpreted as charges for model risk, even if the EBA does not itself use the term.

4.8 Voluntary Commitment by Banks to a Code of “Model Ethics”

A commitment could be made to refrain from aggressive or inappropriate modelling with the sole aim of minimising capital requirements. Banks voluntarily exclude portfolios, such as certain (though by no means all) securitisation portfolios, from the scope of their model if questionable results tend to be generated. This may be regarded as a subitem of the modelling validation issue. The desk approach under Basel 3.5 will help to put this new culture into practice. Since capital requirements will have to be calculated using the standardised approach as well as the IMA, any aggressive modelling should be exposed. At a minimum, banks will have to demonstrate that the standardised approach overstates risk in the portfolio in question. If this cannot be demonstrated, a case of excessively aggressive modelling may be assumed.

4.9 Other Approaches

Other approaches to restoring confidence also deserve a brief mention:

• further incentives to use models appropriately

• opening up of access to trade repository data

• review of models by auditors

• more stringent new product introduction (NPI) processes.

In addition to the code of “moral ethics” discussed in Sect. 4.8, the following additional incentive to use models appropriately could be considered. Establishing a link between traders’ bonuses and model backtesting results could serve to improve the alignment of interests. This idea is also closely connected with the issue of strengthening the use test concept (see Sect. 4.5).

Trade repositories already collect key data, including calculated market values, relating to all derivative contracts, irrespective of whether they are centrally cleared or not. As things stand, banks have no way of accessing the data of other banks. If access were made possible at an anonymised level, for example, banks would be able to carry out internal benchmarking, which could reduce valuation uncertainty and thus model risk (see also Sect. 4.7).

External auditors already review banks’ internal models (both instrument and stochastic) when auditing the annual accounts. Ways could be explored of further improving or extending this process, e.g. to include a review of use test compliance.

In the insurance industry, the chief actuary is personally responsible for the correct pricing of new products. This practice could be adopted in the NPI process used in the banking industry. The CRO would then be responsible for pricing products fairly, including products aimed at retail clients. The NPI process could also be made stricter by requiring external reviewers to approve major new products. Finally, the suitability of proxy approaches, which are extremely important in the NPI process, could be examined more stringently and in greater depth.

5 Conclusion

The key conclusions of this article can be summarised as follows:

• A risk-sensitive and model-based approach to calculating capital requirements for banks should be retained.

• Not only should model-based approaches be formally retained, but there should also continue to be a capital incentive to use these approaches (i.e. no overriding leverage ratio, no floor set at too high a level).

• Non-risk-sensitive approaches to calculating capital requirements should, at most, be used in a complementary capacity, serving merely as indicators and not as binding limits. Otherwise, dangerous perverse incentives will arise.

• There are also dangers associated with risk-sensitive standardised approaches because these typically overestimate or underestimate the actual risk.

• Variation in the area of models is something that needs to be lived with to a certain extent. Some standardisation is nevertheless possible, as are other ways of restoring confidence. But it should not compromise the internal usability of models.