Learning from Money Laundering National Risk Assessments: The Case of Italy and Switzerland

Abstract

The FATF requires each country to undertake a national risk assessment (NRA) to show the government’s knowledge of money laundering risks. There is little guidance as to how these NRAs are to be conducted, and those that have been published show great variation in terms of data used, analytical methods, and the depth of policy analysis. After expounding some of the concepts basic to any risk analysis, we analyze two of the more detailed published NRAs, from Italy and Switzerland. The Italian NRA, focused on domestic criminal threats, relies almost exclusively on expert opinion. Its most distinctive product is an analysis of the high-threat sectors and the need for specific kinds of policy interventions. The Swiss NRA, focused primarily on threats from other countries, presents far more quantitative data, almost exclusively from suspicious activity reports, to supplement expert opinion. Though both NRAs provide useful insights about money laundering risks, neither is conceptually clear; in particular, neither reflects contemporary practice in the use of expert opinion. Our critique is aimed at helping strengthen the next round of NRAs, and identifies lessons learned for all countries. Our recommendations include the use of risk assessment standards from other fields, the addition of a measure of uncertainty, and a more critical assessment by FATF in its NRA evaluations.

Introduction

A central question for money laundering research and policy is to what extent an existing anti-money laundering (AML) regime is effective and efficient. A few studies have examined this (e.g. Masciandaro 1999; Reuter and Truman 2004; Unger et al. 2014), but all have shied away from reaching a general judgment on the question, merely noting factors that would need to be considered.

Performing a national risk assessment (NRA) for money laundering is—or at least should be—focused on answering a narrow version of that question. Implicitly, an NRA is an effort, within the context of the existing set of laws, to determine whether AML resources (including the stringency of monitoring, the severity of sanctions, and the intensity of investigation) are sufficient and whether they could be better allocated across different sectors of the financial system to reduce overall money laundering in a country. It should also examine whether the AML system covers all methods by which money might be laundered. It does not ask whether the laws are optimal.

NRAs are now a requirement for member countries of the Financial Action Task Force (FATF) as part of the fourth round of mutual evaluation reports (MERs). The NRA is intended to demonstrate whether the government has an adequate understanding of the risks associated with money laundering within the various sectors of the economy.¹ The NRAs are often elaborate exercises, involving many different stakeholders and conducted over a period of many months.² They receive extensive reference in the early fourth-round MERs; for example, the Canadian 2016 MER refers to the NRA 10 times just in the executive summary. Thus NRAs appear to be important documents for national governments. Yet the FATF (2013) methodology document, intended to provide guidance to both the countries and the MER assessors, contains little specific advice about how an NRA is to be conducted, leaving a great deal of discretion to the individual countries. It is a guidance document rather than a manual, such as the quite detailed directions that it now offers for MERs. This may reflect the fact that the MERs are now in their fourth round, whereas the NRA process is just beginning.

In practice, NRAs often go beyond the FATF requirements of assessing relative risks and providing recommendations as to how policies and resources should be adapted to increase effectiveness. Most published NRAs claim to show that the AML system is successful in controlling money laundering.³ For example, the NRA of Singapore, a major financial center, states that despite high inherent risks as a destination for international capital, “Singapore has in place a robust AML/CFT [Combating the Financing of Terrorism] regime, grounded in tough regulations, rigorous supervision and effective enforcement that has helped mitigate these risks. There are a few areas where controls need to be strengthened and efforts are underway to address these areas” (p. 5). Some go even further. The United States claims that “AML regulation, supervision, enforcement, and compliance in the United States are generally successful in minimizing money laundering risks” (p. 85), whatever that means.

Thus NRAs are more than assessments of relative risks across sectors. They purport to be evaluations of how well the regime controls particular forms of money laundering. We examine whether they do in fact provide credible support for their conclusions. An NRA could fail in any of three ways:

• Conceptual. Are risks defined appropriately?

• Empirically. Are the data used the best available? Are the data adequately described?

• Analytically: Are the data used appropriately so as to measure the defined risks?

This paper, the first in a program of research, examines two national risk assessments, those of Italy (2014a, b) and Switzerland (2015). These countries are of interest for at least three reasons:

• 1. They were the most explicit about the methodology employed at the time of writing.⁴ Italy is the only country to publish a separate methodology report (32 pages). Switzerland has an appendix presenting a formula for calculating risk for each sector and offers an explanation for its choice of factors to account for that risk variation. Thus one can see more clearly how these two countries reached their assessments. They might reasonably be called “state-of-the-art” NRAs, even though they were relatively early efforts.

• 2. The two countries differ sharply in the nature of the ML threat faced. Though it has exported some of its criminal entrepreneurs and enterprises, Italy’s problem is primarily domestic, and the country has an unusual (perhaps unique) focus on organized crime (Levi 2016). In contrast, Switzerland is primarily concerned with foreign crimes, often involving corruption of senior officials in other countries or fraud in other countries. Thus the two countries present very different challenges for conducting an NRA.

• 3. Both provide relatively fine-grained analyses in terms of the sectors covered and the nature of the policy levers examined.

We conclude that each has problems in the first and third categories (i.e. conceptual and analytical) that give rise to potential failure. While both NRAs use the best available data, they fail to interpret them correctly. For instance, the Swiss NRA emphasizes the value of its suspicious activity report (SAR) data because banks are required to conduct an extensive investigation of a transaction or account before reporting it as suspicious,⁵ whereas many other countries (including Italy) require banks to file a SAR on “first impression”, so to speak. However, that does not make the Swiss SARs better measures of the extent of money laundering in each sector; it reduces the number of false positives (transactions incorrectly labeled as laundering), but (presumably) increases the number of false negatives (the number incorrectly classified as legitimate). Which is better for the specific analytical purposes cannot be established a priori.⁶

Our critique is aimed at helping strengthen the next round of money laundering NRAs. We argue that there are ways of using the same resources and data to generate risk assessments that will better inform AML decisions, with better substantiation. These ideas are just briefly mentioned here and will be developed in subsequent papers.

We begin with a brief discussion of the concepts that underlie a money laundering risk assessment. In particular, we try to provide a clearer understanding of the relationship between threat and vulnerability, on the one hand, and risk on the other. Sections III (Italy) and IV (Switzerland) present our analyses of the individual NRAs. In each we briefly describe the context of money laundering, how the NRA was conducted, how the risk concepts were operationalized, the dimensions of risk examined, the data utilized, and the method for analyzing the data. Section V summarizes and contrasts the findings from the two individual analyses, focusing on whether these NRAs in fact provide meaningful evaluations of how well the AML system is working or how policies should change. The Section concludes with suggestions for how to improve NRAs.

Conceptual Framework

The 2013 FATF methodology for risk assessment refers to threats and vulnerabilities. Threat is the external forces (such as drug trafficking) and groups (e.g. drug trafficking organizations) that might lead to money laundering. Vulnerability refers to those characteristics of a sector or country that make it attractive for money laundering: weaknesses in the prevention, detection, and/or enforcement of policies against money laundering events. While these labels have intuitive appeal as a means of structuring the NRA exercise, they are in fact confusing, as we explain in more detail at the end of this section. The principal conceptual weaknesses of the Italian and Swiss NRAs can be traced to FATF’s articulation, though this in turn reflected the lack of clarity in the academic AML literature.

Consider risk assessment applied to an individual sector, such as banking or life insurance. One intuitive measure of threat is the likelihood that a euro entering the sector is the fruit of a predicate crime; we’ll denote this as p_i(1), and it is measured by laundered euros entering sector i divided by the total number of euros entering sector i, i.e. how contaminated the sector is.⁷ That is a reasonable measure of the threat from the perspective of the sector’s regulator. Vulnerability may be given a specific measure, namely the probability that a laundered euro is not detected by the AML system: p_i(2). This is the number of undetected laundered euros divided by the total number of laundered euros that enter the sector; the latter of course cannot be observed and at best can be estimated. Since AML regulation (prevention, detection, and sanctioning) is expensive, the regulator and regulated want to calibrate the vulnerability to the threat level. If p_i(1) is small, then there is no need to spend large sums in order to keep p_i(2) low. We note that these are theoretical probabilities. Finding proxies to estimate them is a major challenge, but it is important to keep them in mind in assessing the relevance of the calculations that are used.

This theoretical approach, on the surface, appears consistent with the 2013 FATF methodology, but in fact the threat as articulated in that document is not measured on a sectoral basis. FATF conceives of it as an aggregate measure for each crime or offender type. How much of the total revenue that might be laundered is derived from specific crimes such as foreign corruption or from domestic cybercrime? Since, as reflected in passing comments in the two NRAs but elsewhere as well (e.g. Reuter and Truman 2004), different predicate offenders use different money laundering techniques, these aggregate threat measures are marginally relevant to the regulators, who are important consumers of NRAs.

However, in addition to financial regulators, there is a second set of policymakers for risk assessment, namely investigative or police agencies. For them, sector-level threat is conceived differently. It is money that tries to enter sector i, divided by the total money generated by the predicate crimes for which the agency is responsible; let’s designate it as p_i(1)*, i.e. the attractiveness of the sector for launderers of this kind of predicate offense revenue. The drug enforcement agency will want to allocate its AML efforts across sectors⁸ to reflect the sectoral allocation of drug revenues laundered. Or, if there is a single law enforcement agency responsible for all crimes, the agency will want to allocate its AML investigative efforts in accordance with each sector’s share of total money laundering.⁹ The numerator is the same as in the sectoral threat assessment, but the denominator is different, being total laundered euros for the enforcement agency and total sectoral euros for the regulator. Vulnerability is the same for both regulators and criminal investigative agencies.

Note the importance of denominators in this discussion. Though threats appear in the FATF methodology without dimensions, they are best thought of as probabilities. A p_i(1) equal to 1 would mean that all money flowing into the sector is laundered—obviously implausible, but an easily understood interpretation conceptually. A p_i(1)* = 1 would mean that all the crime money was laundered in sector i and none in the other sectors. A p_i(2)=1 means that all laundered euros entering sector i are successfully laundered.

This explication puts the exercise in simple arithmetic form, as though there were credible estimates of the proceeds of crime, by predicate offense, and of the amount of money laundered, by sector. In fact, no such estimates are available for any country. Nonetheless, these are useful constructs against which to compare what is done in the two NRAs, and what information they provide about specific concepts of risk.

Money laundering risk is said by FATF to be a function of threat and vulnerability. Drawing an analogy to a risk assessment for earthquakes shows the logic of this distinction. Japan is faced with the threat of many earthquakes each year, and this requires the construction industry in Japan to build houses that are not vulnerable to earthquakes. Earthquakes are a lesser threat in Germany, which means that the country can be more relaxed about building houses that are vulnerable to earthquakes. The analysis of threat helps to determine which vulnerability level is acceptable. But for money laundering, the threat is not exogenous per se (i.e. determined outside the system), as are earthquakes. Threats are partly determined by vulnerabilities. Consider the extreme: a country in which the likelihood of detection so high that it is almost impossible for criminals to enjoy their ill-gotten gains.¹⁰ The threat will be slight as a consequence of the prevention effect of AML policy. An exception here would be when the threat does not come from domestic crimes, but from foreign crimes. Money made from foreign crimes coming into the country is said to be a threat, but here also one can argue that this threat would be lower when criminals know that the country fights money laundering very effectively.

Italy’s National Risk Assessment

The Italian NRA, like that of Switzerland, includes separate assessments for money laundering and for terrorist financing. We discuss here only the ML risk assessment.

Unlike other NRAs, the Italian NRA consists of two documents: a methodology document and the actual assessment (labeled “synthesis” in English). The methodology is ambitious and transparent, though also conceptually unclear. The published assessment itself is more vague than its methodology, both because it presents no data and because it does not explain how conclusions were reached.¹² This 30-page document does, however, provide an exceptionally fine-grained analysis of the policy measures that should be strengthened in order to reduce the vulnerability of each of a list of 18 distinct modalities by which money can be laundered in Italy. We had access to a longer (160-page) report that was provided to the FATF assessors for the MER (in English translation) and widely distributed to many private and public stakeholders (in Italian); we shall refer to the 160-page document as the “full report” and the shorter version as the “published report”.¹³ The full report gives more insight into the assessment process and what information was used to reach conclusions.

We briefly discuss the methodology and the NRA before raising some concerns about conceptual and methodological aspects. The critique is illustrative rather than exhaustive. While we are critical of it on methodological grounds, the NRA is admirably clear-eyed about the weaknesses of the current AML regime and offers specific remedies.

Methodology

The exercise was carried out by a committee with at least 11 agencies¹⁴; academics and private bodies (e.g. trade associations) were also involved. An ad hoc “group of experts” is referred to, but even the full report provides only a list of organizations and does not identify the nature of the expertise involved; no independent researchers are mentioned. The methodology identified eight distinct types of data (both qualitative and quantitative) that would be used; it also foresaw use of reports by “international bodies, academic studies, and specialized press” (p. 4).

In one sense, the methodology promises a great deal. It describes in broad outlines many activities that might contribute to the goals enunciated at the beginning. However, it provides a muddy vision of the underlying concepts and essentially none of the critical detail about how the various data sources were going to be analyzed. The NRA often did not deliver what was promised in the methodology.

The strategy to first publish a methodology before conducting the NRA aimed to ensure that all authorities agreed on how the NRA was to be conducted and that the process would be robust.¹⁵ This is a sound approach for increasing the perceived integrity of the findings.

The NRA

The methodology document refers to the 2013 FATF terminology, but the NRA itself deviates from that terminology. Instead of measuring threats and vulnerabilities—as the FATF proposes—it measures “inherent risk” and the “effectiveness of the regime”. The inherent risk is a characteristic of the national economy. It is a function of threat (actually, the consequences of crime) and socioeconomic criticalities (which resembles the idea of vulnerability, operationalized with two indicators: use of cash and the shadow economy). No explanation is offered as to how the risk judgment was reached or whether it was a relative or absolute level.

The effectiveness of the regime is assessed for different sectors (such as banks and lawyers), and starts with the determination of the specific risk for that sector (inherent vulnerability of their services and products) before assessing the sector’s vulnerability after preventive measures. With this, Italy departs significantly from the FATF terminology and proposed methodology. But this approach has its merits. Defining what is inherent risk (crime and two country characteristics: use of cash and shadow economy) and what are the inherent vulnerabilities of each sector makes clear the challenges facing AML efforts. The FATF guidance is unclear on how the concepts of threat and vulnerability are related. As noted in our conceptual framework, FATF’s terminology incorrectly views threat and vulnerability as determined independently of each other.

After determining the sector’s inherent risks and vulnerabilities, the Italian NRA assesses the effectiveness of the preventive measures. No explanation is offered on how effectiveness is measured, but the quantity and quality of the suspicious transaction reports (STRs) clearly played a role.¹⁶ The number of STRs is offered as a proxy for the rigor of the sector’s AML efforts. However, it is similarly plausible as an indicator of the severity of the sector’s AML problem. The simple fact is that STR quantity reflects both the rigor of AML efforts and the severity of money laundering, and no one has developed a methodology for distinguishing the two factors. This is a problem for all studies of crime that rely on law enforcement indicators such as arrests to measure the underlying problem. This remains a problem in all the NRAs we have seen so far. (See also our discussion on the Swiss NRA in the next section.) On a positive note, Italy also considers the quality of STRs (e.g., whether all fields of the report template are filled in and whether sufficient information is provided with each STR). The quality of STRs can be compared across sectors, providing a plausible indicator of competence and conscientiousness of each sector. A similar analysis of STR quality should be doable in any country.

The inconsistent use of the terminology makes the analysis more difficult to follow. For example, the socioeconomic criticalities (published Italian NRA, p. 14) that determine the inherent risk together with threats (which are measured by three indicators of the consequences of crime) are also called “weaknesses” (p. 7), “challenges” (p. 6), and “vulnerabilities” (pp. 6–7).

Most countries do not include consequences in their risk assessments for money laundering; FATF itself notes that it is often very challenging to find measures of consequences. Italy is very unusual in that it adds consequences to the analysis. However, the NRA does not include the consequences of money laundering itself, but rather the consequences of the specific crimes (Methodology Report, p. 6).

The two socioeconomic criticalities (use of cash and shadow economy) “are considered the most significant in terms of ability to influence the Country’s inherent risk level” (published Italian NRA, p. 5). However, selecting only the salient factors creates an upward bias in estimated risk. Salient factors are generally the weaknesses of the system. If every country would include only the salient factors in their NRA, each country would conclude that the risks are very significant, as Italy does (published Italian NRA, p. 14). For example, the Swiss NRA emphasizes the opacity of business ownership and an open economy, since the foreign owners of funds are more difficult to investigate than domestic owners. Similarly, the NRA for Canada noted the increased threat arising from the very diverse immigrant population resident in Canada, including many from high-risk countries. Italy has a smaller immigrant population with fewer (at least until recently) from high-risk countries. Including all relevant risk factors in the assessment for Italy would give a more complete description of the money laundering risks and might lead to downward revision of the inherent risk estimate.

The NRA does not reach a conclusion on the vulnerabilities for the country as a whole. The reason given is that “it is not always easy to summarize the system’s overall vulnerability in a synthetic judgment” (published Italian NRA, p. 31). But would it be useful anyway to have a “synthetic judgment”? Although not reported in the published NRA, the full report does provide an assessment of the relative vulnerabilities per sector. This is a more useful result than an overall judgment. The relative vulnerabilities per sector can inform where AML measures or supervision should be increased, as also indicated in more detail in a later table (published Italian NRA, pp. 31–2). It is difficult to see the policy relevance of an overall judgment, especially a relative one.¹⁷ The question then becomes: relative to what? A possible answer might be: relative to other countries. Indeed, overall scores for national AML frameworks can be useful for comparisons between countries, but this is not the goal of an NRA and can only be useful when a similar method is used to assess each country.

On a positive note, the Italian NRA provides clear and specific policy recommendations. Table 7 (published Italian NRA, pp. 31–2) indicates the priority for AML along two dimensions: sectors (banks, notaries, lawyers, etc.) and regulatory interventions (analysis, dialogue and training, operational interventions, and supervision). For example, for lawyers, chartered accountants, and accounting experts, high priority is given to all four intervention types. On the other hand, for insurance brokers, there is low–medium priority for operating interventions and enhancement of supervision, and no need for changes in the other two dimensions. In that sense, this NRA achieved the goal of providing clear policy-relevant advice by showing where the priorities for improving the AML framework are. The problem in assessing this is that these recommendations are not linked to any of the prior analysis in the synthesis or the full report.

Even though many institutions and experts were involved in determining the scores, the NRA does not present any measure of uncertainty of the determined scores, “a chronic disease of planners” (Quade 1975). This reflects the use of a consensus approach¹⁸; the group determining a particular score continued the discussion (sometimes over multiple meetings) until consensus was reached, as in a jury. This has the advantage that one circumvents the problem of how to consolidate multiple opinions into a single score. However, pushing for a consensus ignores the issue of why the experts differ in their opinions (Morgan et al. 1992). Important biases that might need careful consideration here include motivational bias or differences in expertise.

One hypothetical advantage of requiring consensus is that the group ultimately defers to its most knowledgeable member. But it is too optimistic to assume that expertise will dominate, or indeed that there is a single measure of expertise to determine who ought to be the highest authority. There is considerable evidence that face-to-face interaction between group members can create destructive pressures of various sorts, such as domination by particular individuals for reasons of status or personality unrelated to their capability as probability assessors (Myers and Lamm 1975). Seaver (1978) conducted a series of experiments with 10 four-person groups and concluded that simple aggregation of opinions without interaction produces the best results. He also noted that experts have more faith in assessments with face-to-face interaction, which might be important in persuading them to accept the results.

Switzerland’s National Risk Assessment

The NRA for Switzerland is an unusual mix of candor and opaqueness. It provides a much more explicit methodology than any other, even including the Italian with its separate methodology document. However, the Swiss NRA fails to show how that methodology was used to generate the findings. It also provides an extremely fine-grained analysis of the many sets of institutions that are used for money laundering in Switzerland, not simply those that are currently covered by the AML statute. There are details about how different classes of institutions acquire information that leads to SARs,¹⁹ unavailable in any other such document.

Data are presented and explained well to justify the findings. The final judgments about relative risk among sectors are clearly presented and given some justification. The analysis of within-country geographic variation is also distinctive.²⁰ It reaches the usual conclusion that the AML system is working well²¹ but allows that there are substantial risks associated with some important sectors such as private banking, lawyers/notaries, and money transmitters.

The NRA was conducted by the Money Laundering Reporting Office Switzerland (MROS), the country's national financial intelligence unit (FIU). MROS requested information from each regulated sector as well as from certain other professional associations in unregulated sectors that were thought to be used for money laundering (e.g. real estate). The MROS director assembled the report, which was reviewed by a standing committee of relevant agencies, but in its construction, it is less of an inter-agency process than that of other countries for which NRA process details are available.

Our critiques are primarily at the conceptual level. In most other respects this is truly a state-of-the-art NRA.

SARs

SARs played a central role in the analysis of risks. For example, an initial risk categorization for each sector subject to AML reporting requirements (banks, money transmitters, casinos, etc.) was based on five characteristics of SARs filed by the sector: (a) country risk (ranked from 1 to 4), (b) amount of money involved (1 to 4), (c) complexity of the transaction (1 to 4), (d) involvement of domiciliary corporations [essentially the Swiss version of a shell corporation] (0 or 1), and (e) politically exposed person (PEP) involvement (0 or 1). Each factor was assigned a different weight, based on expert judgment; for example, the amount of money involved had a weight of 1, while PEP involvement had a weight of 4. The real importance of these characteristics is the product of the assigned weight and the scale. The two characteristics that are deemed most important by expert judgment (involvement of domiciliary corporations and PEPs), therefore, effectively have the lowest importance for the risk score. The score for each SAR in the sector was then calculated, and the SAR scores were summed and compared to a potential maximum, the total if each SAR had received the highest score. Conceptually, it appears to be a measure of the relative risk that a transaction in the sector involves money laundering, or p_i(1) in our terminology.

On this basis, sectors were given a risk rating of between 0 and 5. The range was substantial: from 3.8 for lawyers and notaries, to 1.3 for lending businesses. This number was treated as the presumptive value for each regulated sector, but it could be overturned on the basis of expert judgment. Occasionally the NRA noted such discrepancies; for example, expert opinion suggested a higher score than the calculated number for the precious metals sector.²² For the five sectors that the NRA considered but were not yet subject to the AML law, there were no SARs, so expert opinion and a few examples were the basis for judgment.

The use of SARs for the purpose of risk assessment is problematic for a complex of reasons. The fact that the SARs in one sector have higher risk characteristics than those in another does not provide information about the riskiness of the full populations of transactions in the two sectors. To illustrate, let us consider two sets of institutions that have a population of transactions of similar riskiness. Institution class A spends twice as much on AML activities per 1000 transactions as does class B. Assume, as seems reasonable, that the riskiest transaction is detected first, the second riskiest transaction is detected second, and so on. Class A then will generate SARs that appear on average less risky than those of Class B, because it reaches deeper into the risk distribution. What is being compared is the level of AML effort.²³

The interpretation of the SAR data is further complicated by the fact that financial intermediaries use specific and sophisticated algorithms to select what in Switzerland are called “red flags”, and would probably be SARs/STRs in other jurisdictions. These algorithms include some of the very variables that are used in the MROS risk analysis. Assume for simplification that they all used exactly the same algorithms as MROS. Then, variation in the MROS sector risk measures based on SARs would simply identify their tolerance for risk. The “high-risk” sectors would be those that had high tolerance for risk and did not submit SARs with lower risk indicators. The term “high risk” would then refer to their “appetite for risk” rather than the nature of the underlying transactions. That might indeed be a basis for targeting those sectors, but it would be a statement about vulnerabilities rather than threats, to use the FATF terminology.

The proper method for comparing the riskiness of transactions by institutional class is to subject a random sample of transactions to this analysis, i.e. to assess each transaction in the sample in terms of country of origin, complexity, and so on. This is in fact a feasible exercise for a bank regulator.

Switzerland deviates from the FATF recommendations in its rules on the filing of SARs. FATF requires that each financial intermediary (FI) or designated non-financial business/profession (DNFBP) report suspicious transactions to the FIU. In some countries (e.g. Canada, the UK), this leads to hundreds of thousands of SARs/STRs—Italy reported 64,000 STRs in 2013. In Switzerland, the FIU requires that the FI or DNFBP conduct an investigation of the transaction to establish its credibility as a “suspicious transaction”. The result is a total of only about 1800 reports in 2015. The NRA very helpfully provides unique data on the number of potential SARs (called “red flags”) and on the number of actual SARs filed by the three largest banks in 2012 and 2013; they account for one-quarter of all SARs filed with MROS (see the table below).

Table 10: Filtering performed by the three banks submitting the most suspicious activity reports, 2012–2013

1. Source: CGMF (2015) Swiss NRA, p. 60

These data show that only about 1% of red flags generated SARs, which are often extremely detailed investigative reports, producing box-loads of documents. The remaining 99% of red flags may fail to generate a SAR for any number of reasons, such as lack of merit or inability to obtain enough information for a true assessment.²⁴ If we are correct that “red flags” are comparable to SARs in other countries, these data suggest just how noisy a signal other countries’ SARs are of the extent and nature of ML transactions.

Concepts of Risk

As noted in our conceptual section, it is important to compare sector threats to a specific denominator, either the volume of transactions of that sector (regulator perspective) or the totality of laundered money (investigator perspective). The Swiss NRA is never explicit about this, but the most plausible interpretation is that threat is assessed against the sector size. For example, it noted that the fact that retail banks and private banks accounted for many more SARs than did universal banks was partly explained by the larger asset base of the first two categories (Swiss NRA, p. 63).

The analysis is sometimes confusing because of its failure to identify a scale measure. Consider the following statement based on data from 2009 to 2013²⁵: “[t]he growing number of money laundering convictions and an analysis of the suspicious activity reports submitted to MROS show not only the higher overall effectiveness of Switzerland’s system but also a real increase in the threat.” (Swiss NRA, p. 34) The threat here is measured in absolute terms, not relative to an appropriate denominator such as the size of the financial sector or GDP. Nor is it obvious why the rising number of convictions is evidence of higher overall effectiveness, particularly given that the ratio of convictions to SARs has declined during the same period.

Despite the comforting overall assessment mentioned at the beginning of this section, the NRA delivers a very refined analysis of individual sectors that is explicit about those sectors that face high risk. We reproduce the summary table of the report below. It classifies two classes of banks as being “high risk” but makes an important distinction about the source of the risks. For universal banks, it is the “very high” threat, along with “medium” vulnerability that generates this classification. For private banks, the threat is less (“high”) but the vulnerability greater (“high”). This suggests, as indicated in our conceptual framework, that the universal banks need more attention from investigative authorities, while the private banks need more attention from regulatory authorities.

The NRA correctly notes that money laundering risk cannot be eliminated. However, it is fair to ask whether a system in which six separate classes of institutions are classified as high risk is consistent with the claim that “Switzerland has a full, coordinated and effective range of legal and institutional resources for combating money laundering and terrorist financing. “(Swiss NRA, p. 4) On the other hand, if the primary purpose of the NRA is to show that authorities have an understanding of risk in the country, then the table delivers exactly the kind of judgment that is being sought.

In contrast to the Italian NRA, the Swiss document does not provide a detailed set of recommendations for reducing sector-level risk. It does, however, put forth a set of recommendations regarding legislation to expand the reach of the AML system.

Concluding Comments

Both the Italian and Swiss authorities have noted that the NRAs are just first efforts and that they plan to repeat the effort in about 2 years. MROS in Switzerland has recently begun issuing narrower risk assessments for specific sectors, such as for non-profits and safety deposit boxes. (see FATF 2016, p. 40) Thus our critique is aimed at helping strengthen the next round of NRAs. We have focused on the way NRAs are executed and not on the commitment to do so. Our critique is that there are ways of using the same resources and only existing data (including expert opinion) to generate risk assessments that will better inform AML decisions with better substantiation.

We note that these two NRAs, like others that we have examined, are silent on many important aspects of methodology. For example, Italy refers to a “group of experts” but provides no information about the number of such experts or the nature of their expertise. This raises issues regarding the objectivity (it can be in the experts’ interest to underestimate or overestimate money laundering risks) and the breadth of their expertise (e.g. how competent they are to assess the quality of crime statistics). The same statements can be made about the Swiss NRA (and plausibly many others). Names are not needed but affiliations and expertise should be provided, as is standard in risk assessments in other fields. Rosqvist (2003) explains that, in accordance with the international standards on risk assessments quality and consistency IEC 60300–3-9,²⁶ one should distinguish five types of experts, who can only be used in their relevant risk assessment phase: decision-maker, referendary (i.e. dispute-settler), normative expert, domain expert and stakeholder. NRAs on money laundering are not transparent about the experts employed, but seem to use mostly experts from the last category: stakeholders. As pointed out by Rosqvist (2003, Table 2, p. 16), stakeholders should not be used for the estimation of risks, only the hazard identification and analysis of policy options. NRAs in the field of money laundering seem to disregard this.

Neither of the NRAs pays any attention to uncertainty. They present exact results, with nothing like a confidence interval or a more broadly defined “uncertainty range”, taking account of non-sampling error. Yet each of them commits to very specific scoring procedures that are arbitrary and which could have a large impact on findings. For example, the Italian NRA scores intensity of threat by predicate offense on a range from 1 to 4. There is no basis for believing that a “very significant” threat is just four times as high as a “non-significant” threat. What if instead of 1, 2, 3, 4 the scores were 1, 2, 10, 20? In the Swiss formula for assessing risk, the financial risk (i.e. amount of money involved) was given a weight of 1, whereas the presence of a PEP was given a weight of 4. No rationale is offered for that; what would have been the consequence of varying those weights? Moreover, the weights in the Swiss formula are modified by the fact that different scales are used, making the characteristics deemed most important by experts the least important for the eventual risk calculation. The scores themselves, as in the Italian analysis, are also arbitrary.

Our critique here is not that there is a simple way of choosing the right weights and scores, but that a systematic analysis has an obligation to acknowledge the possibly large consequences of decisions taken in assigning them. Uncertainty must be taken into account and should be documented. This is again standard in other fields of risk assessment.

Though some version of NRA was occasionally conducted²⁷ before FATF made such an exercise a requirement for the fourth round of mutual evaluations, there is no doubt that the inspiration for the recent rash of assessments was this FATF requirement. For example, the World Bank is providing advice to over 80 countries which are preparing for their MERs. Thus it is interesting to examine how the Italian and Swiss NRAs were assessed in the MERs for their countries.

In neither case is there more than a passing and mildly negative comment. For Italy, the MER explicitly states that it is “of good quality” and “uses multiple sources of information”.²⁸ The most critical comment is: “There are some data gaps (e.g., comprehensive statistics on ML/TF investigations, and international cooperation) and the methodology establishes how to deal with such gaps so as not to undermine the robustness of the assessment.” The MER notes that the NRA has not yet informed a national risk-based approach for AML but attributes that to the recency of the NRA, which was completed only 6 months before the MER team visited Italy.

The Swiss NRA received an overall positive reception but also more specific critiques than did the Italian NRA. The relevant paragraph reads:

Competent authorities in Switzerland generally all have a high level of understanding of ML/TF risks. The June 2015 NRA, to which the private sector contributed, has made an important contribution to this understanding. Overall, the NRA produced high-quality results, although some of the sources, which focused mainly on STRs, do not fully take into account emerging or developing risks. Nevertheless, the assessment is based on a realistic overview of risks, and examined all sectors covered by AML/CFT legislation and other sectors that present risks (e.g. real estate or free ports). Some important information that would provide Switzerland with a full picture of the nature and type of ML/TF risks to which it is exposed has yet to be taken into account. This includes risks associated with the use of cash, the fiscal framework, or legal persons or arrangements. (FATF 2016, p. 6)

The critical comments are mild and mechanical. The claim that Switzerland did not take into account “emerging or developing risks” could be made of any NRA.

Both Switzerland and Italy assert that the aggregate threat of money laundering cannot be reduced, but for very different reasons. Italy argues that the threat arises from broader structural characteristics of the economy, which are not susceptible to effective AML efforts. Switzerland argues that the threat comes from abroad and hence is not responsive to domestic activities. The Swiss argument is less persuasive. Surely the question is why Switzerland is attractive for the deposit of revenues from foreign predicate crimes. Perhaps the right comparison is the riskiness of Switzerland to other international financial centers such as Singapore, the UK, and the USA. Nonetheless, it appears paradoxical to claim that the AML policies of Switzerland are effective and at the same time that it attracts large flows from overseas.

A national risk assessment exercise should be a recurring activity, not a one-time exercise, aimed at ensuring that the system remains robust. A great deal has been learned from the first round, as illustrated by the NRAs for Italy and Switzerland. Not only have authorities identified sectors that need stronger AML action, but each country has learned the limits of their own knowledge and data. The task now is to look beyond these NRAs and to improve the methods and data, taking into account the more sophisticated practices that have been developed in other fields. The mutual evaluation reports have also evolved and improved over successive rounds. Risk assessments in other fields have improved as well, as institutions and experts become more familiar with the process. One easily implemented recommendation that would help this process is to greatly improve the reporting of the methods used so that countries can learn from each other.²⁹ More attention to the basic concepts is perhaps the most critical issue.