Introduction

In 2004, the Dutch National Criminal Intelligence Service produced its first National Threat Assessment for Serious and Organized Crime (NTA), which received a very positive reception from its customers (Landman and Meloen 2005). As a contributing institution to both the first NTA and the next one, scheduled to appear in 2008, the Police Academy of The Netherlands is in a good position to present a critical insider’s account of some of the issues and methodologies involved.

This paper discusses issues related to measuring organized crime as they have become manifest in the NTA, which constitutes the Dutch contribution to the EU Organised Crime Threat Assessment (OCTA). It intends to convey to a wider academic community certain issues of definition, methodology and paradigm, understanding the NTA process in terms of the communication of risks in a context of competitive defining institutions.

Situational context

Organized crime is notoriously hard to count (Vander Beken et al. 2004; Black et al. 2001; van der Heijden 1993; Maltz 1990). Efforts to define and count criminal organizations on a national scale have ended up in unmanageable numbers of incomparable entities, while counting ‘networks’ appears to be almost a contradiction in terms. Police analysts have tried focusing on separate individuals as the units of calculation in measuring organized crime, but such tallies appear to miss the essential element of level of organization.

In The Netherlands, the first national inventory of organized crime groups was drawn up in 1988 (Klerks 2000: 103pp.; van der Heijden 1993). Based on reports provided by the regional criminal intelligence services, analysts tallied no less than 189 criminal groups, although only three of those met all the criteria for organized crime except ‘political influence.’ The 1988 report immediately led the minister of justice to establish five interregional investigative teams. Roughly similar situation reports were produced in 1991, 1993 and 1995, although the defining criteria shifted somewhat through the years. In 1996, a parliamentary inquiry into organized crime and its control led to a substantial revision of policies (Parlementaire enquêtecommissie opsporingsmethoden 1996). From 1998 onwards, the existing measurement system was formally replaced by the ministry of Justice’s WODC organized crime monitor, which appeared in 1998, 2002 and is scheduled to appear again in the autumn of 2007 (Kleemans et al. 1998, 2002). Meanwhile, the police continued to produce their own situation reports and threat assessments (Korps Landelijke Politiediensten 2004). Since the late 1990s, the qualitative analysis and understanding of organized crime has been emphasized, although tallies of criminal groups were still produced for internal use.

Since 2004, member states of the EU are required to provide annual reports on their nation’s organized crime situation to Europol. This European policing agency has produced European Union Organized Crime Reports for 2004 and 2005. In 2006, these were replaced by OCTA: the EU Organised Crime Threat Assessment, a future-oriented assessment of organized crime (Europol 2007). OCTA’s methodology was improved for the second (2007) edition. Still, OCTA’s compilers acknowledge that “creating a ranking of OC groups and phenomena is a major challenge, not only for methodological reasons” (Europol 2007: 7). OCTA aims to identify the threat level from a European perspective. It does so by setting specific criteria that allow for a pre-selection of the most relevant criminal phenomena. By thus providing a qualitative assessment of the threat posed by organized crime, OCTA seeks to support decision-makers both at the EU and national levels.

Problem formulation

The relevancy of the NTA program and the measurement of organized crime activities in a wider sense are evident. Organized crime causes serious harm to society, and if left unchecked it would pose an existential threat to the rule of law and to democratic society as a whole. For that reason, governments put in substantial efforts and budgets to curtail organized crime phenomena through both preventive and repressive activities. These efforts can include certain measures that in themselves have potentially corrosive effects on civil liberties and the rule of law, such as uncontrolled covert policing, extensive monitoring of citizens’ activities, withholding evidence in court and certain disruption tactics.

The scope, intensity and direction of those curtailing efforts are to a large extent guided by threat assessments and in particular the NTA. Therefore, the quality and reliability of the NTA program should be a matter of concern to policymakers and administrators, as well as the academic community.

Issues of confidentiality restrict the access of academic researchers to police information. This sets certain limits to debates over the measurement of organized crime, as much of the relevant elementary information resides in confidential files. In many European countries, even the information presented in court during organized crime prosecutions is only available for research purposes when gathered on the spot. Court sentences in general only contain condensed and selected evidentiary elements needed to support judiciary decisions. These are only of limited use for in-depth organized crime research. Such in-depth research could presumably be based on (often problematic) journalistic accounts and on information gathered by researchers themselves on a first-hand basis, e.g., through interviews and direct observation. Most academic outsiders remain unable to access basic surveillance data gathered through covert observation, infiltration, informers, turncoats, telephone taps, hidden microphones and other investigative techniques.

This can be lamented, but is an obvious and necessary fact of life. The access to secondary intelligence products on organized crime, such as strategic threat assessments, is less troubled by issues of sensitivity. Strategic threat assessments after all often serve as input for political decision-making processes, which should expose them to professional scrutiny on issues of quality and reliability. The current situation in the European Union regarding transparency of the strategic intelligence process relating to organized crime still leaves room for improvement from the viewpoint of democratic control. In certain member states all information on their contribution to Europol’s comprehensive threat assessment is kept under wraps. The Dutch have a relatively open attitude toward strategic threat assessments. The rather detailed National Threat Assessment is available and discussed in parliament, as are the public versions of most regional threat assessments. The specific versions containing perpetrators’ names and details about ongoing investigations remain confidential, and for good reasons.

Where details about operations and tactical reports regarding organized crime should necessarily remain confidential, on the strategic level the information process could be mostly transparent and open for scrutiny. In the interest of professionalism, issues of methodology and reliability should be discussed in universities and professional forums.

Methodology

As the leader of one of the NTA’s project groups contributing to the end product in 2004, the author was involved in the project’s research design, collection and analysis of project data, writing of the project report and the final analytical synthesis of all contributing studies (Klerks et al. 2004; van der Heijden et al. 2004). In preparation of the second NTA, to be finalized in 2008, the author and a colleague produced a study on crime-relevant trends and developments (Klerks and Kop 2007). The coming winter, he will again be involved in the final analytical synthesis of over 30 situation reports and topical studies.

This paper aims to communicate, under the restrictions set by the classified nature of the NTA program, to an academic audience some issues of definition, methodology and paradigm that have surfaced in the course of drawing up both NTA versions. The information provided in this paper was contained in documents as well as in written and oral communications. Data collection was carried out in the course of working in NTA programs and in related research and advisory capacities. Due to the need to respect the official confidentiality requirements, not all relevant data could be quoted.

Data presentation

Organized crime is currently considered by the Dutch National Crime Squad (DNR) as being manifestations of ‘banditism’ (robbery offenses committed or devised by combinations of suspects) or ‘merchant crime’ (offenses in relation to markets for illegal goods and services) (Korps Landelijke Politiediensten, KLPD, dienst Nationale Recherche, DNR, s.a.). The term ‘combination of suspects’ is introduced by the DNR as similar to the more current ‘criminal cooperative’ (“crimineel samenwerkingsverband—CSV”). It is defined as “two or more persons jointly involved in a criminal offense” (Klerks and Kop 2007). For practical purposes, the DNR distinguishes between levels of organized crime as shown in Table 1 (internal instruction, KLPD).

Table 1 Levels of organized crime
Full size table

Analysts in all the 25 Dutch regional police forces are required to produce annual assessments of organized crime as is manifests itself in their area. Various data sources are exploited for this purpose, ranging from open sources to investigative and intelligence reports. These regional organized crime assessments, while varying in quality, are generally satisfactory and matter-of-fact. They primarily serve the regional police force management and the public prosecutor’s office, but as a spin-off, the information is forwarded to the National Criminal Intelligence Service to be used for the NTA and the annual report to Europol. Regional analysts receive detailed written instructions and oral briefings to stimulate their compliance and provide a format for their reporting. Simultaneously, a national inventory is taken of the most powerful and sophisticated criminal cooperatives: the ‘level 3 and 3+’ targets of the 1,000-strong Dutch National Crime Squad. In addition, thematic studies are conducted by KLPD researchers and analysts, based mostly on open sources, expert interviews and restricted police information. Topics include varieties of cyber crime, issues of corruption and integrity, offensive tactics used by organized crime against investigative agencies and judicial institutes, various forms of organized robbery and theft, and the nexus between criminal cooperatives and terror groups (Boerman and Mooij 2006).

The classified nature of much of the studies obviously hinders the free flow of ideas from which the quality of research could otherwise benefit. Given the limited availability and sometimes questionable reliability of data, plus the often very strict deadlines, the research quality is generally quite acceptable. The involvement of some university staffers temporarily assigned to working with the police and the supervision by some highly qualified academic organized crime experts further enhances the value of the reports. Even these conditions, however, cannot replace the open forum of academia where critical colleagues can challenge published research on issues of theory, data collection, methodology and analysis.

All these reports and estimates are collected and synthesized by the NTA Working group into a comprehensive National Threat Assessment, with Expert Panels reflecting on intermediate versions in a quality control cycle. When making the first NTA in 2003–2004, this was the most challenging phase of the program. Especially the weighing of the seriousness of societal consequences of criminal phenomena was problematic for lack of objective criteria (Landman and Meloen 2005). Both customers and researchers have expressed a need to devote more attention to validating the NTA’s results and conclusions. Currently the NTA is still conceived as a periodical physical product, the intention being to evolve toward a continuous monitoring system serving a ‘war room’ from where all national operations aimed at organized crime are coordinated and led.

Europol has developed rather strict format requirements for the reporting agencies tasked with providing their nation’s information on organized crime. Focusing more on qualitative than on quantitative aspects, Europol requires detailed questionnaires to be filled out for every criminal group selected. A common set of selection criteria is provided aimed at filtering out the more serious and dangerous groups. In parallel, questionnaires about selected criminal activities committed by multiple groups also have to be filled out.

Among the information items on criminal groups that have to be reported upon are details about the geographical area of operations, a group’s size, its composition, leading members, its assets, use of legitimate business structures, criminal contacts, level of violence and other threatening features.

Although organized crime investigations ultimately serve to present evidence in court, they are nonetheless of a sensitive nature. Organized crime groups show a great interest in finding out the government’s intentions and capabilities toward them. They have been known to actively sabotage ongoing investigations through espionage, intimidation and violence. Dutch academic researchers and journalists have also been threatened, sued and slandered by organized crime figures. In other European countries such as the UK, Ireland, France, Belgium and Italy, writers about organized crime have become victims of sometimes lethal reprisals. These and other facts provide ample arguments to restrict access to sensitive information concerning organized crime investigations.

Confidentiality in law enforcement contexts is required for a number of reasons.

  1. 1.

    Protection of sensitive sources, such as informants;

  2. 2.

    The need to avoid certain methods and techniques to become exposed, which would reduce their effectiveness;

  3. 3.

    Keeping lawbreakers from knowing the police’s level of knowledge and areas of interest;

  4. 4.

    The need to adhere to the Third Party Rule, i.e., the ban on retransmitting information obtained from partners on condition of confidentiality;

  5. 5.

    The legal requirements to protect the privacy of subjects;

  6. 6.

    Care for the security of police personnel and installations;

  7. 7.

    The need to protect ongoing investigations;

  8. 8.

    The need to avoid misinterpretations, e.g., in the case of draft versions and other internal correspondence;

  9. 9.

    The desire to avoid bureaucracy and extra paperwork;

  10. 10.

    In case of obvious trivia, the need to avoid wasting precious time and resources;

  11. 11.

    The desire to maintain control over information and avoid exposure to other parts of the bureaucracy or ‘competitors’ (as in: “organized crime is our prerogative”);

  12. 12.

    The desire to avoid exposing errors and vulnerabilities to the subject’s counsellor;

  13. 13.

    The desire to hide incompetence or otherwise avoid embarrassment;

  14. 14.

    The desire to hide or cover up improper, irregular or illegal behavior.

In the light of all these arguments, proponents of greater transparency appear to be fighting an up-hill battle. Their arguments for greater control include the legitimacy and appropriateness of the following questions.

  1. (a)

    Is law enforcement doing what it should (proper tasks and targets)?

  2. (b)

    Is police work done in full adherence to the rule of law and according to regulations?

  3. (c)

    Are the applied methods morally acceptable?

  4. (d)

    Is the work done in the most effective way (best practices)?

  5. (e)

    Is the work done in the most efficient way (tax money well spent)?

  6. (f)

    What are the results, and are more or other efforts needed?

  7. (g)

    Are there no undesired side effects?

While transparency is in itself something to strive for, the police in a system such as in Holland, with all the checks and balances and accountability procedures resulting from earlier scandals involving incompetence, have a point when they claim to be already ‘working under a magnifying glass.’ Excessive transparency ultimately does get in the way of ‘getting the work done.’ This calls for a delicate balance between democratic transparency and the need to protect information.

Some of the ground work on this precarious issue has already been done (Gill 1994, 2000; Moynihan 1998; Sulc 1996; Baxter 1991; Ericson 1989; Bok 1986; Robertson 1982).

Analysis

It should be noted beforehand that although critical in attitude, the author is an insider as well as a ‘believer’ in regard to the NTA program. This obviously affects the judgment in matters of appraising the observed practices. The data presented in paragraph 5 indicate that the organized crime reports that form the basis for the NTA are to a large extent factual and empirically sound in nature. Many reports are primarily based on evidence accepted by the courts. Reports on cases still under investigation are based mostly on empirical observations contained in sworn statements by professional police investigators reporting on matters of fact such as criminal offences observed first-hand. The relatively small fraction of purely intelligence-based reports should be treated with care, as such information can for a variety or reasons be sometimes unreliable. Unfortunately, during the processing of information to produce sanitized strategic reports, the distinction between intelligence-based and evidence-based reports may become blurred, which then negatively affects the reliability of the end product.

As to the methodology involved in drawing up the NTA, the production process of the contributing regional and thematic studies is rather thoroughly supervised on issues of quality and rigidity. Definitions and concepts have been made explicit (Korps Landelijke Politiediensten, KLPD, dienst Nationale Recherche, DNR, s.a.). The relatively low threshold for defining a ‘criminal cooperative’ on the regional level could be a reason for concern, but in the centralized analytical process for the NTA ‘minor groups’ are generally discarded as the focus remains on the more serious threats.

It should be conceded that the final synthesis remains to some extent a matter of interactive subjectivity and professional debate among the researchers involved, rather than a formal, explicit and rigid evaluative methodological process. The task of synthesizing so many reports on a wide range of issues and with varying levels of detail still appears to be very hard to capture in a feasible methodological approach. Added to this comes the challenge of having to weigh the seriousness of organized crime’s societal consequences on a number of aspects and on a strategic level.

The country reports drawn up to fulfill the EU reporting requirements in their level of detail and comprehensiveness provide a wealth of data for Europol’s analysts. Perhaps more importantly, the reporting process has motivated those member states that had not yet by themselves drawn up systematic organized crime inventories to develop a better understanding of their country’s problems in this field. The organized crime situation throughout the EU has thus become more transparent and manageable. This in itself has greatly facilitated both practical police cooperation and common policy initiatives.

The added value of Europol’s meta-analyses and thematic Analyst Working Files lies mainly in the identification of transnational links and commonalities between various organized crime structures as they are investigated in the respective member states. An illustrative example of this, drawn from the public version of OCTA 2007 (Europol 2007: 9), could be the emergence of ‘oriented clusters’ in several member states.

“Single criminal groups with common objectives no longer operate in isolation and this creates a powerful convergence of criminal intentions and resources. Strategic direction for their activities can be determined by policies decided by the leaders of the most dominant criminal group or by regular meetings of the most influential representatives of the individual groups. The presence of such “oriented clusters” that are led or, at least, coordinated by a common centre of influence is assessed as a major threat.”Footnote 1

Europol’s OCTA reports are supposedly the apex of organized crime analysis in Europe, where all national and thematic strategic assessments come together in an authoritative intelligence product. The mechanism for exchanging and comparing national crime reports holds important benefits to law enforcement agencies with full information access, but unfortunately the rather superficial character of OCTA’s public version withholds most of the analytical meat from academic researchers and outside experts. The 28-page 2007 OCTA report avoids all analytical depth and lacks any reference to sources. This severely limits its usefulness for academic debate. While political sensitivities about organized crime assessments are in themselves understandable to anyone familiar with the European political arena, Europol should be challenged to make better use of the wealth of information available to its analysts, when producing public strategic assessments.

Without going into details, it can be postulated at this point that international comparisons between organized crime situation reports and assessments are precarious. In many EU member states the data remain unavailable to the public. Those countries that do produce public assessments, with very few exceptions, limit themselves to rather brief abstracts that do not allow academic scrutiny as to methodology and empirical basis. Perhaps this is something that students of organized crime, due to the sensitive nature of their subject, will have to live with for the foreseeable future.

Discussion

The policing practices discussed in this paper concern the ‘high end’ of surveillance. They involve the production of risk knowledge at strategic levels. The shaping of the risk discourse, as conceptualized by Ericson and Haggerty (1997), is in this case intended to enable decision-making regarding the control of organized crime. The end products of surveillance are packaged in terms of profiles of human sub-populations and their risks, a matter that Ericson and Haggerty have addressed at the level of tactical policing. They have portrayed the police as knowledge workers who believe “that the world can be made more secure by ever more perfect knowledge of risk” (Ericson and Haggerty 1997: 295), adding the insight that much police ‘paper work’ is in fact being done to service the information needs of other institutions. At the time, they compared the way the police were deployed to that of social science researchers, as they are both “equipped with extensive closed-ended systematic observation reports along with coding instructions” (Ericson and Haggerty 1997: 429). Such similarities have become even more striking today, with many academically-trained professional researchers and analysts now working within police institutions to produce knowledge with a claim to scientific veracity and reliability. This only serves to confirm Ericson and Haggerty’s initial observation that the police through their communication formats “claim to broker consciousness and social existence, defining what should be taken as objective reality with respect to risks” (Ericson and Haggerty 1997: 430).

In the production of risk knowledge regarding organized crime, it is important to consider that the police are no longer the single defining institution. The department of Justice has its own research institute producing an authoritative periodical organized crime monitor, based on cases that have gone through the judicial process. Academic researchers have come up with competing discourses on organized crime that sometimes challenge the conventional wisdom brought forward by the police (van Duyne et al. 2006). Investigative journalists have written accounts of varying depth and reliability that communicate a more qualitative understanding of organized crime practices (Leistra 2006; Lensink 2006; Middelburg and Vugts 2006). And nowadays, the Internet provides a range of Dutch resources on the subject drawn together by various observers and participants (www.crimesite.nl; steven-brown.web-log.nl; www.onderwereldblog.nl; http://onore.web-log.nl). Finally, the Dutch police are not entirely in unison when it comes to deciding which department or branch should administer criminal intelligence matters.

To the police as knowledge workers and risk communicators in a contested and politicized arena, the importance of credibility is obvious. Unfortunately, this has not led administrators to strive for transparency to enhance the quality of research, but instead to attempts to control information concerning the measurement of organized crime. Reinforced by obligations of confidentiality toward Europol, this results in a communication style that at times appears somewhat constrained. Still, all this should not obscure the fact that the Dutch police are positioned in the European premier league when it comes to professionalism, openness and transparency. Thanks to this noblesse oblige, we can discuss Dutch practices in the measurement of organized crime in some detail today.