Despite a growing interest in migration questions, it is still not possible to systematically analyse immigration policies across time and a large number of countries. Most studies in this field have heretofore focussed on individual cases. Recently, there have been a series of studies that have proposed policy indices that allow for large-N analyses. It appears, however, that these studies have not always adequately addressed the main challenges of index building, that is, conceptualisation, measurement and aggregation. Moreover, they are for the most part limited to individual policy fields or there is a trade-off between the number of countries and years that are covered. The aim of this article is to present the Immigration Policies in Comparison (IMPIC) project, which proposes a new and comprehensive way to measure immigration regulations. The data set covers all major fields and dimensions of immigration policies for thirty-three OECD countries between 1980 and 2010. This article discusses the way immigration policies have been conceptualised, how policies have been measured and aggregated and demonstrates the potential of such a new data set.
For a long time migration has been considered a relatively minor topic in political science, with few scholars working in this research field. Over the last two decades, however, questions related to migration have come to the fore in both public and academic debates. The American Political Science Association’s decision to establish a new Migration and Citizenship section in 2012 is perhaps one of the most telling examples of this subject’s growing importance. In the section’s first newsletter, Hollifield and Wong (2013: 3) opened the discussion with the observation that ‘the scholarly study of migration has, over the past several decades, slowly entrenched itself in the mainstream of political science’.
For five major political science journals, Hollifield and Wong (2013: 7–8) show that there has been a clear upward trend in the number of migration-related articles between 2000 and 2012. The vast majority of these articles focus on attitudes, behaviour or incorporation. The dynamism that has characterised recent engagement with questions relating to migration has not, it seems, communicated itself to the study of immigration policies. There is, of course, a long list of studies that have analysed immigration policies (e.g., Geddes, 2003; Messina, 2007; Schain, 2008). These studies, however, have mostly focussed on individual countries, or only compared a small number of countries.
Bjerre et al (2015) have shown that, with a single exception, only since the mid-2000s have there been studies that compare a large number of cases by quantifying immigration policies (Timmer and Williams, 1998; Thielemann, 2003; Hatton, 2004; Mayda, 2005; Givens and Luedtke, 2005; Lowell, 2005; Oxford Analytica, 2008; Cerna, 2008; Economist Intelligence Unit, 2008; Klugman and Medalho Pereira, 2009; Ortega and Peri, 2009; Ruhs, 2011; Pham and Van, 2014; Fitzgerald et al, 2014; Peters, 2015). The majority of these studies have, however, been conducted by economists. Only very recently have political scientists (but also others) started to build immigration policy indices (for an overview see Helbling et al, 2013; Gest et al, 2014).
“‘Only since the mid-2000s have there been studies that compare a large number of cases by quantifying immigration policies’.”
“‘The Immigration Policies in Comparison (IMPIC) project will remedy these limitations and gaps by providing a more comprehensive data set’.”
All these projects have proposed very innovative ways to measure immigration policies. However, as Bjerre et al (2015) show in their study that compares all existing indices, the three main challenges encountered in index-building (conceptualisation, measurement and aggregation (Munck and Verkuilen, 2002)) have sometimes been inadequately addressed in these studies. Bjerre and her co-authors have shown that such studies thus far include hardly any discussion of the conceptualisation of immigration policies and that justifications of methodological decisions concerning measurement and aggregation are often absent from their pages. It is therefore often difficult to know what a policy index is really measuring and to what extent it constitutes a valid and reliable tool. Moreover, besides not being accessible, the existing data sets are for the most part limited in their empirical scope – either because they only include individual policy fields such as labour migration or asylum policies or because there is a trade-off between the number of countries and years that are covered. As will become clearer throughout this article, the Immigration Policies in Comparison (IMPIC) project will remedy these limitations and gaps by providing a more comprehensive data set. A more detailed conceptualisation is proposed and the empirical scope is extended across cases (thirty-three OECD countries), time (1980–2010) and policy dimensions. Methodological issues concerning measurement and aggregation will be discussed more extensively.Footnote 1
Although economic migration together with refugees and family migrants have existed throughout history, policies specifically addressing these groups of migrants only came into existence during the inter-war period, and were not widely adopted until after World War II. In order to be able to answer a wide range of research questions on the restrictiveness of immigration policies across time and on the convergence or divergence of policies across time, the index must preferably cover as great a period as possible between World War II and the present day. That being said, the retrospective collection of information on immigration policies requires significant resources. In many cases, information is only readily accessible for the most recent policies, and the acquisition and analysis of old legal sources would require too great an investment of time and resources. For these reasons, we decided to cover a shorter period of three decades. This time span not only allows for longitudinal analyses but also for comparison with existing studies on integration and citizenship policies that cover the same period (e.g., Howard, 2009; Koopmans et al, 2012).
By including virtually all OECD member states, it is possible to study countries with very different immigration experiences: there are traditional settler states (Australia, Canada, New Zealand and the United States), countries that have experienced increasing immigration since WWII (e.g., Great Britain, Germany, France), countries that recently turned from emigration to immigration (e.g., Italy, Spain) and countries that have experienced very limited immigration or have not yet been given attention in this regard in the literature (e.g., Finland, Japan, Chile, Mexico).Footnote 2 Among other things, this will allow investigations of how immigration flows shape immigration regimes and the formation of immigration policy in emigration countries (e.g., Poland).
Collecting data on OECD countries is advantageous in that it can be easily matched with demographic and economic data provided by the OECD itself. Moreover, all these countries are established democracies and are thus easily comparable on a broader range of institutional variables. More specifically, they all constitute functioning states that have immigration policies that can be compared. Many non-OECD countries have hardly any formal immigration regulations, while others constitute weak states that do not have the means to impose such regulations. This is, of course, not to say that the functioning of immigration policies is the same in all OECD countries as the context and history of immigration is very different across these cases. Moreover, there is no reason not to include non-OECD countries in further studies. On the contrary, it would be very enriching to include cases that are largely neglected in this field. Many non-OECD countries and autocracies have experienced substantial immigration inflows over the last decades.
The aim of this article is to present a new approach to the measurement of immigration policies. According to Munck and Verkuilen (2002), three aspects need to be taken into account when measuring policies: conceptualisation, measurement and aggregation. The remainder of this article will be structured along these aspects. We first present the way we have defined and delimited immigration policies. By doing so, we differentiate between various immigrant groups that are targeted by immigration policies; between policy outputs and outcomes; between neighbouring fields such as integration and citizenship policies; and finally between different policy dimensions. In the section on measurement, we present the way we have selected the concrete items to be measured; the type of sources we used; how the data has been coded; and which measurement levels we have chosen. In the third section, we discuss aspects regarding aggregation and weighting. In the concluding part of this article we will outline the new avenues for research opened up by this data set. It is our aim to show that the IMPIC database will afford researchers an unprecedented level of depth and detail in their study of immigration policies, particularly with regard to their causes and effects.
To measure a policy, it is first and foremost important to define it. This definition should encompass all relevant aspects of a policy field and account for the potential multidimensionality of a concept. Such a conceptualisation should avoid conceptual redundancy, that is, multiple inclusion of similar aspects and conflation of aspects that need to be kept apart (Munck and Verkuilen, 2002).
As we have already argued in Bjerre et al (2015), defining a complex and multidimensional subject such as ‘Immigration Policy’ is not an easy task. Following others (e.g., Hammar, 1990; Brochmann, 1999; Meyers, 2000; Andreas, 2003), we define immigration policy as government’s statements of what it intends to do or not do (including laws, regulations, decisions or orders) in regards to the selection, admission, settlement and deportation of foreign citizens residing in the country.
“‘Defining a complex and multidimensional subject such as “Immigration Policy” is not an easy task’.”
This definition, as straightforward as it is, nevertheless lacks specificity for choosing concrete indicators. To identify an appropriate set of indicators, four questions need to be addressed:Footnote 3 (1) How should we distinguish between types of immigrants targeted by immigration policy? (2) How do we differentiate between measures of immigration policy output, implementation and outcomes, and which of these should be the focus of the IMPIC index? (3) How can immigration policies be distinguished from the neighbouring fields of integration and citizenship policies? (4) How can the laws that regulate immigration be systematically grouped?
DIFFERENTIATING BETWEEN TYPES OF IMMIGRANTS TARGETED BY IMMIGRATION POLICIES
Many scholars have argued that immigration involves long-term settlement (e.g., Messina, 2007: 23–24, citing Hammar, 1985). Our definition is, to some degree, broader: we define immigration as ‘people moving from one nation-state to another and thereby taking up residence in the destination country’ (Helbling et al, 2013). This definition also includes temporary residence and work permits that stand for forms of migration that are on the rise (see Lenard and Straehle, 2012; Skeldon, 2012). Increasingly, industrialised countries resort to seasonal schemes to ensure the demands of a flexible labour market while at the same time prohibiting the long-term settlement of immigrants. Policies tend to be most restrictive towards temporary low-skilled workers. Hence, not incorporating these schemes would make countries’ immigration policies appear deceivingly liberal. However, we follow Messina’s (2007) and Hammar’s (1985) proposal by excluding certain other forms of short-term mobility, namely commuting and tourism, as well as student migration.
As argued in Helbling et al (2013), and as depicted in Table 1, what we then define as the immigrating population is primarily made up of four fields, which reflect the main reasons why states may accept immigrants: for economic reasons (labour migration) (Freeman, 1978, 1979; Hollifield, 1998), social reasons (family reunification) (Cholewinski, 2002; Honohan, 2009), humanitarian reasons (refugees, asylum) and for cultural and historical reasons (co-ethnics) (Groenendijk, 2006; Jerónimo and Vink, 2011; see also Givens and Luedtke, 2005: 3). The last group concerns people who are entitled to easier access to immigration because of cultural or historical affiliations to the nation-state. This might be because these groups share the same language or religion as the country of destination; because their ancestors emigrated from this country; or because of former colonial ties.
Within each policy field we further differentiate between different groups of migrants. There are, for example, different ways to enter a country as a labour migrant. While in some cases there has been only one permit for migrants entering legally for work purposes, in most countries several permits exist. They are mostly awarded based on immigrants’ skill levels. It is important to account for these different entry routes as regulation with regard to conditions, eligibility criteria, rights and status vary (see Ruhs, 2013). Collecting data on only one of the existing entry routes would therefore lead to a distorted view of a given country’s immigration policy.
Similarly, in the field of family reunification policies, we differentiate between sponsors that are third country nationals and sponsors that are citizens. We exclude sponsors who are citizens of the European Union because these are treated very similarly to citizen sponsors. Among the regulations concerning asylum seekers and refugees we distinguish between asylum seekers, recognised refugees and people with humanitarian/subsidiary protection. In the field of co-ethnics we allow for up to four different entry routes for co-ethnics. Notably, only one country has four different groups of co-ethnics (Germany). Greece is next, with three. A little less than one-third of the countries recognise two groups of co-ethnics. Another third of the countries accepts one group of co-ethnics, while in the remaining third the concept of co-ethnicity as a basis for easier access to immigration and citizenship does not exist.
DIFFERENTIATING BETWEEN POLICY OUTPUTS, IMPLEMENTATION AND OUTCOMES
For the IMPIC data set we adopt a narrow definition of policy by only focusing on policy output. Easton (1965) defines ‘outputs’ as ‘the binding decisions, their implementing actions and […] certain associated kinds of behaviour’ while outcomes are ‘all the consequences that flow from […] the outputs of the system’ (Easton, 1965: 351). Easton (1965) thus subsumes both legal regulations and their implementation under the heading ‘outputs’. We try to be slightly more specific, and refer only to legal regulations as policy outputs, while implementation will be regarded as a separate aspect of policy. We thus understand policy outputs as legally binding regulations, while outcomes are immigration rates. Implementation is the process that links the outcome to the output.
Some scholars argue that focusing only on policy outputs while disregarding how these are interpreted and acted upon in practice is too narrow an approach. Money (1999), for instance, argues that formal regulations do not necessarily lead to the intended outcomes due to diverging implementation processes, among other things. We certainly agree that data on outcomes and implementation is needed for any research question that revolves around the effects of policy. At the same time, we are nevertheless convinced that it is crucial to keep policy output, implementation and outcome analytically separate in order to isolate, for example, the effects of legal binding decisions on immigration rates.
DIFFERENTIATING BETWEEN IMMIGRATION, INTEGRATION AND CITIZENSHIP POLICIES
How can we demarcate immigration policies from the neighbouring fields of integration and citizenship policies? According to Hammar (1990) there are three gates that immigrants have to pass through to acquire the same status as native citizens: the first is the gate of entry to the territory, the second is the gate of being allowed to settle and the third is full membership, that is, citizenship. The differentiation between different levels of social closure is also recognised by Weber (1946) and Brubaker (1992: Chapter1); 2010). The latter differentiates between an ‘entry’ level located at the territorial border, and ‘settlement and full membership’ that is regulated on the territory, that is., after having crossed the territorial border. Givens and Luedtke (2005: 2) agree with this demarcation, and add that because they follow very different political logics the two fields of ‘immigration control’ (‘entry’ in Brubaker’s terminology) and ‘immigrant integration’ (‘settlement and full membership’) need to be treated as two analytically distinct policy areas. Following these approaches, we define immigration policies as policies that concern entry, while integration and citizenship policies deal with settlement and full membership, respectively (Helbling et al, 2013).
However, to some extent there is an overlap, and ‘immigration policies’ leverage does not stop the moment an immigrant has crossed the physical territorial border. Hence, in the construction of the IMPIC Index, we also incorporate regulations concerning the legal length of stay that is associated with a certain type of entry category, as well as the rights to work. While both the duration of permit validity, as well as the access to the labour market, can also be conceptualised as an integration policy, integration policies as a whole also cover political, social and cultural rights of immigrant groups. In other words, while the legal length of stay stipulates for how long immigrants may stay on the territory (and the right to work for how long they can sustain living there), integration policies stipulate how immigrants live in the host country. The exclusion of integration and citizenship policies is furthermore justified by pragmatic reasoning; many indices measuring citizenship policies already exist and it would thus have been a waste of resources to collect the same information again (for an overview on existing indices see Helbling, 2013).
DIFFERENTIATING BETWEEN POLICY DIMENSIONS
It has already become apparent that immigration policy is a multi-dimensional construct that is comprised of a variety of regulations. These regulations are grouped according to their location in a two-dimensional scheme (see Table 1). This framework allows us to give our concept a clear hierarchical structure and to aggregate on different sub-dimensions. On the first dimension, which we call ‘modus operandi’, a distinction between regulations and control mechanisms is made (see Brochmann and Hammar, 1999; Doomernik and Jandl, 2008). Regulations are binding legal provisions that create or constrain rights (Dreher, 2002). Controls, on the other hand, are mechanisms that monitor whether the regulations are adhered to. The ‘modus operandi’ hence tells us how laws operate. To give an example: a regulation might state that immigrants need a work permit to take up a job. The corresponding control mechanism would be sanctions for employing illegal immigrants. Controls differ from implementation, because they are formally regulated in the law.
Within the control mechanisms we also find many elements that refer to irregular immigrants whose entry or stay is considered unlawful. We consider regulations regarding irregular migrants to be different from the other four policy fields, as they concern a category of immigrants that spans across all other four immigration categories. Such immigrants have not been admitted for economic, humanitarian, social, cultural, or other reasons, but have nonetheless crossed national borders, or have remained in the country after their residence permit had expired (i.e., overstayers). Requirements for registration or the possession of personal identification documents, for example, constitute control mechanisms for regular immigrants in order to keep them from overstaying their working or residence permits. On the other hand, sanctions for forged documents, schooling rights for children of irregular migrants or carrier sanctions are control mechanisms that specifically concern irregular migrants.
On the second dimension, we account for the fact that states regulate and control immigration not only at their borders, but also within their territories. The ‘locus operandi’ differentiates between externally and internally targeted laws. Inspired by the classification that was developed by the Migration Integration Policy Index (MIPEX) (MPG, 2006), we further distinguish between different sub-dimensions within the external and internal regulations. External regulations are subdivided into eligibility requirements and conditions. Eligibility requirements stipulate which criteria an immigrant has to fulfil to qualify for a certain entry route. Conditions are the additional requirements that need to be fulfilled. We further distinguish between regulations regarding the security of status, that is, all policies that regulate the duration of permits and access to long-term settlement. Finally, ‘Rights associated’ are all the policies that govern which rights immigrants receive in regard to access to employment, and how they are monitored once they are within the territory.
SELECTION OF ITEMS
Having developed a conceptualisation, we needed to operationalise the different dimensions by selecting specific items that we can measure (see also Bollen, 1980). The following basic rules guided us in this process (see also Koopmans et al, 2005: 33): (1) the aim was to include multiple items per category; (2) we selected items that are widely discussed in the literature and deemed the most important by experts; (3) the items need to exist and be relevant in most OECD countries; (4) items need to vary across countries (at least potentially); and (5) the items need to be relatively easy to compare, in the sense that their meaning should be the same in all cases studied and the sources to measure these items need to be available.
We had no specific number of items in mind for the overall scheme or for the individual boxes in Table 1. The general idea was to include enough items to cover all relevant aspects and thus to allow for enough precision and sensitivity (Elkins, 2000). On the other hand, it was clear that we could not include all existing aspects in our database, as this would not have been possible given our restricted resources. Rather, we aimed at including in each category all relevant items to account for the numerous manifestations of immigration policies (Munck and Verkuilen, 2002: 15).
We first took a look at the relevant secondary literature (mostly case studies), at research reports by international organisations, and at existing indices studies (see overview in Bjerre et al, 2015). For each policy field, we tried to find out which aspects are most often discussed and deemed relevant. We then presented the list of items to field and country experts and asked them to point out missing items, and tell us if they found certain aspects irrelevant. For each field we had two to three experts who were (with a few exceptions) political scientists specialised in one of the policy fields, such as asylum or co-ethnic policies. Country experts were the persons with whom we collaborated for the data collection (see below), and who mostly specialised in migration law.
This stage of the project did not pose any particular difficulties. There seemed to be a large agreement in the literature over which aspects are most relevant in the different fields. The comments of the experts lead to only minor changes of our list.Footnote 4 We were therefore assured that all our items were relevant (to varying degrees) in all OECD countries, and that these could (at least potentially) vary across countries and/or time.
TYPE OF SOURCES
One may draw on different sources to find information on how to measure policy outputs (see Bollen, 1986). This is particularly true with regard to questions of degree of restrictiveness, as there have been attempts in various fields in the past to do expert surveys in which individual policy specialists have been asked to evaluate certain policy aspects on a scale, for example, from liberal to restrictive (e.g., MIPEX, see Niessen et al, 2007). The problem with this kind of approach is that the findings depend on the subjective perception of the expert. Thus, it is rather challenging to the policy aspects, on which the evaluation is based. For example, one does not necessarily know whether an answer is based on his or her knowledge of the concrete regulation or on its implementation and effects. Moreover, even for experts it is difficult to ascertain the degree of restrictiveness of individual regulations. Finally, it is very difficult – if not impossible – to collect historical information as one can hardly distinguish retrospectively which laws have been adopted and come into force in which years. Examples of other sources include reports from states and international organisations, or secondary literature. The problem with these sources is that they most often have already selected specific aspects for their own analyses that might not fully correspond to one’s own list of items.
For these various reasons, we based our data collection on legally binding immigration regulations. By legally binding regulations, we mean both primary law (i.e., law that has come into existence through the parliamentary legislative process, e.g., statute law) and secondary law (i.e., law that is created by executive authority and derived from primary legislation).
CODERS AND CODING RULES
For the analysis of these regulations we closely followed the lead of established projects in the citizenship literature, namely the EUDO citizenship project (Vink and Bauböck, 2013) and the project of the Indices of Citizenship Rights for Immigrants (ICRI) (Koopmans et al, 2012). In a very similar vein, we collaborated extensively with country experts that helped to provide us with the information we needed. This was necessary, given the impossibility of recruiting a research team that can read and analyse all relevant documents in their original language. Moreover, many of the documents were not accessible online (this is especially true for earlier documents). Finally, country experts are crucial to understand and correctly interpret national specificities.
It was very important to us to collaborate with legal scholars given the heavily legal nature of our source base. For obvious reasons, legal scholars have more detailed knowledge of these regulations than social scientists in most cases, and have a better sense of where to find relevant documents. In most cases we were successful in finding an advanced legal scholar who has been working on migration issues for several years or even decades. In some cases, we collaborated with political scientists or economists who are specialists in migration research. We paid them a certain amount of money both as an incentive and to cover some of the costs that they incurred in conducting their research, such as hiring a research assistant for a couple of months.
Coding the legal texts completely by ourselves did not seem a feasible alternative to us for the reasons given above. It would have been virtually impossible to find so many legal scholars in one place especially given the fact that legal scholars who specialise on one country also work in the respective countries. And even if we had managed to hire students from all OECD countries this would have posed the problem that they do not have the expertise of more advanced scholars that have worked in the respective countries. Collaborating with country experts poses of course problems of inter-coder reliability. It was therefore crucial to collaborate closely with each expert and to create a common understanding of the main concepts used in the project.
One of the most time-consuming phases of the project was the construction of the questionnaire and the formulation of item questions in particular. For questions of reliability, it was crucially important that the questions and definitions were clearly understood by the country experts. There were several rounds of revisions during which the country and field experts, but also colleagues from other fields, commented on the structure of the questionnaire and the intelligibility of the questions and the instructions. Finally, we put together a detailed glossary that provided brief definitions of all the specific terms and concepts we used in the questionnaire.
To guarantee high reliability of the data it was not only crucial to create a common understanding of the most important concepts. During the recoding and data cleaning phase we discussed extensively the material and answers they provided to make sure that the questions have been understood correctly and in the same way across countries. In the instructions to the country experts, we clearly stated that we were only interested in information as it is stated in legally binding regulations and thus that we were not interested in subjective statements or how a law is implemented, evaluated or perceived. Therefore, for each item we asked experts to provide details about the legal sources they used to answer the question. One basic rule guided the formulation of item questions: they should allow as little interpretation as possible. For this reason, it was important to make sure that the questions were clearly understood by all country experts, that they were as close as possible to the factual information as it is found in legal documents and that they provided the entire range of possible answer categories. It was important to have questions that asked about the existence of a certain regulation (yes/no) or a concrete number. We thereby avoided questions that allow for any interpretation or evaluative statements, such as questions that ask about the degree of difficulty involved in acquiring a certain permit, or the degree of restrictiveness of a certain regulation.
We also tried to limit the number of open questions. On the other hand, we provided a comments field for all items, to allow country experts to elaborate on their answers in case they had the impression that some of the information they provided to answer our questions might be misleading or may have tended to oversimplify the actual reality in their country. This information was then taken into account during the scoring process.
MEASUREMENT LEVELS AND JUSTIFICATION OF MEASUREMENT LEVELS
All individual items vary between 0 (open) and 1 (restrictive) and thus indicate the level of restrictiveness of a specific regulation (Bjerre et al, 2015). The degree of restrictiveness indicates to what extent a regulation limits or liberalises the rights and freedoms of immigrants (see also Givens and Luedtke, 2005: 4; De Haas et al, 2014: 15). The measurement of a specific measure’s restrictiveness allows us to study both within- and between-country differences. This is an important advantage over studies that only coded policy changes (De Haas et al, 2014; Ortega and Peri, 2009; Mayda, 2005; Hatton, 2004), as one does not know from which level a policy change was initiated.
As Stevens (1946: 677) points out, ‘scales are only possible […] because there is a certain isomorphism between what we can do with the aspects of the objects and the properties of the numerical series’. The design of a scale that allows for the measurement of the restrictiveness of immigration laws needs to be guided by the properties of the raw data, but will nevertheless always involve some degree of arbitrariness (see also Jacoby, 1999). The first step of scale development is, therefore, the thorough review of the raw data and its properties. There are two types of scales in the IMPIC raw data: (1) Interval/ratio scales (e.g., items that measure fees that need to be paid in order to acquire a work permit, or the temporal validity of a permit); and (2) Ordinal scales (e.g., items that measure types of family members permitted to immigrate under family reunification provisions, or whether language tests were a required condition before immigrating etc.)
Having two different measurement levels – which stem from the nature of regulations rather than the way the question was posed – causes certain difficulties when later aggregating indicators into one single measure, as the scales are not comparable. One way to address this problem is to standardise the scales by equalising the range or data variability. This, however, leads to a different problem in which indicators lose their comparability over time. Instead, we made two key decisions that rendered the scales comparable without z-standardising the data: (1) fixing the minimum and maximum at the same value for all items, and (2) applying a threshold at the numerical value of 0.5 for the presence of a legal provision.
First, instead of empirically identifying the minimum and maximum value, we identified the theoretical minimum and maximum. We argue that the theoretical maximum in each item is always identifiable as the most restrictive measure and the theoretical minimum as the least restrictive measure. For example, if in a given country a legal provision on transit through a ‘safe third country’ does not exist, this country would be assigned the theoretical minimum value for all time points under study until this provision was adopted. The theoretical maximum on that item, however, would be if a country does not have any kind of asylum and refugee provisions, so that for a refugee it would not be possible to immigrate into the country for humanitarian reasons.Footnote 5 The minimum is assigned the numerical value of 0 while the maximum is assigned 1. In a way one could argue that this is in fact standardising the items, as all items vary between 0 and 1.
The second decision we took was to fix the presence of a legal provision at the value of 0.5. The reason is that items that are measured on an interval or ratio scale need to be made comparable with items that are measured on an ordinal scale. An example might help illustrate this point: consider the items ‘fee needed to be paid in order to attain a work permit’, and ‘whether a language test was a necessary condition in order to be able to immigrate’. The former item can range from a small to a very large amount, while the latter item is either present as a condition or not. Nevertheless, the presence of a legal provision on both the first and the second item increases the restrictiveness of a country. Only, for the first item we can also distinguish between graduations of restrictiveness. Thus, while having to pay a fee in order to attain a work permit and having to pass a language test would give a country both a score of 0.5, our fine-grained scoring also allows us to assign higher values to countries where the fees are relatively higher, thereby indicating a greater degree of restrictiveness. Nevertheless, this also means that while the language test item varies only between 0, 0.5 and 1, the work permit fee items show greater variability between 0.5 and the restrictive maximum. Hence, we theoretically assume that having to pay US$1000Footnote 6 for a work permit is a more restrictive measure (yielding the value of 0.9) than having to pass a language test (yielding the value of 0.5). If, however, researchers disagree with this assumption, they can apply a weight to the language test item, so that both items have the same influence on the final aggregated index.
As immigrants do not only face certain conditions but also have certain rights, the scoring steps for items measuring immigrants’ rights differ from the ones measuring conditions and requirements. Again, an example might help illustrate this. Take that of applicants for refugee status: if they had the right to appeal a negative decision, then this was scored as the least restrictive value of 0, but if, however, they did not have the right to appeal, this was scored a 0.5. For questions that asked about immigrants’ rights, we also applied a finer-grained scoring if information in the raw data allowed us to do so. The item pertaining to whether asylum seekers were allowed to undertake paid work while their application was pending, for example, allows for more nuanced scoring of restrictiveness. If asylum seekers could take up work right away, this was scored as the least restrictive; if they had to wait for a certain period, this increased restrictiveness by 0.1 steps for certain time intervals. If they had no right to take up paid work while waiting for a decision on their application this was scored a 0.5. Again, as in all other asylum and refugee items, the maximum value of one was only assigned if no legal provisions for seeking asylum or refugee status existed in a country in a given year.
While the differences in step size have certain disadvantages (as discussed above), the strongest argument for having a more fine-grained measure is that it captures changes within countries over time. The passing of a new law is by far rarer than changes or amendments to an already adopted one. Our scoring scheme has the advantage to be able to capture ,for example, if a country increases the required amount of funds an immigrant needs in her bank account in order to be able to immigrate from six months of self-sustainability to twenty-four months of self-sustainability as a restrictive change.Footnote 7 But even for comparisons between countries, a fine-grained scoring scheme has the advantage of being more precise. Being able to distinguish, for instance, between employer sanctions (i.e., fines or penalties for hiring undocumented workers) that can be considered rather negligible (e.g., a $1000 fine) and severe ones (e.g., fines around $100 000) gives us a more precise picture of which country is more restrictive than that yielded by a simple binary measure.
“‘How you weight and aggregate data depends on one’s theoretical framework and specific research question’.”
How you weight and aggregate data depends on one’s theoretical framework and specific research question. There is therefore no standard rule for aggregation. We agree with Nardo et al (2005: 23) that ‘[t]he absence of an “objective” way of determining weights and aggregation methods does not necessarily lead to rejection of the validity of composite indicators, as long as the entire process is transparent. The modeller’s objectives must be clearly stated at the outset, and the chosen model must be checked to see to what extent it fulfils the modeller’s goal’.
AGGREGATION LEVEL AND JUSTIFICATION
The problem of existing immigration policy indices is that they hardly account for the underlying dimensionality of their indices, and most often simply aggregate at a relatively high level (Bjerre et al, 2015). To counter this trend, we will not only provide the raw data with information on the individual items that allow each researcher to choose their own aggregation level, but will also provide aggregate data for each theoretical level of our index (dimensions and policy fields; see Table 1). All these differentiations are theoretically justified and enable us to respect the hierarchical structure of the index; each level can constitute a research topic in itself. This allows us, among other things, to investigate causes and effects of individual dimensions and policy fields.
As the policy fields correspond to different reasons why states admit immigrants, one might doubt whether anything like an overall immigration policy could possibly exist in actual fact. This is also partly an empirical question: To what extent do they constitute different policy fields or are linked to each other. This shows that in any case, disaggregated indices are crucial in this field.
AGGREGATION RULES AND JUSTIFICATION
Most existing immigration policy indices have chosen an additive, mostly unweighted aggregation rule (Bjerre et al, 2015). Additive aggregation means that items can be substituted (compensability). The absence, or lower values, of one item can be compensated for by the presence/higher value of another item. If, however, items constitute necessary features, they should be multiplied (there are different forms of multiplicative approaches, e.g., geometric means). For instance, if a necessary item is absent and thus takes the value of 0, it means that the policy does not exist at all.
Fully compensatory additive indices are problematic when it is normatively assumed that various criteria need to be given (e.g., democracy consists of various components to define a system as democratic). You cannot, for example, simply increase freedom of press rights to compensate for a complete absence of free elections. Full compensation is also problematic if we expect certain thresholds within an index. For example, in democracy studies, although continuous indices are the norm nowadays, one might still argue that a certain number of aspects need to be present – at least to a certain extent – to speak of a democracy.
This is not a problem for immigration policy because, among other things, we are not dealing with a normative/abstract concept, but with one that can be measured ‘quite objectively’ on a linear restrictiveness scale. Our conceptualisation does not imply that the various dimensions need to be there. The purpose of our scheme is rather to be able to cover all relevant aspects of immigration policy, as most other indices failed to include aspects that might play an important role.
In addition, the entire conceptualisation is based on the assumption that the various components can substitute for each other. Some states might have more restrictive external regulations while others focus more on internal regulations. These constitute different strategies to achieve the same goals. Finally, we assume complete linearity; there are no thresholds below which one would argue that there is no immigration policy, and no components of our conceptualisation are necessary. For example, if there are no regulations on rights associated, this does not mean that there is no immigration policy. It instead means that the policy is more restrictive.
In the immigration policy literature, only two projects (deliberately) apply weighting. The Migration Accessibility Index relies on expert judgements and Oxford Analytica on policy outcomes (inflow rates), to weight items. Both approaches are problematic.
The importance of an item might be assessed with its impact, for example, on immigration rates. Such a weighting would, however, violate our differentiation between policy outputs and outcomes. The importance of items constitutes, in this case, an empirical question. Value judgments by experts should be avoided as much as possible as they are presumably very unreliable. This is especially true if data are collected over time as in our case. It is also problematic if a large number of items have to be assessed/ranked. One can perhaps judge/rank a small number of items, but lose track if, as in our case, around 70 items need to be judged. Judgments through citizen surveys, as done in some fields, make no sense here, as citizens do not know the details of immigration policies.
Even if expert judgments should be avoided, value judgements play a small role in our project insofar as field and country experts helped us assess the suitability of our items. We explicitly asked them to tell us whether important items are missing or unimportant items should be cut. This can be considered a form of weighting as unimportant aspects were excluded.
To avoid indirect unequal weighting, we aggregate so that at each level each component has the same weight. Aggregation from one level to the other is therefore always done by averaging the items/components of the lower level (see Table 1). One sub-dimension is the mean of its items, one locus operandi is the mean of its two sub-dimensions, and a modus operandi of the respective internal and external regulations and so on.
THE POTENTIAL OF THE IMPIC DATABASE
“‘The IMPIC database will allow researchers to describe policy variation across time and space, and to study in greater detail the causes and effects of migration policies’.”
Given the conceptual and methodological limitations of earlier projects, we aimed to fill as many gaps as possible and to compile a data set that is as methodologically sound as possible. Our data set is, of course, far from flawless. Resource limitations in particular prevented us from fully addressing all issues of validity and reliability (Carmines and Zeller, 1979; Adock and Collier, 2001). Moreover, as the field is not yet very advanced, it is difficult to assess the validity of indices as this typically involves a comparison of alternative measures of the same concept (convergent or construct validity) (see for other fields for example Elkins, 2000; Marks et al, 2007; Helbling, 2013). Such a comparison is not possible in this case as this constitutes the first comprehensive data set on immigration policies and no widely accepted standard measure is available yet. Moreover, data sets that measure specific aspects of immigration policies and thus partly overlap with our data are not publicly accessible. There are integration indices such as MIPEX (Niessen et al, 2007) or ICRI (Koopmans et al, 2012) that partly overlap with our measure of internal regulation. However, the overlap of the temporal and geographical coverage is so small that results from validity tests would not be very meaningful.
In any case we are convinced that we have made an important step forward in this field, and that we have laid the basis for important future work. The IMPIC database will allow researchers to describe policy variation across time and space, and to study in greater detail the causes and effects of migration policies. Which are the most restrictive and most liberal countries? Have policies become more liberal or restrictive over time? Are there groups of countries whose policies present similar patterns? What factors lead to more restrictive and, conversely, more liberal policies? Do restrictive policies indeed lead to lower immigration rates? How great is the impact of immigration policies on immigration rates compared with other factors? For these and many other questions, it will now be easier to find answers.
In Figures 1 and 2, the variation of family reunification policies and the overall control mechanisms are shown. It appears that there is considerable variation across countries and time, and also that the overall level of regulation change across policy fields. While in some cases policies remained relatively stable, in others we observe movement towards more restrictive or liberal policies. Comparing the two fields, we notice that, overall, family policies remain relatively stable, with some notable exceptions such as, for example, Poland and Portugal. The control mechanisms, on the other hand, vary more significantly and in many cases we observe a development towards more restrictive regulations.
Looking at the overall development of an entire field might, however, be misleading, or at least conceal some more specific, underlying changes. In Figures 3 and 4, we have disaggregated family policies and displayed the development of external and internal regulations. Looking at these two dimensions separately, we observe some more variation and especially some opposing trends. While there are several countries in which external regulations have become more restrictive over the years, we see a completely different trend when it comes to internal regulations. Taking, for example, Denmark and Germany as cases in point, we see that in these two countries conditions and eligibility criteria became more restrictive in the second half of the 1990s and the 2000s, whereas regulations regarding rights and security of status remained more stable in the same period.
These analyses show that disaggregating indices allows for more precise analyses. While there might be a general trend towards more restrictive measures in one field, this does not necessarily mean that we observe the same trend in other fields or that this holds for all dimensions of a policy. It thus becomes clear that ‘immigration policy’ is a more complex phenomenon than it might first appear. This has also become evident in our discussion on conceptualisation that has shown that various aspects need to be taken into account to clearly delimit this domain.
Proposing disaggregated indices also has the advantage that it becomes more transparent how a database was put together or an index built. Transparency is crucial as it allows critical analysis and fosters broader discussions on how to measure policies. Therefore, it was crucial for us to discuss different ways of measurement and aggregation and to clearly justify why we have taken certain methodological decisions. This makes it easier for others to understand what the IMPIC database is actually measuring and thus builds general knowledge.
For further details of the project, publications and to access the data set visit the following webpage: www.impic-project.eu
It was initially planned to include all thirty-four OECD countries in the database, but Slovenia was dropped because of problems of finding a country expert (see section below on coders).
Of course many more items could have been included than the ones we selected. Given our limited resources we, however, tried to select only the most relevant ones.
Of course, this does not mean that the refugee could not qualify as a labour migrant in the same country, thus still be able to immigrate. For the asylum and refugee policy field, however, the country would be highly restrictive.
National currencies are converted into international dollars using purchasing power parity exchange rates.
This happened in Denmark were requirements were altered from 6 to 24 months between 2007 and 2008.
Adock, R. and Collier, D. (2001) ‘Measurement validity: A shared standard for qualitative and quantitative research’, American Political Science Review 95(3): 529–46.
Andreas, P. (2003) ‘Redrawing the line: Borders and security in the twenty-first century’, International Security 28(2): 78–111.
Bjerre, L., Helbling, M., Römer, F. and Zobel, M. (2015) ‘Conceptualizing and measuring immigration policies. A comparative perspective’, International Migration Review 49(3): 555–600.
Bollen, K.A. (1980) ‘Issues in the comparative measurement of political democracy’, American Sociological Review 45(2): 370–90.
Bollen, K.A. (1986) ‘Political rights and political liberties in nations: An evaluation of human rights measures, 1950 to 1984’, Human Rights Quarterly 8(4): 567–91.
Brochmann, G. (1999) ‘The Mechanisms of Control’, in G. Brochmann and T. Hammar (eds.) Mechanisms of Immigration Control. A Comparative Analysis of European Regulation Policies, Oxford and New York: Berg, pp 1–27.
Brochmann, G. and Hammar, T. (eds.) (1999) Mechanisms of Immigration Control. A Comparative Analysis of European Regulation Policies, Oxford and New York: Berg.
Brubaker, R. (1992) Citizenship and Nationhood in France and Germany, Cambridge, MA: Harvard University Press.
Carmines, E.G. and Zeller, R.A. (1979) Reliability and Validity Assessment, Beverly Hills, CA: Sage.
Cerna, L. (2008) Towards an EU Blue Card? The delegation of National High Skilled Immigration Policies to the EU level, Oxford: ESRC Centre on Migration, Policy and Society, COMPAS working paper 65.
Cholewinski, R. (2002) ‘Family reunification and conditions placed on family members: Dismantling a fundamental human right’, European Journal of Migration and Law 4(3): 271–90.
De Haas, H., Natter, K. and Vezzoli, S. (2014) Compiling and coding migration policies: Insights from the DEMIG POLICY database, Oxford: University of Oxford, IMI Working paper 87.
Doomernik, J. and Jandl, M. (eds.) (2008) Modes of Migration Regulation and Control in Europe, Amsterdam: Amsterdam University Press.
Dreher, S. (2002) ‘Regulation’, in D. Nohlen and R.-O. Schultze (eds.) Lexikon der Politikwissenschaft, München: C.H. Beck, pp. 804–05.
Easton, D. (1965) A Systems Analysis of Political Life, Chicago: University of Chicago Press.
Economist Intelligence Unit. (2008) Global Migration Barometer. Methodology, Results and Findings, 12 September, http://www.un.org/esa/population/meetings/seventhcoord2008/GMB_ExecSumEIU.pdf, accessed 10 October 2012.
Elkins, Z. (2000) ‘Gradations of democracy? Empirical tests of alternative conceptualizations’, American Journal of Political Science 44(2): 293–300.
Fitzgerald, J., Leblang, D. and Teets, J. (2014) ‘Defying the law of gravity: The political economy of international migration’, World Politics 66(3): 406–45.
Freeman, G.P. (1978) ‘Immigrant labor and working-class politics: The French and British experience’, Comparative Politics 11(1): 25–41.
Freeman, G.P. (1979) Immigrant Labor and Racial Conflict in Industrial Societies: The French and British Experience, Princeton: Princeton University Press.
Geddes, A. (2003) The Politics of Migration and Immigration in Europe, London: SAGE.
Gest, J., Boucher, A., Challen, S., Burgoon, B., Thielemann, E., Beine, M., McGovern, P., Crock, M., Rapoport, H. and Hiscox, M. (2014) ‘Measuring and comparing immigration, asylum and naturalization policies across countries: Challenges and solutions’, Global Policy 5(3): 261–74.
Givens, T. and Luedtke, A. (2005) ‘European immigration policies in comparative perspective: Issue salience, partisanship and immigrant rights’, Comparative European Politics 3(1): 1–22.
Groenendijk, K. (2006) ‘The Status of Quasi-Citizenship in EU Member States: Why Some States have ‘Almost Citizens’’, in R. Bauböck, E. Ersbøll, K. Groenendijk and H. Waldrauch (eds.) Acquisition and Loss of Nationality. Policies and Trends in 15 European States, Amsterdam: Amsterdam University Press, pp. 411–29.
Hammar, T. (1985) ‘Immigration Regulation and Alien Control’, in T. Hammar (ed.) European Immigration Policy: A Comparative Study, Cambridge: Cambridge University Press, pp. 249–62.
Hammar, T. (1990) Democracy and the Nation State. Aliens, Denizens and Citizens in a World of International Migration, Aldershot: Avbury.
Hatton, T. (2004) ‘Seeking asylum in Europe’, Economic Policy 19(38): 5–62.
Helbling, M. (2013) ‘Validating integration and citizenship policy indices’, Comparative European Politics 11(5): 555–576.
Helbling, M., Bjerre, L., Römer, F. and Zobel, M. (eds.) 2013) ‘How to measure immigration policies, migration and citizenship, newsletter of the American political science association’, Organized Section on Migration and Citizenship 1(2).
Hollifield, J.F. (1998) ‘Migration, trade and the nation-state: The myth of globalization’, UCLA Journal of International Law and Foreign Affairs 3(2): 595–636.
Hollifield, J.F. and Wong, T.K. (2013) ‘International migration: Cause or consequence of political change?’ Migration and Citizenship: Newsletter of American Political Science Association Organized Section on Migration and Citizenship 1(1): 3–9.
Honohan, I. (2009) ‘Reconsidering the claim to family reunification in migration’, Political Studies 57(4): 768–787.
Howard, M.M. (2009) The Politics of Citizenship in Europe, New York: Cambridge University Press.
Jacoby, W.G. (1999) ‘Levels of measurement and political research: An optimistic view’, American Journal of Political Science 43(1): 271–301.
Jerónimo, P. and Vink, M.P. (2011) ‘Citizenship in a postcolonial context: Comparing portugal and The Netherlands’, Portuguese Journal of Political Science and International Relations 6: 109–29.
Klugman, J. and Medalho Pereira, I. (2009) Assessment of National Migration Policies: An emerging picture on admissions, treatment and enforcement in developing and developed countries, Human Development Research Paper. HDRP-2009-48, Human Development Report Office (HDRO), United Nations Development Programme (UNDP).
Koopmans, R., Michalowski, I. and Waibel, S. (2012) ‘Citizenship rights for immigrants: National political processes and cross-national convergence in Western Europe, 1980–2008’, American Journal of Sociology 117(4): 1202–45.
Koopmans, R., Statham, P., Giugni, M. and Passy, F. (2005) Contested Citizenship. Immigration and Cultural Diversity in Europe, Minneapolis: University of Minnesota Press.
Lenard, P.T. and Straehle, C. (2012) ‘Temporary labour migration, global redistribution, and democratic justice’, Politics, Philosophy & Economics 11(2): 206–30.
Lowell, L.B. (2005) Policies and Regulations for Managing Skilled International Migration for Work, New York: United Nations Expert Group Meeting on International Migration and Development.
Marks, G., Hooghe, L., Steenbergen, M.R. and Bakker, R. (2007) ‘Crossvalidating data on party positioning on European integration’, Electoral Studies 26(1): 23–38.
Mayda, A.M. (2005) International Migration: A Panel Data Analysis of Economic and Non-Economic Determinants, IZA Discussion Papers (1590). Bonn: Institute for the Study of Labor.
Messina, A.M. (2007) The Logics and Politics of Post-WWII Migration to Western Europe, New York: Cambridge University Press.
Meyers, E. (2000) ‘Theories of international immigration policy – A comparative analysis’, International Migration Review 34(4): 1245–82.
Money, J. (1999) Fences and Neighbours. The Political Geography of Immigration Control, Ithaca, NY: Cornell University Press.
MPG. (2006) The Migration Integration Policy Index (MIPEX), Brussels: Migration Policy Group British Council.
Munck, G.L. and Verkuilen, J. (2002) ‘Conceptualizing and measuring democracy: Evaluating alternative indices’, Comparative Political Studies 35(1): 5–34.
Nardo, M., Saisana, M., Saltelli, A., Tarantola, S., Hoffman, A. and Giovannini, E. (2005) Handbook on constructing composite indicators: Methodology and user guide, OECD, available at: http://www.oecd.org/std/42495745.pdf, accessed 21 January 2016.
Niessen, J., Huddleston, T. and Citron, L. (2007) Migrant Integration Policy Index (MIPEX), Brussels: British Council and Migration Policy Group.
Ortega, F. and Peri, G. (2009) ‘The causes and effects of international labor mobility: Evidence from OECD countries 1980–2005’, Human Development Research Paper(6): 1–42.
Oxford Analytica. (2008) Labour Migration Policy Index Phase II, IOM, 1 October 2008.
Peters, M. (2015) ‘Open trade, closed borders: Immigration in the Era of globalization’, World Politics 67(1): 114–54.
Pham, H. and Van, P. (2014) ‘Measuring the Climate for Immigrants: A State-by-State Analysis’, in G.J. Chin and C. Hessick (eds.) The Role of the States in Immigration Enforcement and Policy, New York: New York University Press, pp. 21–39.
Ruhs, M. (2011) Openness, Skills and Rights: An Empirical analysis of labour immigration programmes in 46 high-and middle income countries. Oxford: Centre on Migration, Policy and Society, COMPAS working paper 88.
Ruhs, M. (2013) The Price of Rights: Regulating International Labor Migration, Princeton: Princeton University Press.
Schain, M.A. (2008) The Politics of Immigration in France, Britain, and the United States, New York: Palgrave Macmillan.
Skeldon, R. (2012) ‘Going round in circles: Circular migration, poverty alleviation and marginality’, International Migration 50(3): 43–60.
Stevens, S.S. (1946) ‘On the theory of scales of measurement’, Science 103(2684): 677–80.
Thielemann, E.R. (2003) Does policy matter? On Governments’ Attempts to Control Unwanted Migration. Dublin: Institute for International Integration Studies, Trinity College, IIIS Discussion Papers 9.
Timmer, A.S. and Williams, J.G. (1998) ‘Immigration policy prior to the 1930s: Labor markets, policy interactions and globalization backlash’, Population and Development Review 24(4): 739–71.
Vink, M. P. and Bauböck, R. (2013) ‘Citizenship configurations: Analysing the multiple purposes of citizenship regimes in Europe’, Comparative European Politics 11(5): 621–48.
Weber, M. (1946) Essays in Sociology, New York: Oxford University Press.
First of all, the authors would like to thank all country and field experts who helped them with the conceptualisation of the data set and especially the collection of the data. Without their professional help this project would not have been possible. The authors also would like to thank all commentators at various conferences where they presented their project. The authors were especially grateful for the comments they received at various steps of this project from their colleagues at the WZB Berlin Social Science Center and from international colleagues at the conference on ‘Causes and Consequences of Immigration and Citizenship Policies’ that they organised in Berlin in June 2014. Finally, the authors would like to thank Jonas Kahle, Andrea Pürckhauer, Hannah Schilling, Anne Bohm, Florian Eyert, Maren Hahnen, Dorina Kalkum, Gregory Kerr and Jakob Oxenius for their research assistance.
About this article
Cite this article
helbling, m., bjerre, l., römer, f. et al. measuring immigration policies: the IMPIC database. Eur Polit Sci 16, 79–98 (2017). https://doi.org/10.1057/eps.2016.4