How to combine and analyze all the data from diverse sources: a multilevel analysis of institutional trust in the world

Abstract

Scholars who want to perform cross-national comparative research rely on data provided by International survey projects, which study the same concepts in varying countries and periods using different question wordings and scales. In this article, we propose a process to combine and analyse the data pertaining to the same concept—institutional trust—when measures and sources differ. We show how we combined 1327 surveys conducted from 1995 to 2017 by 17 survey projects in 142 countries. The database comprises close to 2 M respondents and 21 M answers to trust questions. We use local regression to visualize the trends in trust for different institutions and sources of data in different parts of the world. We complete these analyses with a 4-level longitudinal analysis of repeated measures. These analyses lead to reliably conclude that institutional trust is a property of the institutions themselves and of the context in which they operate since there is much more variance within respondents than between respondents and more variance between countries than over time. This research contributes to the current debates in political trust research. Since the process presented here can be applied to other fields of research, the research also contributes to enhance the possibilities for comparative cross-national analysis.

Appendices

Appendix 1: The grouping of institutions

Table 6 presents the groupings of the 133 different institutions in 16 broad categories referring to four dimensions. Political trust comprises six categories, that is, president/state, government, parliament, political parties, elections, and supra-national organizations. Between 28% (elections) and 93% (parliament) of the respondents were asked about their trust in an institution grouped in one of these institutional categories. There are one or two major institutions that account for the larger part of the data in each category. For example, 66% of the respondents in the President/State category were asked specifically about the president and 84% were asked about the Government or Congress in the Government category. Therefore, it is unlikely that the inclusion of any institution that has been less surveyed had an impact on the results. The most heterogenous category groups all the international and regional institutions into a “supra-national institutions” category.

Table 6 Institutional groupings

The second group comprises the institutions related to the public administration. Some of the institutions in this grouping are often grouped with the political institutions in a unidimensional scale. When considered separately, they receive various designations: institutions of implementation (Marien, 2011, 2017), of the state (Mattes and Moreno 2017), regulatory (Catterberg and Moreno 2006; Zavecz 2017) or impartial (van der Meer and Ouattara 2019). This dimension includes four categories. Trust in the police (97% of the respondents), in the judicial system, including institutions that fight corruption (90%) and in the army (75% of respondents) are frequently measured. Public administration itself (48% of the respondents) is a quite heterogenous category. It groups all the institutions of the public service including those of the education and health systems.

The third group comprises the institutions of the civil society (Catterberg and Moreno 2006), also called civic institutions (Zavecz 2017). It includes trust in the media (asked of 68% of the respondents), in the Church or religious leaders (66%), in the trade unions (37%) and in the non-governmental organizations (NGO), 30%. Trust in the Media accounts for 10.2% of the measures and therefore is the most frequent institutional category.

Finally, the fourth group comprises the economic institutions, that is, the financial institutions and the enterprises. At most 37% of the respondents were asked about these institutions.

Some institutional categories comprise few different institutions—the army, for example—while others group many different institutions—supra national organizations, for example. However, our empirical and statistical criteria hold and, although the institutions themselves may be different, they may not be assessed differently by respondents (Bovens and Wille, 2011). For some of these institutions—politicians and political parties for example—research shows high correlations when trust about them are asked from the same respondents (Hooghe, 2011; Marien, 2011; Torcal, 2017). Finally, given the way we set up the data base, it remains possible to drop some of these institutions to check whether the choices that we made biased the results.

Other groupings may be criticized. Why group trust in the Prime Minister with trust in the government and not trust in the President? Different political systems have different roles for the President and the Prime Minister. In some systems, the President represents the state and has a protocolary role while the Prime minister leads the government. Empirically, the average trust in the President is usually higher than trust in the government or the Prime Minister, particularly in countries where the two roles coexist. This led us to group trust in the Prime Minister with trust in the government.

Appendix 2: The grouping of countries

We must first deal with the fact that some regional survey projects include countries of the “Western World”. The Latino Barometro includes Spain; the Americas Barometer (LAPOP) includes Canada and the United States. The European Quality of Life project, the European Social Survey and the European Values Surveys are conducted in Austria, Finland, Germany and Greece, countries that are geographically in the eastern part of Europe. Given our stated goal to concentrate on the countries outside the Western World, we could have dropped these countries. However, if we keep them, we benefit from more variance in the contexts and therefore more possibilities for relevant comparisons with the other countries.

What do these “Western World” countries have in common? All of them received the highest grade (10) on the Polity IV index (Marshall, Gurr and Jaggers, 2019; Center for Systemic Peace 2019) over all the period studied. The Polity IV index has two scales, one of democracy and one of autocracy and an index computed as the difference between the two. Therefore, it varies from minus 10—fully autocratic regime—to plus 10—fully democratic regime. We selected this index because it is more fact-based (Marshall, Gurr and Jaggers, 2019) than other indices that are commonly used. Though other indices produced by Freedom House (Freedom House, 2019), V_DEM (Coppedge et al. 2018) and the Economist Intelligence Unit (Economist Intelligence Unit 2019) are commonly used and have their own specificities, all the indices are highly correlated (Elff and Ziaja, 2018).

Since we have a criterion that groups these “Western World” consolidated countries together, we must check whether other countries in our database meet the same criterion, that is, a Polity IV score of 10 over all the period. Ten countries in four regions meet the criterion: Japan and Mongolia in Asia, Costa Rica, Trinidad and Tobago and Uruguay in South and Central America, Cabo Verde and Mauritius in Sub-Saharan Africa and Hungary, Lithuania and Slovenia in Eastern Europe. We group these countries with the “Western World” countries in a “Consolidated Democracies” category. These countries are not a random selection of the consolidated democracies. The next iteration of data combination will seek to include all the countries of the World.

It is also on historical and political criteria that we group together the “Post-communist countries” that did not qualify as consolidated democracies. Indeed, authors who study trust in European countries generally group together the former socialist countries of Eastern Europe and the former soviet republics (Quaranta and Martini, 2016; Zavecz 2017). Some authors divide these countries in two or more sub-regions (Catterberg and Moreno 2006; Zavecz 2017). While Catterberg and Moreno (2006) found differences between the Eastern European countries and the former soviet republics, Zavecz (2017) did not find any. We decided to group all these countries together (except the three countries that are considered consolidated democracies) to have enough units in the category.

The countries that did not meet the former two criteria are grouped according to a socio-geographical criterion. We use the same divisions as Zmerli and van der Meer (2017). We group the other countries of South and Central America in one group, of West Asia and North Africa (WANA)² including Turkey in a second group, of Sub-Saharan Africa in a third group and of Asia in a fourth group. Consolidated democracies include 17 countries, Post-communist countries, 28, South and Central America, 30, Rest of Asia, 22, Sub-Saharan Africa, 30 and the WANA and Turkey grouping, 16. Table 7 in lists all the countries by grouping.

Table 7 Country Groupings

We are aware that these groupings and the criteria we used are not common and may be criticized. Do they represent cultural, political and economic contexts that are homogenous within regions, or even, in our case, characteristics of survey projects? In terms of our first criterion regarding democracy, Table 8 presents the mean and standard deviation of common indices of Democracy—V_DEM Polyarchy Index of Electoral Democracy (Coppedge et al. 2018), Freedom House Political Rights and Civil Liberties indices (Freedom House, 2019) and the Economist Intelligence Unit Democracy Score (Economist Intelligence Unit, 2019)—used in the literature in order to validate our recourse to the Polity IV index as a first criterion. All the indices have been rescaled to a 0–1 scale for comparison purposes. Table 8 validates our decision to group together the consolidated democracies. Whatever the measure of Democracy used, this group of country stands as rather homogenous and different from the other groups of countries.

Table 8 Mean Indices of Democracy According to Country Groupings