1 Introduction

The question of how the trend of decreasing trust in government of citizens in Western democracies can be reversed, has been a research topic in social, political and public management studies for already more than a decade (Nye et al. 1997). The solution to the confidence crisis, according to Robert Putnam (2000), lies in restoring and utilizing the social capital of the populations concerned. The main hypothesis of Bowling Alone has been summarized as “measurable declines in group activities cause bad outcomes” (Sobel 2002). The local level of government is receiving increasing esteem in contemplations on trust in multilevel democratic government systems. Communities can solve problems that otherwise appear as classic market or state failures, because their members have crucial information about other members’ behaviors, capacities, and needs (Bowles and Gintis 2002: 422). Social capital serves as the adhesive which links members within a group and as a bridge which links several groups with each other (Lang and Hornburg 1998). Governments play a vital role in determining the degree of influence that organized interests, and individual citizens, have upon democratic functioning and performance (Lowndes and Wilson 2001: 631). Local governance affects the distribution of social capital, because the city’s ‘political opportunity structure’ provides incentives, expectations and openings for people to undertake collective action within the political system (Kearns 2004: 11).

However, social capital is surrounded with much controversy. The list of discords is almost endless. Social capital is supposed to have both positive (constructive) and negative (destructive) impact (Portes 1998). It is considered at the level of individuals as a condition for institutional trust, where at the same time public institutions are supposed to make the citizens more trusting (see Foley and Edwards 1999, for a study on the use of social capital as dependent or independent variable in scientific literature). Social capital is supposed to have beneficial impact both within the American capitalist and within the European social democratic state models (Navarro 2002). It may serve as the means to have power concerning ones own destiny, and to exercise power over others (see Smith and Kulynych 2002: 158). For some, social capital can only play a role in the distribution of existing prosperity (zero sum game), whereas others consider social capital vital for the creation of new prosperity (Woolcock 1998). Social capital produces positive outcomes, such as economic development and less crime, and at the same time presence of social capital is inferred from these outcomes (Portes 1998: 19). The eminent circularity in the reasoning, that social capital is both cause and effect of good governance, produces tautologies. Durlauf (2002: 474) argues that

The empirical social capital literature is plagued by vague definitions of concepts, poorly measured data, absence of exchangeability conditions, and lack of information necessary to make identification claims in confirmatory statistical analysis plausible.

Another important problem, and the focus of this article, concerns the incompatibility of conclusions reached at aggregate (e.g., municipalities, regions, states) and micro level (e.g., citizens). Regardless of the tautologies in Putnam’s reasoning, his conclusions on the effects of social capital at aggregate level (the southern and northern regions of Italy) are not replicated in studies at micro level (e.g., Pennants 2005 study on diversity, trust and community participation in England) or international level (e.g., Knack and Keefer’s 1997 study on memberships in formal groups and economic performance; Bjornskov’s 2006 cross-national comparison of social capital indicators). This problem, also known within the context of the “ecological inference problem” (e.g., King 1997), places doubts on the cross level consistency of measures of social capital.

In this paper, we try to shed some light on the issue of operationalization of fuzzy concepts by comparing individual, citizen or survey based indicators, with municipal or aggregate level based indicators of social capital and local government performance. With knowledge about the behavior of indicators of social capital and local government performance at micro and aggregate level, we are better equipped to judge the value of studies carried out in isolated contexts, that is, with access to data on either, but not both, levels. The generic conclusions on cross level consistency of the indicators of the study below are probably generalizable to situations where e.g., the aggregate level corresponds to nation states and the micro level to regions or municipalities. Of course, additional research is necessary in order to establish (lack of) cross level consistency of the indicators at these levels. The source of the data and the method of analysis are discussed in the next section. We than elaborate on the specific measures that are to be compared for consistency, were we recognize two distinct groups of variables: indicators of social capital, and indicators of local government performance. Cross level consistency of the analyzed indicators is presented thereafter, followed by a discussion of the results in the final section.

2 Data and Method

2.1 Sources for Micro Level and Aggregate Level Data

Both the micro level and the aggregate level data deal exclusively with the Flemish communities, that is, those within the Dutch speaking region of Belgium. As for the aggregate level data, we rely on a dataset collected by Coffé and Geys (2005), who used a dataset with indicators of social capital, neighborhood quality and local government performance at local level. The major source of this data is Statistics Belgium, one of the many organizations in Belgium responsible for the nation’s official statistics. A second source of aggregate data are the local electoral results at municipal level. Local elections have been held in Belgium in 2000 and 2006.

The municipalities are coded by a five digit coding system, the so-called NIS-code; e.g., Antwerp (that is, downtown Antwerp) is coded 11001. This variable is used to connect the aggregate level data to the micro level (survey) data. The latter consist of a survey of 1,250 randomly selected Flemish citizens (with age between 18 and 80 years) nested within a random sample of 80 Flemish municipalities. The survey data was collected in 2002 in survey research issued by the Catholic University of Leuven commissioned by the ministry of the Flemish Community (see e.g., Kampen 2007). The study was called WADO, which stands for “Werken aan de Overheid”, in English “Working on the Government”. The questionnaire aimed to investigate the relationship of public service quality and trust in government. Thus, the survey provides a variety of items dealing with social capital and local government performance, and that can be used to test the consistency with the indicators measured at aggregate level. To this end, the datasets are merged using the NIS-code as keyed variable. Statistics Belgium also applies a two digit code that classifies municipalities according to population density and cultivation. This variable clusters the municipalities into nine classes ranging from urban to rural taking intermediate values in between. We use this variable in order to explain differences between municipalities.

2.2 Indicators of Social Capital

The respondents were all asked with whom they discussed their personal problems during the past 6 months. They were presented a list of possible persons and were required to answer with a simple ‘yes’, ‘no’ or ‘not applicable’. Of course, the data have to be interpreted while accounting for the fact not all people have e.g., a partner, or colleges, etc.; thus, we consider with each statistic, the parameter f denoting the sample fraction where a response was applicable. On the top of the list, probably not surprisingly, one finds the partner or spouse (93.3% responded ‘yes’, f = 77.6), followed by children (73.7%, f = 67.9), friends (69.7%, f = 97.8) and parents (69.4%, f = 65.3). Brothers and sisters (55.1%, f = 86.0) and colleges (49.4%, f = 73.5) take a middle position, whereas the wider family (24.1%, f = 94.2) and neighbors (18.2, f = 96.6) only play modest roles. The total number of contacts people dispose over to discuss their personal problems may be denoted

$$ F_{i} = \sum\limits_{p = 1}^{P} {f_{ip} } , $$

where f ip  ϵ {0, 1} denotes for respondent i, the feasibility to talk to the pth item in the list of P possible persons. The scope of an individual’s network will be defined

$$ S_{i} = {\frac{{F_{i} }}{P}}, $$

which is the fraction of potential channels that a respondent has at his disposal. If we accept that the number of channels people in effect use to discuss their personal problems serves as an indicator of the degree of social isolation, than we may construct the index

$$ T_{i} = {\frac{{\sum_{p = 1}^{P} {X_{ip} f_{ip} } }}{{F_{i} }}}, $$

with X ip  ϵ {0, 1} denoting the value for the pth item in the list of P possible persons to talk toFootnote 1: ‘no’ = 0 and ‘yes’ = 1. We refer to T i as the interaction index: the higher the index, the more a respondent takes advantage of the available channels to break through his or her potential isolation.

Respondents were also given a list of associations and clubs, and were asked whether at present or in the past, they were a member of the given group. In addition, if at present they were a member, they could report whether they were passive or active members, and if active, whether they were committee member or not. This differentiation is relevant in the context of social capital studies (e.g., Beugelsdijk and van Schaik 2005: 311–318), but due to data sparseness, we shall look only into the categories of ‘present members’, ‘former members’, and ‘never members’. Accordingly, for all individuals, two membership indices are constructed. The first simply counts the number of associations that an individual is at present a member of, divided by the number of possible associations (membership index). The second divides the number of associations that an individual is or has been a member by the number of possible associations (historic membership index). At present, the greatest number of people is member of a sports club or a labor union (26%). The third place is taken by socio-cultural associations (22%), followed by cultural associations (15%) such as reading clubs, charity (10%), associations to improve the environment (7%) such as Greenpeace, and political associations (6%). Less popular memberships include youth organizations (5%), religious associations (5%), neighborhood organizations (4%), and last but not least, advisory boards (4%). Historic membership is highest for sports clubs (49% of all respondents at one point in their lives had been a member), followed by youth organizations (46%), and labor unions in the third place (34%).

A visualization of these memberships in terms of a circle diagram (see Scott 2007: 147) is displayed in Fig. 1. Lines in the circle diagram are drawn if at least 5% of the total sample at one point in their lives had been a member of the pair of associations, with bolder lines corresponding to higher joint historic memberships, bearing to a maximum of 26.8% who in their lives have been a member of a sports club and (simultaneously or not) of a youth organization. The most likely social network appears to be located within the pentagon of the sports club, youth association, labor union, socio-cultural, and cultural organizations.

Fig. 1
figure 1

Joint historical memberships in Flanders (2002)

Full size image

Coffé and Geys (2005) chose the municipal crime level, electoral turn-out and the number of associations per capita as indicators of social capital at aggregate level. Associational life was operationalized as the number of organizations per capita in a municipality, where the organizations mainly included sports clubs and socio-cultural organizations, such as associations of retired people, women’s associations, associations of parents, and the like. Electoral turn-out was computed as the number of valid plus invalid votes casted at the 2000 local elections, divided by the number of registered voters. The crime level, which served as a measure of generalized trust, was defined as the number of crimes per capita as reported by the local police. However, although the choice to operationalize crime level in terms of number of crimes per capita appeals to academia’s norms of objective quantification, local newspapers in general do not take into account that larger communities produce more crime in absolute figures. That is to say, the space in newspapers (and other media) assigned to crimes does not correspond to the percentage of people exposed to criminal facts. As pointed out by Glassner (1999), who found that fear of homicide increased in the USA in the nineties, not because homicide figures were rising (in fact, they dropped) but because media coverage increased (by several hundreds of percents), it is not fact but perception that governs the responses of the general public. Therefore, the raw number of crimes in a municipality must also be considered to be a potentially valid indicator of (lack of) social capital and interpersonal trust.

2.3 Indicators of Local Government Performance

The WADO survey includes three items measured at five point ordinal scales that deal with opinions on the performance of local government. They consist of two questions about the local administration, and one about the Mayor:

  1. (1)

    To what extent are you satisfied with the performance of the local administration? (very little–very much)

  2. (2)

    To what extent do you trust the local administration? (very little–very much)

  3. (3)

    How do you evaluate the image of the Mayor? (very negative–very positive)

The ordinal scale of measurement complicates the making of cross level comparisons with the other indicators, because the computing of means at municipal level yields ill-defined indicator variables (see e.g., Kampen and Swyngedouw 2000). This problem is solved when the three items are dichotomized by separating the first two categories from the other three. Such a dichotomous variable has a ratio scale, and its mean at the level of a municipality simply represents the proportion of respondents within the municipality that are satisfied, trusting and have a positive image, respectively.

Aggregate government performance was defined by Coffé and Geys (2005) by computing the relative budget deficit as the difference of total local government revenues plus total expenditures (including investments) divided by the total revenues. As in the case of crime level, news reporters may not be sensitive to nuances about the relative level of deficit, and therefore, the absolute budget deficit must also be considered as an indicator of local government approval. Performance operationalized in these financial terms is influenced by the ideological orientation of the parties in the local coalitions, because leftist governments are often thought to be supporters of big governments and therefore, of larger government spending and possibly, larger deficits. Coffé and Geys (2005), following Volkerink and De Haan (2001), operationalized ideological fragmentation as the standard deviation of the ideological position of the coalition partners. A socio-economic variable at aggregate level called cost efficiency, finally, was operationalized by Coffé and Geys as a scale composed from the total of students in primary schools, the total surface of recreational facilities (e.g., parks), the population size, and the fraction of elderly people. The idea is that cost efficient municipalities are able to do more with their revenues, which in turn serves as an indicator of local government performance.

The 2000 and 2006 electoral results are used to construct a variable called coalition support. Belgium’s fragmented political landscape usually leads to the building of coalitions at the local level consisting of three or more different parties. Coalition support is a dichotomous variable taking the value 1 if the 2000 coalition was re-installed after the 2006 local elections, and coalition support equals zero otherwise. Coalition support will serve as a proxy for the level of trust in government at aggregate level.

2.4 Statistical Methodology

Generally, in the case of two measures of the same phenomenon, we would expect these measures to show correlation when they are measured at metric scale, stochastic dependence in the case of a categorical measurement scale, and between-variance if one measure is metric and the other categorical. However, interpreting even a well-know statistic as the correlation of an indicator at micro and aggregate level is not as straightforward as it may seem at first glance. Cross-level analysis is deceptively difficult (Achen and Shively 1995: 1). At individual level, some indicator A for individual i nested within municipality j can be decomposed as

$$ V_{ij}^{A} = \mu^{A} + \delta_{j}^{A} + \varepsilon_{ij}^{A} , $$

with \( V_{ij}^{A} \) the observed value of the indicator, \( \mu^{A} \) the overall mean, \( \delta_{j}^{A} \) the deviation of the overall mean within municipality j, and \( \varepsilon_{ij}^{A} \) the deviation of the mean in the jth municipality for individual j, usually called error term. Consider the aggregate level indicator A′

$$ V_{ij}^{A'} = \mu^{A} + \delta_{j}^{A} , $$

which in this example is equivalent to the micro level indicator \( V_{ij}^{A} \) minus the micro level deviation term \( \varepsilon_{ij}^{A} \); in other words, for each individual the indicator A′ simply equals the mean of the indicator A within his/her municipality. Upon abbreviating \( \text{var} \left( {\delta_{j}^{A} } \right) = \omega_{A}^{2} \) and \( \text{var} \left( {\varepsilon_{ij}^{A} } \right) = \sigma_{A}^{2} , \) it is easy to see that the correlation of the micro en aggregate level indicator,

$$ r^{AA'} = {\frac{{{\text{cov} }\left( {V_{ij}^{A} ,V_{ij}^{A'} } \right)}}{{\sqrt {\text{var} \left( {V_{ij}^{A} } \right)\text{var} \left( {V_{ij}^{A'} } \right)} }}} = {\frac{{\omega_{A}^{2} }}{{\sqrt {\left( {\omega_{A}^{2} + \sigma_{A}^{2} } \right)\omega_{A}^{2} } }}}, $$

satisfies r AA′ = 1 if and only if the micro level variance \( \sigma_{A}^{2} \) = 0 (which corresponds to the situation that all individual deviations \( \varepsilon_{ij}^{A} \) equal zero). In other occasions, the individual variance component \( \sigma_{A}^{2} \) has a diminishing effect on the correlation, so that r AA′ < 1. Thus, the seemingly identical indicators A and A′ will not show perfect correlations in empirical settings, even though these indicators would satisfy straightforward requirements of cross level indicator consistency. Moreover, a lack of significance cannot be taken as evidence that the indicators are not associated, because the significance is inversely related to the variance of the micro level error term.

The obvious remedy to this problem is to look only into the behavior of the indicators at aggregate level, but this leads to a major complication regarding the interpretation of the resulting measures of association. For instance, suppose that at aggregate level, higher numbers of associations per capita correlate to higher budget surpluses in municipalities (a finding reported by Coffé and Geys 2005: 495). We may then ask, to what extent are the individual members of associations “responsible” for the higher surpluses? This is an instance of the ecological inference problem, and the aggregate level association is referred to as an ecological correlation (Freedman 1999), as opposed to the “real” correlation which would be measured at micro level.

The only way to obtain certainty about the relationship membership-number of associations-budget surplus, requires access to data at both the individual and the aggregate level. Separate association measures for the two levels can then be computed by applying random effects models (e.g., Bryk and Raudenbush 1992; Snijders and Bosker 1999; Berkhof and Kampen 2004). These models are a generalization of the well known linear regression model. Consider a micro level indicator B,

$$ V_{ij}^{B} = \mu^{B} + \delta_{j}^{B} + \varepsilon_{ij}^{B} . $$

Denote the variance-covariance terms of the indicators by \( \text{var} \left( {\delta_{j}^{B} } \right) = \omega_{B}^{2} , \) \( \text{var} \left( {\varepsilon_{ij}^{B} } \right) = \sigma_{B}^{2} , \) \( \text{cov} \left( {\delta_{j}^{A} ,\delta_{j}^{B} } \right) = \omega_{AB} \) and \( \text{cov} \left( {\varepsilon_{ij}^{A} ,\varepsilon_{ij}^{B} } \right) = \sigma_{AB} . \) The correlation coefficient of indicators A and B is given by the ratio of their covariance and the square root of the product of their respective variances,

$$ r^{AB} = {\frac{{{\text{cov}} \left( {V_{ij}^{A} ,V_{ij}^{B} } \right)}}{{\sqrt {\text{var} \left( {V_{ij}^{A} } \right)\text{var} \left( {V_{ij}^{B} } \right)} }}} = {\frac{{\omega_{AB} + \sigma_{AB} }}{{\sqrt {\left( {\omega_{A}^{2} + \sigma_{A}^{2} } \right)\left( {\omega_{B}^{2} + \sigma_{B}^{2} } \right)} }}}. $$

Upon estimation of each of the variance components, we are able to give an expression of the between municipality correlation and the within municipality correlation coefficients

$$ r_{b}^{AB} = {\frac{{\omega_{AB} }}{{\sqrt {\omega_{A}^{2} \omega_{B}^{2} } }}},\quad r_{w}^{AB} = {\frac{{\sigma_{AB} }}{{\sqrt {\sigma_{A}^{2} \sigma_{B}^{2} } }}}. $$

The between municipality correlation coefficient \( r_{b}^{AB} \) is the estimated correlation of indicators A and B that we would obtain if we had measures at aggregate level only. In other words, it is the ecological correlation. The within correlation coefficient \( r_{w}^{AB} \) is the correlation coefficient that we would obtain on average if we had measures at micro level and within a single randomly selected municipality only. Note that if at least one of the two involved indicators is measured at aggregate level, then all terms including individual level variance-covariance terms vanish from the equations and \( r_{w}^{AB} = 0 \). In the case that we substitute A′ for B, it is easy to see that \( r_{b}^{AA'} = 1, \) and thus, the between correlation coefficient satisfies our requirement that it should show consistency.

Suppose that two indicators A and B are some micro level measure of “social capital”. In that case, if \( r_{b}^{AB} = 0 \), than the two measures of social capital are independent at the municipal level and therefore, inconsistent at aggregate level. If the within coefficient \( r_{w}^{AB} = 0 \), than the measure is inconsistent at micro level. Of course, consistency across measures increases with increasing correlations. In isolated studies, depending on analysis exclusively at micro or aggregate level, the inconsistent indicators will be deemed insignificant and produce a publication bias.

3 The Cross Level Consistency of the Indicators

3.1 A Short Note on the External Validity of the Indicators of Social Capital

Not all of the micro level indicators of social capital discussed in Sect. 2.2 have been used in this precise form in previous empirical sociological research. Therefore, it is worthwhile to assess the external validity of these indicators. The micro level indicators of social capital confirm certain stereotypes related to gender. The t-test for equality of means detects significant differences between males and females for both the interaction index (T) and the historic membership index. Males have on average, more often been a member of some association (p = .001), whereas females tend to have a higher score on the interaction index (p = .003). The scope of possible persons to talk to (S), and the membership index do not differ between the sexes.

A series of Pearson chi-square independence statistics (not printed) render the precise differences between the sexes as follows. Reflecting traditional family organization patterns, males have on average more often colleges than females (p = .003), but they are equally (un)likely to discuss personal problems with their colleges (p = .500). On the other hand, men are slightly more likely than women to discuss their personal problems with their children (p = .003). Men are on average more often member of a sports club (p < .001) and a labor union (p < .001) whereas women are more often member of socio-cultural association (p < .001), most likely associations specifically organized for women. These differences balance in the (present) membership index. Historically, however, and in addition to the differences in present membership, men have more often than women been a member of a political association (p < .001) or an advisory board (p = .005).

The interaction index and the membership index were constructed independently, yet display a correlation of r = .17 (p < .001), indicating that people are more likely to discuss their personal problems with other people when they engage in a group. When we look into the indicator variables that were used to construct the membership index, we find that the indicators ‘cultural association’, ‘socio-cultural association’, ‘advisory board’, ‘charity’, and ‘other’ correlate weakly, but significantly, with the interaction index. Interestingly, the indicator ‘sports club’ correlates significantly with the scope of contacts, but not with the interaction index, suggesting that sporting is a way to meet people but not to interact with them on a more personal level. This, together with the earlier finding that sports clubs have the highest incidence of historic membership, confirms the recent finding by Seippel (2008), who found that sports organizations are influential due to size, but relatively weakly embedded and positioned in civil society. In sum, the selection of social capital indicators shows external validity with well-known sociological facts.

3.2 The Consistency of Indicators of Social Capital

In Table 1, within, between and overall correlations of the indicators of social capital at micro level are displayed. In all, the micro level indicators of social capital can be divided into a group of variables that display predictable, and a set that displays less predictable, behavior. Among the predictable relationships are the positive association of historic membership and (present) membership, the impact of (historic) membership on the number and scope of contacts, and the impact of the number of contacts and the interaction index. Less appealing at first sight, is the negative association between the scope of contacts and the interaction index; however, if we bear in mind that the interaction index is concerned with personal problems, it is evident that one cannot discuss such problems with anybody. Thus, the micro level indicators of social capital appear to be consistent.

Table 1 Within, between and overall correlations of micro level social capital indicators
Full size table

At aggregate level, see Table 2, counter intuitive correlations appear amongst the number of associations in a municipality on the one hand, and the crime level indicators (per capita and absolute) on the other hand. Whereas the correlation between absolute numbers display nothing but a spurious correlation (third variable problem caused by the size of the municipality), the fact that the absolute number of associations also increases with the crime level per capita can only be explained from the apparent fact that bigger cities have more crime. These findings are confirmed when we inspect partial correlations of the involved indicators, controlled for the level of urbanization (see Sect. 2.1). The correlation (not printed) between absolute and relative crime levels vanishes when controlled for this confounding variable. Similar reasoning holds for the correlation of crime per capita and number of associations. The other aggregate level indicators display consistent behavior.

Table 2 Between correlations of aggregate level social capital indicators
Full size table

Cross level correlations of the social capital indicators are weak if significant at all (nominal α = .01); see Table 3. At the individual level, all correlations are insignificant, thus illustrating the diminishing effect of the micro level variance explained in Sect. 2.3. But even at aggregate level, indicators of social capital show inconsistent behavior. Particularly the absence of a significant correlation between membership and the number of associations (absolute or relative i.e., per capita) places doubts on the usefulness of these indicators for operationalizing social capital. The weak relationships between indicators become evident when we look at the explained variance or multiple R² values that are obtained by predicting each of the indicators separately from the other level’s indicators by multiple regression analysis. These values range from 1 (crime level per capita explained by scope, contacts, interaction index, and the membership indices) to 10% (turn-out at the 2000 local elections explain by the same set of indicators). The shared component of these indicators, which we assumed to be social capital, can hardly be distinguished from ordinary noise. Even when we take into account that measurement error of micro level indicators transformed into aggregate level indicators (the means within municipalities) is higher than measurement error of variables measured at municipal level, the available evidence supplies at best a meager basis for presupposing a shared component, let alone that that component can be interpreted as social capital.

Table 3 Cross level correlations of social capital indicators
Full size table

3.3 The Consistency of Indicators of Local Government Performance

The correlations of indicators of local government performance measured at micro level are displayed in Table 4. The correlations of these subjective indicators are not overwhelmingly high (in the order of .40 or 16% mutually explained variance), but they show a consistent pattern of positive association. At aggregate level (Table 5), consistency is less evident. Intuitively appealing is that higher relative budget deficit corresponds to higher budget deficit and lower efficiency. Ideologically fragmentized councils are on the one hand, more efficient, but on the other hand, they have less relative budget deficit and less absolute budget deficit, implying that these local governments are more inclined to play a zero sum game in local spending.

Table 4 Within, between and overall correlations of micro level performance indicators
Full size table
Table 5 Between correlations of aggregate level performance indicators
Full size table

Coalition support decreases with increasing relative and absolute budget deficits, it increases with increasing ideological fragmentation and there is no relationship with cost efficiency. Taking cross level correlations into account (Table 6), the keeping of a financial balance is not rewarded in terms of satisfaction, trust, and image, where cross level correlations are not significant. Financial output measures are more important indicators of satisfaction, trust and image than ideological fragmentation and efficiency.

Table 6 Cross level correlations of government performance indicators
Full size table

When assessing the cross level correlations between the indicators on micro and aggregated level, we are surprised by the negative association between coalition support on the one hand and the trust of the local administration and the image of the Mayor on the other side. It concerns an ecological correlation, and from the fact that the total correlation is not significant we can infer that we cannot generalize the association to the level of the citizens. From further analysis it became clear that the link does not disappear when we correct for the size and population density in the municipality, or for the ideological fragmentation of the coalition. Even if we take into account the fact that the link is weak (the explained variance of coalition support as predicted from the confidence in the local government and the image of the Mayor is 3%), the association is counter-intuitive and cannot be explained by the existing literature on good governance.

On the whole, cross level consistency of indicators of local government performance are somewhat higher than in the case of social capital, still reaching only moderate levels, 16% being the maximum amount of variance that one set of indicators can explain of another indicator in multivariate regression analyses.

3.4 Cross Level Cross Dimension Consistency of the Indicators of Social Capital

Because their causal link has been suggested in the literature (see Sect. 1), a short account of the empirical findings on associations of governance and social capital indicators is in order, even when we are unable to elaborate on the theoretical implications of the findings. The causal relationship between social capital and local government performance (regardless of direction), could reveal itself as significant correlations of their respective indicators. Of course, a significant correlation is not a sufficient condition to imply causality, but it would provide a first line of evidence that a causal relationship exists. Table 7 displays the correlations across dimensions for the indicators at micro level. The results are disappointing. Within and overall correlations are without exception insignificant, meaning that at the most detailed level a causal chain cannot be inferred. At the aggregate level, the image of the Mayor increases significantly, but hardly relevantly, with increasing levels of social capital indicators. Multivariate and at aggregate level, the four indicators jointly explain 2% of the total variance of the image of the Mayor. The impact of micro level social capital indicators on micro level local government performance indicators also approaches zero.

Table 7 Cross dimension cross level correlations of indicators
Full size table

Finally, when we look at all indicators of social capital taken together as predictors of subjective indicators of local government performance, it is only satisfaction with local administration which is worthwhile to discuss, with 20% of its variance at aggregate level explained. The social capital indicators measured at municipal level account for the larger part of its explained variance. Interestingly, absolute numbers of crime and associations are better predictors than their relative (per capita) counterparts. More crime means less satisfaction with the local administration. The negative correlation of the absolute number of associations in a municipality and satisfaction is counter intuitive, but has the usual explanation in terms of the third variable problem that we encountered earlier (Sect. 3.2): the partial correlation of the absolute number of associations and satisfaction equals zero when controlled for urbanisation. In other words, evidence in this data of a causal relationship between social capital and local government performance is weak, and in many instances, cannot be separated from the effect of demographic characteristics of the involved municipalities.

4 Discussion

The consistency of the indicators cross levels of measurement as defined by their mutual correlations is low. The lack of correlation of membership in associations on the one hand, and the number of associations (per capita) in a municipality on the other, defeats all aspiration to operationalize social capital in a simple way. But also more advanced approaches, where in Bjornskov’s (2006) words, “multiple facets of social capital” are brought together into a single multivariate model, produces only poor fit statistics in terms of explained variance, taking values between 2 and 20%. The shared component of the social capital indicators can hardly be distinguished from ordinary noise. It is perhaps useful to remember at this point, that it is good scientific practice following William of Ockham, not to multiply the number of truths unnecessarily. The economy principle means that when we use membership of associations as an indicator of e.g., local government performance, we must continue to speak of the (alleged) impact of membership and not of social capital. This principle is concisely presented by Smith and Kulynych (2002):

What Putnam is really advocating is healthy people, economic prosperity, and genuine democracy. But social capital, broadly defined, only sometimes contributes to these wider goods. So we are left wondering, why not simply advocate health, wealth, and democracy rather than social capital?

For both policy makers and social scientists alike such tangible measures provide a much more solid basis for theorizing than diffuse concepts such as social capital, which is too coarse a notion to explain trends or even isolated events in real life. Beugelsdijk (2008) argues that a distinction needs to be made between issues of measurement and issues of theoretical relations, an argument which we can only fully support on the basis of the present empirical assessment of consistency of measures of social capital.

Of course, the way in which consistency was defined (correlations between measures) has its own limitations. The possibility that there is a causal link between two indicators cannot be dismissed entirely when a zero correlation is observed, because there may be two causal circuits at work. For instance, higher levels of social capital may have positive influence on trust in local government, but social capital may simultaneously increase expectations, leading to a negative impact on trust. The two effects taken together may lead to an observed correlation equal to zero. Such complicated relationships cannot be addressed while taking only simple correlations into account, and future research that accounts for expectations is therefore necessary in order to satisfactorily answer questions on causality.

We shall elaborate on two other conclusions from our empirical research, because they may serve as guidelines for future research on social capital and related issues. As we already said (see Sect. 1), social capital links members within a group and links several groups with each other. It deals in other words, with social networks, and the obvious research methodology for conducting social scientific research on social capital would appear to be social network analysis. In social network analysis (e.g., Scott 2007; Carrington et al. 2005), groups are identified as cliques, social circles and structurally cohesive blocks. The data show evidence that the municipal level appears to be a good unit of analysis of issues regarding social capital (and related concepts). A municipality acts like a self-contained universe with subjects that display, to a certain degree, predictable behavior, as shown by the significant values of the between correlations. The universe is self-contained, because there is evidence that citizens do not look over the “borders” of their own municipality: absolute figures (crime, spending deficit) have greater impact on subjective government performance indicators than relative numbers. The behavior is predictable, because the signs of the correlations have theoretically justifiable directions. A problem that we encountered sometimes is that apparent (causal) relationships are the effect of a confounding variable such as the level of urbanization, but that only means that comparable to micro level analyses, correlation research at the aggregate level must take such demographic variables into account in order to avoid spurious correlations. The theory on social network analysis can be applied with municipalities (or regions, or countries) as the basic unit of analysis, where distances between municipalities can be expressed in terms of the number of exchanged commuters, length of mutual borders, mutual engagement in public–private partnerships, etc. Such analyses on social capital issues would also provide information on the question whether regions (as used in the original Putman contribution, and by e.g., Beugelsdijk and van Schaik 2005), municipalities, or neighborhoods are the appropriate aggregate level of analysis. At the level of citizens, promising work in the area of social network analysis has been carried out by e.g., Van der Gaag and Snijders (2003, 2005).

Another line of research, however, extends to the micro level of analysis and focuses more deeply on the ecological inference problem. Although generally in our empirical study of social capital and governance indicators, the within correlations are of the same magnitude as the corresponding between correlations, there are instances when differences between these association measures are quite large and require further investigation. For instance, the between correlation of the interaction index and the scope of contacts (r b  = −.50; see Table 1) is more than double the size of the within correlation (r w  = −.20), and the same holds for the items concerning the Mayor’s image and trust in local government (r b  = .42, r w  = .22; see Table 4). In other words, the ecological correlation in these cases is considerably higher than the real correlation (see Sect. 2.4; note that empirical within and overall correlations are always of similar magnitude, justifying the use of the word “real”). There is no obvious explanation of this result but it certainly merits the effort of future empirical research. In fact, elaborating on the ecological inference problem may be the key to find explanations for the inconsistency of indicators for social capital, government performance, and comparable hybrid concepts, and the incompatibility of conclusions reached at aggregate and micro level. This is no simple task. The problems of confounding and aggregation bias, says Freedman (1999), are unlikely to be resolved in the proximate future. It is however, a challenge that must be met if the theory on social capital is to produce fruitful results.