1 Introduction

The origin of CARPE (Church Attendance and Religious change Pooled European dataset)Footnote 1 lies in a research project started few years ago and devoted to religious voting in Europe. In that context, those of us involved in the project were asked to provide a reliable description of the change of individual religiosity in Italy over time. The basic idea of harmonizing different studies (the EVS, WVS, ESS, etc.) came from a paper on church attendance trend in Italy (Pisati 2000). Based on this contribution, we decided to go further in the harmonization process in order to include as many datasets as possible. The first results of this work on the Italian case have been published in Vezzoni and Biolcati-Rinaldi (2015). Analysing the harmonized data, the authors depict the peculiar trend of religious change in Italy, with church attendance decreasing at different paces (including a period of stability in the 1980s) during the period between 1968 and 2010.

Since the beginning of research on this subject, it has been clear how fruitful such an approach incorporating different survey programmes can be. It offers the opportunity to tackle multiple research questions: describing trends of religious change, examining subgroup variation, disentangling period and cohort effects, and providing aggregate information for research projects in which religiosity plays the role of contextual variable. In the context of a harmonized dataset, researchers may increase the reliability of the estimates. They may deal with the issue of total survey error from a variety of methodological perspectives: wording, administration modes, sampling, or weighting. Finally, it opens the opportunity for collaboration with researchers willing to replicate the same strategy in different contexts, or with an interest in the theoretical and methodological issues mentioned above.

Given these considerations, we proceeded to extend the Italian dataset into a comparative (European) dimension, with the intended scope to have the widest possible coverage of European countries for the past 40 years. The result was the CARPE project, to which readers will be introduced in this paper. Section 2 of this article explains the purpose and research strategy adopted for the project. Section 3 provides technical details concerning the selection of studies and harmonization procedures used. A reliability analysis of the CARPE estimates is performed in Sect. 4, while the following section illustrates possible research applications, with respect to both the individual and aggregate versions of the dataset. Finally, some limitations of the overall project are discussed. “Appendix 1” presents the main characteristics of the aggregate version of the CARPE dataset which is available through the Data Archive for the Social Sciences of the GESIS—Leibniz Institute for the Social Sciences based in Cologne (Biolcati et al. 2020).

2 Purpose and research strategy

The CARPE project mainly aims to inform the debate about the substance of religious changes that occurred in Europe over the past decades. This being done by using data from a multiplicity of sources with sometimes more, sometimes less differing approaches to measuring religiosity, a major challenge lies in granting the equivalence of these measures across the various data sources. The variability in extant operationalizations partly stems from different theoretical approaches to the concept of religiosity, and partly from purely methodological choices. The former source of variability—that of different meanings attributed to the data—falls in line with what the sociology of religion knows as Glock’s (1959) paradox. Glock highlighted how the different interpretations of religious change in the United States might derive from researchers considering different dimensions and indicators of religiosity. A possible answer to Glock’s paradox consists in a thorough revision of concepts and measures: the concept of religiosity could be redefined, dimensions reshaped, and new indicators be developed. But even with some existing indicators already being good measures for different aspects of what a theoretical revision might yield, this approach could only be fully executed when designing new data collections. Covering all desirable dimensions in a backward-looking way is impossible, as in the existing surveys, there are always gaps in regional or temporal coverage for a number of those dimensions.

The second, methodological, source of variability stems from different survey practices being followed across countries, fieldwork organisations, and points in time, be those related to ad hoc decisions or to conscious attempts at refining (and thus changing) measurement practice. This introduces variation even into what might otherwise count as measures of a single dimension of religiosity, e.g. when church attendance is measured with a six-point scale in one survey and with a seven-point scale in another one, and when response category wordings vary across surveys etc. This requires often difficult decisions about ex post-harmonization, on top of the harmonization problems that naturally arise from the cross-country comparative setting (Wolf et al. 2016).

Knowing that some of this methodological variation, e.g. in relation to sampling methods or non-response treatment, is often not revealed by the available documentation (Kołczyńska and Schoene 2019) makes this issue alone enough of a practical challenge for any claims to a valid harmonisation effort. We therefore decided to focus only on a single indicator of religiosity, on church attendance. Among all candidate indicators, church attendance luckily by and in itself combines the greatest coverage across survey programmes and over time with the smallest opportunities for conceptual variation. Thus we arrive at a large number of surveys of a narrowly defined measure of religiosity. Each survey represents a measurement occasion that can be pooled in a time series (Vezzoni and Biolcati-Rinaldi 2015). This enables us to analyse the consistency of the observed results with much better statistical power, by minimizing the impact of random fluctuations and by possibly detecting and controlling systematic methodological variation, so that the degree of confidence in the final estimates does increase (Firebaugh 2008, pp. 90–119; Kwak and Slomczynski 2019).

Also in terms of concept and measurement practice, focusing on the indicator of church attendance is a natural choice. To begin with, church attendance taps into religious practice, which is admittedly only one aspect of religious change but one that is particularly effective for measuring the ritualistic dimension of individual religiosity. Moreover, it has often been used to detect strong forms of religiosity (‘belonging’ as the counterpart to ‘believing’) in cross-national research (Billiet 2002; Brenner 2016; Davie 1994; Ruiter and Van Tubergen 2009; Voas 2009; Reitsma et al. 2012). Secondly, church attendance has a social dimension that is important in two respects: it exposes the laity to the messages of the clergy and fosters regular social interaction and the strengthening of group identity. Thirdly, the question concerning church attendance is relatively simple to formulate and easy to understand for respondents, given that it requires simply reporting the frequency of a certain behaviour.

3 Building the harmonized dataset

3.1 The selection of studies

We used three criteria for selecting the studiesFootnote 2 to be included in the harmonized dataset. First, we decided to consider only studies produced for non-commercial purposes and with public access, since their data are easy to obtain and they usually provide decent documentation for data quality assessment. Secondly, we selected cross-national comparative studies with some thematic coverage of religion and full coverage of the adult population through nationally representative samples. Thirdly, we considered only repeated cross-sectional surveys, i.e. surveys pertaining to a study repeated over time on the same population but with different samples. This design was chosen in order to reconstruct trends in church attendance fully and comprehensively, since any new edition of the study provides an updated picture of the population and thereby improves our understanding of change at the aggregate level (Duncan and Kalton 1987). The issue of data quality (Blasius and Thiessen 2012; Groves 1989; Harkness 1999) has not been tackled directly; however, given the specific selection criteria we applied, the data quality of the selected studies, despite some heterogeneity across studies, rounds and countries, is well known in the scientific community and well assessed in the literature (Kołczyńska and Slomczynski 2019; Maineri et al. 2017; Oleksiyenko et al. 2019). Based on these criteria, five repeated cross-sectional studies were chosen:

The pooled dataset covers 45 European countries. Those countries are Albania, Austria, Armenia, Belgium, Bosnia Herzegovina, Bulgaria, Belarus, Croatia, Cyprus, Northern Cyprus, the Czech Republic, Denmark, Estonia, Finland, France, Georgia, Germany, Greece, Hungary, Iceland, Ireland, Italy, Kosovo, Latvia, Lithuania, Luxembourg, Macedonia, Malta, Moldova, Montenegro, the Netherlands, Norway, Poland, Portugal, Romania, the Russian Federation, Serbia, the Slovak Republic, Slovenia, Spain, Sweden, Switzerland, Turkey, Ukraine, and the United Kingdom. Due to their historical specificities, a split version is also available for Germany (West and East) and the United Kingdom (Great Britain and Northern Ireland).

At this stage of the project, we decided to focus on European countries in order to limit the heterogeneity of the meaning of church attendance across religious traditions.

3.2 The studies’ specifications

All rounds of the different studies were included in CARPE, except when preliminary analyses detected quality problems or duplications. Specifications for the studies considered follow.

  1. (a)

    Eurobarometer: the Mannheim Eurobarometer Trend File 1970–2002 was combined with Eurobarometers 63.1, 63.4, 64.3, 65.1, 65.2, 65.3, 66.1 and 73.1, covering the period of 2005–2010, the Central and Eastern Eurobarometer (1990–1997) and the Candidate Countries Eurobarometer (2000–2004). Eurobarometer 46.1 was dropped from the harmonized dataset since it overestimates church attendance due to the provision of a specific filter question (self-described religiosity): church attendance was not asked to those not describing themselves as religious and considered as missing for these cases.

  2. (b)

    ESS: Rounds 1 (2002) to 7 (2014) were included in the dataset.

  3. (c)

    EVS and WVS: the integrated version, combining the EVS (four rounds: 1981, 1990, 1999 and 2008) and WVS (six rounds: 1981–1984, 1989–1993, 1994–1998, 1999–2004, 2005–2009 and 2010–2014) were included in the pooled dataset.

  4. (d)

    ISSP: rounds from 1985 to 2016 were included in the dataset. For some study rounds, data referring to two consecutive ISSP ‘modules’ (equivalent to study rounds) have been collected within the same sample in several countries. In order not to duplicate the sample, the following ‘country rounds’ were not included in the harmonized dataset: Austria 1987, 2000, 2003, 2007 and 2009; Ireland 1990, 2004 and 2007; Italy 1986, 1988 and 1990; the Netherlands 2003; Portugal 2003, 2005 and 2008; and Switzerland 1996, 2004, 2006, 2008 and 2011.

There are two versions of the CARPE dataset: an individual level version containing 1,879,751 cases, and an aggregate version in which every case refers to a specific combination of study, year and country (Biolcati et al. 2020; see also “Appendix 1”). The composition of the harmonized dataset is summarized in Table 1. Each cell in the table reports the number of countries surveyed in the specific year-study combination. The harmonized dataset is made up of 1665 distinct surveys. The study contributing the most cases is Eurobarometer (48.0% of the surveys in the harmonized dataset comes from the European Commission study), followed by the ISSP (29.5%). The remaining surveys come from the ESS (10.8%), EVS (7.1%) and WVS (4.5%). The longest time series is Eurobarometer, which covers 37 years from 1973 to 2010. This is followed by the ISSP, which spans 31 years. The remaining studies cover different periods and have a different ‘granularity’. The EVS has run every 9 years from 1981 to 2009, the observation window of the WVS stretches from 1989 to 2008 with approximately 5-year intervals between each round, and the ESS began in 2002 and rounds have been collected every 2 two years since.

Regarding the sample size, the ESS has the largest mean sample size (1453), followed by the EVS (1152), ISSP (1137), and WVS (1104). The surveys with the smallest mean sample sizes are those undertaken by Eurobarometer (805).

Table 1 Surveys distribution by study and year, 1973–2016
Full size table

The distribution of surveys by study and country is reported in Table 2. The median number of surveys per country is 35 for the 44 years of the observation window. If countries with at least 20 surveys are selected, the median number rises to 47. The mean sample size is 1011, ranging from 454 for Luxembourg to 1640 for the Russian Federation.

Table 2 Surveys distribution by study and country, 1973–2016
Full size table

Individuals aged 18–70 were selected. For respondents older than 70, the indicator considered here becomes less valid, since the decrease in church attendance recorded from that age onwards does not necessarily reflect a change in religiosity but rather the increasing difficulties the elderly may have in physically getting to church.

3.3 Harmonization procedures

While the introductory sentence of the question regarding church attendance is quite constant over studies (e.g. “Apart from such special occasions as weddings, funerals, etc., how often do you attend religious services?”—ISSP 2016), the answer options are characterized by a certain degree of heterogeneity. In order to do harmonize the response formats, three different strategies were followed.

A first, easy way was to distinguish between respondents attending church ‘at least weekly’ versus respondents attending ‘less than weekly’. The weekly threshold makes sense for Catholic countries, since participation in Sunday mass is intended to be a precept of Catholicism. This brings with it a loss of accuracy, but the threshold has a clear meaning for most respondents and can be identified in all the surveys considered. However, this choice seems inappropriate for non-Catholic countries, where weekly church attendance is not so widespread and the distinction between ‘regular’ churchgoers and others occurs at different thresholds.

Alternatively, a monthly threshold would work for most denominations, at the expense of losing precision for Catholic countries. However, this has a major drawback, since Eurobarometer response formats do not include a monthly answer category prior to 2002 (see Table 3). This strategy thus results in the loss of a large number of cases, since our pooled dataset relies extensively on the Eurobarometer.

Table 3 Coding scheme for the implied probability of weekly church attendance, by study and years
Full size table

A third strategy to follow does not require setting a fixed threshold for church attendance. We proceeded by imputing an implied probability of weekly church attendance, as proposed by Hout and Greeley (1998, p. 116), derived from the reported frequency of church attendance. Our indicator of implied probability is obtained by converting reported church attendance frequency into a ratio between the weeks per year in which a person attended church and the total number of weeks per year. For example, when respondents declared that they attended church once a month, it was presumed that they attended church in approximately 12 of the 52 weeks of a year. Their implied probability of attending church in any given week therefore was equal to 0.23 (i.e. = 12/52). Table 3 reports the coding plan for all the surveys included in the harmonized dataset. This transformation allows to treat church attendance as a continuous variable and to compare attendance across different studies while retaining the accuracy of the original response formats.

Following Hout and Greeley (1998), a probability of 0.99 has been assigned to those who declared to attend religious services every week. This choice seems reasonable to account for possible random events that could prevent people to attend mass, even if they would like to. In practical terms, the results are not affected by this choice. Some further clarifications might be useful. For “Two–three times a month” we chose a midpoint (30 weeks a year) between two times a month (24 weeks a year) and three times a month (36 weeks a year). Similarly, for “Once to three times a month” we chose the midpoint of 2 times a month and we assumed that this equals a 0.50 probability of weekly church attendance. For “Several/Few times a year” we assumed 6 weeks a year (1 week every 2 months). For “About every 2 or 3 months” we chose a midpoint (5 weeks a year) between every 2 months (6 weeks a year) and every 3 months (4 weeks a year). For “Only on special holy days” we assumed 1–2 (1.5) weeks a year. Considering the overall scale, we assigned a probability of 0.01 to “Less often” and “Rarely”. The implied probability assigned to the answer option “Less frequently” for ISSP changes over time to accommodate for some options that were not present in certain rounds.

Beyond church attendance, a few other target variables were harmonized. We selected four background variables that at the same time were relevant for the analysis of church attendance and feasible with a sustainable workload. The following target variables are included in the dataset: gender, age, level of education, religious denomination. To facilitate the harmonization process, we defined very broad response options for the denomination (Catholic, Protestant, Orthodox, other) and the level of education (lower secondary or less, upper secondary, tertiary).

4 Reliability checks

For CARPE to be used extensively, the quality of the measurements it provides needs first to be assessed.Footnote 3 While assessing their validity is ultimately a question for the substantive research this is meant to invite, assessing certain aspects of their reliability is something that should come before any substantive application. Reliability, when speaking of measures, implies both “consistency” and “repeatability”. A very common definition of reliability is provided by Carmines and Zeller, who affirm that “Reliability concerns the extent to which an experiment, test, or any measuring procedure yields the same results on repeated trials” (1979, p. 11). Similarly, Nunnally (1978) stresses the fact that such measurement consistency should be obtained over a variety of conditions, and Corbetta (2003) emphasizes the reproducibility of measurement results.

With regard to CARPE, two main issues concerning reliability arise: the first is that of the data coming from different sources, the second is that of the losses or biases that the harmonization process might introduce into the original data. For reliability assessments, the different samples from which the aggregate CARPE data set is built can be considered as different trials (Carmines and Zeller 1979), or at least as being a similar procedure performed under different conditions (Nunnally 1978). Measures of the same property (frequency of church attendance in a certain year in a certain country) coming from different sources should ideally be consistent. To put this another way, the different sources should not produce systematically biased measurement outcomes.

First, our focus turns to the consistency between aggregate estimates coming from different survey programmes. The scatter plots in Fig. 1 show the country-year observations shared by different study pairs. For example, the first graph reports the mean implied probabilities of church attendance for the years and countries that overlap in Eurobarometer and EVS/WVS surveys. With the correlations at or above 0.95 for all pairs, this clearly demonstrates that observations coming from the different sources involved are fairly equivalent.

Fig. 1
figure 1

Implied probabilities of weekly church attendance, country-year averages coming from different studies and the correlations between them

Full size image

Moving on to a multivariate and multi-level perspective that also allows the precision of the estimates within each sample to be accounted for, it is possible to reach similar conclusions. Table 4Footnote 4 presents three alternative multi-level estimation approaches. The first column (VARIANCE PARTITION, 4 LEVELS) basically assesses how individual variance in service attendance is accounted for by four levels of nesting implied within the harmonized dataset: the individual level, the study-country-year level, the country-year level and the country level. What is of interest here is the share of variance accounted for by the second level, that of study-country-year. This is because that level refers to the differences between samples (and thus, between survey programmes) within the different country-years (which are modelled as higher levels). This corresponds to the variance accounted for the different survey programmes after having controlled for the variance due to the various countries and years in which the interviews took place. As the VPCFootnote 5 shows, the share of variance due to survey differences within country-years is only 0.7%. This confirms the negligible impact that the survey sources have on the overall measurement.

Table 4 Testing the effects of studies on implied probability of weekly church attendance (4 level variance partition multilevel model and 3 levels fixed effects and random slopes multilevel models)
Full size table

The other two models follow a different strategy to obtain almost the same results. In the FIXED EFFECTS and RANDOM SLOPES 3 LEVELS models, the differences between surveys are not modelled as additional levels like before, but rather as covariates in the fixed part of the model. The only difference between the two is that in the latter model the effects of such covariates are allowed to vary between country-years, as this controls for the different coverage (in terms of years and countries) of the different studies. This, again, makes it possible to model whether different studies lead to different measurement results given that it excludes the effect of the different studies designs in term of period covered and granularity. Generally speaking, the random slope model shows a better fit if compared to the fixed effect one (lr test = 0.000). This, together with the fact that the fixed effects are very small (less than 0.02 on a 0–1 scale) suggests that on average the different survey programmes in the same country-year capture the same level of church attendance.Footnote 6

Overall, the tests presented here lead to the same conclusion: the different survey programmes on which CARPE is based (the EVS/WVS, EB, ISSP and ESS) are convincingly consistent as a foundation on which to build a harmonized form of measurement of church attendance: neither the differences in the sources nor the effects of the implied probability-transformation seem to present any biases or distortions.

5 Possible research applications

Generally speaking, the CARPE dataset can be useful in many ways. Among them, we would stress the following ones:

  • Providing long-term country trends in religious change for individual countries.

  • Comparing and grouping church attendance and religious change trends.

  • Building country and country-year level figures to be used as contextual measures in multilevel models.

  • Disentangling the effects of cohort replacement and period succession in explaining religious change.

While the first three tasks can be accomplished by relying on the aggregate version of the CARPE dataset, the last one needs to be based on individual-level observations. Here we present one application for each observation level of the CARPE dataset. In the first, we have applied group-based trajectory models (GBTM) to the aggregate dataset in order to identify the best way for clustering European countries according to their trends in church attendance. In the second, we have applied a cross-classified multi-level model to the individual-level dataset in order to disentangle the relative contributions of cohort replacement and period succession in describing religious change.

These two examples are primarily focused on the variable concerning church attendance, as it represents the main target of the harmonization procedure. However, this does not exclude the possibility to differentiate such analyses by gender, different denominations, age groups or levels of education. As described, CARPE also contains a harmonized version of those variables.

5.1 A group-based trajectory model for clustering European countries

The main aim of this application is to cluster European countries’ religious trends in the most meaningful and economical way possible. Other attempts have been made in the relevant literature to perform the same exercise, but all were based on data coming from single studies, or even data only from specific rounds of such studies. An ex-ante clustering mainly based on territorial distributions has been used by Voas and Doebler (2011) to present some general trends in religious change across cohorts in Europe. Burkimsher (2014) has also drawn some country trends by inspecting inter-cohort differentials for the 1950s–1980s. Finally, Brenner (2016) has provided a concise summary of religious service attendance trends.

Three main methods have been proposed in the literature for modelling trends and general developmental trajectories (Jen et al. 2010). The first is hierarchical modelling (Bryk and Raudenbush 1987), the second is growth curve modelling (Meredith and Tisak 1990), and finally there is group-based trajectory modelling (Nagin 2005). The first two approaches assume that the higher-level parameters are distributed according to a normal distribution, whereas group-based trajectory modelling does not (Jen et al. 2010). Given this relevant feature, this third approach is more flexible when it comes to identifying the clustering solution that best fits our data.

When applying this strategy to the aggregate version of CARPE, the best-performing solution is a 7-group one in which almost all cluster trends are modelled as a 2nd order polynomial except for cluster 3 (1st order), and cluster 5 (3rd order). Table 5 shows the results of this model estimation.

Table 5 Group-based trajectory model estimation results (description of the 7 groups and corresponding parameters form a third level polynomial interpolation)
Full size table

A brief look at the coefficients makes it possible to outline the main features of the seven groups.Footnote 7 The first group consists of just one country (Ukraine) showing an increasing U-shaped trend. This is followed by a very large group of countries (comprising almost 35% of the observations) whose trend is essentially flat (very small linear and quadratic components) and with very low starting points (around a probability of 0.08). Then come three different groups in which decreasing trends can be observed, starting from different levels; the higher the starting point the steeper the trend. In addition, Group 5 also displays a relevant quadratic component. Group 6 is special: it starts from a quite high level of 0.41 but both its linear and quadratic components are close to 0. An inspection of the trends in the countries belonging to this group shows a great heterogeneity in their functional forms: overall, the shape can be described as (very) slowly declining. The remaining group (7) is the second-smallest group: its trend starts very high (around the probability of 0.92) and declines almost linearly. The features of the seven groups’ trends can be visualized better by looking at Fig. 2.

Fig. 2
figure 2

7 Groups clustering solution for the implied probability of church attendance in Europe between 1973 and 2016

Full size image

Figure 2 clearly depicts the wide observation window spanning from 1973 to 2016. This has been possible only because of the cumulative approach used in building the CARPE dataset.

Starting with this clustering solution, it then becomes possible to provide a unique example of how a harmonized dataset can accentuate the strengths of different studies while at the same time minimizing their weaknesses. Figure 3 isolates two of the studies behind CARPE in order to highlight the potentials of such data harmonization. The EVS has been chosen because of its wide thematic coverage of religion, as well as its large and regular observation window (every 9 years starting with 1981). To contrast with this, the ESS has been chosen because of its much shorter time coverage, but also for its high granularity (every 2 years starting with 2002). As will be shown, CARPE combines the strengths of both studies. Panel 1 shows the trends of the 7 clusters that are drawn by relying only on EVS data, while Panel 2 does the same for ESS data. Panel 3 plots these two figures onto the same graph. From these three panels, the characteristics of the two studies emerge clearly. Panel 4 shows the same 7 trends based on combined EVS and ESS data, while Panel 5 compares these latter trends with those based on the entire CARPE dataset. Panel 6 shows the trends based only on CARPE. Especially when looking at Panel 5, it clearly emerges how a harmonized dataset such as CARPE is able both to increase the granularity and extend the observation window of a cumulative approach investigation in relation to the windows of the individual studies involved, while simultaneously providing very reliable estimations.

Fig. 3
figure 3

Combining different studies to draw aggregate trends: an example from EVS and ESS

Full size image

5.2 A cross-classified model to disentangle period and cohort effects on religious change

A second empirical application aims to disentangle the effects of cohort replacement and period succession on religious change. Far from being a simple empirical exercise, this distinction turns out to have a very strong theoretical value. If religious decline is mainly a matter of period succession (the effect of the passage of time), then the focus of both theoretical argumentation and empirical analysis should be on the phenomena or events that make people move away from religion. If instead cohort replacement is the leading mechanism, then the focus should be on the ways in which religious values, attitudes and behaviours are (no longer) transmitted from older to younger generations.

Separating these two mechanisms of change is complicated by the Age–Period–Cohort problem, which arises from the fact that possible effects of periods and cohorts are perfectly confounded by the effect of people aging. The methodological debate around this is often dominated by very technical solutions trying to solve the main problem directly: the problem of identification arising from the perfect collinearity of age, periods and cohorts (A = P–C).

Generally, the techniques attempting to deal with the Age–Period–Cohort problems can be divided into two main groups. On the one side are approaches trying to find a general procedure that can be applied to any APC problem (Yang and Land 2008). In this group we can place the constrained generalized linear models (Yang and Land 2013), the intrinsic estimator (Yang et al. 2004), and the Hierarchical Age-Period-Cohort models (HAPC) (Yang and Land 2008). On the other side are alternative approaches arguing for a more subject-specific perspective, which would exclude one of the three factors of the equation on theoretical grounds (Bell and Jones 2017).

Our contribution is characterized by three main features. (1) It draws on the Hierarchical Age–Period–Cohort models (HAPC) in which age is treated as a fixed-effect at the individual level and periods and cohorts as two cross-classified higher levels. However, (2) it considers age as irrelevant for explaining long-term aggregate religious change, thus introducing it only as a categorical control variable calibrated to measure three key life-stages for religious behaviour (< 30, 30–59 and > 59). Finally, (3) it takes full advantage of the harmonized CARPE dataset in order to minimize the main estimation bias potentially linked with such a way of modelling. Concerning this last point, the main critique levelled against HAPC models is that they are very sensitive to the ranges of periods and cohorts. As Bell and Jones note, “when a sample based on a small range of cohorts is taken, such that the period range is much greater than the cohort range, the results produced are very different to those produced when cohort groups span a much wider range than periods, as is structurally the case with repeated cross-sectional data” (2017, p. 1). This happens because the model tends to assign higher parameter variance where there is higher variability in the data. Clearly, a cumulative design such as that of CARPE makes it possible to rely on a much more balanced structure (71 birth cohorts and 42 periods) than single cross-sectional designs such as the EVS (just 4 rounds) or ESS (just 9 rounds).

Overall, the use of the individual-level version of the CARPE dataset makes it possible not only to control for this bias, but also to extend the time frame (back to the 1920 cohort and the year of 1973 respectively), while at the same time estimating parameters for every single year, thus providing very reliable and precise figures.

The entire analytical procedure for this analysis is based on three different modelling steps. The first one (M1) is based on periods, cohorts and countries as cross-classified higher levels. This means that every individual in the sample belongs to a unique combination of period, cohort and country. The second step (M2) adds the interaction between periods, cohorts, and countries as additional cross-classified levels, which allows country-specific trends to diverge from the general one. The third step (M3) simply adds age as a three-category fixed effect term.

As before for the reliability assessment, we turn to the VPC statistics to evaluate the relative impact of periods and cohorts in explaining religious change. These estimates are reported in Table 6 and appear to be coherent between the three models: the effect of cohort replacement in explaining religious change is overwhelmingly larger than that of period succession. The main empirical evidence of this is the VPC for cohorts always being higher (7.52 in the last model) than that of periods (0.54). Generally speaking, the proportion of variance that can be explained by cohort replacement is 14 times the size of that which can be explained by period succession.

Table 6 Cross-classified models to disentangle period and cohort effect. VPC results for the three different specifications
Full size table

6 Discussion and conclusions

The limited scale of the CARPE project highlights the burden as well as the potential benefits of its harmonization strategy. When human and financial resources are scarce, this kind of project becomes difficult to start, extend or update. A lot of work is involved in data preparation before analysis can even start: this implies high costs for researchers while returns are not certain. In cases where harmonization is successful, the theoretical and methodological benefits are remarkable. Researchers can pose new questions or assess old questions on a more reliable empirical basis. In addition, the number of methodological issues for which a harmonized dataset can provide potential solutions is remarkable.

With reference to the CARPE project, some limitations need to be highlighted. First, church attendance is a single indicator referring to a single dimension of a concept that is itself just as complex as individual religiosity. This obviously also restricts the conceptual validity of the harmonized measure. Adding further dimensions of religiosity to the dataset surely represents the most interesting possible expansion of the CARPE project. For example, adding subjective religiosity and daily praying could allow to construct a harmonized latent variable that could also be equivalent over time in a theoretically richer way than the single indicator of service attendance can (Billiet 2002). That the meaning attached to not attending church weekly by the Catholic church changed from a “dead sin” before the Vatican II Council to an undesired option thereafter points to changes in the nature of religiosity that a simple quantifying behavioural measure as ours per construction cannot possibly tap into.

Secondly, when investigating church attendance there is always the well-known problem of overreporting bias (for the Italian case, see Castegnaro and Dalla Zuanna 2006). Beyond the traditional explanation in terms of social desirability (Hadaway et al. 1993), recent contributions focus more on an identity-claiming mechanism (Brenner 2011; Vezzoni and Biolcati 2019). In the context of a longitudinal dataset, the potentially changing magnitude of such overreporting should be considered (Vezzoni and Biolcati-Rinaldi 2015, p. 105).

Thirdly, while there is a shortage of other indicators of religiosity, there is also a shortage of indicators of other phenomena related to religiosity. From this perspective, the scientific community should exploit the recurrence of the source datasets over the different harmonization projects: a sort of harmonized ID would give the opportunity to harmonize different datasets according to different research questions.

Fourthly, the issue of weighting is not considered. While all the studies considered here provide weights (but not all of them for all the years and all the countries), the CARPE harmonization project does not include them. Cross-national surveys are characterized by a significant degree of heterogeneity in weighting methods across studies, countries and over time. “Yet the variability in how these weights were calculated, their quality, and their adjustment effects are considerable” (Zieliński et al. 2019, 1035). Since weights are calculated in the different studies according to different strategies, we decided to discard them altogether. A different strategy could be to calculate new harmonized weights using international official statistics. More research is needed on this point. Similarly, it would be useful to assess possible, if any, distortions due to the different sampling, answer options, and placement of questions within the questionnaire.

Lastly, more research is needed to further validate the use of the 'implied probabilities' transformation to the underlying discrete church attendance variable. This is not an issue of the harmonization process per-se, but becomes a relevant point to assess while reasoning about modelling requirements. This kind of reasoning, we believe, must be part of every harmonization design.

To conclude, the application of a harmonization strategy to a target variable reporting a behavior (i.e. church attendance) was illustrated. We think that such a strategy would be applicable to other survey items that measure the frequency of a particular behavior. Nevertheless, the item we dealt with presents two peculiarities that we believe important for the success of the harmonization strategy: firstly, church attendance is a behaviour very well-defined in normative terms; secondly, the wording of the question is relatively stable. As this is not the case for all the items about reported behavior, the success of the harmonization strategy might be more uncertain for some of them. This is something that should be discussed when considering a new harmonization project.