In order to tackle health inequalities, the 30 big cities in the Netherlands are advised to follow the interventions and policy measures recommended by the Dutch advisory committee on socioeconomic inequalities in health (Mackenbach & Stronks 2002). Part of their recommendations was to adapt health promotion programmes to the needs of lower socioeconomic groups. Especially, community interventions were thought to be suitable for this purpose (Mackenbach & Bakker 2003).

One of the 30 big cities in the Netherlands, the city of Eindhoven funded a community intervention programme in two deprived neighbourhoods. The objectives of the programme “Wijkgezondheidswerk” (Dutch for “Working on Healthy Neighbourhoods”) were to improve health-related behaviour among adults aged 18 to 65 years. A community intervention trial with quasi-experimental design was used to assess the effectiveness of the 2-year intervention programme on (intermediate) outcomes of health-related behaviour. An impact evaluation (Kloek et al. in press), process evaluation and participatory action research among stakeholders (Weijters & Koelen 2003) were part of the evaluation study. In this article we report the results of the process evaluation that was conducted during the implementation phase of the intervention from September 2000 to September 2002.

Community interventions are complex interventions that are implemented at multiple levels, include multiple audiences and consist of multiple components, so that process evaluation becomes important in order to know the extent to which all intervention components are actually implemented. Furthermore, process evaluation can help explain why certain results were achieved (Baranowski & Stables 2000; Linnan & Steckler 2002). The purpose of this process evaluation is to examine whether the intervention programme was delivered as planned and to what extent the target population was exposed to the intervention programme. Before discussing the results of the process evaluation we continue with a brief history of the preparatory phase of the programme “Wijkgezondheidswerk”.

The programme “Wijkgezondheidswerk”

The Municipal Health Services (MHS) initiated the programme “Wijkgezondheidswerk” in 1999. The preparatory phase of the intervention from June 1999 to September 2000 was conducted as follows.

Beforehand, the MHS, as the lead organisation, assigned two deprived neighbourhoods (“Tivoli” and “De Bennekel”) to receive the intervention. Geographically, one of these neighbourhoods consisted of two smaller areas. Therefore, three comparison neighbourhoods were matched with the intervention neighbourhoods, using information on sociodemographic and health-related behaviour variables. The programme started in June 1999 with a meeting in which several professional organisations and local grassroots organisations brought up health issues that could be tackled in the neighbourhoods. Thereafter the MHS started to develop two neighbourhood coalitions in both neighbourhoods, which should plan and deliver the intervention programme. In January 2002, the implementation of intervention activities was postponed because of differences in opinion about the intervention goals and allocation of tasks between the neighbourhood coalitions and the MHS. The MHS was bound to its contract with the external funding agency and therefore had to focus on health related behaviour, while the other organisations preferred to implement activities more generally related to health. To solve these competing opinions, the power and control of the programme were given explicitly to the neighbourhood coalitions. Furthermore, an assessment of the capacity of manpower of all participating organisations contributed to a better collaboration between the lead organisation and the neighbourhood coalitions. In May 2000, the neighbourhood coalitions achieved an agreement on the intervention goals with the lead agency. The goals were related to Lalonde’s health fields (Lalonde 1974) and in the field of lifestyle both neighbourhoods chose nutrition, and especially fruits and vegetable consumption and physical activity as their main intervention topics. A third topic in “De Bennekel” was smoking and in “Tivoli” excessive alcohol consumption. The implementation of intervention activities started in September 2000. At this same moment the baseline data for the impact evaluation were collected from a random sample of 4800 residents in the intervention and comparison neighbourhoods aged 18–65 years. A follow-up of the baseline cohort was conducted in September 2002, after the end of the 2-year implementation phase.

The results of the impact evaluation showed that the intervention had no impact on outcomes of vegetable consumption, physical activity, smoking and alcohol consumption, or on intermediate outcomes (including knowledge, attitudes, self-efficacy expectations, awareness of one’s one behaviour, and stages of change) of vegetable consumption or physical activity. There was evidence of a small impact on fruit consumption and intermediate outcomes of fruit consumption including knowledge related to fruit and vegetable consumption, stages of change for fruit consumption, and self-efficacy expectations for fruit consumption.

Methods

Key process evaluation components

To describe the process evaluation data we used the key process evaluation components as defined by Linnan & Steckler (2002): (1) context or environment that may influence intervention implementation, (2) recruitment of participants on the individual and community level, (3) dose delivered by the intervention providers, (4) reach of the target audience, (5) dose received by the target audience, and (6) fidelity of intervention delivery. The measure of fidelity addressed whether the intervention was carried out according to important key principles of the “community approach”; participation of residents in planning and implementing the intervention, intersectoral collaboration of organisations from relevant sectors, tailoring to local needs and existing structures, incorporation of a social ecological perspective, use of multi-strategies and a broad lifestyle approach (van Assema & Willemsen 1998).

Data collection of process evaluation components

A representative of the MHS kept minutes of all community coalitions’ meetings. The minutes of the meetings were used to determine the number of meetings and members during intervention implementation. They were also screened for additional information on intervention activities, especially for activities that were not implemented. Registration forms were used by the MHS to record information on the intervention activities about content and duration of the activity, target groups, number of participants, successful and unsuccessful elements, and sustainability. In the postal survey that was conducted in September 2002 among the general population of the intervention and comparison neighbourhoods, questions were asked about programme awareness and programme participation. To measure programme awareness, we asked, “Do you know the ‘Wijkgezondheidswerk’ Project?”: yes, yes and I also recognize the projects’ logo, I am not sure, no. To measure programme participation, we asked, “Did you participate in activities of the ‘Wijkgezondheidswerk’ Project?”: yes, I am not sure, no. Furthermore, the survey included questions about awareness of and participation in several large-scale intervention activities, the monthly newsletter, and sociodemographic factors. The final survey sample comprised 1 929 respondents who completed both the baseline and follow-up surveys (attrition rate: 31%). Lost to follow-up did not differ between intervention and comparison neighbourhoods. To assess the context of the intervention we used available statistics of the municipality of Eindhoven (Anonymous 1999b; Anonymous 1999a; ten Caten 2003). To assess the dose delivered by the intervention providers and reach of the target audience we used data from the registration forms, and to assess the dose received by the target audience we used the data from the postal survey.

Data analyses

The data on the registration forms were entered in a computerized database. If applicable, additional information from the minutes was also added in this database. “Dose delivered” was computed by summing the number of activities across intervention topic and intervention period for each intervention neighbourhood. “Reach” was computed by summing the number of participants of the intervention activities across topic, objective and channel of communication for each intervention neighbourhood. “Dose received” was computed using the survey measures of programme awareness and participation for each intervention neighbourhood and the comparison neighbourhoods. Analyses were conducted using SAS version 8.2. Frequency distributions were calculated on all process measures in the survey. Multivariate logistic regression was used to identify sociodemographic and health behaviour correlates of intervention exposure and reading the newsletter.

Results

Context

Table 1 shows the socioeconomic characteristics of the assigned neighbourhoods for the community intervention. The deprived neighbourhoods showed a higher presence of ethnic minority groups, higher unemployment rates, higher rates of rented property, lower mean household income, and a higher presence of lower educated residents. Furthermore, a citywide health survey had shown that there were considerable differences in health and lifestyle between the deprived neighbourhoods and the non-deprived neighbourhoods in Eindhoven. Community change was induced at the neighbourhood level when the MHS started to mobilize relevant sectors around the problem of health in the neighbourhoods (Fig. 1). In 1999, the intervention programme started as a top-down programme with a priori defined goals. A further description of the early phases of intervention development can be found in the introduction of this article. During the implementation phase, the neighbourhood coalitions planned most intervention activities. The MHS was the initiator of almost one third of the intervention activities, almost half were initiated by other participating professional organisations and 20 % by either the residents themselves or the neighbourhood coalition. Not all participating organisations had enough manpower for intervention delivery. Therefore, main executers of intervention activities were the MHS (40 %), Social Work organisations (19 %), residents (16 %) and other professional organisations (25 %). The focus of most intervention activities was on lifestyle and not on one of the other determinants of health. The total budget for intervention implementation was 90 000 Euros.

Table 1 Characteristics of intervention and comparison neighbourhoods compared to characteristics of the city of Eindhoven
Full size table
Figure 1
figure 1

Overview of participating organisations in the community intervention “Wijkgezondheidswerk” and intervention development (adapted from Goodman (Goodman 1998))

Full size image

Recruitment

At the neighbourhood level the MHS recruited members from various professional organisations and the local grassroots organisation to form a neighbourhood coalition (Fig. 1). The coalition’s goals involved assessing the situation, deciding what action to take, and implementing the programme. Representatives of the MHS, community social work organisations, Municipal Task Force of Social Development, local grassroots organisation, a general practitioner and researchers participated in the neighbourhood coalitions. In the beginning eight organisations were involved in the coalition (see Tab. 2). After a merger of two community social work organisations, seven organisations were left in the second year of implementation. To ensure awareness of the programme at higher organisational levels of the participating professional organisations a community advisory board was established at the city level.

Table 2 Overview of numbers of members, numbers of meetings and distribution of intervention activities* by neighbourhood coalition and period
Full size table

At the beginning of the project, community participation was limited to participation of a single member of the local grassroots organisation, but as the project progressed, the level of community participation also grew. At the end of the second year, the coalition had implemented several intervention activities that were proposed and organized by residents themselves. In nearly one third of the intervention activities, one or more volunteers were involved.

Participants of the intervention activities were recruited in various ways. Most activities were announced in the programs’ monthly newsletter and also on posters hanging outside the information shop of the local grassroots organisations. Smokers received a letter from their general practitioner to call for their participation in a quit-smoking course. Participants for school-based activities were recruited by means of a letter from their children’s’ teacher. Two activities used a door-to-door method to recruit participants.

Dose delivered

In total the neighbourhood coalitions planned and developed 53 activities of which 10 could not be implemented due to low participation rates of neighbourhood residents, implementation problems or bad weather conditions (Tab. 2). The coalitions implemented, 10 nutritional activities, 15 physical activity activities, 2 smoking activities, 16 activities that were related to health in general, and no activities directed at excessive alcohol consumption. Twenty were implemented in the first year of the implementation phase and 23 in the second year. The “De Bennekel” coalition implemented more activities (n = 27) than the “Tivoli” coalition (n = 16). Table 3 shows an overview of all implemented intervention activities (n = 43) by topic, objective and channel of communication. One third of the implemented activities were directed to attracting attention or providing information, one third was directed at increasing awareness or knowledge, and one third was directed at behaviour change. The most frequently used channels of information were face-to-face multiple session activities (courses with multiple sessions to improve health related behaviour) and one-off special events. Only one activity was directed at the neighbourhood environment and only one activity was evidence-based. (All intervention activities are listed by neighbourhood and topic in the appendix at the end of the article.)

Table 3 Overview of implemented intervention activities (n = 43) by topic, objective and channel of communication
Full size table

Reach

The intervention programme reached about 1 400 residents in De Bennekel and about 1 100 in Tivoli, respectively 21 % and 62 % of the total number of residents (Tab. 3). These numbers could not be corrected for double counts or participating residents from other neighbourhoods and they do not include the households that received the monthly newsletter. Thirty percent of the activities had between 11 and 25 participants. Several special events and activities in schools attracted large numbers of participants (n > 100). Almost half of the interventions were directed at adults in general, one third at adults and their children and there were several activities for ethnic groups (n = 5) or adults over 55 years old (n = 3). The easyto-read monthly newsletter included information on intervention activities to come or that took place and information on health related behaviour topics. Of the respondents who knew the “Wijkgezondheidswerk” programme, 82 % remembered the newsletters. Among the respondents who remembered the newsletters, 41 % reported that they had read the newsletter often/always, 43 % read it sometimes, and 13 % never read it. In general, respondents thought the newsletters were interesting and understandable. Multivariate logistic regression showed that age and ethnicity were independently associated with reading the newsletters. The odds ratios for younger respondents age 18–24 and 25–44 to have read the newsletter were 0.21 (95 % CI 0.08–0.60) and 0.50 (95 % CI 0.30–0.81) respectively when compared to that of respondents aged 45–66. The odds ratios for Dutch respondents was 1.75 (95 % CI 1.03–2.98) compared to respondents from other ethnic origin.

Dose received

Table 4 shows that between 69 %–71 % of the respondents of the postal survey in the intervention neighbourhoods were aware of one or more large-scale intervention activities of the programme and 11 %–13 % participated in one of these activities. In the comparison neighbourhoods, 11 % of the respondents were aware and 1 % participated in one or more large-scale activities in the intervention neighbourhoods. However, only 24 % of the respondents in the intervention neighbourhoods and 7 % of the respondents in the comparison neighbourhoods knew the programme by its name because of e.g. the monthly newsletter, information in the neighbourhoods’ information shop, other residents or the general practitioner. In the intervention neighbourhoods, 3 % of the respondents participated in activities of the programme “Wijkgezondheidswerk”. Multivariate logistic regression showed that neighbourhood and age were independently associated with participation in intervention activities. Residents from intervention neighbourhoods were more likely to participate than respondents from comparison neighbourhoods, but participation in Tivoli was also 2.89 (1.27–6.39) times larger than participation in De Bennekel. The odds ratios for respondents, age 18–24 and 25–44 years, were 0.20 (95 % CI 0.03–1.05) and 0.37 (95 % CI 0.15–0.88) respectively compared to that of older respondents, age 45–66 years.

Table 4 Awareness and participation in large-scale intervention activities of the programme “Wijkgezondheidswerk” among residents
Full size table

Fidelity

Table 5 lists the key principles of the “community approach” and whether they were carried out sufficiently or insufficiently. Most key principles were carried out sufficiently except for community participation and the use of an ecological perspective.

Table 5 Fidelity of intervention delivery
Full size table

Discussion

The programme “Wijkgezondheidswerk” was implemented according to most of the key principles of the “community approach”. Intervention delivery was less sufficient in its ecological reach with regard to the environmental level and in community participation, especially in the first implementation year. Most intervention activities were directed at attracting attention, providing information, and increasing awareness and knowledge. One third of the activities had the objective to change behaviour. Programme awareness and programme participation were 24 % respectively 3 % among residents in the intervention neighbourhoods. What are the lessons learned from this process evaluation?

The difference in goals and priorities found between the neighbourhood coalition and the MHS had two important implications. Firstly, single intervention components using intuitively reasonable methods to promote behaviour change satisfied the need of the neighbourhood coalitions. This approach may be viewed as being in opposition to the need of researchers for evidence-based decision making, to achieve an impact on health behaviour outcomes by using effective intervention components. Secondly, to measure programme effectiveness on health behaviour outcomes preferably all activities should be focused at the chosen health behaviours and target population of the impact evaluation. In practice, it appeared to be impossible to expect that the community coalitions would implement intervention activities under such strict conditions.

Furthermore, the neighbourhood coalition members shared knowledge, skills, expertise, perspectives and connections, but financial resources from outside the coalition paid most of the costs that were associated with intervention implementation and the MHS delivered most of the manpower to organize the activities. In this context, sustainability of such a community coalition will depend on new funding and sustained involvement of the collaborating organisations and in particular the MHS for their manpower.

According to the participatory action research the level of community participation depended on the role residents had in the project, good intersectoral collaboration, the phase of intervention delivery, and the kind of intervention activity (Weijters & Koelen 2003). Some tasks in the delivery of the intervention were just better suited to professionals and full control of community members on these tasks was not necessary. At the beginning of the programme community participation was limited to a single member of the local grassroots organisations, but as the programme progressed, the level of community participation grew.

The results of the impact evaluation showed that the programme “Wijkgezondheidswerk” only demonstrated weak evidence for a small intervention effect on outcomes of fruit consumption and no evidence for an intervention effect on other outcomes of health related behaviour (Kloek et al. in press). The insufficient strength of the total package of intervention activities is likely to play a role in the absence of a clear intervention effect. As mentioned earlier, most of the activities were based on intuitively reasonable methods and only one intervention component in the total package was evidence-based. Furthermore, two thirds of the activities of the intervention programme were directed at attracting attention, providing residents with information and increasing awareness and knowledge of health related behaviour, which are the first steps towards behavioural change. As a result, we may expect programme effects on knowledge, attitudes and beliefs. The intervention only showed a small impact on intermediated outcomes of fruit consumption including, knowledge, stages of change and self-efficacy expectations.

During the 2-year implementation period just one activity was focused on the neighbourhood environment. In one of the intervention neighbourhoods the local greengrocer’s shop advertised in the monthly newsletter with special offers. Although we do not know which intervention components contributed to the small effect on fruit consumption, this same intervention neighbourhood showed a higher increase in fruit consumption than the other intervention neighbourhood. Possibly, the attention for individual characteristics combined with changes in the environment attained synergy in promoting behaviour change. Furthermore, changes at the environmental level are difficult to achieve in a short time frame and they are also more difficult to achieve for health practitioners.

The intervention reached a considerable part of the general population in the intervention neighbourhoods, but probably lacked sufficient tailoring of intervention activities to reach all different segments within the community. Especially, men and younger residents were less often reached with the intervention activities. In addition, several events and activities were school-based, whereas the school population did not belong to the target population of the evaluation study. The monthly newsletter was important in reaching the residents of the intervention neighbourhood, although it was less often read by ethnic minority groups and younger residents. In practice, it was impossible to expect that the neighbourhood coalitions would implement only intervention activities for the chosen target population.

The self-reported awareness of large-scale intervention activities showed that a lot of respondents were aware that these activities were organized in their neighbourhood but most of them were unaware that these activities were part of the programme “Wijkgezondheidswerk”. Participation in the intervention activities was quite low; 3 % of the respondents were aware that they participated in one of the intervention activation. Given this low participation rates, behaviour change at the community level is not very likely.

Although there are a number of projects that evaluated a community intervention in a deprived setting, comparing the results of process evaluations remains difficult, as these are often not available, the setting is too different or other process measures were used. Best comparable are the process outcomes of the Coeur en Santé St-Henri programme (Paradis et al. 1995; O’Loughlin et al. 1999) in which also awareness and participation outcomes were measured. In this programme, like in our study, there was a large difference between awareness of and participation in intervention activities. Furthermore, their results also showed some contamination of the comparison neighbourhoods and their perceived barriers during implementation like the difficulties with sustained commitment of participating organisations were recognizable.

The process evaluation had several limitations. At this moment, there is some general consent about the key components of process evaluation, which were used to structure this process evaluation (Linnan & Steckler 2002). However there are no standardized methods available to measure these individual components. Furthermore, the data collection methods we used could not capture all potentially important process information. For example, the assessment of the “dose received” was only done in the follow-up survey among the general population. Assessment of the “dose received” for all implemented intervention activities among participants of the interventions would have given more information on the short-term effects of these single intervention components. Furthermore, some of our process evaluation measures are based on observation of documents and minutes concerning intervention development and implementation. Although this was done systematically, the results are subject to interpretation bias.

A last remark concerns the complexity of community interventions and the challenges they pose for evaluation. It is important that future investments should be made and judged within the reality of community intervention practice. Beforehand, practitioners and researchers should decide what are realistic and valid outcomes of a proposed community health programme. Are they interested in individual risk behaviour change or in the process of community change? The answer to this question will be clearly linked to the goals of the community intervention. Furthermore, they should decide a priori what the acceptable level of implementation would be. Is a community health programme successfully implemented if e.g. and intervention is implemented according to key principles of the “community approach” or should implementation objectives be directed at more specific goals for the key process evaluation components reach, dose delivered, dose received and fidelity. Measuring programme implementation is critical in avoiding a type III error (Schwartz & Carpenter 1999), i.e. evaluating a programme that has not been adequately implemented, and thus drawing incorrect conclusions about the effectiveness of a given intervention.

In conclusion, this process evaluation indicated that it was feasible to implement a community intervention according to the key principles of the “community approach” in deprived neighbourhoods. However, it is unlikely that the total package of intervention activities had enough strength and sufficient exposure to attain community-wide health behaviour change. The results from this process evaluation were useful to show what was planned and what was achieved and they helped to explain the results from the impact Evaluation. Furthermore, the data can contribute to the international exchanges of experiences with development, implementation and evaluation of community interventions in deprived settings.