Background

HIV has had an enormous impact on the health of communities and on models of service delivery [1–5] particularly in sub-Saharan Africa, and more recently in the Caribbean. While all health providers have had to deal with this epidemic, nurses and midwivesFootnote 1 who deliver direct client care and comprise the largest proportion of the health workforce in low- and middle- income countries (LMICs) have been especially affected. Severe nursing shortages and unrealistically low nurse-to-patient care ratios are entwined with the impact of stigmatisation, workplace safety and health service delivery in HIV prevalent settings [5–13]. These have all contributed to significant gaps in HIV care in LMICs [14–16].

Efforts to address these gaps have been targeted at different facets of the problem. Extensive professional development training and mentoring programmes for nurses have focused on clinical skills and scope of practice [17]. These have included strategies to reduce HIV stigma [12, 18], to improve patient counselling on adherence to anti-retroviral therapy (ART) [19], to enhance voluntary testing and counselling for HIV testing [19, 20] and to increase the update of universal precautions to protect both patients and nurses [21, 22].

Human resource constraints have been addressed at national and district levels with a focus on task sharing with nurses [23–29] and task shifting to community health workers (CHWs) [23–28, 30]. Integrated care models [31–39] have been introduced to improve appropriate referrals, ART initiation, coverage and retention in care. Quality improvement approaches for better care and workplace policies and programmes have been tested such as cascade and systems analyses [15, 40–46]. Achieving the full potential of these approaches, however, requires leadership capacity development for nurses to fully engage in enabling change [47] and to address issues such as navigating professional turf, providing supportive supervision for CHWs [48–50], and shifting the organisational context for care delivery [51].

Utilising the technical knowledge and clinical experiences of nurses in the formulation of related guidelines and policies by senior decision-makers and policy-makers [52–58] also requires nursing leadership. However, many of the leadership interventions to involve nurses in developing and implementing evidence-based guidelines have been undertaken in higher-income countries [59–62]. There is a paucity of studies examining ways to strengthen nursing leadership for improvements in the uptake of evidence-informed HIV care in LMICs [24, 63, 64]. Furthermore, much of the research on building leadership capacity has focused on concentrated, short-term leadership training programmes either targeted directly at nurses [65–67] or aimed at those holding management positions [68]. As Daire et al. [63] pointed out, there is an abundance of training that focuses on the cognitive aspects of leadership rather than on the action component. If nurses in LMICs are to play a more substantial role in reorienting HIV care strategies and workplace conditions to be more evidence-informed, this requires action-based leadership: nurses’ engagement in and capacity to lead, implement and sustain HIV care improvements [69–71]. This study focused on strengthening the leadership and policy engagement capacity of nurses to address gaps in HIV care and workplace policies in all health facilities in their health districts.

Overview of study

The primary objective of this quasi-experimental study was to determine the impact of establishing leadership hubs on HIV care by nurses. We engaged hub members in a participatory action research (PAR) process to improve the uptake of evidence-informed nursing care practices and workplace policies for HIV. This study was undertaken in four countries (Jamaica, Kenya, South Africa and Uganda) between 2008 and 2012, as part of a larger programmeFootnote 2 of research and capacity building [72–74]. The main quantitative study findings are described in this paper. Qualitative results are published elsewhere [9, 56, 75–77]. This study was carried out with support from the Global Health Research Initiative (GHRI), a collaborative research funding partnership of the Canadian Institutes of Health Research, the Canadian International Development Agency, Health Canada, the International Development Research Centre and the Public Health Agency of Canada.

Methods

Leadership hub intervention

District-levelFootnote 3 leadership hubs were established in each country with the aim of stimulating district-wide health improvements in HIV care. The theory of change underlying the study was as follows:

  1. 1.

    Bringing together stakeholders from different system levels to form a district leadership hub; and

  2. 2.

    Providing hub members with training on research and evaluation, policy engagement and leadership; and

  3. 3.

    Using a participatory action research approach with hubs as they reflect on research findings from their districts about HIV care and policies;

  4. 4.

    Will strengthen hub members’ individual agency and collective capacity to plan, develop, implement and monitor district-level, evidence-informed change strategies;

  5. 5.

    And will thereby improve the delivery of evidence-informed HIV care by nurses and strengthen supporting policies in district health facilities.

Indicators related to our theory of change are described in Table 1.

Table 1 Theory of change and related indicators
Full size table

Although Danschroder’s Consolidated Framework for Implementation Research [78] had not been published when we designed the intervention for this study, we drew heavily on some of the literature, which informed this framework, in developing the overall leadership hub approach and specific training modules. Our focus on change within organisations as well as at the district level reflects an often neglected facet of change (the interface between inner and outer organisational contexts). Our intervention was consistent with key elements of Danschroder’s framework including adapting the intervention to the local context, while retaining core components; building agency of the hub members to extend and use their affiliations and power for positive change; and using an active change process aimed at both individuals and organisations.

We had several inclusion criteria specific to the composition of each hub (see Table 2). We aimed to have every leadership hub composed of members from four stakeholder groups— direct care nurses and nurse managers; researchers; decision-makers; and community representatives. This mixed composition was intended to extend the vertical networks (across system layers) of the nurses who joined the hubs and to activate district-level change by stimulating horizontal collaboration among hub members. Leadership hubs were designed to act as the lever for change, or enabling mechanism, that would transform enhanced capacity into action leading to policy and practice change [72, 75, 77]. This hub intervention targeted improvements in policies and patient care within health care institutions as orchestrated through a district-level change process.

Table 2 Inclusion criteria for leadership hubs
Full size table

One hub was established in each intervention district. We aimed for six to nine members per hub, with one person on each hub being a person living with HIV. Hub members were recruited by the country research team who approached individuals or health care organisational leads and consulted with national advisory committees, asking them to identify potential participants. Inclusion criteria for individual hub members are shown in Table 2. Hub membership was voluntary and unpaid. National advisory committees, set up specifically for the research programme, were intended to inform hub activities. Leaders of professional nursing associations, nursing councils, policy-makers and experienced academics were invited to participate on these committees.

Standard elements of the hub intervention, implemented across all four countries, are shown in Table 3. A chronology of the hub intervention and research programme data collection/analysis is illustrated in Fig. 1. A common slate of training workshops (seven) was delivered to hubs over 3 years to build skills in using research evidence to identify gaps in clinical practice and workplace policies for their districts, using evidence generated by the research programme. The training approach was consistent with PAR processes [79] and designed to improve hubs’ abilities to identify, question and take action on setting-specific health system issues, thereby empowering members as change agents [76, 77]. Through the workshops, we addressed four aspects of capacity: ability (knowledge and skills); resources (financial, material, logistics); authority (control, voice, participation); and responsibility (accountability, monitoring). Interactive discussions during the workshops and the use of local examples tailored the content to study districts in each country. Midway into the second year of each hub's life, members were provided with country-specific baseline findings from our surveys of evidence-informed clinical practices and workplace policies and qualitative findings about the impact of HIV on nurses and their clinical practice. In-country research assistants (RAs) encouraged hub members to interrogate and use findings to identify priority activities they could undertake and stakeholders they should engage. Hubs were trained to develop and implement action plans to address gaps identified in workplace or district policies and practices related to HIV care. Commencing in year 3, a bi-monthly project newsletter was distributed to hubs to encourage information sharing. Three country-specific communiques were prepared to share findings and implications with institutional and district-level decision-makers.

Table 3 Leadership hub model—standard intervention elements for development and implementation activities
Full size table
Fig. 1
figure 1

Chronology of the leadership hub intervention and research programme data collection/analysis

Full size image

In years 4 and 5, we offered each hub a small grant (maximum $1000 per project) to plan and undertake an evaluation project examining local district-level gaps in health policy or practice related to HIV nursing care. Country research teams provided mentorship and a peer review process, led by NE, provided further input to strengthen each evaluation project. Their letters of intent and subsequent proposals were formally reviewed by the research team (team members from each study country and Canada) and interns participating in the Jamaica international research internship for health system researchers (a capacity building initiative for junior researchers, also funded by the research programme). Hubs received extensive written feedback from the peer review committee, and RAs and hubs received training in how to respond to peer reviewers. Hub members obtained institutional letters of permission to undertake their projects.

Hubs received regular mentoring throughout the project at all country sites. Country RAs worked closely with hubs, supporting them through field visits and telephone calls. They coordinated hub meetings, facilitated hub training sessions and mentored hubs through the process of developing action plans, and implementing and disseminating evaluation projects. Two international teleconferences facilitated by the Canadian programme manager brought together hubs from all countries for discussions about their evaluation projects, action plans and nursing strategies for common workplace and policy issues. The Canadian programme manager mentored country RAs throughout this process by email and Skype, encouraged peer-to-peer learning and led three face-to-face refresher training sessions later in the project.

Study context

The study was conducted in countries located within the two regions most affected by HIV. In sub-Saharan Africa, which has 71 % of all HIV cases worldwide, an estimated 25.8 million people are living with the disease, a prevalence of 2.8 % of the total population [80, 81]. South Africa has the highest HIV prevalence in the region, at 11.2 % of the total population, while Kenya and Uganda have prevalence rates of 3.8 and 3.7 %, respectively [80]. The Caribbean is the second most affected region of the world with an overall HIV prevalence of 1.1 % (0.9–1.2 %), and 250,000 adults and children in the region living with the disease [81]. In Jamaica, the HIV prevalence rate is 1.2 %.

Country-specific adaptations to the intervention

Since the hub intervention was intended to be implemented using a PAR process, it was essential that the intervention was appropriately tailored to diverse settings. Certain intervention components were intended to be adapted by each country to increase relevance to their context, an important consideration particularly in low-resource settings and given that implementation contexts and priority health issues varied greatly across countries. The proportion of hub members to be drawn from each of the four stakeholder groups was not a pre-set requirement. Country research teams identified members based on receptivity to engagement, availability to commit for 4 years and the local context, and built on existing institutional links between the workplaces of country project leads and other organisations. While most hub members worked at the district level, Uganda (which had one hub based in the national capital) had a hub member who was a national member of parliament. While the aim was to have representation of an HIV-positive person on each hub, this depended on the self-disclosure by potential members of their HIV status; voluntary self-disclosure of HIV status was ethically essential and made it impossible to mandate a required number of HIV-positive members. The number of self-identified HIV-positive hub members ranged from one member per hub (Jamaica) to one member per country (Kenya, Uganda and South Africa).

Flexibility was deliberately built into other intervention elements to allow country research teams to tailor to setting. While all intervention activities were undertaken in the same 4-year period, the sequencing of some activities varied by country, such as the dates that all three hubs in a given country were established (Uganda was first to achieve this, in May 2008; South Africa was last, in February 2009 due to challenges in hub member recruitment). The timing of training workshops depended on participant availability, while the timing of grant approval and rollout of hub projects depended on hub turnaround time in response to sometimes extensive peer review comments.

Communiques shared with hubs with research findings from the programme presented country-specific findings on the same topics: nursing clinical practices, HIV stigma and human resources management; Jamaica developed an additional communique (HIV training and policy). Countries tailored workshop training materials using setting-specific examples and added workshops (one to three workshops per country) based on hub learning needs and interests. The focus for hub action plans and hub evaluation projects was chosen by each hub, within the guidelines set by the project stipulating that the topic had to emerge from gaps identified by the programme’s research findings and be responsive to the local health situation. Hubs could choose whether to undertake their projects on their own or jointly with another hub; in two countries (Kenya and South Africa), two hubs chose to work jointly on a project.

There were also unplanned divergences in implementation. This included the lack of a functioning national advisory committee in one country (the committee in South Africa ceased shortly after inception). In Kenya and Uganda, several hub members responded to nationally mandated redeployment of health personnel by maintaining their commitments, albeit at a distance, to their original hub (while they moved to new workplaces outside of their original intervention district, none had been redeployed to control districts). While all three hubs per country were sustained in three of the study countries, one hub (South Africa) ceased to function before the end of the project.

Design

A prospective quasi-experimental design was used, with baseline and follow-up data collection, pre- and post-establishment of the leadership hubs. Three intervention and three control districts were sampled in each country with the exception of South Africa where only intervention districts were included.Footnote 4 One leadership hub was established in each intervention district. A process evaluation of the leadership hub intervention examined hub activities and self-rated changes in the capacity of hub members. Pre and post measures assessed district-wide changes in clinical practices, quality assurance, workplace policies and stigma by sampling health care institutions in intervention districts regardless of whether participants from those institutions had been involved as hub members.

Multi-stage sampling

Intervention districts were purposively selected by country directorsFootnote 5 with consideration given to high HIV prevalence rates, proximity to the in-country research office, variations in geographic and socioeconomic conditions (e.g. urban versus rural), and the mix of health care institutions within the district. Control districts were selected on the basis of similarities in HIV rates and the distribution of health facilities. Because we aimed to improve district-wide HIV nursing care, eligibility for participation in pre and post measures was not limited only to the health facilities employing those hub members who were health care workers. All public government health care institutions within each study district were listed and categorised using WHO criteria for level of health facility [82]. Three levels of health facilities (WHO level 3, 4 and 5 institutionsFootnote 6) were eligible for inclusion. Level 3 facilities were health centres at the sub-district level (providing primarily health promotion and prevention services); level 4 facilities were district and sub-district hospitals; and level 5 facilities were referral hospitals at the provincial or national level. We excluded level 1 and 2 facilities (community health posts and dispensaries), and we excluded facilities that were officially designated as level 3 health centres but due to staffing limitations were actually functioning as a level 1 or 2 facility.

We used stratified, multi-stage random sampling. See Table 4 describing the number of institutions and participants by WHO institution level at baseline and follow-up. We enrolled 16 health care institutions (as planned) in both the intervention and control districts of each country (with the exception of South Africa where there were no control districts). All national, provincial and district hospitals that met the WHO criteria were included in the sample. In Kenya, a level 5 hospital that served the three control districts was located in an adjacent province and was included in the control sample. In Uganda, two national hospitals in the same intervention district were selected. One of these national hospitals was designated an intervention site, while the other, a specialty hospital, was assigned to the control group as it also provided services to clientele in the adjacent control district. Four or five health centres, which met WHO criteria for staff mix and functions were randomly sampled in each district.Footnote 7

Table 4 Number of institutions and participants by WHO institution level at baseline and follow-up
Full size table

National, provincial and district hospitals were provided with a protocol for randomly sampling staff. All eligible staff in health centres were invited to participate. Eligibility criteria included registered or enrolled nurse, staff nurse or manager; employed in their health care setting for at least 3 months; and fluent in English.

We used the same approach to sample participants for follow-up data collection. While the same institutions were sampled at both data collection points, the identity of respondents was anonymous, and thus, we were unable to match responses to baseline and follow-up surveys for those who participated in both data collection periods. Of the institutions sampled, 82 of the 109 institutions with valid survey data, or 75.2 %, had at least one respondent participate at both data collection periods. Three institutions did not have valid data at either collection point.

Pre and post measures

The same self-administered questionnaire was completed at baseline and follow-up.

Socio-demographic characteristics included sex, professional designation (enrolled nurse, registered nurse), highest education level (diploma or certificate, higher level of education), current work location within the institution (community, obstetrics and/or gynaecology, other) and frequency of contact with HIV patients (daily, less often than daily).

We adapted validated instruments [62, 72, 83] to measure clinical practices, quality assurance initiatives and workplace policies. Participants were asked to report their own clinical practices and those of their nursing co-workers (12 parallel items for each scale) using a five-point Likert scale. Response options were never (1), rarely (2), sometimes (3), most of the time (4) and always (5). Ten-item scales were used to assess both workplace quality assurance initiatives and workplace policies. Response options were yes (1), no (2) or unsure (3).

We assessed two dimensions of HIV stigma and discrimination [84]: stigmatising by nurses against people with HIV (10 items) and stigmatising by co-workers and the community against nurses who provide care to people with HIV (9 items). Responses were captured using a four-point scale: never (1), once or twice (2), several times (3) and most of the time (4).

Data collection

Country RAs were trained by the principal investigators (NE, DK, EK) and project manager (SR) through a 1-week intensive session. Research staff were introduced to the protocol and to PAR, taught how to conduct field sampling and assess eligibility of health care institutions and participants, instructed on how to obtain consent and trained to use all data collection measures [77]. Prior to follow-up data collection, an updated field guide was developed and refresher training held. Day-to-day supervision of RAs was provided by country directors.

Data analysis of pre and post measures

Data were entered in an Excel database and cleaned by RAs at each country site. RAs in Jamaica and Kenya completed preliminary data analysis under the direction of the principal investigators. Final analysis using SPSS [85] and the R statistical programme [86] was undertaken with the assistance of a statistician who worked with the Canadian principal investigator (NE). Data analysis steps are shown in Fig. 2.

Fig. 2
figure 2

Data analysis process

Full size image

Descriptive and bivariate analysis

Descriptive analyses were used to initially examine data and to assess response patterns. A Pearson chi-square was used to compare socio-demographic characteristics of respondents in intervention and control districts at baseline and follow-up.

Several items were mistakenly excluded from one or more countries’ pre-intervention questionnaires. Consequently, we discarded one item from each of the clinical assessment scales and four items from both the work place policies and quality assurance scales. We then conducted confirmatory factor analysis on these four scales and determined that a one-factor solution was optimal for each. Confirmatory factor analysis on the stigma scales yielded the two-factor structure reported in the literature [84].

As recommended [84], we analysed scores on the stigma sub-scales in two ways, using the standardised mean score (ranging from 0 to 1) and a dichotomised score (never (0) versus one or more (1)). For clinical practice scores, we collapsed and recoded responses to the 11 items as never, rarely or sometimes (0), versus most of the time or always (1) and computed a standardised mean score (ranging from 0 to 1) for both scales. For workplace polices and quality assurance measures, we retained six items from each scale and compared yes (1) versus no or unsure (0) responses. If someone had more than one missing value, the entire response for that scale was deemed to be missing; otherwise, the missing item value was imputed using a means score imputation procedure.

Inter-country differences in baseline mean scores were assessed for all scales, with data from respondents in intervention and control groups combined, using ANOVA and post hoc testing (Tukey’s HSD and Scheffe’s tests). We used a paired t test to compare respondents’ scores on the two stigma sub-scales and on quality assurance and workplace policies.

Within-country, pre versus post differences in outcomes between intervention and control groups for Jamaica, Kenya and Uganda, were assessed using the Mantel Haenszel chi-square (for dichotomised stigma scores) and independent t tests on the pre-post differences in mean scores between intervention and control groups (for all other measures). For South Africa, which had no control group, pre-post differences were compared using a Pearson chi-square and independent t test.

Multivariate analysis

Multivariate analysis was completed for Kenya and Jamaica. Uganda was excluded due to a large amount of missing data for district hospitals at follow-up. South Africa was excluded as there was no control group.

Stigma measures were analysed at the individual level, using a fixed effects logistic model. We analysed other outcomes using a repeated measures model and aggregated the data by institutions as this better met the assumptions of normality. This yielded one response for each institution and period. Institutions with surveys from only one period were discarded. Precision weights based on the sample size were used to account for varying numbers of surveys from different institutions and periods.

Common parameters for all models were (a) WHO institution (national/provincial, district and health centre); (b) intervention group (control and intervention, ignoring period); (c) period (pre (baseline) and post (follow-up)); and (d) intervention period, which had one non-zero value for any additional change in the control districts in the post period over what would occur if intervention and period operated independently.

Process evaluation of leadership hubs

Data collection

The characteristics of hub members and composition of hubs were documented by RAs in annual progress reports. The number and content of hub meetings and training sessions was captured from bi-monthly structured reports completed by RAs. Near the end of the project, each hub member was asked to rate their capacity (pre and post hub experience) on seven skill dimensions using a 10-point Likert scale (1 = low capacity, 10 = high capacity).

Data analysis

Turnover rates in hub membership were estimated using standard labour force methods [87] to assess period turnover: L/[(N(i) + N(f))/2], where L = number of hub members who left during the period; N(i) = number of hub members at the beginning of the period; and N(f) = number of hub members at the end of the period. Membership counts were determined for the beginning and end of each period using information in bi-monthly hub reports. At follow-up, hub members were categorised as active if they were listed as contributors to the final evaluation project reports.

Average mean scores for each capacity dimension were calculated. Pre-post differences were compared using a paired t test.

Results

Leadership hubs

At follow-up, leadership hubs averaged 8.4 members per hub; a total of 167 members participated in the hubs over the course of the intervention period. Over half (58.9 %) of hub members were direct care nurses and nurse managers; 11 % were researchers; 31.5 % were decision-makers; and 21.9 % were community representatives. The majority of hub members came from non-sampled health institutions (49.1 %) or from other workplaces or the community (22.2 %). The remainder (28.7 %) came from the district health institutions sampled for pre and post measures. Over one third (35.9 %) of sampled institutions in intervention districts had hub member representation; Jamaica had the largest portion of hub members (57.1 %) from sampled institutions. In six of the hubs, members included one person self-disclosed as living with HIV. Table 5 describes the membership characteristics of leadership hubs by country.

Table 5 Membership and characteristics of leadership hubs, 2008–2012
Full size table

Sample for pre and post measures

A total of 101 and 90 health care institutions participated at baseline and follow-up, respectively (response rates of 90.2 and 80.4 %). There were 866 survey respondents at baseline and 813 at follow-up. A minority of follow-up participants (ranging from 12.4 % in Jamaica to 25 % in South Africa) indicated that they had also completed the baseline interview. All study districts had level 3 health centres (30.4 % of the sample) but some districts also had a level 5 provincial/national hospital and/or a level 4 district hospital (54.5 and 15.2 % of participants, respectively). In Uganda at follow-up, there were no participants from district hospitals (level 4 institutions) in the control group and only five in the intervention group (see Table 4).

Baseline respondents were predominantly female, and 77.2 % were registered nurses. The majority of respondents (71.7 %) had at least daily contact with HIV patients. Participants had worked in their current work setting an average of 9.75 years; 49.3 % of the participants were in a staff nurse position and 50.7 % were managers. Kenya had the largest percentage of male respondents. South Africa had the largest proportion of registered nurses, while Jamaica had the largest proportion of enrolled nurses. The proportion of registered nurses was highest in health centres in both Jamaica and Kenya. Jamaican respondents reported significantly less daily contact with HIV patients (35.2 %) compared with other countries (over 80 %) (see Table 6 for socio-demographic characteristics of all health workers by country and by data collection period).

Table 6 Socio-demographic characteristics of participants on clinical practices, quality assurance, workplace policies and stigma scales. Data included is from all participants who completed pre or post questionnaires for the listed measures. For Jamaica, Kenya and Uganda, the following within-country comparisons were calculated: intervention (pre versus post); control (pre versus post); and pre-intervention versus pre-control versus post-intervention versus post-control. For South Africa, a pre versus post comparison was included
Full size table

In all countries, there were significant differences in the composition of the sample coming from different WHO institutional levels between baseline and follow-up, and in the three countries with control groups, between intervention and control districts. WHO institution level was associated with significant differences in baseline scores for most outcomes (clinical practices (self and peers), quality assurance and workplace policies were assessed): Jamaica (4/4), Kenya (2/4), South Africa (4/4) and Uganda (3/4). However, the direction of these differences varied. In Jamaica, Kenya and South Africa, health centres had the highest (best) scores while, in Uganda, health centres had the lowest (worst) scores (see Table 7).

Table 7 Outcome measures for the clinical practice—self, clinical practice—peers, quality assurance and workplace policies scales. For Jamaica, Kenya and Uganda, the following within-country comparisons were calculated: pre (intervention versus control); and pre-intervention versus pre-control versus post-intervention versus post-control. For South Africa, a pre versus post comparison was included
Full size table

Inter-country comparisons at baseline

At baseline, Jamaica and Uganda had significantly lower (worse) clinical practice scores than either Kenya or South Africa (assessment of self and peers; F = 37.1, p < 0.0001; F = 25.1, p < 0.0001, respectively), while Uganda had significantly lower scores on workplace policies and quality assurance than other countries (F = 37.4, p < 0.00001; and F = 74.8, p < 0.00001, respectively). All four countries had significantly worse scores for quality assurance initiatives than workplace policies (p < 0.0001).

A comparison of scale items revealed four main practice gaps across all countries. These were as follows: assessing the knowledge of family members to prevent HIV transmission, clients’ comfort in disclosing their HIV status to family members, family health needs related to HIV care; and referring family members for voluntary counselling and testing. A similar pattern was observed for related items assessing quality assurance initiatives and workplace policies.

The highest (best) score on the clinical practice scale was for the item “consistently using universal precautions to prevent HIV transmission with clients” (>90 % for all countries). However, quality assurance initiatives to monitor either adherence to universal precautions or the occurrence of occupational exposure to HIV were more variable, ranging within country districts from 47.2 to 80.7 % and 56 to 81 %, respectively.

At baseline, stigma scores on both sub-scales were significantly different across countries. Post hoc analysis indicated that Uganda and Kenya had worse scores on the nurses’ stigmatising scale than South Africa. On the nurses being stigmatised scale, Jamaica had significantly better scores, while Uganda had significantly worse scores than comparator countries. Nurses were significantly more likely to report being stigmatised than stigmatising patients (p < 0.0001) in all countries except Jamaica, where the reverse relationship was found (p = 0.002) (see Table 8).

Table 8 Mean scores and occurrences for the two stigma sub-scales. For Jamaica, Kenya and Uganda, the following within-country comparisons were calculated: pre-intervention versus pre-control versus post-intervention versus post-control. For South Africa, a pre versus post comparison was calculated
Full size table

Pre-post differences

South Africa was the only country with a significant improvement in clinical practices from baseline to follow-up (p = 0.0002) (see Table 7). There were statistically significant improvements in quality assurance and workplace policies for Jamaica and South Africa (p = 0.026 and p = 0.001 for quality assurance and p = 0.025 and p = 0.031 for workplace policies, respectively). In Jamaica, significance levels reflected slight improvements in the intervention group in combination with deteriorations in the control group. Both Kenya and Uganda experienced a significant deterioration in quality assurance scores in intervention relative to control districts from pre to post periods. In Uganda, the same significant decline was seen for workplace policies.

Using the dichotomised score for stigma, both Jamaican and Kenyan participants in intervention institutions reported a significant decrease in pre to post instances of stigmatising HIV patients compared to those in control institutions (see Table 8). Kenyan and South African nurses reported a decline in instances of being stigmatised between pre and post periods. However, in Uganda, the control group reported a significantly larger pre-post reduction in being stigmatised than the intervention group. Overall, there was a downward trend in reports of nurses stigmatising patients in both intervention and control groups in Uganda and Kenya and in intervention groups in South Africa and Jamaica.

Multivariate analysis

There were 24 Jamaican institutions and 24 Kenyan institutions with data for both periods. The intervention effect was not statistically significant for any of the measures. Uganda was not included in this analysis due to the large amount of missing data at follow-up resulting from an administrative error, specifically, data not being collected from provincial/district hospitals (level 4). South Africa did not have a control group due to budgetary constraints and was therefore also excluded from this analysis.

Process evaluation

Hub sustainability and turnover rates

Twelve leadership hubs were set up in intervention districts across four countries. Eleven of 12 hubs remained active through to the end of the project, an average span of 3.8 years (one hub in South Africa became inactive before the project ended, after 2.8 years). The average turnover rate for hub members across all countries was 10.1 %. In the last 18 months, Kenya and Jamaica experienced the highest turnover rates. Primary reasons for turnover were a job transfer followed by being too busy or losing interest. The proportion of hub members reported as active in the final year of the programme ranged from 42.2 to 88.2 %. The proportion of hub members at follow-up who had been members since baseline ranged from 65.0 % (Uganda) to 81.3 % (Jamaica).

Hub training

Training progressed at different rates across countries with some locally driven variations in the sequence and selection of workshop topics, as described earlier. Not all hub members participated in all training workshops, primarily due to turnover. Each workshop was presented once, so members who were unavailable on the workshop day or who joined the hub at a later date missed that formal training session. During their second year of operation, all hubs developed district-focused health action plans targeting gaps identified in the baseline findings. They were encouraged to develop plans that included all institutions in the district, not just those that had been sampled for data collection. However, some hubs decided to focus their action on a particular tier of health facilities within their district (e.g. health centres). Hub members sometimes requested additional analyses from the research team to inform their action plans. These plans focused on issues including improving infection control; providing support activities for HIV-positive nurses; sensitising nurses to HIV-related stigma; and increasing nurses’ awareness of existing HIV workplace policies. Some plans included activities aimed at health issues other than HIV. Despite a common template that was used to guide hubs in developing plans, we observed unevenness in the specificity and level of activity proposed across action plans.

In the final year, 11 hubs completed a total of nine evaluation projects, all with a quality assurance focus (see Table 9). Hubs worked either singly or with other hubs in their country as they completed data collection, analysis and report-writing. The dissemination of findings from these projects took place in the last few months of the programme and in some cases continued after the programme ended. The proportion of intervention institutions that participated in the evaluation projects ranged from 35.3 % (Jamaica) to 88.2 % (Kenya).

Table 9 Leadership hub evaluation projects
Full size table

By the end of the programme, leadership hub members in each country reported statistically significant improvements in their self-rated capacity to identify and act on gaps in clinical care and health system issues for each of the seven capacity dimensions assessed (see Table 10). Pre-post change scores ranged from 3.59 to 4.59 (on a 10-point scale) for the eight hubs included in this analysis. The largest change score was reported for the competency “initiating and undertaking an evaluation project”. Data from two hubs were excluded from significance testing, as only aggregated data, rather than individual data, were provided by the RA. Competency scores from these two hubs showed the same pattern as that for the other hubs. Two other hubs were excluded; one in South Africa that was inactive and one in Uganda, whose members did not respond.

Table 10 Active leadership hub members’ self-assessed changes in capacity
Full size table

Discussion

We were able to set up, train and sustain leadership hubs in all countries over the 4-year project. However, limited changes in outcomes were observed in the study districts. We consider reasons for these overall findings and differences across countries, in the sections below.

Baseline comparisons

At baseline, our study findings showed that all study countries had some significant gaps in evidence-informed HIV care, quality assurance initiatives and workplace policies. Family care, referrals and information exchange between health care settings were the most common gaps reported in all countries. It is not surprising that in busy and understaffed clinics and in-patient units, family-related issues get less attention. Involving family members in patient care is compounded by confidentiality issues related to HIV disclosure to patients’ partners, and concerns about stigmatisation [9, 88]. Nurses may have assumed that family dimensions of HIV care were being addressed through voluntary counselling and testing programmes, or by other members of the health care team. There are also country-specific reasons for these findings. For instance, nurses working in Jamaica’s health centres have little interaction with HIV patients after they are diagnosed, as follow-up care is provided by doctors and adherence counsellors.

There were gaps in workplace policies and quality assurance programmes as well. A number of these mirrored the gaps observed for clinical care. There was also more variability across countries with respect to these policies and programmes. These reflect, in part, considerable differences in the HIV policy context [89] and socio-political influences such as legislation and norms regarding homosexuality.

Pre-post changes in clinical practices, workplace policies and quality assurance

Despite significant improvements in the self-rated capacity of hub members in all countries, there were only small, although statistically significant, pre-post improvements in workplace policies and quality assurance in Jamaica (pre-post, intervention versus control group comparisons) and modest improvements in clinical practices, workplace policies and quality assurance in South Africa (pre-post comparisons). There were small but statistically significant improvements in pre-post scores for nurses stigmatising patients in Jamaica, Kenya and South Africa and for nurses being stigmatised in Kenya, but no statistical adjustment was made for either multiple testing or clustering effects. Multivariate analysis models for Jamaica and Kenya that adjusted for differences across WHO institution levels yielded non-significant results. In Uganda, improvements in the control group for both stigma and quality assurance exceeded those in the intervention groups. The biggest pre-post changes in the intervention groups were seen in South Africa. However, there was no control group in this country.

Co-interventions

The improvement in stigma scores across all countries (nurses’ reports of both stigmatising and being stigmatised), in both intervention and control groups, suggests a trend towards less HIV-related stigma. Stigma reduction has been a priority for many HIV programmes. In Jamaica, for example, an island-wide stigma reduction campaign was introduced during the study period by the Pan Caribbean Partnership against AIDS [90]. In Kenya, Uganda and South Africa, numerous professional development training programmes targeting service delivery issues including voluntary testing and counselling, stigma, access to HIV care and integrated models of care were offered by Ministries, development aid and non-governmental organisations during the study period [4, 12, 19, 20, 31, 34, 42, 45, 47]. Changes in national level policies on access to HIV care were also evident during the study period with some of the biggest change taking place in South Africa [83]. We did not systematically track these co-interventions. It seems likely that these other interventions may have led to some of the improvements observed, particularly in South Africa.

Hub intervention

Since trust among partners is a critical prerequisite for effective PAR [91], this may have influenced how quickly the intervention took hold and what activities were prioritised by hubs. Pre-existing relationships between the institutions where research project leads for the hub intervention were employed and the study districts varied substantially.

Our process evaluation of the hubs highlights some strengths of the intervention. The hubs comprised a corps of nurses and health system stakeholders committed to improving HIV prevention and care-related services and policies. Hub members in all districts reported capacity improvements. They gained introductory skills in research, evaluation and influencing workplace policies. However, our experience working with hub members to develop and refine their evaluation projects suggests that in some cases, their self-reported confidence in using these skills may have outstripped their actual ability to use them independently.

A core group of hub members remained engaged throughout the 4-year project period and 11 of the 12 hubs remained active at the end of the programme; 71.7 % of those who were still in the hub at the end of the study had been involved in the hub since the outset. Country directors were able to oversee the recruitment of hub members and replace those who left early, suggesting that the opportunity to participate as a leadership hub member was valued and hubs as an entity were sustained. The substantial proportion of unremunerated time hub members spent attending training and meetings, and developing and implementing both their action plans and evaluation projects also indicates that this opportunity was viewed positively.

Overall, despite these apparent strengths of the leadership hub intervention, district-wide improvements in practice and policy outcomes were not seen. There are two major plausible explanations for these results: the strength and intensity of the intervention was inadequate, and/or outcome measures were not sensitive to intervention effects.

Intervention strength and intensity

Several factors diluted the potential strength of the hub intervention. Hub activities were significantly disrupted in areas where there were more acute shortages of health care providers and challenging socio-political influences. In both Kenya and Uganda, for example, major reorganisations of their health system led to the redeployment of many health workers, including some hub members. This led to some loss of momentum among hubs as health workers became demoralised as a result of these larger system changes, particularly in Uganda and Kenya.

There were delays in getting baseline data findings to the hubs for their input and feedback, and this may have initially discouraged members. Because we piggy-backed capacity building of country RAs with analysis, results were shared more slowly than initially planned and hubs were sometimes kept waiting for study findings [77]. This may have contributed to some of the unevenness we observed in the content of the action plans across countries and the initial inclusion of some non-HIV-related activities in some of the earlier plans.

While we attempted to ensure links with the formal system by establishing national advisory committees within three countries, requiring that hub members were employed in the study districts and asking hub members to seek their employers’ permission to participate in hub activities, the hubs as an entity were not a recognised formal structure of the health system. Research highlights the importance of integrated governance structures, formal decision-making authority and accountability chains within the system as dimensions of committees that can stimulate wide-scale system change [92–94]. However, these dimensions were missing from our hubs.

Hub members found their work priorities and responsibilities competing with hub time commitments. Furthermore, hub members were not paid by our programme, an approach that was different from the more common practice of non-governmental organisations in the study countries, which normally pay a training allowance or replacement wages to the employee’s institution. Coupled with the fact that leadership hubs were not a formal part of the health care system, this made it difficult for hub members to negotiate release time with their employers. This was even more challenging when districts were dealing with urgent matters such as disease outbreaks (e.g., cholera, H1N1), local natural disasters, national health campaigns, periods of civil unrest and labour disruptions. All of these put extra demands on the scarce human resources available.

Hub members set priorities for two core activities—preparing an action plan and developing an evaluation project. The former posed challenges because the hubs were informal structures, without any direct accountability within the system. This left no formal mechanism for approving their action plans and led to questions about what they were authorised to do. In contrast, the development of evaluation plans provided a turning point for the hubs. Members had to seek administrative approval of participating institutions where they decided to undertake the work. This gave a formal accountability mechanism to provide feedback and recommendations for action. However, in most instances, evaluation projects were completed just a few months before follow-up data for the study were gathered and involved a subset of sampled organisations in the intervention districts. In South Africa, the two active hubs carried out a joint project which involved a literature review and subsequent development of an assessment instrument. Actual implementation of this assessment instrument (to improve continuity of care for HIV patients) was planned for after the project period. Therefore, changes seen in South Africa are not likely due to their evaluation project.

Sensitivity of outcome measures

HIV care outcome measures for the study were chosen in advance of establishing the hubs. Therefore, they did not reflect the specific focus of the hub action plans and evaluation projects. Since the interventions were hub-driven, similar to a “community-driven” intervention [95], this put our PAR approach at odds with our quasi-experimental design and outcome measures. Yet, the quantitative baseline measures were a critical input into the PAR process. Baseline data comprised an essential set of local and evidence-informed HIV health care indicators that were specific to nursing practice, which hub members considered as they developed their action plans and decided on a focus for their evaluation projects.

Action plans developed by hubs were diffuse. Although evaluation projects were focused, they did not nearly cover the breadth of outcomes assessed. We did not impose a requirement for hub activities to be undertaken in the randomly sampled institutions where study data was being collected and more than half of the hub members came from health facilities or other institutions that were not part of the study sample. Furthermore, not all study institutions had hub members. Thus, we had a district-wide, health system improvement orientation to our sampling and measurement, while hubs chose a more targeted approach to their implementation of action plans and evaluation projects, which did not involve all sampled organisations. In retrospect, we set the bar extremely high in terms of expected outcomes, as hub members were neither working to create changes across the full spectrum of study measures nor representing or targeting all sampled study institutions.

Research limitations

There were several limitations of the research design. One of these was planned, while others emerged during the study. We did not have a control group in South Africa. This was due to budgetary constraints. A control group in this country would have helped our interpretation of the pre-post improvements we observed. Uganda experienced the biggest challenges in data collection with some notable gaps in the institutions from which data were collected during follow-up. There were several reasons for this including unanticipated changes in research assistants who had to be retrained and some other internal difficulties in that country. This further limits our interpretation of findings from Uganda and meant that we could not include that country in the multivariate analysis. Working across the four countries was a logistical challenge. We had a strong governance structure for our project which helped us operationalise the work but day-to-day communication when problems arose was difficult due to limited internet and challenging phone connections (particularly in Kenya and Uganda) at the time of the study. Our annual face-to-face meetings were critical to the work of the project and both research assistants and study investigators attended these meetings. However, providing timely support for colleagues when issues arose during field work was more challenging.

Intervention design challenges

In retrospect, this project had three main intervention design challenges. First, leadership hubs did not exist within the formal decision-making and accountability structure of the health system, which would otherwise have provided oversight, support and recognition. Second, we did not give preference to data that could be rapidly and iteratively collected, such as data from district health information systems as reported in some other large-scale evaluations of HIV programmes [96]. This would have complemented data collected by our team and provided timely, ongoing feedback to hub members on progress made. Third, we did not adequately extend the membership reach of the hubs nor did we actively recruit a critical mass of hub members in larger health care institutions. Finally, neither hub members nor their home organisations received any remuneration for participation in hub activities, and we did not offer funding for activities planned as part of hub action plans. Providing some remuneration would have allowed for more rapid implementation of hub activities, although this would fail to address either the underlying constraint of limited human resources or the informal nature of the hubs.

Future research

Although our capacity measures did capture some characteristics of leadership, we did not use a comprehensive measure of leadership among hub members. Other leadership attributes such as interpersonal relationships, future vision and the ability to manage change [97] would be useful to include in future studies. We only measured self-assessed perceptions of capacity among hub members at the end of the project. Repeated measures would allow further delineation of the pathways through which leadership characteristics develop and exert change.

Synchronising hub initiatives with other programmes targeting HIV could help leverage conditions for change and would be a useful area for future inquiry. During the final phase of the study, some of this synchronisation started to take place but we did not actively catalyse these alignment opportunities.

Although we experienced a number of implementation challenges reflecting the diversity of contexts in which the study was undertaken, we think that comparative research across district and country sites remains important. It is through such comparative studies that insights can be gleaned on how interventions interact with dynamic health care systems and how interventions need to be adapted and tailored to local conditions. In this study, action plans and evaluation projects reflected local concerns and the situational analysis undertaken through the PAR process. We provide a more detailed description of how these contexts impacted on the PAR processes and the experience of leadership hub members within the study in another paper [76].

Contribution to new knowledge

This is the only study we are aware of that has attempted to link a PAR and leadership-oriented intervention to outcomes at the district health system level. The study highlights promising elements of an intervention to improve evidence-informed nursing care practices and policies by strengthening nurses’ leadership and policy engagement capacity but illustrates the challenges of achieving health system impact.

Limitations

Our study had several limitations. We were unable to include control districts in South Africa due to budgetary constraints. For some indicators and especially in Kenya, baseline scores on some clinical practices and workplace policies were quite high, with limited room for improvement.

Hub members provided a self-assessment of their pre and post capacity at the end of the study. Had we assessed their skills at the outset of their involvement on hubs, their baseline results may have been different. We do not know if capacity gains were sustained over time.

We did not include quantitative indicators of management, leadership or governance at the level of the health care institutions, other than a human resources management rapid assessment tool [98]. Results from the latter tool, which was completed by a purposeful sample of respondents, are being written up in a separate manuscript. Some dimensions of governance were examined by leadership hubs in their evaluation projects.

Conclusions

PAR, with collaboration between nurses and decision-makers, can bring to light gaps in the health care system and identify ways to improve clinical practice and care. Leadership hubs comprising people capable of and committed to change and provided with capacity building and mentorship can collectively identify issues and act on strategies that may improve practice and policy. It is apparent that targeted change strategies are a more realistic short-term expectation of leadership hubs than district-level health system improvements. Funding hub-led evaluation projects created a necessary mechanism for hub reporting, feedback and accountability within the health care system. This type of mechanism is essential for formalised systems’ change processes.

Overall, leadership hubs did not provide the necessary force for nurses to improve HIV care in their districts. If entities such as leadership hubs are to succeed, they must be integrated within district health authorities as participatory policy and practice mechanisms and become part of established formal, legal organisations (such as nursing associations or academic institutions) in order to regularise and sustain them as a means to improve health system performance.

Disclosures

In 2007, a large multidisciplinary team of researchers and decision-makers from Canada and five LMICs (Barbados, Jamaica, Kenya, Uganda and South Africa) received funding to implement a PAR programme entitled “Strengthening Nurses’ Capacity for HIV Policy Development in sub-Saharan Africa and the Caribbean.” One year after programme funding was received and prior to any data collection, Barbados withdrew from the programme, with the four remaining partner countries continuing [77].

Abbreviations

AIDS, acquired immune deficiency syndrome; ANOVA, analysis of variance; ART, antiretroviral therapy; CHWs, community health workers; HIV, human immunodeficiency virus; HSD, honestly significant difference; LMICs, low- and middle-income countries; PAR, participatory action research; RAs, research assistants; SD, standard deviation; WHO, World Health Organization