Introduction

Non-communicable diseases (NCDs) were predicted to account for over 70% of all disease burden in developing countries in 2020 as compared to just under 50% in 1990 [1]. The current burden of NCDs accounts for 71% of all global deaths (41 million people) each year [2]. Annually, 15 million NCD deaths occur between 30 and 69 years of age and over 85% of these "premature" deaths occur in low- and middle-income countries (LMICs) including Uganda [2]. In sub-Saharan Africa, between 1990 and 2017 the proportion of total disability adjusted life years (DALYs) attributable to NCDs increased from 19 to 30% of the total burden [2]. Therefore, health systems (HS) in these countries face the need to undergo a transition from services largely fashioned on managing communicable diseases like malaria, HIV/AIDS, or tuberculosis (TB) to services that must become able to address both these infections and the increasing burden of NCDs. Due to the urgency of care, the double burden on the already limited available resources means that curative services often get prioritised over preventative ones. Additionally, in LMICs, many NCDs including most cases of hypertension remain undetected due to the lack of active screening efforts [3, 4].

In Uganda, the prevalence of one or more modifiable NCD risk factors is over 94% indicating that NCDs may be largely preventable [5]. Despite the call to global action taken by the UN General Assembly resolution on NCDs control and prevention in 2011 [6], there is still limited domestic and global investment in addressing NCDs. Furthermore, the required response to NCDs among the poorest countries requires the introduction of integrated health service delivery strategies [7].

In 2016, we concluded a large cluster randomised trial to improve NCD care for hypertension (HT) and type-2 diabetes (DM) at primary care facilities in Uganda and Tanzania (the UK-MRC funded East African Chronic Disease Programme, EACDRP) [8]. The trial demonstrated significant improvements in NCD service readiness, with large differences between intervention and control facilities in the availability of functional basic equipment and healthcare worker knowledge. For example, in Uganda, the mean performance score in the intervention facilities was nearly double that in the control (74% vs 43%) and similarly 95% of these intervention facilities provided good quality NCD care according to national guidelines compared to only 8% in the control arm [8].

A comprehensive definition of sustainability of a health services intervention includes three components: (i) continued benefits to those who received the services when the interventions started and to new participants after the supporting funds were discontinued, (ii) continued implementation of intervention activities through a responsible organisation following discontinuation of financial support and (iii) community empowerment to improve their health by continuing intervention activities after its end [9]. Several efforts were made to ensure that NCD services were sustained after the EACDRP trial. These included close involvement of the ministerial and local governance structure in study activities, handover of important intervention resources (e.g., documents, equipment and up to 9 months’ buffer supply of NCD drugs at the end of the trial). The study also encouraged a patient-led initiative to form patient clubs which promoted peer support and contributions to a small-scale communal fund to assist patients with their treatment, by procuring drugs or supplies with a high stock out rate, e.g., bendrofluazide, metformin or glucose test strips over periods when the usual freely provided health facility (HF) supplies were interrupted.

The EACDRP created an excellent platform to assess medium-to-long term sustainability (which we defined as a period of 2 to 5 years after the end of research funding support) of a successful health system NCD intervention on the management of HT and DM within primary care settings in Uganda, the MeLoHanD study. We used this platform to re-evaluate health worker (HW) knowledge, and HT and DM service availability and readiness of HFs, and the quality of care experienced by patients at these HFs, at about 4 years after the research-related funding support had ended.

Methods

Study setting

Between 2013–2016, a cluster randomised trial of a health system intervention was conducted to improve NCD care in 38 primary care facilities and 6 referral facilities in Uganda (the EACDRP trial) [8]. The intervention package included the following components: training of health workers; provision of simple clinical management algorithms and patient registers in line with national guidelines; provision of essential NCD care drugs; active outpatient screening; promotion of NCD awareness and screening in the community outreaches, and regular support supervision visits to monitor the intervention conducted jointly by district and project staff.

Fig. 1
figure 1

Map showing the distribution of participating health facilities across Mpigi and Wakiso Districts in Uganda (Developed using GPS visualizer.com with map data from OpenStreetMap.org, relief from ESRI/ArcGIS)

HC Health centre levels II, III, IV, Hosp Hospital

Full size image

At the end of the research support in 2016, an evaluation was conducted to assess HF service readiness and quality of patient care that the intervention project had achieved [8]. This involved a detailed inspection of each of the intervention and control facilities, a written test of HWs’ knowledge at each facility, and a survey of a random sample of 4 NCD patients from each facility. The surveys used standardised tools and questionnaires.

Fieldwork for the current MeLoHanD study was conducted between January and December 2020 in all 19 former intervention facilities and 3 referral facilities, in two central districts of Uganda: (a) Wakiso district, which forms a horseshoe shape around the capital city of Kampala (Fig. 1) with an urban, peri-urban, and rural population of 2.5 million [10]; (b) Mpigi district, which lies just southwest of Kampala along the shores of Lake Victoria (Fig. 1) and has a population of 250,000 [10]. The population is largely peri-urban and rural which is mainly engaged in subsistence farming, fishing, and artisanship.

Description of health facility levels in Uganda

The primary health care system of Uganda is tiered along the politico-administrative organisation of the country (Table 1) and is overseen by the district health office, led by an experienced medical doctor, who co-ordinates resource distribution and staff deployment [11] to health centres (HC) II, III, IV and district hospitals (Fig. 1). Several districts form a region which is served by a hospital that can provide specialist care. HCIIs and HCIIIs which may include some private-not-for-profit health facilities are expected to diagnose and manage uncomplicated chronic disease cases including DM, HT, asthma, and HIV infection. HCIIs should also be able to diagnose DM, but usually refer DM patients to a HCIII or higher level facilities [12].

Table 1 Description of the levels of public health service delivery in Uganda
Full size table

Randomisation strategy applied during the original trial

Of the 38 Ugandan lower-level HFs (19 in each arm), 22 were randomised individually (as singletons) while the remaining 16 HCIIIs and nearby HCIIs (within a 10 km radius) were randomised jointly as 8 pairs, to minimise potential contamination owing to their proximity. In Uganda, hospitals and HCIVs are referral HFs for lower-level HFs in their catchment area. To ensure consistent care delivery to any patient referred to them from an intervention facility, all the 6 referral HFs (hospitals/HCIVs) in the project area therefore received the intervention without randomisation [13]. For each community in our study, the categories used for urban, peri-urban, or rural localisation followed the classification used by the Uganda’s 2010 national population census [14] and by the Uganda Demographic and Health Survey, 2011 [15].

Study design

Our follow-up evaluation study (MeLoHanD) was a comparison of data from two cross-sectional surveys conducted at two-time points: the end of the EACDRP trial in 2016 and 4 years after the end of the trial, in 2020. We could thus evaluate the durability or sustainability of the service availability and readiness at former intervention HFs, as well as the health service-centred quality of care and patient experience (QoCE) over the 4 years without research-driven support.

The MeLoHanD study was conducted in 3 randomly selected higher-level facilities (HCIVs) of the 6 referral units, and all the 19 lower-level facilities (10 HCIIIs and 9 HCIIs) that constituted the original intervention arm of the EACDRP trial described in the section above (Fig. 1).

Of the 3 randomly selected HCIVs, 1 HCIV (peri-urban) was selected from Mpigi district while 2 HCIVs (1 peri-urban, 1 rural) were selected from Wakiso district. There were 7 lower-level facilities (4 HCIIIs, 3 HCIIs) from Mpigi and 12 lower-level facilities (6 HCIIIs, 6 HCIIs) from Wakiso district. Of these, only Wakiso district (Entebbe) had urban facilities (1 HCIII, 1 HCII) while the remaining 17 facilities (9 HCIIIs, 8HCIIs) were rural (Table 2).

Table 2 Distribution of health facilities by district and facility level
Full size table

A study pilot to test study tools and prepare the study team for their field work was done in two health facilities (an urban HCIII and a peri-urban HCII) prior to this evaluation. Findings from these pilot HFs were not included in the analysis for the MeLoHanD study.

To evaluate service availability and readiness, each HF was inspected using a modified WHO Service Availability and Readiness Assessment (SARA) tool [16]. In addition, a clinical knowledge assessment was administered to all health workers present at the HF on the survey days. This is a validated knowledge test that has been used previously in East Africa [17, 18].

Selection of participants for the MeLoHanD study

For the quality of care and experience (QoCE) assessment, patients were interviewed using a previously used tool as well as any available patient records to check consistency in responses. Twenty active patients who had been in care for at least 3 months at each HF were randomly selected from all HT and DM patients on the patient register. The study team made a pre-survey visit to each HF to confirm the survey days, prepare the HF staff, get the contacts of the identified individuals, and invite them on the survey days. We applied a restricted random selection approach to ensure adequate balance of men and women, and of HT and DM patients. Men and DM patients were found to be heavily under-represented in the study pilot.

All 20 selected patients were invited to visit the HF for participation in the survey. A transport refund was given to those that presented on the survey day. Efforts were made to contact those that did not present on the scheduled days to encourage their participation. We aimed to establish reasons for non-participation. Defaulters were not replaced. To ensure adequate statistical power for the study four times more patients were sampled in 2020 than in 2016. All assessment tools were identical to the versions used for the evaluation of the EACDRP intervention in 2016.

Assessment of service availability and readiness (SAR)

The modified SARA tool assessed five aspects of service delivery elements (listed below). A HF could obtain up to 10 points for each element, so that a maximum of 50 points could be achieved.

  1. i.

    Availability and functionality of essential equipment (e.g., blood pressure (BP) machine, weighing scale, stadiometer, glucometer/urine dipsticks, stethoscope, patient register book, HT/DM screening logbook, referral register book, health education record).

  2. ii.

    Availability of drugs and other consumables (for the treatment of HT and DM and whether drugs were in line with guidelines and in sufficient stock).

  3. iii.

    Quality of records on patients with NCDs (guidelines, essential demographic data, clinical observations, information on diagnosis, treatment, referral and follow up).

  4. iv.

    Healthcare utilisation at facility by patients with NCDs (number of HT and DM patients, evidence for increase in utilisation over past year).

  5. v.

    Prevention of NCDs and their complications (evidence for and number of health education sessions given, evidence for active screening for HT or DM among those that presented with other conditions e.g., back pain evidence of outreach activities).

Details of the adapted SARA tool used are available from Supplementary table 2.

Health worker knowledge was assessed by means of a supervised self-completed multi-choice questionnaire that was based on 3 clinical case scenarios on HT, DM, and HIV infection. For each scenario, the assessment also looked at five elements: 1) essential diagnostic steps (symptoms, signs, tests); 2) risk factors for each of these 3 disease; 3) complications; 4) treatment regimens; and 5) guidelines for referral. The HIV case scenario was included for comparison, as earlier knowledge tests had indicated that HWs in East Africa had generally good knowledge on HIV case management [17, 18]. Each scenario was worth 10 points, so a HW could score a maximum of 30 points.

The inspection score from the SARA tool and the scores from the health worker clinical knowledge assessment were combined into a single aggregate facility performance score (FPS). The score from the SARA tool was converted to a percentage and scaled to a maximum of 60 points (e.g., a facility that scored 40 on the SARA would have achieved 40/50 = 0.80*60 = 48 points). Similarly, HW knowledge scores for each facility were totalled and converted to a percentage, then scaled to a maximum of 40 points. For example, if there were 5 HWs who accumulated a total of 132 points out of 150 (5 × 30) possible points (equivalent to 88%), the facility received a total of (40*0.88) = 35 points (rounded to the nearest whole number) for the knowledge assessment. The SARA and knowledge assessments points were then combined into the FPS with a maximum score of 100. Using the above example, the FPS would then be 48 + 35 = 83 out of a possible maximum of 100 points.

Additional details on health worker characteristics are available from Supplementary table 1.

Assessment of quality of patient care and experience (QoCE)

The QoCE assessment comprised five questions assessing whether: 1) the patient was diagnosed correctly; 2) treatment was provided; 3) the patient received health education; 4) the quality of reception and waiting time were acceptable; and 5) the patient was managed according to national guidelines (Table 3). The observations were summarised into a QoCE score. Each question could result in up to 2 points, for a maximum score of 10. The patient was considered to be adequately managed if their QoCE score was ≥ 7/10.

Table 3 Data collection sheet to assess quality of patient care and experience (QoCE)
Full size table

Statistical methods

Data were collected by a team comprising a study clinician or study nurse and 3 field workers, mainly via hand-held tablets using REDCap® version 7.6.3. Data from REDCap® were checked in the field by the clinician or the nurse and were then actively synced or uploaded on to study servers at the end of each day. Facility inspection data were collected on paper checklists and double entered in REDCap®. All data entry from paper sources was overseen by a senior data manager and REDCap® programmer. If necessary, queries were raised and communicated to the field team for verification.

All analyses were conducted in Stata 17.0 (College Station, Texas, USA). Continuous variables were summarised as means and standard deviations (SD), or medians and interquartile ranges (IQR). Categorical variables were summarised with frequency counts and percentages. Characteristics of health workers and patients surveyed in 2020 and 2016 were compared using the Pearson chi-squared statistic with the second-order correction of Rao and Scott to account for the clustered design.

The aggregate FPS was the primary outcome used to assess the impact of the lack of research-driven support on service availability and readiness assessment in 2020. The analysis used methods designed for pair-matched cluster randomised trials, with each HF in 2020 being treated as paired with its observation from 2016. First, the mean aggregate FPS was calculated for each HF in 2016 and in 2020, and log transformed. Then, the difference in log mean scores between 2020 and 2016 was calculated for each HF, and a paired t-test was performed on the pair-wise differences. A similar analysis was done for each component of the FPS: the facility inspection score and the health worker knowledge score.

The association between the aggregate FPS in 2020 and factors that may have influenced service quality was explored using linear regression. These factors included the functionality of patient clubs, whether the HFs had external support such as extra drug supplies from a non-governmental organisation (NGO) or community-based organisation (CBO), HF level, area (rural, peri-urban, or urban), district (Mpigi or Wakiso), and pairing at randomisation (before the start of the original trial). Initial models evaluated each factor individually and adjusted for the FPS in 2016. All variables that were associated with the 2020 score at p < 0.20 after adjusting for the 2016 score were included in a multivariable model (the ‘fully adjusted’ model).

The QoCE score, and proportion of adequately managed patients (QoCE ≥ 7), was used to assess the impact of the lack of research-driven support on the quality of patient care in 2020, using the same analysis approach as for the FPS. First, the mean QoCE score and the prevalence of adequately managed patients in each HF in 2016, and in 2020, was calculated and log transformed. Then, the difference in log mean scores, and in log prevalence, between 2020 and 2016 was calculated for each HF, and a paired t-test was performed on the pair-wise differences.

The association between QoCE scores in 2020 and factors that may have influenced quality of care was explored in an individual-level analysis using a Tobit regression with random effects to account for the correlation of multiple observations within HF. Since the QoCE score is bounded by a maximum of 10, ordinary linear regression can lead to biased standard errors. Therefore, Tobit regression is used to reduce the bias in the estimation of standard errors when the outcome is censored (bounded) [19]. These factors included the patients’ age, gender, the functionality of patient clubs, whether the HFs had external support such as extra drug supplies from a non-governmental organisation (NGO) or community-based organisation (CBO), HF level, area (rural, peri-urban, or urban), district (Mpigi or Wakiso), and pairing at randomisation. Initial regression models evaluated each factor individually and when adjusted for the facility mean QoCE score in 2016 a priori. All variables that were associated with the QoCE score in 2020 at p < 0.20 after adjusting for the mean score in 2016 were included in a multivariable model.

Results

Service availability and readiness

All 22 health facilities were inspected using the SARA tool. A total of 91 HWs from the 22 facilities were assessed on the clinical knowledge test, compared with 131 from the EACDRP trial in 2016. The lower number in 2020 was due to COVID restrictions and absenteeism. 60 HWs (66%) surveyed had also completed the test in 2016. The age distribution across the two years was similar with a mean (SD) age of 37.0 (7.9) in 2020 compared with 36.6 (9.6) in 2016 (p = 0.01). A slightly lower proportion of respondents in 2020 were female than in 2016 (65% vs 70%, respectively, p = 0.11), and a larger proportion were clinicians 27% in 2020 vs 17% in 2016), although the difference was not statistically significant (p = 0.29). Most respondents were either nurses, midwives, or nursing aides (67% in 2020 and 76% in 2016). In 2020, 47% of HWs were from HCIIIs and 28% were from HCIIs, compared with 60% and 18% in 2016, respectively, while 25% of HWs in 2020 and 22% in 2016 were from HCIVs (p < 0.01) (Table 7).

The mean aggregate FPS from the facility inspection and clinical knowledge assessments in 2020 was lower than in 2016: 70.2 (95%CI = 66.0–74.5) compared with 74.8 (95%CI = 71.3–78.3), respectively (Table 4). However, there was no evidence of a significant difference (p = 0.18).

Table 4 Summary of facility performance scores (FPS) and five elements of service availability and readiness (SAR)
Full size table

In evaluating the separate components of the FPS score, across the 5 elements of the modified SARA tool for facility inspection, we observed a decline in mean scores of 4 of the components, ranging from 2 to 5 points: the availability of essential equipment, essential drugs, utilisation of NCD treatment services and preventive services (Table 4). The utilisation of NCD services for HT and DM was most adversely impacted, with a mean score of 3.0 (95%CI 1.8, 4.2) in 2020 compared with 8.0 (95%CI = 7.3, 8.7) in 2016 (p < 0.001). In contrast, the mean score for the quality of records showed an improvement: 9.8 (95%CI = 9.5, 10.0) in 2020 vs 9.1 (95%CI = 8.7, 9.5) in 2016 (p = 0.001).

Mean health worker knowledge scores were slightly higher in 2020 than 2016: 26.8 (95%CI = 26.1, 27.5) vs 26.0 (95%CI = 25.3, 26.7), although the difference was not significant (p = 0.11) (Table 5). There was some evidence that mean knowledge scores for HIV were higher in 2020 than in 2016 (8.5 vs 8.2, respectively; p = 0.08), but no evidence that knowledge scores differed for the other disease areas or across HF levels (Table 5).

Table 5 Summary of health worker knowledge assessment
Full size table

The FPS in 2020 was inversely associated with FPS in 2016, with mean FPS in 2020 decreasing by 0.45 points (95%CI -0.77, 0.12; p < 0.01) for every point increase in the 2016 score. After adjusting for the FPS in 2016, there was strong evidence that the FPS in 2020 was associated with location of the HF (p < 0.01) (Table 6). The mean FPS was 10.9 points lower in HFs located in rural areas (95%CI = -19.9, -1.91) and 20.6 points lower in those located in urban (95% CI -32.8, -8.42) compared with peri-urban areas. There was borderline association with a HF’s patient club functionality (p = 0.05) with moderate and low functionality associated with a mean FPS that was 5.62 (95%CI = -13.3, 2.00) and 6.82 (95%CI = -14.1, 0.47) points lower, respectively, than patient clubs with high functionality. There was some evidence of an association between FPS and facility level (p = 0.08), with mean FPS scores 1.12 points higher (95%CI = -5.64, 7.89) in HCIIIs than in HCIIs and 8.90 points higher (95%CI = -0.65, 18.5) in HCIVs than in HCIIs. There was also weak evidence that the FPS was lower in Mpigi than in Wakiso (-4.57, 95%CI = -10.7, -1.59, p = 0.11). There was no evidence of an association with post-trial external support received (e.g., from NGOs) (p = 0.22) or pairing at randomisation (0.19) (Table 6).

Table 6 Unadjusted and adjusted analysis of the effect of independent factors on facility performance (FPS)
Full size table

In the final adjusted model, after adjusting for patient club functionality, facility level, HF area, district and pairing at randomisation, the mean FPS in 2020 decreased by 0.55 points (95% CI -0.89, -0.21) for every point increase in the 2016 score (p < 0.01). None of the other factors was found to be an independent predictor of FPS but club functionality showed a weak association (p = 0.09) (Table 6).

Quality of patient care and experience

QoCE assessments were available from interviews with 332 patients in 2020 and 88 patients from 2016. The patients in 2020 were older than in 2016 (mean (95%CI) = 60.1 (58.5, 61.8) vs 55.9 (53.6, 58.1)), respectively (p < 0.01) but the gender distribution was similar across both groups (71% female in 2020 vs 65% in 2016, p = 0.22). Patient distribution across HF levels was not different (p = 0.86). More than half of patients (53%) in 2020 said they were employed compared to 29% in 2016 respectively while 40% in 2020 considered themselves to be homemakers compared to 47% in 2016. Only 7% were either unemployed or retired or reported belonging to other categories in 2020 compared to about a quarter (24%) in 2016 (p < 0.01).

There was good evidence that the mean QoCE score was lower in 2020 than in 2016 (8.72 vs 9.45, respectively, p = 0.02) (Table 7). The proportion of adequately managed patients was also lower in 2020 (88.2% vs 95.5%), but the difference was not significant (p = 0.20). When stratified by HF level, the largest decrease in QoCE scores was seen in HCIIs (a mean decrease of 0.91 points, 95% CI = -1.82, -0.003; not shown).

Table 7 Summary of patient quality of care and experience (QoCE) assessment
Full size table

There was no evidence that mean QoCE scores in 2016 were associated with QoCE scores in 2020 (p = 0.71). After adjusting for QoCE scores in 2016, there was some evidence that scores in 2020 were associated with patient gender, external support (excluding patient club support), district, and patient club functionality (Table 8). Mean QoCE scores in 2020 were 0.43 points higher (95%CI = -0.08, 0.95; p = 0.10) in women than men, 1.35 points higher (95%CI = -0.28, 2.41; p = 0.02) in HFs with external support than those without, and 1.04 points higher (95%CI = -0.09, 2.17; p = 0.08) in Wakiso than Mpigi district. Mean QoCE scores in 2020 were lower in HFs with low or no patient club functionality compared with those with high functionality (-0.92, 95%CI = -2.19, 0.36; p = 0.05). After adjusting for QoCE scores in 2016, gender, patient club functionality, external support, facility level, area, and district, only parent district remained an independent predictor of QoCE scores in 2020, with HFs in Wakiso having scores 1.21 points higher (95%CI = 0.25, 2.18; p = 0.02) than Mpigi.

Table 8 Unadjusted and adjusted analysis of the effect of independent factors on quality of patient care and experience (QoCE) using Tobit regression
Full size table

Discussion

We defined facility performance as having two inputs i) health worker clinical knowledge and ii) the facility inspection with five constituent elements of service availability and readiness.

Overall, health worker knowledge was well preserved, and knowledge scores were similar across all facility levels whether by subject matter or the 3 disease case scenario questions. This indicates that over the post-trial period of 4 years, despite any decay or shortcomings in support supervision or motivation or even underutilisation of the facility; the NCD case management competence of HWs were largely sustained. Only 66% of HWs interviewed in 2020 took part in the assessment in 2016 which suggests that NCD related knowledge among new staff members was also adequate. This finding was also reinforced by a similar finding on adequate quality of care and experience which remained unchanged over this same period.

Health worker knowledge is usually measured as a pre- and post-training assessment we have had the benefit of assessing it at least 4 years after the intervention. Interestingly, most studies in SSA that have assessed task shifting with nurse-led NCD management, have shown good knowledge retention in using a qualitative framework approach [20] or the bundled education and support with text (BEST) method [21]. However, our study like others in Kenya and South Africa [22, 23], also demonstrates that a protocol-driven or clinical knowledge test approach based on national guidelines is essentially comparable.

Despite the decline in most of the elements of service availability and readiness over the period 2016–2020, there was no evidence of difference in overall facility performance owing to the strong performance of HWs on the clinical knowledge test. Of the five constituent elements of SAR, only the quality of records was preserved while the utilisation of NCD services and evidence of preventive activities declined most strongly. The availability of essential drugs and to a lesser extent the availability of essential equipment and consumables also declined, but less steeply. This suggests that the support obtained from functional patient clubs may have alleviated the performance decline to some extent, through the replenishment of consumables, repair or replacement of simple equipment and the direct purchase of essential medicines. However, with regards to the availability of essential medicines, the worst performing HFs only scored < 2 points out of possible 10 in 2020 (not shown), indicating that some HFs were more severely impacted by inconsistent drug supplies than others, and also had little to no patient club support.

The utilisation of primary care services at public facilities usually reflects the availability of essential medicines and other consumables [24,25,26], and therefore the observed decline in utilisation was expected. This decline is most likely also a result of a reduction in screening efforts, consistent with our finding that preventive activities in 2020 occurred much less frequently than in 2016. This in turn implies that only a few new NCD cases were actively detected and put into NCD care.

The association of FPS with patient club functionality (p = 0.05) and facility level (p = 0.08) was borderline. District location (Mpigi vs Wakiso) (p = 0.19) and the peri-urban vs rural or urban location of a health facility (0.23) did not affect service availability and readiness. None of them predicted facility performance independently despite a weak association with patient club functionality (0.09). However, adjusting for 2016 scores, rural and urban HFs mean scores in 2020 were 11 and 21 points lower than peri-urban HFs (p < 0.01) respectively, perhaps a chance finding. Our findings are reminiscent of a recent study from southern Nigeria which found that service readiness increased with the presence of some power sources (electricity, generators, batteries and solar), but was lower among lower-level units that did not have this support. Travel time to headquarters and rural facilities also significantly reduced indices of equipment availability [27]. We did not assess the effect of electricity or power sources on facility performance in this study. However, lower-level facilities in Uganda have at least one power source from either solar panels or public electricity [28]. The availability and functionality of the solar equipment or electricity system lies in the docket of the parent district; however, district location was not found to affect facility performance but was found to be an independent predictor of service quality instead.

Our findings also differ from the Nigerian study with regards to the urban health facility location, which in Uganda we speculate may be due to an intervening local administrative level at the municipality or local town council that usually distorts the district office’s direct influence. A district health officer or their team may not exercise the same direct supervisory oversight and authority over those units within a semi-autonomous local municipality as those outside because these usually report to the local municipality health officer instead. This gap often leads to poor district support in terms of consistent supervisory oversight and timely medical supplies or replenishments especially where the municipality or local authority administration is weak. Public health facilities in urban areas are often poorly utilised as urban-dwelling patients have more health care options such as services from commercial drug shops and private clinics [29]. Public HFs in Uganda are also generally shunned because of inconsistent drug supplies, health worker absenteeism and poor overall supervision [30].

The weak association of FPS with HF level in the initial models where higher-level units performed up to 9-points better than HCIIs (or HCIIIs) can be partly explained by the fact that these facilities are referral clinics and have more consistent drug supplies. They also usually have stronger patient clubs.

The quality of patient care and experience showed a statistically significant decline between 2016 and 2020 (p < 0.02). The proportion of patients receiving adequate quality of care in line with national guidelines also declined but not significantly so (p = 0.24). A U-shaped relationship between service quality and patient volume has been previously described by a service quality assessment in Ethiopia which indicated that service quality increases until a peak patient volume of 90 patients per day and then decreases [31]. Whereas we did not measure patient volume directly as related to service quality, we did measure service utilisation under facility performance which had dropped in 2020 to less than half of that observed in 2016 (8 vs 3-points; p > 0.001). It can be argued that service utilisation in 2020 was only about half of that seen when the intervention had reached its peak optimisation in 2016. This might explain our mixed findings; we would speculate that indeed some HFs had in fact optimised, and also reached their patient volume thresholds with a subsequent dip in service quality thereafter. Whilst other HFs might still be optimising or re-optimising service quality below their patient volume thresholds currently.

The quality of patient care was strongly associated with parent district (p = 0.02). Wakiso district was found to have 1.2-point higher increase in mean QoCE than Mpigi, this association appeared to stronger after controlling for all other factors. Wakiso district geographically encircles the capital city of Uganda, Kampala, and as such benefits from the better road network that radiates from the capital city in a variety of directions. This has the double effect of ensuring quicker and more regular replenishment of essential medical supplies to most of Wakiso district as well as more regular support supervision due to easier access to remote district HFs. However, this could also be a chance finding as logistical support to NCD services did not seem to differ that much between the two districts.

Regarding provider or HS quality of care studies on NCDs in SSA, one study in Lesotho within HIV clinics [32] found that about a third of patients did not have records on NCD outcomes. The main barriers to care were equipment shortage or disrepair and staff shortage which affected the organisational structure for NCD care while inadequate screening for NCDs, poor scheduling and inadequate patient education affected treatment processes [32]. This is not very different from what we found regarding the challenges to facility performance on very similar 4 of the 5 elements of service readiness. However, in our study one element i.e., the quality of records was adequately preserved over the 4 years.

Strengths

The EACDRP intervention project had provided a strong and effective NCD service intervention with fully optimised elements against which it was easy to study potential changes in service readiness or service quality over time. For example, record keeping had been optimised in intervention HFs during the trial, data quality was high in both consecutive surveys. This applied to both routine data collection tools and records e.g., patient registers.

Identical and standardised study tools were used at both time points. To avoid observed bias, care was taken to ensure that observed data collection in 2020 did not involve any staff member from the original intervention (implementation) team. Also, instead, we recruited staff from the original evaluation team, and this helped to ensure comparability of the data sets generated at the two time points. The restricted sampling technique used for the QoCE assessment, allowed us to sample up to 20 eligible patients who were invited without replacement for defaulting, and also to ensure appropriate representation of men and women and patient groups (i.e., for DM and HT). This also helped increase the power of the study.

Lastly, most post-intervention evaluations in a research or programmatic setting are usually done after 2 or 5 years, the 2020 survey was conducted about 4 years after the end of the intervention trial. This interval was sufficiently long to study mid- to long-term sustainability and durability of intervention effects.

Limitations

We were not able to collect data on service readiness and quality of care at any point of time between 2016 and 2020. We, therefore, could not assess whether the observed declines had occurred earlier or later, and whether the decline followed a linear pattern or occurred at certain points in time. Because there were no major changes in policy, infrastructure, or human resource or staff attrition in the assessed intervention facilities over this period; we assume the deterioration occurred slowly.

We also cannot ascertain whether any effect of the intervention may have in fact optimised after 2016, and whether any positive effects observed in 2020 reflected that subsequent optimisation rather than intervention durability. However, the 4-year gap between assessments is likely to be a long enough period to hone out only the durable post-trial effects.

As this study focused on evaluating sustainability in facilities that received the intervention, we did not evaluate the former control facilities for comparison; therefore, it is possible that our findings were a result of other external influences that might have alleviated a possible decline of SAR and/or QoCE, rather than of the durability of the intervention. However, there were no major changes in health policy, health facility management or community initiatives during this period, and we are not aware of any other (e.g., NGO-related) health service efforts to strengthen health service performance, neither among the former intervention nor the former control facilities. Although all primary care centres in Uganda participate in ongoing efforts of the national NCD control programme to strengthen the response to NCDs, the effect of these efforts has been limited: for example, the proportion of health workers trained “during the last year” was 23% in 2020 and 39% in 2016, and there was little difference in having “received any NCD training” (68% in 2020 vs. 65% in 2016).

Our sample size of health workers and patients in 2016 was fixed; therefore, our ability to increase the power of the study was limited. However, where possible, we increased the number of participants in the 2020 survey so that we had reasonable power (> 80%) to detect important changes in service readiness or quality of care.

Both health workers and patients who participated in the MeLoHanD study represent clusters of participants: it can be assumed that individuals from the same HF were likely to be more similar with regards to the variables that we determined in this study than a random selection of independent individuals. We accounted for this clustered design in the statistical analysis.

However, there were some notable differences in the study populations between the two surveys e.g., there were fewer health workers at HCIIs in 2016 than in 2020 (18% vs 28%) and more at HCIIIs (60% vs 47%) which could mean that HCIIs were over-represented or HCIIIs were under-represented in the 2020 health worker sample. This could be partly explained by the fewer number of health workers that were available for survey in 2020 due to COVID restrictions or related absenteeism. In spite of this, there was generally a more equitable spread in 2020.

Conclusions

The clinical knowledge of HWs was sustained over 4 years after the end of the intervention project. This suggests that within similar primary care settings in LMICs, training effects resulting from the original intervention are likely to be maintained for some extended period, even as service availability and readiness decline in the absence of funding support. Nevertheless, we recommend that refresher training of health workers should be routinely provided within a public health care system until such a time when other elements of service readiness are sufficiently optimised.

As expected, overall facility performance was negatively affected because of the interrupted supplies and a decline in regular supervision. Some intervention components declined more strongly in urban or rural than peri-urban settings; and we speculate that this depended on administrative impediments and supervision. Lastly, the availability of functioning patient clubs seemed to have a positive effect with regards to service availability and readiness.

Surprisingly, although the overall quality of care received and experienced by HT and DM patients had declined, the proportion of those receiving adequate care according to national guidelines had not substantially changed even 4 years after the end of funding support to the intervention. District oversight was important to maintaining service quality.

More research in similar primary care settings is needed to clarify the role of well-functioning patient clubs for the sustainability of NCD care: what is their mean time-to-full optimisation and how and when do they begin to affect the continuity of NCD or other chronic care services in the absence of any funding support.

Sustainability or durability of an NCD intervention in similar primary care settings may remain achievable despite the funding instability and logistical challenges that follow withdrawal of research or programme support. Early during an intervention trial, health system managers and researchers should jointly plan how to sustain the intervention beyond the end of the project if the trial demonstrates its effectiveness.