Background

Global, regional, and national health initiatives have responded to the HIV/AIDS pandemic by introducing antiretroviral therapy (ART) to ever-increasing numbers of affected patients. The success of ART relies on life-long, high levels of medication adherence to maximize clinical effectiveness and to minimize the potential population risks associated with the development of drug resistance. Evidence for a goal of maintaining adherence rates of 90% [1, 2] to 95% [3] have been presented, although recent evidence has questioned the validity of that standard [4, 5]. Reviews of over 50 years of research have shown that achieving medication adherence rates of over 80% for treating a range of chronic illnesses has been problematic, even in resource-rich countries [6, 7]. Achieving high rates of adherence in resource poor settings is a serious challenge and performance needs to be closely monitored.

The ability to measure treatment adherence accurately is critical to identifying adherence problems and improving adherence practices. Common methods for measuring adherence to antiretroviral (ARV) medications involve using patient self-reports [8–10]; pill counts [11, 12]; pharmacy dispensing data [7, 13–15] and, primarily in research settings, electronic medication monitors [16–19]. Each method has strengths and limitations, but individually, each method fails to capture important dimensions of ARV treatment adherence [12, 20–22]. All methods produce measurements associated with clinical outcomes in research settings; however, to our knowledge, none of the approaches to measuring adherence have been tested for feasibility and validity using routine data from the array of current treatment program settings in Africa.

Countries and global donors have agreed on the importance of supporting and monitoring adherence among patients on antiretroviral therapy [23], although they have been unable to reach consensus on optimal strategies for measuring or improving adherence at the patient or system level [23, 24]. The assessment reported here was designed to investigate the feasibility of using routine data collected in a systematic way in a wide variety of facilities to construct standardized indicators that reflect the performance of facilities in maintaining their patients on therapy. In a companion article, the validity of these indicators is tested and reported [25].

In 2006, the International Network for the Rational Use of Drugs Initiative on Adherence to Antiretrovirals (INRUD-IAA), in collaboration with the national AIDS control programs from Ethiopia, Kenya, Rwanda, Tanzania, and Uganda, carried out a survey that assessed how existing ART programs and health care facilities tracked patient adherence and treatment defaulting [26]. ART programs had started in Uganda as early as 1991, in Rwanda 1999, Kenya 2001, Ethiopia 2003 and Tanzania 2004. However, the survey showed that many programs and facilities had no processes in place to measure treatment adherence or defaulting at either the patient or program level, and among those facilities that did conduct measurements, definitions and data collection practices varied widely, although they routinely collected useful data [26] .

Using these survey results along with measurement approaches described in the adherence literature [11, 12], INRUD-IAA collaborators defined a draft set of core indicators for monitoring adherence and attendance that could be calculated from data routinely available in ARV treatment clinics and developed standardized methods to collect data and calculate indicators.

The draft adherence indicators and their data sources include:

  • Self-reported, short-term adherence: Percentage of patients that report perfect adherence and the average percentage of doses of ARV medicine they report having missed during the previous three days, using data from interviews with patients exiting from a clinic and from self-reported adherence data recorded in clinical records.

  • Dispensing-based long-term adherence: Percentage of days covered by ARV medicines dispensed over the last six months or a year and percentage of a cohort of patients who had a gap of at least 30 days between dispensing, assessed from pharmacy dispensing records.

  • Consistency of visit attendance: Percentage of patients attending appointments on or before the scheduled day and percentage attending within three days after scheduled appointments, taken from clinic appointment logs.

  • Pill count-based medium-term adherence: Average percentage of days patients took ARVs as expected based on pill counts as noted in clinic or pharmacy records.

Methods

Overall Study Design

We conducted field studies in Ethiopia, Kenya, Rwanda, and Uganda between October 2006 and June 2007 to evaluate the feasibility and the reliability of collecting data from patient interviews and medical and pharmacy records to apply to a set of draft adherence indicators for assessing adherence to ART in resource-limited settings.

Facility Sample

In each country, the sample included 20 health facilities offering different levels of care. Eligible HIV/AIDS treatment facilities were required to have had at least 100 patients receiving ARVs a year before the study. We first stratified all such facilities in each country by region, and then chose study facilities randomly, excluding facilities that were too difficult to reach or that were the focus of other adherence-related initiatives.

Data Collection

Trained data collectors interviewed patients exiting the facility, examined a randomly selected sample of patient records, and interviewed facility staff. Each data collection team included one team leader and three or four data collectors. Data collectors were practicing pharmacists or physicians or senior-level students. The process was designed so that one team could spend one day collecting the data in each facility.

Patient exit interviews

We sought to interview a purposive sample of 30 patients on ARVs as they exited from each facility following their clinic visits on the day of data collection. Data collectors first asked patients for consent and then asked them a standard question about treatment adherence over the previous three days. They also collected information on factors potentially affecting adherence, such as the time spent getting to clinic, time spent waiting in the clinic, and whether patients knew how to take their medicines correctly.

Retrospective sample of patient records

In each facility in Kenya and Rwanda, data collectors examined 100 randomly chosen pharmacy records from patients who had attended the clinic 13 months (Kenya) or 7 months (Rwanda) before the survey, and abstracted days of medicine each patient had been dispensed over the succeeding 12 or 6 months, respectively. In a second sample of 100 records of patients who attended the clinic 4 months prior to the day of data collection, dates of their next clinic appointment were abstracted. Data collectors abstracted pill count and self-reported adherence data from all records where available. Following a review of the experience in the first two surveys, data collectors in Uganda and Ethiopia abstracted records of a single cohort of 100 randomly selected patients in each facility who had attended the clinic 7 months before.

Additional information extracted from clinic records included demographic (age, gender) and clinical data (World Health Organization [WHO] stage at initiation, length of time on ARV therapy, and CD4 and viral load test dates and results, where available).

Data Analysis

We entered all data into formatted Microsoft® Office Excel 2003 spreadsheets and calculated the percentages and means for each indicator for all patients and for each facility, as appropriate. For each country, for each facility, we calculated the summary statistics for all patients sampled. Within country we took the median facility indicator values, as well as the maximum, 75th percentile, 25th percentile, and minimum. Using facility median values helped prevent skewing the summary results with outlying values, since our dual purpose was to identify individual problem facilities and to assess overall performance across facilities. For the dispensing-based indicators, we examined results for the entire sample, including patients who were lost to follow up during the observation period. The data were then entered into the Statistical Analysis System (SAS)® 9.1.3 and we calculated Pearson correlation coefficients between all facility-level summary indicators.

When the four pilot surveys were completed, we revised the methodology based on the overall quality of the data that facilities routinely maintain in medical and pharmacy records, the difficulty of collecting the data, the observed results, correlations among the facility-level summary indicators, and utility of the resulting information. We simplified the methodology to focus on five core adherence and attendance indicators. These five indicators were validated in a later study by comparing them to changes in clinical outcomes observed in patients newly initiating treatment; the results are reported in a companion paper [25].

Ethics committee approval

In three of the countries the national AIDS control program gave permission to conduct the surveys and collaborated in the study as part of their quality improvement processes. In Ethiopia this approval was given by the Drug Administration and Control Authority (DACA). All patients interviewed gave verbal consent and were free to stop the interview at any time. Patient and staff interviews were conducted privately and all results were reported anonymously.

Results

Characteristics of Health Facilities and Patients

Characteristics of the facilities included in the sample can be seen in Table 1. Overall, we collected data from medical and pharmacy records in 78 of 80 targeted health facilities. A number of facilities did not have the target number of 30 patients attending on the day of data collection, so we interviewed a median of 21 per facility. In Ethiopia and Uganda, the median number of records examined was 100. With two samples of records in Kenya and Rwanda, the median numbers of recent (4 months before data collection) and long-term (7 or 13 months before data collection) patient records examined were 70 and 59 respectively.

Table 1 Characteristics of health facilities and patients included in the sample
Full size table

Problems in data collection

Although most data were available in the sample facilities, challenges in collecting data are outlined below.

Patient exit interviews

Interviewers targeted patients on ARVs except those starting ARVs on the day of the survey. In all cases the interviews were voluntary and very few patients refused. However, in some facilities, few patients attended on the day of the survey.

Sampling patient records

Sample patient records were usually identified by linking data from the attendance register for patients on ART in a given prior month with clinical and pharmacy records. In some cases, the attendance register did not distinguish between those on ART and those not on ART, so the sample of current patients was selected in the most feasible manner from the ART Initiation Register, from sequentially numbered clinic or pharmacy records, or from a list of all patients currently in treatment.

Days of medicine dispensed

In most cases, it was feasible to determine the number of days of medicines dispensed over the previous six months. However, in four facilities there were no dispensing records and the facility did not regularly prescribe the same number of days of medicine, so this indicator was not possible to measure.

Self-Reported Adherence in Exit Interviews

Ninety four percent of all patients reported perfect adherence. When analyzed by country, the percentage varied from 91% in Rwanda to 96% in Kenya and Ethiopia (Table 2), and when analyzed by facility, the median facility-specific percentage of patients with 100% self-reported adherence in the previous three days was 100% in Rwanda and 96.7% in the other three countries. However, there were facilities in which the percentage of patients with self-reported perfect adherence was as low as 60%. The overall mean percentage of doses patients reported taking across facilities was 99.4% or above; however, the facility mean was as low as 85%. (Figure 1 and Table 3).

Figure 1
figure 1

Distribution of facility-level percentage of self-reported full adherence by country. Note: Horizontal line represents the value of the outcome in the median facility in each country; box covers the values from the 25th to 75th percentile of facilities; diamonds represent values in individual facilities that lie above the 75th percentile or below the 25th percentile.

Full size image
Table 2 Patient indicators of adherence and clinic attendance across health facilities in four countries based on exit interviews and record review
Full size table
Table 3 Facility-level indicators of adherence and clinic attendance across health facilities in four countries based on exit interviews and record review
Full size table

Dispensing-Based Adherence Measures from Patient Records

Analyses of retrospective dispensing records for 5,890 patients under treatment showed that the percentage of days covered by dispensed medicines for the previous 6 months was high at 91% (95% confidence intervals (CI): 91% to 92%) with 13.7% (CI: 13%-15%) of patients having a gap of 30 days or more in dispensed medicine (Table 2).

The median facility in each country achieved a rate of days covered by dispensed medicine of 90% or above, with 2%-19% of patients experiencing a gap of 30 days or more in their medication coverage. However, the mean coverage in some individual facilities was less than 80% with as many as 42% of patients experiencing a gap of 30 days or more (Figure 2 and Table 3).

Figure 2
figure 2

Distribution of facility-level mean percentage of days covered by dispensed drugs in last 6 months by country. Note: Horizontal line represents the value of the outcome in the median facility in each country; box covers the values from the 25th to 75th percentile of facilities; diamonds represent values in individual facilities that lie above the 75th percentile or below the 25th percentile.

Full size image

The recommended target threshold for adherence in many programs is 95%. Although the average percentage of days covered by medicine dispensed was high, fewer health facilities were able to maintain patients at this target level over the 6, or 12 month study periods. Across the four countries, the median facility-specific performance in achieving the 95% coverage target ranged from 52%-80% of patients (Table 3). In individual facilities, the percentage of patients who achieved at least 95% dispensing coverage varied from 12%-100%.

Clinic Attendance from Clinic Records

The clinic attendance measurement was lower and more variable than the medication coverage measurement. Overall, 76% (CI: 75%-77%) of patients attended their appointment on or before the scheduled day; 87% (CI: 86%-88%) attended within 3 days; and 94% (CI: 94%-95%) attended within 30 days (Table 2). Across countries, the median facility had from 72%-91% of patients attending their next appointment on or before the day scheduled and from 80%-96% attending within 3 days of their appointments (Table 3). Variability across individual facilities was larger, with only 14% of patients attending on time in one facility and 20% attending within 3 days in another facility (Figure 3).

Figure 3
figure 3

Distribution of facility-level values for patient attendance to next appointment. Note: Horizontal line represents the value of the outcome in the median facility in each country; box covers the values from the 25th to 75th percentile of facilities; diamonds represent values in individual facilities that lie above the 75th percentile or below the 25th percentile.

Full size image

Pill Counts and Self-Reports in Clinic Notes

Overall, only 15% of patient records we examined included a pill count and 45% included a self-reported adherence measure; therefore, we excluded these measures.

Correlations between Indicators

The correlations between the facility-level indicators are presented in Table 4. The correlations between the dispensing-based indicators and the other adherence indicators did not differ for the entire sample of patients (indicators 8-10 in Table 4), or for only those still in treatment at the end of the observation period (indicators 1-3). As a result we used the data from the entire sample.

Table 4 Correlation between facility indicators
Full size table

The percentages of days covered by dispensed medicine and of patients with more than 95% of days covered both were significantly correlated with the percentages who attended their appointments on time (i.e., on or before the scheduled day of the appointment), within 3 days, and within 30 days of their appointment, but were not correlated with self-reported adherence either in the clinical notes or in exit interviews. However, self-reported adherence at exit interview was significantly correlated with the percentage of patients who attended within 3 days of their appointment, and the percentage with perfect self-reported adherence had higher correlations with the other indicators than the average self-reported adherence. The indicator for attending appointments within 3 days had higher correlations with all indicators than the indicator for attending on or before the scheduled day of the appointment.

Discussion

We have shown, using routine data sources, that it is possible to use standardized methods to collect data and to measure important aspects of patient treatment adherence and clinic attendance in a wide variety of health facilities in resource-poor settings and that the different facility-level summary indicator results are correlated. Because taking medicine is a private affair, all individual-level adherence measurements are indirect. Aggregated at the facility level, summary results for different indicators can point to interventions in different systems to improve facility performance.

In the four survey countries, facilities generally maintained patients at a high level of treatment adherence; however, for all indicators, we identified some facilities that performed less well, which provided a target for quality improvement efforts. The measurement approaches offer a systematic way to assess and compare adherence measures across facilities and programs and to assess the impact of system-level interventions to improve adherence.

Measuring patient self-report about recent patterns of adherence is a low cost and commonly used method to assess adherence, which has correlated with clinical outcomes both in a meta-analysis [9] and in several resource-poor settings [8, 10, 21, 21, 27], although it has been assessed in various ways [28]. For purposes of a cross-sectional survey, exit interviews provide an opportunity to standardize the self-report question (i.e., number of doses missed in the last three days; achievement of 100% adherence). In the current field studies, a small percentage of clinical records included self-reported adherence and pill counts. In addition, the self-reporting measurement in clinic records suffers from inconsistent questioning and irregular recording. Standardizing the questions and the recording methods allow for a useful comparison. Because self-report measures tend to overestimate true adherence compared to more objective measures [11, 12], and the median levels reported in these surveys are over 95%, this may not be a sensitive enough measure to evaluate interventions to improve adherence.

Facilities with electronic pharmacy records often use dispensing data to calculate adherence to therapy [15]. Several studies in industrialized countries have shown associations between pharmacy-based ARV adherence measures and clinical outcomes, including viral load and CD4 counts [17, 29]. These methods have also been shown to be feasible and valid in an African setting using electronic data [13]. However, using dispensing records to measure adherence in settings with manual record systems has not been widely tested. We have demonstrated that such data are possible to gather in resource-poor settings in East Africa.

The average percentage of days covered by ARV therapy dispensed over six months provides a useful patient population-based measure of intermediate-term adherence. Similarly, the percentage of patients who experience a 30-day gap in treatment measures the ability of health facilities to retain patients in consistent therapy. These measures can help a program manager identify facilities with low levels of dispensing coverage or patient retention. Different factors can contribute to low dispensing coverage and patient retention, including poor drug supply and inadequate dispensing and patient counselling practices, which can be addressed at the system level.

Patients need to attend appointments consistently to manage many clinical issues that may arise during ARV treatment and to avoid gaps in therapy. Missed appointments correlate with other aspects of non-adherence in our data and elsewhere [15, 21, 30, 31]. Because patients often receive a 30-day supply of medicines and a follow-up appointment after 28 days, they will not miss taking their medicine, even if they do not come on their scheduled appointment date. We therefore propose attendance within three days of a scheduled appointment as a key adherence measure. Not all treatment programs in Africa have easy access to information on who is expected to attend on a given day and who has not appeared for their appointment. A standard approach for identifying patients that miss appointments could help programs develop effective community outreach systems; in addition, information on the percentage of patients missing appointments would allow a program manager to target poorly performing facilities and to identify causes and appropriate interventions.

Deciding to take steps to improve adherence levels depends on available human and financial resources. One method of deciding which facilities need to improve is to set target levels for each of the main indicators. The current results could serve as benchmarks for future surveys in other locations. Another method would be to look at the mean scores achieved by most facilities and target the poorest performing 5%, 10%, or 25% for interventions. For example, across facilities, a median of 97% of patients reported full adherence. However, in the bottom 25% of facilities, the measure was 93% or less. In the bottom 10% of facilities, 84% or less of patients reported full adherence, and in the bottom 5% of facilities, the figure fell to 78% or less (Table 5).

Table 5 Distribution of mean values of key adherence and attendance indicators across all countries
Full size table

Encouragingly, most facilities surveyed are maintaining high rates of patient adherence and attendance. However, all countries have facilities with low measures that need systems strengthening. Using simple, low-cost methods to identify poorly performing facilities would allow ART program managers to examine the causes of poor performance and work with facilities to make improvements. This standardized methodology and associated indicators can also be adapted to become part of an ongoing monitoring system. A manual on how to conduct the adherence assessments and spreadsheets for processing the data are available on request from the first author.

Our study made no assessment of patient retention as this was not part of our objectives. We recognize the importance of the assessment of retention. Retention data are already collected by many agencies. Our work reported here is to explore the feasibility of assessing the performance of health facilities in supporting adherence among their patients. The method of selecting patient records was to identify patients who had attended the facility 7 or 13 months before the month of the survey. To do this, the records had to be accessible in the facility. There was an implicit bias towards patients who keep appointments and those whose records were available at the facility; thus, the indicators may reflect the experience of patients still in treatment who maintain better adherence.

Evaluating and monitoring adherence to ARV treatment are critical to using health care resources effectively and efficiently, reducing rates of drug resistance to first-line therapies, and improving patient outcomes. The four field tests provide strong evidence that adherence evaluation is possible using routine data that can be collected systematically from a wide range of health facilities in resource-poor settings. The challenge now will be to introduce monitoring of these indicators on a routine basis and using them as the basis for facility-level quality improvement efforts.

Conclusions

The four field tests have shown that data to measure indicators of adherence based on days covered by dispensed medicine, attendance at appointments and self report can be collected systematically from health facilities in a wide variety of resource-poor settings and that the indicators correlate with each other. These methods allow comparison of the performance of programs and facilities between each other, over time, and to evaluate the success of interventions.

Most facilities surveyed are maintaining high rates of patient adherence and attendance. However, all countries have facilities with low measures that need systems strengthening. Using these simple, low-cost methods to identify poorly performing facilities enables ART program managers to examine the causes of poor performance and work with facilities to make improvements.