Standard Measures are Inadequate to Monitor Pediatric Adherence in a Resource-Limited Setting
- First Online:
- Cite this article as:
- Müller, A.D., Jaspan, H.B., Myer, L. et al. AIDS Behav (2011) 15: 422. doi:10.1007/s10461-010-9825-6
- 488 Downloads
This study aims to compare the use and cost of objective and subjective measures of adherence to pediatric antiretroviral treatment in a primary care facility in South Africa. In a 1-month longitudinal study of 53 caregiver-child dyads, pharmacy refill (PR), measurement of returned syrups (RS), caregiver self-report (3DR) and Visual Analogue Scale (VAS) were compared to Medication Event Monitoring System (MEMS). Adherence was 100% for both VAS and 3DR; by PR and RS 100% and 103%, respectively. MEMS showed that 92% of prescribed doses were administered, but only 66% of these within the correct 12-hourly interval. None of the four measures correlated significantly with MEMS. MEMS data suggest that timing of doses is often more deviant from prescribed than expected and should be better addressed when monitoring adherence. Of all, MEMS was by far the most expensive measure. Alternative, cheaper electronic devices need to be more accessible in resource-limited settings.
KeywordsAdherence Monitoring methods Pediatrics Resource-limited settings
An estimated 2.1 million children were living with HIV in 2007, 90% of whom were in Sub-Saharan Africa . Combination antiretroviral therapy (ART) has transformed pediatric HIV from a disease with 50% mortality before 2 years of age into a chronic illness in high-income settings [2, 3] and increasingly in the developing world .
Adherence to therapy is vital to preventing resistance and maintaining the effectiveness of ART regimens . Adherence to ART is a challenge, particularly for children . Children rely on a caregiver, who often is also infected, to administer medication. Syrups are difficult to measure and frequently unpalatable. These challenges may be exacerbated in resource-limited settings (RLS) that are most affected by the HIV epidemic. Often elderly family members have to care for children whose biological parents have passed away, and end up being burdened with multiple dependents while being frail themselves. Extreme financial insecurities force caregivers to work in multiple temporary employments, making it difficult to stick to a consistently managed medication schedule. The pressures of guaranteeing day-to-day survival in poor areas may overshadow the long-term goal of adherence to medication. With this in mind, adherence is even more crucial in most parts of the developing world, where only one or two ART regimens are available, leaving limited or no options once resistance has developed.
Although there is literature surrounding pediatric adherence in the developed world [7, 8, 9], fewer data exist from sub-Saharan African settings . Data from Cape Town suggest that adherence, as measured by Medication Event Monitoring System (MEMS), may be as good or better than in the developed world . In other African countries adherence was similarly good, however these studies primarily used caregiver report or pill count as adherence measures [12, 13, 14]. A heterogeneous array of pediatric adherence measures and metrics has been used in RLS, leading to incomparable study outcomes .
Although viral load is often used as a surrogate marker of good adherence, viral load data is not always available in RLS, and some common regimens, especially those containing protease inhibitors, can be more “forgiving” of poor adherence to the extent that VL may still be suppressed in the face of poor short-term adherence [15, 16, 17]. While, in adults, pharmacy refill data seems to correlate well with virologic suppression and time-to-death in African settings [18, 19], this measure has not been evaluated for liquid formulations. Of all adherence measures regularly used, MEMS appear to provide the closest correlation with viral load  and was used as the reference method for measuring adherence in this study. MEMS has been shown to be a more accurate adherence measure than self-report and provides the exact time that the cap was removed , provided that it is used correctly; i.e. closed promptly after removal without periods of it being kept open.
There have been few comparisons of adherence measures among children  and almost none in sub-Saharan Africa. Self-report is commonly used, and although readily available and inexpensive, its validity has often been questioned . Objective measures like MEMS, on the other hand, are either not readily available or simply prohibitively expensive to be used outside of well-resourced research settings .
To our knowledge, there has been no cost analysis of adherence measures for children. With the ongoing expansion of ART programs, an accurate, cost-effective method for measuring adherence among children in RLS is needed. Adherence needs to be closely monitored to detect early non-adherence and prevent the subsequent development of resistance—even more so in settings with limited treatment options.
This study compared four readily available measures of pediatric adherence to MEMS in order to assess adherence behavior and find the most accurate and feasible method for a RLS. It aims to identify adequate methods of adherence measurement for ongoing and future pediatric ART programs and program-related research.
The study took place at the Hannan Crusaid Treatment Center (HCTC) in the Nyanga district of Cape Town, which has approximately 300,000 inhabitants and a 28% antenatal HIV seroprevalence . When meeting the criteria to start ART (recurrent hospitalizations for HIV-related disease, modified WHO stage II or III disease, CD4 < 20% or CD4 < 15% in children under or over 18 months old respectively ), most children are commenced on non-nucleoside reverse transcriptase inhibitor-based regimens, usually stavudine (d4T), lamivudine (3TC) and efavirenz (EFV) or nevirapine (NVP). A lopinavir/ritonavir-based therapy (LPV/r) is offered for those who had NNRTI-based Prevention of Mother-To-Child-Transmission Therapy (PMTCT), fail first line treatment or are on Rifamycin-based antituberculous treatment and are below 3 years of age. All regimens, with the exception of EFV, are administered twice daily. Routine follow-up is monthly to month 4 and then every 4 months. VL and CD4 counts are obtained every 4 months. At the time of this study, 185 children and 1750 adults were receiving ART at the center.
Adherence support is a central component of pediatric HIV care. Caregivers attend three group “treatment readiness” sessions in order to ensure treatment preparedness. Trained therapeutic counselors are responsible for up to 120 patients each. Home visits occur at set intervals, with frequency decreasing over time and with success on therapy.
We conducted a 1-month longitudinal study, which was approved by the Health Sciences Research Ethics Committee of the University of Cape Town. Children and caregivers who attended the clinic during July and August 2007 received information about the study in their home language and were enrolled after written informed consent. The caregiver contact was managed by a research-trained nurse in the preferred language of the caregiver. This nurse did not belong to the regular clinic staff, in order to minimize the risk of obtaining answers influenced by social desirability or fear of negative treatment repercussions.
In June and July 2007, 55 eligible caregiver-child dyads were approached to participate in the study. Inclusion criteria were child age less than 7 years (to focus on the specific adherence challenges in this young age group which makes up 69% of the clinic’s pediatric patients), having one designated caregiver with appropriate cognitive ability who consented to be enrolled, and receiving either lamivudine or abacavir in syrup form (as all other liquid antiretrovirals required large volumes and did not fit into one MEMS bottle).
Clinical histories were collected from the medical records. These included age, ART regimen and date of initiation, WHO stage at treatment initiation, changes to ART regimen, and most recent CD4 percentages and VL (within 6 months of enrolment). Caregivers completed a basic demographic questionnaire at the enrollment visit, recording caregiver age, gender, ethnic and language background (black, “coloured” or white South African, primary language at home), relationship to child (biological mother/father/grandmother, extended family or other) and ART status.
We used five adherence measures, of which four could routinely be used in clinical settings with limited resources: MEMS as the method of reference; Visual Analog Scales (VAS); 3 Day Recall of missed doses (3DR); Pharmacy Refill data (PR); and measurement of returned syrups (RS). Adherence was described as a continuous variable as well as a binary variable with a 95% cut-off, based on current literature . The next routine viral load and CD4 percentage taken post-study were also recorded.
Medication Event Monitoring System (MEMS)
Each participant was equipped with a MEMS cap (Aardex Corp., Zug, Switzerland) for a 1-month period. The cap records time and date of all bottle-opening events as presumptive doses taken by the patient; the data is stored in the chip until downloaded onto a computer. The cap was fitted on one of the three liquid antiretroviral (ARV) drugs. Decanting of this drug into the MEMS-equipped bottle was performed at the clinic pharmacy when patients filled their monthly prescriptions. Caregivers received information on the correct handling and were informed that their adherence would be monitored by MEMS. The specific mechanism was not revealed to avoid deliberate cap opening without drug administration.
The overall percentage of doses taken during the 4 week monitoring interval (overallMEMS) and
The percentage of doses that were taken at the prescribed 12-h intervals during the whole monitoring period (with a 2-h window allowed: 1 h before and 1 h after the target dose time; timedMEMS).
Visual Analogue Scale (VAS)
At the 1-month follow-up visit and following the MEMS monitoring period, all caregivers were asked to rate their adherence on a Visual Analogue Scale for the previous month, ranging from 0 to 100%, in bands of 10%, marked on a vertical line. The scale was administered by the research assistant and available in English and isiXhosa. VAS has shown valid results in adults in Uganda , however it seemed less reliable in a pediatric cohort in Cape Town .
Caregiver 3-Day Recall (3DR)
Pharmacy Refill (PR)
Nachega and colleagues demonstrated that pharmacy refill-measured adherence predicted survival times in a cohort of adults in Soweto, South Africa .
Measurement of Returned Syrups (RS)
This method has been used in conjunction with pill count in another South African study, where adherence was associated with virologic outcome .
At the study site, CD4% and VL specimens were routinely collected every 4 months. Since funding restraints did not allow for us to perform study-specific blood tests, we collected the latest CD4% and VL results before study enrolment as well as the earliest ones after finishing the study from the patient records. HIV RNA was measured using the Bayer HIV-1 RNA 3.0 assay. Children were defined as having achieved virologic suppression if there were fewer than 50 copies of HIV RNA/ml in the blood sample (HIV RNA log <1.69).
Costing of the adherence measures used an ingredients-based methodology in order to most accurately assess the cost of the measures . This involves measuring the utilization of person-time and goods (the ingredients) required to perform the task, and the cost of each ingredient. For each adherence measure all tasks performed by staff members over and above standard practice in the absence of adherence measurement were determined. Measurement of returned syrups was excluded from the costing analysis for logistical reasons—the research assistant could not measure more than one action at the time. For each task, the research assistant observed a consecutive convenience sample of events, recording the time taken to perform the task and all consumables and other resources used. The costs of staff time were based on the annual cost of employment of the staff grade performing the task, divided by the number of minutes they were expected to work in each year. The costs of other resources were based on commercial prices and the costs of durable items were depreciated across their expected lifetime. It was assumed that each patient was measured at each of their three scheduled visits to the clinic annually and where items were shared across all patients (i.e. the MEMS USB communicator and software) the cost per patient was based on the total clinic population at the end of August 2008 (2,304). Overhead costs of building infrastructure and program management were not included in this costing since these would be very similar across all adherence measures and comprise a very small proportion of the total cost of this clinic . The total annual cost of each adherence measure was calculated by summing the annual costs for all ingredients. All costs were converted to US dollars at the average exchange rate for the South African Rand and Euro in 2007 (http://www.oanda.com/convert/fxhistory). In the absence of any gold standard against which to evaluate effectiveness, no attempt to conduct a cost-effectiveness analysis was made, and a cost comparison was performed instead.
Descriptive statistics were used to analyze children’s and caregivers’ characteristics and variables related to treatment outcome. Normally distributed continuous variables were described using means and standard deviations (SD); non-normally distributed ones using medians and interquartile ranges (IQR). Discrete data were described using counts and percentages. To express adherence as a binary variable, the defined cut-off value was at 95% of the continuous data for all measures. Patients above this cut-off were defined as adherent, patients below as non-adherent. We used Pearson correlations to compare adherence measures and linear regression to assess the associations between adherence and treatment outcome. The performance criteria for all adherence measures were calculated with two-by-two tables and receiver operating characteristic curves. The level of significance for all analyses was P < 0.05. All statistical analyses were performed using STATA 9.0 (StataCorp, College Station, TX).
Characteristics of Caregivers and Children
Demographic characteristics of the children
Pre-study medians (IQR) or n (%)a
Post-study medians (IQR) or n (%)a
Gender (female), n (%)
Time on ART (months)
Median viral load (log10)
Viral load <50 copies/ml
WHO stage at ART initiation
III, n (%)
IV, n (%)
d4T/3TC/LPV/r, n (%)
d4T/3TC/EFV, n (%)
AZT/3TC/LPV/r, n (%)
AZT/3TC/EFV, n (%)
Of the caregivers, 40 were the children’s biological mothers (75%), eight were grandmothers (15%), two each were fathers and aunts (4%), and one was an older sibling (2%). The median age of caregivers was 30 years (IQR, 26–37), and 33% (n = 17) were enrolled on ART at the center themselves. All caregivers spoke isiXhosa as their first language.
Medication Event Monitoring System
Reported adherence and groups of above 95% and below 80% adherence, by measure
Overall adherence (%), median (IQR)
>95% adherent n (%)
<80% adherent n (%)
Visual Analogue Scale
Medication-administering adherence as self-rated by caregivers on the VAS was 100% for all participants (Table 2).
Three Day Recall
Ninety-two percent of caregivers (n = 45) reported not having missed any dose in the 3 days prior to the follow-up visit. One caregiver reported to have missed one dose, one had missed 3 doses, one 4 doses and one 5 doses. The mean number of missed doses for all children over the whole monitoring period was 0.3 (SD, 1.0). Median 3DR adherence was 100% (IQR, 100; Table 2).
Median adherence by pharmacy records was 100% (IQR, 100). The expected interval between refill visits was 28 days, and 24 caregivers (45%) adhered to their scheduled visits. Seven caregivers (13%) were 1 day late, three caregivers (6%) were 3 days late and five caregivers (9%) were more than 4 days late. Thirteen caregivers (25%) picked up their refills ahead of the scheduled time (Table 2).
Measurement of Returned Syrups
In 32 children (67%), RS showed an adherence rate greater than 100%. Adherence for the MEMS-monitored ARV drug was 104% (IQR, 98–116), 100% (IQR, 93–101) for the second ARV drug and 101% (IQR, 99–107) for the third ARV drug (Table 2).
Adherence as a Categorical Variable
Table 2 shows the classification of patients into adherence groups for each adherence measure. MEMS was the only measure that reported median adherence rates below 100%, and the only method that detected substantial numbers of caregivers with low adherence rates. Only 4 caregivers (9%) were >95% adherent with regards to correct timing of doses within the 12-hourly interval.
Comparison of Adherence Measures
Correlations Between Measures
Correlation between adherence measures, rho correlation coefficients
3-Day Recall and MEMS
The number of doses that caregivers reportedly missed by 3DR was compared with the MEMS data for these days (n = 46 due to the smaller group of children with MEMS data; MEMS data was available for all caregivers who reported less than 100% adherence). While 85% of the caregivers had reported not to have missed any dose, this was confirmed by MEMS in only 43% of cases. MEMS consistently showed higher rates of missed doses than 3DR caregiver report; 17% for one missed dose (2% by 3DR), 9% for 2 missed doses (2% by 3DR), and 15% for more than 2 missed doses (0% by 3DR).
MEMS and Treatment Outcome
MEMS adherence, adjusted for the correct timing of doses and at a cut-off of 95%, showed a specificity of 100% in predicting virologic suppression (AUC 0.57; sensitivity 13%). For unadjusted MEMS, sensitivity was 27%, specificity 47% and AUC 0.43.
Composite Measures of Adherence
Sensitivity, specificity, positive and negative predictive value and area under curve for all adherence measures in predicting MEMS adherence >95%
VAS > 95%
VAS > 80%
3DR > 95%
3DR > 80%
RS > 95%
RS > 80%
PR > 95%
PR > 80%
VAS + 3DR
3DR + RS
VAS + RS
VAS + PR
3DR + PR
RS + PR
VAS + 3DR + RS
VAS + 3DR + PR
3DR + PR + RS
VAS + PR + RS
Costing of Four Methods of Adherence Measurement
Annual per patient costs of adherence monitoring methods
Adherence monitoring method
Annual cost per patient
1. MEMS bottle method
2. Pharmacy Refill
3. Visual Analogue Scale
4. Modified PACTG questionnaire: 3DR
In this cohort of children on ART we sought to determine the most practical and cost-effective method of measuring adherence in a RLS. MEMS was used as the reference method against which two subjective and two objective adherence assessments were compared. Of note, adherence as measured by MEMS (adjusted for dosing interval) in this study correlated with virologic suppression, which remained unchanged pre-and post-study.
Overall adherence by all measures was excellent: both subjective measures (VAS and 3DR) showed a median adherence of 100%. Overestimation of adherence is a common issue with subjective measures, which is why they are often not considered useful alone . Objective measures (PR and SR) however, had similarly high median adherence values in this study. All of the subjective and objective measures appeared to overestimate adherence when compared to MEMS.
The adherence rates in this young age group are similar to findings in older children , except for PR adherence, which was higher in our cohort. It has recently been shown that caregiver motivation and knowledge of ART are important predictors of adherence ,
The fact that adherence rates were not lower in our very young cohort, where medication administration was further complicated by liquid formulations, suggests that motivational factors are of higher predictive importance than regimen complexity.
MEMS data showed that the timing of doses was often deviant from prescription—a finding that no other adherence measure could detect. Unfortunately, at an annual cost of $57 per patient including equipment and staff time, MEMS caps are not affordable in most RLS. While both subjective measures are easy to use and low cost at less than $0.40 per patients per annum per measure, neither correlated significantly with overall MEMS adherence. Only VAS showed significant correlation with timedMEMS—the correlation, however, was weak. Given the fact that VAS showed very little variance, this correlation’s relevance should be interpreted with caution. Similarly, the objective measures showed poor correlation with MEMS. Syrup returns were difficult to manage and required more staff time than simply calculating PR data.
For the objective measures, although PR has been shown to be a highly effective means of monitoring adherence in adults , in our study PR overestimated adherence when compared to MEMS. This difference may be due to the different dispensing practices for syrups versus pills. For adults, exact numbers of pills can be counted and dispensed, whereas syrups can usually only be dispensed according to the manufacturers prepackaged volume. Since the dose of antiretroviral drugs changes with patient weight in children, no pharmacist is able to dispense an exact amount of syrup. The other objective measure, syrup returns, also overestimated adherence when compared to MEMS, possibly due to liquid spillage at dosing times. Another possibility is that of medication dumping—caregivers might deliberately empty medication containers before attending the clinic in order to appear adherent. Regardless, although PR is not as useful a tool in children as in adults, it is the more simple and cost effective measure if an objective method is to be used.
These results should be interpreted in light of several limitations. Due to limited financial and staff resources, we were only able to follow mother–child dyads for a short period of time, and it is possible that self-reported adherence was inflated (relative to MEMS adherence) over this short period. Overestimation of adherence by all measures but PR could also have occurred through the Hawthorne effect, due to the short monitoring period. It is also important to note that we measured only one antiretroviral drug of the regimen of three. Even though we included virologic data from subsequent visits after the adherence-monitoring period, the study cohort showed very little heterogeneity in their virologic response, and this limits the analyses. It is further noteworthy that the wider time frame during which specimen were collected could have influenced results. It is also important to note the limitations of our sample size—even though it is relatively large in comparison with other pediatric MEMS studies and with regards to our clinic population, its size is limiting in interpreting the significance of the detected associations and the overall statistical power of the data.
MEMS can over- or underestimate adherence, and even though we adjusted our data for these limitations, it cannot be guaranteed that all the caregiver reports that the adjustments were based on were correct.
Measuring adherence remains complex and elusive. The data here does highlight some areas where more intensive education should occur in the treatment support process: training with regard to amount and method of dosing syrups should be emphasized, as should timing of dosing. Although MEMS gave realistic adherence data in our study, no single adherence method is ideal. Our data suggest that PR and VAS would give an inexpensive assessment of objective and subjective adherence, but only VAS showed any correlation with MEMS usage. It is unfortunate that the expense of the best method renders it unavailable in RLS. As MEMS is repeatedly shown to be the best method of monitoring adherence, despite its limitations, perhaps more effort should be directed at the exploration of cheaper electronic alternatives for such RLS.
MEMS caps and equipment were donated by Aardex Corp. (Zug, Switzerland). The authors thank the staff and patients of Hannan Crusaid Treatment Center. A.M. thanks Anna Henxi He for her support.
This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.