AIDS and Behavior

, Volume 15, Issue 2, pp 407–421

Assessing the Reporting of Adherence and Sexual Activity in a Simulated Microbicide Trial in South Africa: An Interview Mode Experiment Using a Placebo Gel

Authors

    • Population Council
  • Paul C. Hewett
    • Population Council
  • Sharon Abbott
    • Population Council
  • Johanna Rankin
    • Population Council
  • Sarah Littlefield
    • Population Council
  • Khatija Ahmed
    • Setshaba Research CentreUniversity of Limpopo/Medunsa Campus
  • Nazira Cassim
    • Setshaba Research CentreUniversity of Limpopo/Medunsa Campus
  • Smruti Patel
    • Empilisweni Centre for Wellness StudiesUniversity of Cape Town
  • Gita Ramjee
    • Isipingo ClinicMedical Research Council
  • Thesla Palanee
    • Reproductive Health and HIV Research Unit, Tshireletso Wellness CentreChris Hani Baragwanath Hospital
  • Stan Mierzwa
    • Population Council
  • Stephanie Skoler-Karpoff
    • Memorial Sloan Kettering Cancer Center
Original Paper

DOI: 10.1007/s10461-010-9791-z

Cite this article as:
Mensch, B.S., Hewett, P.C., Abbott, S. et al. AIDS Behav (2011) 15: 407. doi:10.1007/s10461-010-9791-z

Abstract

Misreporting of adherence undermines detection of an association between product use and HIV infection in microbicide trials. This study investigates whether, in a placebo trial, audio computer-assisted self-interviewing (ACASI) produces more accurate reporting of adherence and sexual behavior than a face-to-face interview (FTFI). At three South African clinics, 849 women were enrolled and instructed to use applicators filled with placebo gel; participants were randomly assigned to FTFI or ACASI. Behavioral reports were validated through two biomarkers that detect product usage and unprotected sex. For most behaviors, ACASI generated significantly higher reporting, although differences by interview mode appeared to diminish over time. ACASI participants were more likely to report having had sex without gel, but reported and tested applicators did not indicate greater honesty about gel insertion with ACASI. While comparisons of reported unprotected sex with the validated biomarker revealed more agreement with ACASI than with FTFI, differences were small.

Keywords

ACASIAdherenceBiomarkersMicrobicidesSexual activity

Introduction

In the last 25 years, HIV prevalence rates have risen sharply in sub-Saharan Africa with women disproportionately affected. While incidence rates are now declining, new infections still number around 2 million annually [1, 2]. Given limited options for women to protect themselves against HIV, considerable investment has been made in developing female initiated methods, such as microbicides, and testing them in randomized controlled trials [3]. The recent results from the CAPRISA 004 trial, which assessed the effectiveness of Tenofovir gel—an antiretroviral based product—and found a significant reduction in HIV acquisition, have given renewed hope to scientists involved in HIV prevention [4].

One of the major challenges confronting clinical trials is the need to achieve high levels of product adherence [58]. If participants are unwilling or unable to use the product, it stands to reason that primary analyses on the intention-to-treat population are unlikely to demonstrate efficacy. Indeed, simulations have confirmed that microbicide non-adherence will result in an underestimate of efficacy [9, 10].

Since the majority of microbicides tested to date have not been systemically absorbed, their presence in urine or plasma cannot be readily detected; thus trials have relied on behavioral reports of product use and sexual activity to assess adherence. Given the explicit expectation that trial participants comply with project protocols, accurate data on sensitive topics—such as frequency of vaginal and anal intercourse and use or nonuse of the candidate microbicide, as well as condoms—may be difficult to obtain. Because of social desirability bias, fear of being excluded from the trial, and/or embarrassment, participants may be unwilling to admit that they have not been compliant with study protocols. An ancillary study of the reliability of behavioral data collected during a phase 2 trial of the candidate microbicide Carraguard suggests that participants are frequently untruthful when responding to sensitive behavioral questions. An audio computer-assisted self-interview among a sub-set of trial participants during one of the monthly study visits included questions about the frequency with which participants had given inaccurate or misleading answers during face-to-face behavioral interviews; nearly four–fifths of participants reported giving false answers to trial interviewers [11].

The recent results from the phase 3 trial of Carraguard further illustrate the difficulties that arise in assessing product use. In the trial, adherence was measured both by self-reports and by a staining assay that indicated whether applicators had been inserted in the vagina [12]. Estimates of the number of sex acts in which the product was used were calculated by dividing the average number of applicator insertions determined in the applicator test by the average number of sex acts reported in response to a question in the quarterly behavioral interview about the number of vaginal sex acts in the 2 weeks prior. While the rate of self-reported gel use based on last sex act was over 96%, the computation based on the applicator test indicated that gel was used in approximately 42% of sex acts. The substantial discrepancy between these adherence estimates highlights the challenge of obtaining accurate measures of product use. It also underscores the potential problem with relying on reports of behavior at last sex act or last day, when use may actually be higher in anticipation of a clinic visit, to generalize about usage over a longer time interval.

Reliable behavioral data are critical for successful implementation of microbicide trials. Knowing who does and does not use the product and the pattern of usage, both in terms of “persistence” (time to discontinuation) and “quality of execution” (observance of the prescribed regimen), can help study staff develop appropriate and tailored counseling messages [13]. Accurate behavioral data are also crucial for detecting effectiveness in secondary analyses among the per-protocol population. If participants over report product use, a candidate microbicide may be deemed ineffective when in fact low adherence is the underlying problem [7, 14].

The goal of this study was to determine whether audio computer-assisted self-interviewing (ACASI) produces more accurate reporting of adherence to microbicide clinical trial protocols and risky sexual behaviors than conventional face-to-face interviewing (FTFI). Two biomarkers were used to validate selected self reports. The advantage of ACASI over FTFI is that neither the investigator, nor anyone else in the area where the interview is being conducted, hears the question or response, presumably reducing social desirability bias. Moreover, unlike self-administered questionnaires, which require literacy, ACASI can be conducted without requiring that the respondent read the questions on the computer screen. The other advantage of ACASI is that the interviewing process is standardized; thus variability in interviewer characteristics, behavior and questioning style does not come into play as it does in an FTFI. Numerous studies have demonstrated the feasibility of ACASI in developing country settings [1517]. Although in the absence of programmed consistency checks ACASI data are more likely to be internally discrepant [1820], ACASI often yields higher reports of sensitive behaviors than FTFI [11, 2124]. Few studies, however, have used biomarkers to validate self reports of sexual activity by interview mode. Although one study in Zimbabwe found that self reports of recent sexual activity were not confirmed by a biomarker of recent semen exposure for both ACASI and FTFI [25], another study in Brazil revealed stronger associations between self reported risk behaviors and sexually transmitted infections with ACASI compared to FTFI [26].

Methods

Study Design and Participants

Between March 2007 and April 2008, 849 women were enrolled in a methodological experiment at the South African sites that participated in the phase 3 Carraguard trial. Participants were instructed to use applicators filled with a placebo gel for 3 months and randomly assigned to FTFI or ACASI for monthly behavioral interviews. The three sites were the University of Cape Town (UCT) Empilisweni Centre for Wellness Studies in Gugulethu, Western Cape; the University of Limpopo/Medunsa campus (Medunsa), Setshaba Research Centre in Soshanguve, Gauteng; and the Medical Research Council (MRC) Isipingo Clinic in KwaZulu-Natal. The eligibility criteria were similar to those used in microbicide trials with active products. Eligible participants were aged 18–40, willing and able to give written informed consent, agreed to be tested for HIV and told their results at screening and at the final study visit, HIV-negative, were not pregnant or planning to become pregnant in the next 3 months, had sex at least once in the past month, currently had at least one sex partner, and were citizens or permanent residents of South Africa. Women were excluded who were within 4 weeks of the last pregnancy, had a papanicolaou (Pap) smear at screening graded as carcinoma, injected illicit drugs in the 12 months prior to screening, were participants in another research study, or had been participants in the phase 3 Carraguard trial.

Procedures

The study specified five visits: a screening visit, an enrollment visit approximately 2 weeks later and three additional visits at 1 month intervals. Initial eligibility was assessed via (1) a short questionnaire that included two sexual activity questions about whether the woman engaged in sex in the last month and had a steady sexual partner, (2) a pelvic exam, and collection of a specimen for a pap smear and vaginal swabs for Neiserria gonorrhea and Chlamydia trachomatis, and (3) collection of a blood sample for syphilis testing, and a urine specimen for an hCG pregnancy test. Participants received counseling before and after HIV testing; two HIV rapid tests were done on blood samples using Determine (Abbott laboratories) and OraQuick diagnostic tests (Orasure Technologies, Bethlehem, PA, USA). For discordant test results, a confirmatory ELISA test (AxSYM® HIV1/2 gO Abbott laboratories, Wiesbaden, Germany) was conducted. Specimens were processed at the Bio Analytic Research Corporation, Johannesburg, South Africa.

A total of 1,413 women were screened. Participants were provided with pre and post test counseling for HIV as well as safer sex counseling at the screening visit. As part of the informed consent process, participants were told that the purpose of the study was to compare different ways of interviewing women about sensitive issues. They were also told that they would either be interviewed by a study staff member or would use a computer to complete behavioral interviews.

At the enrollment visit, results from the STI tests and Pap smear were reviewed and positive infections treated unless already treated syndromically at screening. A urine specimen was collected and an hCG pregnancy test conducted. A pelvic exam was also conducted and a vaginal swab collected for a semenogelin analysis. At the third monthly visit, participants were tested again for HIV. If clinically indicated, whether at monthly or unscheduled visits, wet mount, whiff tests, and vaginal pH tests were conducted on site for the detection of bacterial vaginosis, Trichomonas vaginalis and yeast infections. Treatment was prescribed as appropriate.

At the enrollment visit, women were instructed to insert single-use Microlax®-type applicators filled with methylcellulose, the placebo gel used in the Carraguard trial, up to 1 h prior to each sex act. Methylcellulose (manufactured by Clean Chemical Sweden in Borlänge, Sweden) a clear, odorless, tasteless gel was chosen for the Carraguard trial on the basis of its safety profile and lack of effectiveness against HIV [12]. Women were told to use the gel vaginally and counseled not to use it orally or rectally. They were also counseled to use male condoms, which were provided at the study sites. At each of 3 monthly follow-up visits women returned used applicators, received condoms, had a pelvic exam, pregnancy test and a vaginal swab collected for a semenogelin analysis and completed a behavioral interview.

Using a block 10 randomization scheme stratified by site, participants were assigned to FTFI or ACASI for behavioral interviews. Staff were instructed to conduct behavioral interviews prior to HIV or STI testing. The enrollment and monthly interviews included close-ended questions about sexual behavior including the number of sexual partners, whether the respondent thinks her regular sexual partner has other partners, frequency of sex in the prior month, consistency of condom use and frequency of condom use last month, oral and anal sex last month, vaginal hygiene last month, transactional sex last month, and forced sex last month. The monthly interview also included questions about gel and condom use in the previous month. Interviews were conducted in Xhosa at UCT, Setswana and English at Medunsa, and Zulu at MRC.

Participants randomized to ACASI used a laptop set up in a private room. They were provided with instructions on how to input responses using an external mini-keypad connected to the computer. This hardware configuration had been used in prior studies conducted in Brazil, Vietnam and Kenya. Although some keys were color-coded to simplify tasks, e.g., moving to the next question, replaying the audio, repeating the previous question, respondents were required to enter numeric responses to answer questions, e.g., 1 = yes or 2 = no. If a question required a continuous response, a range check was performed. If the participant gave an answer outside the range, she was not able to proceed to the next question until a valid response was entered. The instructions and questions were audible through headphones. If they were able, participants could read the corresponding text on the computer screen. Note that the ACASI and FTFI interviews used identically worded questions; no pictures or diagrams were used for the computerized interviewing. In addition, interviewers were provided with written guidelines about conducting the FTFI that attempted to minimize variability between the two modes such that the only difference was the interviewing technology. The ACASI program did not enforce logical consistency in the respondents’ answers. Thus, for example, a woman who indicated she had ten sex acts in the last month could report using gel for 12 sex acts. In addition, in the FTFI, study staff were explicitly instructed not to reconcile inconsistent answers. We chose not to confront participants with inconsistent responses either in the ACASI or the FTFI because we wanted to assess their ability to understand and answer the questions and to compare inconsistencies across interview modes. Moreover, if a participant wanted to change an answer to an earlier question in the FTFI, the interviewer was only permitted to do so for the previous question because that was how the ACASI was programmed. The ACASI software was developed at the Population Council using Microsoft Visual Basic 6.0 and Access 97. The custom application was data-driven so that all translated text and associated audio files, as well as skip logic patterns, were stored in database tables. The content of the interview, displayed as text on the laptop screen and recorded in the audio files, was translated into the three local languages. The ACASI laptops were configured to run only the ACASI software and were password protected and rebooted after each interview. Daily backups were performed and the units were stored in a locked location at the end of the day. Case record forms (CRFs) were used to record all clinical and laboratory data as well as responses from all FTFI. These forms were then sent to the Population Council in New York via Datafax. ACASI data were electronically captured in real time and merged with the CRF data.

Self reports of gel and recent condom use were validated externally through two biomarkers, the Population Council’s applicator test and the Rapid Stain Identification of Human Semen (RSID®) test. The applicator test determines whether or not the applicator has been vaginally inserted. Vaginal mucous present on inserted applicators will stain blue in a characteristic pattern when exposed to FD&C Blue No. 1 Granular Food Dye. The applicator test, which has been shown to be effective at detecting insertion regardless of whether applicators have undergone long term storage and/or been extensively handled, is reported to have 97.5% sensitivity and 96% specificity [27, 28]. The presence of semenogelin, a protein produced by seminal vesicles that causes seminal fluid to form a coagulum subsequent to ejaculation in the vagina indicates that a woman has been exposed to a partner’s ejaculate in the previous 48 h [29] and thus is a biomarker of sex without a condom. The RSID® test, a qualitative test which detects the presence of semenogelin, was developed by Independent Forensics (Lombard IL). The test does not cross react with saliva, other vaginal fluids, vaginal products or menstrual blood and can detect semenogelin with as little as one μl of semen [30]. The manufacturer also indicates that the test can detect semenogelin from vasectomized ejaculate and that “on occasion” it can detect semen up to 72 h after intercourse, (e-mail communication with Dr. Karl Reich, Chief Scientific Officer, February 2010). Thus it is possible that a few participants who honestly report no sex in the prior 48 h act may test positive for semenogelin because they had sex between 48 and 72 h prior to the interview. However, the effect of this imprecision in our analysis should not vary by interview mode. Both the applicator test and the RSID tests were conducted on-site by medical technologists out of sight of participants; in two of the three sites they were done in real time. Women were not explicitly told that their applicators would be tested for insertion or their vaginal swabs would be tested for semen. During the informed consent process they were told that blood and fluid samples taken during clinical visits might be used in other tests to help researchers learn more about sexual behaviors and microbicides, including women’s use of a vaginal gel. Participants were assured that test results would not affect their treatment or healthcare.

The study was reviewed and approved by the Population Council Institutional Review Board (New York); the University of KwaZulu Natal Biomedical Research Ethics Committee for the Medical Research Council; the University of Limpopo, Medunsa Campus, Research, Ethics and Publications Committee; and the University of Cape Town Research Ethics Committee.

Statistical Analyses

For the analyses comparing differences in background characteristics by site and interview mode (ACASI vs. FTFI) and completeness of information by interview mode, two-sided t tests, F tests and Chi-square tests were used. To compare the reporting of sensitive behaviors by interview mode, z tests were conducted. It was estimated that a sample of 810 women, 270 per site, needed to be enrolled to detect a 10% difference in reporting by interview mode with a power of 80 and a significance criterion of 0.05. For the combined analyses across all follow-up visits, generalized estimating equation (GEE) logistic regression was used to account for the within participant correlation.

Note that for the RSID analysis we used the same primary endpoint as Minnis et al. [25] for their PSA analysis. That is, we compared the percentage of women by interview mode who reported no sex for the past 2 days yet tested positive for RSID. We also compared the percentage testing positive who reported unprotected sex in the prior 2 days.

For the applicator analysis, we compared the mean difference between applicators that the participant reported inserting and applicators that tested positive for insertion. Computations of tested applicators were adjusted for women who reported inserting gel without subsequent sex. However, because the question about gel insertion without subsequent sex did not ask the precise number of times this occurred but instead included the responses “never,” “once,” “two–four times,” and “five or more times,” we had to provide estimates for the last two categories; we estimated “three” for the “two–four” category and “five” for the “five or more” category. Those who didn’t remember the number of times applicators were inserted without sex taking place (ranging from five women to seven women per month depending on the month) were treated as missing for analyses of tested applicators.

Results

Figure 1 shows the trial profile and indicates loss to follow-up and discontinuation as well as incomplete data collection, either missing behavioral interviews (BI) or missing biomarkers. Of the 1,413 women screened, 849 were enrolled and randomly assigned to an interview mode; HIV positive status was the principal reason for exclusion. Overall 47 (11.1%) of the FTFI group and 59 (13.9%) of the ACASI group were lost to follow up; 26 (6.1%) of the FTFI group and 11 (2.6%) of the ACASI group discontinued early. The primary reason for discontinuation was pregnancy; 15 of the 26 FTFI group who discontinued and five of the 11 ACASI group who discontinued became pregnant. Additionally, some participants did not complete all the components of a visit; 16.3% of ACASI versus 5.9% of FTFI participants had some but not all information missing for one or more of their visits. In month 3 41 women were missing a behavioral interview for the ACASI group, compared with only two women in the FTFI group; 27 of these 41 women were from the UCT site.
https://static-content.springer.com/image/art%3A10.1007%2Fs10461-010-9791-z/MediaObjects/10461_2010_9791_Fig1_HTML.gif
Fig. 1

Trial profile

For analyses involving only behavioral interview information, women who had complete interview data were included, even if some of their biomarker information was missing. For analysis involving biomarkers, only women who had relevant biomarker information as well as the particular month’s interview data were included in the analysis. It should be noted that the few women who completed their RSID test on a different date from their behavioral interview were excluded from RSID analyses.

As shown in Table 1, although baseline socio-demographic characteristics were significantly different across sites, no significant differences were observed by interview mode. As for baseline STIs, no significant differences were observed by interview mode; nor were there differences by site, despite the fact that HIV rates at screening were substantially higher at MRC. (36.6% of women screened at MRC, 29.2% of women screened at Medunsa and 19.5% of the women screened at UCT were HIV positive).
Table 1

Baseline characteristics, by site and interview mode

 

Site

Interview mode

Total

UCT (N = 297)

Medunsa (N = 281)

MRC (N = 271)

Test statistic

ACASI (N = 425)

FTFI (N = 424)

Test statistic

(N = 849)

Age (years)

 Mean

27.1

(26.4–27.8)

26.7

(26.0–27.4)

24.6

(24.0–25.2)

F = 15.40 P < 0.01

26.3

(25.7–26.9)

26.0

(25.5–26.6)

t = 0.64 P = 0.52

26.2

(25.8–26.6)

 18–24

127 (42.8%)

139 (49.5%)

186 (68.9%)

χ2 = 41.273 P < 0.01

224 (52.8%)

228 (53.8%)

χ2 = 0.076 P = 0.78

452 (53.3%)

 25–40

170 (57.2%)

142 (50.5%)

84 (31.1%)

200 (47.2%)

196 (46.2%)

396 (46.7%)

Marital status

 Single, never married

181 (61.2%)

183 (65.1%)

235 (86.7%)

χ2 = 50.744 P < 0.01

289 (68.3%)

310 (72.9%)

χ2 = 2.181 P = 0.14

599 (70.6%)

 Married/formerly married

115 (38.8%)

98 (34.9%)

36 (13.3%)

134 (31.7%)

115 (27.1%)

249 (29.4%)

Years of education

 Mean

10.6

(10.4–10.9)

10.8

(10.6–11.0)

11.0

(10.8–11.2)

F = 3.63 P < 0.05

10.9

(10.7–11.0)

10.7

(10.5–10.9)

t = 1.75 P = 0.08

10.8

(10.7–10.9)

 0–9 years

55 (18.6%)

41 (14.6%)

30 (11.2%)

χ2 = 6.001 P < 0.05

54 (12.8%)

72 (17.1%)

χ2 = 3.023 P = 0.08

126 (14.9%)

 10+ years

241 (81.4%)

240 (85.4%)

237 (88.8%)

368 (87.2%)

350 (82.9%)

718 (85.1%)

Home language

 Sepedi

0

69 (24.5%)

0

χ2 = 1.3e03 P < 0.01

35 (8.3%)

34 (8.0%)

χ2 = 1.486 P = 0.83

69 (8.1%)

 Zulu

1 (0.3%)

41 (14.6%)

251 (92.6%)

145 (34.2%)

148 (34.8%)

293 (34.5%)

 Setswana

0

90 (32%)

0

49 (11.6%)

41 (9.7%)

90 (10.6%)

 Xhosa

293 (98.7%)

3 (1.1%)

13 (4.8%)

155 (36.6%)

154 (36.2%)

309 (36.4%)

 Othera

3 (1.0%)

78 (27.8%)

7 (2.6%)

40 (9.4%)

48 (11.3%)

88 (10.4%)

Gonorrhea

 Negative

29 (9.9%)

17 (6.6%)

22 (8.6%)

χ2 = 1.955 P = 0.38

26 (6.6%)

42 (10.3%)

χ2 = 0.568 P = 0.06

68 (8.5%)

 Positive

263 (90.1%)

240 (93.4%)

234 (91.4%)

370 (93.4%)

367 (89.7%)

737 (91.2%)

Chlamydia

 Negative

56 (19.1%)

54 (21.0%)

50 (19.5%)

χ2 = 0.343 P = 0.84

76 (19.1%)

84 (20.6%)

χ2 = 0.301 P = 0.58

160 (19.8%)

 Positive

237 (80.9%)

203 (79.0%)

207 (80.5%)

323 (81.0%)

324 (79.4%)

647 (80.2%)

Syphilis

 Negative

41 (14.7%)

33 (12.4%)

29 (15.6%)

χ2 = 1.054 P = 0.59

47 (13.1%)

56 (15.0%)

χ2 = 0.536 P = 0.46

103 (14.1%)

 Positive

238 (85.3%)

233 (87.6%)

157 (84.4%)

311 (86.9%)

317 (85.0%)

628 (85.9%)

aOther languages: English, Afrikaans, Venda, Tsonga, Sesotho, Ndebele, Swati

Because of the loss to follow-up and early discontinuation, as well as missing behavioral interviews, the demographic characteristics by mode for Months 1–3 combined were analyzed. Although the differences that exist by interview mode, language group, and site were expected because of the missing ACASI interviews at UCT, there are no significant differences by demographic characteristics (see Table 2). This finding suggests that missing an interview was not selective of key demographic characteristics of certain participants.
Table 2

Differences in completeness of monthly interviews, by interview mode, site, and background characteristics

 

Complete

Incomplete

Test statistic

Total

580 (68.3%)

269 (31.7%)

 

Interview mode

 ACASI

275 (64.9%)

149 (35.1%)

χ2 = 4.677 P < 0.05

 FTFI

305 (71.7%)

120 (28.2%)

Site

 UCT

186 (62.6%)

111 (37.4%)

χ2 = 8.681 P < 0.01

 Medunsa

208 (74.0%)

73 (26.0%)

 MRC

186 (68.6%)

85 (31.4%)

Age (years)

 18–24

306 (52.9%)

146 (54.3%)

χ2 = 0.076 P = 0.70

 25–40

273 (47.2%)

123 (45.7%)

Language group

 Sepedi

54 (9%)

15 (6%)

χ2 = 12.107 P < 0.05

 Zulu

211 (36.4%)

82 (30.5%)

 Setswana

62 (10.7%)

28 (10.4%)

 Xhosa

190 (32.8%)

119 (44.2%)

 Othera

63 (10.9%)

25 (9.3%)

Marital status

 Never married

411 (71.0%)

118 (69.9%)

χ2 = 0.106 P = 0.74

 Married/formerly married

168 (29.0%)

81 (30.1%)

Education

 Primary

83 (14.4%)

43 (16.2%)

χ2 = 0.468 P = 0.49

 Secondary

495 (85.6%)

223 (83.8%)

Other languages: English, Afrikaans, Venda, Tsonga, Sesotho, Ndebele, Swati

FTFI Versus ACASI

Table 3 shows the prevalence of reported behaviors by interview mode for 17 different outcomes at enrollment and each of the 3 monthly visits. The 17 outcomes are divided between those for sexual behavior and vaginal hygiene (N = 7), and variables measuring adherence and condom use (N = 10).
Table 3

Prevalence of reported sensitive behaviors by interview mode

 

Enrollment

Month 1

Month 2

Month 3

ACASI N = 414–424 (%)

FTFI N = 418–425 (%)

Test statistic

ACASI N = 378–389 (%)

FTFI N = 365–375 (%)

Test statistic

ACASI N = 369–374 (%)

FTFI N = 340–351 (%)

Test statistic

ACASI N = 308–313 (%)

FTFI N = 342–350 (%)

Test statistic

>1 sex partner, last month

13.8

(10.5–17.1)

1.7

(0.4–2.9)

z = 6.72

P < 0.01

9.0

(6.1–11.9)

1.3

(0.2–2.5)

z = 4.80

P < 0.01

9.7

(6.6–12.7)

0.6

(0.0–1.4)

z = 5.58

P < 0.01

6.7

(3.9–9.5)

2.3

(0.7–3.9)

z = 2.78

P < 0.01

Thinks current partner has other partners

26.3

(22.1–30.6)

20.1

(16.3–23.9)

z = 2.14

P < 0.05

22.3

(18.2–26.5)

18.3

(14.3–22.2)

z = 1.39

P = 0.17

21.4

(17.2–25.6)

19.3

(15.1–23.4)

z = 0.71

P = 0.48

19.4

(15.0–23.9)

18.6

(14.5–22.7)

z = 0.26

P = 0.79

Forced to have sex, last month

7.1

(4.6–9.6)

2.8

(1.2–4.4)

z = 2.88

P < 0.01

7.5

(4.8–10.1)

2.4

(0.8–4.0)

z = 3.21

P < 0.01

4.3

(2.2–6.3)

1.4

(0.2–2.7)

z = 2.30

P < 0.05

4.8

(2.4–7.2)

2.9

(1.1–4.6)

z = 1.31

P = 0.19

Anal sex, last month

7.8

(5.2–10.4)

1.9

(0.6–3.4)

z = 4.06

P < 0.01

4.9

(2.7–7.1)

1.6

(0.3–2.9)

z = 2.57

P < 0.01

7.0

(4.4–9.5)

1.1

(0.0–2.3)

z = 3.96

P < 0.01

5.4

(2.9–8.0)

1.7

(0.3–3.1)

z = 2.62

P < 0.01

Washed vagina after sex, last vaginal sex

61.1

(56.4–65.8)

25.8

(21.6–30.0)

z = 11.03

P < 0.01

60.4

(55.5–65.3)

21.5

(17.3–25.7)

z = 11.81

P < 0.01

58.5

(53.4–63.5)

23.3

(18.9–27.8)

z = 10.21

P < 0.01

53.4

(47.8–59.0)

25.3

(20.7–30.0)

z = 7.68

P < 0.01

Washed vagina after inserted gel but before sex, last sex

   

31.4

(26.8–36.1)

3.0

(1.2–4.7)

z = 11.12

P < 0.01

26.5

(22.0–31.0)

1.4

(0.2–2.7)

z = 10.25

P < 0.01

24.4

(19.5–29.2)

0.6

(0.0–13.9)

z = 10.08

P < 0.01

Inserted something in vagina (to clean, dry, tighten) after inserted gel but before sex, last sex

   

9.5

(6.7–12.5)

2.4

(0.9–4.0)

z = 4.13

P < 0.01

8.1

(5.3–10.9)

0.9

(0.0–1.8)

z = 4.69

P < 0.01

6.8

(4.0–9.6)

1.2

(0.0–2.3)

z = 3.78

P < 0.01

Always use condoms

25.2

(21.0–29.4)

24.5

(20.4–28.6)

z = 0.26

P = 0.80

36.2

(31.4–41.0)

36.0

(31.1–40.9)

z = 0.05

P = 0.96

38.8

(33.8–43.7)

38.7

(33.6–43.9)

z = 0.01

P = 0.99

38.8

(33.4–44.3)

42.6

(37.4–47.8)

z = −0.10

P = 0.33

Used condom, last sex

38.5

(33.8–43.2)

42.2

(37.5–46.9)

z = −1.11

P = 0.27

55.0

(50.1–60.0)

55.5

(50.4–60.6)

z = −0.14

P = 0.89

59.4

(54.4–64.4)

58.9

(53.7–64.1)

z = 0.12

P = 0.90

57.3

(51.7–62.8)

62.2

(57.1–67.4)

z = −1.28

P = 0.20

Used gel, last sex

   

79.3

(75.3–83.4)

81.7

(77.7–85.6)

z = −0.81

P = 0.42

83.2

(79.3–87.0)

79.9

(75.7–84.1)

z = 1.13

P = 0.26

80.3

(75.8–84.7)

82.6

(78.5–86.6)

z = −0.75

P = 0.45

Used condom + gel, last sex

   

63.7

(58.9–68.5)

55.1

(49.9–60.2)

z = 2.42

P < 0.05

64.2

(59.3–69.1)

56.5

(51.2–61.8)

z = 2.11

P < 0.05

63.0

(57.6–68.4)

55.7

(50.4–61.0)

z = 1.90

P = 0.06

Ever had sex without gel last month

   

59.7

(54.8–64.6)

40.8

(35.8–45.8)

z = 5.30

P < 0.01

52.4

(47.3–57.5)

34.8

(29.8–39.8)

z = 4.86

P < 0.01

46.9

(41.3–52.5)

30.9

(26.0–35.7)

z = 4.28

P < 0.01

Squeezed gel into vagina without having sex last month

   

20.6

(16.6–24.7)

12.8

(9.4–16.2)

z = 2.90

P < 0.01

15.5

(11.8–19.2)

9.1

(6.1–12.1)

z = 2.62

P < 0.01

13.4

(9.6–17.2)

8.9

(5.9–11.8)

z = 1.88

P = 0.06

Told partner(s) about gel

   

75.7

(71.4–80.0)

86.7

(83.2–90.2)

z = −3.88

P < 0.01

71.7

(67.1–76.2)

88.2

(84.8–91.6)

z = −5.63

P < 0.01

74.7

(69.8–79.6)

86.6

(82.9–90.2)

z = −3.89

P < 0.01

 

Enrollment

Month 1

Month 2

Month 3

ACASI

N = 373

FTFI

N = 387

Test statistic

ACASI

N = 338

FTFI

N = 338

Test statistic

ACASI

N = 317

FTFI

N = 303

Test statistic

ACASI

N = 256

FTFI

N = 312

Test statistic

Tested positive for RSID, reported no sex in past 2 days

11.5

(8.3–14.8)

17.1

(13.3–20.8)

z = −2.18

P < 0.05

12.4

(8.9–16.0)

17.2

(13.1–21.2)

z = −1.74

P = 0.08

14.2

(10.3–18.1)

12.5

(8.8–16.3)

z = 0.60

P = 0.55

14.1

(9.8–18.3)

18.0

(13.7–22.2)

z = −1.25

P = 0.21

Tested positive for RSID, reported unprotected sex in past 2 days

12.9

(9.5–16.3)

6.7

(4.2–9.2)

z = 2.87

P < 0.01

10.9

(7.6–14.3)

7.1

(4.3–9.9)

z = 1.75

P = 0.08

7.6

(4.6–10.5)

6.6

(3.8–9.4)

z = 0.47

P = 0.64

12.1

(8.1–16.1)

5.4

(2.9–8.0)

z = 2.86

P < 0.01

    

(Mean) N = 373

(Mean) N = 364

Test statistic

(Mean) N = 364

(Mean) N = 338

Test statistic

(Mean) N = 293

(Mean) N = 326

Test statistic

Difference in reported and tested applicators, last month

   

3.2

(2.4–4.0)

2.3

(1.8–2.8)

z = 1.75

P = 0.08

3.1

(2.4–3.8)

3.9

(3.3–4.6)

z = −1.60

P = 0.11

4.1

(3.3–4.9)

3.8(3.2–4.3)

z = 0.59

P = 0.56

ACASI respondents were significantly more likely to report sexual behaviors than those interviewed face-to-face. Notably reporting of anal sex was higher in the ACASI arm, an indication that this behavior is likely underreported by women in microbicide trials that conduct conventional face-to-face interviews. Indeed, in the phase 3 Carraguard trial, where women were interviewed face-to-face, between 1 and 2% of women reported anal sex [12], similar to the percentage reporting anal sex among the FTFI group here, but considerably less than the 5–8% reported by ACASI participants. If unprotected anal sex is common and a large proportion of HIV infections are due to receptive anal intercourse, the ability to detect a protective effect in a vaginal product will be undermined [31].

ACASI participants were also substantially more likely to report vaginal hygiene practices, including inserting something after gel insertion to clean, dry or tighten the vagina. In two of the three sites participants were explicitly told not to clean after gel insertion; but at UCT (results not shown) where the instruction was not given there were still significant differences by interview mode suggesting that these questions were sensitive at all three sites.

Note that for the six of seven outcomes in which ACASI respondents reported a greater likelihood of engaging in a particular behavior, the difference between modes became smaller with time. One explanation for this finding is that ACASI’s effect on reporting diminished as participants became more familiar with the technology, since the prevalence of reported behaviors was lower at later visits among those interviewed with the computer whereas the responses of the FTFI participants remained fairly stable. Alternatively, differential loss to follow-up especially in the ACASI group may have occurred if those who missed an interview or were lost to follow-up were more likely to engage in these behaviors than those who were interviewed all 3 months. We re-ran the analysis shown in Table 3 for those who did not miss an interview to see if the effect of interview mode decreased over time (see Table 4). The results indicated that reporting in ACASI was lower at month 3 than at enrollment or month 1 for all seven behaviors; a similar pattern was not observed for the FTFI group.
Table 4

Prevalence of reported sensitive behaviors by interview mode (Only those participants who completed all 3 months of the study)

 

Enrollment

Month 1

Month 2

Month 3

ACASI (%)

FTFI (%)

Test statistic

ACASI (%)

FTFI (%)

Test statistic

ACASI (%)

FTFI (%)

Test statistic

ACASI (%)

FTFI (%)

Test statistic

>1 sex partner, last month

N = 274–275 ACASI

N = 300–304 FTFI

12.4

(8.5–16.3)

0.7

(0.0–1.6)

z = 5.96

P < 0.01

9.1

(5.7–12.5)

1.3

(0.0–2.6)

z = 4.35

P < 0.01

8.8

(5.4–12.1)

0.3

(0.0–1.0)

z = 5.08

P < 0.01

6.2

(3.3–9.0)

2.0

(0.4–3.5)

z = 2.60

P < 0.01

Thinks current partner has other partners

N = 268–275 ACASI

N = 301–304 FTFI

22.8

(17.7–27.8)

20.1

(15.5–24.6)

z = 0.78

P = 0.43

21.0

(16.1–25.8)

18.2

(13.8–22.6)

z = 0.83

P = 0.41

21.1

(16.2–25.9)

20.9

(16.3–25.6)

z = 0.05

P = 0.96

18.8

(14.1–23.4)

18.6

(14.2–23.0)

z = 0.05

P = 0.96

Forced to have sex, last month

N = 274–275 ACASI

N = 303–305 FTFI

8.8

(5.4–12.1)

3.3

(1.3–5.3)

z = 2.82

P < 0.01

9.1

(5.7–12.5)

3.0

(1.0–4.9)

z = 3.42

P < 0.01

5.5

(2.8–8.2)

1.6

(0.2–3.1)

z = 2.52

P < 0.01

5.5

(2.8–8.2)

3.0

(1.0–4.9)

z = 1.51

P = 0.13

Anal sex, last month

N = 274–275 ACASI

N = 303–305 FTFI

8.4

(5.1–11.7)

1.7

(0.2–3.1)

z = 3.82

P < 0.01

4.4

(1.9–6.8)

2.0

(0.4–3.5)

z = 1.66

P = 0.10

6.9

(4.0–9.9)

1.3

(0.0–2.6)

z = 3.46

P < 0.01

5.8

(3.0–8.6)

2.0

(0.4–3.5)

z = 2.43

P < 0.05

Washed vagina after sex, last vaginal sex

N = 269–274 ACASI

N = 300–305 FTFI

61.0

(55.1–66.8)

25.6

(20.6–30.5)

z = 9.16

P < 0.01

60.8

(55.0–66.6)

22.8

(18.1–27.6)

z = 10.00

P < 0.01

58.4

(52.5–64.3)

24.0

(19.1–28.9)

z = 8.94

P < 0.01

51.8

(45.9–57.8)

25.3

(20.3–30.2)

z = 6.80

P < 0.01

Washed vagina after inserted gel but before sex, last sex

N = 271–275 ACASI

N = 301–302 FTFI

   

29.0

(23.6–34.5)

3.3

(1.3–5.3)

z = 9.08

P < 0.01

25.8

(20.6–31.0)

1.0

(0.0–2.1)

z = 9.56

P < 0.01

22.5

(17.5–27.5)

0.3

(0.0–1.0)

z = 9.10

P < 0.01

Inserted something in vagina (to clean, dry, tighten) after inserted gel but before sex, last sex

   

9.7

(6.1–13.2)

2.3

(0.6–4.0)

z = 3.79

P < 0.01

7.4

(4.2–10.5)

0.3

(0.0–1.0)

z = 4.53

P < 0.01

6.6

(3.6–9.6)

1.3

(0.0–2.6)

z = 3.32

P < 0.01

N = 269–272 ACASI

N = 299–301 FTFI

In contrast to the sexual behavior and vaginal practice questions, there were no significant differences by interview mode in reported use of condoms or gel use at last sex (Table 3). The only exception is combined gel and condom use at last sex; for this indicator, contrary to expectations, ACASI respondents were significantly more likely to report combined use of gel and condoms at months 1 and 2.

Validation by Biomarkers

For the RSID analysis we report the percentage of participants whose test result was positive, indicating the presence of sperm in the vagina. Approximately 25–36% of participants tested positive in any month. The first RSID outcome indicates the percentage of participants who tested positive, despite reporting no sex in the 2 days before the interview. For three of the four visits, FTFI participants were more likely than ACASI participants to deny having sex despite testing positive for the presence of sperm; the difference between modes, however, is only significant at enrollment. The next row indicates positive RSID results for participants who acknowledged having unprotected sex in the 2 days before the interview. These results are as expected with a larger percentage in the ACASI group reporting unprotected sex and testing positive for RSID than in the FTFI group, significantly so at enrollment and month 3.

The results from the applicator test are shown in the last row of Table 3. Of the 40,485 applicators distributed, 36,576 (90.3%) were returned, of which 17,159 (46.9%) were opened. The dye test indicated that of those opened, 8,920 (52.0%) had been inserted vaginally. The mean difference in reported applicators and applicators that tested positive was similar by interview mode. Indeed for both groups, the number of applicators testing positive for insertion is less than the number reported to have been used, suggesting that gel use is over-reported in both modes. Estimates of the percentage of sex acts covered by gel use were calculated by dividing the number of applicators returned opened and inserted based on the applicator test by “rounds of sex” reported for the prior month adjusting for insertion without subsequent sex. (Data not shown in tables.) Gel use was estimated to be used in 53.2% of sex acts in month 1 (53.9% ACASI, 52.5% FTFI), 47.4% in month 2 (50.7% ACASI, 43.9% FTFI) and 39.9% in month 3 (41.4% ACASI, 38.6% FTFI). Estimates of gel use based on self reports indicated a much higher level of coverage: 85.2% in month 1, 87.8% in month 2, and 86.3% in month 3. Self reports differed by interview mode only for month 1, with 88.1% of acts reported as covered in ACASI and 82.2% in FTFI.

Whereas a comparison of reported and tested applicators revealed no interview mode effect, questions that explicitly asked about nonuse of gel in the last month did show significant and substantial differences, with ACASI respondents much more likely to have reported nonuse. When asked about specific reasons they didn’t use gel, ACASI participants were significantly more likely in months 1 and 2 to admit to nonuse for all indicators specified (ran out, thought there was enough gel in vagina from previous round of sex, not enough privacy to insert, no time to insert, partner did not allow, worried would cause harm); in month 3 the percentages were larger for ACASI for all six reasons but the differences between modes were only significant for three of the six (results not shown). The reason that consistently generated the largest difference by mode for all 3 months was “no time to insert” with, for example, nearly twice as many ACASI respondents reporting this in month 1: 36.0% vs. 18.1% (results not shown). In addition to higher reports of non use, ACASI participants were significantly less likely to report that they told their partners about gel use in all 3 months (month 1: 75.7% ACASI versus 86.7% FTFI z = −3.88, P < 0.01).

In sum, the results of the differences in reporting by mode separately by visit vary according to the outcome although the results are either in “favor” of ACASI or indicate no difference by interview mode. For only one outcome at only one visit is FTFI significantly “better.” For sexual behaviors and vaginal practices, significantly higher reporting of sensitive behaviors was found among ACASI participants for six of seven outcomes; for some outcomes the differences were substantial. For questions about sex without gel use, ACASI also yielded higher reporting. However, for adherence questions about gel and condom use at last sex and for analyses assessing mean difference in reported and tested applicators, few differences were observed by interview mode. ACASI respondents were as or more likely to over-report gel insertion as those interviewed FTF. The RSID results are more supportive of ACASI; the differences between modes are in the expected direction with ACASI respondents more likely to have reported honestly about sexual activity in the 2 days prior to the interview; however, the differences are small and not always significant. Moreover, depending on the month, between 11.5 and 14.2% of ACASI respondents reported abstaining from sex in the 2 days prior to the interview, although the RSID results indicate the presence of semenogelin. While less than the comparable percentage for FTFI respondents, the fraction is not trivial.

The Effect of a Male Interviewer

One factor that might affect the difference in reporting by interview mode is the sex of the interviewer. At the outset of the study there was no discussion about whether interviewers for the FTFI should all be female. At study close-out, it became apparent to all investigators for the first time that three of the interviewers at the MRC site were male. We re-ran the analysis of Table 3 for the FTFI group comparing those interviewed by female staff with those interviewed by male staff, approximately 50 participants in any 1 month. Significant differences were found for three outcomes: participants interviewed by men were significantly less likely to state that they had sex without gel and that they told partner(s) about gel, and significantly more likely to report no sex in the past 2 days albeit testing positive for RSID.

GEE Analysis

The GEE results combining analyses across visits are shown in Table 5. The table indicates the effect of the interview mode variable, coded as 1 for ACASI and 0 for FTFI, on each outcome. Both unadjusted analyses and analyses controlling for age, ethnic group, site, marital status, education, and male interviewer are displayed. The results are consistent with the month by month results shown in Table 3, with 11 of the 17 outcomes indicating significant differences by interview mode in the expected direction for both unadjusted and adjusted analyses. For example, women interviewed with the computer were 6.4 (95% CI 3.6–11.2) times as likely to report having more than one sex partner as those interviewed FTF. When adjusted for covariates, the effect increases slightly. With the exception of one of 17 outcomes, the GEE analysis indicates that ACASI either significantly increased the reporting of sensitive behaviors or had no effect.
Table 5

Effect of interview mode on sensitive behaviors

 

Unadjusted odds ratio (95% CI)

P value

N

Adjusteda odds ratio (95% CI)

P value

N

>1 sex partner, last month

6.4 (3.6–11.2)

<0.01

2985

7.0 (3.9–12.4)

<0.01

2969

Thinks current partner has other partners

1.2 (0.9–1.6)

0.22

2969

1.2 (0.9–1.6)

0.26

2953

Forced to have sex, last month

2.4 (1.4–4.0)

<0.01

2997

2.4 (1.4–4.1)

<0.01

2981

Anal sex, last month

3.8 (2.2–6.8)

<0.01

2997

4.0 (2.3–7.1)

<0.01

2981

Washed vagina after sex, last vaginal sex

5.6 (4.5–6.9)

<0.01

2970

5.8 (4.7–7.3)

<0.01

2954

Washed vagina after inserted gel but before sex, last sex

18.1 (10.8–30.3)

<0.01

2152

20.3 (11.9–35.0)

<0.01

2141

Inserted something in vagina (to clean, dry, tighten) after inserted gel but before sex, last sexb

5.1 (2.8–9.2)

<0.01

1999

5.5 (2.9–10.2)

<0.01

1991

Always use condoms

1.0 (0.8–1.3)

1.00

2974

1.0 (0.8–1.3)

0.84

2958

Used condom, last sex

0.9 (0.8–1.2)

0.63

2973

1.0 (0.8–1.2)

0.83

2957

Used gel, last sex

1.0 (0.8–1.3)

0.92

2133

1.0 (0.8–1.3)

0.91

2122

Used condom + gel, last sex

1.4 (1.1–1.8)

<0.01

2116

1.5 (1.2–1.9)

<0.01

2105

Ever had sex without gel last month

1.9 (1.5–2.3)

<0.01

2152

2.0 (1.6–2.5)

<0.01

2141

Squeezed gel into vagina without having sex last month

1.6 (1.2–2.2)

<0.01

2151

1.5 (1.1–2.1)

<0.01

2140

Told partner(s) about gel

0.4 (0.3–0.5)

<0.01

2127

0.4 (0.3–0.5)

<0.01

2116

Tested positive for RSID, reported no sex in past 2 days

0.8 (0.6–1.1)

0.18

2690

0.8 (0.6–1.0)

0.10

2677

Tested positive for RSID, reported unprotected sex in past 2 days

1.6 (1.1–2.3)

<0.01

2690

1.6 (1.2–2.2)

<0.01

2677

 

Unadjusted coefficient 95% CI

P value

N

Adjusteda coefficient 95% CI

P value

N

Mean difference in reported and tested applicators, last month

0.08 (−0.67 to 0.84)

0.83

2057

0.05 (−0.68 to 0.77)

0.90

2046

GEE logistic regression results: odds ratio reported is for ACASI. The omitted category is FTFI with a female interviewer. FTFI with a male interviewer is included as a covariate in both unadjusted and adjusted models

aControlling for age, site, ethnic group, marital status, and education (Wald test P values reported)

bThose with male interviewer excluded because no woman with a male interviewer reported behavior

With regard to the biomarker analyses, women’s reports of recent sexual activity with ACASI were significantly more likely to be consistent with the RSID test: The adjusted analyses indicate that of those who tested positive, the ACASI group were 20% less likely than the FTFI group to report no sex in the past 2 days (odds ratio = 0.8, 95% CI 0.6–1.0, P = 0.10) and 1.6 times more likely to report unprotected sex in the past 2 days (95% CI 1.2–2.2, P < 0.01). However, results from the applicator test showed no effect of interview mode. Indeed, the analysis of the applicator data indicated considerable over-reporting of gel use among both the ACASI and FTFI groups.

Discussion

We investigated whether audio computer-assisted self-interviewing produced more accurate reporting of adherence and higher reporting of sexual behavior than a standard face-to-face interview in a 3-month study of a vaginal gel that had been used as a placebo in a prior phase 3 microbicide trial; two biomarkers were used to validate self reports.

We found considerable differences between interview modes in reporting of sensitive behaviors. For the most part ACASI generated substantially and significantly higher reporting of sexual behavior and vaginal hygiene practices, although the effect of the computer on reporting appeared to diminish somewhat over time.

In contrast to the Minnis et al. [25] cross-sectional study in Zimbabwe, which used prostate specific antigen as a biomarker and found that self report was not a good predictor of recent sexual activity regardless of interview mode, validation of the behavioral reports with the RSID test provided objective evidence that data generated with ACASI were slightly more accurate. Nonetheless each month between 11 and 15% of women interviewed with the computer reported no sex in the 2 days prior to the interview despite testing positive for RSID.

Adherence self-reports varied by the question asked. Whereas comparisons of reported and tested applicators did not indicate greater honesty about gel insertion with ACASI, according to the GEE analysis women with computerized interviews were about two times more likely to report having had sex without gel. There has been some concern expressed about the precision of the applicator test in measuring gel insertion because of a reliance on the naked eye for identification. But even if accuracy in this study were lower than the 93–95% in the clinical trial site validation study in South Africa or the 97% in the controlled clinical site validation study at a hospital in New York [28], the expectation is that it would be similar across interview modes, thereby not affecting the estimate of the difference between modes in reported and tested applicators.

One advantage of the ACASI study over the Carraguard Phase 3 trial was that tested applicators were separated by month of use rather than cumulated over the duration of the trial, permitting a monthly estimate of coverage rather than an aggregated estimate over the entire study or trial period. The Carraguard Phase 3 trial and the ACASI study also differed in the calculation of the denominator, the number of vaginal sex acts. In the trial this estimate was based on reports in the 2 weeks prior to each visit, multiplied by the number of weeks of trial participation since the previous visit divided by two, and then aggregated across all visits to get a total number of sex acts during the trial. While the ACASI study had the benefit of using the same reference period for both numerator and denominator, it had the disadvantage of having a longer recall period for reported sex acts, 1 month vs. 2 weeks. Note that both for the trial and the ACASI study, women whose reported percentage of gel-covered sex acts exceeded 100% were re-coded as having 100% of sex acts covered. Assuming that women returned all used applicators and the applicator test was accurate, coverage estimates over 100% using the applicator test were due either to underreporting of sex acts or insertion of applicators without subsequent sex, which was not considered in the trial calculations and was estimated in the ACASI study.

A placebo trial may not be predictive of the level of adherence in a microbicide trial with an active product. It is possible that women may be more likely to use the gel in a placebo trial because they are not worried about potential side effects. Alternatively women may be less inclined to use the product because they have been told it does not affect HIV. As mentioned earlier, in the Carraguard Phase 3 trial, gel was estimated to be used in 42% of sex acts based on tested applicators whereas in the ACASI study, gel use was estimated to be higher (approximately 50%) in the first 2 months and about the same as in the Carraguard trial in month 3 but the difference may be due to shorter duration—on average 2.65 months for ACASI, versus 16.20 months in the trial. Whether measured by the applicator test or by self reports, adherence declined from month 1 to 2 and then again from month 2 to 3 in the ACASI study. Presumably if women were enrolled in a placebo study for as long as they were enrolled in the Carraguard Phase 3 trial, overall adherence would be lower.

One clear drawback of ACASI, at least in the absence of programmed consistency checks, is that it is more likely to produce internally discrepant data [19, 20]. Interviewers typically reconcile inconsistencies whether explicitly trained to do so or not. Indeed in this study in month 1, 19.2% of ACASI participants reported more acts with gel use than total sex acts compared to 3.3% of FTFI participants. The analogous percentages for later visits were 13.7 vs. 2.3% in month 2, and 10.4 vs. 1.2% in month 3. These results as well as results from other studies using ACASI point to the need to simplify behavioral questions and to identify which time interval generates the most accurate recall.

While ACASI, which provides greater privacy than a standard FTFI, encouraged markedly higher reporting of sexual behavior and vaginal hygiene practices, the results from the biomarker analysis suggest that even self-interviewing with a computer does not lead to full disclosure. The behavioral assessment that is of greatest interest in a microbicide trial, namely gel adherence, does not appear to be more credible when generated by a computer interview than by a face-to-face interview, at least as measured by the proportion of sex acts in which gel was inserted. In microbicide trials, where participant compliance is so critical, social desirability is likely to be more of a problem for questions about product use than for questions about sexual activity. A computerized self interview, although clearly of some benefit for adherence reporting as evidenced by the much higher reports of at least one instance of non use, may not fully overcome participants’ reluctance to reveal that they did not behave as they were instructed.

The results from this study together with the results from the Carraguard Phase 3 trial point to the difficulties researchers confront in conducting microbicide trials where adherence is so critical to the success of the trial. In the absence of a reliable biomarker to measure adherence, researchers must rely on self reports of product usage and sexual behavior. Yet, eliciting accurate information from participants is a challenging task. ACASI appears to encourage greater honesty than conventional FTF interviews, but as this study demonstrates, it is not a panacea and it is not without limitations. Electronic data capture introduces further complexities, including the need for sufficient redundancy in file-backup and data storage to prevent data loss, and additional time and cost for setting up the computerized interview program, training participants, and designing and implementing a process for dealing with technological problems.

There is a concern that extensive behavioral interviewing is burdensome to staff and participants in clinical trials [32]. Given that poor adherence can significantly reduce the likelihood that a microbicide product will be found to be efficacious, researchers should explore new and innovative ways to collect behavioral data. For example, simple-message-service (SMS) tools, which employ cell phones to monitor product usage on a close to real time basis, may reduce recall error and lower reporting biases. SMS goes one step further than ACASI in removing clinic staff from participants’ reports of their behavior during the trial. In addition, utilizing mixed methods and triangulating data, as was done in the MDP 301 trial of Pro2000/5 [33], should be encouraged in future trials as it may provide greater confidence in behavioral reports. Finally, if barriers to adherence are to be understood and effective counseling provided, data are required about factors that determine product use including risk perceptions, attitudes toward product attributes, motivations for joining trials, the decision making of partners, and the particular circumstances in which participants find products difficult to use.

Acknowledgments

The authors thank Barbara Friedland, for her assistance in developing this project, for providing guidance, and for her careful reading of the manuscript. We also acknowledge Sumen Govender, Marlena Gehret, Vuyelwa Mehlomakulu, and Robin Maguire for their contributions. Funding for this research was provided by the Office of Population and Reproductive Health, Bureau for Global Health, United States Agency for International Development (USAID) under Award No. GPO-A-00-04-00019, and the William and Flora Hewlett Foundation. The opinions expressed herein are those of the author(s) and do not necessarily reflect the views of the U.S. Agency for International Development.

Copyright information

© Springer Science+Business Media, LLC 2010