Introduction

The ultimate goal of any approach to HIV prevention is a meaningful reduction in new HIV infections. There are, unfortunately, few prevention interventions that have examined HIV infections as their endpoint and very few have achieved the goal of reducing HIV incidence. However, once demonstrated to reduce HIV incidence, interventions have had an impact on public health policy [1–3]. In order to use HIV incidence as their endpoint, clinical trials must be large in size, expansive in scope, and expensive in cost. For these reasons, HIV prevention trials typically rely on proxy outcomes, such as non-HIV incident sexually transmitted infections (STI), and behaviors that mediate (i.e., unprotected sex and injection equipment sharing) or mitigate (i.e., condom use and injection equipment cleaning) HIV transmission [4, 5]. Findings from numerous carefully controlled randomized clinical trials are remarkably positive and consistent; behavioral interventions that are guided by principles of behavior change effectively increase protective practices across nearly every population of importance in HIV prevention [6–8]. Unfortunately, because these intervention outcomes focus on behavior change, the results have been met with skepticism and often dismissed with claims that self-report is invalid and behavior change does not necessarily lead to disease prevention [9–11].

The next generation of HIV prevention trials answered the limitations of self-reported behavior change by increasing the size and scope of studies to include incident STI as their endpoint. This shift in direction allowed for reasonable approximations of HIV prevention without the expense of conducting trials powered for detecting reductions in incident HIV infections. The inference that an incident STI approximates HIV transmission risk is based on the fact that HIV is itself an STI. In addition, there is high-prevalence of other STI in populations at greatest risk for HIV infection and STI play a pivotal role in HIV susceptibility and infectiousness [12, 13]. Findings from prevention trials that have included incident STI as their endpoint mirror those trials that report only behavioral outcomes [14, 15]. Nevertheless, skeptics of behavioral interventions for HIV prevention remain unsatisfied, claiming that the biological bases and transmission dynamics of incident STI are too different from HIV to permit inferences regarding HIV prevention. Thus, once again the goal posts were pushed back; the only convincing evidence of HIV prevention efficacy would be reducing HIV incidence [16].

In the current paper, we examined the outcomes of the EXPLORE study—the only behavioral intervention trial aimed to detect differences in sexually transmitted HIV incidence. We set out to review how the outcomes of the EXPLORE study have been judged in the years following its completion. We begin with an overview of the design of the trial and rationale for evaluating its effectiveness. We then provide a review of the published research that has cited the EXPLORE study outcome paper [17] to determine how the HIV incidence outcomes have been judged. Finally, we discuss how interpretations of the EXPLORE study have influenced HIV prevention policy.

The EXPLORE Study (HPTN 015)

Designed as the definitive test of behavior change interventions to reduce HIV incidence, the EXPLORE study [17] enrolled 4,295 men who have sex with men between 1999 and 2003 in six US cities—Boston, Chicago, Denver, New York, San Francisco, and Seattle. The counseling in this trial was based on principles of motivational interviewing and social-cognitive behavioral skills building models for health behavior change [18]. Preliminary studies identified key behavioral targets aimed directly at reducing HIV risks. For example, as a result of initial developmental studies, sexual pleasure was considered the basis for engaging in unprotected anal sex and was incorporated into the counseling. Substance use was also a prominent feature of risk and became a major focus in counseling. Men randomized to the EXPLORE study experimental counseling received ten prevention counseling modules in just as many sessions over 4–6 months. Each session lasted an average of just under 40 min. Nearly 75 % of EXPLORE participants had received all of the counseling sessions, with an additional 12 % receiving between six and nine sessions. EXPLORE also provided maintenance counseling every 3 months, as well as HIV testing every 6 months for the duration of the study. Men in the comparison intervention received HIV testing every 6 months accompanied by ongoing risk reduction counseling that has itself been demonstrated highly-effective for reducing incident STI in long-term follow-up [19].

Rationale for Defining Success

The EXPLORE study defined effectiveness a priori as reducing HIV incidence by 35 %. The decision rules described in the trial executive summary state:

To be specific, the EXPLORE study was designed so that the intervention strategy would be declared beneficial if the reduction in HIV incidence was statistically significantly above 10 % (that is, that the lower bound of the confidence interval was above 10 %). If not, and the reduction in HIV incidence was statistically significantly below 35 % (that is, the upper bound of the confidence interval was below 35 %), the benefit of the intervention strategy would be ruled out. In case neither was true, the intervention would be considered plausibly efficacious with merit for further evaluation, possibly with select refinements. With the target sample size of 4,350 and an expected HIV incidence of 1.55 per 100 person years in the standard arm, if the true reduction in HIV incidence was 35 %, there would be 3.0 % chance of ruling out benefit, 50.0 % chance of declaring benefit, and 46.9 % chance of stating plausibly efficacious. Furthermore, if the true reduction in HIV incidence was 0 %, there would be an 75.0 % chance of ruling out benefit. [(EXPLORE Executive Summary [20], p. 3)].

Defining HIV prevention success as 35 % reduction in HIV incidence appears rooted in epidemiological models used to estimate vaccine efficacy. The benchmark of 35 % reduction in HIV incidence may therefore be practical for ‘permanently’ protective interventions, such as vaccines and male circumcision. However, it is unreasonable to expect that the effects of any behavioral intervention would hold for the duration of one’s lifetime. Is expecting a 35 % reduction in HIV incidence over 48 months a legitimate standard to apply to behavioral interventions? Furthermore, given that permanently protective interventions are rare (there is no effective HIV vaccine), limited by risk group (data suggest limited benefits of male circumcision for MSM) and partially protective (male circumcision is 50 % effective), is it reasonable to declare behavioral interventions that are effective over 18 months but not 48 months as unsuccessful? [8, 21].

Over 48 months of follow-up, the HIV incidence for men who received the EXPLORE counseling was 1.9 infections per 100 person years, compared to 2.3 per 100 person years for men who received semi-annual testing and counseling, an 18 % difference in HIV incidence. Thus, over the course of 4 years (average follow-up 3.25 years), the difference between the conditions was not deemed significant. However, the 12-month follow-up outcomes exceeded the 35 % bar set for reducing in HIV incidence, demonstrating a difference between conditions of 39 % [22]. The impact of the EXPLORE counseling has therefore been judged against two standards. As a behavioral intervention, a 39 % reduction in HIV incidence over 12–18 months is an irrefutable success. As discussed by Coates et al. [22]. “Had the study terminated when behavioural studies are usually stopped (i.e., at 12 months’ follow-up), the intervention would have been declared effective.” (p. 672) In contrast, when held to a standard set for HIV vaccines, an 18 % reduction in HIV incidence over 48 months is considered a failure. The policy implications of these two disparate conclusions have the potential to determine the direction of HIV prevention research, policy, and practice. We therefore examined how the field has thus far judged the outcomes of the EXPLORE study.

Method of Review

We conducted a systematic search using Thomson Reuters Web of Knowledge (version 1.0) and Google Scholar to identify all published articles that have cited the EXPLORE study outcome paper by Koblin et al. [17]. Our initial search was conducted in October 2012 and we performed a final search in March 2013. We retrieved the full text for all published papers and searched the articles for the EXPLORE study citation. We extracted the verbatim text where the EXPLORE study was cited in the original content. We then coded the extracted text blind to author, title, and journal.

Articles were first sorted into two groups. The first group was defined as non-judging, that is the EXPLORE study was cited but not judged as to its effectiveness for preventing HIV infections. These articles cited EXPLORE for use in secondary analyses, use of measurement instruments, with reference to the type of counseling, or with respect to the non-HIV outcomes without drawing conclusions on the impact of the intervention on HIV incidence. For example, one article that cited the EXPLORE study stated “Reviews showed that intervention programs for MSM are effective in reducing risk behaviors among HIV risk groups” [23], without drawing conclusions on the HIV incidence endpoint. In addition, articles that discussed the rates of behavior and disease in the trial without reference to the intervention effects were considered non-judging; for example, “In the EXPLORE cohort study, a significant increase in the risk of HIV infection was associated with recently acquired and prevalent HSV-2 infection-by 3.6-fold and 1.5-fold, respectively” [24]. Finally, articles that made broad statements about intervention outcomes without specifically discussing EXPLORE were coded as non-judging; for example, “Behavioral interventions, contingency management programs and referral to outpatient treatment programs have shown promise in decreasing methamphetamine use and reducing risk behaviors associated with the spread of HIV infection.” [25].

The balance of articles judged the impact of EXPLORE counseling on HIV incidence. These articles were subjected to a second stage of coding to determine how the outcomes were judged: (a) favorable HIV outcomes (i.e., demonstrated significant reduction in HIV incidence), (b) unfavorable outcomes (i.e., failing to significantly reduce HIV incidence). Two independent coders (the first two authors) examined the extracted text and reached 92 % agreement. Discrepancies were settled by discussion.

Interpretation of the EXPLORE Study in Peer-Reviewed Literature

Results showed that the EXPLORE study outcome paper was referenced soon after publication, with 53 articles citing the outcomes in the first four years after publication, 19 in the fifth year, and 23 articles citing the outcomes in the sixth year. In total we identified 127 articles citing the EXPLORE outcome paper. For the sake of comparison, we conducted a similar citation search for the primary outcome study from Project RESPECT, which demonstrated reductions from counseling in non-HIV STI that was widely heralded as successful and formed the basis for the control condition in the EXPLORE study. Results showed that in the first 8 years after Project RESPECT was published it had been cited 299 times, more than twice as often as the EXPLORE study, a statistically significant difference, t(14) = 4.04, p < 0.001 (see Fig. 1).

Fig. 1
figure 1

Number of articles citing project respect (1999–2006) and the EXPLORE study (2005–2012)

Full size image

Content coding of the articles that cited EXPLORE showed that 92 (72 %) of the 127 articles citing the study did not judge the HIV incidence outcomes. A total of 34 articles drew conclusions regarding the effects of EXPLORE counseling on HIV incidence. Table 1 shows the positive and Table 2 shows the negative coded judgments of the EXPLORE study HIV incidence outcomes. Results showed that 27 of the articles (80 %) judged EXPLORE as ineffective at reducing HIV incidence and 7 (20 %) judged EXPLORE as effectively reducing HIV incidence. Examining the content of Table 2 shows that even the most positive interpretations of the EXPLORE study outcomes take into account the 48-month follow-up and do not suggest overwhelming success. Thus, across all citations of the EXPLORE outcome paper, only 5 % judged the trial effective in reducing HIV incidence compared to 21 % that judged the outcomes ineffective.

Table 1 Negative interpretations of the EXPLORE study HIV incidence outcomes in published articles
Full size table
Table 2 Positive interpretations of the EXPLORE study HIV incidence outcomes in published articles
Full size table

Interpretations of the EXPLORE Study in Public Policy

Whether or not EXPLORE counseling should be held to the same standard as an HIV vaccine may be worthy of academic debate. Nevertheless concluding that EXPLORE was not effective in reducing HIV incidence has had a significant real world impact on public health policy. At worst, the 48-month outcomes from the EXPLORE study have provided long-time skeptics of behavioral interventions with a definitive confirmation for their beliefs that risk reduction counseling is ineffective. For example, in their 2006 HIV Testing Guidelines, the CDC refers to EXPLORE as offering limited evidence of efficacy for risk reduction counseling. Specifically, as part of their basis for removing the requirement to conduct risk reduction counseling in conjunction with HIV testing, the CDC stated the following about EXPLORE;

The benefit of providing prevention counseling in conjunction with HIV testing is less clear. HIV counseling with testing has been demonstrated to be an effective intervention for HIV-infected participants, who increased their safer behaviors and decreased their risk behaviors; HIV counseling and testing as implemented in the studies had little effect on HIV-negative participants. However, randomized controlled trials have demonstrated that the nature and duration of prevention counseling might influence its effectiveness. (referenced the EXPLORE study) Carefully controlled, theory-based prevention counseling in STD clinics has helped HIV-negative participants reduce their risk behaviors compared with participants who received only a didactic prevention message from health-care providers. A more intensive intervention among HIV-negative MSM at high risk, consisting of ten theory-based individual counseling sessions followed by maintenance sessions every 3 months, resulted in reductions in unprotected sex with partners who were HIV-infected or of unknown status, compared with MSM who received structured prevention counseling only twice yearly. (references the EXPLORE study). [[26] p. 15].

The implication is that risk reduction counseling may change behavior and may even reduce the risks for STI, but not HIV.

The CDC has also identified EXPLORE counseling as an ‘effective intervention’ for distribution in its Diffusion of Effective Behavioral Interventions (DEBI) program. (see http://www.cdc.gov/hiv/topics/research/prs/resources/factsheets/EXPLORE.htm) In describing the “Key Intervention Effects”, the CDC notes that EXPLORE decreased risk behavior. They specifically state, “Men in the intervention group were significantly less likely to report any unprotected anal sex, serodiscordant unprotected anal sex, and serodiscordant receptive unprotected anal sex at the 12 and 18-month follow-ups as compared to men receiving the standard comparison intervention (p’s < 0.001).” However, under a separate heading, “Considerations”, The CDC states, “A modest 18 % reduction in HIV incidence, the primary outcome, was observed in the intervention relative to control arm; however, this did not achieve statistical significance.”

The US National HIV Prevention Strategy, arguably the single most important policy statement guiding US HIV prevention efforts, only cites the EXPLORE study once. Consistent with the bulk of the research literature and the conclusions of the CDC, the National Strategy cites EXPLORE by stating the following, “Even though these [risk reduction] interventions have not been proven to reduce HIV infections, they promote responsible sexual behaviors that may lower a person’s risk for becoming infected with HIV and some have been associated with reducing STIs” [[27], p. 17] Given the importance of the US National HIV Prevention Strategy, its conclusion that behavioral interventions have not been proven to reduce HIV infections surely impacts policy decisions, resource allocation, and statewide prevention planning.

Implications

The question as to whether EXPLORE counseling reduced HIV incidence clearly depends on how success is defined. Held to a vaccine standard, EXPLORE′s outcomes over 48 months do indeed fall short. For instance, the most successful HIV vaccine trial has demonstrated 31 % efficacy over 3 years [28]. However, when held to a standard that is fitting for behavioral interventions, 18 months of follow-up far exceeds what would typically be expected for durability of change in behavior. Thus, reducing HIV incidence over 35 % in this time frame is an unquestionable success. The EXPLORE study outcomes also parallel those of behavioral interventions delivered in far fewer sessions, suggesting the potential for even greater public health impacts of risk reduction counseling [15].

There are other features of the EXPLORE study that question using the 48-month follow-up to define success. Only those participants enrolled in the first year of the study, a fairly small number of men to reliably assess HIV incidence, were represented at 48 months. The HIV incidence in the EXPLORE trial did not decline as a result of epidemiologic trends, therefore yielding adequate statistical power for the analysis. In fact, during the years that the EXPLORE study was conducted there were no significant reductions in HIV incidence among MSM in major US cities, including cities where the trial was conducted [29]. The epidemiologic context of the EXPLORE study therefore further bolsters confidence in the 18-month reductions in HIV [30].

Concluding that EXPLORE counseling was more effective in reducing HIV incidence than suggested in the original outcome paper requires using the appropriate standard for a behavioral intervention. We are not the first to point out misrepresentations of the EXPLORE outcomes. Others have similarly concluded that the trial outcomes for reducing HIV incidence should be judged against behavioral interventions and that the methodological features of the trial should be taken into account. For example, although Safren et al. [31] draw conclusions that the HIV incidence outcomes were not statistically significant, they described the EXPLORE study as successful on rational grounds after considering the overlapping features of the experimental and comparison conditions as well as the duration of follow-up. In addition, Eaton et al. [32] performed secondary analyses and concluded on statistical grounds that EXPLORE was effective in reducing HIV incidence through direct and indirect effects of the intervention.

As prevention planners and policy makers contemplate the role of risk reduction counseling and other behavioral interventions in a context that sees little hope for an effective HIV vaccine and in a new era of combination approaches to HIV prevention, the interpretation of the EXPLORE study outcomes will become even more critical. We conclude that when held to an appropriate standard for behavioral interventions, the EXPLORE counseling was indeed highly-effective in reducing HIV incidence and offers the potential to significantly alter the course of HIV epidemics. Future references to this uniquely important clinical trial should judge the intervention outcomes more fairly.