AIDS and Behavior

, Volume 17, Issue 4, pp 1231–1244

A Systematic Review to Identify Challenges of Demonstrating Efficacy of HIV Behavioral Interventions for Gay, Bisexual, and Other Men Who Have Sex with Men (MSM)

Authors

    • Prevention Research Branch, Division of HIV/AIDS PreventionU.S. Centers for Disease Control and Prevention
  • Nicole Crepaz
    • Prevention Research Branch, Division of HIV/AIDS PreventionU.S. Centers for Disease Control and Prevention
  • Khiya J. Marshall
    • Prevention Research Branch, Division of HIV/AIDS PreventionU.S. Centers for Disease Control and Prevention
  • Linda Kay
    • Prevention Research Branch, Division of HIV/AIDS PreventionU.S. Centers for Disease Control and Prevention
  • H. Waverly Vosburgh
    • Prevention Research Branch, Division of HIV/AIDS PreventionU.S. Centers for Disease Control and Prevention
  • Pilgrim Spikes
    • Prevention Research Branch, Division of HIV/AIDS PreventionU.S. Centers for Disease Control and Prevention
  • Cynthia M. Lyles
    • Prevention Research Branch, Division of HIV/AIDS PreventionU.S. Centers for Disease Control and Prevention
  • David W. Purcell
    • Prevention Research Branch, Division of HIV/AIDS PreventionU.S. Centers for Disease Control and Prevention
Original Paper

DOI: 10.1007/s10461-013-0418-z

Cite this article as:
Higa, D.H., Crepaz, N., Marshall, K.J. et al. AIDS Behav (2013) 17: 1231. doi:10.1007/s10461-013-0418-z

Abstract

Gay, bisexual, and other men who have sex with men (MSM) are disproportionately affected by HIV but few MSM-specific evidence-based interventions (EBIs) have been identified for this vulnerable group. We conducted a systematic review to identify reasons for the small number of EBIs for MSM. We also compared study, intervention and sample characteristics of EBIs versus non-EBIs to better understand the challenges of demonstrating efficacy evidence. Thirty-three MSM-specific studies were evaluated: Nine (27 %) were considered EBIs while 24 (73 %) were non-EBIs. Non-EBIs had multiple methodological limitations; the most common was not finding a significant positive effect. Compared to EBIs, non-EBIs were less likely to use peer intervention deliverers, include sexual communication in their interventions, and intervene at the community level. Incorporating characteristics associated with EBIs may strengthen behavioral interventions for MSM. More EBIs are needed for substance-using MSM, MSM of color, MSM residing in the south and MSM in couples.

Keywords

HIV preventionMen who have sex with menBehavioral interventionsSystematic review

Introduction

In the fourth decade of the HIV/AIDS pandemic, gay, bisexual, and other men who have sex with men (MSM) continue to be the most vulnerable group for HIV infection in the U.S. MSM comprise 51 % of the estimated 784,701 persons living with HIV in the U.S. [1] Although MSM comprise ~2 % of the U.S. population [2], they account for 61 % of all new HIV infections [3]. The rate of new HIV diagnoses for MSM is at least 44 times that of other men [2] and since the year 2000, MSM are the only group where new infections are rising annually [4]. Given these recent and alarming trends, it is critical to examine current HIV prevention approaches to better understand how efforts can be improved to help reverse the trends. One area that may benefit from this kind of examination is behavioral interventions for MSM.

Behavioral Interventions and HIV Prevention

Behavioral interventions dominated early HIV prevention efforts when effective biomedical and treatment options were not yet available and reducing risky behaviors was the best option [5, 6]. For MSM and other HIV risk groups, the majority of risk reduction behavioral interventions have been delivered to individuals 1-on-1 or in small groups, using cognitive-behavioral approaches [7]. These approaches attempt to change an individual’s beliefs, attitudes, and behaviors to reduce HIV risk. Community-level behavioral interventions using popular opinion leaders, diffusion of innovation theory, or community mobilization have also been conducted [8, 9]. Behavioral interventions have demonstrated evidence of reducing risk behavior across different populations [10], including MSM [1115], but have not shown substantial evidence for reducing biomedical outcomes such as STI or HIV infection [16, 17]. Like any single prevention approach, behavioral interventions may be insufficient as a stand-alone prevention strategy for producing an impactful and sustainable reduction in HIV infection [18].

With advances in HIV treatment and biomedical approaches to HIV prevention, and a growing awareness of the importance of structural and social determinants of health, HIV prevention is moving towards a high-impact, multi-level approach. This high-impact approach uses a mixture of different types and levels of interventions that is evidence-informed, cost-effective, and tailored for a particular community in order to make the greatest sustained impact on reducing new HIV infections [5]. Analogous to using antiretroviral therapy (ART) or a combination of different classes of HIV drugs to treat a person living with HIV, a high-impact prevention approach is based on the HIV epidemiological profile of a community and uses the most effective combination of biomedical, structural, and behavioral strategies [18]. Biomedical interventions such as ART and pre-exposure prophylaxis (PrEP) [19], in particular, have generated much excitement and reflect the increasing “medicalization” of HIV prevention. However, there are many behavioral and social implications of biomedical interventions that must be addressed for these interventions to be effective [20]. For example, increased risk behavior can outweigh the effectiveness of ART in reducing HIV incidence by as much as 30 % [21] and the outcomes of recent PrEP trials highlight the importance of adherence behaviors in obtaining the most impact from PrEP [19]. Furthermore, the continued importance of efficacious behavioral interventions is underscored in the National HIV/AIDS strategy (NHAS) that prioritizes scalable behavioral interventions for decreasing sexual and drug-use risk behaviors of MSM and other HIV vulnerable groups [22]. Given this emphasis on high-impact prevention in NHAS, the need for evidence-based behavioral interventions for MSM remains a crucial piece of a high-impact approach to HIV prevention among MSM.

Evidence-Based HIV Behavioral Interventions

In 1996, the Centers for Disease Control and Prevention (CDC) created the Prevention Research Synthesis (PRS) team to systematically review and summarize HIV behavioral intervention research literature with the goal of translating scientific evidence from the research literature into evidence-based HIV prevention recommendations [23]. PRS developed efficacy criteria to assess a study’s design, quality of study implementation and analysis, and strength of findings through multiple consultations with internal and external HIV prevention researchers and methodology experts (http://www.cdc.gov/hiv/topics/research/prs/efficacy_criteria.htm). Mirroring the criteria used by other projects such as the Community Guide and Grades of Recommendation Assessment, Development and Evaluation (GRADE), PRS efficacy criteria focus on internal validity “to ensure a reasonable level of confidence that the observed changes can be attributed to the intervention” [23]. Interventions meeting PRS efficacy criteria are identified as evidence-based behavioral interventions (EBIs).

Although U.S. MSM are disproportionately affected by HIV, relatively few MSM-specific EBIs have been identified. Out of 74 US-based EBIs PRS identified as of May 2011 in the CDC’s Compendium of Evidence-Based Behavioral Interventions (http://www.cdc.gov/hivtopics/research/prs/rr_chapter.htm), only 14 EBIs were listed for MSM. Of these, only 8, or 11 %, of 74 were specifically designed for MSM. Considering that MSM comprise the majority of new HIV cases, it is puzzling that only 11 % of the PRS-identified EBIs have explicitly focused on this vulnerable population. Is the small number of EBIs reflective of few behavioral HIV prevention interventions having been specifically designed for MSM or is it because many interventions have been developed for MSM but few have demonstrated sufficient evidence of efficacy? To answer these questions, we conducted a qualitative systematic review of behavioral HIV prevention interventions specifically designed for MSM. Our objectives were to (1) identify the number of behavioral HIV prevention interventions specifically designed for MSM (2) understand the challenges of demonstrating efficacy for MSM behavioral interventions by examining the study design, implementation, analysis, and strength of findings of interventions that did not meet efficacy criteria (non-EBIs), and (3) compare study, intervention and sample characteristics of non-EBIs with EBIs. We believed a better understanding of these issues will facilitate the development of efficacious behavioral interventions for MSM.

Methods

Search Strategy

We searched the CDC’s PRS database for evaluations of MSM-specific behavioral interventions in January 2010. Another search was conducted in May 2011 to locate studies not captured in the initial search. The PRS database is annually updated by two research librarians who conduct systematic searches of several bibliographic databases including CINAHL, EMBASE, MEDLINE, PsycINFO, and Sociological Abstracts [24]. The last electronic systematic update of the PRS database was February 2011 for articles indexed in searched databases by December 2010. Quarterly hand searches of 38 crucial HIV/AIDS journals, reference lists checks, and contacting key authors are also implemented to supplement electronic database searches. We also examined online registries (i.e., Cochrane Library, ISI Web of Knowledge, RePORTER, and CRD databases) for any related research. The full search strategy for each database is available from the authors [24].

Eligible studies for this review were: (1) behavioral interventions to reduce HIV infection and transmission: (2) specifically designed for MSM; (3) conducted in the U.S.; (4) tested in randomized controlled trials with a comparison arm; (5) measured HIV behavioral or biological outcomes (e.g., condom use, number of sex partners, sexually transmitted infections, HIV infection); (6) and published between January 1988 and December 2010. We excluded studies not published in peer reviewed journals, studies not specifically designed for MSM even though they included a majority of MSM in their samples, and pilot studies if the full-scale efficacy trials were eligible. Adaptations of interventions were included but replications were excluded to avoid overweighting characteristics from previous interventions already included in the review.

Data Extraction and Analysis

Two trained coders independently coded eligible intervention studies, entered codes in the PRS database, and met to reconcile all discrepancies. If a study did not report critical information needed to determine intervention efficacy, we contacted the primary study investigator to obtain missing information or additional clarification. The final efficacy determination for each study was reached by PRS team consensus.

For each eligible study, we coded study characteristics (e.g., study date, location, eligibility), intervention content (e.g., behavioral determinants of risks), intervention characteristics (e.g., time span or duration of the intervention measured in weeks, number of sessions, level of delivery), and participant characteristics (e.g., racial/ethnic background, HIV status) on standardized coding forms. Following the standardized efficacy review procedure (http://www.cdc.gov/hiv/topics/research/prs/efficacy_criteria.htm), we evaluated research design, study implementation, analysis, and strength of findings of each eligible study against efficacy criteria established as good evidence for either individual- and group-level interventions (http://www.cdc.gov/hiv/topics/research/prs/efficacy_good-evidence.htm) or community-level interventions (http://www.cdc.gov/hiv/topics/research/prs/efficacy_good-evidence_CLIs.htm) An SPSS data file of final codes was generated from the PRS database.

Methodological and Analytic Approaches

Eligible interventions were classified into four categories based on the efficacy review:
  • EBIs that met all efficacy criteria;

  • “Rigorous non-EBIs” that met all efficacy criteria except for a significant positive finding;

  • “Positive non-EBIs” that reported a significant positive finding but did not meet at least one other PRS criterion; and

  • “Other non-EBIs” that did not have a significant positive finding and did not meet at least one other PRS criterion.

Using SPSS Version 18, we examined study, intervention, and participant characteristics with descriptive statistics (e.g., medians, frequencies). We excluded community-level interventions when calculating median number of intervention sessions, session duration (length of an individual session measured in hours), intervention time spans (the duration of the intervention measured in weeks), and total intervention hours (number of sessions multiplied by session duration) because intervention exposure at the community level is likely to differ at the individual level and estimates for these characteristics were not reported. To examine differences in study characteristics, intervention content, and participant characteristics between EBIs and non-EBIs, we conducted Fisher’s exact tests and non-parametric independent samples median tests. We compared EBIs with all non-EBIs as a whole and with each non-EBI subgroup. We also compared studies that did not find a significant positive effect (a combined group of rigorous and other non-EBIs) to EBIs to identify potential characteristics that may contribute to a significant positive effect.

Results

Literature Search

PRS evaluated 353 US-based behavioral interventions that were conducted between January 1988 and December 2010 (Fig. 1). Out of this total, 47 interventions (13 %) were specifically tailored for MSM or tested with majority of MSM in their samples. Of these, 14 were not eligible: 11 were interventions designed for persons living with HIV and had majority of MSM (over 50 %) in their samples [2535]. These studies were ineligible for the review because they were not specifically designed for MSM. The remaining three ineligible interventions included a replication [36], unpublished manuscript [37], and a pilot study whose full-scale trial was included in the review [38]. A total of 33 interventions met review eligibility criteria and were examined using PRS efficacy criteria. Descriptive characteristics of the studies are presented in Table 1.
https://static-content.springer.com/image/art%3A10.1007%2Fs10461-013-0418-z/MediaObjects/10461_2013_418_Fig1_HTML.gif
Fig. 1

Flow chart of search

Table 1

Descriptive characteristics of 33 HIV Prevention Interventions for MSM

Primary author, reference

Study years

Regiona

Intervention (# sessions/tot. h)b

Comparison type

Levelc

Sub populationd (baseline sample size)

Evidence-based interventions (EBIs)e (k = 9)

 Choi et al. [51]

1992–1995

W

Living well (1/3)

Wait-list

G

Asian Pacific Islanders (329)

 Dilley et al. [43]

1997–2000

W

Personal cognitive counseling (PCC) (1/1)

HIV-related

I

HIV- (248)

 Dilley et al. [44].

2002–2005

W

PCC with paraprofessionals (1/1)

HIV-related

I

HIV- (305)

 Kegeles et al. [8]

Pub 1996

W

Mpowerment (NA/NA)

Wait-list

C

Young (300)

 Kelly et al. [9]

1989–1992

S

Popular opinion leader (NA/NA)

Wait-list

C

Bar patrons (659)

 Koblin et al. [45]

1999–2005

NE, MW, W

EXPLORE (17/17)

HIV-related

I

HIV- (4,295)

 McKirnan et al.f [52]

2004–2006

MW

Treatment advocacy Program (4/9)

Wait-list

I

HIV+ (313)

 Wilton et al. [53]

2005–2007

NE

Many men, many voices (1/18)

Wait-list

G

African American, HIV- (338)

 Wolitski et al. [50]

2000–2001

NE, W

SUMIT (6/18)

HIV-related

G

HIV+ (811)

Non-evidence based interventions (non-EBIs) (k = 24)

 Rigorous non-EBIsg (k = 7)

 

  Carballo-Dieguez et al. [54]

1998–2002

NE

LEO (8/16)

Wait-list

G

Latino (180)

  Mansergh et al. [46]

2004–2008

NE, MW, W

MIX (6/12)

Non-HIV

G

Substance users (1,686)

  Menza et al. [58]

2007–2008

W

Contingency management (36/NR)

Non-HIV

I

Meth users (127)

  Morgenstern et al. [60]

2004–2007

NE

Motivational interviewing (4/4)

HIV-related

I

Club drug users (150)

  Rosser et al. [63]

2005–2007

NE, W,S

Positive connections (1/16)

HIV-related

G

HIV+ (675)

  Velasquez et al. [49]

1999–2003

NE

Motivational interviewing (8/NR)

HIV-related

G

Alcohol users, HIV+ (253)

  Williams et al. [68]

2003–2006

W

S-HIM (6/12)

Non-HIV

G

African American, Latino, HIV+ (137)

Positive non-EBIsh (k = 11)

 Carpenter et al. [39]

2006–2007

I

Multi-media (1/2)

Non-HIV

I

Young, HIV- (112)

 Coates et al. [55]

Pub 1989

W

Stress management (9/24)

Wait-list

G

HIV+ (64)

 Kelly et al. [56]

Pub 1989

S

ARIES (12/18)

Wait-list

G

Not specified (104)

 Mausbach et al. [57]

1999–2005

W

EDGE (8/12)

Non-HIV

I

HIV+, meth users (341)

 Peterson et al. [61]

1989–1992

W

Risk reduction (3/9)

Wait-list

G

African American (318)

 Read et al. [48]

Pub 2006

W

Interactive video (1/NR)

HIV-related

G

HIV- (136)

 Roffman et al. [62]

1989–1991

W

Relapse prevention (17/34)

Wait-list

G

Not specified (159)

 Roffman et al. [41]

1992–1993

T

Telephone counseling (14/21)

Wait-list

G

Not specified (548)

 Rosser et al. [69]

1997–1999

MW

Man-to-man sexual health (2/8)

HIV-related

I

Not specified (422)

 Rosser et al. [40]

2007–2009

I

Sexpulse (1/5)

Wait-list

I

Not specified (560)

 Valdiserri et al. [67]

1986–1988

NE

Peer-led Skills training (1/4)

HIV-related

G

Not specified (584)

Other Non-EBIsi (k = 6)

 Coleman et al. [42]

2006–2007

NE

Social cognitive (4/8)

Non-HIV

G

HIV+ African American (60)

 Miller et al. [59]

1991–1993

NE

Keep it up (1/5)

Wait-list

G

Not specified (150)

 Picciano et al. [47]

2002–2005

W

Motivational interviewing (3/5)

HIV-related

I

Not specified (391)

 Serovich et al. [64]

2005–2006

MW

HIV-related disclosure (4/5)

Wait-list

G

HIV+ (77)

 Shoptaw et al. [65]

1998–2001

W

Contingency manage. (48/72)

Non-HIV

G

Meth-users (162)

 Stall et al. [66]

1990–1994

W

Relapse prevention (32/48)

Non-HIV

G

Substance users (147)

aRegion: NE northeast, MW midwest, S south, W west, I internet based, T telephone based

bTotal hours: NA not applicable because community intervention, NR not reported

cLevel: I individual, G group, C community

dSub population: Not specified intervention did not focus on a specific MSM sub population

eInterventions that met all PRS efficacy criteria

fCurrently not listed in the Compendium of Evidence-Based Behavioral Interventions (http://www.cdc.gov/hivtopics/research/prs/rr_chapter.htm)

gInterventions that were evaluated as methodologically rigorous but did not find a significant positive effect

hInterventions that found a significant positive effect but did not meet at least one PRS efficacy criteria

iInterventions that did not find a significant positive effect and did not meet at least one other PRS efficacy criteria

Overall Characteristics of MSM Behavioral Interventions

Introduction of ART

Among the 33 studies we examined, 11 (33 %) were conducted before ART or prior to 1996 while 22 studies (67 %) were conducted after ART. The majority (82 %) of pre-ART studies were conducted in small group settings while the remaining studies were implemented as community-level interventions (18 %). No individual-level interventions for MSM were conducted in this time period. In the post-ART period, half of the interventions were conducted as individual-level interventions while the other half were delivered in small groups. No post-ART studies were conducted as community-level interventions.

Intervention Sites

The majority of studies (n = 26, 79 %) were single-site studies, while four (12 %) were multi-site studies. The three remaining studies were conducted nationwide using the internet [39, 40] and telephone [41]. The most commonly reported intervention sites were urban settings where higher proportions of MSM congregate: New York City, San Francisco, Los Angeles, and Seattle. Regionally, most interventions were conducted in the West, followed by the Northeast. The fewest interventions were conducted in the South.

Targeted MSM Subpopulations

In terms of MSM subpopulations, 6 studies (18 %) exclusively focused on MSM of color and 7 studies (21 %) intervened with substance-using MSM. Nine (27 %) studies concentrated on HIV-positive MSM and 6 (18 %) targeted HIV-negative MSM; the majority of studies (n = 18, 55 %) did not include HIV status as an eligibility criterion. A higher percent of MSM studies conducted post-ART focused on high-risk MSM when compared to the pre-ART period: 36 versus 9 % focused on HIV-positive MSM and 27 versus 9 % focused on substance-using MSM respectively.

Intervention Design

Approximately 42 % of the studies used wait-list groups as comparisons (n = 14) while 11 studies (33 %) used HIV-related comparison groups and eight (24 %) used non-HIV related comparison groups. The majority of interventions were conducted in small groups (n = 20, 61 %). Eleven (33 %) were individual- level interventions while two (6 %) were community-level interventions. No interventions focused on couples. Nine (27 %) studies reported power analysis results [4250] (not shown in a table).

PRS Categories of Interventions

Of the 33 eligible MSM intervention studies, nine (27 %) met all PRS efficacy criteria and are considered EBIs [8, 9, 4345, 5053] while the remaining 24 (73 %) interventions that did not meet PRS efficacy criteria were identified as non-EBIs [39, 41, 42, 4649, 5468]. The most commonly identified failed criteria among the 24 non-EBIs were not finding significant positive intervention effects on a relevant behavioral or biological outcome (n = 13, 39 %) [42, 46, 47, 49, 54, 5860, 6366, 68], having follow-up assessments less than 1 month after the intervention (n = 6, 18 %) [41, 47, 55, 56, 62, 66], having analytic sample sizes less than 40 study participants per arm (n = 5, 15 %) [42, 48, 55, 64, 65] having less than a 60 % retention rate of study participants per arm (n = 3, 9 %) [39, 57, 65]. In addition, several other limitations such as substantial missing data, significant negative findings, data analysis issues or inconsistent findings were problems for 6 interventions (18 %) [40, 47, 59, 61, 63, 64].

The 24 interventions that did not meet at least one PRS efficacy criterion fell into 3 non-EBI groups. Seven interventions (21 %) met all efficacy criteria except for a significant positive finding and were categorized as rigorous non-EBIs [46, 49, 54, 58, 60, 63, 68]. Eleven interventions (33 %) reported a significant positive finding on a relevant behavioral or biological outcome but did not meet at least one other PRS criterion and were classified as positive non-EBIs [3941, 48, 5557, 61, 62, 67, 69]. Finally, the remaining six (18 %) interventions did not have a significant positive finding on a relevant behavioral or biological outcome and did not meet at least one other PRS criterion [42, 47, 59, 6466]. These interventions were identified as other non-EBIs.

Non-EBIs as a whole significantly differed from EBIs on study and intervention characteristics (Table 2). Non-EBIs were significantly more likely to exclusively focus on HIV-negative MSM and significantly less likely to be pilot tested, delivered by peers, and include intervention content on sexual communication (e.g., sexual negotiation) compared to EBIs. No EBIs included substance use as an eligibility criterion. Over half of the EBIs were conducted in the West and most EBIs used wait-list comparison groups. EBIs reported the most variety in terms of intervention delivery level (e.g., individual, small group, community) whereas the majority of non-EBIs were conducted in small groups. Excluding the 2 community-level interventions, EBIs reported the shortest median time span for an intervention and lowest median total number of hours allocated for an intervention.
Table 2

Percents and medians of select characteristics of evidence-based interventions (EBIs) and non-EBIs

 

EBIs

(n = 9)

n (%)

Non-EBIs (n = 24)

All non-EBIs

(n = 24)

n (%)

Rigorous non-EBIs

(n = 7)

n (%)

Positive non-EBIs

(n = 11)

n (%)

Other non-EBIs

(n = 6)

n (%)

Conducted pre or post ART

 Pre-ART

3 (33)

8 (33)

0

6 (55)

2 (33)

 Post-ART

6 (67)

16 (67)

7 (100)

5 (45)

4 (67)

Eligibility criteria

 Substance use

02,4

7 (29)

4 (57)2,4

1 (9)

2 (33)4

 Unprotected or discordant anal sex

2 (22)

10 (42)

3 (43)

5 (45)

1 (17)

 Exclusive focus on HIV-positive MSM

2 (22)

7 (29)

3 (43)

2 (18)

2 (33)

 Exclusive focus on HIV-negative MSM

4 (44)1,4

2 (8)1

04

2 (18)

04

 Exclusive focus on MSM of color

2 (22)

4 (17)

2 (29)

1 (9)

1 (17)

Regiona

 Northeast

3 (33)

8 (33)

5 (71)

1 (9)

2 (29)

 South

1 (11)

2 (8)

1 (14)

1 (9)

0

 Midwest

2 (22)

3 (13)

1 (14)

1 (9)

1 (14)

 West

6 (67)

13 (54)

4 (57)

6 (55)

3 (43)

 Internet

0

2 (8)

0

2 (18)

0

 Telephone

0

1 (4)

0

1 (9)

0

Study design

 Comparison group

  Wait-list

5 (56)

9 (38)

1 (14)

6 (55)

2 (29)

  HIV-related

4 (44)

7 (29)

3 (43)

3 (27)

1 (14)

  Non-HIV

04

8 (33)

3 (43)4

2 (18)

3 (43)4

Intervention characteristic

 Delivery level

  Individual

4 (44)

7 (29)

2 (29)

4 (36)

1 (14)

  Group

3 (38)

17 (71)

5 (71)

7 (64)

5 (83)

  Community

2 (25)

0

0

0

0

Intervention characteristic

 Median intervention time span (weeks)b

1.0

8.0

7.0

5.5

14.0

 Median total time of intervention (h)b

6.0

12.0

12.0

12.0

6.5

 Median session duration (h)b

2.0

2.0

2.0

2.0

2.0

 Median number of sessionsb

3.5

6

7

3

4.5

 Pilot tested

6 (67)1,3,4

6 (25)1

1 (14)4

5 (45)

03,4

 Conducted in a MSM setting

5 (56)2

4 (17)

02

2 (18)

2 (33)

 Delivered by peers

6 (67)1

4 (17)1

2 (29)

2 (18)

1 (17)

 Focused on sexual communication

8 (89)1,2,4

9 (38)1

2 (29)2,4

5 (45)

2 (33)4

Sample characteristic

 Median % black

8.0

13.7

39.0

7.5

10.8

 Median % Latino

13.6

13.9

18.9

15.1

9.0

 Median % white

72.5

70.7

36.3

84.1

76.7

 Median average age

31.1

35.4

38.2

33.8

37.0

 Median % high school graduate or less

22.62

29.2

39.42

16.1

29.0

1Significant Fisher’s Exact test for difference between EBIs and all non-EBIs (p < 0.05); 2Significant Fisher’s Exact test between EBIs and Rigorous non-EBIs (p < 0.05); 3Significant Fisher’s Exact test between EBIs and Other Non-EBIs (p < 0.05); 4Significant Fisher’s Exact test between EBIs and Rigorous and Other Non-EBIs combined (p < 0.05)

aSome studies were multi-regional studies

bDoes not include community level interventions

Rigorous Non-EBIs

Seven studies that met all PRS efficacy criteria with the exception of finding a significant positive effect were classified as rigorous non-EBIs. All rigorous non-EBIs were conducted post-ART. When compared to EBIs, rigorous non-EBIs were significantly more likely to include substance use as an eligibility criterion and report study participants with a high school education or less. Also, rigorous non-EBIs were significantly less likely to be conducted in MSM settings and include sexual communication as part of the intervention in contrast to EBIs (Table 2).

Positive Non-EBIs

Although these 11 studies found a significant positive effect, they did not meet other efficacy criteria. Having only an immediate follow-up assessment (n = 4, 36 %) [41, 55, 56, 62] was the most common reason for not meeting efficacy criteria. Other common reasons included having analytic sample sizes less than 40 per arm (n = 2, 18 %) [48, 55] and having less than 60 % retention in at least one study arm (n = 2, 18 %) [39, 57]. The remaining interventions had other study limitations such as reported contradictory findings, substantial missing data, significant negative findings or re-assigned participants from the intervention arm to the control arm or excluded participants who missed interventions sessions from the analyses (n = 4, 36 %) [40, 61, 67, 69]. As shown in Table 1, the internet- and telephone-based interventions were classified as positive non-EBIs. Although not statistically significant, positive non-EBIs reported the lowest median number of intervention sessions, lowest median percent of blacks, and highest median percent of whites in their samples compared to EBIs, rigorous, and other non EBIs. They also reported the lowest median percent of study participants with a high school education or less. When positive non-EBIs were compared to EBIs, no statistically significant differences in study characteristics, intervention content, or participant characteristics emerged.

Other Non-EBIs

Besides lacking a significant positive effect for a relevant behavioral or biological outcome, these 6 studies also did not meet at least one other efficacy criterion. For other non-EBIs, reasons for not meeting PRS criteria were having analytic sample sizes less than 40 per arm (n = 3, 50 %) [42, 64, 65], having follow-up assessments less than 1 month (n = 2, 33 %) [47, 66], having significant negative findings (n = 2, 33 %) [47, 64] and biased allocation of participants to intervention and comparison arms [59]. Other non-EBIs reported the highest percent of interventions conducted in groups (n = 5, 83 %) compared to EBIs, rigorous non-EBIs, and positive non-EBIs. Other non-EBIs were also significantly less likely to pilot test interventions compared to EBIs (Table 2).

Rigorous & Other Non-EBIs Combined

When we combined rigorous and other non-EBIs (n = 13) or studies that did not find a significant positive intervention effect and compared this group with EBIs (not shown), we found similar results from previous comparisons. For example, rigorous and other non-EBIs combined were significantly more likely to include substance use (33 % vs. 0, p = 0.05) and exclusively focus on HIV negative men (44 % vs. 0, p = 0.02) as eligibility criteria compared to EBIs. In addition, the combined group of rigorous and other non-EBIs was significantly less likely to be pilot tested (8 vs. 67 %, p = 0.01) and include sexual communication as part of the intervention (31 vs. 89 %, p = 0.01) in contrast to EBIs. Finally, rigorous and other non-EBIs combined were more likely to use a non-HIV related comparison group in contrast to EBIs (46 % vs. 0, p = 0.05).

Discussion

Of the 353 funded behavioral intervention studies evaluated by PRS since 1996, only 13 %, or 47, exclusively focused on MSM. Similarly, of the 74 EBIs identified by PRS, only 8 (11 %) were specifically designed for MSM. These percentages are much smaller than would be expected given the cumulative impact of HIV on MSM. The similarity of proportions between studies funded exclusively for MSM (13 %) and those found to be efficacious (11 %) suggests that the low number of EBIs for MSM is more attributable to misaligned funding priorities than to particular issues with interventions for MSM. Another possible explanation may be that few MSM-focused intervention studies are being proposed. Although 24 (73 %) of the 33 MSM-specific interventions included in this review were determined by PRS to be non-efficacious, they offer important lessons that can guide further research.

Lessons Learned from Non-Efficacious MSM-Specific Interventions

Not Finding A Significant Positive Effect

Findings from this study suggest that it may be difficult to demonstrate significant positive changes in sexual behavior with MSM who experience substance use issues. MSM with substance use issues often experience other psychosocial health problems like depression and partner violence [70]. From a “syndemics” perspective, it may be difficult for MSM who experience multiple interconnected issues such as substance use, depression, a history of sexual child abuse or partner violence to change sexual risk behaviors [46, 70, 71]. Efficacious interventions for substance-using MSM who experience multiple vulnerabilities may require a comprehensive approach that addresses substance use specifically (e.g., methadone treatment, individual therapy, contingency management) [72], accounts for various MSM populations (e.g., MSM who use injection drugs, MSM who use recreational drugs on the weekends), and targets venues combining sexual activity and substance use [71]. Addressing interconnected syndemic factors (e.g., treating depression, counseling for childhood sexual abuse or partner violence) while using biomedical strategies (e.g., PrEP and/or PEP) and structural interventions (e.g., decreasing stigma and discrimination related to being gay) should also be a part of this comprehensive approach [72, 73].

For substance-using and other high-risk MSM, it may be more efficacious to use individual-level interventions rather than group-level interventions. In our review, a larger proportion of non-EBIs (71 %) used groups to deliver the intervention compared to EBIs (33 %). Delivering interventions to high-risk MSM in a group setting may have some unintended outcomes. Some evidence indicates that small group formats may actually reinforce and sustain risk behaviors for high-risk MSM through meeting similarly or more-risky men [14]. An advantage of one-on-one interventions such as EXPLORE is that they can be tailored to meet a high-risk individual’s specific prevention needs [74]. However, to have maximum impact, it may be necessary to use a combination approach that includes other kinds of strategies such as biomedical or structural interventions to reinforce engaging in safer behaviors for high-risk MSM.

Another factor that may contribute to interventions not finding effects is the type of comparison group. In our review, interventions that did not find significant positive effects were more likely to use a non-HIV comparison group (i.e. served as attention control) while EBIs were more likely to use wait-list or HIV-related comparison groups. In addition, the number of MSM studies that used wait-list controls decreased over time while more recent studies were more likely to use HIV-related comparison groups as demand controls. These findings suggest that using non-HIV specific comparison groups that focus on general health promotion or stress reduction may encourage comparison participants to think about and possibly make changes in sexual health behaviors [39, 68]. However, using wait-list and HIV-related comparison groups also have their disadvantages. Some researchers argue for using control groups with minimal or no treatment when conducting rigorous intervention evaluations [58], but the problem with this approach is that it does not control for attention or demand effects. Also, temporarily withholding interventions from individuals and communities severely impacted by HIV may be perceived as unethical. A potential disadvantage of using HIV-related comparison groups is the use of enhanced or exceptional HIV prevention programs or diluted versions of the new intervention as comparison groups that may greatly reduce the ability to detect effects [16]. Researchers should consider the advantages and disadvantages of using each type of comparison group and address issues related to controlling for attention and demand effects. Using a standardized comparison arm that the HIV prevention field could agree upon as a prevention standard for MSM can facilitate comparing intervention effects across studies.

Sample Sizes and Retention Rates

One of the major reasons why interventions in our review did not meet efficacy criteria was not having at least 40 study participants per arm at follow-up assessment. Some of the interventions were conducted as small scale pilot studies to test feasibility and thus started with small sample sizes at baseline [42, 64]. Other interventions suffered from attrition problems [48, 65]. A prior power analysis can help to determine the adequate sample size for detecting an intervention effect but only 28 % of the studies in our review reported power analysis results. More transparent reporting of power analyses would facilitate evaluation of evidence. Finally, recruiting and retaining MSM in HIV behavioral prevention studies, especially studies with a substance use focus, may be more challenging than in the past [75]. The HIV prevention field would greatly benefit from comparative research and a compendium of “best practices” specifically focusing on recruiting and retaining MSM in intervention studies [75]. Newer recruitment methods such as respondent-driven, venue-day-time, and internet sampling show promise [76, 77] but more research is needed to evaluate their strengths and limitations.

Follow-Up Assessments

Having a follow-up assessment at least 1 month post-intervention was another common criterion that non-EBIs failed to meet in our review. These interventions only reported intervention effects immediately after the interventions were completed and thus the sustainability of the intervention effects is unclear. Given the emphasis on sustainable interventions in NHAS, HIV prevention researchers should consider longer follow-up assessments extending past immediate post- intervention to provide stronger evidence for sustainable changes in risk-reduction behavior.

Considerations for Future MSM Behavioral Interventions

MSM Inclusion and Focus in Interventions

A few intervention characteristics of EBIs that distinguished these interventions from those not meeting efficacy criteria might be helpful to consider when designing future behavioral interventions for MSM. For example, involving MSM in the design and implementation of interventions may be important. In our review, pilot testing and delivering the intervention through peers were more often characteristic of EBIs than non-EBIs. Pilot testing the intervention in the MSM community can provide information to improve the intervention. Using peers as intervention deliverers may help create an intervention environment that is safe and socially comfortable for MSM.

Researchers should also consider including intervention content on sexual communication, as it was associated with efficacious interventions. Other studies have also found sexual communication as an important component in behavioral interventions for MSM [11, 13]. Sexual negotiation and other communication skills such as HIV disclosure are critical if MSM rely on serostatus-based prevention strategies such as serosorting and strategic positioning to reduce their HIV risk. Innovative interventions are particularly important to help men with sexual negotiation and HIV disclosure skills in public and commercial sex environments where silence may be the norm [78]. As technological sites such as websites and phone apps become more popular to meet sexual partners among MSM, appropriate messages and strategies to enhance sexual communication will need to be developed with these new environments in mind.

MSM of Color

There was an increase in the number of interventions conducted with high-risk MSM post-ART compared to pre-ART, but compared to the number of interventions PRS has evaluated, the number of interventions specifically developed for MSM of color is relatively few. Developing efficacious behavioral interventions for these highly impacted MSM subgroups, especially African American and Latino MSM is urgently needed. EBIs and non-EBIs reported similar median percentages for including black, Latino, and white MSM in their samples, but only 2 EBIs in our review exclusively focused on MSM of color [51, 53]. One was developed for Asian and Pacific Islander MSM and was conducted in the pre-ART era, while the other EBI is a group intervention designed for HIV-negative African Americans. Currently, there are no efficacious behavioral interventions that have been specifically developed for Latino MSM or those that exclusively focus on HIV-positive MSM of color. Researchers should consider addressing the cultural and contextual factors that have been associated with risk behavior with MSM of color [79, 80]. Acknowledging and addressing the health disparities as well as the economic and social inequities confronting MSM of color are also needed as a part of a comprehensive approach for reducing HIV in these groups.

HIV Positive MSM

In this review, the percentage of interventions that exclusively focused on HIV-positive MSM were not significantly different between EBIs and non-EBIs, but EBIs were significantly more likely than non-EBIs to exclusively focus on HIV-negative MSM. This finding suggests that it may be challenging to demonstrate significant positive effects with HIV-positive MSM compared to HIV-negative MSM. Several efficacious behavioral interventions have been developed for HIV-positive persons in general (http://www.cdc.gov/hiv/topics/research/prs/subset-best-evidence-interventions.htm#link2.6) and a large percentage of study participants in these studies were MSM. In our review, however, only 2 [50, 52] out of 9 interventions for HIV-positive MSM were efficacious. A factor that may contribute to non-efficacious behavioral interventions for HIV-positive MSM is the level of intervention. In this review, the majority of non-EBIs for HIV-positive MSM were group-level interventions. Intervening at the individual level reduces sexual risk for HIV-positive persons in general [81] and may be particularly important for HIV-positive MSM. As mentioned earlier, group-level interventions may inadvertently reinforce risky behavior for high-risk MSM and have fewer opportunities for specific tailoring that may be important for behavior change. Another factor that may explain why we found relatively few efficacious behavioral interventions for HIV-positive MSM is the intervention setting or where the intervention is delivered. Previous research shows behavioral interventions conducted in clinical settings where HIV-positive persons regularly receive medical or social services may be more effective in reducing sexual risk than non-clinical settings [81]. Future behavioral interventions to reduce sexual risk for HIV-positive MSM should consider individual-level interventions and delivering these interventions in settings where other services are also available.

Other Gaps

HIV is greatly impacting MSM in the South [82], but we found the fewest interventions conducted there in our review. Because many MSM interventions have been developed and conducted in the West and Northeast, researchers should consider the cultural differences in the South (e.g., being located in the Bible belt) that can make HIV prevention more challenging for MSM and other vulnerable populations.

Our review also found that efficacious technology-based behavioral interventions have yet to be developed for MSM. Retention in these interventions may be challenging [39, 83] but behavioral interventions for MSM using digital media via the Internet, mobile phones, and computers have demonstrated the potential for increasing HIV testing [84], increasing HIV disclosure [85], and reaching high-risk MSM [86] and rural MSM [87].

Another gap in the MSM intervention literature is the lack of interventions for couples [88]. Some evidence indicates that most HIV transmissions occur within primary partner relationships [89] and, thus, developing couple-based behavioral interventions for MSM should be a priority. A recent pre-posttest design study that evaluated a couple-based intervention for methamphetamine-using black MSM found encouraging results for reducing sexual risk and illicit drug use [90].

Finally, early interventions that were evaluated as efficacious pre-ART should be replicated to determine if they are still efficacious post-ART [91]. It is important to recognize that interventions that worked 10 years ago may not be comparably effective today and that they should be continually assessed for effectiveness as they are implemented in practice [92].

From Research to Practice

While the identification of EBIs is informative, several more steps are necessary to translate scientific knowledge into impactful practice. PRS efficacy criteria currently focus on internal validity to ascertain causality between the intervention and behavior change, but scalability, sustainability, and cost-effectiveness are additional critical factors to determine which EBIs and combinations of EBIs are likely to achieve high-impact outcomes [22]. Emphasizing these other qualities reflects the importance of examining an intervention’s external or “viable” validity—or the extent to which the intervention is “practical, affordable, suitable, evaluable, and helpful in the real world” [93]. Also, standards for evaluating external validity or the capacity for an intervention to generalize to other populations or settings would be helpful for effective intervention implementation in affected communities. More implementation research about how interventions work in the field and how to maximize their reach and impact is also important for facilitating the translation of research into best practices.

Limitations

Our study has several limitations that warrant caution when interpreting the results. Although we contacted primary investigators to confirm the efficacy evaluation, the coding of study and intervention characteristics are based on published reports that may not provide complete information about the intervention. While we observed some patterns that may explain differences among EBIs and non-EBIs, there are multiple factors that may contribute to an intervention’s lack of evidence and it is difficult to disentangle a specific reason or combination of reasons. Also, our review focused on original studies; replications were excluded from this review. Replications are informative especially when examining generalizability, but few were conducted. These issues should be further examined when more studies become available in the literature. We excluded interventions that were not specifically designed for MSM but had substantial numbers of MSM in their samples (e.g., HIV clinic patients). Future studies may want to compare characteristics of interventions tailored specifically to MSM with general interventions tested with a majority of MSM in their samples. Finally, we did not examine cost data because few original studies provided the information.

Conclusions

In the fourth decade of HIV prevention, behavioral interventions will continue to play an important role in HIV prevention efforts for MSM but need to be strengthened to make a substantial impact as part of a comprehensive HIV prevention package. They should target MSM at the highest risk of transmitting and acquiring HIV and consider the gaps in HIV behavioral intervention research for MSM. Behavioral interventions should also focus on relevant issues for MSM and include MSM in their design and implementation. In short, HIV behavioral interventions will need to continually evolve, as they have throughout the epidemic, to help reduce HIV vulnerability for gay, bisexual, and other men who have sex with men.

Copyright information

© Springer Science+Business Media New York (outside the USA) 2013