AIDS and Behavior

, 15:1691

Evaluation of an HIV Prevention Intervention for African Americans and Hispanics: Findings from the VOICES/VOCES Community-Based Organization Behavioral Outcomes Project


    • Division of HIV/AIDS Prevention, National Center for HIV/AIDS, Viral Hepatitis, STD, and TB PreventionCenters for Disease Control and Prevention
  • A. Patel-Larson
    • Division of HIV/AIDS Prevention, National Center for HIV/AIDS, Viral Hepatitis, STD, and TB PreventionCenters for Disease Control and Prevention
  • K. Green
    • Division of HIV/AIDS Prevention, National Center for HIV/AIDS, Viral Hepatitis, STD, and TB PreventionCenters for Disease Control and Prevention
  • E. Shapatava
    • Ginn Group Inc.
  • G. Uhl
    • Division of HIV/AIDS Prevention, National Center for HIV/AIDS, Viral Hepatitis, STD, and TB PreventionCenters for Disease Control and Prevention
  • E. J. Kalayil
    • MANILA Consulting Group, Inc.
  • A. Moore
    • MANILA Consulting Group, Inc.
  • W. Williams
    • MANILA Consulting Group, Inc.
  • B. Chen
    • MANILA Consulting Group, Inc.
Original Paper

DOI: 10.1007/s10461-011-9961-7

Cite this article as:
Fisher, H.H., Patel-Larson, A., Green, K. et al. AIDS Behav (2011) 15: 1691. doi:10.1007/s10461-011-9961-7


There is limited knowledge about whether the delivery of evidence-based, HIV prevention interventions in ‘real world’ settings will produce outcomes similar to efficacy trial outcomes. In this study, we describe longitudinal changes in sexual risk outcomes among African American and Hispanic participants in the Video Opportunities for Innovative Condom Education and Safer Sex (VOICES/VOCES) program at four CDC-funded agencies. VOICES/VOCES was delivered to 922 high-risk individuals in a variety of community settings such as substance abuse treatment centers, housing complex centers, private residences, shelters, clinics, and colleges. Significant risk reductions were consistently observed at 30- and 120-days post-intervention for all outcome measures (e.g., unprotected sex, self-reported STD infection). Risk reductions were strongest for African American participants, although Hispanic participants also reported reducing their risky behaviors. These results suggest that, over a decade after the first diffusion of VOICES/VOCES across the U.S. by CDC, this intervention remains an effective tool for reducing HIV risk behaviors among high-risk African American and Hispanic individuals.


HIV sexual risk behaviorsHIV risk reduction interventionsVOICES/VOCESAfrican AmericansHispanicsHIV/AIDS


Historically, African Americans and Hispanics in the United States have been disproportionately affected by HIV and other sexually transmitted diseases (STDs), compared with non-Hispanic whites [14]. In 2008, African Americans represented 13% of the U.S. population but they accounted for 52% of all new diagnoses of HIV infection; Hispanics accounted for 13% of the U.S. population but they represented 17% of all new diagnoses [5, 6]. The HIV incidence rate is seven times as high among African Americans and 2.5 times as high among Hispanics, as among whites [7]. Moreover, the overall lifetime risk of being infected is much higher for African Americans and Hispanics as for whites [8]. Racial and ethnic disparities are also evident in STD rates [4]. STD infection facilitates HIV transmission such that HIV-positive individuals who have other STDs have an increased likelihood of spreading HIV to others, and HIV-negative individuals who have other STDs have an increased likelihood of acquiring HIV [9]. Given these disparities, culturally sensitive HIV prevention programs that emphasize behavior change in risky sexual practices are critical to slowing HIV/STD infection among minority populations. Past research indicates culturally tailored behavioral interventions are, in fact, more effective among African American and Hispanic populations in reducing sexual risk behaviors than interventions that are not culturally sensitive [1012].

The Video Opportunities for Innovative Condom Education and Safer Sex (VOICES, or VOCES for Spanish-speaking participants) program is a single-session HIV/STD prevention intervention that emphasizes condom use and condom negotiation skills among African American and Hispanic men and women. During the session, a culturally specific video on condom use and negotiation is shown to a small group of same-sex individuals, followed by a group discussion led by a facilitator and role-play exercises among the participants [1315]. In general, the VOICES/VOCES intervention communicates prevention messages in a brief period of time (45 min to an hour) and is structured so that it can be integrated into an assortment of services already provided by service providers.

VOICES/VOCES efficacy research measured self-reported change in knowledge, attitudes, and behavioral intentions for condom use [13, 14] and biological markers [15] and focused on African American and Hispanic male and female clients visiting STD clinics. Intervention participants reported an increase in knowledge, more positive attitudes and greater efficacy for condom use, increased intentions to use condoms, and they were more likely to later redeem condom coupons, relative to control participants [13, 14]. Moreover, the infection rate of new STDs was significantly lower in male clients after intervention participation compared to the control group [15]. Further evidence supporting the use of VOICES/VOCES as a preventative intervention for the transmission of HIV and other STDs comes from a cost-effectiveness study that tracked male clients through the New York City surveillance system and collected data on STD rates and condom acquisition [16]. Results showed that participation in VOICES/VOCES was highly cost-effective in terms of HIV treatment costs, number of HIV infections averted, and intervention implementation in STD clinics. Furthermore, reductions in STD outcomes persisted over time and were greatest among men who reported having multiple sex partners at the time of study enrollment. These findings suggest that participation in VOICES/VOCES may lead to reductions in risky behavior among very high-risk individuals which could, in turn, have a positive impact on HIV transmission rates within a community.

A major goal of HIV prevention efforts is to reduce the number of new infections that occur each year. Behavioral interventions like VOICES/VOCES that target high-risk, HIV-negative persons and HIV-positive persons play a significant role in the Centers for Disease Control and Prevention’s (CDC) HIV/AIDS Prevention Strategy [17]. CDC regularly conducts systematic literature reviews to identify behavioral interventions with strong evidence of efficacy for reducing HIV risk behaviors, and then works with researchers and community-based partners to translate evidence-based interventions into user-friendly intervention packages [18]. Through the Diffusion of Effective Behavioral Interventions (DEBI) project [19], intervention materials, resources, training, and capacity building assistance are disseminated nationwide to community-based organizations (CBOs), state and local health departments, and other service providers. As of 2010, materials and trainings had been disseminated for 26 Effective Behavioral Interventions (EBIs), including VOICES/VOCES. From 2002 to 2009, CDC trained 775 agencies on the delivery of VOICES/VOCES [20].

While data from RCTs have demonstrated EBIs to be efficacious under experimental conditions, there is limited knowledge about whether the delivery of EBIs in ‘real world’ settings will produce similar outcomes [2123]. One issue to consider is that CBO implementation may differ significantly from intervention implementation in the original efficacy trials. For example, although the majority of agencies funded by CDC to deliver VOICES/VOCES have reported adherence to the intervention’s core elements (i.e., key components of the intervention that are most likely to produce intended outcomes), 41% of agencies reported delivery to audiences for whom the intervention was not originally designed and tested in research trials [24]. Additionally, EBIs may be adapted in a variety of ways by local providers depending on agency resources and the needs of the populations served by the agency. Public health officials acknowledge programmatic adaptations can enhance local suitability of the intervention and help service providers obtain buy-in from stakeholders. However, program impact may be compromised if alterations are not appropriate for the target population, or if critical core elements are modified [25]. As a first step toward evaluating EBIs implemented in the field with federal funds, CDC began the Community-based Organization Behavioral Outcomes Project in 2006 to conduct outcome monitoring on selected EBIs being delivered at CDC-funded CBOs. Outcome monitoring is defined as the collection of participant behavioral data before and after exposure to an intervention to determine whether the intervention achieved its outcome-related objectives. The primary intent of these evaluation studies was to determine if the implementation of specific EBIs in community settings would produce similar outcomes as those reported in the original efficacy trials. VOICES/VOCES was one of the first EBIs to be evaluated through this project.

The purpose of this paper is to describe longitudinal changes in sexual risk behavior and STD outcomes among VOICES/VOCES participants at four CDC-funded CBOs. This paper will address the following questions: (1) do clients who participate in VOICES/VOCES in community settings report changes in sexual risk behaviors 1 and 4 months after receiving the intervention; (2) which client-level factors are associated with reported changes in sexual risk behaviors; and (3) do participants report fewer STD diagnoses at follow-up?

Based on the original efficacy studies which reported increased intentions and efficacy for condom use and increased condom acquisition [13, 14], we anticipated the intervention would have a positive effect on clients’ sexual risk behaviors at follow-up. We also sought to determine whether clients would report significantly fewer STD diagnoses after participation in VOICES/VOCES. In the original research, STD infection rates among clients at STD clinics were reduced after participation in VOICES/VOCES [15]; however, it is unknown if a similar reduction in self-reported STD diagnoses would be observed among a more diverse population of individuals who received the intervention in a variety of settings.



Fifteen CBOs were funded through a competitive cooperative agreement by CDC from July 1, 2004 to June 30, 2010 to implement VOICES/VOCES to members of racial/ethnic minority communities at high risk for HIV infection. Four of these 15 CBOs were given additional funding (through a separate, competitive cooperative agreement) to conduct a program evaluation on VOICES/VOCES from September 1, 2006 to August 31, 2008. Eligibility criteria for agency participation in the evaluation were: (1) at least 6 months’ experience delivering VOICES/VOCES to high-risk individuals, (2) capacity to recruit at least 200 participants into the evaluation and to retain 80% of them at each of the two follow-up time points, (3) capacity to develop data and program quality assurance plans prior to evaluation data collection, and (4) implementation of VOICES/VOCES must include all core elements of the intervention. VOICES/VOCES has 4 core elements (i.e., fundamental components of the intervention that are most likely to produce intended outcomes) [19]: (1) show culture-specific videos portraying condom negotiation; (2) convene small group, skill-building sessions to work on overcoming barriers to condom use; (3) educate clients about different types of condoms and their features; and (4) distribute samples of condoms identified by clients as best meeting their needs. The agencies were located in Houston, TX (CBO A), Columbia, SC (CBO B), Decatur, GA (CBO C), and Laredo, TX (CBO D). All four agencies were moderately small in size (less than 50 employees at each agency) and offered HIV prevention programs and HIV counseling, testing, and referral services to local clients for an average of 14 years. CBOs C and D also offered outpatient drug and alcohol treatment services. All CBOs served clients in metropolitan areas; CBOs B and D also served clients in rural areas.

VOICES/VOCES Implementation

Although CDC and CBO staff worked collaboratively throughout the project period to improve evaluation procedures as needed, an important goal of the evaluation was to avoid interfering with the actual implementation of the intervention. CBO staff were instructed to deliver VOICES/VOCES as they had delivered it before the evaluation, according to CDC and DEBI training guidelines and based on guidance provided by CDC program project officers. CDC CBOP staff did not provide assistance on recruitment or delivery of the intervention. Additionally, CDC staff were careful to avoid interfering with any services that may have been provided to VOICES/VOCES clients during the evaluation period. This section describes how VOICES/VOCES was delivered at these agencies during the evaluation project period.


CBO staff recruited VOICES/VOCES participants from a variety of sites, including community centers, shelters, substance abuse treatment centers, barber and beauty shops, transitional housing programs, and family-planning clinics. In general, agency staff focused their recruitment efforts on sites where they were more likely to find clients that were at very high risk for HIV infection or transmission. Participants were also recruited through social networks, referrals from other agencies, and from in-house programs at their own agencies such as HIV counseling and testing programs. The specific populations targeted by each CBO included the same target populations used in the original VOICES/VOCES efficacy research (i.e., HIV-negative individuals at high risk for HIV/STD infection) [1315]. CBOs A, B, and C targeted African Americans, and CBOs A and D targeted Hispanics. In addition, all four CBOs expanded their recruitment efforts to target risk populations that were not originally included in the research studies. For example, CBO A targeted substance abusers; CBO B targeted homeless individuals; CBOs C and D targeted men who have sex with men (MSM), HIV-positive individuals, and current and previous injection drug users (IDU); and CBOs B and C targeted persons who were unaware of their HIV status and/or enrolled into an HIV testing program. Although each agency used slightly different eligibility criteria for enrollment into VOICES/VOCES, potential participants were deemed to be at high risk for HIV/STD infection if they reported risky sexual or drug activities in the previous 3 months (e.g., unprotected sex with an HIV-positive partner, unprotected sex in exchange for money or drugs, needle sharing, having multiple sex partners).

Intervention Delivery

VOICES/VOCES was delivered many times at the four CBOs during the program evaluation time period. Because each CBO was only required to recruit and retain a total of 200 participants in the evaluation, details about intervention delivery were only documented for the sessions attended by evaluation participants. Table 1 describes characteristics of the VOICES/VOCES sessions that were delivered to evaluation participants. VOICES/VOCES was delivered at the agencies and in substance abuse treatment centers, housing complex community centers, private residences, shelters, family planning clinics, and other community settings (Table 1). To conduct the intervention, facilitators convened small groups of VOICES/VOCES participants in a space or area that allowed for a private discussion within the group. Across CBOs, the average number of participants attending an intervention session was 5.9 (SD = 1.8). Whenever possible, groups were comprised of individuals with the same gender and ethnicity to help everyone feel comfortable and to allow for the exchange of culturally appropriate, gender-specific HIV prevention messages during discussions. Information on HIV risk behaviors and condom use was delivered using a video (Love Exchange) or movie clips (Booty Call, Noah’s Ark) that targeted African Americans, or a video that targeted Hispanics (Porque Si). A group discussion led by one or more intervention facilitators followed. Skill-building in condom use and condom negotiation were modeled in the videos, demonstrated afterward by agency staff using penile models, and then role-played and practiced by participants during discussion. A board display that showed features of various condom brands in both English and Spanish was also used during the session to facilitate discussion. At the end of the session, participants were given condom samples. All agencies reported implementing all intervention core elements [19], with one exception related to condom distribution (at CBO D, noted below). Across agencies, intervention sessions were, on average, 48.7 min long (SD = 6.9).
Table 1

Characteristics of VOICES/VOCES intervention sessions attended by evaluation participants, by agency—September 2006 to August 2008






Agency location

Houston, Texas

Columbia, SC

Decatur, GA

Laredo, TX

Intervention delivery sites

Substance abuse treatment centers, shelters, parole offices, housing complex community centers, public health clinics, community enrichment centers

Mobile unit (parked at homeless shelters, bus terminals, laundromats, and store fronts), agency

Private residences, substance abuse treatment centers, shelters, housing complex community centers, agency

Shelters, private residences, community/trade colleges, family planning clinics, Job Corps centers, substance abuse treatment centers, agency

Average number of intervention participants per session

6.7 (SD = 1.5)

5.7 (SD = 1.7)

6.1 (SD = 1.9)

5.5 (SD = 1.8)

Average session length (in min)

50.7 (SD = 8.1)

46.1 (SD = 6.7)

55.1 (SD = 8.8)

46.5 (SD = 3.2)




• Showed 2 new culture-specific videos for African Americans and MSM

• Developed facilitation guide for MSM

• HIV-negative and HIV-positive participants scheduled for different sessions

• In correctional settings, gave participants condom coupons for use after release

• HIV-negative and HIV-positive participants scheduled for different sessions

SD standard deviation, MSM men who have sex with men

Two CBOs adapted the intervention materials or procedures (Table 1). CBO C showed two new videos that were tailored specifically to African Americans [one for heterosexuals and one for MSM] and developed an accompanying facilitation guide for MSM. CDC assisted agency staff with these adaptations and approved all final materials. When delivering VOICES/VOCES in a jail, CBO D did not perform the condom demonstration or distribute condoms since these activities were prohibited in the correctional setting. Instead, participants (n = 6) were given condom coupons they could redeem upon their release. Additionally, CBOs C and D conducted separate intervention sessions for HIV-negative and HIV-positive individuals with appropriate peer facilitators.

All CBOs offered monetary (e.g., gift cards) or non-monetary (e.g., condoms, hygiene kit, bus tokens, t-shirts) incentives to VOICES/VOCES participants and/or to individuals who hosted the intervention in their homes.

Program Evaluation


Staff recruited participants for the evaluation activity in two ways: (1) by recruiting participants for VOICES/VOCES and the program evaluation at the same time, and (2) by inviting individuals who had already agreed to participate in VOICES/VOCES to participate in the evaluation. To be eligible for the evaluation project, participants must have agreed to provide consent, agreed to supply sufficient contact information for follow-up interviews, agreed to provide demographic and behavioral risk information at each time point, been 16 years old or older, and indicated they were at high risk for HIV infection (as described in “VOICES/VOCES Implementation” section above). Persons were permitted to participate in the intervention regardless of whether they agreed to participate in the program evaluation. Data at follow-up time points were collected regardless of whether an individual completed the intervention based on an intent-to-treat approach. Only one participant (at CBO C) who enrolled into the evaluation project did not complete the intervention. It should also be noted that not all clients who received the VOICES/VOCES intervention during this project period were recruited into the evaluation study. The evaluation participants represented approximately 49% of the clients who received VOICES/VOCES at these four agencies during the evaluation time period (range across CBOs = 31–81%).

The study followed a repeated measures design: data were collected prior to the start of the intervention session (baseline) and at two time points after participation in the intervention, at approximately 1 month post-intervention (follow-up 1) and at approximately 4 months post-intervention (follow-up 2). A comparison group was not used in this study. The second follow-up time point was included to assess whether any observed behavioral change at the first follow-up would be sustained over time. The window period for scheduling interviews was 1 week before to 2 weeks after the 30-day mark for follow-up 1, and 2 weeks before to 2 weeks after the 120-day mark for follow-up 2. Shorter follow-up periods were chosen rather than longer follow-up periods (e.g., six and 12 months after the intervention) because there was concern that it would be difficult to retain clients participating in a short, single session intervention over a long period of time. A low retention rate could result in a smaller final data set whereby there might not be enough statistical power to conduct the planned analyses [26].

Baseline and follow-up data were collected via in-person client- or staff-administered questionnaires. Follow-up data were collected via client- or staff-administered questionnaires in-person or via phone interviews. Standardized data collection instruments were not used as the original intent of the project was to monitor VOICES/VOCES outcomes without interruption or alteration of service delivery at the agencies. Although a standardized instrument was not used, all questionnaires used in this study were developed by CDC staff and then tailored as needed by staff at each agency. All CBOs provided monetary incentives to evaluation participants at follow-up interviews. A detailed discussion of incentives and other retention strategies used in this study is described elsewhere [27].


All agencies funded by CDC to conduct HIV prevention services are required to collect and report a core set of standardized variables (National HIV Prevention Program Monitoring and Evaluation, or NHM&E, variables), such as demographics and general risk behaviors. The CBOs included in this study were funded to collect additional variables for the purposes of evaluation, for example, detailed risk behaviors collected at three time points. All data were entered, managed, and submitted to CDC using the Program Evaluation Monitoring System (PEMS), a web-based software designed for secure data entry and management [28].

The following participant information was collected for the purposes of this evaluation project: current gender, age, race, ethnicity, primary language, relationship status, education, HIV test history, and most recent HIV test result. Information about HIV-related risk factors (90-day recall period) was also collected at baseline: incarceration, exchanging sex for drugs or money, sex with someone the participant met on the Internet, housing status, self-reported STD diagnosis, sex events (total sex events and only unprotected sex events), and sex partners (male, female, transgender). HIV risk group (previous 90 days) was also computed for each participant: MSM = male who reported sex with another male; IDU = person who reported injecting drugs; heterosexual contact = male who reported only sex with females or female who reported only sex with males; other risk = transgender participant or female who reported only sex with other females; no drug or sexual risk = person who reported no sex events or injection drug use. HIV risk groups were mutually exclusive and were assigned to each participant based on the risk behavior reported at baseline. When multiple risk behaviors were reported, the groups were assigned in the following order (from the highest level of transmission risk to the lowest level of transmission risk): MSM/IDU, MSM, IDU, and heterosexual contact.

Outcome measures were based on sexual activities that occurred during a 30-day recall period. For all sex-related variables, a “sex event” was defined as an episode of anal or vaginal intercourse with a partner; “unprotected sex” was defined as engaging in anal or vaginal sex without a condom. Participants reported the following sex outcomes for a 30-day recall period: number of sex partners, number of sex events, number of unprotected sex events, number of unprotected sex events with someone who exchanges sex for drugs or money, and number of unprotected sex events while high or intoxicated. From these variables, the following outcome measures were calculated as dichotomous (yes/no): multiple sex partners (i.e., more than one), any unprotected sex events, any unprotected sex events with someone who exchanges sex for drugs or money, any unprotected sex events while high or intoxicated, and reporting any unprotected sex events in combination with reporting multiple sex partners.

Statistical Analysis

Descriptive statistics for sociodemographic, HIV risk factor, and HIV risk behavior variables were generated for all participants using baseline data. Descriptive statistics are reported by each CBO and overall.

Several data cleaning procedures were performed before analysis of outcomes. Where inconsistencies across outcome measures were observed (e.g., 0 sex partners and 1 or more sex events were reported for a particular participant), a series of recodes was performed at each time point to make all data for each participant consistent: (1) if number of total sex events = 0, recode number of sex partners to 0; (2) if number of sex partners = 0, recode all sex outcomes to 0; (3) if number of sex partners > number of sex events, recode number of sex partners to match number of sex events; and (4) if number of unprotected sex events > number of total sex events, recode number of total sex events to match number of unprotected sex events. Outcome measures for 33 participants (4% of sample) were changed as a result of these recodes. Outcome measures for an additional 184 participants had to be recoded because of PEMS data entry errors that resulted in both the selection of a “no risk identified” variable and the entry of risk behaviors for the same client.

The second step was to impute missing values when number of sex partners, number of sex events, and number of unprotected sex events for a participant were all missing at a specific time point. Missing follow-up data were imputed with data from the previous time point first and, if those data were missing, data from the remaining time points were used. Missing baseline data were imputed with follow-up 1 data first, if available, and then follow-up 2 data. (To avoid the possibility that new inconsistencies across outcome measures would be introduced, data were not imputed when only one or two of these outcome measures were missing at a specific time point.) This imputation process followed a conservative approach for replacing missing data with actual values: by replacing missing data with data reported at an adjacent time point, we assumed that there was no change across these two time points. The outcome data for a total of 148 participants were modified at one or more time points as a result of this process.

The third step was to inspect all outcome variables for skewed distributions and outliers. Number of unprotected sex events, number of unprotected sex events with someone who exchanges sex for drugs or money, and number of unprotected sex events while high or intoxicated were non-normally distributed. To understand any change in behavior after intervention participation, these measures were converted to dichotomous (yes/no) variables prior to conducting the regression analyses described below. Although these measures could have been analyzed as rates by applying a Poisson model, the dichotomized variables were applied in order to evaluate the risk of ever engaging in the risk behavior over the recall period. A relatively high proportion of individuals reported zero sex events and the dichotomization served to minimize the effect of potentially over-reported values amongst few individuals. If a participant indicated engaging in a risk behavior, a “1” was assigned for that variable. If a participant indicated not engaging in a risk behavior, a “0” was assigned. Missing values for the original variable were also coded as missing for the dichotomous variable.

After the data cleaning procedures were completed, generalized estimating equations (GEE) were used to estimate odds ratios (ORs), rate ratios (RRs), and 95% confidence intervals (CIs) for the associations between time, CBO, race/ethnicity, HIV risk group, and each outcome. GEE models were introduced by Liang and Zeger [29, 30] as a method to account for the correlation of responses within subjects commonly observed during analysis of repeated measures data with non-normal response variables. Logistic regression with GEE was used for all dichotomous outcomes and a Poisson regression with GEE was used for count data (i.e., the number of partners). The exponential of the logistic model parameters represent ORs which are calculated by dividing the odds of an event in one group by the odds of the event in another group. The exponential of the Poisson model parameters represent RRs which are calculated by dividing the mean partners per 30 days in one group by the mean partners per 30 days in the referent group. GEE is an expansion upon the generalized linear model which accounts for correlated data in calculating variance and P-values. GEE models were developed to account for expected correlation in repeated measurements for each individual within each CBO.

Model results, 95% CIs, and P-values are reported for number of sex partners, unprotected sex events, unprotected sex events with someone who exchanges sex for drugs or money, unprotected sex events while high or intoxicated, unprotected sex events in combination with multiple sex partners, and STD diagnosis.


Evaluation Participant Characteristics at Baseline

The study goal was to enroll 800 clients into the evaluation project, 200 per agency. A total of 922 people were enrolled into the evaluation component and completed a baseline data collection questionnaire. Of the 922 participants, 60% were male, 40% were female, and 0.4% (n = 4) were transgender (Table 2). Participants had a mean age of 37.0 years (SD = 11.4; range = 16–64 years old). Almost two-thirds of participants (63%) reported their race/ethnicity as non-Hispanic African American, 31% as Hispanic, 5% as non-Hispanic white, 0.3% as non-Hispanic American Indian, and 0.2% as non-Hispanic Asian. The majority of participants reported that their primary language was English (86%), 57% indicated they were single and had never been married, and 36% of participants reported they had a high school diploma, GED, or equivalent. Most of the participants reported a previous HIV test (79%) and about one-fifth (19%) of those previously tested reported they were HIV-positive.
Table 2

Evaluation project participant characteristics at baseline, by agency and overall


CBO A (n = 200)

No. (%)a

CBO B (n = 220)

No. (%)a

CBO C (n = 245)

No. (%)a

CBO D (n = 257)

No. (%)a

Total (N = 922)

No. (%)a



54 (27)

149 (68)

115 (47)

127 (50)

367 (40)


146 (73)

71 (32)

128 (52)

128 (49)

551 (60)


0 (0)

0 (0)

2 (0.8)

2 (0.8)

4 (0.4)

Age (years)


0 (0)

1 (1)

5 (2)

0 (0)

6 (0.7)


30 (15)

12 (6)

31 (13)

89 (35)

162 (18)


59 (30)

33 (15)

48 (20)

94 (37)

234 (25)


42 (21)

79 (36)

60 (25)

62 (24)

243 (26)


69 (35)

95 (43)

101 (41)

12 (5)

277 (30)



30 (15)

4 (2)

4 (2)

252 (98)

290 (31)

 African American, not Hispanic

122 (61)

214 (97)

240 (98)

1 (0.4)

577 (63)

 White, not Hispanic

45 (23)

1 (1)

0 (0)

3 (1)

49 (5)


3 (2)

1 (1)

1 (0.4)

1 (0.4)

6 (0.7)

Primary language


192 (96)

220 (100)

245 (100)

133 (52)

790 (86)


4 (2)

0 (0)

0 (0)

109 (42)

113 (12)


4 (2)

0 (0)

0 (0)

15 (6)

19 (2)

Relationship status

 Single, never married

94 (47)

139 (63)

151 (62)

139 (54)

523 (57)

 Married or partnered

32 (16)

11 (5)

19 (8)

76 (30)

138 (15)

 Married, separated

29 (15)

31 (14)

24 (10)

18 (7)

102 (11)

 Divorced or widowed

44 (22)

39 (18)

50 (20)

24 (9)

157 (17)


 Less than high school degree

57 (29)

91 (41)

57 (23)

114 (44)

319 (35)

 High school degree, GED, or equivalent

75 (38)

93 (42)

76 (31)

87 (34)

331 (36)

 Some college

53 (27)

32 (15)

76 (31)

52 (20)

213 (23)

 Bachelor’s or postgraduate degree

15 (8)

4 (2)

36 (15)

4 (2)

59 (7)

Previous HIV test

154 (77)

165 (75)

221 (90)

191 (74)

731 (79)

 Of those previously tested, those HIV-positive

22 (11)

19 (9)

85 (35)

11 (4)

137 (19)

aFrequencies and percentages represent the number of participants at a given agency who have a specific characteristic. Percentages may not add to 100% because of missing data or rounding. Missing data were only reported for relationship status (n = 2, 0.2%), previous HIV test (n = 5, 0.5%), and (of those previously tested) HIV status (n = 15, 2%)

Table 3 describes participant HIV risk factors and behaviors reported for the 90 days preceding baseline. Fourteen percent of participants reported they had been incarcerated, 24% had exchanged sex for money, 7% reported having sex with someone they met via the Internet, and 31% reported they did not have permanent housing. Eight percent of participants reported a recent STD diagnosis. With regard to HIV risk group, most participants (74%) reported heterosexual contact, 13% were MSM, 8% were IDUs, 5% reported no drug or sexual risk, and 1% reported other risks. An additional four participants reported both MSM and IDU behaviors.
Table 3

Evaluation project participant HIV risk factors and HIV risk behaviors (past 90 days) at baseline, by agency and overall


CBO A (n = 200)

No. (%)a

CBO B (n = 220)

No. (%)a

CBO C (n = 245)

No. (%)a

CBO D (n = 257)

No. (%)a

Total (N = 922)

No. (%)


58 (29)

37 (17)

17 (7)

21 (8)

133 (14)

Exchanged sex for money

45 (23)

100 (46)

49 (20)

30 (12)

224 (24)

Sex with Internet partner

11 (6)

3 (1)

40 (16)

7 (3)

61 (7)

Non-permanent housing status

71 (36)

103 (47)

76 (31)

38 (15)

288 (31)

STD diagnosis

13 (7)

9 (4)

17 (7)

35 (14)

74 (8)

HIV risk group


0 (0)

0 (0)

2 (0.8)

2 (0.8)

4 (0.4)


9 (5)

25 (11)

61 (25)

20 (8)

115 (13)


16 (8)

2 (1)

7 (3)

48 (19)

73 (8)

 Heterosexual contact

141 (71)

191 (87)

168 (69)

179 (70)

679 (74)

 Other riskb

1 (0.5)

0 (0)

5 (2)

3 (1)

9 (1)

 No drug or sexual riskc

33 (17)

2 (1)

2 (1)

5 (2)

42 (5)

MSM men who have sex with men, IDU injection drug user

aFrequencies and percentages represent the number of participants at a given agency who have a specific characteristic

bOther risk group represents transgender clients (n = 3) or females who reported sex with other females (n = 6)

cNo risk group represents clients who reported no sexual activity or injection drug use during recall period

Attrition and Retention

At follow-up 1, a total of 833 participants (90%) were retained in the program evaluation. A total of 840 participants (91%) were retained at follow-up 2. Overall, 883 participants (96%) returned for at least one follow-up interview. On average, follow-up 1 data collection occurred 35.3 days (SD = 9.9) after the intervention date (goal was 30 days); follow-up 2 data collection occurred 124.2 days (SD = 11.2) after the intervention date (goal was 120 days). Compared to the 883 participants who returned for at least one follow-up interview, the 39 participants lost to any follow-up were significantly more likely to be 18–34 years old [X2(4) = 14.11, P < 0.01], Hispanic or white [X2(3) = 22.36, P < 0.0001], and primary Spanish speakers [X2(2) = 12.03, P < 0.01]. No differences in attrition were observed for gender, relationship status, education, HIV testing history, or HIV status. Additional information about retention strategies utilized by each agency in the evaluation project is detailed elsewhere [27].

Changes in Behavioral Outcomes Over Time

Table 4 shows the outcome measures at baseline, follow-up 1, and follow-up 2, averaged across the four CBOs. The data included in this table were recoded for consistency (as described in “Methods” section) but no other data transformations were made. The purpose of this table is to show the general risk reduction trends that were observed for all outcome measures across all agencies. On average, all sexual risk behaviors decreased from baseline to follow-up 1, and from follow-up 1 to follow-up 2. Self-reported STD diagnoses decreased from baseline to follow-up 2. (STD diagnoses reported at follow-up 1 are not included because the 90 day recall period overlaps with the baseline data recall period.)
Table 4

Sexual risk behaviors (past 30 days) and STD diagnosis (past 90 days) reported by evaluation project participants, by time point


Baseline (N = 807)a

1 month (N = 716)a

4 months (N = 729)a

Mean (SD)

N (%)

Mean (SD)

N (%)

Mean (SD)

N (%)

Number of sex partners

3.34 (5.87)

2.42*** (2.70)

2.00*** (2.17)

Number of sex events

10.41 (12.59)

9.23^ (12.31)

9.11* (9.60)

Number of unprotected sex events

7.80 (11.67)

5.61*** (12.08)

4.64*** (9.03)

 With someone who exchanges sex for drugs or money

1.83 (6.42)

0.76*** (2.93)

0.45*** (1.96)

 While high or intoxicated

4.31 (8.63)

2.30*** (6.66)

1.58*** (4.25)

Unprotected sex events and multiple sex partnersb

486 (60)

308*** (43)

219*** (30)

STD diagnosisc

61 (8)


14 (2)

P < 0.05; * P < 0.01; ** P < 0.001; *** P < 0.0001

aSexual risk behaviors at each time point are reported for participants who reported at least one sex event at that time point

bPeople who reported more than one sex partner and at least one unprotected sex event

cSTD diagnosis is self-reported; follow-up 1 data are not reported because of overlap between the recall period and baseline

After consistency recodes and missing data imputations were completed, data for participants with the following conditions were excluded from the data set before conducting the GEE analysis: (1) number of sex partners, number of total sex events, and number of unprotected sex events were all missing (n = 5); (2) 0 sex partners and 0 sex events were reported at all time points (n = 32); (3) under the age of 18 (n = 6); (4) self-reported seroconversion to HIV-positive status at either follow-up (n = 6); and (5) completed the baseline interview after completion of the intervention at CBOs A and B (n = 6). These exclusions reduced the data set to 867 observations. All CBO C participants (n = 237) were also excluded because of evidence that baseline data collection occurred after the intervention session at this agency for an unknown number of participants. A final sample size of 630 observations was included in the GEE analysis.

Table 5 presents the GEE results for each outcome measure in four sets of analyses. In Analysis 1, models were developed to calculate the change in outcomes reported at each of the two follow-up time points, as compared to the baseline measure, not controlling for any variable. In Analysis 2, CBO was added to the model to calculate the change in outcomes over time after adjusting for CBO effects. CBO A was set as the reference group. In Analyses 3 and 4, race/ethnicity and HIV risk group (reported at baseline) were added to the models to calculate the change in outcomes over time after adjusting for race/ethnicity and HIV risk group effects. Non-Hispanic white and heterosexual contact groups were set as the reference groups. Analysis 3 and Analysis 4 models also included interactions between time and group membership to allow an examination of effects at each time point within groups. A backward selection method was used to remove interaction terms that were not significant (cutoff of P = 0.15) and increase the statistical power available to detect possible associations among variables.
Table 5

Generalized estimating equations (GEE) regression results (evaluation project participants at CBOs A, B, and D)


Number of sex partners

RR (95% CI)

Unprotected sex events

OR (95% CI)

Unprotected sex events with someone who exchanges sex for drugs or money

OR (95% CI)

Unprotected sex events while high or intoxicated

OR (95% CI)

Unprotected sex events and multiple sex partners

OR (95% CI)

STD diagnosis

OR (95% CI)

Analysis 1


  Follow-up 1

0.73 (0.65, 0.81)***

0.29 (0.23, 0.37)***

0.49 (0.40, 0.60)***

0.39 (0.32, 0.47)***

0.44 (0.37, 0.52)***


  Follow-up 2

0.62 (0.54, 0.72)***

0.29 (0.22, 0.38)***

0.31 (0.24, 0.40)***

0.32 (0.26, 0.39)***

0.30 (0.24, 0.37)***

0.20 (0.10, 0.38)***

Analysis 2


  Follow-up 1

0.73 (0.65, 0.81)***

0.28 (0.26, 0.40)***

0.40 (0.31, 0.51)***

0.38 (0.31, 0.46)***

0.41 (0.34, 0.50)***


  Follow-up 2

0.62 (0.54, 0.72)***

0.28 (0.26, 0.41)***

0.24 (0.17, 0.32)***

0.31 (0.25, 0.38)***

0.28 (0.22, 0.34)***

0.20 (0.10, 0.38)***



1.55 (1.20, 1.99)**

1.14 (0.83, 1.55)

3.10 (2.17, 4.41)***

2.36 (1.74, 3.20)***

3.13 (2.28, 4.29)***

0.57 (0.24, 1.34)


1.13 (0.79, 1.63)

2.74 (1.94, 3.87)**

0.13 (0.07, 0.24)***

1.79 (1.32, 2.44)**

1.15 (0.82, 1.61)

2.46 (1.28, 4.72)*

Analysis 3


  Follow-up 1

0.72 (0.64, 0.82)***

0.57 (0.42, 0.78)***

0.37 (0.28, 0.49)***

0.30 (0.23, 0.39)***

0.58 (0.46, 0.74)***


  Follow-up 2

1.00 (0.67, 1.51)

0.58 (0.41, 0.82)*

0.27 (0.20, 0.37)***

0.40 (0.30, 0.53)***

0.77 (0.35, 1.68)

0.20 (0.10, 0.38)***



1.23 (0.68, 2.24)

1.74 (0.99, 3.06)

0.15 (0.07, 0.31)***

1.20 (0.73, 1.97)

1.50 (0.79, 2.86)

2.36 (0.71, 7.86)

  African American

1.69 (0.98, 2.91)

2.58 (1.30, 5.13)*

2.17 (1.23, 3.85)*

1.91 (1.14, 3.22)^

4.58 (2.34, 8.95)***

0.89 (0.26, 3.08)

 Time × race/ethnicity

  FU1, Hispanic



1.84 (0.96, 3.52)

1.92 (1.34, 2.76)**



  FU2, Hispanic

0.69 (0.44, 1.07)




0.47 (0.21, 1.09)


  FU1, African American


0.28 (0.17, 0.46)***



0.53 (0.37, 0.77)***


  FU2, African American

0.54 (0.35, 0.84)**

0.28 (0.16, 0.48)***


0.66 (0.44, 0.99)^

0.27 (0.12, 0.63)*


Analysis 4


  Follow-up 1

0.73 (0.65, 0.82)***

0.22 (0.16, 0.29)***

0.48 (0.39, 0.59)***

0.36 (0.30, 0.44)***

0.38 (0.31, 0.46)***


  Follow-up 2

0.60 (0.51, 0.69)***

0.21 (0.15, 0.28)***

0.31 (0.23, 0.40)***

0.29 (0.24, 0.36)***

0.25 (0.20, 0.32)***

0.20 (0.10, 0.38)***

 HIV risk group


1.49 (0.87, 2.56)

1.10 (0.67, 1.79)

1.99 (1.18, 3.37)*

1.49 (0.93, 2.37)

1.14 (0.62, 2.09)

1.12 (0.41, 3.03)


1.06 (0.65, 1.73)

0.89 (0.42, 1.89)

0.65 (0.37, 1.13)

1.50 (1.00, 2.23)^

0.39 (0.23, 0.65)**

0.53 (0.19, 1.50)

 Time × HIV risk group

  FU1, MSM





1.57 (0.87, 2.84)


  FU2, MSM





1.88 (0.98, 3.59)


  FU1, IDU


2.69 (1.36, 5.30)*



1.73 (1.14, 2.63)*


  FU2, IDU

1.36 (1.08, 1.72)*

3.12 (1.33, 7.27)*



1.80 (0.96, 3.35)


Race/ethnicity and HIV risk group represent data reported at baseline. Referent groups are baseline, CBO A, non-Hispanic white, and heterosexual contact

RR rate ratio, OR odds ratio, CI confidence interval, MSM men who have sex with men, IDU injection drug user

P < 0.05; * P < 0.01; ** P < 0.001; *** P < 0.0001

Covariate removed from the model based upon backward selection methods (cutoff of P = 0.15)

All models in Analysis 1 showed statistically significant decreases in behavioral risk over time (P < 0.0001) (Table 5). The number of partners was statistically significantly lower at each follow-up compared to baseline. The proportions of participants who reported unprotected sex events (any, with someone who exchanges sex for drugs or money, while high or intoxicated, and in combination with multiple partners) were lower at each follow-up relative to baseline. The proportion of participants who reported an STD diagnosis at follow-up 2 was also statistically significantly lower compared to baseline. Figure 1 depicts the prevalence of the sexual risk outcomes and the mean number of partners at each time point (unadjusted, with reduced data set, n = 630).
Fig. 1

Unadjusted sexual risk outcomes (past 30 days) at each time point

In Analysis 2, when agency was included as a covariate, all outcome measures showed statistically significant decreases at both follow-up times, indicating the positive behavior change persisted after controlling for the effects of CBO (P < 0.0001) (Table 5). CBO B participants reported significantly more partners overall than the number of partners reported by CBO A participants (P < 0.001). A greater proportion of participants at CBO B reported unprotected sex (with someone who exchanges sex for drugs or money, while high or intoxicated, and with multiple partners) compared to CBO A (P < 0.0001). A greater proportion of CBO D participants reported unprotected sex (any, while high or intoxicated) than the proportion reported by CBO A participants (P < 0.001). A lower proportion of participants at CBO D reported unprotected sex with someone who exchanges sex for drugs/money, relative to CBO A (P < 0.0001). Finally, a greater proportion of CBO D participants reported an STD diagnosis at follow-up 2, relative to CBO A (P < 0.01).

In Analysis 3, when race/ethnicity was included in each model as a covariate along with interactions between race/ethnicity and follow-up time, all outcome measures showed significant decreases at follow-up 1, and all outcome measures except number of sex partners and unprotected sex in addition to multiple sex partners showed statistically significant decreases at follow-up 2 in the reference group (Table 5). Figure 2 depicts the unadjusted outcomes associated with significant race/ethnicity effects, by time point (using the reduced data set, n = 630). With regard to race/ethnicity effects, the only statistically significant effect among Hispanics was that the prevalence of unprotected sex events with someone who exchanges sex for drugs or money was significantly lower among Hispanics than the prevalence observed among whites (OR = 0.15; 95% CI = 0.07, 0.31). Among African Americans, statistically significant effects were observed for all outcome measures except number of sex partners and STD diagnosis: in each case, the risk behavior was more likely to occur among African Americans than among whites. The interaction terms indicate how changes in behavior after the intervention may have been different for the race/ethnicity groups. At follow-up 1, the effect of the intervention on the proportion of participants who reported unprotected sex events while high or intoxicated was significantly reduced for Hispanics (OR = 1.92; 95% CI = 1.34, 2.76), meaning that the decrease in this risk behavior for this group was significantly less than the decrease observed among whites. No other statistically significant interactions were observed for Hispanics. Among African Americans, statistically significantly greater decreases were observed for any unprotected sex events and for unprotected sex events and multiple sex partners, at both follow-up time points, relative to whites. Greater reductions in number of sex partners (OR = 0.54; 95% CI = 0.35, 0.84) and the prevalence of unprotected sex events while high or intoxicated (OR = 0.66; 95% CI = 0.44, 0.99) were also observed among African Americans at follow-up 2, as compared to whites.
Fig. 2

Unadjusted sexual risk outcomes (past 30 days) at each time point, by race/ethnicity

In Analysis 4, when HIV risk group was included in the model as a covariate, all outcome measures showed statistically significant decreases at both follow-up times (Table 5). Additionally, prevalence among MSM for unprotected sex with someone who exchanges sex for drugs or money was twice as high as the prevalence observed among heterosexuals (OR = 1.99; 95% CI = 1.18, 3.37). Compared to heterosexuals, a greater proportion of IDU participants reported unprotected sex while high or intoxicated (OR = 1.50; 95% CI = 1.00, 2.23) but a smaller proportion reported they had multiple sex partners and unprotected sex events (OR = 0.39; 95% CI = 0.23, 0.65). The effect of the intervention was not significantly modified by MSM for any outcome; however, weaker reductions in risk behaviors were observed for IDU for number of partners (follow-up 2) (OR = 1.36; 95% CI = 1.08, 1.72), unprotected sex events (both follow-up time points) (follow-up 1: OR = 2.69; 95% CI = 1.36, 5.30; follow-up 2: OR = 3.12; 95% CI = 1.33, 7.27), and multiple sex partners plus unprotected sex events (follow-up 1) (OR = 1.73; 95% CI = 1.14, 2.63).

To assess intervention effects at each individual CBO, regression analyses were re-run separately on each CBO sample (CBOs A, B, and D). The findings were very similar to the findings when these CBOs’ data were combined. Statistically significant declines in each outcome at each CBO were observed at follow-up as compared to baseline, with the following exceptions. Reductions associated with the number of partners and any unprotected sex events at CBO A for the second follow-up were not statistically significant. Additionally, statistical significance of reductions in any unprotected sex events with someone who exchanges sex for drugs or money at CBO D, and the reduction in STD diagnoses at CBO B, could not be estimated due to the observation of zero events at follow-up 2 (thus, the generalized estimating equation failed to converge).


The findings of this study suggest that participation in the VOICES/VOCES intervention in a variety of community settings may lead to sexual risk reduction among various high-risk populations. For most outcomes examined in this study, risk reduction was more pronounced for African American and Hispanic participants than for white participants, particularly at the second follow-up time point (approximately 120 days post intervention). These results suggest, over a decade after the first diffusion of VOICES/VOCES across the U.S. by CDC, this intervention remains an effective tool for reducing HIV risk behaviors among high-risk African American and Hispanic individuals. Furthermore, the delivery of VOICES/VOCES in community settings other than STD clinics (for which the intervention was originally developed) appears to be feasible for CDC-funded CBOs and has the potential to produce positive changes in sexual risk outcomes in different subpopulations.

In this project, clients who participated in a VOICES/VOCES intervention delivered by CDC-funded agencies reported changes in risk behaviors that persisted up to four months after the intervention (Fig. 1). These effects were observed in a variety of contexts. For example, the intervention was conducted in many different types of settings ranging from the agency to private residences to mobile units parked at bus terminals. In addition, across CBOs, a diverse group of subpopulations at high risk for HIV infection/transmission participated in the intervention such as MSM, IDU, heterosexuals, HIV-negative individuals, HIV-positive individuals, homeless individuals, sex workers. The GEE results also indicated substantial variability across CBOs with regard to the overall risk level of individuals who participated in the evaluation. For example, the prevalence of unprotected sex with someone who exchanged sex for drugs or money was three times as high at CBO B as at CBO A, while the prevalence of self-reported STD incidence was 2.5 times as high at CBO D compared to CBO A. Furthermore, the same videos/video clips were not shown at all CBOs. Previous research has reported CBO staff frequently make such modifications to VOICES/VOCES to accommodate specific target populations, such as MSM [24, 31]. CDC supports making changes to key characteristics of an EBI as long as the relevant core element (show a culturally specific video that portrays condom negotiation) is not altered.

Despite the wide range of circumstances under which the intervention was delivered at these sites, and the heterogeneity of the participants at each agency, significant risk reductions were consistently observed at both follow-up time points for all outcome measures. The robustness of these findings may be attributable to the fact that each agency received the same preparation and materials through CDC-sponsored DEBI training when they began implementing the VOICES/VOCES program. If all CBOs were implementing the core elements, even with various adaptations tailored toward community needs, this could explain the consistent findings across CBOs. CDC encourages CBOs to adapt key characteristics of EBIs to improve their fit with relevant target populations in their community as long as fidelity to the original core elements is preserved.

In our models, race/ethnicity was observed to be an important client-level factor that predicted changes in sexual risk behaviors at both follow-up time points (Fig. 2). Greater reductions in almost all risk outcomes were observed among African Americans after intervention participation, relative to whites. The decline in risk behaviors among African Americans, relative to whites, tended to be even stronger at the second follow-up time point than the first, suggesting a longer lasting impact of the intervention among African Americans. Among Hispanics, intervention effects tended to be better for some outcomes (number of partners, multiple sex partners and unprotected sex) than others (unprotected sex with someone who exchanges sex for drugs/money, unprotected sex while high/intoxicated), relative to whites (Fig. 2). Weaker reductions among Hispanic participants should not be interpreted as VOICES/VOCES did not “work” for these individuals but rather that the declines were simply not as great for Hispanics as for African Americans and whites for certain outcomes.

In previous efficacy research, Hispanics in the intervention conditions were the most likely to redeem condom coupons and African Americans were also likely to redeem coupons but the effect was not as strong [13]. Different videos were shown to African Americans in the current project (Love Exchange, Booty Call, Noah’s Ark) than the one used in the previous efficacy study (Let’s Do Something Different), although the same video (Porque Si) was shown to Hispanics in both studies. The African American videos used in this project may have done a better job at communicating prevention messages than the other videos. It also should be noted that the reduction in number of sex partners among both Hispanics and African Americans at follow-up 2 (in Analysis 3) suggests that the reductions at follow-up 2 in Analyses 1 and 2 were largely due to decreased risk behavior in these two race/ethnicity groups. A similar pattern across race/ethnicity groups was evident for unprotected sex in combination with multiple sex partners. Thus, while positive effects were observed for some outcomes in each race/ethnicity group, much of the significant decline in risky sexual behavior appears to be driven by risk reduction among African American and Hispanic participants.

HIV risk group was also identified as a client-level factor that was significantly associated with decreased risky behavior over time. Overall, the Analysis 4 results suggested the intervention had a similar effect on outcomes for heterosexuals and MSM but the reductions were less pronounced for IDUs. There was no difference when comparing change over time for MSM and heterosexual participants, but the reduction in risk behaviors observed among IDU participants was less than the reductions observed among heterosexual participants. Specifically, smaller declines were observed among IDUs for number of sex partners (follow-up 1), unprotected sex events (both follow-up time points), and unprotected sex events with multiple sex partners (follow-up 2). These findings are not surprising, given that the VOICES/VOCES intervention involves communication of messages about condom use and safer sex practices and was originally intended for high-risk heterosexuals. Reducing sexual risk behaviors among high-risk injection drug users is a continuing challenge for HIV prevention and may require more than just a single-session intervention designed for high-risk heterosexuals. CDC recommends CBOs enroll their IDU clients in interventions that are specifically designed for them (e.g., Safety Counts). Agencies that offer services to IDU populations may also incorporate prevention messages into their VOICES/VOCES sessions that address both risky drug and sexual practices.

The current findings extend previous VOICES/VOCES research by determining if participation in VOICES/VOCES would be associated with positive changes in client-level, sexual risk behaviors. Previous studies reported that participation in VOICES/VOCES had significant and positive effects on efficacy, behavioral intentions, and attitudes pertaining to condom use [13, 14, 20]. The outcomes utilized in this study went one step further by assessing whether participants reported the actual use of protection (condoms) during sexual activity after participation in VOICES/VOCES. The results indicated that participants in VOICES/VOCES may reduce risky sexual activity especially among individuals who report engaging in more than one risk behavior (i.e., having unprotected sex and having multiple sex partners). In addition, consistent with previous work [15, 23], the prevalence of self-reported STD diagnoses in the current sample decreased significantly from baseline (8%) to follow-up (2%), perhaps because of the overall decrease observed in unprotected sexual behaviors. Although STD incidence in the previous studies was confirmed through STD surveillance databases, only self-report data were available for this study; however, the majority of participants who reported having an STD diagnosis in the previous 90 days indicated the diagnosis was confirmed with a laboratory test (77% at baseline and 73% at follow-up 2).

These findings are subject to a number of limitations. First, randomized sampling procedures were not used and, thus, the results reported here may not be generalizable to other individuals who participate in VOICES/VOCES (at the same or other agencies). The sample used for this study was a function of agency-specific recruitment strategies, sampling quotas, and other unique criteria. Second, the specific outcomes analyzed were dictated by the NHM&E variable set and PEMS which were designed for use by health departments and CBOs when reporting key HIV program data to CDC. Data collection was limited by previously established variable definitions and by the functionality of the PEMS software. Third, while the variables were standardized across agencies, the questions used to solicit the answers were not, although guidance and templates were provided. Although CDC provided data collection templates to each agency, the CBOs could adapt these instruments as needed. Variability in methods was observed to some extent across agencies at different points in the project as CBOs could adapt procedures and instruments as needed. Fourth, regression toward the mean, demand effect, history, and other sources of error cannot be eliminated as possible explanations of behavior change. There was no comparison group and, thus, it is possible that participants exaggerated their reports of behavior change or gave socially desirable responses at follow-up. Fifth, information about whether participants were incarcerated or residing in substance treatment facilities at the time of baseline or follow-up interviews was not collected. Current incarceration or participation in live-in substance abuse programs could have had an impact on participants’ sexual activity during the 30-day recall period. Finally, all measurements of HIV risk behaviors reported in this paper are based on self-report and recall and, thus, all caveats associated with this kind of data apply.

These findings indicate that CDC-funded CBOs have the potential to monitor the outcomes of their programs, make changes as necessary to improve recruitment of appropriate clients into EBIs, make appropriate adaptations to meet the needs of the agency and the target population, and improve EBI delivery to strengthen expected outcomes. The agencies that participated in this evaluation project demonstrated flexibility and creative use of their resources by effectively delivering the intervention in an array of settings that considered the needs of their target populations. These results suggest other CDC-funded CBOs also have the potential to conduct outcome monitoring (perhaps with other EBIs) with the aid of technical assistance and other resources provided by in-house or CDC program evaluators. Furthermore, future EBI evaluations should include the collection of intervention process data so that CDC can identify the specific intervention/agency/staff elements that are associated with intervention effectiveness. It was not possible to collect detailed process monitoring data in the current project but other EBI evaluations currently in progress include this activity. Additionally, adaptations to VOICES/VOCES, such as using alternative videos, highlight the fact that service providers in the field can successfully alter specific aspects of the intervention (while maintaining fidelity to its core elements) to provide their participants with appropriate and relevant information and still produce positive intervention outcomes.

In sum, this outcome monitoring project represents one of CDC’s first attempts at monitoring EBI outcomes in the field and identifying changes in clients’ sexual risk behaviors that occur after EBI participation in community settings [32]. These findings make an important contribution to the expanding knowledge base of HIV prevention intervention implementation science. Further, the results have important implications for specific high-risk populations, such as MSM, IDUs, and African American heterosexuals, for whom there are presently a limited number of evidence-based prevention interventions available in the field.


We thank the efforts of the staff at each funded agency for their support and assistance in conducting this outcome monitoring project. We thank Qian An, Linda Andes, Felicia Hardnett, Venkat Mannam, Susan Moss, and Stephen Tregear for their assistance and consultation related to the data analysis. We thank Bryce Smith, Marla Vaughan, and Joanna Wooster for their contributions to the project and protocol. Finally, we thank Marla Vaughan for reviewing an earlier draft of this manuscript.

Copyright information

© Springer Science+Business Media, LLC (outside the USA)  2011