Background

Despite several decades of work [1, 2], there is still insufficient evidence of how to effectively reach men who have sex with men (MSM) in low and middle-income countries with effective prevention programming, particularly in settings where stigma and homophobia are prevalent and MSM are difficult to reach. An estimated 112,000 people in Central America were newly infected with HIV in 2010. MSM in Central America are one of the most affected key populations in the region, with HIV prevalence ranging from 7.5% to 11.1% [3, 4]. Investments have been made in Central America to target key populations including MSM, but evidence is extremely limited on the effectiveness of these investments [5–7]. We aim here to assess the effectiveness of one combination prevention program for reducing HIV risks among MSM across 5 countries in Central America.

That MSM in Central America experience elevated HIV risks is well documented. High levels of unprotected anal intercourse have been reported in Nicaragua and El Salvador. Condom use is frequent, but the picture of consistent condom use is not clear, and in 2013, 60.2% of MSM reported sometimes or never using condoms with their partners [8, 9]. Use of HIV testing and counseling (HTC) is variable across the region, despite high levels of awareness of services [8–10]. HIV testing in the past year ranged from 54% in San Salvador, El Salvador in 2008 to 72% in Chinandega, Nicaragua in 2009 [11, 12].

HIV risk behaviors among MSM occur in a context of stigma and violence. Central America experiences high levels of generalized violence along with homophobia targeted against MSM and transgender women [13–15]. Stigmatizing social environments increase the vulnerability of MSM to a range of physical and mental health risks [1, 8, 16]. Violence against men, especially homosexual men, including forced sex, is common, along with MSM being refused services from health care providers [8, 13, 16, 17].

Given structural factors, such as homophobia and violence, it is difficult to reach targeted populations at impactful levels of coverage and challenging to maintain these contacts. In Central America and elsewhere, the HIV community is increasingly mobilized around combination prevention strategies, recognizing that it is not possible to get traction on the epidemic through isolated efforts that do not address contexts of risk [18, 19]. A combination prevention approach includes biomedical interventions that block the biological mechanisms through which HIV transmission occurs or reduce the chance of acquisition of infection, such as provision of condoms, STI treatment, pre-exposure prophylaxis, and post-exposure prophylaxis, as well as evidence-based behavioral interventions that use communications channels to promote behaviors that reduce HIV risk – which on their own can be effective but are difficult to sustain - as well as generate demand for biomedical interventions [20, 21]. Critically, combination prevention also includes structural interventions to address the social drivers of the epidemic to create environments more supportive for vulnerable populations [18, 20–22]. Structural interventions cover a wide scope of approaches, for example cash transfers, community dialogue, sensitivity training for law enforcement and health providers, and political advocacy, to address the multiple levels of social interactions that influence HIV risk and transmission [22]. These interventions operate at the population level and aim to address an individual’s risk through indirect, upstream mechanisms that shape norms, behaviors and health outcomes in the population as a whole [23]. Best practice recommends that the specific combination of interventions to be implemented from within the three components should be designed based on local epidemiology and social conditions [24].

While combination prevention is increasingly the consensus framework for HIV prevention, there is as yet little evidence on the effectiveness of different combination prevention strategies [25]. The need to evaluate the combination prevention approach is clear, given the complexity of evaluating multi-level programming, but few dedicated studies have been conducted [26].

Program description

With support from USAID (United States Agency for International Development), the Pan-American Social Marketing Organization (PASMO) and Population Services International (PSI)/Mexico began implementing a Combination Prevention Program for HIV with community-based partners in 2011 across Belize, Guatemala, El Salvador, Nicaragua, Panama, Costa Rica, and Mexico. The program is designed to increase access to HIV prevention interventions in order to reduce risk behaviors among key populations and reduce social hostility that contributes to stigma and discrimination.

PASMO has defined an essential package of interventions for MSM following a combination prevention approach [27]. To receive a complete package, an individual client should have a) participated in at least three behavior change communication interventions conducted by outreach teams or through online outreach, b) received a referral for biomedical services such as HTC or screening for sexually transmitted infections (STI), and c) received a referral to a set of complementary services.

The behavioral component includes integrated behavior change communications activities conducted by outreach workers. Based on the transtheoretical model and Population Services International’s PERForM framework, these activities were designed to generate demand for the program’s products and services, while also encouraging individuals reached to assess their health risks and build skills in HIV prevention [28–30]. Activities are conducted in person one-on-one, one-to-group or online through cyber-educators. Using Prochaska’s Stages of Change framework, outreach workers identify the stage a targeted individual is in to be able to practice a specific behavior being promoted, such as condom use or use of HTC, and tailor delivery of activities according to that stage through discussion and reflection [29]. Outreach workers also ensure that condoms and lubricants are available for sample or purchase near to where activities are being implemented [31].

The biomedical component is core to the Combination Prevention program’s design. Services provided include HTC, STI screening and treatment, and for people living with HIV, referrals for care and treatment, including antiretroviral therapy. A variety of service providers participate in the program, including private laboratories, private and NGO clinics, and some public sector facilities. HTC is provided through fixed and mobile services, according to where the program is operating [31].

The addition of complementary services was designed to support program clients to locate resources that help them negotiate potential stigma and discrimination [31]. These specifically include referrals to designated service providers for support groups, drug and alcohol treatment, legal support, and violence prevention services. In addition, the program operates beyond the interpersonal level to address stigma and discrimination and change social norms through sensitization and training of health care providers to improve perceived quality of care, media training, and mobilization of civil society leaders to foster public dialogue on homophobia and discrimination against people living with HIV.

Integrated HIV prevention teams conduct behavioral interventions and referrals to biomedical and complementary services in identified high-risk zones in program cities. For MSM, these zones are areas with large concentrations of nightclubs, bars, parks and areas where community organizations have identified that MSM are likely to meet each other. Teams include outreach workers, PASMO sales staff responsible for ensuring availability and access to condoms, community leaders, and service providers. When clients are contacted by an outreach worker, they receive an in-person behavioral intervention plus a packet of vouchers for other behavioral interventions, condoms, biomedical services, and complementary services. The program tracks client contacts and use of referral vouchers via unique identifier codes assigned to each client contacted [32]. While conducting direct service delivery and coordinating with other program partners, PASMO and partners have also implemented a set of advocacy strategies as part of the structural component of the program. Advocacy is designed to encourage public discussion of HIV transmission and encourage reduction in social hostility towards key populations including MSM [33].

We aimed in this paper to provide early results on PASMO’s combination prevention program, as targeted to MSM in Central America. Two years into implementing the program, our objectives were to 1) assess population-level coverage of specific and combined intervention components and 2) determine whether program exposure to any or a combination of components was associated with HIV risk reduction behaviors, specifically condom and water-based lubricant use, HIV testing and counseling, and appropriate treatment of STIs.

Methods

Data sources and measures

We used behavioral survey data and data extracted from PASMO’s program monitoring system for this analysis.

Regional survey of MSM

From September-November 2012, PASMO recruited MSM from 10 cities in six Central American countries into a survey using respondent-driven sampling (n = 4231). The survey was conducted in areas where the Combination Prevention program operates: Belize City in Belize, San Jose in Costa Rica, San Salvador and Santa Ana in El Salvador, Guatemala City and Quetzaltenango in Guatemala, Managua and Chinandega in Nicaragua, and Panama City and Colón in Panama. These cities were selected because they had on-going program operations and national level evidence indicated high projected HIV prevalence in each city [3, 34]. Men were eligible to participate in the study if they were aged 18–40, had had anal sex with another man in the past 3 months, were resident in the study city, and knew the name or alias of the person who had recruited that respondent to participate. Respondent-driven sampling is a chain referral sampling technique developed to improve the population representativeness of surveys of hard-to-reach populations [35, 36]. We recruited 1–3 seeds in each city. We selected seeds based on socio-economic status, self-identification of sexual orientation, and working in commercial sex. Each seed was asked to recruit three peers within his social network to participate in the study, and subsequent study participants were asked to recruit an additional three peers.

Data was collected through in-person interviews administered by a male enumerator in a location that ensured the respondent’s privacy. The survey instrument included questions on socio-demographic characteristics, sexual behaviors, STI symptoms, use of STI treatment and HTC services, mass media and program exposure, knowledge and attitudes regarding HIV prevention, and experience with violence. The PSI Research Ethics Board provided ethical approval of this survey along with Ministries of Health in Belize, El Salvador, Guatemala and Nicaragua. The Panama National HIV/AIDS Program and the University of Medical Sciences in Costa Rica approved the study as well. Individuals were asked to provide informed consent before participating in the interview. No identifying information was collected from respondents.

For this analysis, we did not use data from Belize because program operations were not completely comparable. We also excluded transgender women (n = 400 self-identified cases) from this analysis because transgender women are increasingly considered to be a separate population programmatically and epidemiologically [37]. Our final analytical sample was 3531 men.

Routine data

We also used data extracted from PASMO’s routine program monitoring system. PASMO’s system for applied monitoring (SAM) tracks BCC activities and service utilization across all program implementers in six countries. A key feature of the SAM is the use of unique identifier codes (UICs) to track individual clients across program contacts. Unique identifier codes are based on personal information that does not vary over time and that clients can easily recall. Monitoring data are collected on BCC activities by type of activity and target population reached, messages and behaviors, referrals initiated and completed, plus client use of HTC, and STI treatment, and any referrals to other services, such as CD4 count and viral load testing for people living with HIV. Data on program contacts and service utilization are aggregated by country and reported to partners on a monthly basis. For this analysis, we extracted data from October 1, 2011-September 30, 2012. No identifying information on program clients was collected, and analysis of this data was determined to not constitute research with human subjects. From extraction of routine data, we identified 15031 program contacts across the five countries.

Measures

Outcome measures

Five outcome measures were used for this analysis, all taken from behavioral survey data: 1) condom use at last sex, 2) condom use with water-based lubricant at last sex, 3) consistent condom use in the past 30 days, 4) receipt of an HIV test in the past 12 months, and 5) seeking medical care for a suspected STI symptom in the past 12 months. Consistent condom use was defined as using a condom from start to finish for all sex acts with any partner type for the past 30 days. We measured consistent condom use by partner type for commercial, casual and regular partners. Commercial partners were partners from whom the respondent received money or to whom the respondent gave money for sex. Casual partners were defined as partners with whom the respondent had sex with no emotional attachment and no financial transaction. Regular partners were defined as partners with whom the respondent had engaged in sex for more than 3 months and felt connected to, either by love or friendship.

Program exposure

We created several measures of program exposure from behavioral survey data. These measures were designed to capture specific components of the combination prevention approach in order to capture program operations and assess potentially additive effects of the different components. Exposure to behavioral interventions was defined as having been contacted by an outreach worker representing the project. Exposure to biomedical interventions was defined as having received a voucher for HIV testing and counseling, to capture the referral process. Exposure to complementary interventions was defined as receiving a voucher for other services. We defined any exposure to the program as receipt of any one of these components. We also created a measure of having received all three components, defined as the combination package.

After extracting individual records from the program monitoring system, we created measures of receipt of behavioral interventions (y/n, and number of interventions), receipt of biomedical services (y/n and number of services), and receipt of complementary interventions (y/n and number of services) for this analysis.

Other measures

We considered a range of other factors from the survey data for this analysis. We looked at socio-demographic factors of age, education (primary education or less, some secondary education, any higher education), and an index of socio-economic status. The index of socio-economic status was constructed following methods of the Mexican Market Research Association and grouped individuals into categories of high, medium and low socio-economic status [38]. We also considered how men described their sexual identity (gay, bisexual, heterosexual).

We considered a series of factors assessing men’s social interactions with gay, transgender and other MSM communities. These included: friends’ knowing the respondent’s sexual orientation, having participated in a gay/transgender event in the past 12 months, and going to a gay bar/disco in the past 30 days. We also considered exposure to other HIV/AIDS programming targeting MSM, measured as whether the respondent received a condom for free in the past 12 months. All factors were treated as dichotomous variables.

Analysis of survey data

Weighting

For descriptive sample characteristics, we present un-weighted data. We conducted weighted analyses to assess program effects using multiple logistic regression. Because of our sampling and analysis strategy, we calculated weights using pooled data. Several types of weights were calculated and adjusted for in regression models. First, we calculated RDS weights using RDSAT software, which accounts for network size and recruitment patterns for each city [39]. Second, we estimated city weights to account for differential proportions of the sample and estimated population size of each city. Because reliable estimates of the population size of men who have sex with men in all 9 cities were not available, we weighted by the city population of men 15–39 years [40]. Population size estimates were retrieved from national statistics bureaus in each country [41–45]. Age- and sex-specific population size estimates were not available for Guatemala, and we therefore applied the national proportion of men in Guatemala to the age distribution for each city. A third set of weights was used as well, derived from the matching procedure described below. These weights were multiplied together and applied in regression models. Data analysis was conducted in Stata 13 and SPSS 22.

Statistical matching

We next used a statistical matching technique, coarsened exact matching (CEM), to improve the quality of our estimates of program effects [46–48]. Coarsened exact matching was used to create statistically equivalent groups of survey respondents exposed and not exposed to any program component. This quasi-experimental approach allowed us to designate a counterfactual (no exposure to any Combination Prevention program components) when an experimental design was not feasible, since program implementation was ongoing at scale across several countries and was not designed or implemented with intervention and control groups [47, 49]. Coarsened exact matching is a monotonic imbalance matching method designed to reduce imbalance between treatment and control groups in observational data [49, 50]. CEM assigns each case into one of a specified set of strata in which members are exactly matched on a set of coarsened, or categorized, variables [46]. Matched cases are then assigned a weight specific to that stratum and representative of the proportion of all cases present in the stratum [51, 52].

We matched survey respondents on factors likely to influence exposure to PASMO’s combination prevention program, given the strong likelihood of selection bias in a setting where MSM are highly stigmatized [53]. We matched on education (categorized as primary education or less, any secondary education, any higher education), sexual orientation (heterosexual, homosexual, bisexual), whether the respondent’s friends knew he had sex with other men, whether the respondent had participated in a gay/transgender event in the past 12 months, whether the respondent had been to a gay bar or disco in the past 30 days, and whether the respondent had received free condoms in the past 12 months. We assessed relationships between potential matching variables and any program exposure using Spearman correlation coefficients. We matched variables that had a statistically significant correlation with program exposure at p < 0.05. An exact match made on the coarsened variables yielded an L1 measure of 4.814e-16, indicating minimal imbalance between respondents exposed and not exposed [46].

Modeling strategy

We calculated un-weighted descriptive statistics for key measures using country data and pooled data. Descriptive statistics in the pooled dataset were calculated for the full and the matched samples. We next compared distributions of program exposure in the survey data to program exposure from monitoring data by country. We then used weighted multivariable logistic regression to estimate odds ratios of average treatment effects for each measure of program participation, applying all three types of weights (RDSAT weights, city weights, CEM weights). We first estimated unadjusted odds ratios for each outcome of interest. We next adjusted for age and socio-economic status – factors not included in the matching procedure. We also investigated the effects of receiving two combination prevention channels (behavioral + biomedical and behavioral + complementary), with and without controls. The same modeling strategy was used in the full sample and with the matched sub-sample to estimate how estimates of treatment effects changed following matching.

Analysis of combined survey and MIS data

Finally, we created a new sub-sample of cases where we could link unique identifiers between routine data and survey data. Because of the limited sample size for these linked cases, we did not apply any sampling weights, and we did not match because all cases were exposed. We conducted chi-square tests of association between type of program exposure and the outcomes of interest by country.

Results

More than 80% of respondents across all countries were under 30 (Table 1). Half of the surveyed men were educated at the secondary level and 40% had some university, except for Costa Rica where a quarter of the sample had only primary education. The majority of men were of middle or lower socio-economic status. Half of respondents identified as homosexual or gay, 40.5% as bisexual, and 8.4% as heterosexual. Trends in sexual identity varied by country; Panama had a relatively greater proportion of men indicating they were gay/homosexual while Costa Rica and El Salvador had larger proportions of men reporting they were heterosexual. The majority (70%) of respondents indicated that they had disclosed their sexual identity to their friends, 33% had participated in a gay/transgender event in past 12 months, and 63% had visited a gay bar or disco in the past 30 days.

Table 1 Characteristics of men who have sex with men in Central America, 2012
Full size table

Of the outcomes of interest, 85% of men reported having used a condom at last sex, and 48% used a condom with water-based lubricant at last sex. Regarding consistent condom use, 82% reported consistent condom use in the past 30 days with all partners. Consistent condom use was substantially higher when respondents reported having casual partners (90%) and commercial partners (92%). Just over half of respondents reported having had an HIV test in the last 12 months (52%), and of those with STI symptoms, 76% sought medical treatment.

Almost a third (32.2%) of respondents reported exposure to any component of the Combination Prevention program (Table 2). Across the five countries, exposure to the behavioral component reached 17.4% and was highest in Guatemala (27.9% overall and 74.2% of those with any exposure) and lowest in Costa Rica and Panama. Coverage of the biomedical component reached 16.4% of men regionally, with El Salvador and Panama achieving the greatest levels of biomedical exposure. Exposure to complementary interventions was reported by 11.2% of respondents regionally; the reach of the complementary component was minimal in Costa Rica but reached 65% of those with any exposure in El Salvador. Only 2.8% of respondents received all three components of the program, with El Salvador (6.1%) and Guatemala (5.5%) achieving the greatest levels of population-based coverage of combination prevention exposure.

Table 2 Exposure to combination prevention intervention channels among men who have sex with men
Full size table

We also report trends in program exposure from program monitoring data. Based on how the data were collected, exposure to behavioral interventions was considerably greater than to any other component across all countries, ranging from 84 to 100% of reported contacts. Only El Salvador and Guatemala reported anything other than minimal levels of biomedical contacts (16.5% and 22.7%, respectively), while El Salvador and Nicaragua had the greatest reported contacts with complementary services (both at 3.4%).

Table 3 reports findings from models estimating program effects for the outcomes of interest. We do not show modeling results for consistent condom use with casual partners or for STI treatment seeking, where there were no findings. Men exposed to any programmatic component were more likely to use condoms consistently with regular partners (OR 1.69; 95% CI 1.09, 2.62) and to have been tested for HIV (AOR 2.98; 95% CI 1.82, 4.87). Men exposed to the behavioral component were more likely to use a condom with water-based lubricant the last time they had sex (OR 1.84; 95% CI 1.08, 3.14), to use condoms consistently with regular partners (AOR 1.88; 95% CI 1.09, 3.25), and to have tested for HIV in the past 12 months (OR 1.76; 95% CI 1.001, 3.10). Exposure to the complementary component was also associated with HIV testing (OR 1.97; 95% CI 1.0, 4.05). The odds of condom and water-based lubricant use were greater when men received both the behavioral and biomedical component (OR 3.05; 96% CI 1.08, 8.64). Exposure to both behavioral and complementary components was associated with consistent condom use with commercial partners in the full sample prior to matching (OR 2.47; 95% CI 1.00, 6.09), but after matching, these results were not statistically significant. Overall, we found no evidence that receipt of all three program components was associated with any of the outcomes of interest, although this measure of exposure was rare.

Table 3 Associations with combination prevention program exposure for behavioral outcomes among MSM across five Central American countries
Full size table

We were able to link a total of 520 unique individuals between monitoring and survey data (Table 4). These cases were comparable to the full sample on socio-demographic factors, except that cases in Guatemala were somewhat younger on average than all men (results not shown). Consistent condom use with regular partners was higher for linked cases, except in Nicaragua. Use of condoms with water-based lubricant and HIV testing were both greater among linked cases in Costa Rica than in the full sample. Most MSM had been exposed to 1–2 behavioral activities, although in Costa Rica a quarter were exposed to more than five. Comparatively fewer of the linked cases had received any biomedical services. The country with the highest level of biomedical service use was Guatemala (22% of cases). Use of complementary services was rare. Overall, El Salvador and Nicaragua achieved the greatest levels of combination prevention exposure, but coverage was still low at 7.3% and 6.8%, respectively. No relationship between program exposure and behavioral outcomes was statistically significant when tested with chi-square tests of association.

Table 4 Behavioral outcomes and program exposure for MSM cases with program monitoring and survey data, n = 520
Full size table

Discussion

In light of the need for a larger evidence base on the effectiveness of combination prevention, particularly from Latin America, we have aimed in this study to systematically evaluate a program strategy that seeks to link multiple program components for a priority population in Central America. We aimed to assess population-level coverage of a combination prevention program targeted to men who have sex with men and to determine whether program exposure was associated with HIV risk reduction in this population. We present preliminary evidence here that suggests this strategy is associated with condom use with water-based lubricant at last sex, consistent condom use with regular partners, and HIV testing.

Two years into implementation, PASMO’s Combination Prevention program has achieved fairly modest levels of coverage in the target population of MSM, a population that is notably difficult to reach [54, 55]. Condom use was relatively high in this population, but condom use with water-based lubricant less so. These levels of condom use are consistent with other reports from Central America [3, 56]. Condom use in our study tended to be slightly higher, particularly with casual partners, than recent IBBS surveys in in El Salvador, Guatemala and Nicaragua [11, 12, 57]. Over half of the population had received an HIV test in the past year, except in Costa Rica where receipt of testing was less common. These results were similar to the 2008 IBBS in El Salvador, but the 2009 Nicaragua IBBS reported higher levels of HIV testing in the past year compared to our study [11, 12]. STI treatment practices were highly variable across countries, but overall trends indicate that appropriate treatment-seeking was common.

Program coverage varied across countries, indicating that the program was at different stages of penetration into the target population. Exposure to the behavioral component of the program was most common at a population level, followed closely by exposure to referral for biomedical services Panama and Costa Rica both had relatively lower levels of exposure to the behavioral component, reflecting that these countries were relatively newer sites for program implementation than Guatemala or El Salvador. Variation in biomedical program exposure was likely due in part to differing periods of program operation.

The picture of program exposure based on records of contacts from monitoring data is somewhat different. Several countries showed high service utilization, but trends differed from survey data. This suggests issues with under-reporting in the monitoring system, likely due to problems with voucher redemption, and is compounded by likely under-reporting of program exposure through recall bias in survey data. Despite the potential for discrepancies, exposure to the complementary component was consistently low across data sources and countries. The notable exception was Guatemala, where complementary exposure was 45% of exposure amongst survey respondents with any exposure. This suggests that the program was more successful in implementing the voucher mechanism and service linkages in that country.

Modalities of program implementation also account for some of the differences in exposure trends. Specifically in Costa Rica, the program operates solely through referrals to the public sector Social Security Institute, following government policy on maintaining HTC services as a public sector service. Rapid HIV tests are not yet available in Costa Rican clinics, which may have influenced outcomes there. Based on coverage levels in other countries, including in Panama where the program also only recently began implementation, loosening policy guidelines on service providers and types of HTC services that can be provided in Costa Rica may be merited to increase the use of HTC.

Despite modest levels of coverage and inconsistencies across data sources, there is evidence that participation in the Combination Prevention program was associated with HIV risk reduction. The behavioral component was associated with consistent condom use with regular partners and with using condoms and water-based lubricants at last sex. Both of these behaviors are focuses of messaging during behavioral interactions. Print materials are distributed that demonstrate the importance of using condoms with water-based lubricant, and counselors are trained to talk about lubricant use in pre- and post-counseling. Effect estimates more than doubled for men who had received both the behavioral component and referral to biomedical services, suggesting that a greater dose of exposure was particularly effective at promoting water-based lubricant use with condoms.

Based on previously collected data showing comparatively lower levels of condom use with regular partners, compared to with casual or commercial partners, PASMO specifically focused messaging within BCC activities on condom use with regular partners during the period evaluated by this study [58–61]. The finding that exposure to the behavioral component was associated with condom use with regular partners suggests that interpersonal communications and a structured approach to BCC can be effective for influencing specific behaviors. Condom use across all partner types is now relatively high, suggesting that future messaging should focus on continued maintenance of these behaviors, along with greater adoption of the use of water-based lubricants.

There was also evidence that program exposure was associated with HIV testing, particularly the exposure to the behavioral component. The effectiveness of referral to complementary services was attenuated in the matched sample, and having received both program components was associated with an even higher likelihood of HIV testing, but these results were not statistically significant, likely due to the small numbers of men who reported both components. These results suggest that BCC strategies are effectively able to promote use of HIV testing and counseling services. The program’s emphasis on improving quality of care within testing service providers, adapting service hours to be convenient for the population being served, and helping service providers to treat client respectfully likely contributed to these results, along with making referrals to these service providers. Subsequent evaluations of this program should continue to investigate whether service linkages and making services MSM-friendly are able to achieve increases in demand for and use of HTC services. This study was not designed to assess links between HIV testing and care and treatment services, but other data from the Combination Prevention program suggest that the program has contributed to improved access to care and treatment for people living with HIV [62].

After linking cases from monitoring with survey data, we found no evidence that program exposure was associated with any of the outcomes. Again, this was likely due to the rarity of exposure, a result of challenges in locating men exposed to the program during survey data collection and perhaps inconsistencies in reporting of unique identifiers. Future survey research to evaluate this program may be able to address these challenges of recall during data collection.

The Combination Prevention program results are aligned with recent evidence from the HIV prevention literature and contribute unique results on condom use with water-based lubricants, especially in regards to the doubling of effect size for men who were exposed to both the behavioral and biomedical components in the program. Several evaluations from large-scale HIV prevention efforts focused on MSM show an increase in consistent condom use with regular male partners, some with all male partners [63–65]. Community-based peer interventions show evidence of increased condom use at last sex with a male partner and increased perception of behavioral control in taking up HIV testing and counseling [66]. The Combination Prevention program, as well other programs for MSM in low and middle-income countries found to be effective, require additional investments to increase depth and breadth of program coverage.

Limitations

This analysis has several limitations. In the survey data, recall of outcomes may have been biased. However, levels of condom use and of HIV testing were comparable to other reports from the region [56]. Program exposure recall may have been biased as well. Comparison with program monitoring data suggests that program exposure recall from survey data may have been under-reported, thus attenuating estimates of coverage and effectiveness. The survey did not collect data on HIV status, either through self-report or biomarkers, nor were we able to go beyond experience with HIV testing to look at whether this program links HIV testing and counseling to the HIV treatment cascade, priority areas for understanding the effectiveness of HIV prevention strategies and linking prevention to a continuum of care [67]. We also suspect under-reporting of program contacts in monitoring data, due to the need for program beneficiaries to redeem vouchers before registering in the system. Under-reporting may have attenuated estimates.

Another potential study limitation was in the coarsening of variables before matching. The matching procedure we used faces a trade-off in increasing the number of matched pairs at the expense of less exact matching [49, 51]. However, categorical variables were less likely to be affected by this potential source of bias. This analysis is also at risk for omitted variable bias, as we were not able to account for the effects of unobservable factors.

The measurement approach taken here primarily assesses program exposure and outcomes based on individual-level measurement. It has been argued that combination prevention programs require a multi-level exposure measurement approach or evaluation platform approach to assess the effects of structural factors [25, 26]. We were not able to measure changes at a structural level that may have been related to the program. However, measurement of the complementary component may stand in for some of these gaps, in that these services, including legal support and counseling on drug and alcohol use, influence other factors that affect risk. Further research and later rounds of the evaluation may investigate how to capture the advocacy and social mobilization components better.

Conclusions

We have early evidence that a combination prevention strategy is associated with HIV risk reduction among MSM in Central America. Additional strategies are needed to expand program coverage into a population that is difficult to reach due to stigma and discrimination. Service linkages appear to strengthen program impacts, but measurement may be hampered by under-reporting. Future evaluations should take a multi-level approach to account for complex program operations and the multiple levels (societal, social network, interpersonal) at which a combination prevention strategy operates.