Archives of Sexual Behavior

, Volume 47, Issue 7, pp 2135–2148 | Cite as

Attrition and HIV Risk Behaviors: A Comparison of Young Men Who Have Sex with Men Recruited from Online and Offline Venues for an Online HIV Prevention Program

  • Krystal Madkins
  • George J. Greene
  • Eric Hall
  • Ruben Jimenez
  • Jeffrey T. Parsons
  • Patrick S. Sullivan
  • Brian Mustanski
Original Paper


In addition to offline venue-based and time–space sampling, recruitment of young men who have sex with men (YMSM) into HIV surveillance and prevention studies has expanded over time to include Internet-based approaches. Despite broadening recruitment strategies, enrolling racially and ethnically diverse YMSM who are disproportionately impacted by HIV continues to be challenging. Additionally, there is little literature on the impact of recruitment venue on participant characteristics and likelihood to enroll YMSM into online randomized control trials (RCT). This study used data from the multisite RCT, Keep It Up! 2.0, to examine the impact of recruitment venue on participant demographics, behavioral HIV risks, and enrollment. A total of 2984 participants were screened for eligibility from community-based organizations, Facebook, dating apps, and outreach events. There were significant differences by venue in age (χ2[3] = 54.38, p < .001), race/ethnicity (χ2[9] = 110.78, p < .001), sexual orientation (χ2[3] = 7.85, p < .05), relationship status (χ2[6] = 27.71, p < .001), and region of recruitment (χ2[6] = 1480.51, p < .001). There were no significant differences by venue in attrition during the enrollment process. The only difference in attrition was by race/ethnicity. Compared to White participants, eligible Black (OR: 0.35, p < .01) and Latino (OR: 0.46, p < .05) participants were significantly less likely to enroll in the intervention. There were also no significant differences by recruitment venue in sexual risk behaviors among enrolled participants. These findings suggest that recruitment into online HIV interventions from a variety of venues is feasible for diverse YMSM who are at similar risk for HIV infection.


Men who have sex with men HIV Sexually transmitted infections eHealth Sexual orientation 



This study was funded by the National Institute on Drug Abuse and National Institute of Mental Health (Grant Number R01DA035145). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute on Drug Abuse or the National Institute of Mental Health.

Compliance with Ethical Standards

Conflict of interest

All authors received grants from the National Institute on Drug Abuse (NIDA) and National Institute of Mental Health (NIMH) in support of the work under consideration for publication.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all individual participants in the study.


  1. Beasley, T. M., & Schumacker, R. E. (1995). Multiple regression approach to analyzing contingency tables: Post hoc and planned comparison procedures. Journal of Experimental Education, 64(1), 79–93. Scholar
  2. Bowen, A., Williams, M., & Horvath, K. (2004). Using the internet to recruit rural MSM for HIV risk assessment: Sampling issues. AIDS and Behavior, 8(3), 311–319.CrossRefPubMedPubMedCentralGoogle Scholar
  3. Bull, S. S., Lloyd, L., Rietmeijer, C., & McFarlane, M. (2004). Recruitment and retention of an online sample for an HIV prevention intervention targeting men who have sex with men: The Smart Sex Quest Project. AIDS Care, 16(8), 931–943.CrossRefPubMedGoogle Scholar
  4. CDC. (2015a). Diagnoses of HIV infection in the United States and dependent areas, 2014. HIV Surveillance Report, 26, 1–123.Google Scholar
  5. CDC. (2015b). HIV among gay and bisexual men. Retrieved from
  6. CDC. (2015c). HIV surveillance—Men who have sex with men (MSM). Retrieved from
  7. CDC. (2016). Trends in U.S. HIV Diagnoses, 20052014. Retrieved from
  8. CDC. (2017). Compendium of evidence-based interventions and best practices for HIV prevention. Retrieved from
  9. Chiasson, M. A., Hirshfield, S., Remien, R. H., Humberstone, M., Wong, T., & Wolitski, R. J. (2007). A comparison of on-line and off-line sexual risk in men who have sex with men: An event-based on-line survey. Journal of Acquired Immune Deficiency Syndromes, 44(2), 235–243.CrossRefPubMedGoogle Scholar
  10. Clark, R. C., & Mayer, R. E. (2003). E-Learning and the science of instruction: proven guidelines for consumers and designers of multimedia learning. San Francisco, CA: Jossey-Bass/Pfeiffer.Google Scholar
  11. Dhalla, S., & Poole, G. (2014). Effect of race/ethnicity on participation in HIV vaccine trials and comparison to other trials of biomedical prevention. Human Vaccines & Immunotherapeutics, 10(7), 1974–1984. Scholar
  12. Dodge, B., Reece, M., & Gebhard, P. H. (2008). Kinsey and beyond: Past, present, and future considerations for research on male bisexuality. Journal of Bisexuality, 8(3–4), 175–189. Scholar
  13. Du Bois, S. N., Johnson, S. E., & Mustanski, B. (2012). Examining racial and ethnic minority differences among YMSM during recruitment for an online HIV prevention intervention study. AIDS and Behavior, 16(6), 1430–1435. Scholar
  14. Fernandez, M. I., Varga, L. M., Perrino, T., Collazo, J. B., Subiaul, F., Rehbein, A., … Bowen, G. S. (2004). The Internet as recruitment tool for HIV studies: Viable strategy for reaching at-risk Hispanic MSM in Miami? AIDS Care, 16(8), 953–963.CrossRefPubMedGoogle Scholar
  15. Fisher, J. D., & Fisher, W. A. (2002). The information-motivation-behavioral skills model. In R. J. DiClemente, R. A. Crosby, & M. C. Kegler (Eds.), Emerging theories in health promotion practice and research: Strategies for improving public health (pp. 40–70). San Francisco: Jossey-Bass.Google Scholar
  16. Fisher, J. D., Fisher, W. A., Williams, S. S., & Malloy, T. E. (1994). Empirical tests of an information-motivation-behavioral skills model of AIDS-preventive behavior with gay men and heterosexual university students. Health Psychology, 13(3), 238–250.CrossRefPubMedGoogle Scholar
  17. Frandsen, M., Thow, M., & Ferguson, S. G. (2016). The effectiveness of social media (Facebook) compared with more traditional advertising methods for recruiting eligible participants to health research studies: A randomized, controlled clinical trial. JMIR Research Protocols, 5(3), e161. Scholar
  18. Goedel, W. C., & Duncan, D. T. (2015). Geosocial-networking app usage patterns of gay, bisexual, and other men who have sex with men: Survey among users of Grindr, a mobile dating app. JMIR Public Health Surveillance, 1(1), e4. Scholar
  19. Greene, G. J., Madkins, K., Andrews, K., Dispenza, J., & Mustanski, B. (2016). Implementation and evaluation of the Keep It Up! Online HIV prevention intervention in a community-based setting. AIDS Education and Prevention, 28(3), 231–245. Scholar
  20. Grosskopf, N. A., Harris, J. K., Wallace, B. C., & Nanin, J. E. (2011). Online sex-seeking behaviors of men who have sex with men in New York City. American Journal of Men’s Health, 5(5), 378–385. Scholar
  21. Grov, C. (2012). HIV risk and substance use in men who have sex with men surveyed in bathhouses, bars/clubs, and on Venue of recruitment matters. AIDS and Behavior, 16(4), 807–817. Scholar
  22. Grov, C., Cain, D., Rendina, H. J., Ventuneac, A., & Parsons, J. T. (2016a). Characteristics associated with urethral and rectal Gonorrhea and Chlamydia diagnoses in a US national sample of gay and bisexual men: Results from the One Thousand Strong Panel. Sexually Transmitted Diseases, 43(3), 165–171. Scholar
  23. Grov, C., Cain, D., Whitfield, T. H., Rendina, H. J., Pawson, M., Ventuneac, A., & Parsons, J. T. (2016b). Recruiting a U.S. national sample of HIV-negative gay and bisexual men to complete at-home self-administered HIV/STI testing and surveys: Challenges and Opportunities. Sexuality Research and Social Policy, 13(1), 1–21. Scholar
  24. Grov, C., Rendina, H. J., Jimenez, R., & Parsons, J. T. (2016c). Using online settings to identify gay and bisexual men willing to take or with experience taking PrEP: Implications for researchers and providers. AIDS Education and Prevention, 28(5), 378–392. Scholar
  25. Grov, C., Rendina, H. J., & Parsons, J. T. (2014). Comparing three cohorts of MSM sampled via sex parties, bars/clubs, and Implications for researchers and providers. AIDS Education and Prevention, 26(4), 362–382. Scholar
  26. Halkitis, P. N., Wolitski, R. J., & Millett, G. A. (2013). A holistic approach to addressing HIV infection disparities in gay, bisexual, and other men who have sex with men. American Psychologist, 68(4), 261–273. Scholar
  27. Hatfield, L. A., Ghiselli, M. E., Jacoby, S. M., Cain-Nielsen, A., Kilian, G., McKay, T., & Rosser, B. R. (2010). Methods for recruiting men of color who have sex with men in prevention-for-positives interventions. Prevention Science, 11(1), 56–66. Scholar
  28. Hernandez-Romieu, A. C., Sullivan, P. S., Sanchez, T. H., Kelley, C. F., Peterson, J. L., Del Rio, C., … Rosenberg, E. S. (2014). The comparability of men who have sex with men recruited from venue-time-space sampling and facebook: A cohort study. JMIR Research Protocols, 3(3), e37. Scholar
  29. Huang, E., Marlin, R. W., Young, S. D., Medline, A., & Klausner, J. D. (2016). Using Grindr, a smartphone social-networking application, to increase HIV self-testing among Black and Latino Men who have sex with men in Los Angeles, 2014. AIDS Education and Prevention, 28(4), 341–350. Scholar
  30. Jenkins, R. A. (2012). Recruiting substance-using men who have sex with men into HIV prevention research: Current status and future directions. AIDS and Behavior, 16, 1411–1419. Scholar
  31. Landovitz, R. J., Tseng, C. H., Weissman, M., Haymer, M., Mendenhall, B., Rogers, K., … Shoptaw, S. (2013). Epidemiology, sexual risk behavior, and HIV prevention practices of men who have sex with men using GRINDR in Los Angeles, California. Journal of Urban Health, 90(4), 729–739. Scholar
  32. Lehmiller, J. J., & Ioerger, M. (2014). Social networking smartphone applications and sexual health outcomes among men who have sex with men. PLoS ONE, 9(1), e86603. Scholar
  33. Lemp, G. F., Hirozawa, A. M., Givertz, D., Nieri, G. N., Anderson, L., Lindegren, M. L., … Katz, M. (1994). Seroprevalence of HIV and risk behaviors among young homosexual and bisexual men. The San Francisco/Berkeley Young Men’s Survey. Journal of the American Medical Association, 272(6), 449–454.CrossRefPubMedGoogle Scholar
  34. MacKellar, D., Valleroy, L., Karon, J., Lemp, G., & Janssen, R. (1996). The Young Men’s Survey: Methods for estimating HIV seroprevalence and risk factors among young men who have sex with men. Public Health Reports, 111(Suppl. 1), 138–144.PubMedPubMedCentralGoogle Scholar
  35. Mustanski, B. S. (2001). Getting wired: Exploiting the Internet for the collection of valid sexuality data. Journal of Sex Research, 38(4), 292–301. Scholar
  36. Mustanski, B., Feinstein, B. A., Madkins, K., Sullivan, P., & Swann, G. (2017a). Prevalence and risk factors for rectal and urethral sexually transmitted infections in self-collected samples among young men who have sex with men participating in the Keep It Up! 2.0 randomized controlled trial. Sexually Transmitted Diseases, 44(8), 483–488. Scholar
  37. Mustanski, B., Garofalo, R., Monahan, C., Gratzer, B., & Andrews, R. (2013). Feasibility, acceptability, and preliminary efficacy of an online HIV prevention program for diverse young men who have sex with men: The Keep It Up! intervention. AIDS and Behavior, 17(9), 2999–3012. Scholar
  38. Mustanski, B., Madkins, K., Greene, G. J., Parsons, J. T., Johnson, B. A., Sullivan, P., … Abel, R. (2017b). Internet-based HIV prevention with at-home sexually transmitted infection testing for young men having sex with men: Study protocol of a randomized controlled trial of Keep It Up! 2.0. JMIR Research Protocols, 6(1), e1. Scholar
  39. Mustanski, B., Newcomb, M., Du Bois, S. N., Garcia, S. C., & Grov, C. (2011). HIV in young men who have sex with men: A review of epidemiology, risk and protective factors, and interventions. Journal of Sex Research, 48(2–3), 218–253. Scholar
  40. Mustanski, B., Ryan, D. T., & Garofalo, R. (2014a). Associations of sexually transmitted infections with condom problems among young men who have sex with men. Sexually Transmitted Diseases, 41(7), 427–432. Scholar
  41. Mustanski, B., Starks, T., & Newcomb, M. E. (2014b). Methods for the design and analysis of relationship and partner effects on sexual health. Archives of Sexual Behavior, 43(1), 21–33. Scholar
  42. Parsons, J. T., Vial, A. C., Starks, T. J., & Golub, S. A. (2013). Recruiting drug using men who have sex with men in behavioral intervention trials: A comparison of internet and field-based strategies. AIDS and Behavior, 17(2), 688–699. Scholar
  43. Pequegnat, W., Simon Rosser, B. R., Bowen, A. M., Bull, S. S., DiClemente, R. J., Bockting, W. O., … Zimmerman, R. (2007). Conducting Internet-based HIV/STD prevention survey research: Considerations in design and evaluation. AIDS and Behavior, 11(4), 505–521. Scholar
  44. Prescott, T. L., Phillips Ii, G., DuBois, L. Z., Bull, S. S., Mustanski, B., & Ybarra, M. L. (2016). Reaching adolescent gay, bisexual, and queer men online: Development and refinement of a national recruitment strategy. Journal of Medical Internet Research, 18(8), e200. Scholar
  45. Raymond, H. F., Rebchook, G., Curotto, A., Vaudrey, J., Amsden, M., Levine, D., & McFarland, W. (2010). Comparing internet-based and venue-based methods to sample MSM in the San Francisco Bay Area. AIDS and Behavior, 14(1), 218–224. Scholar
  46. Rendina, H. J., Jimenez, R. H., Grov, C., Ventuneac, A., & Parsons, J. T. (2014). Patterns of lifetime and recent HIV testing among men who have sex with men in New York City who use Grindr. AIDS and Behavior, 18(1), 41–49. Scholar
  47. Rhodes, S. D., DiClemente, R. J., Cecil, H., Hergenrather, K. C., & Yee, L. J. (2002). Risk among men who have sex with men in the United States: A comparison of an Internet sample and a conventional outreach sample. AIDS Education and Prevention, 14(1), 41–50.CrossRefPubMedGoogle Scholar
  48. Ross, M. W., Tikkanen, R., & Mansson, S. A. (2000). Differences between Internet samples and conventional samples of men who have sex with men: Implications for research and HIV interventions. Social Science and Medicine, 51(5), 749–758.CrossRefPubMedGoogle Scholar
  49. Sanchez, T., Smith, A., Denson, D., Dinenno, E., & Lansky, A. (2012). Internet-based methods may reach higher-risk men who have sex with men not reached through venue-based sampling. Open AIDS Journal, 6, 83–89. Scholar
  50. Saxton, P., Dickson, N., & Hughes, A. (2013). Who is omitted from repeated offline HIV behavioural surveillance among MSM? Implications for interpreting trends. AIDS and Behavior, 17(9), 3133–3144. Scholar
  51. Stueve, A., O’Donnell, L. N., Duran, R., San Doval, A., & Blome, J. (2001). Time-space sampling in minority communities: Results with young Latino men who have sex with men. American Journal of Public Health, 91(6), 922–926.CrossRefPubMedPubMedCentralGoogle Scholar
  52. Sullivan, P. S., Grey, J. A., & Simon Rosser, B. R. (2013). Emerging technologies for HIV prevention for MSM: What we have learned, and ways forward. Journal of Acquired Immune Deficiency Syndromes, 63(Suppl. 1), S102–S107. Scholar
  53. Sullivan, P. S., Khosropour, C. M., Luisi, N., Amsden, M., Coggia, T., Wingood, G. M., & Diclemente, R. J. (2011). Bias in online recruitment and retention of racial and ethnic minority men who have sex with men. Journal of Medical Internet Research, 13(2), e38.CrossRefPubMedPubMedCentralGoogle Scholar
  54. Swann, G., Newcomb, M. E., & Mustanski, B. (2018). Validation of the HIV Risk Assessment of Sexual Partners (H-RASP): Comparison to a two month prospective diary study. Archives of Sexual Behavior, 47(1), 121–131. Scholar
  55. U.S. Census Bureau. (2015). Census regions and divisions of the United States. Retrieved from
  56. Vial, A. C., Starks, T. J., & Parsons, J. T. (2014). Finding and recruiting the highest risk HIV-negative men who have sex with men. AIDS Education and Prevention, 26(1), 56–67. Scholar
  57. Yang, Z., Zhang, S., Dong, Z., Jin, M., & Han, J. (2014). Prevalence of unprotected anal intercourse in men who have sex with men recruited online versus offline: A meta-analysis. BMC Public Health, 14, 508. Scholar
  58. Young, S. D., Szekeres, G., & Coates, T. (2013). Sexual risk and HIV prevention behaviours among African-American and Latino MSM social networking users. International Journal of STD and AIDS, 24(8), 643–649. Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Krystal Madkins
    • 1
  • George J. Greene
    • 1
  • Eric Hall
    • 2
    • 4
  • Ruben Jimenez
    • 3
  • Jeffrey T. Parsons
    • 3
  • Patrick S. Sullivan
    • 2
    • 4
  • Brian Mustanski
    • 1
  1. 1.Department of Medical Social Sciences, Feinberg School of Medicine and Institute for Sexual and Gender Minority Health and WellbeingNorthwestern UniversityChicagoUSA
  2. 2.Programs, Research and Innovation in Sexual Minority Health, Rollins School of Public HealthEmory UniversityAtlantaUSA
  3. 3.Center for HIV Educational Studies and TrainingHunter College and the Graduate Center of the City University of New YorkNew YorkUSA
  4. 4.Department of EpidemiologyEmory UniversityAtlantaUSA

Personalised recommendations