AIDS and Behavior

, Volume 17, Issue 6, pp 2202–2210

An Empirical Examination of Respondent Driven Sampling Design Effects Among HIV Risk Groups from Studies Conducted Around the World

  • Lisa G. Johnston
  • Yea-Hung Chen
  • Alfonso Silva-Santisteban
  • H. Fisher Raymond
Original Paper


For studies using respondent driven sampling (RDS), the current practice of collecting a sample twice as large as that used in simple random sampling (SRS) (i.e. design effect of 2.00) may not be sufficient. This paper provides empirical evidence of sample-to-sample variability in design effects using data from nine studies in six countries among injecting drug users, female sex workers, men who have sex with men and male-to-female transgender (MTF) persons. We computed the design effect as the variance under RDS divided by the variance under SRS for a broad range of demographic and behavioral variables in each study. We also estimated several measures for each variable in each study that we hypothesized might be related to design effect: the number of waves needed for equilibrium, homophily, and mean network size. Design effects for all studies ranged from 1.20 to 5.90. Mean design effects among all studies ranged from 1.50 to 3.70. A particularly high design effect was found for employment status (design effect of 5.90) of MTF in Peru. This may be explained by a “bottleneck”—defined as the occurrence of a relatively small number of recruitment ties between two groups in the population. A design effect of two for RDS studies may not be sufficient. Since the mean design effect across all studies was 2.33, an effect slightly above 2.00 may be adequate; however, an effect closer to 3.00 or 4.00 might be more appropriate.


Respondent driven sampling Design effects HIV/AIDS surveillance Hard-to-reach populations 


  1. 1.
    Carlson RG, Wang J, Siegal HA, Falck RS, Guo J. An ethnographic approach to targeted sampling: problems and solutions in AIDS prevention research among injection drug and crack-cocaine users. Hum Organ. 1994;53(3):279–86.Google Scholar
  2. 2.
    Magnani R, Sabin K, Saidel T, Heckathorn D. Review of sampling hard-to-reach populations for HIV surveillance. AIDS. 2005;19(Suppl 2):S67–72.PubMedCrossRefGoogle Scholar
  3. 3.
    Family Health International. Behavioral surveillance surveys: guidelines for repeated behavioral surveys in populations at risk of HIV. Arlington: Family Health International. 2000. Accessed 15 Nov 2012.
  4. 4.
    Goel S, Salganik MJ. Assessing respondent-driven sampling. Proc Natl Acad Sci USA. 2010;107:6743–7.PubMedCrossRefGoogle Scholar
  5. 5.
    Salganik MJ. Variance estimation, design effects and sample size calculations for respondent driven sampling. J Urban Health. 2006;83(Suppl 7):98–112. doi:10.1007/s11524-006-9106-x.CrossRefGoogle Scholar
  6. 6.
    Heckathorn D. Respondent driven sampling: a new approach to the study of hidden populations. Soc Probl. 1997;44(2):174–99.CrossRefGoogle Scholar
  7. 7.
    Malekinejad M, Johnston LG, Kendall C, Kerr L, Rifkin M, Rutherford G. Using respondent-driven sampling methodology for HIV biological and behavioral surveillance in international settings: a systematic review. AIDS Behav. 2008;12(suppl 1):105–30.CrossRefGoogle Scholar
  8. 8.
    Johnston LG, Malekinejad M, Rifkin MR, Rutherford GW, Kendall C. Implementation challenges to using respondent-driven sampling methodology for HIV biological and behavioral surveillance: field experiences in international settings. AIDS Behav. 2008;12(Suppl 1):131–41.CrossRefGoogle Scholar
  9. 9.
    Heckathorn D. Respondent driven sampling II: deriving valid population estimates from Chain-referral samples of hidden populations. Soc Probl. 2002;49(1):11–34.CrossRefGoogle Scholar
  10. 10.
    Wejnert C, Pham H, Krishna N, Le B, DiNenno E. Estimating design effect and calculating sample size for respondent-driven sampling studies of injection drug users in the United States. AIDS Behav. 2012;16(4):797–806.PubMedCrossRefGoogle Scholar
  11. 11.
    Ruan S, Yang H, Zhu Y, et al. HIV prevalence and correlates of unprotected anal intercourse among men who have sex with men, Jinan. China. AIDS Behav. 2008;12(3):469–75.CrossRefGoogle Scholar
  12. 12.
    Lane T, Raymond HF, Dladla S, et al. High HIV prevalence among men who have sex with men in Soweto, south Africa: results from the Soweto men’s study. AIDS Behav. 2011;15(3):626–34.PubMedCrossRefGoogle Scholar
  13. 13.
    Johnston LG, Saumtally A, Corceal S, Mahadoo I, Oodally F. High HIV and hepatitis C prevalence amongst injecting drug users in Mauritius: findings from a population size estimation and respondent driven sampling survey. Int J Drug Policy. 2011;22(4):252–8. Epub 2011 Jun 22.PubMedCrossRefGoogle Scholar
  14. 14.
    Johnston LG.: Integrated behavioral and biological surveillance survey among injecting drug users in Mauritius, 2009; Mauritius Ministry of Health, Port Louis. 2009. Accessed 15 Nov 2012.
  15. 15.
    Taran YS, Johnston LG, Pohorilac NB, Saliuk T. Correlates of HIV risk among injecting drug users in sixteen Ukrainian cities. AIDS Behav. 2011;15(1):65–74.PubMedCrossRefGoogle Scholar
  16. 16.
    Heckathorn DD. Extensions of respondent-driven sampling: analyzing continuous variables and controlling for differential recruitment. Sociol Methodol. 2007;37:151–207.CrossRefGoogle Scholar
  17. 17.
    Volz E, Heckathorn DD. Probability-based estimation theory for respondent-driven sampling. J Off Stat. 2008;24(Suppl 1):79–97.Google Scholar
  18. 18.
    Salganik MJ, Heckathorn DD. Sampling and estimation in hidden populations using respondent driven sampling. Sociol Methodol. 2004;34:193–239.CrossRefGoogle Scholar
  19. 19.
    McPherson M, Smith-Lovin L, Cook JM. Birds of a feather: homophily in social networks. Ann Rev Sociol. 2011;27:415–44.CrossRefGoogle Scholar
  20. 20.
    Szwarcwald CL, de Souza Júnior PR, Damacena GN, Junior AB, Kendall C. Analysis of data collected by RDS among sex workers in 10 Brazilian cities, 2009: estimation of the prevalence of HIV, variance, and design effect. J Acquir Immune Defic Syndr. 2011;57(Suppl 3):S129–35.PubMedCrossRefGoogle Scholar
  21. 21.
    Johnston LG, Khanam R, Reza M, et al. The effectiveness of respondent driven sampling for recruiting males who have sex with males in Dhaka, Bangladesh: a pilot study. AIDS Behav. 2007;2(2):294–304.Google Scholar
  22. 22.
    Goel S, Salganik MJ. Respondent-driven sampling as Markov chain Monte Carlo. Stat Med. 2009;28:2202–29.PubMedCrossRefGoogle Scholar
  23. 23.
    Poon AFY, Brouwer KC, Strathdee SA, et al. Parsing social network survey data from hidden populations using stochastic context-free grammars. PLoS ONE. 2009;4(9):e6777. doi:10.1371/journal.pone.0006777.PubMedCrossRefGoogle Scholar
  24. 24.
    Gile K J. Improved inference for Respondent-Driven sampling data with application to HIV prevalence estimation. J Am Stat Assoc. 2011;106(493):135146.CrossRefGoogle Scholar
  25. 25.
    Gile KJ, Handcock MS. Respondent-driven sampling: an assessment of current methodology. Sociol Methodol. 2010;40:285–327.PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Lisa G. Johnston
    • 1
    • 4
  • Yea-Hung Chen
    • 2
  • Alfonso Silva-Santisteban
    • 3
  • H. Fisher Raymond
    • 2
  1. 1.University of California, San Francisco, Global Health SciencesSan FranciscoUSA
  2. 2.San Francisco Department of Public HealthSan FranciscoUSA
  3. 3.Unit of Health, Sexuality and Human DevelopmentCayetano Heredia University School of Public HealthLimaPeru
  4. 4.AmsterdamThe Netherlands

Personalised recommendations