Archives of Sexual Behavior

, Volume 47, Issue 4, pp 1069–1084 | Cite as

Estimates of Non-Heterosexual Prevalence: The Roles of Anonymity and Privacy in Survey Methodology

  • Ronald E. Robertson
  • Felix W. Tran
  • Lauren N. Lewark
  • Robert Epstein
Original Paper
First Online:
Received:
Revised:
Accepted:

Abstract

When do people feel comfortable enough to provide honest answers to sensitive questions? Focusing specifically on sexual orientation prevalence—a measure that is sensitive to the pressures of heteronormativity—the present study was conducted to examine the variability in U.S. estimates of non-heterosexual identity prevalence and to determine how comfortable people are with answering questions about their sexual orientation when asked through commonly used survey modes. We found that estimates of non-heterosexual prevalence in the U.S. increased as the privacy and anonymity of the survey increased. Utilizing an online questionnaire, we rank-ordered 16 survey modes by asking people to rate their level of comfort with each mode in the context of being asked questions about their sexual orientation. A demographically diverse sample of 652 individuals in the U.S. rated each mode on a scale from −5 (very uncomfortable) to +5 (very comfortable). Modes included anonymous (name not required) and non-anonymous (name required) versions of questions, as well as self-administered and interviewer-administered versions. Subjects reported significantly higher mean comfort levels with anonymous modes than with non-anonymous modes and significantly higher mean comfort levels with self-administered modes than with interviewer-administered modes. Subjects reported the highest mean comfort level with anonymous online surveys and the lowest with non-anonymous personal interviews that included a video recording. Compared with the estimate produced by an online survey with a nationally representative sample, surveys utilizing more intrusive methodologies may have underestimated non-heterosexual prevalence in the U.S. by between 50 and 414%. Implications for public policy are discussed.

Keywords

Sexual orientation Prevalence of non-heterosexuality Survey methodology Heteronormativity 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgements

Earlier versions of this article were presented at the 2013 annual meeting of the Society for the Scientific Study of Sexuality and at the 2014 annual meeting of the Western Psychological Association. The authors would like to thank Brian W. Ward, Debby Herbenick, Randy Sell, and Tom W. Smith for their helpful comments and suggestions.

Compliance with Ethical Standards

Conflict of interest

All authors declare no conflicts of interest.

Human and Animal Rights

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. This article does not contain any studies with animals performed by any of the authors.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

References

  1. Aquilino, W. S. (1994). Interview mode effects in surveys of drug and alcohol use: A field experiment. Public Opinion Quarterly, 58, 210–240.CrossRefGoogle Scholar
  2. Aquilino, W. S., & LoSciuto, L. A. (1990). Effects of interview mode on self-reported drug use. Public Opinion Quarterly, 54, 362–393.CrossRefGoogle Scholar
  3. Badgett, M. V. (2009). Best practices for asking questions about sexual orientation on surveys. Los Angeles, CA: The Williams Institute.Google Scholar
  4. Baker, R. P., Bradburn, N. M., & Johnson, R. A. (1995). Computer-assisted personal interviewing: An experimental evaluation of data quality and cost. Journal of Official Statistics, 11, 413–431.Google Scholar
  5. Barnett, J. (1998). Sensitive questions and response effects: An evaluation. Journal of Managerial Psychology, 13, 63–76.CrossRefGoogle Scholar
  6. Beatty, P., & Herrmann, D. (2002). To answer or not to answer: Decision processes related to survey item nonresponse. In R. M. Groves, D. A. Dillman, J. L. Eltinge, & R. J. A. Little (Eds.), Survey nonresponse (pp. 71–85). New York, NY: Wiley.Google Scholar
  7. Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012). Evaluating online labor markets for experimental research: Amazon.com’s Mechanical Turk. Political Analysis, 20, 351–368.CrossRefGoogle Scholar
  8. Black, M. C., Basile, K. C., Breiding, M. J., Smith, S. G., Walters, M. L., Merrick, M. T., & Stevens, M. R. (2011). National intimate partner and sexual violence survey. Atlanta, GA: Centers for Disease Control and Prevention.Google Scholar
  9. Bonett, D. G. (2008). Confidence intervals for standardized linear contrasts of means. Psychological Methods, 13, 99–109.PubMedCrossRefGoogle Scholar
  10. Booth-Kewley, S., Larson, G. E., & Miyoshi, D. K. (2007). Social desirability effects on computerized and paper-and-pencil questionnaires. Computers in Human Behavior, 23, 463–477.CrossRefGoogle Scholar
  11. Bostwick, W. B., Boyd, C. J., Hughes, T. L., & McCabe, S. E. (2010). Dimensions of sexual orientation and the prevalence of mood and anxiety disorders in the United States. American Journal of Public Health, 100, 468–475.PubMedPubMedCentralCrossRefGoogle Scholar
  12. Bradburn, N. M., Frankel, M. R., Hunt, E., Ingels, J., Schoua-Glusberg, A., Wojcik, M., & Pergamit, M. R. (1991, May 16–19). A comparison of computer-assisted personal interviews (CAPI) with personal interviews in the national longitudinal survey of labor market behavior-youth cohort. Lecture presented at the annual meeting of the American Association for Public Opinion Research, Phoenix, AZ.Google Scholar
  13. Bradburn, N., & Mason, W. (1964). The effect of question order on responses. Journal of Marketing Research, 1, 57–61.CrossRefGoogle Scholar
  14. Bradford, J. B., & Mayer, K. H. (2008). Demography and the LGBT population: What we know, don’t know, and how the information helps to inform clinical practice. In H. Makadon, K. Mayer, J. Potter, & H. Goldhammer (Eds.), The Fenway guide to lesbian, gay, bisexual, and transgender health (pp. 25–41). Philadelphia, PA: American College of Physicians.Google Scholar
  15. Bradford, J., & Mustanski, B. (2014). Health disparities among sexual minority youth: The value of population data. American Journal of Public Health, 104, 197. doi: 10.2105/AJPH.2013.301818.
  16. Brim, O. G., Baltes, P. B., Bumpass, L. L., Cleary, P. D., Featherman, D. L., Hazzard, W. R., & Shweder, R. A. (2011). National Survey of Midlife Development in the United States (MIDUS), 1995–1996 (ICPSR02760-v8). Ann Arbor, MI: Inter-University Consortium for Political and Social Research.Google Scholar
  17. Buchanan, T. (2000). Potential of the Internet for personality research. In M. H. Birnbaum (Ed.), Psychological experiments on the Internet (pp. 121–140). San Diego, CA: Academic Press.CrossRefGoogle Scholar
  18. Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6, 3–5.PubMedCrossRefGoogle Scholar
  19. Burkill, S., Copas, A., Couper, M. P., Clifton, S., Prah, P., Datta, J., et al. (2016). Using the web to collect data on sensitive behaviours: A study looking at mode effects on the British National Survey of Sexual Attitudes and Lifestyles. PLoS One, 11(2), e0147983. doi: 10.1371/journal.pone.0147983.PubMedPubMedCentralCrossRefGoogle Scholar
  20. Casler, K., Bickel, L., & Hackett, E. (2013). Separate but equal? A comparison of participants and data gathered via Amazon’s MTurk, social media, and face-to-face behavioral testing. Computers in Human Behavior, 29, 2156–2160.CrossRefGoogle Scholar
  21. CDC. (n.d.-a). National health interview survey questionnaires, datasets, and related documentation 1997 to the present. Retrieved September 2, 2015 from http://www.cdc.gov/nchs/nhis/quest_data_related_1997_forward.htm.
  22. CDC. (n.d.-b). National health and nutritional examination survey (NHANES) 2011–2012 questionnaire data overview. Retrieved from http://www.cdc.gov/nchs/nhanes/nhanes2011-2012/quexdoc_g.htm.
  23. Centers for Disease Control and Prevention (CDC) National Center for Health Statistics (NCHS). (2009–2012). National health and nutrition examination survey questionnaire (or examination protocol, or laboratory protocol). Hyattsville, MD: U.S. Department of Health and Human Services, CDC.Google Scholar
  24. Cerny, J. A., & Janssen, E. (2011). Patterns of sexual arousal in homosexual, bisexual, and heterosexual men. Archives of Sexual Behavior, 40, 687–697.PubMedCrossRefGoogle Scholar
  25. Chandler, J., & Shapiro, D. (2016). Conducting clinical research using crowdsourced convenience samples. Annual Review of Clinical Psychology, 12, 53–81.PubMedCrossRefGoogle Scholar
  26. Chandra, A., Mosher, W. D., & Copen, C. E. (2013). Sexual behavior, sexual attraction, and sexual identity in the United States: Data from the 2006–2010 national survey of family growth. In A. K. Baumle (Ed.), International handbook on the demography of sexuality (pp. 45–66). Dordrecht: Springer.CrossRefGoogle Scholar
  27. Chang, L., & Krosnick, J. A. (2002). A comparison of the random digit dialing telephone survey methodology with Internet survey methodology as implemented by Knowledge Networks and Harris Interactive. Columbus, OH: Department of Psychology, Ohio State University.Google Scholar
  28. Chang, L., & Krosnick, J. A. (2009). National surveys via RDD telephone interviewing versus the Internet comparing sample representativeness and response quality. Public Opinion Quarterly, 73, 641–678.CrossRefGoogle Scholar
  29. Chang, L., & Krosnick, J. A. (2010). Comparing oral interviewing with self-administered computerized questionnaires: An experiment. Public Opinion Quarterly, 74, 154–167.CrossRefGoogle Scholar
  30. Chivers, M. L., Rieger, G., Latty, E., & Bailey, J. M. (2004). A sex difference in the specificity of sexual arousal. Psychological Science, 15, 736–744.PubMedCrossRefGoogle Scholar
  31. Chivers, M. L., Seto, M. C., & Blanchard, R. (2007). Gender and sexual orientation differences in sexual response to sexual activities versus gender of actors in sexual films. Journal of Personality and Social Psychology, 93, 1008–1121.CrossRefGoogle Scholar
  32. Cochran, S. D., & Mays, V. M. (2006). Estimating prevalence of mental and substance-using disorders among lesbians and gay men from existing national health data. In A. M. Omoto & H. S. Kurtzman (Eds.), Sexual orientation and mental health: Examining identity and development in lesbian, gay, and bisexual people (pp. 143–165). Washington, DC: American Psychological Association.CrossRefGoogle Scholar
  33. Cochran, S. D., & Mays, V. M. (2007). Physical health complaints among lesbians, gay men, and bisexual and homosexually experienced heterosexual individuals: Results from the California Quality of Life Survey. American Journal of Public Health, 97, 2048–2055.PubMedPubMedCentralCrossRefGoogle Scholar
  34. Coffman, K. B., Coffman, L. C., & Ericson, K. M. M. (2016). The size of the LGBT population and the magnitude of anti-gay sentiment are substantially underestimated. Management Science. doi: 10.1287/mnsc.2016.2503.Google Scholar
  35. Conron, K. J., Mimiaga, M. J., & Landers, S. J. (2010). A population-based study of sexual orientation identity and gender differences in adult health. American Journal of Public Health, 100, 1953–1960.PubMedPubMedCentralCrossRefGoogle Scholar
  36. Crowne, D. P., & Marlowe, D. (1964). The approval motive: Studies in evaluative dependence. New York: Wiley.Google Scholar
  37. Curtin, R., Presser, S., & Singer, E. (2000). The effects of response rate changes on the index of consumer sentiment. Public Opinion Quarterly, 64, 413–428.PubMedCrossRefGoogle Scholar
  38. Dahlgreen, W., & Shakespeare, A. (2015, August 16). 1 in 2 young people say they are not 100% heterosexual. Retrieved from https://yougov.co.uk/news/2015/08/16/half-young-not-heterosexual/.
  39. Dahlhamer, J. M., Galinsky, A. M., Joestl, S. S., & Ward, B. W. (2014). Sexual orientation in the 2013 National Health Interview Survey: A quality assessment. Vital and Health Statistics. Series 2. Data Evaluation and Methods Research, 169, 1–32.Google Scholar
  40. DeMaio, T. J. (1984). Social desirability and survey measurement: A review. In C. Turner & E. Martin (Eds.), Surveying subjective phenomena (Vol. 2, pp. 257–282). New York, NY: Sage.Google Scholar
  41. Diamond, L. M. (2008). Sexual fluidity: Understanding women’s love and desire. Cambridge, MA: Harvard University Press.Google Scholar
  42. Diaz, R. M., Ayala, G., Bein, E., Henne, J., & Marin, B. V. (2001). The impact of homophobia, poverty, and racism on the mental health of gay and bisexual Latino men: Findings from 3 U.S. cities. American Journal of Public Health, 91, 927–932.PubMedPubMedCentralCrossRefGoogle Scholar
  43. Dilley, J. A., Simmons, K. W., Boysun, M. J., Pizacani, B. A., & Stark, M. J. (2010). Demonstrating the importance and feasibility of including sexual orientation in public health surveys: Health disparities in the Pacific Northwest. American Journal of Public Health, 100, 460–467.PubMedPubMedCentralCrossRefGoogle Scholar
  44. Dillman, D. A., Smyth, J. D., & Christian, L. M. (2014). Internet, phone, mail, and mixed-mode surveys: The tailored design method. Hoboken, NJ: Wiley.Google Scholar
  45. Durant, L. E., Carey, M. P., & Schroder, K. E. (2002). Effects of anonymity, gender, and erotophilia on the quality of data obtained from self-reports of socially sensitive behaviors. Journal of Behavioral Medicine, 25, 439–467.CrossRefGoogle Scholar
  46. Dwight, S. A., & Feigelson, M. E. (2000). A quantitative review of the effect of computerized testing on the measurement of social desirability. Educational and Psychological Measurement, 60, 340–360.CrossRefGoogle Scholar
  47. Epstein, R., McKinney, P., Fox, S., & Garcia, C. (2012). Support for a fluid-continuum model of sexual orientation: A large-scale Internet study. Journal of Homosexuality, 59, 1356–1381.PubMedCrossRefGoogle Scholar
  48. Feinstein, B. A., Dyar, C., Bhatia, V., Latack, J. A., & Davila, J. (2016). Conservative beliefs, attitudes toward bisexuality, and willingness to engage in romantic and sexual activities with a bisexual partner. Archives of Sexual Behavior, 45, 1535–1550.PubMedCrossRefGoogle Scholar
  49. Fortson, B. L., Scotti, J. R., Ben, K. S. D., & Chen, Y. C. (2006). Reliability and validity of an Internet traumatic stress survey with a college student sample. Journal of Traumatic Stress, 19, 709–720.PubMedCrossRefGoogle Scholar
  50. Fredriksen-Goldsen, K. I., Kim, H. J., & Barkan, S. E. (2012). Disability among lesbian, gay, and bisexual adults: Disparities in prevalence and risk. American Journal of Public Health, 102, e16–e21.PubMedPubMedCentralCrossRefGoogle Scholar
  51. Fu, H., Darroch, J. E., Henshaw, S. K., & Kolb, E. (1998). Measuring the extent of abortion underreporting in the 1995 national survey of family growth. Family Planning Perspectives, 30, 128–138.PubMedCrossRefGoogle Scholar
  52. Gates, G. J. (2013a). Demographics and LGBT health. Journal of Health and Social Behavior, 54, 72–74.PubMedCrossRefGoogle Scholar
  53. Gates, G. J. (2013b). Geography of the LGBT population. In A. K. Baumle (Ed.), International handbook on the demography of sexuality (pp. 229–242). Dordrecht: Springer.CrossRefGoogle Scholar
  54. Gates, G. J. (2014a). LGBT demographics: Comparisons among population-based surveys. Los Angeles, CA: The Williams Institute.Google Scholar
  55. Gates, G. J. (2014b). In U.S., LGBT more likely than non-LGBT to be uninsured. Retrieved from http://www.gallup.com/poll/175445/lgbt-likely-non-lgbt-uninsured.aspx.
  56. Gates, G. J., & Newport, F. (2012). Special report: 3.4% of U.S. adults identify as LGBT. Retrieved from http://www.gallup.com/poll/158066/special-report-adults-identify-lgbt.aspx.
  57. Gay and Lesbian Medical Association (GLMA). (2001). Healthy People 2010: Companion document for lesbian, gay, bisexual and transgender (LGBT) health. San Francisco, CA: GLMA.Google Scholar
  58. Gilman, S. E., Cochran, S. D., Mays, V. M., Hughes, M., Ostrow, D., & Kessler, R. C. (2001). Risk of psychiatric disorders among individuals reporting same-sex sexual partners in the national comorbidity survey. American Journal of Public Health, 91, 933–939.PubMedPubMedCentralCrossRefGoogle Scholar
  59. Gnambs, T., & Kaspar, K. (2015). Disclosure of sensitive behaviors across self-administered survey modes: A meta-analysis. Behavior Research Methods, 47, 1237–1259.PubMedCrossRefGoogle Scholar
  60. Goffman, E. (1963). Stigma: Notes on the management of spoiled identity. New York, NY: Simon and Schuster.Google Scholar
  61. Graham, R., Berkowitz, B., Blum, R., Bockting, W., Bradford, J., de Vries, B., … Makadon, H. (2011). The health of lesbian, gay, bisexual, and transgender people: Building a foundation for better understanding. Washington, DC: Institute of Medicine.Google Scholar
  62. Graham, A., & Papandonatos, G. (2008). Reliability of Internet- versus telephone-administered questionnaires in a diverse sample of smokers. Journal of Medical Internet Research, 10, e8.PubMedPubMedCentralCrossRefGoogle Scholar
  63. Gruskin, E. P., Greenwood, G. L., Matevia, M., Pollack, L. M., & Bye, L. L. (2007). Disparities in smoking between the lesbian, gay, and bisexual population and the general population in California. American Journal of Public Health, 97, 1496–1502.PubMedPubMedCentralCrossRefGoogle Scholar
  64. Herbenick, D., Reece, M., Schick, V., Sanders, S. A., Dodge, B., & Fortenberry, J. D. (2010). Sexual behavior in the United States: Results from a national probability sample of men and women ages 14–94. Journal of Sexual Medicine, 7, 255–265.PubMedCrossRefGoogle Scholar
  65. Hinkins, S., Crego, A., English, N., Pedlow, S., & Wolter, K. M. (n.d.). 2010 national sample frame. Retrieved from http://www.norc.org/Research/Projects/Pages/2010-national-sample-frame.aspx.
  66. Holbrook, A. L., & Krosnick, J. A. (2010). Social desirability bias in voter turnout reports tests using the item count technique. Public Opinion Quarterly, 74, 37–67.CrossRefGoogle Scholar
  67. Hosmer, D. W., & Lemeshow, S. (2000). Applied logistic regression. New York: Wiley.CrossRefGoogle Scholar
  68. Hyman, H. (1944). Do they tell the truth? Public Opinion Quarterly, 8, 557–559.CrossRefGoogle Scholar
  69. Internet Live Stats. (2016). Internet users in the U.S.A. (2016*). Retrieved from http://www.internetlivestats.com.
  70. Itaborahy, L., & Zhu, J. (2014). State-sponsored homophobia, a world survey of laws: Criminalisation, protection and recognition of same-sex love (9th ed.). Geneva: International Lesbian, Gay, Bisexual and Intersex Associates.Google Scholar
  71. Johnson, T., & Fendrich, M. (2005). Modeling sources of self-report bias in a survey of drug use epidemiology. Annals of Epidemiology, 15, 381–389.PubMedCrossRefGoogle Scholar
  72. Joinson, A. (1999). Social desirability, anonymity, and Internet-based questionnaires. Behavior Research Methods, Instruments, & Computers, 31, 433–438.CrossRefGoogle Scholar
  73. Joinson, A. N., Woodley, A., & Reips, U. D. (2007). Personalization, authentication and self-disclosure in self-administered Internet surveys. Computers in Human Behavior, 23, 275–285.CrossRefGoogle Scholar
  74. Joloza, T., Evans, J., O’Brien, R., & Potter-Collins, A. (2010). Measuring sexual identity: An evaluation report. Newport, UK: Office for National Statistics.Google Scholar
  75. Jones, E. F., & Forrest, J. D. (1992). Underreporting of abortion in surveys of U.S. women: 1976 to 1988. Demography, 29, 113–126.PubMedCrossRefGoogle Scholar
  76. Jones, R. K., & Kost, K. (2007). Underreporting of induced and spontaneous abortion in the United States: An analysis of the 2002 National Survey of Family Growth. Studies in Family Planning, 38, 187–197.PubMedCrossRefGoogle Scholar
  77. Kann, L., Olsen, E. O., McManus, T., Kinchen, S., Chyen, D., Harris, W. A., & Wechsler, H. (2011). Sexual identity, sex of sexual contacts, and health-risk behaviors among students in grades 9–12-youth risk behavior surveillance, selected sites, United States, 2001–2009. Morbidity and Mortality Weekly Report. Surveillance Summaries, 60(7), 1–133.PubMedGoogle Scholar
  78. Kanuha, V. (1999). The social process of passing to manage stigma: Acts of internalized oppression or acts of resistance? Journal of Sociology and Social Welfare, 26(4), 27–46.Google Scholar
  79. Katz-Wise, S. L. (2015). Sexual fluidity in young adult women and men: Associations with sexual orientation and sexual identity development. Psychology & Sexuality, 6, 189–208.CrossRefGoogle Scholar
  80. Keeter, S., McGeeney, K., Igielnik, R., Mercer, A., & Mathiowetz, N. (2015, May 13). From telephone to the web: The challenge of mode of interview effects in public opinion polls. Retrieved from http://www.pewresearch.org/2015/05/13/from-telephone-to-the-web-the-challenge-of-mode-of-interview-effects-in-public-opinion-polls/.
  81. Keeter, S., Miller, C., Kohut, A., Groves, R. M., & Presser, S. (2000). Consequences of reducing nonresponse in a national telephone survey. Public Opinion Quarterly, 64, 125–148.PubMedCrossRefGoogle Scholar
  82. Kitzinger, C. (2005). Heteronormativity in action: Reproducing the heterosexual nuclear family in after-hours medical calls. Social Problems, 52, 477–498.CrossRefGoogle Scholar
  83. Kleck, G., & Roberts, K. (2012). What survey modes are most effective in eliciting self-reports of criminal or delinquent behavior? In L. Gideon (Ed.), Handbook of survey methodology for the social sciences (pp. 417–439). New York, NY: Springer.CrossRefGoogle Scholar
  84. Kreuter, F., Presser, S., & Tourangeau, R. (2008). Social desirability bias in CATI, IVR, and Web surveys: The effects of mode and question sensitivity. Public Opinion Quarterly, 72, 847–865.CrossRefGoogle Scholar
  85. Krosnick, J. A., & Alwin, D. F. (1987). An evaluation of a cognitive theory of response-order effects in survey measurement. Public Opinion Quarterly, 51, 201–219.CrossRefGoogle Scholar
  86. Krumpal, I. (2013). Determinants of social desirability bias in sensitive surveys: A literature review. Quality & Quantity, 47, 2025–2047.CrossRefGoogle Scholar
  87. Langhaug, L. F., Sherr, L., & Cowan, F. M. (2010). How to improve the validity of sexual behaviour reporting: Systematic review of questionnaire delivery modes in developing countries. Tropical Medicine & International Health, 15, 362–381.CrossRefGoogle Scholar
  88. Lee, R. M. (1993). Doing research on sensitive topics. Thousand Oaks, CA: Sage.Google Scholar
  89. Lee, S., McClain, C., Webster, N., & Han, S. (2016). Question order sensitivity of subjective well-being measures: Focus on life satisfaction, self-rated health, and subjective life expectancy in survey instruments. Quality of Life Research, 25, 2497–2510.PubMedCrossRefGoogle Scholar
  90. Lensvelt-Mulders, G. J., Hox, J. J., & Van Der Heijden, P. G. (2005). How to improve the efficiency of randomised response designs. Quality & Quantity, 39, 253–265.CrossRefGoogle Scholar
  91. Locke, S. D., & Gilbert, B. O. (1995). Method of psychological assessment, self-disclosure, and experiential differences: A study of computer, questionnaire, and interview assessment formats. Journal of Social Behavior and Personality, 10, 255–263.Google Scholar
  92. Lucas, G. M., Gratch, J., King, A., & Morency, L. P. (2014). It’s only a computer: Virtual humans increase willingness to disclose. Computers in Human Behavior, 37, 94–100.CrossRefGoogle Scholar
  93. Mason, W., & Suri, S. (2012). Conducting behavioral research on Amazon’s Mechanical Turk. Behavior Research Methods, 44, 1–23.PubMedCrossRefGoogle Scholar
  94. Mayer, K. H., Bradford, J. B., Makadon, H. J., Stall, R., Goldhammer, H., & Landers, S. (2008). Sexual and gender minority health: What we know and what needs to be done. American Journal of Public Health, 98, 989–995.PubMedPubMedCentralCrossRefGoogle Scholar
  95. McLaughlin, K. A., Hatzenbuehler, M. L., & Keyes, K. M. (2010). Responses to discrimination and psychiatric disorders among Black, Hispanic, female, and lesbian, gay, and bisexual individuals. American Journal of Public Health, 100, 1477–1484.PubMedPubMedCentralCrossRefGoogle Scholar
  96. Millar, M. M., & Dillman, D. A. (2011). Improving response to Web and mixed-mode surveys. Public Opinion Quarterly, 75, 249–269.CrossRefGoogle Scholar
  97. Miller, J. D. (1984). A new survey technique for studying deviant behavior. Doctoral dissertation, George Washington University.Google Scholar
  98. Mock, S. E., & Eibach, R. P. (2012). Stability and change in sexual orientation identity over a 10-year period in adulthood. Archives of Sexual Behavior, 41, 641–648.PubMedCrossRefGoogle Scholar
  99. Moore, P. (2015). A third of young Americans say they aren’t 100% heterosexual. Retrieved from https://today.yougov.com/news/2015/08/20/third-young-americans-exclusively-heterosexual/?sid=5388f1ffdd52b8ed110008bc&wpsrc=slatest_newsletter.
  100. Morral, A. R., McCaffrey, D., & Iguchi, M. Y. (2000). Hardcore drug users claim to be occasional users: Drug use frequency underreporting. Drug and Alcohol Dependence, 57, 193–202.PubMedCrossRefGoogle Scholar
  101. Mustanski, B., Van Wagenen, A., Birkett, M., Eyster, S., & Corliss, H. L. (2014). Identifying sexual orientation health disparities in adolescents: Analysis of pooled data from the Youth Risk Behavior Survey, 2005 and 2007. American Journal of Public Health, 104, 211–217.PubMedPubMedCentralCrossRefGoogle Scholar
  102. Nathanson, A. T., & Reinert, S. E. (1999). Windsurfing injuries: Results of a paper-and Internet-based survey. Wilderness & Environmental Medicine, 10, 218–225.CrossRefGoogle Scholar
  103. National Institute on Alcohol Abuse and Alcoholism (NIAAA). (2010). U.S. alcohol epidemiological data reference manual (2nd ed., Vol. 8). Retrieved from http://pubs.niaaa.nih.gov/publications/NESARC_DRM2/NESARC2DRM.pdf.
  104. National Survey of Family Growth (NSFG). (n.d). 2006–2010 NSFG questionnaires. Retrieved from http://www.cdc.gov/nchs/nsfg/nsfg_2006_2010_questionnaires.htm.
  105. NORC at the University of Chicago. (n.d.). 2010 national sample frame. Retrieved from http://www.norc.org/Research/Projects/Pages/2010-national-sample-frame.aspx.
  106. O’Reilly, J., Hubbard, M., Lessler, J., Biemer, P., & Turner, C. F. (1994). Audio computer assisted self-interviewing: New technology for data collection on sensitive issues and special populations. Journal of Official Statistics, 10, 197–214.PubMedPubMedCentralGoogle Scholar
  107. Paolacci, G., & Chandler, J. (2014). Inside the Turk: Understanding Mechanical Turk as a participant pool. Current Directions in Psychological Science, 23, 184–188.CrossRefGoogle Scholar
  108. Parry, H. J., & Crossley, H. M. (1950). Validity of responses to survey questions. Public Opinion Quarterly, 14, 61–80.CrossRefGoogle Scholar
  109. Paulhus, D. L. (1984). Two-component models of socially desirable responding. Journal of Personality and Social Psychology, 46, 598.CrossRefGoogle Scholar
  110. Paulhus, D. L. (1986). Self-deception and impression management in test responses. In A. Angleitner & J. S. Wiggins (Eds.), Personality assessment via questionnaires (pp. 143–165). New York, NY: Springer.CrossRefGoogle Scholar
  111. Paulhus, D. L. (2002). Socially desirable responding: The evolution of a construct. In H. I. Braun, D. N. Jackson, & D. E. Wiley (Eds.), The role of constructs in psychological and educational measurement (pp. 51–77). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
  112. Peer, E., Vosgerau, J., & Acquisti, A. (2014). Reputation as a sufficient condition for data quality on Amazon Mechanical Turk. Behavior Research Methods, 46, 1023–1031.PubMedCrossRefGoogle Scholar
  113. Pew Research Center. (2013, June 13). A survey of LGBT Americans. Retrieved from http://www.pewsocialtrends.org/2013/06/13/a-survey-of-lgbt-americans/.
  114. Pew Research Center. (n.d.). Global views on morality. Retrieved from http://www.pewglobal.org/2014/04/15/global-morality/.
  115. Preisendörfer, P., & Wolter, F. (2013). Asking sensitive questions: An evaluation of the randomized response technique versus direct questioning using individual validation data. Sociological Methods & Research, 42, 321–353.CrossRefGoogle Scholar
  116. Ramo, D. E., Hall, S. M., & Prochaska, J. J. (2011). Reliability and validity of self-reported smoking in an anonymous online survey with young adults. Health Psychology, 30, 693–701.PubMedPubMedCentralCrossRefGoogle Scholar
  117. Rasinski, K. A., Willis, G. B., Baldwin, A. K., Yeh, W., & Lee, L. (1999). Methods of data collection, perceptions of risks and losses, and motivation to give truthful answers to sensitive survey questions. Applied Cognitive Psychology, 13, 465–484.CrossRefGoogle Scholar
  118. Remafedi, G., French, S., Story, M., Resnick, M. D., & Blum, R. (1998). The relationship between suicide risk and sexual orientation: Results of a population-based study. American Journal of Public Health, 88, 57–60.PubMedPubMedCentralCrossRefGoogle Scholar
  119. Rhodes, S. D., Bowie, D. A., & Hergenrather, K. C. (2003). Collecting behavioural data using the World Wide Web: Considerations for researchers. Journal of Epidemiology and Community Health, 57, 68–73.PubMedPubMedCentralCrossRefGoogle Scholar
  120. Roberts, A. L., Austin, S. B., Corliss, H. L., Vandermorris, A. K., & Koenen, K. C. (2010). Pervasive trauma exposure among U.S. sexual orientation minority adults and risk of posttraumatic stress disorder. American Journal of Public Health, 100, 2433–2441.PubMedPubMedCentralCrossRefGoogle Scholar
  121. Russell, S. T., & Joyner, K. (2001). Adolescent sexual orientation and suicide risk: Evidence from a national study. American Journal of Public Health, 91, 1276–1281.PubMedPubMedCentralCrossRefGoogle Scholar
  122. Rust, P. R. (2002). Bisexuality: The state of the union. Annual Review of Sex Research, 13, 180–240.PubMedGoogle Scholar
  123. Ryff, C., Almeida, D. M., Ayanian, J. S., Carr, D. S., Cleary, P. D., Coe, C., … Mroczek, D. K. (2012). National Survey of Midlife Development in the United States (MIDUS II), 2004–2006 (ICPSR04652-v6). Ann Arbor, MI: Inter-University Consortium for Political and Social Research.Google Scholar
  124. Savin-Williams, R. C. (2006). Who’s gay? Does it matter? Current Directions in Psychological Science, 15, 40–44.CrossRefGoogle Scholar
  125. Sell, R. L., & Becker, J. B. (2001). Sexual orientation data collection and progress toward Healthy People 2010. American Journal of Public Health, 91, 876–882.PubMedPubMedCentralCrossRefGoogle Scholar
  126. Sell, R. L., & Holliday, M. L. (2014). Sexual orientation data collection policy in the United States: Public health malpractice. American Journal of Public Health, 104, 967–969.PubMedPubMedCentralCrossRefGoogle Scholar
  127. Singer, E., Von Thurn, D. R., & Miller, E. R. (1995). Confidentiality assurances and response a quantitative review of the experimental literature. Public Opinion Quarterly, 59, 66–77.CrossRefGoogle Scholar
  128. Smith, T. W., Marsden, P. V., & Hout, M. (2015). General social surveys, 1972–2014 [data file]. Retrieved from http://publicdata.norc.org:41000/gss/documents/BOOK/GSS_Codebook.pdf.
  129. Stocké, V., & Hunkler, C. (2007). Measures of desirability beliefs and their validity as indicators for socially desirable responding. Field Methods, 19, 313–336.CrossRefGoogle Scholar
  130. Tourangeau, R., Couper, M. P., & Conrad, F. G. (2013). “Up means good”: The effect of screen position on evaluative ratings in web surveys. Public Opinion Quarterly, 77, 69–88.PubMedPubMedCentralCrossRefGoogle Scholar
  131. Tourangeau, R., Rips, L. J., & Rasinski, K. (2000). The psychology of survey response. Cambridge, MA: Cambridge University Press.CrossRefGoogle Scholar
  132. Tourangeau, R., & Smith, T. W. (1996). Asking sensitive questions the impact of data collection mode, question format, and question context. Public Opinion Quarterly, 60, 275–304.CrossRefGoogle Scholar
  133. Tourangeau, R., & Smith, T. W. (1998). Collecting sensitive information with different modes of data collection. In M. P. Couper, R. P. Baker, J. Bethlehem, C. Z. Clark, J. Martin, W. L. Nicholls II, et al. (Eds.), Computer assisted survey information collection (pp. 431–454). New York, NY: Wiley.Google Scholar
  134. Tourangeau, R., & Yan, T. (2007). Sensitive questions in surveys. Psychological Bulletin, 133, 859–883.PubMedCrossRefGoogle Scholar
  135. Trau, R. N., Härtel, C. E., & Härtel, G. F. (2013). Reaching and hearing the invisible: Organizational research on invisible stigmatized groups via web surveys. British Journal of Management, 24, 532–541.CrossRefGoogle Scholar
  136. Tsuchiya, T., Hirai, Y., & Ono, S. (2007). A study of the properties of the item count technique. Public Opinion Quarterly, 71, 253–272.CrossRefGoogle Scholar
  137. Turner, C. F., Forsyth, B. H., O’Reilly, J. M., Cooley, P. C., Smith, T. K., Rogers, S. M., & Miller, H. G. (1998). Automated self-interviewing and the survey measurement of sensitive behaviors. In M. P. Couper, R. P. Baler, J. Bethlehem, C. Z. Clark, J. Martin, W. Nichols, & J. M. O’Reilly (Eds.), Computer assisted survey information collection (pp. 455–473). New York, NY: Wiley.Google Scholar
  138. Turner, C. F., Ku, L., Sonenstein, F. L., & Pleck, J. H. (1996). Impact of ACASI on reporting of male-male sexual contacts: Preliminary results from the 1995 national survey of adolescent males. In R. B. Warnecke (Ed.), Health survey research methods (pp. 171–176). Hyattsville, MD: National Center for Health Statistics.Google Scholar
  139. Turner, C. F., Lessler, J. T., & Devore, J. (1992). Effects of mode of administration and wording on reporting of drug use. In C. F. Turner, J. T. Lessler, & J. C. Groerer (Eds.), Survey measurement of drug use: Methodological studies (pp. 177–220). Rockville, MD: National Institute on Drug Abuse.Google Scholar
  140. Villarroel, M. A., Turner, C. F., Eggleston, E., Al-Tayyib, A., Rogers, S. M., Roman, A. M., … Gordek, H. (2006). Same-gender sex in the United States impact of T-ACASI on prevalence estimates. Public Opinion Quarterly, 70, 166–196.PubMedPubMedCentralCrossRefGoogle Scholar
  141. Vrangalova, Z., & Savin-Williams, R. C. (2012). Mostly heterosexual and mostly gay/lesbian: Evidence for new sexual orientation identities. Archives of Sexual Behavior, 41, 85–101.PubMedCrossRefGoogle Scholar
  142. Walters, M. L., Chen, J., & Breiding, M. J. (2013). The national intimate partner and sexual violence survey (NISVS): 2010 findings on victimization by sexual orientation. Atlanta, GA: National Center for Injury Prevention and Control, Centers for Disease Control and Prevention.Google Scholar
  143. Ward, B. W., Dahlhamer, J. M., Galinsky, A. M., & Joestl, S. S. (2014). Sexual orientation and health among U.S. adults: National health interview survey, 2013. National Health Statistics Reports (no. 77). Atlanta, GA: Centers for Disease Control and Prevention.Google Scholar
  144. Warner, M. (Ed.). (1993). Fear of a queer planet: Queer politics and social theory (Vol. 6). Minneapolis, MN: University of Minnesota Press.Google Scholar
  145. Wright, K. B. (2005). Researching Internet-based populations: Advantages and disadvantages of online survey research, online questionnaire authoring software packages, and web survey services. Journal of Computer-Mediated Communication. doi: 10.1111/j.1083-6101.2005.tb00259.x.Google Scholar
  146. Wright, D. L., Aquilino, W. S., & Supple, A. J. (1998). A comparison of computer-assisted and paper-and-pencil self-administered questionnaires in a survey on smoking, alcohol, and drug use. Public Opinion Quarterly, 62, 331–353.CrossRefGoogle Scholar
  147. YouGov U.S. (n.d.). YouGov Omnibus. Retrieved from https://today.yougov.com/find-solutions/omnibus/.

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  • Ronald E. Robertson
    • 1
  • Felix W. Tran
    • 1
  • Lauren N. Lewark
    • 1
  • Robert Epstein
    • 1
  1. 1.American Institute for Behavioral Research and TechnologyVistaUSA

Personalised recommendations