Skip to main content

Advertisement

SpringerLink
Log in

Evaluating HIV Knowledge Questionnaires Among Men Who Have Sex with Men: A Multi-Study Item Response Theory Analysis

  • Original Paper
  • Published:
Archives of Sexual Behavior Aims and scope Submit manuscript

Abstract

Knowledge about the transmission, prevention, and treatment of HIV remains a critical element in psychosocial models of HIV risk behavior and is commonly used as an outcome in HIV prevention interventions. However, most HIV knowledge questions have not undergone rigorous psychometric testing such as using item response theory. The current study used data from six studies of men who have sex with men (MSM; n = 3565) to (1) examine the item properties of HIV knowledge questions, (2) test for differential item functioning on commonly studied characteristics (i.e., age, race/ethnicity, and HIV risk behavior), (3) select items with the optimal item characteristics, and (4) leverage this combined dataset to examine the potential moderating effect of age on the relationship between condomless anal sex (CAS) and HIV knowledge. Findings indicated that existing questions tend to poorly differentiate those with higher levels of HIV knowledge, but items were relatively robust across diverse individuals. Furthermore, age moderated the relationship between CAS and HIV knowledge with older MSM having the strongest association. These findings suggest that additional items are required in order to capture a more nuanced understanding of HIV knowledge and that the association between CAS and HIV knowledge may vary by age.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. The TRACK study is the single study that did not fit into this categorization because only prospective measurement of CAS was collected. Accordingly, in order to avoid losing data, TRACK participants were coded as engaging in CAS if they reported that behavior in the 6 months after completing the HIV Knowledge questionnaire.

  2. The relationships between the full scale (i.e., 38 items) and each of the substantive independent variables, including the interactions terms, is identical to that of the 17-item reduced scales.

References

  • Aaro, L. E., Breivik, K., Klepp, K. I., Kaaya, S., Onya, H. E., Wubs, A., … Flisher, A. J. (2011). An HIV/AIDS knowledge scale for adolescents: Item response theory analyses based on data from a study in South Africa and Tanzania. Health Education Research, 26(2), 212–224. doi:10.1093/her/cyq086.

    Article  PubMed  Google Scholar 

  • Adam, P. C., de Wit, J. B., Toskin, I., Mathers, B. M., Nashkhoev, M., Zablotska, I., … Rugg, D. (2009). Estimating levels of HIV testing, HIV prevention coverage, HIV knowledge, and condom use among men who have sex with men (MSM) in low-income and middle-income countries. Journal of Acquired Immune Deficiency Syndromes, 52(Suppl. 2), S143–S151. doi:10.1097/QAI.0b013e3181baf111.

    Article  PubMed  Google Scholar 

  • Allison, P. D. (2003). Missing data techniques for structural equation modeling. Journal of Abnormal Psychology, 112(4), 545–557. doi:10.1037/0021-843x.112.4.545.

    Article  PubMed  Google Scholar 

  • Bauer, D. J., & Hussong, A. M. (2009). Psychometric approaches for developing commensurate measures across independent studies: Traditional and new models. Psychological Methods, 14(2), 101–125. doi:10.1037/a0015583.

    Article  PubMed  PubMed Central  Google Scholar 

  • Bazargan, M., Kelly, E. M., Stein, J. A., Husaini, B. A., & Bazargan, S. H. (2000). Correlates of HIV risk-taking behaviors among African-American college students: The effect of HIV knowledge, motivation, and behavioral skills. Journal of the National Medical Association, 92(8), 391–404.

    PubMed  PubMed Central  Google Scholar 

  • Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate—A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society Series B-Methodological, 57(1), 289–300.

    Google Scholar 

  • Burke, H. M., Fleming, P. J., & Guest, G. (2014). Assessment of the psychometric properties of HIV knowledge items across five countries. AIDS Education and Prevention, 26(6), 577–587. doi:10.1521/aeap.2014.26.6.577.

    Article  PubMed  Google Scholar 

  • Carey, M. P., Morrison-Beedy, D., & Johnson, B. T. (1997). The HIV-knowledge questionnaire: Development and evaluation of a reliable, valid, and practical self-administered questionnaire. AIDS and Behavior, 1(1), 61–74. doi:10.1023/a:1026218005943.

    Article  Google Scholar 

  • Carey, M. P., & Schroder, K. E. (2002). Development and psychometric evaluation of the brief HIV Knowledge Questionnaire. AIDS Education and Prevention, 14(2), 172–182.

    Article  PubMed  PubMed Central  Google Scholar 

  • Centers for Disease Control. (2016). HIV surveillance—Men who have sex with men (MSM). Retrieved from http://www.cdc.gov/hiv/library/slideSets/index.html.

  • Centers for Disease Control and Prevention. (2015). Diagnoses of HIV infection in the United States and dependent areas, 2011. HIV Surveillance Report, 2013, 23.

    Google Scholar 

  • Djokic, D., Englund, J., Daum, R., Martin, R., Dozier, T., Potts, S., … Marcinak, J. (2009). HIV knowledge and attitudes toward HIV testing of South Side Chicago Housing Authority residents. AIDS Patient Care and STDS, 23(1), 23–28. doi:10.1089/apc.2008.0110.

    Article  PubMed  Google Scholar 

  • Ebrahim, S. H., Anderson, J. E., Weidle, P., & Purcell, D. W. (2004). Race/ethnic disparities in HIV testing and knowledge about treatment for HIV/AIDS: United States, 2001. AIDS Patient Care and STDS, 18(1), 27–33. doi:10.1089/108729104322740893.

    Article  PubMed  Google Scholar 

  • Eggers, S. M., Aaro, L. E., Bos, A. E. R., Mathews, C., & de Vries, H. (2014). Predicting condom use in South Africa: A test of two integrative models. AIDS and Behavior, 18(1), 135–145. doi:10.1007/s10461-013-0423-2.

    Article  PubMed  Google Scholar 

  • Ferguson, E., Cox, T., Irving, K., Leiter, M., & Farnsworth, B. (1995). A measure of knowledge and confidence in relation to HIV and AIDS—Reliability and validity. AIDS Care, 7(2), 211–228. doi:10.1080/09540129550126722.

    Article  PubMed  Google Scholar 

  • Finch, H. (2005). The MIMIC model as a method for detecting DIF comparison with Mantel-Haenszel, SIBTEST, and the IRT likelihood ratio. Applied Psychological Measurement, 29(4), 278–295. doi:10.1177/0146621605275728.

    Article  Google Scholar 

  • Fisher, J. D., & Fisher, W. A. (1992). Changing AIDS-risk behavior. Psychological Bulletin, 111(3), 455–474.

    Article  PubMed  Google Scholar 

  • Garofalo, R., Gayles, T., Bottone, P. D., Ryan, D., Kuhns, L. M., & Mustanski, B. (2015). Racial/ethnic difference in HIV-related knowledge among young men who have sex with men and their association with condom errors. Health Education Journal, 74(5), 518–530. doi:10.1177/0017896914549485.

    Article  PubMed  Google Scholar 

  • Glockner-Rist, A., & Hoijtink, H. (2003). The best of both worlds: Factor analysis of dichotomous data using item response theory and structural equation modeling. Structural Equation Modeling, 10(4), 544–565.

    Article  Google Scholar 

  • Goetz, C., Coste, J., Lemetayer, F., Rat, A. C., Montel, S., Recchia, S., … Guillemin, F. (2013). Item reduction based on rigorous methodological guidelines is necessary to maintain validity when shortening composite measurement scales. Journal of Clinical Epidemiology, 66(7), 710–718. doi:10.1016/j.jclinepi.2012.12.015.

    Article  PubMed  Google Scholar 

  • Gomes, R Rd F M, Batista, J. R., Ceccato, Md G B, Kerr, L. R. F. S., & Guimarães, M. D. C. (2014). HIV/AIDS knowledge among men who have sex with men: Applying the item response theory. Revista de Saúde Publica, 48(2), 206–215.

    Article  PubMed  PubMed Central  Google Scholar 

  • Hambleton, R. K. (2006). Good practices for identifying differential item functioning—Commentary. Medical Care, 44(11), S182–S188. doi:10.1097/01.mlr.0000245443.86671.c4.

    Article  PubMed  Google Scholar 

  • Herbst, J. H., Sherba, R. T., Crepaz, N., Deluca, J. B., Zohrabyan, L., Stall, R. D., & Lyles, C. M. (2005). A meta-analytic review of HIV behavioral interventions for reducing sexual risk behavior of men who have sex with men. Journal of Acquired Immune Deficiency Syndromes, 39(2), 228–241.

    PubMed  Google Scholar 

  • Hughes, A. K., & Admiraal, K. R. (2012). A systematic review of HIV/AIDS knowledge measures. Research on Social Work Practice, 22(3), 313–322. doi:10.1177/1049731511426277.

    Article  Google Scholar 

  • Jin, F. Y., Prestage, G. P., Mao, L. M., Poynten, I. M., Templeton, D. J., Grulich, A. E., & Zablotska, I. (2015). “Any condomless anal intercourse” is no longer an accurate measure of HIV sexual risk behavior in gay and other men who have sex with men. Frontiers in Immunology, 6, 86. doi:10.3389/fimmu.2015.00086.

    Article  PubMed  PubMed Central  Google Scholar 

  • Jones, R. N. (2006). Identification of measurement differences between English and Spanish language versions of the Mini-Mental State Examination—Detecting differential item functioning using MIMIC modeling. Medical Care, 44(11), S124–S133. doi:10.1097/01.mlr.0000245250.50114.0f.

    Article  PubMed  Google Scholar 

  • Kalichman, S. C., Picciano, J. F., & Roffman, R. A. (2008). Motivation to reduce HIV risk behaviors in the context of the information, motivation and behavioral skills (IMB) model of HIV prevention. Journal of Health Psychology, 13(5), 680–689. doi:10.1177/1359105307082456.

    Article  PubMed  Google Scholar 

  • McCown, W., & Johnson, J. (1991). The Basic HIV Disease Knowledge Questionnaire: A Rasch-scaled instrument to measure essential HIV knowledge. Psychological Reports, 69(2), 543–549. doi:10.2466/Pr0.69.6.543-549.

    Article  PubMed  Google Scholar 

  • Medley, A., Kennedy, C., O’Reilly, K., & Sweat, M. (2009). Effectiveness of peer education interventions for HIV prevention in developing countries: A systematic review and meta-analysis. AIDS Education and Prevention, 21(3), 181–206.

    Article  PubMed  PubMed Central  Google Scholar 

  • Mustanski, B., DuBois, L. Z., Prescott, T. L., & Ybarra, M. L. (2014a). A mixed-methods study of condom use and decision making among adolescent gay and bisexual males. AIDS and Behavior, 18(10), 1955–1969. doi:10.1007/s10461-014-0810-3.

    Article  PubMed  PubMed Central  Google Scholar 

  • Mustanski, B., Garofalo, R., Monahan, C., Gratzer, B., & Andrews, R. (2013a). Feasibility, acceptability, and preliminary efficacy of an online HIV prevention program for diverse young men who have sex with men: The Keep it Up! intervention. AIDS and Behavior, 17(9), 2999–3012. doi:10.1007/s10461-013-0507-z.

    Article  PubMed  Google Scholar 

  • Mustanski, B., Johnson, A. K., Garofalo, R., Ryan, D., & Birkett, M. (2013b). Perceived likelihood of using HIV pre-exposure prophylaxis medications among young men who have sex with men. AIDS and Behavior, 17(6), 2173–2179. doi:10.1007/s10461-012-0359-y.

    Article  PubMed  PubMed Central  Google Scholar 

  • Mustanski, B., Lyons, T., & Garcia, S. C. (2011). Internet use and sexual health of young men who have sex with men: A mixed-methods study. Archives of Sexual Behavior, 40(2), 289–300. doi:10.1007/s10508-009-9596-1.

    Article  PubMed  Google Scholar 

  • Mustanski, B., Ryan, D. T., & Garofalo, R. (2014b). Associations of sexually transmitted infections with condom problems among young men who have sex with men. Sexually Transmitted Diseases, 41(7), 427–432. doi:10.1097/OLQ.0000000000000150.

    Article  PubMed  PubMed Central  Google Scholar 

  • Muthen, B. (1984). A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators. Psychometrika, 49(1), 115–132. doi:10.1007/Bf02294210.

    Article  Google Scholar 

  • Nelson, L. E., Wilton, L., Agyarko-Poku, T., Zhang, N. H., Aluoch, M., Thach, C. T., … Adu-Sarkodie, Y. (2015). The association of HIV stigma and HIV/STD knowledge with sexual risk behaviors among adolescent and adult men who have sex with men in Ghana, West Africa. Research in Nursing & Health, 38(3), 194–206. doi:10.1002/nur.21650.

    Article  Google Scholar 

  • Newcomb, M. E. (2013). Moderating effect of age on the association between alcohol use and sexual risk in MSM: Evidence for elevated risk among younger MSM. AIDS and Behavior, 17(5), 1746–1754. doi:10.1007/s10461-013-0470-8.

    Article  PubMed  PubMed Central  Google Scholar 

  • Newcomb, M. E., & Mustanski, B. (2014a). Cognitive influences on sexual risk and risk appraisals in men who have sex with men. Health Psychology, 33(7), 690–698. doi:10.1037/hea0000010.

    Article  PubMed  Google Scholar 

  • Newcomb, M. E., & Mustanski, B. (2014b). Developmental change in the relationship between alcohol and drug use before sex and sexual risk behavior in young men who have sex with men. AIDS and Behavior, 18(10), 1981–1990. doi:10.1007/s10461-014-0766-3.

    Article  PubMed  PubMed Central  Google Scholar 

  • Oglesby, W. H., & Alemagno, S. A. (2013). Psychometric properties of an HIV knowledge scale administered with populations at high risk for HIV infection. Health Promotion Practice, 14(6), 859–867. doi:10.1177/1524839912470453.

    Article  PubMed  Google Scholar 

  • Peltzer, K., Matseke, G., Mzolo, T., & Majaja, M. (2009). Determinants of knowledge of HIV status in South Africa: Results from a population-based HIV survey. BMC Public Health, 9, 174. doi:10.1186/1471-2458-9-174.

    Article  PubMed  PubMed Central  Google Scholar 

  • Prejean, J., Song, R., Hernandez, A., Ziebell, R., Green, T., Walker, F., … HIV Incidence Surveillance Group. (2011). Estimated HIV incidence in the United States, 2006–2009. PLoS ONE, 6(8), e17502. doi:10.1371/journal.pone.0017502.

    Article  PubMed  PubMed Central  Google Scholar 

  • Scott-Sheldon, L. A. J., Carey, M. P., Vanable, P. A., Senn, T. E., Coury-Doniger, P., & Urban, M. A. (2010). Predicting condom use among STD clinic patients using the information-motivation-behavioral skills (IMB) model. Journal of Health Psychology, 15(7), 1093–1102. doi:10.1177/1359105310364174.

    Article  PubMed  PubMed Central  Google Scholar 

  • Skinta, M. D., Murphy, J. L., Paul, J. P., Schwarcz, S. K., & Dilley, J. W. (2012). Thoughts, attitudes, and feelings of HIV-positive MSM associated with high transmission-risk sex. Health Education & Behavior, 39(3), 315–323. doi:10.1177/1090198111427390.

    Article  Google Scholar 

  • Strauss, B. B., Greene, G. J., Philips, G. L., II, Bhatia, R., Madkins, K., Parsons, J. T., & Mustanski, B. (2016). Exploring patterns of awareness and use of HIV pre-exposure prophylaxis among young men who have sex with men. AIDS and Behavior. doi:10.1007/s10461-016-1480-0.

    Article  PubMed  Google Scholar 

  • Sullivan, P. S., Peterson, J., Rosenberg, E. S., Kelley, C. F., Cooper, H., Vaughan, A., … Sanchez, T. H. (2014). Understanding racial HIV/STI disparities in black and white men who have sex with men: A multilevel approach. PLoS ONE, 9(3), e90514. doi:10.1371/journal.pone.0090514.

    Article  PubMed  PubMed Central  Google Scholar 

  • Sullivan, P. S., Rosenberg, E. S., Sanchez, T. H., Kelley, C. F., Luisi, N., Cooper, H. L., … del Rio, C. (2015). Explaining racial disparities in HIV incidence in black and white men who have sex with men in Atlanta, GA: A prospective observational cohort study. Annals of Epidemiology, 25, 445–454.

    Article  PubMed  PubMed Central  Google Scholar 

  • Teresi, J. A. (2006). Overview of quantitative measurement methods—Equivalence, invariance, and differential item functioning in health applications. Medical Care, 44(11), S39–S49. doi:10.1097/01.mlr.0000245452.48613.45.

    Article  PubMed  Google Scholar 

  • Thissen, D., Steinberg, L., & Kuang, D. (2002). Quick and easy implementation of the Benjamini–Hochberg procedure for controlling the false positive rate in multiple comparisons. Journal of Educational and Behavioral Statistics, 27(1), 77–83. doi:10.3102/10769986027001077.

    Article  Google Scholar 

  • Villar-Loubet, O. M., Cook, R., Chakhtoura, N., Peltzer, K., Weiss, S. M., Shikwane, M. E., & Jones, D. L. (2013). HIV knowledge and sexual risk behavior among pregnant couples in South Africa: The PartnerPlus project. AIDS and Behavior, 17(2), 479–487. doi:10.1007/s10461-012-0360-5.

    Article  PubMed  PubMed Central  Google Scholar 

  • Wagenaar, B. H., Sullivan, P. S., & Stephenson, R. (2012). HIV knowledge and associated factors among internet-using men who have sex with men (MSM) in South Africa and the United States. PLoS ONE, 7(3), e32915. doi:10.1371/journal.pone.0032915.

    Article  PubMed  PubMed Central  Google Scholar 

  • Walsh, J. L., Senn, T. E., Scott-Sheldon, L. A. J., Vanable, P. A., & Carey, M. P. (2011). Predicting condom use using the information-motivation-behavioral skills (IMB) model: A multivariate latent growth curve analysis. Annals of Behavioral Medicine, 42(2), 235–244. doi:10.1007/s12160-011-9284-y.

    Article  PubMed  PubMed Central  Google Scholar 

  • Wang, W. C., Shih, C. L., & Yang, C. C. (2009). The MIMIC method with scale purification for detecting differential item functioning. Educational and Psychological Measurement, 69(5), 713–731. doi:10.1177/0013164409332228.

    Article  Google Scholar 

  • Zumbo, B. D. (2007). Three generations of DIF analyses: Considering where it has been, where it is now, and where it is going. Language Assessment Quarterly, 4(2), 223–233.

    Article  Google Scholar 

Download references

Funding

This work was supported by Grant R01DA035145 from the National Institute of Mental Health and National Institute of Drug Abuse.

Author information

Authors and Affiliations

  1. Department of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, 625 N. Michigan Ave., Suite 2700, Chicago, IL, 60611, USA

    Patrick Janulis, Michael E. Newcomb & Brian Mustanski

  2. Department of Epidemiology, Rollins School of Public Health, Emory University, Atlanta, GA, USA

    Patrick Sullivan

Authors
  1. Patrick Janulis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael E. Newcomb
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Patrick Sullivan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Brian Mustanski
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Patrick Janulis.

Ethics declarations

Conflict of interest

Patrick Janulis has no conflict of interest to declare. Michael E. Newcomb has no conflict of interest to declare. Patrick Sullivan has no conflict of interest to declare. Brian Mustanski has no conflict of interest to declare.

Ethical Approval

All procedures performed in this study were in accordance with the ethical standards and approved by our institutional review board.

Informed Consent

Informed consent was obtained from all participants in this study.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Janulis, P., Newcomb, M.E., Sullivan, P. et al. Evaluating HIV Knowledge Questionnaires Among Men Who Have Sex with Men: A Multi-Study Item Response Theory Analysis. Arch Sex Behav 47, 107–119 (2018). https://doi.org/10.1007/s10508-016-0910-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10508-016-0910-4

Keywords

Search

Navigation