Sex Roles

, Volume 64, Issue 3–4, pp 276–289

Content Analysis—A Methodological Primer for Gender Research

Original Article

Abstract

This article is intended to serve as a primer on methodological standards for gender scholars pursuing content analytic research. The scientific underpinnings of the method are explored, including the roles of theory, past research, population definition, objectivity/intersubjectivity, reliability, validity, generalizability, and replicability. Both human coding and computer coding are considered. The typical process of human-coded content analysis is reviewed, including the steps of unitizing, sampling, measurement, coder training, reliability assessment, and reportage of methods. Numerous applications to research on gender roles and related issues are reviewed. Practical checklists are offered for content analysis preparation and methodological execution.

Keywords

Content analysis Methodological standards Gender research Gender roles 

References

  1. An, D., & Kim, S. (2007). Relating Hofstede’s masculinity dimension to gender role portrayals in advertising. International Marketing Review, 24, 181–207.Google Scholar
  2. Anderson, D. A., & Hamilton, M. (2005). Gender role stereotyping of parents in children’s picture books: The invisible father. Sex Roles, 52, 145–151.Google Scholar
  3. Argamon, S., Koppel, M., Fine, J., & Shimoni, A. R. (2003). Gender, genre, and writing style in formal written texts. Retrieved from http://lingcog.iit.edu/doc/gendertext04.pdf
  4. Babbie, E. R. (2010). The practice of social research (12th ed.). Belmont: Wadsworth Cengage.Google Scholar
  5. Baker, C. N. (2005). Images of women’s sexuality in advertisements: A content analysis of black- and white-oriented women’s and men’s magazines. Sex Roles, 52, 13–27.Google Scholar
  6. Balmas, M., & Scheafer, T. (2010). Candidate image in election campaigns: Attribute agenda setting, affective priming, and voting intentions. International Journal of Public Opinion Research, 22(2), 204–229.Google Scholar
  7. Banerjee, M., Capozzoli, M., McSweeney, L., & Sinha, D. (1999). Beyond kappa: A review of interrater agreement measures. Canadian Journal of Statistics, 27(1), 3–23.Google Scholar
  8. Bem, S. (1981). Bem sex role inventory professional manual. Palo Alto: Consulting Psychologists Press.Google Scholar
  9. Bridges, J. S. (1993). Pink or blue: Gender-stereotypic perceptions of infants as conveyed by birth congratulatory cards. Psychology of Women Quarterly, 17(2), 193–205.Google Scholar
  10. Brinson, S. L., & Winn, J. E. (1997). Talk shows’ representations of interpersonal conflicts. Journal of Broadcasting & Electronic Media, 41, 25–39.Google Scholar
  11. Carmines, E. G., & Zeller, R. A. (1979). Reliability and validity assessment. Newbury Park: Sage.Google Scholar
  12. Cissna, K. N., Garvin, B. J., & Kennedy, C. W. (1990). Reliability in coding social interaction: A study of confirmation. Communication Reports, 3(2), 58–69.Google Scholar
  13. Clarke, J. N., & Everest, M. M. (2006). Cancer in the mass print media: Fear, uncertainty and the medical model. Social Science & Medicine, 62, 2591–2600.Google Scholar
  14. Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46.Google Scholar
  15. Collins, R. L., Elliott, M. N., & Miu, A. (2009). Linking media content to media effects: The RAND Television and Adolescent Sexuality Study. In A. B. Jordan, D. Kunkel, J. Manganello, & M. Fishbein (Eds.), Media messages and public health: A decisions approach to content analysis (pp. 154–172). New York: Routledge.Google Scholar
  16. Cressman, D. L., Callister, M., Robinson, T., & Near, C. (2009). An analysis of profanity in US teen-oriented movies, 1980-2006. Journal of Children and Media, 3(2), 117–134.Google Scholar
  17. Curtin, P., & Gaither, K. (2003). Public relations and propaganda in cyberspace: A quantitative content analysis of Middle Eastern government websites. Paper presented at the annual meeting of the International Communication Association, San Diego, CA.Google Scholar
  18. Danaher, B. G., Boles, S. M., Akers, L., Gordon, J. S., & Severson, H. H. (2006). Defining participant exposure measures in web-based health behavior change programs. Journal of Medical Internet Research, 8, Article 3.Google Scholar
  19. Dietz, T. L. (1998). An examination of violence and gender role portrayals in video games: Implications for gender socialization and aggressive behavior. Sex Roles, 38, 425–442.Google Scholar
  20. Domhoff, G. W. (1999). New directions in the study of dream content using the Hall and Van de Castle coding system. Dreaming, 9, 115–137.Google Scholar
  21. Dozier, D. M., Lauzen, M. M., Day, C. A., Payne, S. M., & Tafoya, M. R. (2005). Leaders and elites: Portrayals of smoking in popular films. Tobacco Control, 14(1), 7–9.PubMedGoogle Scholar
  22. Ebersole, S. (2000). Uses and gratifications of the web among students. Journal of Computer-Mediated Communication, 6(1). Retrieved from http://jcmc.indiana.edu/vol6/issue1/ebersole.html
  23. Eco, U. (1976). A theory of semiotics. Bloomington: Indiana University Press.Google Scholar
  24. Eschholz, S., Bufkin, J., & Long, J. (2002). Symbolic reality bites: Women and racial/ethnic minorities in modern film. Sociological Spectrum, 22, 299–334.Google Scholar
  25. Evans, L., & Davies, K. (2000). No sissy boys here: A content analysis of the representation of masculinity in elementary school reading textbooks. Sex Roles, 42, 255–270.Google Scholar
  26. Fernandez-Villanueva, C., Revilla-Castro, J. C., Dominguez-Bilbao, R., Gimeno-Jimenez, L., & Almagro, A. (2009). Gender differences in the representation of violence on Spanish television: Should women be more violent? Sex Roles, 61, 85–100.Google Scholar
  27. Fields, A. M., Swan, S., & Kloos, B. (2010). “What it means to be a woman:” Ambivalent sexism in female college students’ experiences and attitudes. Sex Roles, 62, 554–567.Google Scholar
  28. Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76, 378–382.Google Scholar
  29. Fleiss, J. L. (1981). Statistical methods for rates and proportions. New York: Wiley.Google Scholar
  30. Fleiss, J. L., Levin, B., & Paik, M. C. (2003). Statistical methods for rates and proportions (3rd ed.). Hoboken: Wiley-Interscience.Google Scholar
  31. Franiuk, R., Seefelt, J. L., & Vandello, J. A. (2008). Prevalence of rape myths in headlines and their effects on attitudes toward rape. Sex Roles, 58, 790–801.Google Scholar
  32. Fullerton, J. A., & Kendrick, A. (2000). Portrayal of men and women in U.S. Spanish-language television commercials. J&MC Quarterly, 77, 128–142.Google Scholar
  33. Garvin, B. J., Kennedy, C. W., & Cissna, K. N. (1988). Reliability in category coding systems. Nursing Research, 37(1), 52–58.PubMedGoogle Scholar
  34. George, A. L. (1959). Propaganda analysis: A study of inferences made from Nazi propaganda in World War II. Westport: Greenwood.Google Scholar
  35. Ghose, S., & Dou, W. (1998). Interactive functions and their impacts on the appeal of Internet presence sites. Journal of Advertising Research, 38(2), 29–43.Google Scholar
  36. Goffman, E. (1979). Gender advertisements. New York: Harper and Row.Google Scholar
  37. Goodwin, L. D. (2001). Interrater agreement and reliability. Measurement in Physical Education and Exercise Science, 5(1), 13–34.Google Scholar
  38. Gottschalk, L. A., & Bechtel, R. J. (Eds.). (2008). Computerized content analysis of speech and verbal texts and its many applications. New York: Nova Science.Google Scholar
  39. Gray, J. H., & Densten, I. L. (1998). Integrating quantitative and qualitative analysis using latent and manifest variables. Quality & Quantity, 32, 419–431.Google Scholar
  40. Gregory, R. L., & Zangwill, O. L. (Eds.). (1987). The Oxford companion to the mind. Oxford: Oxford University Press.Google Scholar
  41. Groom, C. J., & Pennebaker, J. W. (2005). The language of love: Sex, sexual orientation, and language use in online personal advertisements. Sex Roles, 52, 447–461.Google Scholar
  42. Guetzkow, H. (1950). Unitizing and categorizing problems in coding qualitative data. Journal of Clinical Psychology, 6, 47–58.Google Scholar
  43. Haninger, K., & Thompson, K. M. (2004). Content and ratings of teen-rated video games. JAMA, 291(7), 856–865.PubMedGoogle Scholar
  44. Hardy, C., Harley, B., & Phillips, N. (2004). Discourse analysis and content analysis: Two solitudes? Qualitative methods: Newsletter of the American Political Science Association Organized Section on Qualitative Methods, 2(1), 19–22.Google Scholar
  45. Harrison, K., & Hefner, V. (2006). Media exposure, current and future body ideals, and disordered eating among preadolescent girls: A longitudinal panel study. Journal of Youth and Adolescence, 35, 153–163.Google Scholar
  46. Hart, R. P. (2000). The text-analysis program: Diction 5.0. Austin: Digitext.Google Scholar
  47. Hayes, A. F., & Krippendorff, K. (2007). Answering the call for a standard reliability measure for coding data. Communication Methods and Measures, 1(1), 77–89.Google Scholar
  48. Hofstede, G. (1980). Culture’s consequences: International differences in work-related values. Beverly Hills: Sage.Google Scholar
  49. Hubbell, A. P., & Dearing, J. W. (2003). Local newspapers, community partnerships, and health improvement projects: Their roles in a comprehensive community initiative. Journal of Community Health, 28, 363–376.PubMedGoogle Scholar
  50. Ibroscheva, E. (2007). Caught between East and West? Portrayals of gender in Bulgarian television advertisements. Sex Roles, 57, 409–418.Google Scholar
  51. Janis, I. (1965). The problem of validating content analysis. In H. D. Lasswell, N. Leites, et al. (Eds.), Language of politics (pp. 55–82). Cambridge: MIT.Google Scholar
  52. Jansen, B. J., & Spink, A. (2006). How are we searching the World Wide Web? A comparison of nine search engine transaction logs. Information Processing & Management, 42(1), 248–263.Google Scholar
  53. Janstova, P., Neuendorf, K. A., & Lieberman, E. A. (2010). Empirical testing of auteur theory via content analysis. Manuscript in preparation, Cleveland State University.Google Scholar
  54. Johnston, C. A. B., & Morrison, T. G. (2007). The presentation of masculinity in everyday life: Contextual variations in the masculine behaviour of young Irish men. Sex Roles, 57, 661–674.Google Scholar
  55. Jones, J., & Himelboim, I. (2010). Just a guy in pajamas? Framing the blogs in mainstream U.S. newspaper coverage (1999-2005). New Media & Society, 12, 271–288.Google Scholar
  56. Kalis, P., & Neuendorf, K. A. (1989). Aggressive cue prominence and gender participation in MTV. Journalism Quarterly, 66, 148–154, 229.Google Scholar
  57. Kinney, N. T. (2005). Engaging in “loose talk”: Analyzing salience in discourse from the formulation of welfare policy. Policy Sciences, 38, 251–268.Google Scholar
  58. Knobloch, L. K. (2008). The content of relational uncertainty within marriage. Journal of Social and Personal Relationships, 25, 467–495.Google Scholar
  59. Kolbe, R. H., & Burnett, M. S. (1991). Content-analysis research: An examination of application with directives for improving research reliability and objectivity. Journal of Consumer Research, 18, 243–250.Google Scholar
  60. Krippendorff, K. (2004). Content analysis: An introduction to its methodology (2nd ed.). Thousand Oaks: Sage.Google Scholar
  61. Kunkel, D., Wilson, B., Donnerstein, E., Linz, D., Smith, S., Gray, T., et al. (1995). Measuring television violence: The importance of context. Journal of Broadcasting & Electronic Media, 39, 284–291.Google Scholar
  62. Laird, L. D., de Marrais, J., & Barnes, L. L. (2007). Portraying Islam and Muslims in MEDLINE: A content analysis. Social Science & Medicine, 65, 2425–2439.Google Scholar
  63. Lauzen, M. M., Dozier, D. M., & Cleveland, E. (2006). Genre matters: An examination of women working behind the scenes and on-screen portrayals in reality and scripted prime-time programming. Sex Roles, 55, 445–455.Google Scholar
  64. Lieberman, E. A., Neuendorf, K. A., Denny, J., Skalski, P. D., & Wang, J. (2009). The language of laughter: A quantitative/qualitative fusion examining television narrative and humor. Journal of Broadcasting & Electronic Media, 53, 497–514.Google Scholar
  65. Lin, L. I. (1989). A concordance correlation coefficient to evaluate reproducibility. Biometrics, 45, 255–268.PubMedGoogle Scholar
  66. Lindner, K. (2004). Images of women in general interest and fashion magazine advertisements from 1955 to 2002. Sex Roles, 51, 409–421.Google Scholar
  67. Lombard, M., Snyder-Duch, J., & Bracken, C. C. (2002). Content analysis in mass communication: Assessment and reporting of intercoder reliability. Human Communication Research, 28, 587–604.Google Scholar
  68. Lombard, M., Snyder-Duch, J., & Bracken, C. C. (2004). A call for standardization in content analysis reliability. Human Communication Research, 30, 434–437.Google Scholar
  69. Markson, E. W., & Taylor, C. A. (2000). The mirror has two faces. Ageing and Society, 20, 137–160.Google Scholar
  70. Martins, N., Williams, D. C., Harrison, K., & Ratan, R. A. (2009). A content analysis of female body imagery in video games. Sex Roles, 61, 824–836.Google Scholar
  71. Mastro, D. E., Eastin, M. S., & Tamborini, R. (2002). Internet search behaviors and mood alterations: A selective exposure approach. Media Psychology, 4, 157–172.Google Scholar
  72. McAdams, D. P., & Zeldow, P. B. (1993). Construct validity and content analysis. Journal of Personality Assessment, 61, 243–245.PubMedGoogle Scholar
  73. McCroskey, J. C. (1993). An introduction to rhetorical communication (6th ed.). Englewood Cliffs: Prentice Hall.Google Scholar
  74. McMillan, S. J. (2000). The microscope and the moving target: The challenge of applying content analysis to the World Wide Web. Journalism and Mass Communication Quarterly, 77, 80–98.Google Scholar
  75. Messineo, M. J. (2008). Does advertising on Black Entertainment Television portray more positive gender representations compared to broadcast networks? Sex Roles, 59, 752–764.Google Scholar
  76. Milburn, S. S., Carney, D. R., & Ramirez, A. M. (2001). Even in modern media, the picture is still the same: A content analysis of clipart images. Sex Roles, 44, 277–294.Google Scholar
  77. Miller, M. K., & Summers, A. (2007). Gender differences in video game characters’ roles, appearances, and attire as portrayed in video game magazines. Sex Roles, 57, 733–742.Google Scholar
  78. Mooney, C. Z. (1997). Monte Carlo simulation. Thousands Oaks: Sage.Google Scholar
  79. Neuendorf, K. A. (2002). The content analysis guidebook. Thousand Oaks: Sage.Google Scholar
  80. Neuendorf, K. A. (2004). Content analysis: A contrast and complement to discourse analysis. Qualitative methods: Newsletter of the American Political Science Association Organized Section on Qualitative Methods, 2(1), 33–36.Google Scholar
  81. Neuendorf, K. A. (2009). Reliability for content analysis. In A. B. Jordan, D. Kunkel, J. Manganello, & M. Fishbein (Eds.), Media messages and public health: A decisions approach to content analysis (pp. 67–87). New York: Routledge.Google Scholar
  82. Neuendorf, K. A., & Kane, C. L. (2010). The content analysis guidebook online. Retrieved from http://academic.csuohio.edu/kneuendorf/content
  83. Neuendorf, K. A., & Skalski, P. D. (2010). Content analysis: Description, prediction, and explanation. Paper presented to the Social Science and Social Computing Workshop, University of Hawaii, Honolulu, HI.Google Scholar
  84. Neuendorf, K. A., Gore, T. D., Dalessandro, A., Janstova, P., & Snyder-Suhy, S. (2010). Shaken and stirred: A content analysis of women’s portrayals in James Bond films. Sex Roles, 62, 747–761.Google Scholar
  85. Norris, P. (2003). Preaching to the converted? Pluralism, participation and party websites. Party Politics, 9(1), 21–45.Google Scholar
  86. Odekerken-Schroder, G., De Wulf, K., & Hofstee, N. (2002). Is gender stereotyping in advertising more prevalent in masculine countries? International Marketing Review, 19, 408–419.Google Scholar
  87. Ogletree, S. M., Merritt, S., & Roberts, J. (1994). Female/male portrayals on U.S. postage stamps of the twentieth century. Communication Research Reports, 11, 77–85.Google Scholar
  88. Pan, P.-L., Meng, J., & Zhou, S. (2010). Morality or equality? Ideological framing in news coverage of gay marriage legitimization. The Social Science Journal, 47, 630–645.Google Scholar
  89. Pasadeos, Y., Huhman, B., Standley, T., & Wilson, G. (1995). Applications of content analysis in news research: A critical examination. Paper presented to the Communication Theory and Methodology Division of the Association for Education in Journalism and Mass Communication, Washington, DC.Google Scholar
  90. Patchin, J. W., & Hinduja, S. (2010). Trends in online social networking: Adolescent use of MySpace over time. New Media & Society, 12(2), 197–216.Google Scholar
  91. Pennebaker, J. W., Booth, R. J., & Francis, M. E. (2007). Linguistic inquiry and word count (LIWC2007). Austin, TX: www.licw.net.
  92. Petersson, H., Gill, H., & Ahlfeldt, H. (2002). A variance-based measure of inter-rater agreement in medical databases. Journal of Biomedical Informatics, 35, 331–342.PubMedGoogle Scholar
  93. Pollock, J. C., & Yulis, S. G. (2004). Nationwide newspaper coverage of physician-assisted suicide: A community structure approach. Journal of Health Communication, 9, 281–307.PubMedGoogle Scholar
  94. Potter, W. J. (2009). Defining and measuring key content variables. In A. B. Jordan, D. Kunkel, J. Manganello, & M. Fishbein (Eds.), Media messages and public health: A decisions approach to content analysis (pp. 35–52). New York: Routledge.Google Scholar
  95. Potter, W. J., & Levine-Donnerstein, D. (1999). Rethinking validity and reliability in content analysis. Journal of Applied Communication Research, 27, 258–284.Google Scholar
  96. Potter, J., Linz, D., Wilson, B. J., Kunkel, D., Donnerstein, E., Smith, S. L., et al. (1998). Content analysis of entertainment television: New methodological developments. In J. T. Hamilton (Ed.), Television violence and public policy (pp. 55–103). Ann Arbor: The University of Michigan Press.Google Scholar
  97. Radwin, L. E., & Cabral, H. J. (2010). Trust in Nurses Scale: Construct validity and internal reliability evaluation. Journal of Advanced Nursing, 66, 683–689.PubMedGoogle Scholar
  98. Ricciardelli, R., Clow, K. A., & White, P. (2010). Investigating hegemonic masculinity: Portrayals of masculinity in men’s lifestyle magazines. Sex Roles, 63, 64–78.Google Scholar
  99. Riffe, D., Lacy, S., & Fico, F. G. (2005). Analyzing media messages: Using quantitative content analysis in research (2nd ed.). Mahwah: Lawrence Erlbaum.Google Scholar
  100. Roberts, C. W. (Ed.). (1997). Text analysis for the social sciences: Methods for drawing statistical inferences from texts and transcripts. Mahwah: Lawrence Erlbaum.Google Scholar
  101. Scharrer, E. (2001). From wise to foolish: The portrayal of the sitcom father, 1950s–1990s. Journal of Broadcasting & Electronic Media, 45, 23–40.Google Scholar
  102. Schlenker, J. A., Caron, S. L., & Halteman, W. A. (1998). A feminist analysis of Seventeen magazine: Content analysis from 1945 to 1995. Sex Roles, 38, 135–149.Google Scholar
  103. Schmader, T., Whitehead, J., & Wysocki, V. H. (2007). A linguistic comparison of letters of recommendation for male and female chemistry and biochemistry job applicants. Sex Roles, 57, 509–514.PubMedGoogle Scholar
  104. Shapiro, G., & Markoff, J. (1997). A matter of definition. In C. W. Roberts (Ed.), Text analysis for the social sciences: Methods for drawing statistical inferences from texts and transcripts (pp. 9–34). Mahwah: Lawrence Erlbaum.Google Scholar
  105. Shoemaker, P. J., & Reese, S. D. (1996). Mediating the message: Theories of influences on mass media content (2nd ed.). White Plains: Longman.Google Scholar
  106. Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420–428.PubMedGoogle Scholar
  107. Simon, T. F., Fico, F., & Lacy, S. (1989). Covering conflict and controversy: Measuring balance, fairness and defamation in local news stories. Journalism Quarterly, 66, 427–434.Google Scholar
  108. Skymeg Software. (2009). Program for reliability assessment with multiple coders (PRAM). Unpublished manuscript, Cleveland, OH.Google Scholar
  109. Smith, A. M. (1999). Girls on film: Analysis of women’s images in contemporary American and "Golden Age" Hollywood films. Masters thesis, Cleveland State University, Cleveland, OH.Google Scholar
  110. Spence, J., Helmreich, R., & Stapp, J. (1974). The Personal Attributes Questionnaire: A measure of sex-role stereotypes and masculinity-femininity. JSAS Catalog of Selected Documents in Psychology, 4, 43.Google Scholar
  111. Uray, N., & Burnaz, S. (2003). An analysis of the portrayal of gender roles in Turkish television advertisements. Sex Roles, 48, 77–87.Google Scholar
  112. Valls-Fernandez, F., & Martinez-Vicente, J. M. (2007). Gender stereotypes in Spanish television commercials. Sex Roles, 56, 691–699.Google Scholar
  113. Weare, C., & Lin, W. Y. (2000). Content analysis of the World Wide Web—Opportunities and challenges. Social Science Computer Review, 18, 272–292.Google Scholar
  114. Weber, R. P. (1990). Basic content analysis (2nd ed.). Newbury Park: Sage.Google Scholar
  115. Weber, R., Behr, K.-M., Tamborini, R., Ritterfeld, U., & Mathiak, K. (2009). What do we really know about first-person-shooter games? An event-related, high-resolution content analysis. Journal of Computer-Mediated Communication, 14, 1016–1037.Google Scholar
  116. West, M. D. (Ed.). (2001). Theory, method, and practice in computer content analysis. Westport: Ablex.Google Scholar
  117. Xue, F., & Ellzey, M. (2009). What do couples do? A content analysis of couple images in consumer magazine advertising. Journal of Magazine and New Media Research, 10(2), 1–17.Google Scholar
  118. Zwick, R. (1988). Another look at interrater agreement. Psychological Bulletin, 103, 347–387.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.School of CommunicationCleveland State UniversityClevelandUSA

Personalised recommendations