Guidelines for Qualitative Research Being Published in Sex Roles
As I wrote in 2008 (Frieze 2008), and is still the case, a growing number of papers submitted to and ultimately published in Sex Roles involve the analysis of qualitative data. As is also true for quantitative data analysis, each journal editor makes decisions about what types of methods and reporting are acceptable within this larger methodology. Since we receive many questions about qualitative papers (but rarely are asked about quantitative data presentations), these guidelines for qualitative research reports are being published now and will appear on the journal website so that our requirements are readily available to potential authors.
The first questions we ask in evaluating a manuscript submitted for publication consideration that reports on qualitative data is what data were collected and how the data were coded. We ask whether the data collection and analysis were done in a scientific way. As in any scientific paper, it is essential that sufficient information be provided so that another researcher would be able to find similar information and would also interpret the same data in the same way as the author of the paper being evaluated. Interpretation of interview data is a highly subjective process. In order to validate the analysis of the data, ideally, someone other than the author should be categorizing the responses. Confirmation biases of finding what one expects to find are well known. If it is impossible to have the researcher’s data categorized entirely by someone else, at least some of the categorization should be verified by someone else who is not aware of the predictions or assumptions of the original researcher.
To further elaborate on the specific requirements for this journal, we ask that the researcher provide a listing of all issues coded in the data, with possible codes for each of these issues. Each of these possible coding categories should be defined in a way that would allow another researcher to code similar data using these same categories. These definitions should be provided in a table, so they can be readily seen by the reader. Secondly, it is required that the researcher explain clearly how the assignment of coding categories was validated. Ideally, there should be a report of inter-coder reliability, with kappa values for each code. The identity of each coder and exactly what data were coded by each person should be provided. If this is not possible, and someone other than the author has done the coding, it would be possible to establish validity by providing a summary of the codes and the results of the coding analysis to each of the original study participants to see if the conclusions drawn reflect what he or she said in the interview. If this alternative method of validation is used, a full report of how people were contacted, exactly what they said, and the number of people responding needs to be provided. Similar guidelines need to be followed if the researcher is coding pictures, printed text, or some other form of qualitative data.
If data are coded using a computerized procedure, validation of the final counts is not needed. However, we do need the complete list of search terms used to classify each coding category. Data should also be presented about exactly how the programs were set up and whether additional terms were added in the coding process.
If the data are handled in an acceptable way, we then move to other aspects of the paper in deciding whether or not we can accept the paper for publication. Sex Roles does not publish completely exploratory studies. Thus, we require that the researcher have some idea of what types of issues he or she hopes to examine in the data. Formal hypotheses are welcomed, but not required for qualitative data analysis. Research questions, expressed in terms of particular coding categories, should be clearly expressed in the Introduction section of the paper. Theory and previously published research about why these particular coding categories would be of interest is needed. The researcher should make sure that the importance of the issues being coded is clear within the culture in which the study is done. A previous editorial (Frieze and Dittrich 2008) explains how cultural information should be considered and presented in Sex Roles. This should be done in terms of establishing the cultural context for study, as well as in considering cited research done in other countries and how these studies are relevant for the current paper.
Next we turn to the Method section. This should begin by explaining clearly who the study participants were, or how the coded materials were selected. For interview data, explain how study participants were recruited and what they were told about the study. Note what country and region they come from and provide as much demographic information as possible. Explain exactly what interview questions they were asked and how the responses were recorded. For observations, how were the situations being observed selected?
For more information about qualitative research, especially as it relates to content analysis, see Neuendorf (2011). Researchers from many different countries work to analyze television or other media in their own country, and submit the results of their efforts for publication in Sex Roles. Unfortunately, we are unable to publish many of these because of their failure to follow the practices outlined here or in Neuendorf (2011).
Since formal coding of data is required, the Results should be a report of counts and percentages of material within each coding category. Typically, such data are analyzed using chi squares. Parametric statistics are generally not appropriate. Make sure the presentation of findings follows the structure of the research questions listed at the end of the Introduction. Detailed quotations, if used at all, would be to explain a particular coding category more fully.
Since the type of data analysis described here is not possible to do with case studies or with focus groups or other less structured forms of data analysis such as discourse analysis, such methodologies are generally not acceptable for publication in Sex Roles. As Editor, I am happy to answer questions. Any author who has questions about any of these guidelines should write to us at firstname.lastname@example.org for more information.