Abstract
This paper explores differentials in student ratings of instructors (SRIs) by both race (white and nonwhite) and sex (male and female), taking into account not only the race and sex of class instructors but also the race-sex percentage composition of their enrolled students. Our dataset is by far the largest in the literature to date and includes all course evaluations over Academic Years 2006–2012 at Occidental College, a selective liberal arts institution with relatively high levels of diversity by race and sex of both students and faculty. We examine the data with multilevel mixed-effects linear and ordered probit regression specifications that include an extensive set of non-demographic control variables. Our findings include evidence that is consistent with the existence of bias on the part of white students.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
The broader academic literature on ratings differentials of instructors by race is likewise scant and, to date, does not control for the sorts of potential contributing factors that distinguishes the existing economics literature. Smith (2007) and Smith and Hawkins (2011) report lower average ratings for Black faculty compared to both White and Other nonwhite faculty in a sample of student evaluations from the College of Education at a research institution in the Southern United States. Reid (2010) examines SRI data from RateMyProfessor.com and concludes that “racial minority faculty, particularly Blacks and Asians, were evaluated more negatively than White faculty in terms of overall quality, helpfulness, and clarity, but were rated higher on ‘easiness.’”.
“Gender” prevails in the existing literature where it is implicitly interchangeable with “sex.” As in Hamermesh and Parker (2015), except when quoting others we employ the term “sex,” which conforms to the category label and the dichotomous “male/female” classification of our College data.
Wagner et al. (2016) use the term “ethnicity” throughout their paper. The term “race” predominates in the other studies we cite.
In the other study above by McPherson and Jewell (2007), the students in the sample consist only of those taking classes taught by economists in the Master’s degree program in Economics at their University. The authors found that white instructors receive a SET score amounting to 0.20 standard deviations higher than nonwhite instructors. They highlight this result in the paper’s abstract by indicating that they “find evidence of bias against nonwhite faculty,” but they caution in a later footnote that this conclusion should be viewed with “some skepticism” due to the very limited sample of nonwhite instructors (3 nonwhite instructors in a sample of 22 total instructors, as we noted previously).
While other economists have also studied student evaluations of instructors and especially whether such evaluations are correlated with student learning—for example, Braga, et al. (2014) or Carrell and West (2010)—we have limited our literature review to the subset of economics studies that include variables for race/ethnicity.
A similar approach was used by Burdekin et al. (2005) in a different context. That study found that basketball teams in areas with larger concentrations of white residents in the surrounding metro area had significant revenue (fan attendance) gains when they included white players on their roster. We thank a referee for pointing out to us this similar approach.
The Herfindahl indexes are calculated as the sum of the squares of the individual race-sex shares of employment for full-time faculty and of enrollment for full-time students. The data come from the IPEDS Data Center of the National Center for Education Statistics <http://nces.ed.gov/ipeds/datacenter/>.
Our dataset did not have sufficient numbers of some faculty race-sex categories to allow for a more disaggregated examination of instructors by race.
Note from Table 4 that the median instructor SRI is 5.95. Average SRIs lag the median because the SRIs skew toward the upper range of the 7-point scale.
We also estimated an equation that replaced the various divisional and course-level dummy variables with fixed-effect dummy variables for each course level within each subject area (153 variables in total). The estimated results for both the demographic and non-demographic variables were virtually unchanged from those that we report here.
The implicit assumption of this interpretation is that the expected bias of each incremental enrolled student equals the average bias of the student’s demographic category. We estimated an alternative specification which included a quadratic term for each of the 16 student demographic percentages and could not reject the hypothesis that the entire group of 16 quadratic terms was equal to zero.
A more disaggregated analysis reveals that the negative relationship pertains exclusively to Asian female students enrolled in classes taught by Asian female instructors.
To the extent that course enrollments consist of a heterogeneous mix of biased and unbiased students, our estimated race-sex differentials constitute weighted averages of bias intensity and prevalence, making them lower-bound estimates for the subset of enrolled students who do discriminate. For example, the estimated negative differentials for cross-group instructors by white students reported in Part D of Table 6 average − 0.43 ratings points, large enough to drop a median-rated Occidental instructor’s SRI from 5.95 to 5.52 and the instructor’s ranking to the 30th percentile. If only half of the enrolled white students discriminate, the estimated average differential doubles for those who do so, implying that those students would rate the instructor at an average of 5.09, which ranks at just the 17th percentile.
The enrollment shares in these outlier courses equal 19% for WMs, 75% for WFs, 0% for NMs and 6% for NFs, compared to the average values reported in Table 1, row 5 of 26% for WMs, 31% for WFs, 15% for NMs and 22% for NFs. Combining these enrollment differences with the estimated ratings differentials for nonwhite male instructors reported in row 19 of Table 6 yields the calculated impact on the instructor’s SRI: − 0.698 × (0.19 − 0.26) + − 0.956 × (0.75 − 0.31) + − 0.669 × (0.06 − 0.22) = − 0.26.
Of course, teaching is not the only criterion used for tenure consideration. Professional development (research) and service to the college are also important criteria.
References
Anderson, Kathryn H., and J. Siegfried. 1997. Gender Differences in Rating the Teaching of Economics. Eastern Economic Journal 23 (3): 347–357.
Boring, Anne. 2017. Gender Biases in Student Evaluations of Teaching. Journal of Public Economics 145 (1): 27–41.
Braga, Michela, Marco Paccagnella, and Michele Pellizzare. 2014. Evaluating Students’ Evaluations of Professors. Economics of Education Review 30 (4): 641–653.
Burdekin, R.C.K., R.T. Hossfeld, and J.K. Smith. 2005. Are NBA Fans Becoming Indifferent to Race? Journal of Sports Economics 6 (2): 144–159.
Carrell, Scott E., and James E. West. 2010. Does Professor Quality Matter? Evidence from a Random Assignment of Students to Professors. Journal of Political Economy 118 (3): 409–432.
Centra, J.A., and N.B. Gaubatz. 2000. Is There Gender Bias in Student Evaluations of Teaching? Journal of Higher Education 70 (1): 17–33.
Feldman, K.A. 1993. College Students’ Views of Male and Female College Teachers: Part II-Evidence from Students’ Evaluations of Their Classroom Teachers. Research in Higher Education 34 (2): 151–211.
Hamermesh, D.S., and A.M. Parker. 2005. Beauty in the Classroom: Instructors’ Pulchritude and Putative Pedagogical Productivity. Economics of Education Review 24 (4): 369–376.
Hativa, N. 2013. Student Ratings of Instruction: Recognizing Effective Teaching. United States: Oron Publications.
McPherson, Michael, and R.Todd Jewell. 2007. Leveling the Playing Field: Should Student Evaluation Scores Be Adjusted? Social Science Quarterly 88 (3): 868–881.
McPherson, Michael, R.Todd Jewell, and Myungsup Kim. 2009. What Determines Student Evaluation Scores? A Random Effects Analysis of Undergraduate Economics Classes. Eastern Economic Journal 35: 37–51.
Mengel, Friederike, Jan Sauermann, and Ulf Zolitz. 2019. Gender Bias in Teaching Evaluations. Journal of the European Economic Association 17 (2): 535–566.
Reid, Landon. 2010. The Role of Perceived Race and Gender in the Evaluation of College Teaching on RateMyProfessors.com. Journal of Diversity in Higher Education 3 (3): 137–152.
Saunders, Kent T., and Phillip Saunders. 1999. The Influence of Instructor Gender on Learning and Instructor Ratings. Atlantic Economic Journal 27 (4): 460–473.
Smith, B.P. 2007. Student Ratings of Teaching Effectiveness: An Analysis of End-of-Course Faculty Evaluations. College Student Journal 41 (4): 788–800.
Smith, B.P., and B. Hawkins. 2011. Examining Student Evaluations of Black College Faculty: Does Race Matter? The Journal of Negro Education 80 (2): 149–162.
Theall, Michael, and Jennifer Franklin. 2001. Looking for Bias in All the Wrong Places: A Search for Truth or a Witch Hunt? New Directions for Institutional Research 109: 45–56.
Wagner, N., M. Rieger, and K.J. Voorvelt. 2016. Gender, Ethnicity and Teaching Evaluations: Evidence from Mixed Teaching Teams. Economics of Education Review 54: 79–94.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Moore, R.L., Song, H. & Whitney, J.D. Do Students Discriminate? Exploring Differentials by Race and Sex in Class Enrollments and Student Ratings of Instructors. Eastern Econ J 47, 135–162 (2021). https://doi.org/10.1057/s41302-020-00176-2
Published:
Issue Date:
DOI: https://doi.org/10.1057/s41302-020-00176-2