The extent of gender bias in academia continues to be an object of inquiry, and recent research has begun to examine the particular gender biases emblematic in letters of recommendations. This current two-part study examines differences in the number of doubt raisers that are written in 624 authentic letters of recommendations for 174 men and women applying for eight assistant professor positions (study 1) and the impact of these doubt raisers on 305 university professors who provided evaluations of recommendation letters (study 2). The results show that both male and female recommenders use more doubt raisers in letters of recommendations for women compared to men and that the presence of certain types of doubt raisers in letters of recommendations results in negative outcomes for both genders. Since doubt raisers are more frequent in letters for women than men, women are at a disadvantage relative to men in their applications for academic positions. We discuss the implications and need for additional future research and practice that (1) raises awareness that letter writers are gatekeepers who can improve or hinder women’s progress and (2) develops methods to eliminate the skewed use of doubt raisers.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Aamodt, M. G., Nagy, M. S., & Thompson, N. (1998). Employment references: Who are we talking about?, Paper presented at the International Personnel Management Association Assessment Council, Chicago, IL.
Abbott, A., Cyranoski, D., Jones, N., Maher, B., Schiermeier, Q., & Van Noorden, R. (2010). Metrics: Do metrics matter? Nature News, 465(7300), 860–862.
Adamo, S. A. (2013). Attrition of women in the biological sciences: Workload, motherhood, and other explanations revisited. Bioscience, 63(1), 43–48.
Aguirre Jr, A. (2000). Women and minority faculty in the academic workplace: Recruitment, retention, and academic culture. ASHE-ERIC Higher Education Report, Volume 27, Number 6. Jossey-Bass Higher and Adult Education Series. Jossey-Bass, 350 Sansome St., San Francisco, CA 94104–1342.
Aiston, S. J. (2014). Leading the academy or being led? Hong Kong women academics. Higher Education Research & Development, 33(1), 59–72. https://doi.org/10.1080/07294360.2013.864618.
Anderson, J. C., & Gerbing, D. W. (1988). Structural equation modeling in practice: A review and recommended two-step approach. Psychological Bulletin, 103(3), 411–423. https://doi.org/10.1037/0033-2909.103.3.411.
APPIC (2005). Members survey: APPIC predoctoral internship members. http://www.APPIC.Org
Applegate, B. K., Cable, C. R., & Sitren, A. H. (2009). Academia’s most wanted: The characteristics of desirable academic job candidates in criminology and criminal justice. Journal of Criminal Justice Education, 20(1), 20–39.
Bagozzi, R. P., & Yi, Y. (1988). On the evaluation of structural equation models. Journal of the Academy of Marketing Science, 16(1), 74–94. https://doi.org/10.1007/BF02723327.
Bailyn, L. (2003). Academic careers and gender equity: Lessons learned from MIT1. Gender, Work & Organization, 10(2), 137–153. https://doi.org/10.1111/1468-0432.00008.
Benson, T. A., & Buskist, W. (2005). Understanding “excellence in teaching” as assessed by psychology faculty search committees. Teaching of Psychology, 32(1), 47–49.
Broughton, W., & Conlogue, W. (2001). What search committees want. Profession, 39–51.
Burgess, D., & Borgida, E. (1999). Who women are, who women should be: Descriptive and prescriptive gender stereotyping in sex discrimination. Psychology, Public Policy, and Law, 5, 665–692. https://doi.org/10.1037/1076-89220.127.116.115.
Carnegie Classification of Institutions of Higher Education (n.d.). About Carnegie classification. Retrieved from http://carnegieclassifications.iu.edu/.
Ceci, S. J., & Williams, W. M. (2015). Women have substantial advantage in STEM faculty hiring, except when competing against more-accomplished men. Frontiers in Psychology, 6, 1532.
Ceci, S. J., Ginther, D. K., Kahn, S., & Williams, W. M. (2014a). Women in academic science: A changing landscape. Psychological Science in the Public Interest, 15(3), 75–141.
Ceci, S. J., Ginther, D. K., Kahn, S., & Williams, W. M. (2014b). Women in academic science: A changing landscape. Psychological Science in the Public Interest, 15(3), 75–141.
Cejka, M. A., & Eagly, A. H. (1999). Gender-stereotypic images of occupations correspond to the sex segregation of employment. Personality and Social Psychology Bulletin, 25(4), 413–423. https://doi.org/10.1177/0146167299025004002.
Crocker, J., Major, B., & Steele, C. (1998). Social stigma. In D. T. Gilbert & S. T. Fiske (Eds.), The handbook of social psychology (Vol. 2, 4th ed., pp. 504–553). New York: McGraw-Hill.
Crockett, W. H. (1988). Schemas, affect, and communication. In L. Donohew, H. Sypher, & E. Higgins (Eds.), Communication, social cognition, and affect. Lawrence Erlbaum Association: Hillsdale, NJ.
Deo, M. E. (2014). Looking forward to diversity in legal academia. Berkeley Journal of Gender, Law & Justice, 29(2), 352.
Ding, W. W., Murray, F., & Stuart, T. E. (2013). From bench to board: Gender differences in university scientists’ participation in corporate scientific advisory boards. Academy of Management Journal, 56(5), 1443–1464. https://doi.org/10.5465/amj.2011.0020.
Duehr, E. E., & Bono, J. E. (2006). Men, women, and managers: Are stereotypes finally changing? Personnel Psychology, 59(4), 815–846.
Dutt, K., Pfaff, D. L., Bernstein, A. F., Dillard, J. S., & Block, C. J. (2016). Gender differences in recommendation letters for postdoctoral fellowships in geoscience. Nature Geoscience, 9(11), 805–808.
Eagly, A. H., & Johannesen-Schmidt, M. C. (2001). The leadership styles of women and men. Journal of Social Issues, 57, 781–797. https://doi.org/10.1111/0022-4537.00241.
Eagly, A. H., & Karau, S. J. (2002). Role congruity theory of prejudice toward female leaders. Psychological Review, 109, 573–598. https://doi.org/10.1037/0033-295X.109.3.573.
Easterly, D. M., & Ricard, C. S. (2011). Conscious efforts to end unconscious bias: Why women leave academic research. Journal of Research Administration, 42(1), 61–73.
Ellemers, N., van den Heuvel, H., de Gilder, D., Maas, A., & Bovini, A. (2004). The underrepresentation of women in science: Differential commitment or the queen bee syndrome? British Journal of Social Psychology, 43, 1–24. https://doi.org/10.1348/0144666042037999.
Eveline, J. (2005). Woman in the ivory tower: Gendering feminised and masculinised identities. Journal of Organizational Change Management, 18(6), 641–658. https://doi.org/10.1108/09534810510628558.
Fiske, S. T., & Linville, P. W. (1980). What does the schema concept buy us? Personality and Social Psychology Bulletin, 6, 543–557. https://doi.org/10.1177/014616728064006.
Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39–50. https://doi.org/10.2307/3151312.
Fuerstman, D., & Lavertu, S. (2005). The academic hiring process: A survey of department chairs. PS: Political Science & Politics, 38(4), 731–736.
Gatewood, R., & Feild, H. (2001). Human resource selection: Application forms, training and experience evaluations, and reference checks (5th ed.). Mason, OH: Roche, M.
Gaucher, D., Friesen, J., & Kay, A. C. (2011). Evidence that gendered wording in job advertisements exists and sustains gender inequality. Journal of Personality and Social Psychology, 101(1), 109–128. https://doi.org/10.1037/a0022530.
Guion, R. M. (1998). Assessment, measurement, and prediction for personnel decisions. Mahwah, NJ: Erlbaum.
Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2010). Multivariate data analysis: A global perspective. New York, NY: Pearson.
Hebl, M. R., Madera, J. M., & King, E. B. (2007). Exclusion, avoidance, and social distancing. In K. M. Thomas (Ed.), Diversity resistance: Manifestation and solutions (pp. 127–150). Mahwah, NJ: Lawrence Erlbaum Associates.
Hebl, M. R., Tickle, J., & Heatherton, T. F. (2000). Awkward moments in interactions between nonstigmatized and stigmatized individuals. In T. Heatherton, R. Kleck, M. Hebl, & J. Hull’s (Eds.), The social psychology of stigma. New York, NY: Guilford Press.
Hedricks, C. A., Robie, C., & Oswald, F. L. (2013). Web-based multisource reference checking: An investigation of psychometric integrity and applied benefits. International Journal of Selection and Assessment, 21(1), 99–110.
Heilman, M. E. (1983). Sex bias in work settings: The lack of fit model. Research in Organizational Behavior, 5, 269–298.
Heilman, M. E. (2001). Description and prescription: How gender stereotypes prevent women’s ascent up the organizational ladder. Journal of Social Issues, 57, 657–674. https://doi.org/10.1111/0022-4537.00234.
Heilman, M. E. (2012). Gender stereotypes and workplace bias. Research in Organizational Behavior, 32, 113–135. https://doi.org/10.1016/j.riob.2012.11.003.
Heilman, M. E., & Okimoto, T. G. (2007). Why are women penalized for success at male tasks? The implied communality deficit. Journal of Applied Psychology, 92, 81–92. https://doi.org/10.1037/0021-9010.92.1.81.
Heilman, M. E., Wallen, A. S., Fuchs, D., & Tamkins, M. M. (2004). Penalties for success: Reactions to women who succeed at male tasks. Journal of Applied Psychology, 89, 416–427. https://doi.org/10.1037/0021-9010.89.3.416.
Hough, L. M., Oswald, F. L., & Ployhart, R. E. (2001). Determinants, detection and amelioration of adverse impact in personnel selection procedures: Issues, evidence and lessons learned. International Journal of Selection and Assessment, 9(1–2), 152–194.
Howe-Walsh, L., & Turnbull, S. (2016). Barriers to women leaders in academia: Tales from science and technology. Studies in Higher Education, 41(3), 415–428.
Isaac, C., Chertoff, J., Lee, B., & Carnes, M. (2011). Do students’ and authors’ genders affect evaluations? A linguistic analysis of medical student performance evaluations. Academic Medicine, 86(1), 59–66. https://doi.org/10.1097/ACM.0b013e318200561d.
Johnson, M., Elam, C., Edwards, J., Tayor, D., Heldberg, C., Hinkley, R., & Comeau, R. (1998). Medical school admission committee members’ evaluations of and impressions from recommendation letters. Academic Medicine, 73, S41–S43. https://doi.org/10.1097/00001888-199810000-00040.
Kaminski, D., & Geisler, C. (2012). Survival analysis of faculty retention in science and engineering by gender. Science, 335(6070), 864–866.
Kelly, B. T., & McCann, K. I. (2014). Women faculty of color: Stories behind the statistics. The Urban Review, 46(4), 681–702.
Kervyn, N., Bergsieker, H. B., & Fiske, S. T. (2012). The innuendo effect: Hearing the positive but inferring the negative. Journal of Experimental Social Psychology, 48(1), 77–85. https://doi.org/10.1016/j.jesp.2011.08.001.
Knouse, S. B. (1983). The letter of recommendation: Specificity and favorability of information. Personnel Psychology, 36, 331–341. https://doi.org/10.1111/j.1744-6570.1983.tb01441.x.
Koenig, A. M., Eagly, A. H., Mitchell, A. A., & Ristikari, T. (2011). Are leader stereotypes masculine? A meta-analysis of three research paradigms. Psychological Bulletin, 137, 616–642.
Kuncel, N. R., Kochevar, R. J., & Ones, D. S. (2014). A meta-analysis of letters of recommendation in college and graduate admissions: Reasons for hope. International Journal of Selection and Assessment, 22, 101–107. https://doi.org/10.1111/ijsa.12060.
LaCroix, P. P. (1985). Sex in recs: gender bias in recommendation writing. Journal of College Admission, 109, 24–26.
Landrum, R. E., & Clump, M. A. (2004). Departmental search committees and the evaluation of faculty applicants. Teaching of Psychology, 31(1), 12–17.
Landrum, R., Jeglum, E., & Cashin, J. (1994). The decision-making process of graduate admissions committees in psychology. Journal of Social Behavior and Personality, 9, 239–248.
LeBreton, J. M., & Senter, J. L. (2007). Answers to 20 questions about interrater reliability and interrater agreement. Organizational Research Methods, 11(4), 815–852. https://doi.org/10.1177/1094428106296642.
Lee, Y. J., & Won, D. (2014). Trailblazing women in academia: Representation of women in senior faculty and the gender gap in junior faculty’s salaries in higher educational institutions. The Social Science Journal, 51(3), 331–340.
Lerback, J., & Hanson, B. (2017). Journals invite too few women to referee. Nature, 541(7638), 455–457.
Levine, R. B., Lin, F., Kern, D. E., Wright, S. M., & Carrese, J. (2011). Stories from early-career women physicians who have left academic medicine: A qualitative study at a single institution. Academic Medicine, 86(6), 752–758.
Liu, O. L., Minsky, J., Ling, G., & Kyllonen, P. (2009). Using the standardized letters of recommendation in selection: Results from a multidimensional Rasch model. Educational and Psychological Measurement, 69, 475–492. https://doi.org/10.1177/0013164408322031.
Maass, A., & Arcuri, L. (1996). Language and stereotyping. In C. N. Macrae, C. Stangor, & M. Hewstone (Eds.), Stereotypes and stereotyping (pp. 193–226). New York, NY: Guilford Press.
Madera, J. M., Hebl, M. R., & Martin, R. C. (2009). Gender and letters of recommendation for academia: Agentic and communal differences. Journal of Applied Psychology, 94(6), 1591–1599. https://doi.org/10.1037/a0016539.
McCarthy, J. M., & Goffin, R. D. (2001). Improving the validity of letters of recommendation: An investigation of three standardized reference forms. Military Psychology, 13, 199–222. https://doi.org/10.1207/S15327876MP1304_2.
Meizlish, D., & Kaplan, M. (2008). Valuing and evaluating teaching in academic hiring: A multidisciplinary, cross-institutional study. The Journal of Higher Education, 79(5), 489–512.
Mittenberg, W., Peterson, R. S., Cooper, J. T., Strauman, S., & Essig, S. M. (2000). Selection criteria for clinical neuropsychology internships. The Clinical Neuropsychologist, 14, 1–6.
Morgan, W. B., Elder, K. B., & King, E. B. (2013). The emergence and reduction of bias in letters of recommendation. Journal of Applied Social Psychology, 43(11), 2297–2306. https://doi.org/10.1111/jasp.12179.
Moss-Racusin, C. A., Dovidio, J. F., Brescoll, V. L., Graham, M. J., & Handelsman, J. (2012). Science faculty’s subtle gender biases favor male students. Proceedings of the National Academy of Sciences, 109(41), 16474–16479.
National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. (2007). Beyond bias and barriers: Fulfilling the potential of women in academic science and engineering. Washington, DC: The National Academies Press.
National Research Council (NRC). (2009). Gender differences at critical transitions in the careers of science, engineering and mathematics faculty. Washington, DC: National Academy Press.
National Science Foundation, Division of Science Resources Statistics (2004). Gender differences in the careers of academic scientists and engineers, NSF 04-323, Project Officer, Alan I. Rapoport (Arlington, VA).
Nicklin, M. J., & Roch, S. G. (2009). Letters of recommendation: Controversy and consensus from expert perspectives. International Journal of Selection and Assessment, 17, 76–91. https://doi.org/10.1111/j.1468-2389.2009.00453.x.
Pennebaker, J. W., Francis, M. E., & Booth, R. J. (2001). Linguistic inquiry and word count (LIWC 2001): A computerized text analysis program. Mahwah, NJ: Erlbaum.
Perry, G., Moore, H., Edwards, C., Acosta, K., & Frey, C. (2009). Maintaining credibility and authority as an instructor of color in diversity-education classrooms: A qualitative inquiry. The Journal of Higher Education, 80(1), 230–244.
Peterson, N. B., Friedman, R. H., Ash, A. S., Franco, S., & Carr, P. L. (2004). Faculty self-reported experience with racial and ethnic discrimination in academic medicine. Journal of General Internal Medicine, 19(3), 259–265.
Pyke, J. (2013). Women, choice and promotion or why women are still a minority in the professoriate. Journal of Higher Education Policy and Management, 35(4), 444–454. https://doi.org/10.1080/1360080X.2013.812179.
Ragins, B. R., & Sundstrom, E. (1989). Gender and power in organizations. Psychological Bulletin, 105, 51–88. https://doi.org/10.1037/0033-2909.105.1.51.
Ragins, B. R., Townsend, B., & Mattis, M. (1998). Gender gap in the executive suite: CEOs and female executives report on breaking the glass ceiling. Academy of Management Executive, 12, 28–42 http://www.jstor.org/stable/4165439.
Ralston, S. M., & Thameling, C. A. (1988). Effects of vividness of language on information value of reference letters and job applicants’ recommendation. Psychological Reports, 62, 867–870. https://doi.org/10.2466/pr0.1918.104.22.1687.
Raudenbush, S., Bryk, A., Cheong, Y. F., & Congdon, R. (2004). HLM 6: Hierarchical and nonlinear modeling [computer software]. Lincolnwood, IL: Scientific Software International.
Rubini, M., & Menegatti, M. (2014). Hindering women’s careers in academia gender linguistic bias in personnel selection. Journal of Language and Social Psychology, 0261927X14542436.
Rudman, L. A., & Glick, P. (2001). Perspective gender stereotypes and backlash toward agentic women. Journal of Social Issues, 57, 743–762. https://doi.org/10.1111/0022-4537.00239.
Schmader, T., Whitehead, J., & Wysocki, V. H. (2007). A linguistic comparison of letters of recommendation for male and female chemistry and biochemistry job applicants. Sex Roles, 57(7–8), 509–514. https://doi.org/10.1007/s11199-007-9291-4.
Settles, I. H., Cortina, L. M., Malley, J., & Stewart, A. J. (2006). The climate for women in academic science: The good, the bad, and the changeable. Psychology of Women Quarterly, 30(1), 47–58.
Sheehan, E. P., McDevitt, T. M., & Ross, H. C. (1998). Looking for a job as a psychology professor? Factors affecting applicant success. Teaching of Psychology, 25, 8–11. https://doi.org/10.1207/s15328023top2501_3.
Shen, H. (2013). Mind the gender gap. Nature, 495(7439), 22–24.
Stanley, C. A. (2006). Coloring the academic landscape: Faculty of color breaking the silence in predominantly White colleges and universities. American Educational Research Journal, 43(4), 701–736.
Su, R., Rounds, J., & Armstrong, P. I. (2009). Men and things, women and people: A meta-analysis of sex differences in interests. Psychological Bulletin, 135(6), 859–884. https://doi.org/10.1037/a0017364.
Taylor, D. (2007). Employment preferences and salary expectations of students in science and engineering. Bioscience, 57, 175–185. https://doi.org/10.1641/B570212.
Taylor, P. J., Pajo, K., Cheung, G. W., & Stringfield, P. (2004). Dimensionality and validity of a structured telephone reference check procedure. Personnel Psychology, 57(3), 745–772.
Treviño, L. J., Gomez-Mejia, L. R., Balkin, D. B., & Mixon, F. G. (2015). Meritocracies or masculinities? The differential allocation of named professorships by gender in the academy. Journal of Management., 44, 972–1000. https://doi.org/10.1177/0149206315599216.
Trix, F., & Psenka, C. (2003). Exploring the color of glass: Letters of recommendation for female and male medical faculty. Discourse and Society, 14, 191–220. https://doi.org/10.1177/0957926503014002277.
U.S. Department of Commerce (2011). Women in STEM: A gender gap to innovation. Executive summary. Economics and Statistics Administration. ESA Issue Brief #04-11. August Retrieved on 1/10/2015 at url: http://www.esa.doc.gov/sites/default/files/reports/documents/womeninstemagaptoinnovation8311.pdf.
U.S. Department of Education, National Center for Education Statistics (2015). The condition of education 2016 (NCES 2016-144), characteristics of postsecondary faculty. Retrieved from https://nces.ed.gov/fastfacts/display.asp?id=61
Valian, V. (1998). Why so slow? The advancement of women. Cambridge: M.I.T. Press.
Van den Brink, M., & Benschop, Y. (2012). Slaying the seven-headed dragon: The quest for gender change in academia. Gender, Work & Organization, 19(1), 71–92. https://doi.org/10.1111/j.1468-0432.2011.00566.x.
Westring, A. F., Speck, M. R. M., Sammel, M. D., Scott, M. P., Tuton, L. W., Grisso, J. A., & Abbuhl, S. (2012). A culture conducive to women’s academic success: Development of a measure. Academic Medicine: Journal of the Association of American Medical Colleges, 87(11), 1622–1631. https://doi.org/10.1097/ACM.0b013e31826dbfd1.
Williams, W. M., & Ceci, S. J. (2015). National hiring experiments reveal 2:1 faculty preference for women on STEM tenure track. Proceedings of the National Academy of Sciences, 112(17), 5360–5365.
Wood, W., & Eagly, A. H. (2000). Once again, the origins of sex differences. American Psychologist, 55(9), 1062–1063. https://doi.org/10.1037/0003-066X.55.9.1062.
Yost, E., Winstead, V., Cotten, S. R., & Handley, D. M. (2013). The recruitment and retention of emerging women scholars in stem: Results from a national web-based survey of graduate students, postdoctoral fellows, and junior faculty. Journal of Women and Minorities in Science and Engineering, 19(2), 143–163. https://doi.org/10.1615/JWomenMinorScienEng.2013003021.
This paper was funded by an NIH Grant (1R01GM088530).
Appendix 1. Data Transparency Appendix
The data reported in this submitted manuscript (study 1 data only) have been previously published. Findings from the data collection have been reported in separate manuscripts. MS 1 (published) focuses on communal and agentic descriptions of applicants in letters of recommendations for academic positions as the dependent variables. MS 2 (the current submitted manuscript) focuses on doubt raiser descriptions of applicants in letters of recommendations as the dependent variables. The table below displays where each data variable appears in each study, as well as the current status of each study.
|Variables in the complete dataset||
(status = pub)
(status = current)
|Doubt raisers: negatives||x|
|Doubt raisers: hedges||x|
|Doubt raisers: faint praises||x|
|Doubt raisers: irrelevancies||x|
|Letter writer gender||x||x|
Appendix 2. Letter Exemplar
Dear Search Committee,
It is with enthusiasm that I recommend AA for a tenure track faculty position (Assistant Professor) within the <DEPT> at WR99. I was AA’s doctoral research advisor at WRNR and I know AA both professionally and personally. As a graduate student, AA also served as my teaching assistant for two undergraduate laboratory classes. AA was an impressive student who I have had the pleasure to work with at WRNR.
<MANIPULATION HERE> being successful in developing an independent research program at your institution. I have seen AA mature into a more careful scientist who demonstrates competence, leadership skills, and curiosity. I have kept in close contact with AA during <his/her> post doctoral training and know that <he/she> has matured scientifically and has expanded <his/her> knowledge base into other closely-related fields. AA has aptitude to continue developing in the field. In terms of research, AA has published two manuscripts based on <his/her> thesis work in my lab, and a third manuscript is pending submission. I know that AA detailed this work in <his/her> research statement so I will only state here that it is published in a solid journal and is theoretically strong and methodologically sound.
AA projects professionalism, whether it is in the lecture room and undergraduate laboratory, the research laboratory, or at conferences. AA is hardworking and also willing to take time to teach others. AA became a leader in my research lab, taking time to mentor undergraduate students and less senior PhD students. AA has given a series of tutorial lectures on statistics in Psychology to the PhD students at WR99. AA is very willing to help others and I believe <he/she> demonstrates natural teaching abilities plus <he/she> greatly enjoys it. Both AA’s skills and his vision are broad and fine-tuned.
In conclusion, I have come to regard AA with respect over the past several years. I hope you interview <him/her>. If you have any further questions about AA, please do not hesitate to phone me at [number removed].
Associate Professor of <DEP>
About this article
Cite this article
Madera, J.M., Hebl, M.R., Dial, H. et al. Raising Doubt in Letters of Recommendation for Academia: Gender Differences and Their Impact. J Bus Psychol 34, 287–303 (2019). https://doi.org/10.1007/s10869-018-9541-1
- Letters of recommendation
- Gender schemas
- Sex roles