Skip to main content
SpringerLink
Log in

Peer Review: From “Sacred Ideals” to “Profane Realities”

  • Chapter
  • First Online:
Higher Education: Handbook of Theory and Research

Part of the book series: Higher Education: Handbook of Theory and Research ((HATR,volume 32))

Abstract

Peer review, a socially structured process of evaluating scholarly and scientific performance, is a ubiquitous condition of role performance in the professoriate and central to the production of knowledge. Focusing on the evaluation of publication, this chapter directs attention to three features of peer review: its functional ideals and relationship with the academic reward system; the social organizational basis of peer review and trends that constrain it; and dysfunctions that arise related to reliability, bias, and violation of anonymity. The discussion underscores structural and cultural characteristics of peer review that seemingly shatter its idealized image. While all faculty who conduct research subject themselves to the evaluation of their peers, extant research on peer review is disproportionately based on studies of relatively prestigious journals in the social sciences and medicine. The review thus identifies promising paths for future empirical studies of peer review that would examine understudied disciplines and publication venues, ideally through comparative frameworks.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    For other, secondary arenas where peer review operates, readers can consult illustrative treatments: for example, tenure and promotion (Fairweather, 2002; Hearn & Anderson, 2002; Lawrence, Celis, & Ott, 2014; Long, Allison, & McGinnis, 1993; Perna, 2001, 2005; Youn & Price, 2009); post-tenure review (Aper & Fry, 2003; O’Meara, 2004; Patriquin et al., 2003; Wood & Johnsrud, 2005); grants (General Accounting Office, 1994; Gillespie, Chubin, & Kurzon 1985; Langfeldt, 2001; Liebert, 1976; Roy, 1985); research fellowships (Bornmann & Daniel, 2005; Lamont, 2009); and salaries (Perna, 2003).

  2. 2.

    Requests to review, however, are not universally a sign of recognition. The role of recognition in this respect is historically contingent, pre-dating the rise of electronic submissions. Authors are now often required to indicate areas of interest when submitting a manuscript to a journal, selections that provide editors and editorial staff with a pool of reviewers to approach in the future. Thus, even a relatively unknown researcher who might have rarely published can be approached to review scholarship. Therefore, reputation and the exigencies of managing a journal likely both come into play in reviewer selection.

  3. 3.

    Juhasz et al. (1975) do not specify which fields comprise their sample. They note in a comment about fields that “the conclusions of our study do not relate to any differences between the acceptance-rejection ratios between the humanistic literature and the scientific and technical literature” (p. 184).

  4. 4.

    The length of time that authors take to re-submit manuscripts for consideration elsewhere is a question open to empirical inquiry. Length of time may function, for example, from reviews received and decisions to revise before sending manuscripts to different outlets. Length of time may also function by career stage; the younger the stage, the lesser time on the shelf. In the earliest stages, shelves may be altogether short: anecdotal evidence suggests that many junior scholars are advised always to keep completed manuscripts under review, under the premise that they will “hit” somewhere. The premise itself underscores the socially situated variability of peer review processes (i.e., manuscripts of all sorts will be accepted somewhere).

References

  • Aarssen, L. W., Lortie, C. J., Budden, A. E., Koricheva, J., Leimu, R., & Tregenza, T. (2009). Does publication in top-tier journals affect reviewer behavior? Plos One, 4(7), e6283. doi:10.1371/journal.pone.0006283.

    Article  Google Scholar 

  • Abbott, A. (2011). Personal communication to second author, annual board meeting of the American Journal of Sociology, Atlanta, Georgia.

    Google Scholar 

  • Abramowitz, S. I., Gomes, B., & Abramowitz, C. V. (1975). Publish or politic: Referee bias in manuscript review. Journal of Applied Social Psychology, 5(3), 187–2000.

    Article  Google Scholar 

  • Adair, R. K., Carlon, H. R., & Sherman, C. (1981). Anonymous refereeing. Physics Today, 34(6), 13–15.

    Article  Google Scholar 

  • Alam, M., Kim, N.A., Havey, J., Rademaker, A., Ratner, D., Tregre, B.,…Coleman, W.P. (2011). Blinded vs. unblended peer review of manuscripts submitted to a dermatology journal: A randomized multi-rater study. British Journal of Dermatology, 165(3), 563–567.

    Google Scholar 

  • Alperts, B. (2013). Impact factor distortions. Science, 340(6134), 787.

    Article  Google Scholar 

  • Amrein, K., Langmann, A., Fahrleitner-Pammer, A., Pieber, T. R., & Zollner-Schwetz, I. (2011). Women underrepresented on editorial boards of 60 major medical journals. Gender Medicine, 8(6), 378–387.

    Article  Google Scholar 

  • Aper, J. P., & Fry, J. E. (2003). Post-tenure review at graduate institutions in the United States. Journal of Higher Education, 74(3), 241–260.

    Article  Google Scholar 

  • Appel, C. S. (1994). University press editing and publishing. In R. J. Simon & J. J. Fyfe (Eds.), Editors as gatekeepers: Getting published in the social sciences (pp. 179–194). Lanham, MD: Rowman & Littlefield Publishers.

    Google Scholar 

  • Bacon, F. (1620). Novum Organum: Or, true suggestions for the interpretation of nature. London, UK: Routledge and Sons.

    Google Scholar 

  • Bazerman, C. (1988). Shaping written knowledge: The genre and activity of the experimental article in science. Madison, WI: University of Wisconsin Press.

    Google Scholar 

  • Becher, T., & Trowler, P. R. (2001). Academic tribes and territories. Buckingham, UK: Open University Press.

    Google Scholar 

  • Becker, H. S. (1984). Art worlds. Berkeley, CA: University of California Press.

    Google Scholar 

  • Ben-David, J. (1965). The scientific role: The conditions of its establishment in Europe. Minerva, 4(1), 15–54.

    Article  Google Scholar 

  • Bernard, H. R. (1980). Report from the editor. Human Organization, 39(4), 366–369.

    Article  Google Scholar 

  • Bess, J. L. (1988). Collegiality and bureaucracy in the modern university: The influence of information and power on decision-making structures. New York: Teachers College Press.

    Google Scholar 

  • Beveridge, W. I. B. (1950). The art of scientific investigation. New York: Vintage Books.

    Google Scholar 

  • Biagioli, M. (2002). From book censorship to academic peer review. Emergences: Journal for the Study of Media & Composite Cultures, 12(1), 11–45.

    Article  Google Scholar 

  • Björk, B. C., Roos, A., & Lauri, M. (2009). Scientific journal publishing: Yearly volume and open access availability. Information Research, 14(1). http://InformationR.net/ir/14-1/paper391.html

  • Björk, B., Welling, P., Laakso, M., Majlender, P., Hedlund, T., & Guõnason, G. (2010). Open access to the scientific literature: Situation 2009. PLoS ONE, 5(6), e11273. doi:10.137/journal.pone.0011273.

    Google Scholar 

  • Björk, B. C., & Solomon, D. (2013). The publishing delay in scholarly peer-reviewed journals. Journal of Infometrics, 7(4), 914–923.

    Article  Google Scholar 

  • Blank, R. M. (1991). The effects of double-blind versus single-blind reviewing: Experimental evidence from the American Economic Review. American Economic Review, 81(5), 1041–1067.

    Google Scholar 

  • Bohannon, J. (2013). Who’s afraid of peer review? Science, 342(6154), 60–65.

    Article  Google Scholar 

  • Bornmann, L., & Daniel, H. (2005). Selection of research fellowship recipients by committee peer review: Reliability, fairness and predictive validity of board of trustees’ decisions. Scientometrics, 63(2), 297–320.

    Article  Google Scholar 

  • Bornmann, L., Mutz, R., & Daniel, H. (2010a). A reliability-generalization study of journal peer reviews: A multi-level meta-analysis of inter-rater-reliability and its determinants. PloS One, 5(12), e14331. doi:10.1371/journalpone.0014331.

    Article  Google Scholar 

  • Bourdieu, P. (1984). Distinction: A social critique of the judgement of taste. Cambridge, MA: Harvard University Press.

    Google Scholar 

  • Bradley, J. V. (1981). Pernicious publication practices. Bulletin of the Psychonomic Society, 18(1), 31–34.

    Article  Google Scholar 

  • Braxton, J. M., & Hargens, L. L. (1996). Variation among academic disciplines: Analytical frameworks and research. In J. C. Smart (Ed.), Higher education: Handbook of theory and research (Vol. XI, pp. 1–46). New York, NY: Agathon.

    Google Scholar 

  • Budden, A. E., Tregenza, T., Aarssen, L. W., Koricheva, J., Leimu, R., & Lortie, C. J. (2008). Double-blind review favours increased representation of female authors. TRENDS in Ecology and Evolution, 23(1), 4–6.

    Article  Google Scholar 

  • Burnham, J. C. (1990). The evolution of editorial peer review. Journal of the American Medical Association, 263(10), 1323–1329.

    Article  Google Scholar 

  • Calhoun, C. (2010). On Merton’s legacy and contemporary sociology. In R. K. Merton (Ed.), Sociology of science and sociology as science (pp. 1–32). New York, NY: Columbia University Press.

    Chapter  Google Scholar 

  • Campanario, J. M. (1993). Consolation for the scientist: Sometimes it is hard to publish papers that are later highly-cited. Social Studies of Science, 23, 342–362.

    Article  Google Scholar 

  • Campanario, J. M. (1998). Peer review for journals as it stands today—Part 1. Science Communication, 19(1), 181–211.

    Article  Google Scholar 

  • Ceci, S. J., & Peters, D. (1984). How blind is blind peer review? American Psychologist, 39(12), 1491–1492.

    Article  Google Scholar 

  • Ceci, S. J., & Williams, W. M. (2011). Understanding current causes of women’s underrepresentation in science. Proceedings of the National Academy of Sciences, 108(8), 3157–3162.

    Google Scholar 

  • Chase, J. M. (1970). Normative criteria for scientific publication. American Sociologist, 5(3), 262–265.

    Google Scholar 

  • Cho, A. H., Johnson, S. A., Schuman, C. E., Adler, J. M., Gonzalez, O., Graves, S. J., et al. (2014). Women are underrepresented on the editorial boards of journals in environmental biology and natural resource management. PeerJ, 2, e542. doi:10.7717/peerj.542.

    Article  Google Scholar 

  • Chubin, D. E., & Hackett, E. J. (1990). Peerless science: Peer review and U.S. science policy. Stony Brook, NY: State University Press of New York.

    Google Scholar 

  • Cicchetti, D. V. (1991). The reliability of peer review for manuscript and grant submissions: A cross-disciplinary investigation. Behavioral and Brain Sciences, 14(1), 119–186.

    Article  Google Scholar 

  • Cicchetti, D. V., & Eron, L. D. (1979). The reliability of manuscript reviewing for the Journal of Abnormal Psychology. Proceedings of the American Statistical Association, 22, 596–600.

    Google Scholar 

  • Cleary, F. R., & Edwards, D. J. (1960). The origins of the contributors to the A.E.R. during the ‘fifties. The American Economic Review, 50(5), 1011–1014.

    Google Scholar 

  • Cole, S. (1992). Making science: Between nature and reality. Cambridge, MA: Harvard University Press.

    Google Scholar 

  • Cole, S., & Cole, J. R. (1973). Social Stratification in Science. Chicago, IL: University of Chicago Press.

    Google Scholar 

  • Cole, S., Cole, J. R., & Simon, G. (1981). Chance and consensus in peer review. Science, 214(4523), 881–886.

    Article  Google Scholar 

  • Cole, S., Rubin, L., & Cole, J. R. (1978). Peer review in the National Science Foundation: Phase one of a study. Washington, DC: National Academy of Sciences.

    Google Scholar 

  • Coser, L. A., Kadushin, C., & Powell, W. W. (1982). Books: The culture and commerce of publishing. Chicago, IL: University of Chicago Press.

    Google Scholar 

  • Coursol, A., & Wagner, E. E. (1986). Effect of positive findings on submission and acceptance rates: A note on meta-analysis bias. Professional Psychology: Research and Practice, 17(2), 136–137.

    Article  Google Scholar 

  • Crane, D. (1965). Scientists at major and minor universities: A study of productivity and recognition. American Sociological Review, 30, 699–714.

    Article  Google Scholar 

  • Crane, D. (1967). The gatekeepers of science: Some factors affecting the selection of articles for scientific journals. The American Sociologist, 2(4), 195–201.

    Google Scholar 

  • Cummings, L., Frost, P. J., & Vakil, T. F. (1985). The manuscript review process: A view from the inside on coaches, critics, and special cases. In L. L. Cummings & P. J. Frost (Eds.), Publishing in the organizational sciences (pp. 469–508). Homewood, IL: Irwin.

    Google Scholar 

  • Dar, R. (1987). Another look at Meehl, Lakatos, and the scientific practices of psychologists. American Psychologist, 42(2), 145–151.

    Article  Google Scholar 

  • De Vries, R., Anderson, M. S., & Martinson, B. C. (2006). Normal misbehavior: Scientists talk about the ethics of research. Journal of Empirical Research on Human Research Ethics, 1(1), 43–50.

    Article  Google Scholar 

  • Dewitt, N., & Turner, R. (2001). Bad peer reviewers. Nature, 413(6852), 93.

    Article  Google Scholar 

  • Dressler, A. J. (1970). Nobel prizes: 1970 awards—Physics. Science, 170, 604–606.

    Article  Google Scholar 

  • Edwards, L.N., & Ferber, M.A. (1986). Journal reviewing practices and the progress of women in the economics profession: Is there a relationship? Newsletter of the Committee on the Status of Women in the Economics Profession, 1–7.

    Google Scholar 

  • Emerson, G. B., Winston, J. W., Wolf, F., Heckman, J., Brand, R., & Leopold, S. S. (2010). Testing for the presence of positive-outcome bias in peer review: A randomized controlled trial. Archives of Internal Medicine, 170(21), 1934–1939.

    Article  Google Scholar 

  • Epstein, W. (1990). Confirmational response bias among social work journals. Science, Technology, and Human Values, 15(1), 9–38.

    Google Scholar 

  • Ernst, E., & Kienbacher, T. (1991). Chauvinism. Nature, 353(6336), 560.

    Article  Google Scholar 

  • Ernst, E., Resch, K. L., & Uher, E. M. (1992). Reviewer bias. Annals of Internal Medicine, 116(1), 958.

    Article  Google Scholar 

  • Fairweather, J. S. (2002). The ultimate faculty evaluation: Promotion and tenure decisions. New Directions for Institutional Research, 114, 97–108.

    Article  Google Scholar 

  • Fanelli, D. (2010). “Positive” results increase down the hierarchy of the sciences. PLos One, 5(4), e10068. doi:10.1371/journal.pone.0010068.

    Article  Google Scholar 

  • Ferber, M. A., & Teiman, M. (1980). Are women economists at a disadvantage in publishing journal articles? Eastern Economic Journal, 6(3–4), 189–193.

    Google Scholar 

  • Finke, R. A. (1990). Recommendations for contemporary editorial practices. American Psychologist, 45(5), 669–670.

    Article  Google Scholar 

  • Fisher, M., Freidman, S. B., & Strauss, B. (1994). The effects of blinding on acceptance of research papers by peer review. JAMA, 272(2), 143–148.

    Article  Google Scholar 

  • Fox, M. F. (1994). Scientific misconduct and editorial peer review processes. Journal of Higher Education, 65(3), 298–309.

    Article  Google Scholar 

  • Friedkin, N. (1983). Horizons of observability and limits of informal control in organizations. Social Forces, 62(1), 54–77.

    Article  Google Scholar 

  • Fyfe, J. J. (1994). Cops and robbers in academe: Editing Justice Quarterly. In R. Simons & J. J. Fyfe (Eds.), Editors as gatekeepers: Getting published in the social sciences (pp. 59–72). Boston, MA: Rowman and Littlefield.

    Google Scholar 

  • Garfield, E. (1955). Citation indexes to science: A new dimension in documentation through association of ideas. Science, 122(3159), 108–111.

    Article  Google Scholar 

  • Garfunkel, J. M., Ulshen, M. H., Hamrick, H. J., & Lawson, E. (1994). Effect of institutional prestige on reviewers’ recommendations and editorial decisions. JAMA, 272(2), 137–138.

    Article  Google Scholar 

  • General Accounting Office. (1994). Peer review: Reforms needed to ensure fairness in federal agency grant selection. Washington, DC: General Accounting Office.

    Google Scholar 

  • Geiger, R. L. (1986). To advance knowledge: The growth of American research universities, 1900–1940. New York, NY: Oxford University Press.

    Google Scholar 

  • Gilbert, J. R., Williams, E. S., & Lundberg, G. D. (1994). Is there a gender bias in JAMA’s review process? JAMA, 272(2), 137–138.

    Article  Google Scholar 

  • Gillespie Jr., G. W., Chubin, D. E., & Kurzon, G. M. (1985). Experience with NIH peer review: Researchers’ cynicism and desire for change. Science, Technology, and Human Values, 10(3), 44–54.

    Article  Google Scholar 

  • Goldberg, P. (1968). Are women prejudiced against women? Trans-Action, 5(5), 28–30.

    Google Scholar 

  • Goodrich, D. W. (1945). An analysis of manuscripts received by the editors of the American Sociological Review from May 1, 1944 to September 1, 1945. American Sociological Review, 10(6), 716–725.

    Article  Google Scholar 

  • Greaves, S., Scott, J., Clarke, M., Miller, L., Hannay, T., Thomas, A., & Campbell, P. (2006). Overview: Nature’s trial of open peer review. Nature. doi:10.1038/nature05535.

    Google Scholar 

  • Greenwald, A. G. (1975). Consequences of prejudice against the null hypothesis. Psychological Bulletin, 82, 1–20.

    Article  Google Scholar 

  • Grouse, L. D. (1981). The Ingelfinger rule. Journal of the American Medical Association, 245(4), 375–376.

    Article  Google Scholar 

  • Guest, A. M. (1994). Gatekeeping among the demographers. In R. Simons & J. J. Fyfe (Eds.), Editors as gatekeepers: Getting published in the social sciences (pp. 85–106). Boston, MA: Rowman and Littlefield.

    Google Scholar 

  • Guetzkow, J., Lamont, M., & Mallard, G. (2004). What is originality in the humanities and social sciences? American Sociological Review, 69(2), 190–212.

    Article  Google Scholar 

  • Gustin, B. H. (1973). Charisma, recognition, and the motivation of scientists. American Journal of Sociology, 78(5), 1119–1134.

    Google Scholar 

  • Hall, R. A., & Hall, M. B. (Eds.) (1966). The correspondence of Henry Oldenburg, Vol. 2. Madison, WI: University of Wisconsin Press.

    Google Scholar 

  • Hamermesh, D. S. (1994). Facts and myths about refereeing. The Journal of Economic Perspectives, 8(1), 153–163.

    Article  Google Scholar 

  • Hargens, L. L. (1988). Scholarly consensus and journal rejection rates. American Sociological Review, 53, 139–151.

    Article  Google Scholar 

  • Hargens, L. L., & Herting, J. R. (1990). Neglected considerations in the analysis of agreement among journal referees. Scientometrics, 19, 91–106.

    Google Scholar 

  • Hersen, M., & Miller, D. J. (1992). Future directions: A modest proposal. In D. J. Miller & M. Hersen (Eds.), Research fraud in the behavioral and biomedical sciences (pp. 225–244). New York, NY: John Wiley.

    Google Scholar 

  • Harnad, S. (1982). Peer commentary on peer review: A case study in scientific quality control. New York, NY: Cambridge University Press.

    Google Scholar 

  • Hearn, J. C., & Anderson, M. S. (2002). Conflict in academic departments: An analysis of disputes over faculty promotion and tenure. Research in Higher Education, 43(5), 503–529.

    Article  Google Scholar 

  • Hermanowicz, J. C. (1998). The stars are not enough: Scientists—Their passions and professions. Chicago, IL: University of Chicago Press.

    Google Scholar 

  • Hermanowicz, J. C. (2009). Lives in science: How institutions shape academic careers. Chicago, IL: University of Chicago Press.

    Book  Google Scholar 

  • Hermanowicz, J. C. (2016a). The proliferation of publishing: Economic rationality and ritualized productivity in a neoliberal era. American Sociologist, 47, 174–191.

    Article  Google Scholar 

  • Hermanowicz, J. C. (2016b). Universities, academic careers, and the valorization of ‘shiny things.’. Research in the Sociology of Organizations, 46, 303–328.

    Article  Google Scholar 

  • Herrera, A. J. (1999). Language bias discredits the peer-review system. Nature, 297(6719), 467.

    Article  Google Scholar 

  • Hess, D. J. (1997). Science studies: An advanced introduction. New York, NY: New York University Press.

    Google Scholar 

  • Hirschauer, S. (2010). Editorial judgments: A praxeology of ‘voting’ in peer review. Social Studies of Science, 40(1), 71–103.

    Article  Google Scholar 

  • Hood, J. (1985). The lone scholar myth. In M. F. Fox (Ed.), Scholarly writing and publishing (pp. 111–124). Boulder, CO: Westview Press.

    Google Scholar 

  • Horton, R. (2000). Genetically modified food: consternation, confusion, and crack-up. Medical Journal of Australia, 177, 148–149.

    Google Scholar 

  • Jacobs, J. A. (2013). In defense of disciplines: Interdisciplinarity and specialization in the research university. Chicago,IL: University of Chicago Press.

    Google Scholar 

  • Jacobs, J. A., & Winslow, S. E. (2004). Overworked faculty: Job stresses and family demands. The Annals of the American Academy of Political and Social Science, 596(1), 104–129.

    Article  Google Scholar 

  • Jauch, L. R., & Wall, J. L. (1989). What they do when they get your manuscript: A survey of Academy of Management reviewer practices. Academy of Management Journal, 32(1), 157–173.

    Article  Google Scholar 

  • Judson, H. F. (1994). Structural transformations of the sciences and the end of peer review. Journal of the American Medical Association, 272(2), 92–94.

    Article  Google Scholar 

  • Juhasz, S., Calvert, E., Jackson, T., Kronick, D., & Shipman, J. (1975). Acceptance and rejection of manuscripts. IEEE Transactions on Professional Communication, 18(3), 177–185.

    Article  Google Scholar 

  • Katz, D. S., Proto, A. V., & Olmsted, W. W. (2002). Incidence and nature of unblinding by authors: Our experience at two radiology journals with double-blinded peer review policies. American Journal of Roentgenology, 179(6), 1415–1417.

    Article  Google Scholar 

  • Kerr, S., Tolliver, J., & Petree, D. (1977). Manuscript characteristics which influence acceptance for management and social science journals. Academy of Management Journal, 20(1), 132–141.

    Article  Google Scholar 

  • Knight, J. (2003). Negative results: Null and void. Nature, 422(6932), 554–555.

    Article  Google Scholar 

  • Knorr-Cetina, K. (1981). The manufacture of knowledge: An essay on the constructivist and contextual nature of science. Oxford,UK: Pergamon Press.

    Google Scholar 

  • Kravitz, D. J., & Baker, C. I. (2011). Toward a new model of scientific publishing: Discussion and a proposal. Frontiers in Computational Neuroscience, 5(55), 1–12.

    Google Scholar 

  • Kuhn, T. S. (1962). The structure of scientific revolutions. Chicago, IL: University of Chicago Press.

    Google Scholar 

  • Lamont, M. (2009). How professors think: Inside the curious world of academic judgment. Cambridge, MA: Harvard University Press.

    Book  Google Scholar 

  • Langfeldt, L. (2001). The decision-making constraints and processes of grant peer review, and their effects on the review outcome. Social Studies of Science, 31(6), 820–841.

    Article  Google Scholar 

  • Lawrence, J. H., Celis, S., & Ott, M. (2014). Is the tenure process fair? What faculty think. Journal of Higher Education, 85(2), 155–192.

    Article  Google Scholar 

  • Leahey, E., & Cain, C. L. (2013). Straight from the source: Accounting for scientific success. Social Studies of Science, 43(6), 927–951.

    Article  Google Scholar 

  • Lee, C. J., & Schunn, C. D. (2011). Social biases and solutions for procedural objectivity. Hypatia: A Journal of Feminist Philosophy, 26(2), 352–373.

    Article  Google Scholar 

  • Lee, C. J., Sugimoto, C. R., Zhang, G., & Cronin, B. (2013). Bias in peer review. Journal of the American Society for Information Science and Technology, 64(1), 2–17.

    Article  Google Scholar 

  • Levenson, H., Burford, B., Bonno, B., & Davis, L. (1975). Are women still prejudiced against women? A replication and extension of Goldberg’s study. Journal of Psychology, 89, 67–71.

    Article  Google Scholar 

  • Liebert, R. J. (1976). Productivity, favor, and grants among scholars. American Journal of Sociology, 82(3), 664–673.

    Article  Google Scholar 

  • Link, A. M. (1998). US and non-US submissions. Journal of the American Medical Association, 280(3), 246–247.

    Google Scholar 

  • Lloyd, M. E. (1990). Gender factors in reviewer recommendations for manuscript publication. Journal of Applied Behavior Analysis, 23(4), 539–543.

    Article  Google Scholar 

  • Long, J. S., Allison, P. D., & McGinnis, R. (1993). Rank advancement in academic careers: Sex differences and the effects of productivity. American Sociological Review, 58, 703–722.

    Article  Google Scholar 

  • Long, J. S., & Fox, M. F. (1995). Scientific careers: Universalism and particularism. Annual Review of Sociology, 21, 45–71.

    Article  Google Scholar 

  • Loonen, M. P. J., Hage, J. J., & Kon, M. (2005). Who benefits from peer review? An analysis of the outcome of 100 requests for review by Plastic and Reconstructive Surgery. Plastic and Reconstructive Surgery, 116(5), 1461–1472.

    Article  Google Scholar 

  • Mahoney, M. J. (1977). Publication prejudices: An experimental study of confirmatory bias in the peer review system. Cognitive Therapy and Research, 1(2), 161–175.

    Article  Google Scholar 

  • McAfee, R. P. (2014). Edifying editors. In M. Szenberg & L. Ramrattan (Eds.), Secrets of Economics Editors (pp. 33–44). Cambridge, MA: MIT Press.

    Google Scholar 

  • McLachlan, J. C. (2010). Integrative medicine and the point of credulity. BMJ, 341, c6979.

    Article  Google Scholar 

  • McNutt, R. A., Evans, A. T., Fletcher, R. H., & Fletcher, S. W. (1990). The effects of blinding on the quality of review. JAMA, 263(10), 1371–1376.

    Article  Google Scholar 

  • Merton, R. K. (1973a). The normative structure of science. In N. W. Storer (Ed.), The sociology of science: Theoretical and empirical investigations (pp. 267–278). Chicago, IL: University of Chicago Press (Article originally published in 1942).

    Google Scholar 

  • Merton, R. K. (1973b). Priorities in scientific discovery. In N. W. Storer (Ed.), The sociology of science: Theoretical and empirical investigations (pp. 286–324). Chicago, IL: University of Chicago Press (Article originally published in 1957).

    Google Scholar 

  • Merton, R. K. (1973c). The Matthew effect in science. In N. W. Storer (Ed.), The sociology of science: Theoretical and empirical investigations (pp. 439–459). Chicago, IL: University of Chicago Press (Article originally published in 1968).

    Google Scholar 

  • Merton, R. K., & Zuckerman, H. (1973). Age, aging, and the age structure in science. In N. W. Storer (Ed.), The sociology of science: Theoretical and empirical investigations (pp. 497–559). Chicago, IL: University of Chicago Press (Article originally published in 1972).

    Google Scholar 

  • Moossy, J., & Moossy, Y. R. (1985). Anonymous authors, anonymous referees: An editorial explanation. Journal of Neuropathology and Experimental Neurology, 44(3), 225–228.

    Article  Google Scholar 

  • Morton, H. C., & Price, A. J. (1986). The ACLS survey of scholars: Views on publications, computers, libraries. Scholarly Communication, 5, 1–16.

    Google Scholar 

  • Mulkay, M. (1980). Interpretation and the use of rules: The case of the norms of science. In T. F. Gieryn (Ed.), Science and social structure: A Festschrift for Robert K. Merton (pp. 111–125). New York, NY: New York Academy of Sciences.

    Google Scholar 

  • National Science Board. (2010). Science and engineering indicators 2010. Arlington, VA: National Science Foundation.

    Google Scholar 

  • Olson, C. M., Rennie, D., Cook, D., Dickersin, K., Flanagin, A., Hogan, J., … Pace, B. (2002). Publication bias in editorial decision making. Journal of the American Medical Association, 287(21), 2825–2828.

    Google Scholar 

  • O’Meara, K. A. (2004). Beliefs about post-tenure review: The influence of autonomy, collegiality, career stage, and institutional context. Journal of Higher Education, 75(2), 178–202.

    Article  Google Scholar 

  • Opthof, T., Coronel, R., & Janse, M. (2002). The significance of the peer review process against the background of bias: Priority ratings of reviewers and editors and the prediction of citation, the role of geographical bias. Cardiovascular Research, 56(3), 339–346.

    Article  Google Scholar 

  • Orr, R. & Kassab, J. (1965). Peer group judgments on scientific merit: Editorial refereeing. Presentation to the Congress of the International Federation for Documentation, Washington, DC. October 15.

    Google Scholar 

  • Paludi, M. A., & Bauer, W. D. (1983). Goldberg revisited: What’s in an author’s name? Sex Roles, 9(3), 387–390.

    Article  Google Scholar 

  • Patriquin, L., Bensimon, E. M., Polkinghorne, D. E., Bauman, G., Bleza, M. G., Oliverez, P. M., & Soto, M. (2003). Posttenure review: The disparity between intent and implementation. Review of Higher Education, 26(3), 275–297.

    Article  Google Scholar 

  • Perna, L. W. (2001). Sex and race differences in faculty tenure and promotion. Research in Higher Education, 42(5), 541–567.

    Article  Google Scholar 

  • Perna, L. W. (2003). Studying faculty salary equity: A review of theoretical and methodological approaches. In J. C. Smart (Ed.), Higher education: Handbook of theory and research (Vol. 18, pp. 323–388).

    Chapter  Google Scholar 

  • Perna, L. W. (2005). Sex differences in faculty tenure and promotion: The contribution of family ties. Research in Higher Education, 46(3), 277–307.

    Article  Google Scholar 

  • Peters, D. P., & Ceci, S. J. (1982). Peer-review practices of psychological journals: The fate of accepted, published articles, submitted again. Behavioral and Brain Sciences, 5(2), 187–195.

    Google Scholar 

  • Polanyi, M. (1946). Science, faith, and society. Chicago, IL: University of Chicago Press.

    Google Scholar 

  • Powell, W. W. (1985). Getting into print: The decision-making process in scholarly publishing. Chicago, IL: University of Chicago Press.

    Google Scholar 

  • Power, M. (1997). The audit society: Rituals of verification. Oxford, UK: Oxford University Press.

    Google Scholar 

  • Price, D. J. (1975). Science since Babylon. New Haven, CT: Yale University Press.

    Google Scholar 

  • Primack, R. B., Ellwood, E., Miller-Rushing, A. J., Marrs, R., & Mulligan, A. (2009). Do gender, nationality, or academic age affect review decisions? An analysis of submission to the journal Biological Conservation. Biological Conservation, 142, 2415–2418.

    Article  Google Scholar 

  • Primack, R. B., & Marrs, R. (2008). Bias in the review process. Biological Conservation, 141(12), 2919–2920.

    Article  Google Scholar 

  • Ramos-Alvarez, M. M., Moreno-Fernandez, M. M., Valdes-Conroy, B., & Catena, A. (2008). Criteria of the peer review process for publication of experimental and quasi-experimental research in psychology: A guide for creating research papers. International Journal of Clinical and Health Psychology, 8(3), 751–764.

    Google Scholar 

  • Ripp, A. (1985). Peer review is alive and well in the United States. Science, Technology, and Human Values, 10(3), 82–86.

    Article  Google Scholar 

  • Rosenblatt, A., & Kirk, S. A. (1981). Recognition of authors in blind review of manuscripts. Journal of Social Science Research, 3(4), 383–394.

    Google Scholar 

  • Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638–641.

    Article  Google Scholar 

  • Rowson, R. C. (1994). A formula for successful scholarly publishing: Policy-oriented research and the humanities. In R. J. Simon & J. J. Fyfe (Eds.), Editors as gatekeepers: Getting published in the social sciences (pp. 195–208). Lanham, MD: Rowman & Littlefield Publishers.

    Google Scholar 

  • Roy, R. (1985). Funding science: The real defects of peer review and an alternative to it. Science, Technology, and Human Values, 10(3), 73–81.

    Article  Google Scholar 

  • Schuster, J. H., & Finkelstein, M. J. (2006). The American faculty: The restructuring of academic work and careers. Baltimore, MD: Johns Hopkins University Press.

    Google Scholar 

  • Seglen, P. O. (1997). Why the impact factor of journals should not be used for evaluating research. BMJ, 314(7079), 498–502.

    Article  Google Scholar 

  • Shadish Jr., W. R. (1989). The perception and evaluation of quality in science. In B. Gholson, W. R. Shadish Jr., R. A. Neimeyer, & A. C. Houts (Eds.), Psychology of science: Contributions to metascience (pp. 383–426). Cambridge, UK: Cambridge University Press.

    Chapter  Google Scholar 

  • Shadish Jr., W. R., Doherty, M., & Montgomery, L. M. (1989). How many studies in the file drawer? An estimate from the family/marital psychotherapy literature. Clinical Psychology Review, 9(5), 589–603.

    Article  Google Scholar 

  • Shatz, D. (2004). Peer review: A critical inquiry. Lanham, MD: Rowman & Littlefield.

    Google Scholar 

  • Shils, E. (1997). The order of learning: Essays on the contemporary university. New Brunswick, NJ: Transaction.

    Google Scholar 

  • Siler, K., Lee, K., & Bero, L. (2015). Measuring the effectiveness of scientific gatekeeping. Proceedings of the National Academy of Sciences, 112(2), 360–365.

    Article  Google Scholar 

  • Silverman, R. (1988). Peer judgment: An ideal typification. Knowledge: Creation, Diffusion, Utilization, 9, 362–382.

    Google Scholar 

  • Simon, R. J., Bakanic, V., & McPhail, C. (1986). Who complains to journal editors and what happens. Sociological Inquiry, 56(2), 259–271.

    Article  Google Scholar 

  • Smart, R. G. (1964). The importance of negative results in psychological research. Canadian Psychology, 5, 225–232.

    Article  Google Scholar 

  • Smigel, E. O., & Ross, H. L. (1970). Factors in the editorial decision. The American Sociologist, 5(1), 19–21.

    Google Scholar 

  • Smith, J. A., Nixon, R., Bueschen, A. J., Venable, D. D., & Henry, H. H. (2002). The impact of blinded versus unblended abstract review on scientific program content. Journal of Urology, 168(5), 2123–2125.

    Article  Google Scholar 

  • Sokal, A. D. (1996). A physicists experiments with cultural studies. Lingua Franca, 6(4), 62–64.

    Google Scholar 

  • Starbuck, W. H. (2003). Turning lessons into lemonade: Where is the value in peer reviews? Journal of Management Inquiry, 12(4), 344–351.

    Article  Google Scholar 

  • Sterling, T. (1959). Publication decisions and their possible effects on inferences drawn from tests of significance—Or vice versa. Journal of the American Statistical Association, 54(285), 30–34.

    Google Scholar 

  • Stossel, T. P. (1985). Refinement in biomedical communication: A case study. Science, Technology, & Human Values, 10(3), 39–43.

    Article  Google Scholar 

  • Strang, D., & Siler, K. (2015). Revising as reframing: Original submissions versus published papers in Administrative Science Quarterly, 2005–2009. Sociological Theory, 33(1), 71–96.

    Article  Google Scholar 

  • Sugimoto, C. R., Larivière, V., Ni, C., & Cronin, B. (2013). Journal acceptance rates: a cross-disciplinary analysis of variability and relationships with journal measures. Journal of Informatics, 7(4), 897–906.

    Google Scholar 

  • Tenopir, C., & King, D. W. (2009). The growth of journals publishing. In B. Cope & A. Phillips (Eds.), The future of the academic journal (pp. 105–124). Oxford, UK: Chandos Publishing.

    Chapter  Google Scholar 

  • Teplitskiy, M. (2015). Frame search and re-search: How quantitative sociological articles change during peer review. American Sociologist. doi:10.1008/s12108-015-9288-3.

    Google Scholar 

  • Thatcher, S. G. (1994). Listbuilding at university presses. In R. J. Simon & J. J. Fyfe (Eds.), Editors as gatekeepers: Getting published in the social sciences (pp. 209–258). Lanham, MD: Rowman & Littlefield Publishers.

    Google Scholar 

  • Travis, G., & Collins, H. M. (1991). New light on old boys: cognitive and institutional particularism in the peer review system. Science, Technology, & Human Values, 16(3), 322–341.

    Article  Google Scholar 

  • Tuchman, G. (2009). Wannabe U: Inside the Corporate University. Chicago, IL: University of Chicago Press.

    Book  Google Scholar 

  • Valkonen, L., & Brooks, J. (2011). Gender balance in Cortex acceptance rates. Cortex, 47, 763–770.

    Article  Google Scholar 

  • Wanderer, J. J. (1966). Academic origins of contributors to the “American Sociological Review”, 1955–1965. The American Sociologist, 1(5), 241–243.

    Google Scholar 

  • Ward, C. (1981). Prejudice against women: Who, when, why? Sex Roles, 7(2), 163–171.

    Article  Google Scholar 

  • Ware, M. & Monkman, M. (2008). Peer review in scholarly journals: Perspective of the scholarly community—An international study. Publishing Research Consortium.

    Google Scholar 

  • Waters, M. (1989). Collegiality, bureaucratization, and professionalization: A Weberian Analysis. American Journal of Sociology, 94(5), 945–972.

    Article  Google Scholar 

  • Webb, T. J., O’Hara, B., & Freckleton, R. P. (2008). Does double-blind review benefit female authors? Trends in Ecology and Evolution, 23(7), 351–353.

    Article  Google Scholar 

  • Weisse, A. B. (1986). Say it isn’t no: Positive thinking and the publication of medical research. Hospital Practice, 21(3), 23–25.

    Article  Google Scholar 

  • Weller, A. C. (2001). Editorial peer review: Its strengths and weaknesses. Medford, NJ: American Society for Information Science and Technology.

    Google Scholar 

  • Wessely, S. (1996). What do we know about peer review? Psychological Medicine, 26, 883–886.

    Article  Google Scholar 

  • Wessely, S., Brugha, T., Cowen, P., Smith, L., & Paykel, E. (1996). Do authors know who refereed their paper? A questionnaire survey. BMJ, 313(7066), 1185.

    Article  Google Scholar 

  • Whittaker, R. J. (2008). Journal review and gender equality: A critical comment on Budden et al. Trends in Ecology and Evolution, 23(9), 478–479.

    Article  Google Scholar 

  • Wilhite, A. W., & Fong, E. A. (2012). Coercive citation in academic publishing. Science, 335(6068), 542–543.

    Article  Google Scholar 

  • Wood, M., & Johnsrud, L. (2005). Post-tenure review: What matters to faculty. Review of Higher Education, 28(3), 393–420.

    Article  Google Scholar 

  • Yankauer, A. (1990). Who are the peer reviewers and how much do they review? JAMA, 263(10), 1338–1340.

    Article  Google Scholar 

  • Yankauer, A. (1991). How blind is blind review? American Journal of Public Health, 81(7), 843–845.

    Article  Google Scholar 

  • Youn, T. I. K., & Price, T. M. (2009). Learning from the experience of others: The evolution of faculty tenure and promotion rules in comprehensive institutions. Journal of Higher Education, 80(2), 204–237.

    Article  Google Scholar 

  • Ziman, J. M. (1968). Public knowledge: An essay concerning the social dimension of science. Cambridge, UK: Cambridge University Press.

    Google Scholar 

  • Zuckerman, H. (1977). Scientific elite: Nobel laureates in the United States. New York, NY: The Free Press.

    Google Scholar 

  • Zuckerman, H., & Merton, R. K. (1971). Patterns of evaluation in science: Institutionalization, structure and the functions of the referee system. Minerva, 9, 66–100.

    Article  Google Scholar 

  • Zuckerman, H. (1988). Sociology of science. In N. J. Smelser (Ed.), Handbook of sociology (pp. 511–574). Newbury Park, CA: Sage.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Education, The University of Nevada – Reno, 89557, Reno, NV, USA

    David R. Johnson

  2. Department of Sociology, The University of Georgia, 30602-1611, Athens, GA, USA

    Joseph C. Hermanowicz

Authors
  1. David R. Johnson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joseph C. Hermanowicz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David R. Johnson .

Editor information

Editors and Affiliations

  1. Department of Educational Policy and Leadership Studies N491 Lindquist Center, The University of Iowa, Iowa City, Iowa, USA

    Michael B. Paulsen

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Cite this chapter

Johnson, D.R., Hermanowicz, J.C. (2017). Peer Review: From “Sacred Ideals” to “Profane Realities”. In: Paulsen, M. (eds) Higher Education: Handbook of Theory and Research. Higher Education: Handbook of Theory and Research, vol 32. Springer, Cham. https://doi.org/10.1007/978-3-319-48983-4_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-48983-4_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-48982-7

  • Online ISBN: 978-3-319-48983-4

  • eBook Packages: EducationEducation (R0)

Publish with us

Policies and ethics

Search

Navigation