Skip to main content
SpringerLink
Log in

Peer Review as an Evolving Response to Organizational Constraint: Evidence from Sociology Journals, 1952–2018

  • Published:
The American Sociologist Aims and scope Submit manuscript

Abstract

Double-blind peer review is a central feature of the editorial model of most journals in sociology and neighboring social scientific fields, yet there is little history of how and when its main features developed. Drawing from nearly 70 years of annual reports of the editors of American Sociological Association journals, this article describes the historical emergence of major elements of editorial peer review. These reports and associated descriptive statistics are used to show that blind review, ad hoc review, the formal requirement of exclusive submission, routine use of the revise and resubmit decision, and common use of desk rejection developed separately over a period of decades. The article then argues that the ongoing evolution of the review model has not been driven by intellectual considerations. Rather, the evolution of peer review is best understood as the product of continuous efforts to steward editors’ scarce attention while preserving an open submission policy that favors authors’ interests.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. Because of this article’s concern with the development of journal peer review, it does not consider written reports or descriptive statistics from the ASA’s monograph series or Contemporary Sociology and Contexts, journals that employ different editorial models.

  2. Managing the editor’s workload has also spurred a range of other responses not directly concerned with reviewing itself. The appointment of deputy editors and proliferation of editorial assistants was common during the period when little of the administration or physical production of a journal could be automated. More recently, it has become common for journals to be operated by pairs or teams of editors-in-chief. All ASA journals were headed by a single editor-in-chief from the 1950s until the late 1990s, but since that time, most ASA journals have had at least one team of editors. Team editing now appears to be a firm norm at ASR.

  3. Article length has increased dramatically at both AJS and ASR. Between 1996, when Paula England had brought order to the revision pipeline (ASR 96), and 2013, when the journal reached its present size, page allocation increased nearly 40%, while the number of published articles dropped nearly 30%, meaning that average article length nearly doubled in less than 20 years.

  4. There is no simple way to reckon up the reviewing load in the field, but there can be little doubt that this burden has increased. Sociologists, like academic social scientists generally, face strong pressures to publish more frequently and at earlier stages in their career (Warren 2019). The number of refereed journals has expanded. The rise of English as an academic lingua franca, the growth in impact and performance metrics that strongly favor American journals, and the development of electronic submission models may all tend to increase international submissions to American journals that remain reliant on a mostly-domestic pool of reviewers. And the standard number of reviews per paper has increased: until 1990s, ASA journals typically sought two reviews per paper, but now commonly obtain three or four. The size of the sociology professoriate, however, is relatively stable.

  5. The following discussion implies no opinion of my own about the intellectual state of sociological theory or social psychology—it simply reports editors’ judgments about the need for change in these fields, and their own avowals of difficulty in securing the changes they envisioned.

References

  • Abbott, A. (1999). Department and discipline: Chicago sociology at one hundred. Chicago: University of Chicago Press.

    Book  Google Scholar 

  • Abbott, A. (2007). Notes on replication. Sociological Methods & Research, 36(2), 210–219.

    Article  Google Scholar 

  • Abbott, A. (2019). Career stage and publication in American academia. Sociologia, Problemas e Practicas, 90, 9–30.

    Google Scholar 

  • American Sociological Association. (2020). Doctorates Awarded in Sociology, by Gender. https://www.asanet.org/research-and-publications/research-sociology/trends/doctorates-awarded-sociology-gender. Accessed 8 July 2020.

  • Azar, O. H. (2005). The review process in economics: Is it too fast? Southern Economic Journal, 72(2), 482–491.

    Article  Google Scholar 

  • Bence, V., & Oppenheim, C. (2004). The role of academic journal publications in the UK research assessment exercise. Learned Publishing, 17(1), 53–68.

    Article  Google Scholar 

  • Bornmann, L., Nast, I., & Daniel, H.-D. (2008). Do editors and referees look for signs of scientific misconduct when reviewing manuscripts? A quantitative content analysis of studies that examined review criteria and reasons for accepting or rejecting manuscripts for publication. Scientometrics, 77(3), 415–432.

    Article  Google Scholar 

  • Borts, G. H. (1974). Report of the managing editor: American economic review. American Economic Review, 64(2), 476–482.

    Google Scholar 

  • Burnham, J. C. (1990). The evolution of editorial peer review. Journal of the American Medical Association, 263(10), 1323–1329.

    Article  Google Scholar 

  • Campanario, J. M. (2009). Rejecting and resisting Nobel class discoveries: Accounts by Nobel laureates. Scientometrics, 81(2), 549–565.

    Article  Google Scholar 

  • Childress, C. C. (2017). Under the cover: The creation, production, and reception of a novel. Princeton: Princeton University Press.

    Book  Google Scholar 

  • Childress, C. C., & Nault, J.-F. (2018). Encultured biases: The role of products in pathways to inequality. American Sociological Review, 84(1), 115–141.

    Article  Google Scholar 

  • Cotton, C. (2013). Submission fees and response times in academic publishing. American Economic Review, 103(1), 501–509.

    Article  Google Scholar 

  • Feeney, M. K., Carson, L., & Dickinson, H. (2018). Power in editorial positions: A feminist critique of public administration. Public Administration Review, 79(1), 46–55.

    Article  Google Scholar 

  • Fourcade, M. (2016). Ordinalization. Sociological Theory, 34(3), 175–195.

    Article  Google Scholar 

  • Freese, J. (2007). Replication standards for quantitative social science: Why not sociology? Sociological Methods & Research, 36(2), 153–172.

    Article  Google Scholar 

  • Gans, J. S., & Shepherd, G. B. (1994). How are the mighty fallen: Rejected classic articles by leading economists. Journal of Economic Perspectives, 8(1), 165–179.

    Article  Google Scholar 

  • Gerber, A., & Childress, C. C. (2015). The MFA in creative writing: The uses of a ‘useless’ credential. Professions and Professionalism, 5(2), 1–16.

    Google Scholar 

  • Germano, W. (2016). Getting it published. Chicago: University of Chicago Press.

    Book  Google Scholar 

  • Gurley, J. G. (1967). Report of the managing editor for the year ending December 1966. American Economic Review, 57(2), 702–707.

    Google Scholar 

  • Haley, B. F. (1955). Report of the managing editor for the year ending December 1954. American Economic Review, 45(2), 671–674.

    Google Scholar 

  • Johnson, D. R., & Hermanowicz, J. C. (2017). Peer review: From ‘sacred ideals’ to ‘profane realities’. In M. B. Paulsen (Ed.), Higher education: Handbook of theory and research (pp. 485–527). Cham: Springer.

    Chapter  Google Scholar 

  • Jones, C. O., & Beck, P. A. (1977). Report of the American political science review, 1976-77. PS, 10(4), 448–456.

    Google Scholar 

  • Jubb, M. (2016). Peer review: The current landscape and future trends. Learned Publishing, 29, 13–21.

    Article  Google Scholar 

  • König, T., & Ropers, G. (2018). Gender and editorial outcomes at the American Political Science Review. PS: Political Science and Politics, 51(3), 849–853.

    Google Scholar 

  • Lamont, M. (2009). How professors think: Inside the curious world of academic judgment. Cambridge: Harvard University Press.

    Book  Google Scholar 

  • Lesser, W. (1999). The Amateur: An Independent Life in Letters. New York: Pantheon Books.

    Google Scholar 

  • Luey, B. (Ed.). (2008). Revising your dissertation: Advice from leading editors. Berkeley: University of California Press.

    Google Scholar 

  • Martin-Sardesai, A., Guthrie, J., Tooley, S., & Chaplin, S. (2019). History of research performance measurement systems in the Australia Higher Education Sector. Accounting History, 24(1), 40–61.

    Article  Google Scholar 

  • Merriman, B. (2017a). The promise and problems of unsolicited submission. Michigan Quarterly Review, 56(2), 308–320.

    Google Scholar 

  • Merriman, B. (2017b). The editorial meeting at a little magazine: An ethnography of group judgment. Journal of Contemporary Ethnography, 46(4), 440–463.

    Article  Google Scholar 

  • Mulligan, A., Hall, L., & Raphael, E. (2013). Peer review in a changing world: An international study measuring the attitudes of researchers. Journal of the American Society for Information Science and Technology, 64(1), 132–161.

    Article  Google Scholar 

  • Nosek, B. A., Aarts, A. A., Anderson, J. E., Barry, H., et al. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716.

    Article  Google Scholar 

  • Petersen, J. (2017). How innovative are editors?: Evidence across journals and disciplines. Research Evaluation, 26(3), 256–268.

    Article  Google Scholar 

  • Peterson, D. A. M. (2018). Author behavior and editorial outcomes at Political Behavior. PS: Political Science and Politics, 51(3), 866–869.

    Google Scholar 

  • Powell, W. W. (1985). Getting into print: The decision-making process in scholarly publishing. Chicago: University of Chicago Press.

    Google Scholar 

  • Samuels, D. (2018). Gender and editorial outcomes at Comparative Political Studies. PS: Political Science and Politics, 51(3), 854–858.

    Google Scholar 

  • Smigel, E. O., & Ross, H. L. (1970). Factors in the editorial decision. The American Sociologist, 5(1), 19–21.

    Google Scholar 

  • Starbuck, W. H. (2005). How much better are the Most-prestigious journals? The statistics of academic publication. Organization Science, 16(2), 180–200.

    Article  Google Scholar 

  • Starbuck, W. H. (2016). 60th anniversary essay: How journals could improve research practices in social science. Administrative Science Quarterly, 61(2), 165–183.

    Article  Google Scholar 

  • Teplitskiy, M. (2016). Frame search and re-search: How quantitative sociology articles change during peer review. The American Sociologist, 47(2–3), 264–288.

    Article  Google Scholar 

  • Teplitskiy, M., & Bakanic, V. (2016). Do peer reviews predict impact? Evidence from the American Sociological Review, 1978 to 1982. Socius, 2, 1–13.

    Article  Google Scholar 

  • Warren, J. R. (2019). How much do you have to publish to get a job in a top sociology department? Or to get tenure? Trends over a generation. Sociological Science, 6, 172–196.

    Article  Google Scholar 

  • Weisshaar, K. (2017). Publish and perish? An assessment of gender gaps in promotion to tenure in academia. Social Forces, 96(2), 529–560.

    Article  Google Scholar 

  • Weller, A. C. (2001). Editorial peer review: Its strengths and weaknesses. Medford: American Society for Information Science and Technology.

    Google Scholar 

Download references

Acknowledgments

I thank Andrew Abbott, Daniel Alvord, and the anonymous reviewers for their comments on this article. I dedicate this work to the memory of George Frederickson—a generous colleague and a great editor.

Author information

Authors and Affiliations

  1. School of Public Affairs & Administration, University of Kansas, 4060 Wescoe Hall, 1445 Jayhawk Blvd., Lawrence, KS, 66045, USA

    Ben Merriman

Authors
  1. Ben Merriman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ben Merriman.

Ethics declarations

Conflict of Interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix. Bibliographic Information for Editors’ Reports

Appendix. Bibliographic Information for Editors’ Reports

This article uses a simplified reference system for the editorial reports: reports are cited by journal initials and the last two numerals of the year of report publication.

Journal Initials

ASR:

American Sociological Review

JHSB:

Journal of Health and Social Behavior

SPQ:

Sociometry/Social Psychology/Social Psychology Quarterly

SM:

Sociological Methodology

ST:

Sociological Theory

SOE:

Sociology of Education

SOC:

Socius

TS:

Teaching Sociology

TAS:

The American Sociologist

Sources of Reports

Editors’ reports were published in the American Sociological Review from 1952 to 1964. Volume, issue and page numbers for the reports are as follows:

  • 1952 17(6):780

  • 1953 18(6):679-680

  • 1955 20(6):730-731

  • 1956 21(6):759-761

  • 1957 22(6):735-736

  • 1958 23(6):700-701

  • 1959 24(6):873-875

  • 1960 25(6):940-942

  • 1961 26(6):983-985

  • 1962 27(6):918-920

  • 1963 28(6):1005-1007

  • 1964 29(6):898-899

Editors’ reports were published in The American Sociologist from 1965 to 1972.

Volume, issue and page numbers for the reports are as follows:

  • 1965 1(1):36-37

  • 1966 1(5):283-285

  • 1967 2(4):234-237

  • 1968 3(4):330-334

  • 1969 4(4):348-351

  • 1970 5(4):399-402

  • 1971 6(4):345-349

  • 1972 7(7):26-27

Editors’ reports were published in Footnotes from 1973 to 2005.

Volume, issue and page numbers for the reports are as follows:

  • 1973 1(6):13-14

  • 1974 2(6):12-15

  • 1975 3(6):12-13

  • 1976 No reports published due to change in reporting deadlines

  • 1977 5(3):9

  • 1978 6(3):8-10

  • 1979 7(2):6-7

  • 1980 8(3):10-11

  • 1981 9(3):14-15

  • 1982 10(3):11-12

  • 1983 11(6):13-14

  • 1984 12(4):10-11

  • 1985 13(4):15-16

  • 1986 14(5):13-16

  • 1987 15(5):14-16

  • 1988 16(3):13-16

  • 1989 17(4):10-13

  • 1990 18(4):14-16

  • 1991 19(4):13-16

  • 1992 20(5):17-20

  • 1993 21(5):20-22

  • 1994 22(6):23

  • 1995 23(4):13-15

  • 1996 24(4):9-11

  • 1997 25(4):13-15

  • 1998 26(3):10-12

  • 1999 27(4):13-15

  • 2000 28(3):14-14

  • 2001 29(3):9-11

  • 2002 30(3):13-15

  • 2003 31(4):12-15

  • 2004 32(4):12-15

  • 2005 33(4):12-15

Since 2005, editors’ reports have been published on the American Sociological Association website. This change also led to a shift in the timing of reports: where the printed reports described the editorial activity of the previous calendar year, online reports are dated according to the calendar year of activity summarized.

https://www.asanet.org/research-and-publications/journal-resources/editors-reports/previous-editors-reports.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Merriman, B. Peer Review as an Evolving Response to Organizational Constraint: Evidence from Sociology Journals, 1952–2018. Am Soc 52, 341–366 (2021). https://doi.org/10.1007/s12108-020-09473-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12108-020-09473-x

Keywords

Search

Navigation