Skip to main content
SpringerLink
Log in

(Why) Do you trust your reviewers? Influence behaviors, trustworthiness, and commitment to peer review

(Warum) Trauen Sie Ihren Gutachtern? Einflussnahme, Vertrauenswürdigkeit und Commitment gegenüber Peer Review

  • Originalartikel
  • Published:
Managementforschung

Abstract

Peer review in academic publishing relies on the voluntary engagement of scholars who are, at best, committed to that practice. Current debates on peer review suggest that this commitment is diminishing. Conceptualizing peer review as an instance of social exchange allows us to propose a conceptual model of commitment to peer review and test it by means of a structural equation analysis. Our empirical study is based on survey data from the social sciences (n = 359). Results show that authors are more committed to the practice of peer review if reviewers base their recommendations on rational arguments so that authors can trust them for their competence. By contrast, benevolent reviewers who try to collaborate with authors are not effective in fostering trust and commitment. Within the limitations of our data and with regard to reviewers’ behaviors and characteristics, we cannot support sweeping criticisms of the operational reliability of academic journals.

Zusammenfassung

Peer Review im akademischen Publizieren basiert auf dem freiwilligen Engagement von Wissenschaftlerinnen und Wissenschaftlern, die, im besten Fall, ein hohes Commitment zu dieser Praxis aufweisen. Aktuelle Debatten über den Peer-Review-Prozess deuten darauf hin, dass dieses Commitment abnimmt. Basierend auf der Social-Exchange-Theorie entwickeln wir ein konzeptionelles Modell, das wir mithilfe eines Strukturgleichungsmodells testen. Unsere empirische Studie basiert auf einer Umfrage in den Sozialwissenschaften (n = 359). Die Ergebnisse zeigen, dass Autoren ein höheres Commitment zur Praxis des Peer Review aufweisen, wenn Gutachter ihre Kommentare auf rationalen Argumenten gründen und die Autoren ihnen aufgrund dieser Kompetenz vertrauen können. Sind Gutachter allerdings zu wohlwollend oder versuchen sie mit den Autoren zu kollaborieren, erwecken sie kein Vertrauen und vermindern das Commitment der Autoren. Im Hinblick auf die Eigenschaften und Verhaltensweisen von Gutachtern und unter Berücksichtigung der Limitationen unserer empirischen Studie können wir die Kritik an der Funktionsweise des Peer Reviews in wissenschaftlichen Zeitschriften nicht bestätigen.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1

Similar content being viewed by others

References

  • Aguinis H, Shapiro D, Antonacopoulou EP, Cummings TG (2014) Scholarly impact: a pluralist conceptualization. Acad Manage Learn Educ 13:1–17

    Google Scholar 

  • Antonakis J, Bendahan S, Jacquart P, Lalive R (2010) On making causal claims: a review and recommendations. Leadersh Q 21:1086–1120

    Google Scholar 

  • Aryee S, Budhwar, Chen ZX (2002) Trust as a mediator of the relationship between organizational justice and work outcomes: test of a social exchange model. J Organ Behav 23:267–286

    Google Scholar 

  • Bandura A (1977) Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev 84:191–215

    Google Scholar 

  • Bedeian AG (2003) The manuscript review process: the proper roles of authors, referees, and editors. J Manag Inq 12:331–338

    Google Scholar 

  • Bedeian AG (2004) Peer review and the social construction of knowledge in the management discipline. Acad Manag Learn Educ 3:198–216

    Google Scholar 

  • Berkenkotter C (1995) The power and the perils of peer review. Rhetor Rev 13:245–248

    Google Scholar 

  • Blau PM (1964) Exchange and power in social life. John Wiley, New York

    Google Scholar 

  • Bollen K (1990) Overall fit in covariance structure models: two types of sample size effects. Psychol Bull 107:256–259

    Google Scholar 

  • Bornmann L, Wolf M, Daniel H‑D (2011) Closed versus open reviewing of journal manuscripts: how far do comments differ in language use? Scientometrics 91:843–856

    Google Scholar 

  • Bornmann L, Herich H, Joos H, Daniel H‑D (2012) In public peer review of submitted manuscripts, how do reviewer comments differ from comments written by interested members of the scientific community? A content analysis of comments written for atmospheric chemistry and physics. Scientometrics 93:915–929

    Google Scholar 

  • Bradley JV (1982) Editorial overkill. Bull Psychon Soc 19:271–274

    Google Scholar 

  • Campanario JM (1998a) Peer review for journals as it stands today—part 1. Sci Commun 19:181–211

    Google Scholar 

  • Campanario JM (1998b) Peer review for journals as it stands today—part 2. Sci Commun 19:277–306

    Google Scholar 

  • Cardy RL, Dobbins GH (1986) Affect and appraisal accuracy: liking as an integral dimension in evaluating performance. J Appl Psychol 71:672–678

    Google Scholar 

  • Chubin DE, Hackett EJ (1990) Peerless science: peer review and U.S. science policy. Sunny Press, New York

    Google Scholar 

  • Clark T, Wright M (2007) Reviewing journal rankings and revisiting peer reviews: editorial perspectives. J Manag Stud 44:612–621

    Google Scholar 

  • Colquitt JA, Scott BA, LePine JA (2007) Trust, trustworthiness, and trust propensity: a meta-analytic test of their unique relationships with risk taking and job performance. J Appl Psychol 92:909–927

    Google Scholar 

  • Cropanzano R, Mitchell MS (2005) Social exchange theory: an interdisciplinary review. J Manage 31:874–900

    Google Scholar 

  • Crossman A, Lee-Kelley L (2004) Trust, commitment and team working: the paradox of virtual organizations. Glob Netw (Oxf) 4:375–390

    Google Scholar 

  • Cullen JB, Johnson JL, Sakano T (2000) Success through commitment and trust: the soft side of strategic alliance management. J World Bus 35:223–240

    Google Scholar 

  • D’Andrea R, O’Dwyer JP (2017) Can editors save peer review from peer reviewers? PLoS ONE 12:18–20

    Google Scholar 

  • Dulac T, Coyle-Shapiro JA, Henderson DJ, Wayne SJ (2008) Not all responses to breach are the same: the interconnection of social exchange and psychological contract processes in organizations. Acad Manage J 51:1079–1098

    Google Scholar 

  • Eisenhart M (2002) The paradox of peer review: admitting too much or allowing too little? Res Sci Educ 32:241–255

    Google Scholar 

  • Elsbach KD (2004) Managing images of trustworthiness in organizations. In: Kramer R, Cook K (eds) Trust and distrust in organizations. SAGE, New York, pp 275–292

    Google Scholar 

  • Emerson RM (1976) Social exchange theory. Annu Rev Sociol 2:335–362

    Google Scholar 

  • Frey BS (2003) Publishing as prostitution? Choosing between one’s own ideas and academic success. Public Choice 116:205–223

    Google Scholar 

  • Fu P, Yukl G (2000) Perceived effectiveness of influence tactics in the United States and China. Leadersh Q 11:251–266

    Google Scholar 

  • Giebe T, Gürtler O (2012) Optimal contracts for lenient supervisors. J Econ Behav Organ 81:403–420

    Google Scholar 

  • Glogoff S (1988) Reviewing the gatekeepers: a survey of referees of library journals. J Am Soc Inf Sci 39:400–407

    Google Scholar 

  • Gould-Williams J, Davies F (2005) Using social exchange theory to predict the effects of HRM practice on employee outcomes: an analysis of public sector workers. Public Manag Rev 7:1–24

    Google Scholar 

  • Graf C, Wager E, Bowman A, Fiack S, Scott-Lichter D, Robinson A (2007) Best practice guide-lines on publication ethics: a publisher’s perspective. Int J Clin Pract 61:1–26

    Google Scholar 

  • Hausman A, Johnston WJ (2010) The impact of coercive and non-coercive forms of influence on trust, commitment, and compliance in supply chains. Ind Mark Manag 39:519–526

    Google Scholar 

  • Hemlin S, Rasmussen S (2006) The shift in academic quality control. Sci Technol Hum Values 31:173–198

    Google Scholar 

  • Iacobucci D (2010) Structural equations modeling: fit indices, sample size, and advanced topics. J Consum Psychol 20:90–98

    Google Scholar 

  • Kingshott RPJ, Pecotich A (2007) The impact of psychological contracts on trust and commitment in supplier-distributor relationships. Eur J Mark 41:1053–1072

    Google Scholar 

  • Klein HJ, Molloy JC, Brinsfield CT (2012) Reconceptualizing workplace commitment to redress a stretched construct: revisiting assumptions and removing confounds. Acad Manage Rev 37:130–151

    Google Scholar 

  • Klein HJ, Cooper JT, Molloy JC, Swanson JA (2014) The assessment of commitment: advantages of a unidimensional, target-free approach. J Appl Psychol 99:222–238

    Google Scholar 

  • Korsgaard MA, Brower HH, Lester SW (2015) It isn’t always mutual: a critical review of dyadic trust. J Manage 41:47–70

    Google Scholar 

  • Lance CE (2006) The sources of four commonly reported cutoff criteria: what did they really say? Organ Res Methods 9:202–220

    Google Scholar 

  • Langfred CW (2004) Too much of a good thing? Negative effects of high trust and individual autonomy in self-managing teams. Acad Manage J 47:385–399

    Google Scholar 

  • van Maanen J, Sørensen JB, Mitchell TR (2007) The interplay between theory and method. Acad Manage Rev 32:1145–1154

    Google Scholar 

  • MacKinnon DP, Fairchild AJ, Fritz MS (2007) Mediation analysis. Annu Rev Psychol 58:593–614

    Google Scholar 

  • Mahoney MJ (1977) Publication prejudices: an experimental study of confirmation bias in the peer review system. Cognit Ther Res 1:161–175

    Google Scholar 

  • Malay DS (2009) Peer review and scientific misconduct: bad authors and trusting reviewers. J Foot Ankle Surg 48:283–284

    Google Scholar 

  • Maloney P, Grawitch MJ, Barber LK (2011) Strategic item selection to reduce survey length: reduction in validity? Consult Psychol J Pract Res 63:162–175

    Google Scholar 

  • Mayer R, Davis J (1999) The effect of the performance appraisal system on trust for management: a field quasi-experiment. J Appl Psychol 84:123–136

    Google Scholar 

  • Mayer R, Davis J, Schoorman F (1995) An integrative model of organizational trust. Acad Manage Rev 20:709–734

    Google Scholar 

  • Meade AW, Watson AM, Kroustalis CM (2007) Assessing common methods bias in organizational research. Annual Meeting of the Society for Industrial and Organizational Psychology. Society for Industrial and Organizational Psychology, New York

    Google Scholar 

  • Merton R (1973) Recognition and excellence: instructive ambiguities. In: Merton R (ed) The sociology of science. Theoretical and empirical investigations. University of Chicago Press, Chicago, pp 419–437

    Google Scholar 

  • Meyer JP, Allen NJ (1984) Testing the “side-bet theory” of organizational commitment: some methodological considerations. J Appl Psychol 69:372–378

    Google Scholar 

  • Meyer JP, Stanley DJ, Herscovitch L, Topolnytsky L (2002) Affective, continuance, and normative commitment to the organization: a meta-analysis of antecedents, correlates, and consequences. J Vocat Behav 61:20–52

    Google Scholar 

  • Miller CC (2006) Peer review in the organizational and management sciences: prevalence and effects of reviewer hostility, bias, and dissensus. Acad Manage J 49:425–431

    Google Scholar 

  • Miller CC, Van de Ven AH (2015) Peer review, root canals, and other amazing life events. Acad Manag Discov 1:117–123

    Google Scholar 

  • Miner JB (2003) Commentary on Arthur Bedeian’s “The manuscript review process: the proper roles of authors, referees, and editors.”. J Manag Inq 12:339–343

    Google Scholar 

  • Molm LD, Takahashi N, Peterson G (2000) Risk and trust in social exchange: an experi-mental test of a classical proposition. Am J Sociol 105:1396–1427

    Google Scholar 

  • Morgan R, Hunt S (1994) The commitment-trust theory of relationship marketing. J Mark 58:20–38

    Google Scholar 

  • Mowday RT, Porter LW, Steers RM (1982) Employee-organization linkages: the psychology of commitment, absenteeism, and turnover. Academic Press, New York

    Google Scholar 

  • Mulligan A, Hall L, Raphael E (2013) Peer review in a changing world: an international study measuring the attitudes of researchers. J Am Soc Inf Sci Technol 64:132–161

    Google Scholar 

  • Murnighan JK (2002) The effects of contracts on interpersonal trust. Adm Sci Q 47:534–559

    Google Scholar 

  • Newton DP (2010) Quality and peer review of research: an adjudicating role for editors. Account Res 17:130–145

    Google Scholar 

  • Osterloh M (2010) Governance by numbers. Does it really work in research? Anal Krit 32:267–283

    Google Scholar 

  • Osterloh M, Kieser A (2015) Double-blind peer review: how to slaughter a sacred cow. In: Welpe IM, Wollersheim J, Ringelhan S, Osterloh M (eds) Incentives and performance. Springer, Cham, pp 307–321

    Google Scholar 

  • Petersen J, Hattke F, Vogel R (2017) Editorial governance and journal impact: a study of management and business journals. Scientometrics 112:1593–1614

    Google Scholar 

  • Pillai R, Scandura TA, Williams EA (1999) Leadership and organizational justice: similarities and differences across cultures. J Int Bus Stud 30:763–779

    Google Scholar 

  • Placier M (1995) But I have to have an A: probing the cultural meanings and ethical dilemmas of grades in teacher education. Teach Educ Q 22:45–62

    Google Scholar 

  • Podsakoff PM, MacKenzie SB, Lee J‑Y, Podsakoff NP (2003) Common method biases in behavioral research: a critical review of the literature and recommended remedies. J Appl Psychol 88:879–903

    Google Scholar 

  • Prendergast C (1999) The provision of incentives in firms. J Econ Lit 37:7–63

    Google Scholar 

  • Prendergast C, Topel RH (1996) Favoritism in organizations. J Polit Econ 104:958–978

    Google Scholar 

  • Rabine JL (1999) Putting your trust in reviews: the ethics of book reviewing. Libr Collect Acquis Tech Serv 23:202–203

    Google Scholar 

  • Raja U, Johns G, Ntalianis F (2004) The impact of personality on psychological contracts. Acad Manage J 47:350–367

    Google Scholar 

  • Ramarajan L (2014) Past, present and future research on multiple identities: toward an intrapersonal network approach. Acad Manag Ann 8:589–659

    Google Scholar 

  • Robinson SL (1996) Trust and breach of the psychological contract. Adm Sci Q 41:574–599

    Google Scholar 

  • Roccas S, Brewer MB (2002) Social identity complexity. Pers Soc Psychol Rev 6:88–106

    Google Scholar 

  • Rousseau DM (2001) Schema, promise and mutuality: the building blocks of the psychological contract. J Occup Organ Psychol 74:511–541

    Google Scholar 

  • Rousseau DM, Tijoriwala SA (1998) Assessing psychological contracts: issues, alternatives and measures. J Organ Behav 19:679–695

    Google Scholar 

  • Schoorman F, Mayer R, Davis J (2007) An integrative model of organizational trust: past, present, and future. Acad Manage Rev 32:344–354

    Google Scholar 

  • Seibert S (2006) Anatomy of an R&R (Or, reviewers are an author’s best friends). Acad Manage J 49:203–207

    Google Scholar 

  • Settoon RP, Bennett N, Liden RC (1996) Social exchange in organizations: perceived organizational support, leader-member exchange, and employee reciprocity. J Appl Psychol 81:219–227

    Google Scholar 

  • Shapiro SP (1987) The social control of impersonal trust. Am J Sociol 93:623–658

    Google Scholar 

  • Smith R (2006) Peer review: a flawed process at the heart of science and journals. J R Soc Med 99:178–182

    Google Scholar 

  • Spector PE (2006) Method variance in organizational research. Truth or urban legend? Organ Res Methods 9:221–232

    Google Scholar 

  • Spector PE, Brannick MT (2011) Methodological urban legends: the misuse of statistical control variables. Organ Res Methods 14:287–305

    Google Scholar 

  • Squazzoni F, Bravo G, Takács K (2013) Does incentive provision increase the quality of peer review? An experimental study. Res Policy 42:287–294

    Google Scholar 

  • Stanton JM, Sinar EF, Balzer WK, Smith PC (2002) Issues and strategies for reducing the length of self-report scales. Pers Psychol 55:167–194

    Google Scholar 

  • Starbuck WH (2003) Turning lemons into lemonade: where is the value in peer reviews? J Manag Inq 12:344–351

    Google Scholar 

  • Van der Stede WA, Young SM, Chen CX (2005) Assessing the quality of evidence in empirical management accounting research: the case of survey studies. Account Organ Soc 30:655–684

    Google Scholar 

  • Teixeira da Silva JA, Dobránszki J (2015) Problems with traditional science publishing and finding a wider niche for post-publication peer review. Account Res 22:22–40

    Google Scholar 

  • Tett RP, Meyer JP (1993) Job satisfaction, organizational commitment, turnover intention, and turnover: path analyses based on meta-analytic findings. Pers Psychol 46:259–293

    Google Scholar 

  • Tsang E (2014) Ensuring manuscript quality and preserving authorial voice: the balancing act of editors. Manag Organ Rev 10:191–197

    Google Scholar 

  • Tsang EWK, Frey BS (2007) The as-is journal review process: let authors own their ideas. Acad Manag Learn Educ 6:128–136

    Google Scholar 

  • Vogel R, Hattke F (2018) How is the use of performance information related to performance of public sector professionals? Evidence from the field of academic research. Public Perfom Manag Rev 41(2):390–414

    Google Scholar 

  • Vogel R, Hattke F, Petersen J (2017) Journal rankings in management and business studies: what rules do we play by? Res Policy 46:1707–1722

    Google Scholar 

  • Wheeler B (2011) The ontology of the scholarly journal and the place of peer review. J Sch Publ 42:307–322

    Google Scholar 

  • Whitley R (1984a) The intellectual and social organization of the sciences. Oxford University Press, Oxford

    Google Scholar 

  • Whitley R (1984b) The fragmented state of management studies: Reasons and consequences. J Manag Stud 21:331–348

    Google Scholar 

  • Whyte G, Saks AM, Hook S (1997) When success breeds failure: the role of self-efficacy in escalating commitment to a losing course of action. J Organ Behav 18:415–432

    Google Scholar 

  • Yukl G, Tracey JB (1992) Consequences of influence tactics used with subordinates, peers, and the boss. J Appl Psychol 77:525–535

    Google Scholar 

  • Yukl G, Lepsinger R, Lucia T (1992) Preliminary report on development and validation of the influence behavior questionnaire. In: Clark KE, Clark MB, Campbell DP (eds) Impact of leadership. Center for creative leadership, Greensboro, pp 417–427

    Google Scholar 

  • Yukl G, Chavez C, Seifert CF (2005) Assessing the construct validity and utility of two new influence tactics. J Organ Behav 26:705–725

    Google Scholar 

  • Yukl G, Seifert CF, Chavez C (2008) Validation of the extended influence behavior questionnaire. Leadersh Q 19:609–621

    Google Scholar 

  • Zahra SA, Neubaum DO (2006) Revising to be published: building trust to win the acceptance of journal editors and reviewers. In: Baruch Y, Sullivan SE, Schepmyer HN (eds) Winning reviews: a guide for evaluating scholarly writing. Palgrave Macmillan, Houndmills, pp 205–223

    Google Scholar 

Download references

Acknowledgements

This research is supported by the German Federal Ministry of Education and Research (Grant No. 01PY13014).

Author information

Authors and Affiliations

  1. Organization and Management, University of Hamburg, Hamburg, Germany

    Fabian Hattke

  2. CHESS Center for Higher Education and Science Studies, University of Zurich, Zurich, Switzerland

    Fabian Hattke

  3. Management, Strategy and Leadership, University of Konstanz, Konstanz, Germany

    Isabel Bögner

  4. Public Management, University of Hamburg, Hamburg, Germany

    Rick Vogel

Authors
  1. Fabian Hattke
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Isabel Bögner
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rick Vogel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fabian Hattke.

Appendix

Appendix

 

Table 6 Scales and items
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hattke, F., Bögner, I. & Vogel, R. (Why) Do you trust your reviewers? Influence behaviors, trustworthiness, and commitment to peer review. Managementforschung 28, 61–86 (2018). https://doi.org/10.1365/s41113-018-0022-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1365/s41113-018-0022-2

Keywords

Schlüsselwörter

Search

Navigation