Advertisement

Managementforschung

, Volume 28, Issue 1, pp 61–86 | Cite as

(Why) Do you trust your reviewers? Influence behaviors, trustworthiness, and commitment to peer review

  • Fabian Hattke
  • Isabel Bögner
  • Rick Vogel
Originalartikel

Abstract

Peer review in academic publishing relies on the voluntary engagement of scholars who are, at best, committed to that practice. Current debates on peer review suggest that this commitment is diminishing. Conceptualizing peer review as an instance of social exchange allows us to propose a conceptual model of commitment to peer review and test it by means of a structural equation analysis. Our empirical study is based on survey data from the social sciences (n = 359). Results show that authors are more committed to the practice of peer review if reviewers base their recommendations on rational arguments so that authors can trust them for their competence. By contrast, benevolent reviewers who try to collaborate with authors are not effective in fostering trust and commitment. Within the limitations of our data and with regard to reviewers’ behaviors and characteristics, we cannot support sweeping criticisms of the operational reliability of academic journals.

Keywords

Academic journals Trust Peer control Reviewing Science studies 

(Warum) Trauen Sie Ihren Gutachtern? Einflussnahme, Vertrauenswürdigkeit und Commitment gegenüber Peer Review

Zusammenfassung

Peer Review im akademischen Publizieren basiert auf dem freiwilligen Engagement von Wissenschaftlerinnen und Wissenschaftlern, die, im besten Fall, ein hohes Commitment zu dieser Praxis aufweisen. Aktuelle Debatten über den Peer-Review-Prozess deuten darauf hin, dass dieses Commitment abnimmt. Basierend auf der Social-Exchange-Theorie entwickeln wir ein konzeptionelles Modell, das wir mithilfe eines Strukturgleichungsmodells testen. Unsere empirische Studie basiert auf einer Umfrage in den Sozialwissenschaften (n = 359). Die Ergebnisse zeigen, dass Autoren ein höheres Commitment zur Praxis des Peer Review aufweisen, wenn Gutachter ihre Kommentare auf rationalen Argumenten gründen und die Autoren ihnen aufgrund dieser Kompetenz vertrauen können. Sind Gutachter allerdings zu wohlwollend oder versuchen sie mit den Autoren zu kollaborieren, erwecken sie kein Vertrauen und vermindern das Commitment der Autoren. Im Hinblick auf die Eigenschaften und Verhaltensweisen von Gutachtern und unter Berücksichtigung der Limitationen unserer empirischen Studie können wir die Kritik an der Funktionsweise des Peer Reviews in wissenschaftlichen Zeitschriften nicht bestätigen.

Schlüsselwörter

Wissenschaftliche Zeitschriften Vertrauen Peer Control Begutachtungsprozesse Wissenschaftsforschung 

Notes

Acknowledgements

This research is supported by the German Federal Ministry of Education and Research (Grant No. 01PY13014).

References

  1. Aguinis H, Shapiro D, Antonacopoulou EP, Cummings TG (2014) Scholarly impact: a pluralist conceptualization. Acad Manage Learn Educ 13:1–17Google Scholar
  2. Antonakis J, Bendahan S, Jacquart P, Lalive R (2010) On making causal claims: a review and recommendations. Leadersh Q 21:1086–1120Google Scholar
  3. Aryee S, Budhwar, Chen ZX (2002) Trust as a mediator of the relationship between organizational justice and work outcomes: test of a social exchange model. J Organ Behav 23:267–286Google Scholar
  4. Bandura A (1977) Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev 84:191–215Google Scholar
  5. Bedeian AG (2003) The manuscript review process: the proper roles of authors, referees, and editors. J Manag Inq 12:331–338Google Scholar
  6. Bedeian AG (2004) Peer review and the social construction of knowledge in the management discipline. Acad Manag Learn Educ 3:198–216Google Scholar
  7. Berkenkotter C (1995) The power and the perils of peer review. Rhetor Rev 13:245–248Google Scholar
  8. Blau PM (1964) Exchange and power in social life. John Wiley, New YorkGoogle Scholar
  9. Bollen K (1990) Overall fit in covariance structure models: two types of sample size effects. Psychol Bull 107:256–259Google Scholar
  10. Bornmann L, Wolf M, Daniel H‑D (2011) Closed versus open reviewing of journal manuscripts: how far do comments differ in language use? Scientometrics 91:843–856Google Scholar
  11. Bornmann L, Herich H, Joos H, Daniel H‑D (2012) In public peer review of submitted manuscripts, how do reviewer comments differ from comments written by interested members of the scientific community? A content analysis of comments written for atmospheric chemistry and physics. Scientometrics 93:915–929Google Scholar
  12. Bradley JV (1982) Editorial overkill. Bull Psychon Soc 19:271–274Google Scholar
  13. Campanario JM (1998a) Peer review for journals as it stands today—part 1. Sci Commun 19:181–211Google Scholar
  14. Campanario JM (1998b) Peer review for journals as it stands today—part 2. Sci Commun 19:277–306Google Scholar
  15. Cardy RL, Dobbins GH (1986) Affect and appraisal accuracy: liking as an integral dimension in evaluating performance. J Appl Psychol 71:672–678Google Scholar
  16. Chubin DE, Hackett EJ (1990) Peerless science: peer review and U.S. science policy. Sunny Press, New YorkGoogle Scholar
  17. Clark T, Wright M (2007) Reviewing journal rankings and revisiting peer reviews: editorial perspectives. J Manag Stud 44:612–621Google Scholar
  18. Colquitt JA, Scott BA, LePine JA (2007) Trust, trustworthiness, and trust propensity: a meta-analytic test of their unique relationships with risk taking and job performance. J Appl Psychol 92:909–927Google Scholar
  19. Cropanzano R, Mitchell MS (2005) Social exchange theory: an interdisciplinary review. J Manage 31:874–900Google Scholar
  20. Crossman A, Lee-Kelley L (2004) Trust, commitment and team working: the paradox of virtual organizations. Glob Netw (Oxf) 4:375–390Google Scholar
  21. Cullen JB, Johnson JL, Sakano T (2000) Success through commitment and trust: the soft side of strategic alliance management. J World Bus 35:223–240Google Scholar
  22. D’Andrea R, O’Dwyer JP (2017) Can editors save peer review from peer reviewers? PLoS ONE 12:18–20Google Scholar
  23. Dulac T, Coyle-Shapiro JA, Henderson DJ, Wayne SJ (2008) Not all responses to breach are the same: the interconnection of social exchange and psychological contract processes in organizations. Acad Manage J 51:1079–1098Google Scholar
  24. Eisenhart M (2002) The paradox of peer review: admitting too much or allowing too little? Res Sci Educ 32:241–255Google Scholar
  25. Elsbach KD (2004) Managing images of trustworthiness in organizations. In: Kramer R, Cook K (eds) Trust and distrust in organizations. SAGE, New York, pp 275–292Google Scholar
  26. Emerson RM (1976) Social exchange theory. Annu Rev Sociol 2:335–362Google Scholar
  27. Frey BS (2003) Publishing as prostitution? Choosing between one’s own ideas and academic success. Public Choice 116:205–223Google Scholar
  28. Fu P, Yukl G (2000) Perceived effectiveness of influence tactics in the United States and China. Leadersh Q 11:251–266Google Scholar
  29. Giebe T, Gürtler O (2012) Optimal contracts for lenient supervisors. J Econ Behav Organ 81:403–420Google Scholar
  30. Glogoff S (1988) Reviewing the gatekeepers: a survey of referees of library journals. J Am Soc Inf Sci 39:400–407Google Scholar
  31. Gould-Williams J, Davies F (2005) Using social exchange theory to predict the effects of HRM practice on employee outcomes: an analysis of public sector workers. Public Manag Rev 7:1–24Google Scholar
  32. Graf C, Wager E, Bowman A, Fiack S, Scott-Lichter D, Robinson A (2007) Best practice guide-lines on publication ethics: a publisher’s perspective. Int J Clin Pract 61:1–26Google Scholar
  33. Hausman A, Johnston WJ (2010) The impact of coercive and non-coercive forms of influence on trust, commitment, and compliance in supply chains. Ind Mark Manag 39:519–526Google Scholar
  34. Hemlin S, Rasmussen S (2006) The shift in academic quality control. Sci Technol Hum Values 31:173–198Google Scholar
  35. Iacobucci D (2010) Structural equations modeling: fit indices, sample size, and advanced topics. J Consum Psychol 20:90–98Google Scholar
  36. Kingshott RPJ, Pecotich A (2007) The impact of psychological contracts on trust and commitment in supplier-distributor relationships. Eur J Mark 41:1053–1072Google Scholar
  37. Klein HJ, Molloy JC, Brinsfield CT (2012) Reconceptualizing workplace commitment to redress a stretched construct: revisiting assumptions and removing confounds. Acad Manage Rev 37:130–151Google Scholar
  38. Klein HJ, Cooper JT, Molloy JC, Swanson JA (2014) The assessment of commitment: advantages of a unidimensional, target-free approach. J Appl Psychol 99:222–238Google Scholar
  39. Korsgaard MA, Brower HH, Lester SW (2015) It isn’t always mutual: a critical review of dyadic trust. J Manage 41:47–70Google Scholar
  40. Lance CE (2006) The sources of four commonly reported cutoff criteria: what did they really say? Organ Res Methods 9:202–220Google Scholar
  41. Langfred CW (2004) Too much of a good thing? Negative effects of high trust and individual autonomy in self-managing teams. Acad Manage J 47:385–399Google Scholar
  42. van Maanen J, Sørensen JB, Mitchell TR (2007) The interplay between theory and method. Acad Manage Rev 32:1145–1154Google Scholar
  43. MacKinnon DP, Fairchild AJ, Fritz MS (2007) Mediation analysis. Annu Rev Psychol 58:593–614Google Scholar
  44. Mahoney MJ (1977) Publication prejudices: an experimental study of confirmation bias in the peer review system. Cognit Ther Res 1:161–175Google Scholar
  45. Malay DS (2009) Peer review and scientific misconduct: bad authors and trusting reviewers. J Foot Ankle Surg 48:283–284Google Scholar
  46. Maloney P, Grawitch MJ, Barber LK (2011) Strategic item selection to reduce survey length: reduction in validity? Consult Psychol J Pract Res 63:162–175Google Scholar
  47. Mayer R, Davis J (1999) The effect of the performance appraisal system on trust for management: a field quasi-experiment. J Appl Psychol 84:123–136Google Scholar
  48. Mayer R, Davis J, Schoorman F (1995) An integrative model of organizational trust. Acad Manage Rev 20:709–734Google Scholar
  49. Meade AW, Watson AM, Kroustalis CM (2007) Assessing common methods bias in organizational research. Annual Meeting of the Society for Industrial and Organizational Psychology. Society for Industrial and Organizational Psychology, New YorkGoogle Scholar
  50. Merton R (1973) Recognition and excellence: instructive ambiguities. In: Merton R (ed) The sociology of science. Theoretical and empirical investigations. University of Chicago Press, Chicago, pp 419–437Google Scholar
  51. Meyer JP, Allen NJ (1984) Testing the “side-bet theory” of organizational commitment: some methodological considerations. J Appl Psychol 69:372–378Google Scholar
  52. Meyer JP, Stanley DJ, Herscovitch L, Topolnytsky L (2002) Affective, continuance, and normative commitment to the organization: a meta-analysis of antecedents, correlates, and consequences. J Vocat Behav 61:20–52Google Scholar
  53. Miller CC (2006) Peer review in the organizational and management sciences: prevalence and effects of reviewer hostility, bias, and dissensus. Acad Manage J 49:425–431Google Scholar
  54. Miller CC, Van de Ven AH (2015) Peer review, root canals, and other amazing life events. Acad Manag Discov 1:117–123Google Scholar
  55. Miner JB (2003) Commentary on Arthur Bedeian’s “The manuscript review process: the proper roles of authors, referees, and editors.”. J Manag Inq 12:339–343Google Scholar
  56. Molm LD, Takahashi N, Peterson G (2000) Risk and trust in social exchange: an experi-mental test of a classical proposition. Am J Sociol 105:1396–1427Google Scholar
  57. Morgan R, Hunt S (1994) The commitment-trust theory of relationship marketing. J Mark 58:20–38Google Scholar
  58. Mowday RT, Porter LW, Steers RM (1982) Employee-organization linkages: the psychology of commitment, absenteeism, and turnover. Academic Press, New YorkGoogle Scholar
  59. Mulligan A, Hall L, Raphael E (2013) Peer review in a changing world: an international study measuring the attitudes of researchers. J Am Soc Inf Sci Technol 64:132–161Google Scholar
  60. Murnighan JK (2002) The effects of contracts on interpersonal trust. Adm Sci Q 47:534–559Google Scholar
  61. Newton DP (2010) Quality and peer review of research: an adjudicating role for editors. Account Res 17:130–145Google Scholar
  62. Osterloh M (2010) Governance by numbers. Does it really work in research? Anal Krit 32:267–283Google Scholar
  63. Osterloh M, Kieser A (2015) Double-blind peer review: how to slaughter a sacred cow. In: Welpe IM, Wollersheim J, Ringelhan S, Osterloh M (eds) Incentives and performance. Springer, Cham, pp 307–321Google Scholar
  64. Petersen J, Hattke F, Vogel R (2017) Editorial governance and journal impact: a study of management and business journals. Scientometrics 112:1593–1614Google Scholar
  65. Pillai R, Scandura TA, Williams EA (1999) Leadership and organizational justice: similarities and differences across cultures. J Int Bus Stud 30:763–779Google Scholar
  66. Placier M (1995) But I have to have an A: probing the cultural meanings and ethical dilemmas of grades in teacher education. Teach Educ Q 22:45–62Google Scholar
  67. Podsakoff PM, MacKenzie SB, Lee J‑Y, Podsakoff NP (2003) Common method biases in behavioral research: a critical review of the literature and recommended remedies. J Appl Psychol 88:879–903Google Scholar
  68. Prendergast C (1999) The provision of incentives in firms. J Econ Lit 37:7–63Google Scholar
  69. Prendergast C, Topel RH (1996) Favoritism in organizations. J Polit Econ 104:958–978Google Scholar
  70. Rabine JL (1999) Putting your trust in reviews: the ethics of book reviewing. Libr Collect Acquis Tech Serv 23:202–203Google Scholar
  71. Raja U, Johns G, Ntalianis F (2004) The impact of personality on psychological contracts. Acad Manage J 47:350–367Google Scholar
  72. Ramarajan L (2014) Past, present and future research on multiple identities: toward an intrapersonal network approach. Acad Manag Ann 8:589–659Google Scholar
  73. Robinson SL (1996) Trust and breach of the psychological contract. Adm Sci Q 41:574–599Google Scholar
  74. Roccas S, Brewer MB (2002) Social identity complexity. Pers Soc Psychol Rev 6:88–106Google Scholar
  75. Rousseau DM (2001) Schema, promise and mutuality: the building blocks of the psychological contract. J Occup Organ Psychol 74:511–541Google Scholar
  76. Rousseau DM, Tijoriwala SA (1998) Assessing psychological contracts: issues, alternatives and measures. J Organ Behav 19:679–695Google Scholar
  77. Schoorman F, Mayer R, Davis J (2007) An integrative model of organizational trust: past, present, and future. Acad Manage Rev 32:344–354Google Scholar
  78. Seibert S (2006) Anatomy of an R&R (Or, reviewers are an author’s best friends). Acad Manage J 49:203–207Google Scholar
  79. Settoon RP, Bennett N, Liden RC (1996) Social exchange in organizations: perceived organizational support, leader-member exchange, and employee reciprocity. J Appl Psychol 81:219–227Google Scholar
  80. Shapiro SP (1987) The social control of impersonal trust. Am J Sociol 93:623–658Google Scholar
  81. Smith R (2006) Peer review: a flawed process at the heart of science and journals. J R Soc Med 99:178–182Google Scholar
  82. Spector PE (2006) Method variance in organizational research. Truth or urban legend? Organ Res Methods 9:221–232Google Scholar
  83. Spector PE, Brannick MT (2011) Methodological urban legends: the misuse of statistical control variables. Organ Res Methods 14:287–305Google Scholar
  84. Squazzoni F, Bravo G, Takács K (2013) Does incentive provision increase the quality of peer review? An experimental study. Res Policy 42:287–294Google Scholar
  85. Stanton JM, Sinar EF, Balzer WK, Smith PC (2002) Issues and strategies for reducing the length of self-report scales. Pers Psychol 55:167–194Google Scholar
  86. Starbuck WH (2003) Turning lemons into lemonade: where is the value in peer reviews? J Manag Inq 12:344–351Google Scholar
  87. Van der Stede WA, Young SM, Chen CX (2005) Assessing the quality of evidence in empirical management accounting research: the case of survey studies. Account Organ Soc 30:655–684Google Scholar
  88. Teixeira da Silva JA, Dobránszki J (2015) Problems with traditional science publishing and finding a wider niche for post-publication peer review. Account Res 22:22–40Google Scholar
  89. Tett RP, Meyer JP (1993) Job satisfaction, organizational commitment, turnover intention, and turnover: path analyses based on meta-analytic findings. Pers Psychol 46:259–293Google Scholar
  90. Tsang E (2014) Ensuring manuscript quality and preserving authorial voice: the balancing act of editors. Manag Organ Rev 10:191–197Google Scholar
  91. Tsang EWK, Frey BS (2007) The as-is journal review process: let authors own their ideas. Acad Manag Learn Educ 6:128–136Google Scholar
  92. Vogel R, Hattke F (2018) How is the use of performance information related to performance of public sector professionals? Evidence from the field of academic research. Public Perfom Manag Rev 41(2):390–414Google Scholar
  93. Vogel R, Hattke F, Petersen J (2017) Journal rankings in management and business studies: what rules do we play by? Res Policy 46:1707–1722Google Scholar
  94. Wheeler B (2011) The ontology of the scholarly journal and the place of peer review. J Sch Publ 42:307–322Google Scholar
  95. Whitley R (1984a) The intellectual and social organization of the sciences. Oxford University Press, OxfordGoogle Scholar
  96. Whitley R (1984b) The fragmented state of management studies: Reasons and consequences. J Manag Stud 21:331–348Google Scholar
  97. Whyte G, Saks AM, Hook S (1997) When success breeds failure: the role of self-efficacy in escalating commitment to a losing course of action. J Organ Behav 18:415–432Google Scholar
  98. Yukl G, Tracey JB (1992) Consequences of influence tactics used with subordinates, peers, and the boss. J Appl Psychol 77:525–535Google Scholar
  99. Yukl G, Lepsinger R, Lucia T (1992) Preliminary report on development and validation of the influence behavior questionnaire. In: Clark KE, Clark MB, Campbell DP (eds) Impact of leadership. Center for creative leadership, Greensboro, pp 417–427Google Scholar
  100. Yukl G, Chavez C, Seifert CF (2005) Assessing the construct validity and utility of two new influence tactics. J Organ Behav 26:705–725Google Scholar
  101. Yukl G, Seifert CF, Chavez C (2008) Validation of the extended influence behavior questionnaire. Leadersh Q 19:609–621Google Scholar
  102. Zahra SA, Neubaum DO (2006) Revising to be published: building trust to win the acceptance of journal editors and reviewers. In: Baruch Y, Sullivan SE, Schepmyer HN (eds) Winning reviews: a guide for evaluating scholarly writing. Palgrave Macmillan, Houndmills, pp 205–223Google Scholar

Copyright information

© Springer Fachmedien Wiesbaden GmbH, ein Teil von Springer Nature 2018

Authors and Affiliations

  1. 1.Organization and ManagementUniversity of HamburgHamburgGermany
  2. 2.CHESS Center for Higher Education and Science StudiesUniversity of ZurichZurichSwitzerland
  3. 3.Management, Strategy and LeadershipUniversity of KonstanzKonstanzGermany
  4. 4.Public ManagementUniversity of HamburgHamburgGermany

Personalised recommendations