Skip to main content

In search of rigor, relevance, and legitimacy: what drives the impact of publications?

Journal of Business Economics volume 84, pages 99–128 (2014)Cite this article

Abstract

In this study, we introduce the concept of legitimacy to the rigor-relevance debate and investigate empirically how rigor, relevance, and legitimacy are related to the process of knowledge dissemination within a research field. We argue that this analysis has been a missing piece in the debate on rigor and relevance when institutional logics about what constitutes both elements lead researchers to act according to what they perceive to be appropriate behavior in the research field. We draw on insights from the micro and macro levels of institutional theory to show how researchers aiming to bestow legitimacy on their own work conform to these “rules of the game”. Using meta-analytical techniques, we focus on the field of strategic entrepreneurship and analyze how rigor- and relevance-related characteristics of studies in this field are linked to their legitimacy and therefore to the impact they have in the research community.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

References

  • Adler NJ, Harzing A-W (2009) When knowledge wins: transcending the sense and nonsense of academic rankings. Acad Manag Learn Education 8:72–95

    Google Scholar 

  • Aguinis H, Suárez-González I, Lannelongue G, Joo H (2012) Scholarly impact revisited. Acad Manag Perspect 26:105–132

    Google Scholar 

  • Aiken LS, West SG (1991) Multiple regression: Testing and interpreting interactions. Sage, Newbury Park

    Google Scholar 

  • Anderson N, Herriot P, Hodgkinson GP (2001) The practitioner-researcher divide in industrial, work and organizational (IWO) psychology: where are we now, and where do we go from here? J Occup Organ Psychol 74:391–411

    Google Scholar 

  • Augier M, March JG (2007) The pursuit of relevance in management education. Calif Manag Rev 49:129–146

    Google Scholar 

  • Backes-Gellner U, Sadowski D (1990) Organizational implementation of bibliometric indicators. Scientometrics 19:385–395

    Google Scholar 

  • Baldridge DC, Floyd SW, Markóczy L (2004) Are managers from Mars and academicians from Venus? Toward an understanding of the relationship between academic quality and practical relevance. Strateg Manag J 25:1063–1074

    Google Scholar 

  • Bennis WG, O’Toole J (2005) How business schools lost their way. Harv Bus Rev 83:96–104

    Google Scholar 

  • Bettis RA (1991) Strategic management and the straightjacket: an editorial essay. Organ Sci 2:315–319

    Google Scholar 

  • Beyer JM, Chanove RG, Fox WB (1995) The review process and the fates of manuscripts submitted to AMJ. Acad Manag J 38:1219–1260

    Google Scholar 

  • Borenstein M, Hedges L, Higgins J, Rothstein H (2005) Comprehensive meta-analysis, 2nd edn. Wiley, Englewood

    Google Scholar 

  • Bozart JD, Roberts RR (1972) Signifying significant significance. Am Psychol 27:774–775

    Google Scholar 

  • Bresser RK (1984) Structural dimensions of university departments and their context: the case of West Germany. Organ Stud 5:119–146

    Google Scholar 

  • Bresser RK, Dunbar RLM (1986) Context, structure, and academic effectiveness: evidence from West Germany. Organ Stud 7:1–24

    Google Scholar 

  • Bresser RK, Millonig K (2003) Institutional capital: competitive advantage in light of the new institutionalism in organization theory. Schmalenbach Bus Rev 55:220–241

    Google Scholar 

  • Buckley MR, Ferris GR, Bernardin HJ, Harvey MG (1998) The disconnect between the science and practice of management. Bus Horiz 41:31–38

    Google Scholar 

  • Chlosta K, Pull K (2010) The incentive effects of appointment tournaments in German higher education. Schmalenbach Bus Rev 62:378–400

    Google Scholar 

  • Cohen J, Cohen P (1983) Applied multiple regression/correlation analysis for the behavioral sciences, 2nd edn. Erlbaum, Hillsdale

    Google Scholar 

  • Combs JG (2010) Big samples and small effects: let’s not trade relevance and rigor for power. Acad Manag J 53:9–13

    Google Scholar 

  • Conger AJ (1974) A revised definition for suppressor variables: a guide to their identification and interpretation. Educ Psychol Measur 34:35–46

    Google Scholar 

  • Cooper H (1998) Synthesizing research: A guide for literature reviews. Sage, Thousand Oaks

    Google Scholar 

  • Covin JG, Green KM, Slevin DP (2006) Strategic process effects on the entrepreneurial orientation–sales growth rate relationship. Entrepreneurship: Theory Pract 30:57–81

    Google Scholar 

  • Daft RL, Lewin AY (2008) Rigor and relevance in organization studies: idea migration and academic journal evolution. Organ Sci 19:177–183

    Google Scholar 

  • Dilger A (2000) Plädoyer für einen sozialwissenschaftlichen Zitationsindex. Die Betriebswirtschaft 60:473–484

    Google Scholar 

  • DiMaggio PJ, Powell WW (1983) The iron cage revisited: institutional isomorphism and collective rationality in organizational fields. Am Sociol Rev 48:147–160

    Google Scholar 

  • Donohue JM, Fox JB (2000) A multi-method evaluation of journals in the decision and management sciences by US academics. Omega 28:17–36

    Google Scholar 

  • Duncan WJ (1974) Transferring management theory to practice. Acad Manag J 17:724–738

    Google Scholar 

  • Dyckhoff H, Gilles R (2004) Messung der Effektivität und Effizienz produktiver Einheiten. Zeitschrift für Betriebswirtschaft 74:765–783

    Google Scholar 

  • Dyckhoff H, Schmitz C (2007) Forschungsleistungsmessung mittels SSCI Oder SCI-X? Internationale Sichtbarkeit und Wahrnehmung der Betriebswirtschaftslehre von 1990-2004. Die Betriebswirtschaft 67:638–662

    Google Scholar 

  • Dyckhoff H, Rassenhövel S, Gilles R, Schmitz C (2005a) Beurteilung der Forschungsleistung und das CHE-Forschungsranking betriebswirtschaftlicher Fachbereiche. Wirtschaftswissenschaftliches Studium 34:62–69

    Google Scholar 

  • Dyckhoff H, Thieme A, Schmitz C (2005b) Die Wahrnehmung deutschsprachiger Hochschullehrer für Betriebswirtschaft in der internationalen Forschung: eine Pilotstudie zu Zitationsverhalten und möglichen Einflussfaktoren. Die Betriebswirtschaft 65:350–372

    Google Scholar 

  • Ellis PD (2006) Market orientation and performance: a meta-analysis and cross-national comparisons. J Manage Stud 43:1089–1107

    Google Scholar 

  • Fabel O, Heße F (1999) Befragungsstudie vs. Publikationsanalyse: zur Interpretation von Ranglisten der Forschungsaktivitäten deutscher betriebswirtschaftlicher Fachbereiche. Die Betriebswirtschaft 59:196–204

    Google Scholar 

  • Fiedler M, Welpe IM, Picot A (2006) Terra Incognita—Forschungsleistungen und Qualifizierungswege des deutschsprachigen Hochschullehrernachwuchses für Betriebswirtschaftslehre. Die Betriebswirtschaft 66:464–486

    Google Scholar 

  • Fiedler M, Welpe IM, Lindlbauer K, Sattler K (2008) Denn wer da hat, dem wird gegeben: Publikationsproduktivität des BWL Hochschullehrernachwuchses und deren wissenschaftlicher Betreuer. Zeitschrift für Betriebswirtschaft 78:477–508

    Google Scholar 

  • Fleming P, Spicer A (2003) Working at a cynical distance: implications for power, subjectivity and resistance. Organization 10:157–179

    Google Scholar 

  • Frey BS, Rost K (2010) Do rankings reflect research quality? J Appl Econ 13:1–38

    Google Scholar 

  • Friedland R, Alford R (1991) Bringing society back in: symbols, practices, and institutional contradictions. In: Powell WW, DiMaggio PJ (eds) The new institutionalism in organizational analysis. University of Chicago Press, Chicago, pp 232–263

    Google Scholar 

  • Ghoshal S (2005) Bad management theories are destroying good management practices. Acad Manag Learn Education 4:75–91

    Google Scholar 

  • Gomez-Mejia LR, Balkin DB (1992) Determinants of faculty pay: an agency theory perspective. Acad Manag J 35:921–955

    Google Scholar 

  • Gopinath C, Hoffman RC (1995) The relevance of strategy research: practitioner and academic viewpoints. J Manage Stud 32:575–594

    Google Scholar 

  • Gordon RA, Howell JE (1959) Higher education for business. Columbia University Press, New York

    Google Scholar 

  • Gottfredson SD (1978) Evaluating psychological research reports: dimensions, reliability, and correlates of quality judgements. Am Psychol 33:920–934

    Google Scholar 

  • Greenwood R, Suddaby R, Hinings CR (2002) Theorizing change: the role of professional associations in the transformation of institutional fields. Acad Manag J 45:58–80

    Google Scholar 

  • Gulati R (2007) Tent poles, tribalism, and boundary spanning: the rigor-relevance debate in management research. Acad Manag J 50:775–782

    Google Scholar 

  • Harzing A-W (2012) Journal quality list, 46th edn, http://www.harzing.com

  • Hedges LV, Olkin I (1985) Statistical methods for meta-analysis. Academic Press, Orlando

    Google Scholar 

  • Hedges LV, Vevea JL (1998) Fixed- and random-effects models in meta-analysis. Psychol Methods 3:486–504

    Google Scholar 

  • Hitt MA, Ireland RD, Sirmon DG, Trahms CA (2011) Strategic entrepreneurship: creating value for individuals, organizations, and society. Acad Manag Perspect 25:57–75

    Google Scholar 

  • Hodgkinson GP, Rousseau DM (2009) Bridging the rigour–relevance gap in management research: it’s already happening! J Manag Stud 46:534–546

    Google Scholar 

  • Horst P (1941) The role of prediction variables which are independent of the criterion. In: Horst P (ed) The prediction of personal adjustment. Social Science Research Council Bulletin, New York, pp 431–436

    Google Scholar 

  • Hunter JE, Schmidt FL (1990) Methods of meta-analysis. Sage, Newbury Park

    Google Scholar 

  • Hunter JE, Schmidt FL (2004) Methods of meta-analysis: correcting error and bias in research findings, 2nd edn. Sage, Thousand Oaks

    Google Scholar 

  • Hunter JE, Schmidt FL, Jackson GB (1982) Meta-analysis: cumulating research findings across studies. Sage, Beverly Hills

    Google Scholar 

  • Jermier JM, Knights D, Nord WRE (1994) Resistance & power in organizations. Routledge, London

    Google Scholar 

  • Johnson JL, Podsakoff PM (1994) Journal influence in the field of management: an analysis using Salancik’s index in a dependency network. Acad Manag J 37:1392–1407

    Google Scholar 

  • Judge TA, Cable DM, Colbert AE, Rynes SL (2007) What causes a management article to be cited—article, author, or journal? Acad Manag J 50:491–506

    Google Scholar 

  • Kacmar KM, Whitfield JM (2000) An additional rating method for journal articles in the field of management. Organ Res Methods 3:392–406

    Google Scholar 

  • Kailer N (2009) Entrepreneurship education: empirical findings and proposals for the design of entrepreneurship education concepts at universities in German-speaking countries. J Enterp Culture 17:201–231

    Google Scholar 

  • Kieser A (2005) Wie wichtig sind Forschungsrankings? Wie wichtig werden sie in Zukunft sein? Journal für Betriebswirtschaft 55:169–170

    Google Scholar 

  • Kieser A, Leiner L (2009) Why the rigour–relevance gap in management research is unbridgeable. J Manage Stud 46:516–533

    Google Scholar 

  • Kieser A, Leiner L (2011) On the social construction of relevance: a rejoinder. J Manage Stud 48:891–898

    Google Scholar 

  • Kieser A, Nicolai AT (2005) Success factor research: overcoming the trade-off between rigor and relevance? J Manag Inquiry 14:275–279

    Google Scholar 

  • Köhler R (2004) Publish or perish. Die Betriebswirtschaft 64:127–130

    Google Scholar 

  • Kraus S, Eggers F, Harms R, Hills G, Hultman C (2011a) Diskussionslinien der Entrepreneurial Marketing-Forschung: ergebnisse einer Zitationsanalyse. Zeitschrift für Betriebswirtschaft 81:27–58

    Google Scholar 

  • Kraus S, Kauranen I, Reschke CH (2011b) Identification of domains for a new conceptual model of strategic entrepreneurship using the configuration approach. Manag Res Rev 34:58–74

    Google Scholar 

  • Laband DN (1986) Article popularity. Econ Inquiry 24:173–180

    Google Scholar 

  • Laband DN, Piette MJ (1994) A citation analysis of the impact of blinded peer review. J Am Med Assoc 272:147–149

    Google Scholar 

  • Lipsey MW, Wilson DB (2001) Practical meta-analysis. Sage, Thousand Oaks

    Google Scholar 

  • Locke K, Golden-Biddle K (1997) Constructing opportunities for contribution: structuring intertextual coherence and “problematizing” in organizational studies. Acad Manag J 40:1023–1062

    Google Scholar 

  • Long RG, Bowers WP, Barnett T, White MC (1998) Research productivity of graduates in management: effects of academic origin and academic affiliation. Acad Manag J 41:704–714

    Google Scholar 

  • Long CP, Bendersky C, Morrill C (2011) Fairness monitoring: linking managerial controls and fairness judgments in organizations. Acad Manag J 54:1045–1068

    Google Scholar 

  • Luhmann N (1998) Die Wissenschaft der Gesellschaft. Suhrkamp, Frankfurt am Main

    Google Scholar 

  • Maassen GH, Bakker AB (2001) Suppressor variables in path models: definitions and interpretations. Sociol Methods Res 30:241–270

    Google Scholar 

  • Macharzina K, Wolf J, Oesterle M-J (1993) Quantitative evaluation of German research output in business administration. Manag Int Rev 33:65–83

    Google Scholar 

  • Macharzina K, Wolf J, Rohn A (2006) Zur Forschungsleistung der Betriebswirte im deutschsprachigen Raum—Eine personen- und institutionenbezogene Längsschnittanalyse. In: Wolf J, Rohn A, Macharzina K (eds) Forschungsleistung in der deutschsprachigen Betriebswirtschaftslehre—Konzeption und Befunde einer empirischen Untersuchung. Gabler, Wiesbaden, pp 1–34

    Google Scholar 

  • March JG (1996) Continuity and change in theories of organizational action. Adm Sci Q 41:278–287

    Google Scholar 

  • Mazen AM, Graf LA, Kellogg CE, Hemmasi M (1987) Statistical power in contemporary management research. Acad Manag J 30:369–380

    Google Scholar 

  • McGahan A (2007) Academic research that matters to managers: on zebras, dogs, lemmings, hammers, and turnips. Acad Manag J 50:748–753

    Google Scholar 

  • Meyer JW, Rowan B (1977) Institutional organizations: formal structures as myth and ceremony. Am J Sociol 83:340–363

    Google Scholar 

  • Neter J, Kutner MH, Nachtsheim CJ, Wasserman W (1996) Applied linear statistical models, 4th edn. Irwin, Chicago

    Google Scholar 

  • Nicolai A, Seidl D (2010) That’s relevant! Different forms of practical relevance in Management Science. Organ Stud 31:1257–1285

    Google Scholar 

  • Nicolai AT, Schulz A-C, Göbel M (2011) Between sweet harmony and a clash of cultures: does a joint academic–practitioner review reconcile rigor and relevance? J Appl Behav Sci 47:53–75

    Google Scholar 

  • Nkomo SM (2009) The seductive power of academic journal rankings: challenges of searching for the otherwise. Acad Manag Learn Education 8:106–112

    Google Scholar 

  • Nunnally JC (1978) Psychometric Theory. McGraw-Hill, New York

    Google Scholar 

  • Ocasio W (1997) Towards an attention-based view of the firm. Strateg Manag J 18:187–206

    Google Scholar 

  • Oesterle MJ (2006) Wahrnehmung betriebswirtschaftlicher Fachzeitschriften durch Praktiker. Die Betriebswirtschaft 66:307–325

    Google Scholar 

  • Oswald AJ (2007) An examination of the reliability of prestigious scholarly journals: evidence and implications for decision-makers. Economica 74:21–31

    Google Scholar 

  • Palmer D, Dick B, Freiburger N (2009) Rigor and relevance in organization studies. J Manag Inquiry 18:265–272

    Google Scholar 

  • Park SH, Gordon ME (1996) Publication records and tenure decisions in the field of strategic management. Strateg Manag J 17:109–128

    Google Scholar 

  • Pedhazur EJ (1997) Multiple regression in behavioral research: explanation and prediction. Harcourt, Fort Worth

    Google Scholar 

  • Pettigrew AM (1997) The double hurdles for management research. In: Clarke T (ed) Advancement in organizational behaviour: essays in honour of D. S. Pugh. Dartmouth, London, pp 277–296

    Google Scholar 

  • Pettigrew AM (2001) Management research after modernism. Br J Manag 12:S61–S70

    Google Scholar 

  • Pfeffer J, Fong CT (2002) The end of business schools? Less success than meets the eye. Acad Manag Learn Education 1:78–95

    Google Scholar 

  • Pierson FC (1959) The education of American businessmen. MacGraw-Hill, New York

    Google Scholar 

  • Podsakoff PM, MacKenzie SB, Bachrach DG, Posakoff NP (2005) The influence of management journals in the 1980s and 1990s. Strateg Manag J 26:473–488

    Google Scholar 

  • Porter LW, McKibbin LE (1988) Management education and development: Drift or thrust into the 21st century?. MacGraw-Hill, New York

    Google Scholar 

  • Rafols I, Leydesdorff L, O’Hare A, Nightingale P, Stirling A (2012) How journal rankings can suppress interdisciplinary research: a comparison between innovation studies and business & management. Res Policy 41:1262–1282

    Google Scholar 

  • Rauch A, Wiklund J, Lumpkin GT, Frese M (2009) Entrepreneurial orientation and business performance: an assessment of past research and suggestions for the future. Entrepreneurship Theory Pract 33:761–787

    Google Scholar 

  • Rosenthal R (1979) The “file drawer problem” and tolerance for null results. Psychol Bull 86:638–641

    Google Scholar 

  • Rynes SL, Giluk TL, Brown KG (2007) The very separate worlds of academic and practitioner periodicals in human resource management: implications for evidence-based management. Acad Manag J 50:987–1008

    Google Scholar 

  • Schiele H, Krummaker S (2011) Consortium benchmarking: collaborative academic–practitioner case study research. J Bus Res 64:1137–1145

    Google Scholar 

  • Schrader U, Hennig-Thurau T (2009) VHB-Jourqual2: Method, results, and implications of the German Academic Association for Business Research’s Journal Ranking. BuR—Bus Res 2:180–204

    Google Scholar 

  • Scott WR (2001) Institutions and organizations, 2nd edn. Sage, Thousand Oaks

    Google Scholar 

  • Seglen PO (1997) Why the impact factor of journals should not be used for evaluating research. Br Med J 314:498–502

    Google Scholar 

  • Seidl D (2005) Glossary to Niklas Luhmann’s terminology. In: Seidl D, Becker KH (eds) Niklas Luhmann and organization studies. Copenhagen Business School Press, Kristianstad, pp 403–410

    Google Scholar 

  • Shadish WR, Tolliver D, Gray M, Sen Gupta SK (1995) Author judgements about works they cite: three studies from psychology journals. Soc Stud Sci 25:477–498

    Google Scholar 

  • Shapiro DL, Kirkman BL, Courtney HG (2007) Perceived causes and solutions of the translation problem in management research. Acad Manag J 50:249–266

    Google Scholar 

  • Shrivastava P (1987) Rigor and practical usefulness of research in strategic management. Strateg Manag J 8:77–92

    Google Scholar 

  • Singh G, Haddad KM, Chow CW (2007) Are articles in “top” management journals necessarily of higher quality? J Manag Inquiry 16:319–331

    Google Scholar 

  • Starbuck WH (2005) How much better are the most-prestigious journals? The statistics of academic publication. Organ Sci 16:180–200

    Google Scholar 

  • Starkey K, Madan P (2001) Bridging the relevance gap: aligning stakeholders in the future of management research. Br J Manag 12:S3–S26

    Google Scholar 

  • Starkey K, Hatchuel A, Tempest S (2009) Management research and the new logics of discovery and engagement. J Manage Stud 46:547–558

    Google Scholar 

  • Straub DW, Ang S (2008) Readability and the relevance versus rigor debate. MIS Quarterly 32:iii–xiii

    Google Scholar 

  • Suchman MC (1995) Managing legitimacy: strategic and institutional approaches. Acad Manag Rev 20:571–610

    Google Scholar 

  • Thornton PH, Ocasio W (1999) Institutional logics and the historical contingency of power in organizations: executive succession in the higher education publishing industry, 1958–1990. Am J Sociol 105:801–843

    Google Scholar 

  • Tolbert PS, Zucker LG (1983) Institutional sources of change in the formal structure of organizations: the diffusion of civil service reform, 1880–1935. Adm Sci Q 28:22–39

    Google Scholar 

  • Weiss CH, Bucuvalas MJ (1980) Truth tests and utility tests: decision-makers’ frames of reference for social science research. Am Sociol Rev 45:302–313

    Google Scholar 

  • Wiklund J (1998) Entrepreneurial orientation as predictor of performance and entrepreneurial behavior in small firms. In: Reynolds PD, Bygrave WD, Carter NM, Manigart S, Mason CM, Meyer GD, Shaver KG (eds) Frontiers of Entrepreneurship Research. Babson College, Wellesley, pp 281–296

    Google Scholar 

  • Wiklund J, Shepherd DA (2003) Knowledge-based resources, entrepreneurial orientation, and the performance of small and medium-sized businesses. Strateg Manag J 24:1307–1314

    Google Scholar 

  • Wolf J, Rosenberg T (2012) How individual scholars can reduce the rigor-relevance gap in management research. BuR—Bus Res 5:178–196

    Google Scholar 

  • Wolf J, Rohn A, Macharzina K (2005) Institution und Forschungsproduktivität—Befunde und Interpretationen aus der deutschsprachigen Betriebswirtschaftslehre. Die Betriebswirtschaft 65:62–77

    Google Scholar 

  • Zucker LG (1977) The role of institutionalization in cultural persistence. Am Sociol Rev 42:726–743

    Google Scholar 

  • Zucker LG (1987) Institutional theories of organizations. Annual Rev Sociol 13:443–464

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Passau, Innstr. 27, 94032, Passau, Germany

    Miriam Flickinger & Marina Fiedler

  2. Ludwig-Maximilians-University Munich, Ludwigsstr. 28, 80539, Munich, Germany

    Anja Tuschke

  3. University of Linz, Altenbergerstr. 69, 4040, Linz, Austria

    Tina Gruber-Muecke

Authors
  1. Miriam Flickinger
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anja Tuschke
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tina Gruber-Muecke
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Marina Fiedler
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Miriam Flickinger.

Appendix A

Appendix A

Appendix A Studies included in the meta-analysis
Full size table

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Flickinger, M., Tuschke, A., Gruber-Muecke, T. et al. In search of rigor, relevance, and legitimacy: what drives the impact of publications?. J Bus Econ 84, 99–128 (2014). https://doi.org/10.1007/s11573-013-0692-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11573-013-0692-2

Keywords

  • Rigor
  • Relevance
  • Legitimacy
  • Citation rates

JEL Classification

  • M10
  • Z1