Skip to main content
SpringerLink
Log in

Is It Possible to Assess Progress in Science?

  • Chapter
  • First Online:
Multi-Level Governance in Universities

Part of the book series: Higher Education Dynamics ((HEDY,volume 47))

Abstract

Rankings create, at best, only a snapshot impression of the importance of specific research results within a discipline (see e.g. the currently most popular ranking based on journal impact factors, offered by Thomson Reuters). Rankings do not help to evaluate to what extent the respective research results contribute to scientific progress. They tend to impede progress of science rather than to advance it. This paper discusses (if and) how science progresses and whether it is generally possible to evaluate scientific progress. We discuss different concepts of scientific progress and elaborate the hypothesis that the creation of disciplines and subdisciplines – a process that is frequently driven by invisible colleges – significantly contributes to scientific progress. Our paper identifies prerequisites for a successful development of an invisible college towards a (sub-)discipline that have to be taken into account in attempts to develop indicators for scientific progress.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 119.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 159.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Tolstoy’s novel Anna Karenina starts with the sentence: „Happy families are all alike; every unhappy family is unhappy in its own way.” The implication being that to succeed in their striving for happiness, a family has to fulfill all preconditions. If one precondition, e.g. financial security, is not met, the likelihood for unhappiness is very high.

References

  • Adler, N. J., & Harzing, A. W. (2009). When knowledge wins: Transcending the sense and nonsense of academic rankings. Academy of Management Learning & Education, 8(1), 72–95.

    Article  Google Scholar 

  • Alvesson, M., & Sandberg, J. (2013). Has management studies lost its way? Ideas for more imaginative and innovative research. Journal of Management Studies, 50(1), 128–152.

    Article  Google Scholar 

  • Alvesson, M., & Sandberg, J. (2014). Habitat and habitus: Boxed-in versus box-breaking research. Organization Studies, 35(7), 967–987. doi:10.1177/0170840614530916.

    Article  Google Scholar 

  • Augier, M. (2013). The early evolution of the foundations for behavioral organization theory and strategy. European Management Journal, 31(1), 72–81. doi:10.1016/j.emj.2012.11.005.

    Article  Google Scholar 

  • Augier, M., & Prietula, M. (2007). Historical roots of the a behavioral theory of the firm model at GSIA. Organization Science, 18, 507–522.

    Article  Google Scholar 

  • Augier, M., Kreiner, K., & March, J. G. (2000). Introduction: Some roots and branches of organizational economics. Industrial and Corporate Change, 9, 555–566.

    Article  Google Scholar 

  • Augier, M., March, J. G., & Sullivan, B. N. (2005). Notes on the evolution of a research community: Organization studies in anglophone North America, 1945–2000. Organization Science, 16(1), 85–95. doi:10.1287/orsc.1040.0108.

    Article  Google Scholar 

  • Barnes, B. (1982). T. S. Kuhn and social science. London: Macmillan.

    Book  Google Scholar 

  • Baum, J. A. (2011). Free-riding on power laws: Questioning the validity of the impact factor as a measure of research quality in organization studies. Organization, 18(4), 449–466.

    Article  Google Scholar 

  • Bedeian, A. G. (2004). Peer review and the social construction of knowledge in the management discipline. Academy of Management Learning & Education, 3(2), 198–216.

    Article  Google Scholar 

  • Bendor, J., Moe, T. M., & Shotts, K. W. (2001). Recycling the garbage can: An assessment of the research program. American Political Science Review, 95, 169–702.

    Google Scholar 

  • Blute, M. (1972). The growth of science and economic development. American Sociological Review, 37, 455–464.

    Article  Google Scholar 

  • Bornmann, L., & Marx, W. (2012). The Anna Karenina principle: A way of thinking about success in science. Journal of the American Society for Information Science and Technology, 63(10), 2037–2051. doi:10.1002/asi.22661.

    Article  Google Scholar 

  • Burrows, R. (2012). Living with the h-index? Metric assemblages in the contemporary academy. The Sociologica Review, 60(2), 355–372.

    Article  Google Scholar 

  • Campanario, J. M. (1996). Have referees rejected some of the most-cited articles of all times? Journal of the American Society for Information Science, 47, 302–310.

    Article  Google Scholar 

  • Cohen, M. D., March, J. G., & Olsen, J. P. (1972). A garbage can model of organizational choice. Administrative Science Quarterly, 17(1), 1–25.

    Article  Google Scholar 

  • Cole, S. (1983). The hierarchy of the sciences? American Journal of Sociology, 89(1), 111–139.

    Article  Google Scholar 

  • Crane, D. (1969). Social structure in a group of scientists: A test of the “invisible college” hypothesis. American Sociological Review, 34, 335–352.

    Article  Google Scholar 

  • Crane, D. (1972). Invisible colleges: Diffusion of knowledge in scientific communities. Chicago/London: University of Chicago Press.

    Google Scholar 

  • Cronin, B. (1982). Invisible colleges and information transfer: A review and commentary with particular reference to the social sciences. Journal of Documentation, 38(3), 212–236.

    Article  Google Scholar 

  • Cyert, R. M., & March, J. G. (1963). A behavioral theory of the firm. Englewood Cliffs: Prentice Hall.

    Google Scholar 

  • De Rond, M., & Miller, A. N. (2005). Publish or perish: Bane or boon of academic life? Journal of Management Inquiry, 14(4), 321–329.

    Article  Google Scholar 

  • De Solla Price, D. J., & Beaver, D. (1966). Collaboration in an invisible college. American Psychologist, 21(11), 1011–1018.

    Article  Google Scholar 

  • Fagerberg, J., & Verspagen, B. (2009). Innovation studies. The emerging structure of a new scientific field. Research Policy, 38(2), 218–233.

    Article  Google Scholar 

  • Feyerabend, P. (1993). Against method. London: Verso.

    Google Scholar 

  • Gans, J. S., & Shepherd, G. B. (1994). How are the mighty fallen: Rejected classic articles by leading economists. Journal of Economic Perspectives, 8(1), 165–180.

    Article  Google Scholar 

  • Garfield, E. (1994). Using the impact factor. http://wokinfo.com/benefits/essays/usingimpactfactor/. Accessed 22 Aug 2012.

  • Garfield, E. (2006). The history and meaning of the journal impact factor. Journal of the American Medical Association, 295(1), 90–93.

    Article  Google Scholar 

  • Gibbons, R. (2003). Team theory, garbage cans and real organizations: Some history and prospects of economic research on decision-making in organizations. Industrial and Corporate Change, 12, 753–787.

    Article  Google Scholar 

  • Glimcher, P. W., Camerer, C., Fehr, E., & Poldrack, R. A. (Eds.). (2013). Neuroeconomics: Decision making and the brain. London: Academic.

    Google Scholar 

  • Gmür, M. (2003). Co-citation analysis and the search for invisible colleges: A methodological evaluation. Scientometrics, 57(1), 27–57.

    Article  Google Scholar 

  • Goodall, A. H. (2008). Why have the leading journals in management (and other social sciences) failed to respond to climate change? Journal of Management Inquiry, 17(4), 408–420. doi:10.1177/1056492607311930.

    Article  Google Scholar 

  • Grey, C. (2010). Organizing studies: Publications, politics and polemic. Organization Studies, 31(6), 677–694.

    Article  Google Scholar 

  • Habermas, J. (1970). Erkenntnis und Interesse. Frankfurt am Main: Suhrkamp.

    Google Scholar 

  • Hall, M. B. (2002). Promoting experimental learning: Experiment and the Royal Society 1660–1727. Cambridge: Cambridge University Press.

    Google Scholar 

  • Hambrick, D. C., & Chen, M. J. (2008). New academic fields as admittance-seeking social movements: The case of strategic management. Academy of Management Review, 33(1), 32–54.

    Article  Google Scholar 

  • Hattke, F., Vogel, R., & Woiwode, H. (2016). When professional and organizational logics collide: Balancing invisible and visible colleges in institutional complexity. In J. Frost, F. Hattke, & M. Reihlen (Eds.), Multilevel governance in universities (pp. 235–256). Dordrecht: Springer.

    Google Scholar 

  • Hummon, N. P., & Carley, K. (1993). Social networks as normal science. Social Networks, 15(1), 71–106.

    Article  Google Scholar 

  • Keohane, R. (2002). Intergovernmental organizations and garbage can theory. Journal of Public Administration Research & Theory, 12(2), 155.

    Article  Google Scholar 

  • Kuhn, T. S. (2012). The structure of scientific revolutions. Chicago: University of Chicago Press.

    Book  Google Scholar 

  • Laudel, G., & Origgi, G. (2006). Introduction to a special issue on the assessment of interdisciplinary research. Research Evaluation, 15(1), 2–4.

    Article  Google Scholar 

  • Levitt, B., & Nass, C. (1989). The lid on the garbage can: Institutional constraints on decision making in the technical core of college-text publishers. Administrative Science Quarterly, 34(2), 190–207.

    Article  Google Scholar 

  • Leydesdorff, L., Wagner, C. S., Park, H. W., & Adams, J. (2013). International collaboration in science: The global map and the network. El profesional de la infomación, 22(1), 87–95.

    Article  Google Scholar 

  • Lievrouw, L. A. (1989). The invisible college reconsidered bibliometrics and the development of scientific communication theory. Communication Research, 16(5), 615–628.

    Article  Google Scholar 

  • Lindsey, D. (1989). Using citation counts as a measure of quality in science: Measuring what’s measurable rather than what’s valid. Scientometrics, 15(3–4), 187–203.

    Google Scholar 

  • Lo, M., & Bao, Y. (2016). Are overall journal rankings a good mapping for article quality in specialty fields? Journal of Business & Economic Statistics, 34(1), 62–67.

    Article  Google Scholar 

  • Lomi, A., & Harrison, R. J. (Eds.). (2012). The garbage can model of organizational choice: Looking forward at forty (Vol. 36). Bingley: Emerald Books.

    Google Scholar 

  • March, J. G. (1991). Exploration and exploitation in organizational learning. Organization Science, 2(1), 71–87.

    Article  Google Scholar 

  • March, J. G. (2004). Parochialism in the evolution of a research community: The case of organization studies. Management and Organization Review, 1(1), 5–22.

    Article  Google Scholar 

  • March, J. G., & Olsen, J. P. (1986). Garbage can models of decision making in organizations. In J. G. March & R. Weissinger-Baylon (Eds.), Ambiguity and command (pp. 11–36). Cambridge, MA: Ballinger.

    Google Scholar 

  • March, J. G., & Simon, H. A. (1958). Organizations. New York: Wiley.

    Google Scholar 

  • Markóczy, L., & Deeds, D. L. (2009). Theory building at the intersection: Recipe for impact or road to nowhere? Journal of Management Studies, 46(6), 1076–1088.

    Article  Google Scholar 

  • Mayo-Smith, R. (1888). Statistics and economics. Baltimore: American Economic Association.

    Google Scholar 

  • Merton, R. K. (1968). The Matthew effect in science. Science, 159(3810), 56–63.

    Article  Google Scholar 

  • Mezias, S. J., & Scarselletta, M. (1994). Resolving financial reporting problems: An institutional analysis of the process. Administrative Science Quarterly, 39(4), 654–678.

    Article  Google Scholar 

  • Mingers, J., & Willmott, H. (2013). Taylorizing business school research: On the ‘one best way’ performative effects of journal ranking lists. Human Relations, 66(8), 1051–1073. doi:10.1177/0018726712467048.

    Article  Google Scholar 

  • Mullins, N. C. (1972). The development of a scientific specialty: The phage group and the origins of molecular biology. Minerva, 10, 51–82.

    Article  Google Scholar 

  • Münch, R. (2007). Die akademische Elite: Zur sozialen Konstruktion wissenschaftlicher Exzellenz. Frankfurt am Main: Suhrkamp.

    Google Scholar 

  • Münch, R. (2011a). Akademischer Kapitalismus. Zur Politischen Ökonomie der Hochschulreform. Berlin: Suhrkamp.

    Google Scholar 

  • Münch, R. (2011b). Sieger und Besiegte. Wie der ökonomische Wettbewerb zunehmend den wissenschaftlichen kolonisiert. Forschung und Lehre, 18(7), 512–515.

    Google Scholar 

  • Nahapiet, J., & Ghoshal, S. (1998). Social capital, intellectual capital, and the organizational advantage. Academy of Management Review, 23(2), 242–266.

    Google Scholar 

  • Nobuyuki, I. (2015). Garbage can code: Mysteries in the original simulation model. Annals of Business Administrative Science, 14(1), 15–34. doi:10.7880/abas.14.15.

    Article  Google Scholar 

  • Oswick, C., Fleming, P., & Hanlon, G. (2011). From borrowing to blending: Rethinking the processes of organizational theory building. Academy of Management Review, 36(2), 318–337. doi:10.5465/AMR.2011.59330932.

    Google Scholar 

  • Overington, M. A. (1977). The scientific community as audience: Toward a rhetorical analysis of science. Philosophy and Rhetoric, 10, 143–164.

    Google Scholar 

  • Padgett, J. F. (2013). Review essay on Alessandro Lomi and J. Richard Harrison (Eds.), Research in the sociology of organizations, vol. 36: The garbage can model of organizational choice: Looking forward at forty. Administrative Science Quarterly, 58(3), 472–482. doi:10.1177/0001839213500231.

    Article  Google Scholar 

  • Paisley, W. (1972). The role of invisible colleges in scientific information transfer. Educational Researcher, 1(4), 5–19.

    Article  Google Scholar 

  • Parkhe, A. (1993). “Messy” research, methodological predispositions, and theory development in international joint ventures. Academy of Management Review, 18(2), 227–268.

    Google Scholar 

  • Pfeffer, J. (1993). Barriers to the advance of organization science: Paradigm development as a dependent variable. Academy of Management Review, 18(4), 599–620.

    Google Scholar 

  • Polsby, N. W. (1998). Social science and scientific change: A note on Thomas S. Kuhn’s contribution. Annual Review of Political Science, 1, 199–210.

    Article  Google Scholar 

  • Popper, K. (1962). Conjectures and refutations. New York: Harper & Row.

    Google Scholar 

  • Rafols, I., Leydesdorff, L., O’Hare, A., Nightingale, P., & Stirling, A. (2012). How journal rankings can suppress interdisciplinary research: A comparison between innovation studies and business & management. Research Policy, 41(7), 1262–1282. doi:10.1016/j.respol.2012.03.015.

    Article  Google Scholar 

  • Rinia, E. J., van Leeuwen, T. N., van Vuren, H. G., & van Raan, A. F. J. (2001). Influence of interdisciplinarity on peer-review and bibliometric evaluations in physics research. Research Policy, 30(3), 357–361. doi:http://dx.doi.org/10.1016/S0048-7333(00)00082-2.

    Google Scholar 

  • Sager, F., & Rielle, Y. (2013). Sorting through the garbage can: Under what conditions do governments adopt policy programs? Policy Sciences, 46(1), 1–21. doi:10.1007/s11077-012-9165-7.

    Article  Google Scholar 

  • Saßmannshausen, S. P., & Volkmann, C. (2013). A bibliometric based review on social entrepreneurship and its establishment as a field of research. Schumpeter Discussion Papers.

    Google Scholar 

  • Shore, C., & Wright, S. (2000). Coercive accountability: The rise of audit culture in higher education. In M. Strathern (Ed.), Audit cultures: Anthropological studies in accountability, ethics and the academy (pp. 57–89). London: Routledge.

    Google Scholar 

  • Siedlok, F., & Hibbert, P. (2014). The organization of interdisciplinary research: Modes, drivers and barriers. International Journal of Management Reviews, 16(2), 194–210.

    Article  Google Scholar 

  • Simon, H. A. (1976). Administrative behavior (3rd ed.). New York: The Free Press.

    Google Scholar 

  • Stichweh, R. (1979). Differenzierung der Wissenschaft. Zeitschrift für Soziologie, 8(1), 82–101.

    Article  Google Scholar 

  • Stichweh, R. (1992). The sociology of scientific disciplines: On the genesis and stability of the disciplinary structure of modern science. Science in Context, 5(1), 3–15.

    Article  Google Scholar 

  • Stirling, A. (1998). On the economics and analysis of diversity. SPRU Electronic Working Papers, 28. http://www.sussex.ac.uk/Units/spru/publications/imprint/sewps/sewp28/sewp28.pdf. Accessed 17 Dec 2014.

  • Stone, D. (2007). Recycling bins, garbage cans or think tanks? Three myths regarding policy analysis institutes. Public Administration, 85(2), 259–278. doi:10.1111/j.1467-9299.2007.00649.x.

    Article  Google Scholar 

  • Teasley, C. E., III, & Harrell, S. W. (1996). A real garbage can decision model: Measuring the costs of politics with a computer assisted decision support software (DSS) program. Public Administration Quarterly, 19(4), 479–492.

    Google Scholar 

  • Togia, A., & Tsigilis, N. (2006). Impact factor and education journals: A critical examination and analysis. International Journal of Educational Research, 45(6), 362–379.

    Article  Google Scholar 

  • Verspagen, B., & Werker, C. (2004). Keith Pavitt and the invisible college of the economics of technology and innovation. Research Policy, 33(9), 1419–1431.

    Article  Google Scholar 

  • Vogel, R. (2012). The visible colleges of management and organization studies: A bibliometric analysis of academic journals. Organization Studies, 33(8), 1015–1043.

    Article  Google Scholar 

  • Wagner, C. S. (2008). The new invisible college: Science for development. Washington, DC: Brookings Institution Press.

    Google Scholar 

  • Whitley, R. (1984). The intellectual and social organization of the sciences. Oxford: Oxford University Press.

    Google Scholar 

  • Willmott, H. (2011). Journal list fetishism and the perversion of scholarship: Reactivity and the ABS list. Organization, 18(4), 429–442.

    Article  Google Scholar 

  • Zahra, S. A., & Newey, R. L. (2009). Maximizing the impact of organization science: Theory-building at the intersection of disciplines and/or fields. Journal of Management Studies, 46(6), 1059–1075.

    Article  Google Scholar 

  • Zaltman, G. (1974). A note on an international invisible college for information exchange. Journal of the American Society for Information Science, 25(2), 113–117.

    Article  Google Scholar 

  • Zuccala, A. (2006). Modeling the invisible college. Journal of the American Society for Information Science and Technology, 57(2), 152–168.

    Article  Google Scholar 

Download references

Acknowledgments

This research is supported by the German Federal Ministry of Education and Research (Grant No. 01PY13014).

Author information

Authors and Affiliations

  1. Management Theory, Zeppelin University, Friedrichshafen, Germany

    Isabel Bögner & Alfred Kieser

  2. Public Management, Hamburg University, Hamburg, Germany

    Jessica Petersen

Authors
  1. Isabel Bögner
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jessica Petersen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alfred Kieser
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Isabel Bögner .

Editor information

Editors and Affiliations

  1. Organization and Management, University of Hamburg, Hamburg, Germany

    Jetta Frost

  2. Organization and Management, University of Hamburg, Hamburg, Germany

    Fabian Hattke

  3. Strategic Management, Leuphana University Lüneburg, Lüneburg, Germany

    Markus Reihlen

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Bögner, I., Petersen, J., Kieser, A. (2016). Is It Possible to Assess Progress in Science?. In: Frost, J., Hattke, F., Reihlen, M. (eds) Multi-Level Governance in Universities. Higher Education Dynamics, vol 47. Springer, Cham. https://doi.org/10.1007/978-3-319-32678-8_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-32678-8_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-32676-4

  • Online ISBN: 978-3-319-32678-8

  • eBook Packages: EducationEducation (R0)

Publish with us

Policies and ethics

Search

Navigation