Abstract
Educational researchers have been evaluating ourselves and our colleagues’ work using criteria that are based almost exclusively on bibliometrics, which are in turn based on quantities of publications and citations. In misguided attempts to conduct scientific and ‘objective’, evaluations of research, institutions (universities, foundations, and governments) have adopted instead populist, reductionist, and non-rigorous approaches to evaluating productivity in educational research. It is an example of governmentality operating as a technology of normalization that oversimplified notions of productivity have been imported uncritically to govern many research-evaluation exercises. This chapter problematizes bibliometric criteria of evaluation, and suggests instead that we pay attention to purposeful rigor and ethics in assessments of research productivity. Fortuitously, research-based conceptualizations of productivity, together with literal readings of institutional policy language open the door for the inclusion of ethical judgments into the process of evaluating productivity in educational research.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Geert Bouckaert is a KU-Leuven professor of Public Administration.
- 2.
This may appear to be somewhat ironic since most academic governance policies claim to be adhering to a business-model approach for evaluating educational research.
References
Abramo, G., & D’Angelo, C. A. (2014 November). How do you define and measure research productivity? Scientometrics, 101(2), 1129–1144.
Alexander von Humboldt Foundation. (2015). Beyond bibliometrics: Identifying the best proceedings of the 8th forum on the internationalization of sciences and humanities. Available https://www.humboldt-foundation.de/pls/web/docs/F-698959692/duzSpecial_AvH_M04_2015_WEB.pdf.
Allen, D. (2002). Getting things done: The art of stress-free productivity. New York: Penguin Books.
Ball, S. J. (2010). New voices, new knowledges and the new politics of educational research: The gathering of a perfect storm? European Educational Research Journal, 9(2), 124–137.
Borchardt, R., & Hartings, M. R. (2019). The academic papers researchers regard as significant are not those that are highly cited | Impact of Social Sciences. London School of Economics and Political Science Impact Blog. https://blogs.lse.ac.uk/impactofsocialsciences/2018/05/14/the-academic-papers-researchers-regard-as-significant-are-not-those-that-are-highly-cited/.
Boulding, K. E. (1972). The schooling industry as a possibly pathological section of the American economy. Review of Educational Research, 42(1), 129–143. https://doi.org/10.3102/00346543042001129.
Bouckaert, G. (1990 Autumn). The history of the productivity movement. Public Productivity and Management Review, 14(1), 53–89.
Boyer, E. L. (1990). Scholarship reconsidered: Priorities of the professoriate. Special report from the Carnegie Foundation for the Advancement of Teaching. Available https://depts.washington.edu/gs630/Spring/Boyer.pdf.
Dalbert. (2011 May 12). Ten important reasons to include the humanities in your preparation for a scientific career. Science. Available http://blogs.sciencemag.org/sciencecareers/2011/05/ten-important-r.html.
Douglass, A. (Ed.) (2016). The new flagship university: Changing the paradigm from global ranking to national relevancy. Houndmills, Basingstoke, Hampshire: Palgrave Macmillan.
Duhigg, C. (2016). Smarter faster better: The secrets of being productive in life and business. New York: Random House.
Fallows, J. (2004 July/August). Organize your life! The Atlantic. Available https://www.theatlantic.com/magazine/archive/2004/07/organize-your-life/303455/.
Greiner, B. (2014). The wealth of nations part 2: The history of productivity. Forbes. Available https://www.forbes.com/sites/billgreiner/2014/06/25/the-wealth-of-nations-part-2-the-history-of-productivity/#1265996d6933.
Hatry, H. P., & Fisk, D. M. (1971). Improving productivity and productivity measurement in local governments. [Washington].
Kirchherr, J. (2018 August 9). A Ph.D. should be about improving society, not chasing academic kudos. Higher Education Network: The Guardian. Available https://www.theguardian.com/higher-education-network/2018/aug/09/a-phd-should-be-about-improving-society-not-chasing-academic-kudos.
Konkiel, S. (2016). Altmetrics: diversifying the understanding of influential scholarship. Palgrave Communications, 2, 16057. https://doi.org/10.1057/palcomms.2016.57.
Konkiel, S. (2017 October 19). Transforming research: Can an exploration of values move research evaluation practices forward? Humane Metrics Initiative. Available http://humetricshss.org/blog/transforming-research-2017/.
Lindner, M. D. (2018). Scientific productivity: An exploratory study of metrics and incentives. PLoS One. 13(4): e0195321. Published online Apr 3. https://doi.org/10.1371/journal.pone.0195321. PMCID: PMC5882165. PMID: 29614101. Available: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5882165/.
Long, C. (2017 July 10). Creating humane metrics for the Humanities and Social Sciences. (blogpost). Michigan State University Spartan Ideas. Available https://spartanideas.msu.edu/2017/07/10/creating-humanemetrics-for-the-humanities-and-social-sciences/.
Michigan State University. (2017). Faculty handbook: Appointment, Reappointment, Tenure, and Promotion Recommendations. Available https://www.hr.msu.edu/policies-procedures/faculty-academic-staff/faculty-handbook/recommendations.html.
REF 2021. (2018). What is the REF? Available http://www.ref.ac.uk/about/whatref/.
San Francisco Declaration on Research Assessment. (2012). Available https://sfdora.org/read/.
Schierenbeck, C. (2013). Fixing higher education: A business manager’s take on how to boost productivity in higher education. Wiesbaden: Springer Fachmedien.
Shahjahan, R. A. (2015). Being ‘lazy’ and slowing down: Toward decolonizing time, our body, and pedagogy. Educational Philosophy and Theory, 47(5), 488–501.
Smeyers, P., & Burbules, N. C. (2011). How to improve your impact factor: Questioning the quantification of academic quality. Journal of the Philosophy of Education, 45(1), 1–17.
Stephan, P., Veugelers, R., & Wang, J. (2017 April 27). Blinkered by bibliometrics. Nature, 544. Available https://www.nature.com/polopoly_fs/1.21877!/menu/main/topColumns/topLeftColumn/pdf/544411a.pdf.
University of Illinois. (2017). Promotion and tenure: Office of the provost communication No. 9. Available https://provost.illinois.edu/files/2017/06/Communication_9-effective-5-5-2017.pdf.
Zantal-Wiener, A. (2017 January 30). A brief history of productivity: How getting stuff done became an industry. Hubspot. Available https://blog.hubspot.com/marketing/a-brief-history-of-productivity.
Zuccala, A. (2012). Quality and influence in literary work: Evaluating the educated imagination. Research Evaluation, 21(3), 229-241.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this chapter
Cite this chapter
Fendler, L. (2021). Evaluating Productivity in Educational Research: Evaluation Criteria of Rigor and Ethics. In: Smeyers, P., Depaepe, M. (eds) Production, Presentation, and Acceleration of Educational Research: Could Less be More?. Educational Research, vol 11. Springer, Singapore. https://doi.org/10.1007/978-981-16-3017-0_5
Download citation
DOI: https://doi.org/10.1007/978-981-16-3017-0_5
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-3016-3
Online ISBN: 978-981-16-3017-0
eBook Packages: EducationEducation (R0)