Abstract
Since 2011, in Kazakhstan, one of the main indicators for assessing the effectiveness of researchers is their publication activity in journals indexed by Scopus. The policy implemented by the government had a positive effect on the growth of the number of publications, however, simultaneously, dishonest practices and the use of questionable journals as channels for publication became widespread. This study identifies how the publication management system at a university in Kazakhstan is changing the publication strategies of its staff. The study was conducted based on the data of the internal rating conducted by Al-Farabi Kazakh National University (KazNU) during 2015–2019. Such data were obtained using an automated publication tracking system based on API Scopus. This study shows that introduction of mechanisms aimed to stimulate publication activity in KazNU had a positive effect on the research productivity, but simultaneously led to attempts to manipulate rating indicators on the part of individual researchers. The introduction of indicators based on the quality and impact metrics of the journals, in turn, positively influenced the publication strategy in terms of choice of journals and decrease of publications in journals with dubious reputations. The study results can be used to develop a strategy for publication activity in a university, especially in developing countries faced with publication inflation, an unprecedented increase in publications in “predatory” journals, and the emergence of dishonest practices aimed at manipulating bibliometric indicators.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
List of journals discontinued by Scopus is official released by Elsevier. See, https://www.elsevier.com/__data/assets/excel_doc/0005/877523/Discontinued-sources-from-Scopus.xlsx.
References
Aagaard, K. (2015). How incentives trickle down: Local use of a national bibliometric indicator system. Science and Public Policy, 42(5), 725–737. https://doi.org/10.1093/scipol/scu087
Aagaard, K. (2018). Performance-based research funding in Denmark: The adoption and translation of the Norwegian model. Journal of Data and Information Science, 3(4), 20–30. https://doi.org/10.2478/jdis-2018-0018
Aagaard, K., Bloch, C., & Schneider, J. W. (2015). Impacts of performance-based research funding systems: The case of the Norwegian publication indicator. Research Evaluation, 24(2), 106–117. https://doi.org/10.1093/reseval/rvv003
Abritis, A., & McCook, A. (2017). Cash bonuses for peer-reviewed papers go global. Science, 357, 541. https://doi.org/10.1126/science.aan7214
Agasisti, T., Catalano, G., Landoni, P., & Verganti, R. (2012). Evaluating the performance of academic departments: An analysis of research-related output efficiency. Research Evaluation, 21(1), 2–14. https://doi.org/10.1093/reseval/rvr001
Ahlgren, P., & Waltman, L. (2014). The correlation between citation-based and expert-based assessments of publication channels: SNIP and SJR vs. Norwegian quality assessments. Journal of Informetrics, 8(4), 985–996. https://doi.org/10.1016/j.joi.2014.09.010
Aida, M., & Watanabe, S. P. (2016). Quantifying faculty productivity in Japan: Development and application of the Achievement-Motivated Key Performance Indicator. University of California at Berkeley, Center for Studies in Higher Education.
Aksnes, D. W. (2005). Citations and their use as indicators in science policy: Studies of validity and applicability issues with a particular focus on highly cited papers. Doctoral Thesis, Enschede, the Netherlands: University of Twente.
Antonowicz, D., Kohoutek, J., Pinheiro, R., & Hladchenko, M. (2017). The roads of “excellence” in Central and Eastern Europe. European Educational Research Journal, 16(5), 547–567. https://doi.org/10.1177/1474904116683186
Butler, L. (2010). Impacts of performance-based research funding systems: A review of the concerns and the evidence. Performance-based Funding for Public Research in Tertiary Education Institutions: Workshop Proceedings. Paris, OECD, 127–165. https://doi.org/10.1787/9789264094611-7-en
Butler, L. (2003). Explaining Australia’s increased share of ISI publications - The effects of a funding formula based on publication counts. Research Policy, 32(1), 143–155. https://doi.org/10.1016/S0048-7333(02)00007-0
de Rijcke, S., Wouters, P. F., Rushforth, A. D., Franssen, T. P., & Hammarfelt, B. (2016). Evaluation practices and effects of indicator use-a literature review. Research Evaluation, 25(2), 161–169. https://doi.org/10.1093/reseval/rvv038
Franzoni, C., Scellato, G., & Stephan, P. (2011). Changing Incentives to Publish. Science, 333(6043), 702–703. https://doi.org/10.1126/science.1197286
Gao, X., & Zheng, Y. (2020). ‘Heavy mountains’ for Chinese humanities and social science academics in the quest for world-class universities. Compare-a Journal of Comparative and International Education, 50(4), 554–572. https://doi.org/10.1080/03057925.2018.1538770
Glänzel, W., Debackere, K., Thijs, B., & Schubert, A. (2006). A concise review on the role of author self-citations in information science, bibliometrics and science policy. Scientometrics, 67(2), 263–277. https://doi.org/10.1007/s11192-006-0098-9
Good, B., Vermeulen, N., Tiefenthaler, B., & Arnold, E. (2015). Counting quality? The Czech performance-based research funding system. Research Evaluation, 24(2), 91–105. https://doi.org/10.1093/reseval/rvu035
Gordeeva, E. (2019). State programs for innovative development of the national economy: Theoretical foundations, applied performance and strategic priorities. Doctoral Thesis, Karaganda, Kazakhstan: E.A. Buketov Karaganda State University.
GRK. (2017). Resolution of the government of the Republic of Kazakhstan dated November 29, 2017 No. 790 “On approval of the state planning system in the Republic of Kazakhstan.” Republic of Kazakhstan. Retrieved January 10, 2021, from http://adilet.zan.kz/rus/docs/P1700000790.
GRK. (2019). Resolution of the government of the Republic of Kazakhstan dated December 27, 2019 No. 988 “On approval of the State Program for the Development of Education and Science of the Republic of Kazakhstan for 2020 — 2025.” Republic of Kazakhstan. Retrieved January, 10, 2021, from http://adilet.zan.kz/rus/docs/P1900000988.
Guskov, A. E., Kosyakov, D. V., & Selivanova, I. V. (2018). Boosting research productivity in top Russian universities: The circumstances of breakthrough. Scientometrics, 117(2), 1053–1080. https://doi.org/10.1007/s11192-018-2890-8
Hammarfelt, B., & de Rijcke, S. (2015). Accountability in context: Effects of research evaluation systems on publication practices, disciplinary norms, and individual working routines in the faculty of Arts at Uppsala University. Research Evaluation, 24(1), 63–77. https://doi.org/10.1093/reseval/rvu029
Hammarfelt, B., Nelhans, G., Eklund, P., & Åström, F. (2016). The heterogeneous landscape of bibliometric indicators: Evaluating models for allocating resources at Swedish universities. Research Evaluation, 25(3), 292–305. https://doi.org/10.1093/reseval/rvv040
Hansen, H. F., Geschwind, L., Kivistö, J., Pekkola, E., Pinheiro, R., & Pulkkinen, K. (2019). Balancing accountability and trust: University reforms in the Nordic countries. Higher Education, 78(3), 557–573. https://doi.org/10.1007/s10734-019-0358-2
Hartley, M., Gopaul, B., Sagintayeva, A., & Apergenova, R. (2016). Learning autonomy: Higher education reform in Kazakhstan. Higher Education, 72(3), 277–289. https://doi.org/10.1007/s10734-015-9953-z
Hicks, D. (2012). Performance-based university research funding systems. Research Policy, 41(2), 251–261. https://doi.org/10.1016/j.respol.2011.09.007
Horta, H., & Shen, W. (2020). Current and future challenges of the Chinese research system. Journal of Higher Education Policy and Management, 42(2), 157–177. https://doi.org/10.1080/1360080X.2019.1632162
Hossain, M. N., & Ahmed, S. M. Z. (2020). Use of scholarly communication and citation-based metrics as a basis for university ranking in developing country perspective. Global Knowledge, Memory and Communication, 69(6/7), 461–482. https://doi.org/10.1108/GKMC-09-2019-0112
Jiang, X., Borg, E., & Borg, M. (2017). Challenges and coping strategies for international publication: Perceptions of young scholars in China. Studies in Higher Education, 42(3), 428–444. https://doi.org/10.1080/03075079.2015.1049144
Jonbekova, D. (2020). Educational research in Central Asia: Methodological and ethical dilemmas in Kazakhstan, Kyrgyzstan and Tajikistan. Compare-a Journal of Comparative and International Education, 50(3), 352–370. https://doi.org/10.1080/03057925.2018.1511371
Kenessov, B. (2020). Percentage of articles and reviews by countries in sources discontinued by Scopus. https://data.mendeley.com/datasets/k8pvz45gp2/2.
Korytkowski, P., & Kulczycki, E. (2019). Examining how country-level science policy shapes publication patterns: The case of Poland. Scientometrics, 119(3), 1519–1543. https://doi.org/10.1007/s11192-019-03092-1
Kotsemir, M., & Shashnov, S. (2017). Measuring, analysis and visualization of research capacity of university at the level of departments and staff members. Scientometrics, 112(3), 1659–1689. https://doi.org/10.1007/s11192-017-2450-7
Kulczycki, E., Engels, T. C. E., Pölönen, J., Bruun, K., Duškoviá, M., Guns, R., Nowotniak, R., Petr, M., Sivertsen, G., Istenič Starčič, A., & Zuccala, A. (2018). Publication patterns in the social sciences and humanities: Evidence from eight European countries. Scientometrics, 116(1), 463–486. https://doi.org/10.1007/s11192-018-2711-0
Kulczycki, E., Korzeń, M., & Korytkowski, P. (2017). Toward an excellence-based research funding system: Evidence from Poland. Journal of Informetrics, 11(1), 282–298. https://doi.org/10.1016/j.joi.2017.01.001
Kurambayev, B., & Freedman, E. (2021). Publish or perish? The steep, steep path for Central Asia journalism and mass communication faculty. Journalism and Mass Communication Educator, 76(2), 228–240. https://doi.org/10.1177/1077695820947259
Kuzhabekova, A., & Ruby, A. (2018). Raising research productivity in a post-soviet higher education system: A case from Central Asia. European Education, 50(3), 266–282. https://doi.org/10.1080/10564934.2018.1444942
Ma, L. (2019). Money, morale, and motivation: A study of the output-based research support scheme in University College Dublin. Research Evaluation, 28(4), 304–312. https://doi.org/10.1093/reseval/rvz017
Ma, L., & Ladisch, M. (2019). Evaluation complacency or evaluation inertia? A study of evaluative metrics and research practices in Irish universities. Research Evaluation, 28(3), 209–217. https://doi.org/10.1093/reseval/rvz008
Macháček, V., & Srholec, M. (2021). Predatory publishing in Scopus: Evidence on cross-country differences. Scientometrics, 126(3), 1897–1921. https://doi.org/10.1007/s11192-020-03852-4
MES. (2011a). Law of the Republic of Kazakhstan dated February 18, 2011 No. 407-IV “On Science.” Retrieved January 10, 2021, from https://online.zakon.kz/document/?doc_id=30938581#pos=57.
MES 127. (2011b). Order of the Minister of Education and Science of the Republic of Kazakhstan dated March 31, 2011 No. 127 “On approval of the Rules for awarding degrees.” Retrieved January 10, 2021, from http://adilet.zan.kz/rus/docs/V1100006951.
MES 128. (2011c) Order of the minister of education and science of the Republic of Kazakhstan dated March 31, 2011 No. 128 “On approval of the Rules for conferring academic titles (associated professor (docent), professor).” Retrieved January 10, 2021, from http://adilet.zan.kz/rus/docs/V1100006939.
Mutanov, G. (2016). Report on research and innovation activities of Al-Farabi KazNU for 2015 (p. 325). Al-Farabi Kazakh National University.
Mutanov, G. (2017). Report on research and innovation activities of Al-Farabi KazNU for 2016 (p. 582). Al-Farabi Kazakh National University.
Mutanov, G. (2018). Report on research and innovation activities of Al-Farabi KazNU for 2017 (p. 541). Al-Farabi Kazakh National University.
Mutanov, G. (2019). Report on research and innovation activities of Al-Farabi KazNU for 2018 (p. 520). Al-Farabi Kazakh National University.
Mutanov, G. (2020). Report on research and innovation activities of Al-Farabi KazNU for 2019 (p. 708). Al-Farabi Kazakh National University.
Nafukho, F. M., Wekullo, C. S., & Muyia, M. H. (2019). Examining research productivity of faculty in selected leading public universities in Kenya. International Journal of Educational Development, 66, 44–51. https://doi.org/10.1016/j.ijedudev.2019.01.005
Nazarbayev, N. (2006). Speech by the President of the Republic of Kazakhstan N.A. Nazarbayev at a joint meeting of the Houses of Parliament, January 18, 2006. Retrieved January 10, 2021, from https://tengrinews.kz/zakon/prezident-respubliki-kazahstan/konstitutsionnyiy-stroy-i-osnovyi-gosudarstvennogo-upravleniya/id-K060002006_/.
Nazarbayev, N. (2010a). Address by the President of the Republic of Kazakhstan N. Nazarbayev to the people of Kazakhstan, January 29, 2010 “New Decade — New Economic Recovery — New Opportunities for Kazakhstan.” Retrieved January 10, 2021, from https://www.akorda.kz/ru/addresses/addresses_of_president/poslanie-prezidenta-respubliki-kazakhstan-n-a-nazarbaeva-narodu-kazakhstana-29-yanvarya-2010-goda_1340624693.
Nazarbayev, N. (2010b). Decree of the president of the republic of Kazakhstan dated December 7, 2010 No. 1118 "On approval of the State Program for the Development of Education and Science of the Republic of Kazakhstan for 2011 — 2020.” Retrieved January 10, 2021, from http://adilet.zan.kz/rus/docs/U1000001118.
Nazarbayev, N. (2012). Address by the president of the republic of Kazakhstan, Leader of the Nation, N.Nazarbayev, December 14, 2012 “Strategy Kazakhstan-2050”: New political course of the established state. Retrieved January 10, 2021, from https://www.akorda.kz/en/events/astana_kazakhstan/participation_in_events/address-by-the-president-of-the-republic-of-kazakhstan-leader-of-the-nation-nnazarbayev-strategy-kazakhstan-2050-new-political-course-of-the-established-state-1.
Nazarbayev, N. (2018). Decree of the president of the republic of Kazakhstan dated February 15, 2018 No. 636 “On approval of the Strategic Development Plan of the Republic of Kazakhstan until 2025 and invalidation of some decrees of the President of the Republic of Kazakhstan.” Retrieved January 10, 2021, from https://online.zakon.kz/m/document/?doc_id=38490966.
Neff, M. W. (2018). Publication incentives undermine the utility of science: Ecological research in Mexico. Science and Public Policy, 45(2), 191–201. https://doi.org/10.1093/scipol/scx054
O’Connor, K., Knox, C., & Janenova, S. (2021). Bureaucrats, authoritarianism, and role conceptions. Review of Public Personnel Administration, 41(2), 358–383. https://doi.org/10.1177/0734371X19888009
OECD. (2018). Reviews of national policies for education: Higher Education in Kazakhstan 2007. Paris: OECD Publishing. Retrieved January 10, 2021, from https://read.oecd-ilibrary.org/education/higher-education-in-kazakhstan-2017/foreword_9789264268531-1-en#page1.
Osterloh, M. (2010). Governance by numbers Does it really work in research? Analyse and Kritik, 32(2), 267–283. https://doi.org/10.1515/auk-2010-0205
Prakhov, I., & Rudakov, V. (2018). The determinants of faculty pay in Russian universities: Incentive contracts. Higher School of Economics Research. https://doi.org/10.2139/ssrn.3247107
QS. (2019). Quacquarelli symonds world university rankings methodology. Retrieved January 10 2021, from https://www.topuniversities.com/qs-world-university-rankings/methodology.
Quan, W., Chen, B., & Shu, F. (2017). Publish or impoverish: An investigation of the monetary reward system of science in China (1999–2016). Aslib Journal of Information Management, 69(5), 486–502. https://doi.org/10.1108/AJIM-01-2017-0014
Saarela, M., & Kärkkäinen, T. (2020). Can we automate expert-based journal rankings? Analysis of the finnish publication indicator. Journal of Informetrics, 14(2), 101008. https://doi.org/10.1016/j.joi.2020.101008
Sīle, L., & Vanderstraeten, R. (2019). Measuring changes in publication patterns in a context of performance-based research funding systems: The case of educational research in the University of Gothenburg (2005–2014). Scientometrics, 118(1), 71–91. https://doi.org/10.1007/s11192-018-2963-8
Sivertsen, G. (2018). The Norwegian model in Norway. Journal of Data and Information Science, 3(4), 3–19. https://doi.org/10.2478/jdis-2018-0017
Sivertsen, G. (2019). Understanding and evaluating research and scholarly publishing in the social sciences and humanities (SSH). Data and Information Management, 3(2), 61–71. https://doi.org/10.2478/dim-2019-0008
Smagulov, K., Makanova, A., & Burshukova, G. (2018). Analysis of scientometric indicators of Kazakhstani authors’ publication activity in journals included in the Scopus database. Journal of Economic Research and Business Administration, 123(1), 233–253. https://doi.org/10.26577/be-2018-1-2078
Sorz, J., Glänzel, W., Ulrych, U., Gumpenberger, C., & Gorraiz, J. (2020). Research strengths identified by esteem and bibliometric indicators: A case study at the University of Vienna. Scientometrics, 125(2), 1095–1116. https://doi.org/10.1007/s11192-020-03672-6
THE. (2020). Times higher education world university rankings methodology. Retrieved January 10, 2021, from https://www.timeshighereducation.com/sites/default/files/breaking_news_files/the_2021_world_university_rankings_methodology_24082020final.pdf.
Tokayev, K.-J. (2020). President of Kazakhstan Kassym-Jomart Tokayev’s State of the Nation Address, September 1, 2020 “Kazakhstan in a new reality: time for action.” Retrieved January 10, 2021, from https://www.akorda.kz/en/addresses/addresses_of_president/president-of-kazakhstan-kassym-jomart-tokayevs-state-of-the-nation-address-september-1-2020.
Vanecek, J., & Pecha, O. (2020). Fast growth of the number of proceedings papers in atypical fields in the Czech Republic is a likely consequence of the national performance-based research funding system. Research Evaluation, 29(3), 245–262. https://doi.org/10.1093/reseval/rvaa005
Yamshchikov, G. V., & Schmid, G. P. (2013). Publication practices and attitudes towards evidence-based medicine in central Asia (Letter). Lancet Global Health, 1(2), e73–e74. https://doi.org/10.1016/S2214-109X(13)70022-6
Yassinova, K. (2019). Factors impacting research productivity at higher educational institutions in Kazakhstan. Master Thesis, Nur-Sultan, Kazakhstan: Nazarbayev University.
Yessirkepov, M., Nurmashev, B., & Anartayeva, M. (2015). A scopus-based analysis of publication activity in Kazakhstan from 2010 to 2015: Positive trends, concerns, and possible solutions. Journal of Korean Medical Science, 30(12), 1915–1919. https://doi.org/10.3346/jkms.2015.30.12.1915
Acknowledgements
We gratefully acknowledge Prof. Bulat Kenessov for providing helpful comments on earlier drafts of this study. We would also like to thank the anonymous referees for their careful reading and suggestions to help improve the manuscript.
Funding
This work was funded by the Science Committee of the Ministry of Education and Science of the Republic of Kazakhstan [Grant No. AP08051974].
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Renata Kudaibergenova, Sandugash Uzakbay, Asselya Makanova, Kymbat Ramadinkyzy, Erlan Kistaubayev, Ruslan Dussekeev, Kadyrzhan Smagulov. The first draft of the manuscript was written by Kadyrzhan Smagulov, Renata Kudaibergenova and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors have no conflicts of interest to declare that are relevant to the content of this article.
Rights and permissions
About this article
Cite this article
Kudaibergenova, R., Uzakbay, S., Makanova, A. et al. Managing publication change at Al-Farabi Kazakh National University: a case study. Scientometrics 127, 453–479 (2022). https://doi.org/10.1007/s11192-021-04139-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-021-04139-y