Impelling research productivity and impact through collaboration: a scientometric case study of knowledge management
- 778 Downloads
A case study for impelling university research productivity and impact through collaboration is presented. Scientometric results support the hypothesis that a knowledge management model increased research collaboration and thereby boosted a university’s number of publications and citations. Results come from fifteen years of data at a Mexican university with 2400 researchers who produced 24,000 works in fifteen research disciplines. These data are treated with social network visualizations and algorithms to identify patterns of collaboration and clustering, as well as with normalizations to make disciplines comparable and to verify increasing citation impact. The knowledge management model implemented in the study may be a cost-effective way for universities to intensify collaboration and improve research performance.
KeywordsKnowledge management Social network analysis Scientometrics Case study University Research
Katz and Martin made the clairvoyant observation in 1997 that scientometrics, in the sense of measuring science through publications and citations, could change research management (Katz and Martin 1997). Since then, scientometric studies have shown that, depending on the scientific discipline and other factors, research collaboration can have a positive relationship with number of publications (Abramo et al. 2009; Lee and Bozeman 2005) as well as number of citations (Abramo et al. 2017; Lancho-Barrantes et al. 2012), and collaboration has become a priority in research management around the globe. Because of the relationship between collaboration, publications, and citations, universities are looking for ways to increase collaboration. One idea is to form interdisciplinary laboratory teams. Porac et al. (2004) study two teams that show convergence to similar research themes and productivity boosts to team members. Another idea is to accelerate the maturity of entire higher education systems. Results show that a higher education system’s maturity is not the only predictor of collaborative behavior, but rather that the decisions of institutional actors can make a difference (Kim et al. 2017). Collaboration is typically measured as co-authorship of an indexed publication and analyzed with social network analysis methods. Contandriopoulos et al. (2016) use social network analysis to study collaboration within a region of Canada, and find that collaboration is a predictor of research quantity and quality, as represented by the h-index (Hirsch 2005). The combination of scientometrics and social network analysis have made a powerful case in favor of research collaboration, but as research management increasingly draws from scientometrics and social network analysis, knowledge management is rarely considered as the antecedent of these phenomena.
Although it may be unexpected, research management is a popular concern in Mexico, where young universities are just developing research capabilities. Gregorutti (2010) found professors in Mexico looking for “a collaborative environment with a view towards partnership,” however, almost no studies of research collaboration use data from Latin America. The research here presents the effect of knowledge management on research collaboration and scientometrics using data from a Mexican institution and Web of Science. The form of knowledge management implemented in the study was published by Cantú-Ortiz et al. (2009). By analyzing the results, this investigation shows that knowledge management increased internal and external collaboration intensity, and that this had the secondary effect of increasing scientific publications and citations.
This investigation’s originality lies in its fifteen-year longitudinal consideration of knowledge management, social network analysis, and scientometrics. The theoretical compatibility of these disciplines and their practical synergy give direction to this study. “Background” section introduces the knowledge management model studied. “Data” section outlines the data, which are analyzed following the methodologies explained in “Methodology” section. “Results” section presents results, the implications of which are discussed in “Findings with implications for the university” section. Conclusions and ideas for future research come in “Conclusion” section.
This paper presents a case study of a university adopting knowledge management, driven by social networks, and measured by scientometric results. The university in question is Tecnológico de Monterrey, a system of 31 campuses across all regions of Mexico; herein it is called the University. Cantú-Ortiz et al. (2009) described the implementation of a knowledge management model in the University in terms of intellectual capital and knowledge assets, research-based learning, and human capital. The model they describe is herein denoted as the Knowledge Management Model, and the results of its implementation are studied in this research. This present research focuses primarily on how knowledge management intensified research collaboration, and as a secondary effect increased the quantity and quality of research publications.
Managing a university requires the capacity to absorb knowledge, move knowledge within the organization, and create new knowledge. This view of the university is based on theories of knowledge management, particularly knowledge economy, knowledge assets, and knowledge networks. The knowledge economy treats knowledge as a commodity for production, distribution, and consumption across value chains (Baskerville and Dulipovici 2006). It is bought and sold in the marketplace. The knowledge economy idea encompasses the theory of knowledge assets. A knowledge asset, such as a research paper, is a single input or output of a knowledge activity (Baird and Henderson 2001). Knowledge network theory explains that networks serve to share and even create knowledge (Inkpen and Tsang 2005; Baskerville and Dulipovici 2006). The rate of these processes depends on the “dense” or “sparse” structure of the network, a theoretical classification that comes from social network analysis (Walker et al. 1997). Dense networks have many redundant contacts in which people are closely connected with all other people. Advantages may include trust and the capacity to transfer large amounts of complex information. Disadvantages may include group-think and lack of innovation. A sparse network is the opposite; people are not connected in closed circuits. In this configuration, people may have a large number of contacts, but those contacts come from different groups and are therefore not themselves connected. People in this network have access to a greater array of information, but may lack strong relationships. From this perspective, network structure affects knowledge held by individuals in the network.
Individuals in universities often form communities such as research teams, laboratories, or research centers, which are increasingly dominant over individual researchers in knowledge production (Wuchty et al., 2007). Communities of practice are characterized by the combination of heterogeneous and homogeneous knowledge within them (Wenger & Snyder, 2000). Heterogeneous knowledge that is specialized tends to be separated in the network, whereas homogeneous knowledge that is general tends to diffuse across broader parts of the network (El Louadi, 2008). Managing the flow of knowledge between networked communities can boost university performance. In one example, Romano et al. (2014) concluded that management explained scientific outputs, measured in terms of patents, at an Italian university. It therefore behooves universities, as knowledge-based organizations, to manage their networks of collaboration. A management implementation has four phases: plan, organize, execute, and control. Control is a sub-process that contains: measure, evaluate, adjust, and return to plan. The following paragraphs describe how knowledge management was implemented at the University.
Knowledge management was introduced to the University in this study in 2003 to intensify research collaboration, and thereby improve scientific performance. Designated research groups were central to the Knowledge Management Model. In the Model, the University provided seed money to a principal researcher to work with a group of faculty, postdoctoral researchers, and graduate students in a designated line of research. Undergraduate students were sought to participate in the research according to the model described by Galeano et al. (2012). The seed money was intended to initiate a project, and on the strength of preliminary results the research team was expected to attract outside funding sources proportional to internal seed funding.
The Knowledge Management Model required new managers; these were the new administrators in the research management office and the new academic leaders of the research groups. The research office was responsible for managing indicators, evaluation, and coordination. Academic group leaders were responsible for directing the group members in the realization of peer-reviewed research.
The Model required new management processes. Chief among these were a new system of scientific performance evaluation, academic peer evaluations every three to five years, and rules for financing conditional on continued success. The main thrust of the evaluation structure was a team-based scoring system based on scientometrics. Team-based scientometrics sensitive to changes in collaboration such as publications/professor communicated the importance of collaboration to the groups (Persson et al. 2004).
The Model communicated the relative value of each produced knowledge asset to the research groups. Different points were awarded for publications in journals, conferences, and books, and points were awarded for collaborative activities such as organizing or participating in conferences, advising theses, and establishing faculty collaboration across different campuses.
In this case in which a knowledge management model was implemented at a university, knowledge management, networks, and scientometrics can be used to answer the question: Did knowledge management intensify research collaboration, and thereby increase research quantity and quality? The data and methodology used to answer the question are presented next, followed by the results, and then a discussion of their implications.
Data are longitudinal over 15 years and include 24,844 publications by 2401 researchers. The sample is comparable in number of researchers to related studies, such as Contandriopoulos et al. (2016), and additionally balanced because it comes from activity at 31 campuses spread across Mexico. The 15-year period is divided into five triennia: 2000–2002, 2003–2005, 2006–2008, 2009–2011, and 2012–2014 because research cycles typically last more than one year. The publications are divided into the 15 disciplines most relevant to the priorities of the University, and these are summarized in 5 broad research areas: Life Science, Natural Science, Engineering, Social Science and Management, and Arts and Humanities.
There are two types of data in this study, collaboration data and scientometric data. Collaboration data include internal, interdisciplinary, and international co-authorships. Scientometric data include publication counts in various publication categories, citations, and normalized citation impact. Both data types were collected from two databases. Data were collected from the University’s CRIS (“current research information system”: a class of software systems commonly deployed to manage research resources at universities), which is described by Cantú-Ortiz and Ceballos (2010), as well as Clarivate Analytic’s Web of Science (owned and operated by Thomson Reuters during the period of study). Current research information systems are implemented inside of research institutions to track, manage, and evaluate research. CRIS holds references to all institutional publications starting from 2000. Its advantage is that it holds references to more publications than Web of Science, especially books and book chapters, giving a much larger picture of scientific activity at the University. Its disadvantage is that CRIS fell out of use starting in 2012, and therefore analyses of CRIS data are only valid until 2011. Clarivate Analytic’s Web of Science database is used as the second data source from 2000 to 2014. Web of Science provides three main advantages: it is a safeguard against systemic data errors in the CRIS; it reports more scientometric indicators than CRIS, and it only reports the world’s highest level of scientific work published in top sources.
This study is designed as a longitudinal trend study, which analyzes changes in variables based on observation over time rather than experimental controls (Hernández-Sampieri et al. (2010). Causes and effects of the observed variables are supported by inferences from the data. A characteristic of longitudinal trend studies is that they center on a population, in this case University researchers. Although the population may have inflows and outflows in each period of the study, as a population it exhibits behaviors that can be measured and compared across time. Longitudinal trend studies are used to describe trends and draw conclusions about the relationships between variables. The period of study was set at 2000–2014 because these fifteen years covered a baseline period from 2000 to 2002 before knowledge management as well as twelve years from 2003 to 2014 when knowledge management was put into force and changes could be analyzed.
Social network analysis methods are used to measure collaboration. The language of graph theory is often used in these methods. Researchers are called “nodes” of the network and are represented by single points. Two researchers are connected by a line called an “edge” if they have co-authored a publication together. Nodes have a “degree” that equals the number of edges they have. Nodes have a “weighted degree” that sums the weights of all the edges on a node, where weight is defined as the number of co-authored papers shared along the edge. For example, two researchers who have written two articles together and no other publications are translated into two nodes with one edge between them, and each node has degree one and weighted degree two. Another example: three authors who collaborate on one paper are translated into three nodes, connected in a triangle; each node has degree two and weighted degree two. Graph theory allows for visualization of the network and mathematical calculations of its properties. “Modules” is a specific term from graph theory describing sub-networks within large networks. For the purpose of describing modules in this work, we substitute the more common word “cluster” for “module.” For the purpose of calculating clusters, the Louvain algorithm is used for classifying nodes into clusters such that nodes within clusters are more densely connected to each other than to nodes outside of the cluster (Blondel et al. 2008).
Scientometric methods are used to measure publications and citations. These methods are almost entirely descriptive and observatory because controlled tests cannot be performed on corpora of published works. Data are generally taken from large international databases such as Clarivate Analytics’ Web of Science (WOS). At times, publication data are taken from a Current Research Information System (CRIS). It is important to normalize citation indicators to control for differences across academic disciplines. For example, Category Normalized Citation Impact (CNCI) is normalized by the average citations per paper in an academic discipline such that a paper with the average number of citations in its discipline has a CNCI of 1. A paper with more citations than the average in its discipline scores above 1, and a paper with fewer citations than the average in its discipline scores less than 1 (Thomson Reuters 2014).
Social network data are analyzed in Gephi, an open source. Gephi facilitates visualization and a handful of algorithms to calculate graph properties. Scientometric indicators, graph properties, and visualizations are measured in a longitudinal manner across the five triennia to describe the population’s trends in scientific collaboration and productivity. Special attention is given to the evolution and concurrent changes in knowledge management, network variables, and scientific performance. In summary, the study is designed to provide value by measuring collaboration correlated to knowledge management, and in a secondary way by measuring publications and citations correlated to collaboration.
Internal scientific collaboration
Figure 1b groups researchers into disciplines to highlight their different collaboration patterns. Each node is a research discipline and lines between nodes represent interdisciplinary collaboration. Collaboration intensity varies by academic discipline; it is highest in Information & Communication Technology and lowest in Regional Development, which did not produce any interdisciplinary research from 2000 to 2002. Several engineering disciplines produce interdisciplinary work, as does the discipline of Sustainable Development.
Figure 2b shows that every research discipline has participated in interdisciplinary research under the Knowledge Management Model. Information & Communication Technology is again a hub in the network. Business and Education have drastically increased their interdisciplinary collaboration and become hubs as well. The number of interdisciplinary connections in total has tripled after knowledge management.
To corroborate that collaboration contributed to higher publication output within the Knowledge Management Model, scientists were tracked during participation and non-participation in the Model. The average production per researcher per year while not participating in a designated research group was 1.48 documents in CRIS. The average production per researcher per year while participating in a designated research group was 3.47 documents in CRIS, an improvement of 133% during participation in the Knowledge Management Model.
Scientometric outputs from Web of Science and CRIS
CRIS journal articles
CRIS conference proceedings
CRIS books chapters
During the Knowledge Management Model, the rise in publications was steady and large. In twelve years from 2003 to 2014, the number of University publications per year in Web of Science rose from 78 to 384 (calculated from Table 1). Web of Science publications represent the highest level of science globally. The University was able to advance from almost no footprint on this stage to an established presence, also evidenced by the number of citations to University publications. Citations are considered a proxy for the quality and impact of scientific work. The number of citations to the University per triennium has doubled due to a larger corpus of published work and higher quality represented throughout that corpus.
Particularly, productive collaborations were found between the University and the University of California system, universities in Arizona, and M.I.T. The University has 1,171 citations on 72 papers, or 16.26 citations/paper, with the California system. It shows 15.6 citations/paper on 63 papers with the University of Arizona and Arizona State University, and it shows 9.5 citations/paper on 21 papers published with M.I.T., all within the last four years. These figures should continue to grow, especially the citation figures associated with M.I.T., as all of the papers are recent.
Findings with Implications for the University
The overall finding is that knowledge management measurably increased scientific collaboration at the University from 2003 to 2014. Choo and De Alvarenga (2010) found that knowledge-based organizations either flourish or flounder based on strategy and structure. The present findings demonstrate that management is also a critical determinant of knowledge networks and production of knowledge assets. Visualizations in Figs. 1 and 2 show that under knowledge management an extensive collaboration network between individuals and across disciplines was generated in the University. Collaboration intensity increased across time in all research areas as shown in Fig. 3, especially in Natural Sciences.
Ponomariov and Boardman (2010) studied the impact that affiliating with a research center has on an individual researcher’s production. They found that affiliation with large-scale and mature university research centers increased an author’s cross-discipline and inter-institutional collaboration. The Knowledge Management Model achieves similar results without the investment and infrastructure of a fully established research center. This Model could be a cost-effective way for universities to reap the benefits of research collaboration.
Data in Figs. 4 show that the number of detected co-author clusters and designated research groups converged. Contandriopoulos et al. (2016) asked if deliberately creating social networks among researchers would increase collaboration and performance. This result suggests that designating groups in coordinated and strategic lines of research did positively impact research collaboration and performance.
Figure 5 shows that collaborative output is growing more rapidly than single-author output at the University, and as a result the majority of University research is internally collaborative. Figure 6 shows external collaboration increasing at a faster-than-linear rate. It is a noteworthy finding that the Knowledge Management Model designed for internal collaboration also had a positive impact on external collaboration. This is a valuable result for the University because the returns on international collaboration are high (Lancho Barrantes et al. 2012; Ynalvez and Shrum 2011).
A second finding is that scientific quantity and quality has also measurably increased, via collaboration, as a result of knowledge management. Table 1 shows a steady increase in quantity of articles and other scientific publications across the University. To the extent that citations measure research quality, quality has also increased. Figure 7 shows that the University has a historically high CNCI, and that the entire University is above the global average impact threshold for the first time ever. When the Knowledge Management Model began in 2003, new investigators joined research activities. Amid the swell of new research activities, the relative share of international collaboration fell, as did the average quality of the research. These indicators recovered after 2005. Such transition periods could be expected at any institution setting a new managerial direction, but the Knowledge Management Model could be an option for intensifying a university’s internal and external collaboration and thereby increasing its publication and citation performance.
In the case studied here, publications and citations increased across the institution during a period of increasing collaboration. Some studies have provided counterexamples. Ynalvez and Shrum (2011) studied collaboration networks in the Philippines. They concluded that collaboration is not necessarily correlated to publication output, but only collaboration with the right people. In the context of the Philippines, this meant collaboration with researchers in more scientifically renowned countries such as the U.S.A., Australia, and Japan. Findings from the present study enter the limited discussion with Ynalvez and Shrum around scientific collaboration in developing countries, in favor of internal collaboration and external collaboration. In a cautionary example, Uzzi (2008) found that collaboration, when it becomes insular, can be as detrimental to science as the lack of collaboration. This could become a problem if an overemphasis is placed on collaboration.
A final finding is that the theoretical combination of knowledge management, networks, and scientometrics has allowed organizational learning as described in knowledge management literature (Peters et al. 2016; Morey and Frangioso 1998; Gavin 1993). The University has signed formal international institutional partnerships in several research disciplines, partly in response to data like those in Figs. 6 and 7, which show the growing relevance and impact of international collaboration. The University also learned to control the results of the Model by tweaking the scoring of different publication types. This was done to communicate a preference for journal articles rather than conference papers. The concept of a relative award system could help other universities manage their knowledge production.
In this research, data suggest that knowledge management at a university increases research collaboration and thereby increases research quantity and quality. Social network analysis provided the methodological tools for measuring collaboration, and scientometrics provided references for measuring scientific performance. This research illustrates the theoretical synergies of knowledge management, social network analysis, and scientometrics. Far from providing a decisive account, this research suggests that further work can develop the connection between these disciplines.
Further research could focus on the spread of research collaboration through an institution. Fowler and Christakis (2010) report that cooperative behavior ripples in human networks to people separated by up to three degrees; their concept could be tested in terms of research collaboration. Such research would be useful to knowledge managers because it would approximate the indirect consequences of their interventions. Another branch of research stemming from this data could measure the correlation between internal and external collaboration. This would be useful for knowledge managers because it would permit them to estimate their influence beyond the institution. A final data point that invites further investigation is the short-term negative effect that the Knowledge Management Model had on Citation Normalized Citation Impact. Considering theories of human and intellectual capital, particularly of investments in such forms of capital, it may be possible to anticipate the circumstances under which knowledge management will work in the short or long term.
These findings demonstrate a case in which university knowledge management boosted research collaboration and increased scientific performance via that collaboration. The research presents knowledge management complemented by network analysis and scientometric methodologies. One model does not fit all circumstances; therefore, the results can make a useful contribution to understand knowledge management in context.
The authors acknowledge Dr. Alberto Bustani for proposing the Cátedras de Investigación Research Program, Dr. Arturo Molina, Vice-Rector of Research and Technology Transfer, Dr. David Garza, Academic Vice-Rector, and leaders of the research and members of the Research Program. The authors acknowledge Gustavo Parada, Pablo Mazzuchi, and Alejandra Casas for their work as technical specialists.
- Baird, L., and J. Henderson. 2001. The knowledge engine: How to create fast cycles of knowledge-to-performance and performance-to-knowledge. San Francisco: Berrett-Koehler.Google Scholar
- Galeano, N., R. Morales-Menéndez, and F.J. Cantú-Ortiz. 2012. Developing research skills in undergraduate students through an internship model in research and innovation. International Journal of Engineering Education 28 (1): 48–56.Google Scholar
- Gavin, D.A. 1993. Building a learning organization. Harvard Business Review 71 (4): 78–91.Google Scholar
- Hernández-Sampieri, R., C. Fernández-Collado, and P. Baptista-Lucio. 2010. Metodología de la investigación. Mexico City: McGraw Hill.Google Scholar
- Porac, J.F., J.B. Wade, H.M. Fischer, J. Brown, A. Kanfer, and G. Bowker. 2004. Human capital heterogeneity, collaborative relationships, and publication patterns in a multidisciplinary scientific alliance: A comparative case study of two scientific teams. Research Policy 33: 661–678.CrossRefGoogle Scholar
- Reuters, Thomson. 2014. InCites indicators handbook. Philadelphia: Thomson Reuters.Google Scholar
- Uzzi, B. 2008. A social network’s changing statistical properties and the quality of human innovation. Journal of Physics: Mathematical and Theoretical 41: 1–12.Google Scholar
- Wenger, E., and W. Snyder. 2000. Communities of practice: The organizational frontier. Harvard Business Review 78: 139–145.Google Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.