Ranking Methodology of the World’s Most Innovative Universities

Abstract

Reuters News Agency started the idea of ranking the World’s Most Innovative Universities. It relied on data compiled by Clarivate Analytics and several of its research platforms [25]. The latter company used to be called “Intellectual Property and Science Business of Thomson Reuters.” The process began by identifying approximately 600 academic and government organizations that published the greatest number of articles in scholarly journals. Reuters, then, put ten criteria for the innovation to evaluate the chosen organizations. These criteria are presented later in Sect. 2.3.

2.1 Ranking Organization of the World’s Most Innovative Universities

Reuters News Agency started the idea of ranking the World’s Most Innovative Universities. It relied on data compiled by Clarivate Analytics and several of its research platforms [25]. The latter company used to be called “Intellectual Property & Science Business of Thomson Reuters.” The process began by identifying approximately 600 academic and government organizations that published the greatest number of articles in scholarly journals. Reuters, then, put ten criteria for the innovation to evaluate the chosen organizations. These criteria are presented later in Sect. 2.3.

2.2 Data Collection Methodology

Reuters relied on data compiled by Clarivate Analytics and several of its research platforms [25]. The platforms are:

• 1. InCites,

• 2. Web of Science,

• 3. Derwent Innovations Index,

• 4. Derwent World Patents Index,

• 5. Patents Citation Index.

Reuters began by identifying approximately 600 academic and government organizations that published the greatest number of articles in scholarly journals from 2011 to 2016, as indexed in the Clarivate Web of Science Core Collection database. The list was cross-referenced against the number of patents filed by each organization during the same time period in the Derwent World Patents Index and the Derwent Innovations Index. Patent equivalents, citing patents, and citing articles were included up to March 2018. The timeframe allows for the articles and patent activity to receive citations, thereby contributing to that portion of the methodology.

Next, the list was reduced to just those institutions that filed 70 or more world (World Intellectual Property Organization, WIPO) patents, the bulk of which were universities. The restriction to ranking only those institutions with 70 or more world patents is different for this global ranking than for our regional rankings.

For the 2018 surveys of Europe’s Most Innovative Universities and Asia’s Most Innovative Universities, the threshold was 50 world patents, because it allowed a more in-depth view of the most active institutions relative to others in their geographic area.

Each candidate university was then evaluated using various indicators including how often a university’s patent applications were granted, how many patents were filed with global patent offices and local authorities, and how often the university's patents were cited by others. Universities were also evaluated in terms of how often their research papers were cited by patents and the percentage of articles that featured a co-author from industry.

Since some university systems, such as the University of California, centralize their patent administration, it is not possible to identify which of the various campuses was responsible for the initial research; so in these cases, the entire system was ranked, as opposed to an individual campus. Further complicating matters are the fact that not all universities publicly list their names on their patents or use complex name variants. For example, patents by the University of Oxford are filed under the name ISIS Innovations Ltd. In such instances, the name of the entity administrating patents for a university was identified and the patents were then associated with the university by the Clarivate analysts. There are several related changes to the global ranking methodology this year. In the past, Clarivate ranked the entire University of North Carolina together, instead of breaking out individual members, but in 2018 each member institution was ranked independently. Following this change, only one UNC institution placed in the top 100: The flagship University of North Carolina at Chapel Hill.

2.3 Innovation Criteria

The following are the criteria that contributed to each university’s composite score, which in turn determined the ranking of the universities according to innovative capacity and achievement [25]:

• 1. Patent Volume

Reuters used “Derwent World Patents Index” and “Derwent Innovations Index” to obtain the number of basic patents (patent families) filed by the organization. This is an indication of research output that has a potential for commercial value. The number is limited only to those patents that are registered with the World Intellectual Property Organization (WIPO).

• 2. Patent Success

Here also, Reuters used “Derwent World Patents Index” and “Derwent Innovations Index” to get the ratio of patent applications to grants over the assessed timeframe. This indicates the university’s success in filing applications that are then accepted.

• 3. Global Patents

Here again, Reuters used “Derwent World Patents Index” and “Derwent Innovations Index” to obtain the percentage of patents for which coverage was sought with the US, European, and Japanese patent offices. Filing an international patent is an expensive and laborious process, and filing in multiple countries or regions is an indication that the invention is considered to be non-trivial and has commercial value.

• 4. Patent Citations

Reuters used “Patents Citation Index” to find the total number of times a patent has been cited by other patents. As part of the patent inspection process, the patent office examiner will cite significant prior art. The number of times a patent has been cited is an indication that it has an impact on other commercial R&D.

• 5. Patent Citation Impact

Here too, Reuters used “Patents Citation Index” to find the patent citation impact. This is an indication of how much impact a patent has had. Because it is a ratio (or average), it is not dependent on the size of the organization. Note that the indicator Percent of Patents Cited (listed below) is closely related to this indicator; therefore, these two indicators are given half the weighting of all others.

• 6. Percent of Patents Cited

Here again, Reuters used “Patents Citation Index” to find the percent of patent cited. This indicator is the proportion of patents that have been cited by other patents one or more times. As mentioned, it is closely tied to the Patent Citation Impact indicator.

• 7. Patent to Article Citation Impact

Reuters used the “Patents Citation Index,” “Derwent World Patents Index,” and “Web of Science Core Collection” to obtain the patent to article citation impact. This indicator, similar to Patent Citation Impact, measures the average number of times a journal article has been cited by patents. This unique indicator demonstrates that basic research conducted in an academic setting (as recorded in scholarly articles) has had influence and impact in the realm of commercial research & development (as measured by patents).

• 8. Industry Article Citation Impact

Here, Reuters used “Web of Science Core Collection” to get this impact. Article-to-article citations are an established indicator of influence and research impact. By limiting the citing articles only to those from industry, this indicator reveals the influence and impacts that basic research conducted in an academic setting has had on commercial research.

• 9. Percent of Industry Collaborative Articles

Here again, Reuters used “Web of Science Core Collection” to get the percentage of all articles of a university that contain one or more co-authors from a commercial entity. This indicator shows the percentage of research activity that is conducted in collaboration with industry, suggesting potential future economic impact of the research project jointly undertaken.

• 10. Total Web of Science Core Collection Papers

Here also, Reuters used “Web of Science Core Collection” to get the total number of journal articles published by the organization. This is a size-dependent measure of the research output of the university.

• 11. Final Score

The indicators were used to rank each of the universities, and the composite score was achieved by summing the ranks for each criterion for each university. Each indicator was weighted equally with the exception of Patent Citation Impact and Percent of Patents Cited, which received 50% weighting each since they are closely related in measuring the same phenomenon.

2.4 Tables of the Most Innovative Universities

Reuters has issued three different tables for the most innovative universities. These are:

• 1. The World’s Most Innovative Universities, starting from 2015 and updated every year.

• 2. Europe’s Most Innovative Universities, starting from 2016 and updated every year.

• 3. Asia-Pacific’s Most Innovative Universities, starting from 2016 and updated every year.

It should be noted that an institution’s ranking relative to others may be different in the global and the regional rankings since the ranking is dependent on summarizing the ranks of 10 indicators among others in the population, resulting in a composite score specifically geared to those in the comparison group. As mentioned above, in the global list, the institutions with 70 or more world patents were included, while, in Europe and Asia lists, the threshold was 50 world patents. Tables 2.1 and 2.2 show some examples of ranking orders in the global and the regional rankings.

Table 2.1. Examples in the differences in ranking order between the global and Europe’s lists (2018)

Table 2.2. Examples in the differences in ranking order between the global and Asia-Pacific’s lists (2018)