The 2021 performance measurements are composed of eight indicators. These indicators together represent three different criteria of scientific paper performance: research productivity, research impact, and research excellence. Table 1 lists the indicators and shows the respective weightings of each indicator.
Table 1 The Criteria and Indicators, and Their Respective Weightings, Used for the NTU World University Performance-Based Ranking
| Criteria | 2021 NTU World University Performance Indicators | Weighting | |
|---|---|---|---|
| Research Productivity | Number of articles in the last 11 yearsa,b (2010-2020) Number of articles in the current yearb (2020) |
10% 15% |
25% |
| Research Impact | Number of citations in the last 11 yearsa,b (2010-2020) Number of citations in the last 2 yearsb (2019-2020) Average number of citations in the last 11 yearsa (2010-2020) |
15% 10% 10% |
35% |
| Research Excellence | h-index of the last 2 years (2019-2020) Number of Highly Cited Papersa (2010-2020) Number of articles in high-impact journals in the current year (2019-2020) |
10% 15% 15% |
40% |
Research Productivity
The number of articles published in peer-reviewed academic journals is frequently used to indicate the productivity of a research institution. To objectively represent a university’s current and ongoing research productivity, this ranking system employs two indicators: the number of articles in the last 11 years (2010-2020), and the number of articles in the current year (2020). “Number of articles in the last 11 years” draws data from ESI, which includes 2010-2020 statistics articles published in journals indexed by SCI and SSCI
"Number of articles in the current year” was extracted from the 2020 data obtained from SCI and SSCI, which were released in April 2021. The previous practice was to extract data every January, but after some yearly observations, it is found that papers published in 2020 would continue to be indexed under the databases into early 2021. As a result, it is decided that the time of data extraction should be postponed to April in order to gather a more complete set of data.
Research Impact
The number of citations of a particular academic article within a specific time frame is a commonly accepted indicator of that article’s impact. This ranking system considers both the long-term and short-term impact of particular research and seeks to provide a fairer representation of a university’s research impact regardless of its size or number of faculty. Thus, this ranking system measures research impact by the number of citations in the last 11 years, the number of citations in the last 2 years, and the average number of citations in the last 11 years.
〝Number of citations in the last 11 years” draws 2010-2020 citation statistics from ESI. "Number of citations in the last 2 years" draws 2019-2020 citation statistics from SCI and SSCI in WOS, which include citation statistics updated to the dates of retrieval. “Average number of citations in the last 11 years” is the number of citations in the last 11 years divided by the number of articles in the last 11 years.
Research Excellence
This ranking system assesses each university’s research excellence by the following indicators: the h-index of the last 2 years, the number of Highly Cited Papers from ESI, and the number of articles in high-impact journals in the current year. “h-index of the last 2 years” measures both the quantity and quality of a university’s research via the use of the 2019-2020 SCI and SSCI data. Employing Hirsch’s (2005) concept of h-index, a university has index h if h of its Np papers in the last two years have at least h citations each and the other (Np–h) papers have ≦h citations each.
“Number of Highly Cited Papers” utilizes data from ESI, which includes statistics of “Highly Cited Papers” from 2010 to 2020. ESI defines Highly Cited Papers as SCI/SSCI-indexed papers that are cited most (in the top 1% of the total papers indexed in the same year) within the last 11 years.
“Number of articles in high-impact journals in the current year” employs data from JCR, which supplies the impact factor of each journal in its subject field. The impact factor of a journal is the number of citations of the papers published in that particular journal within the previous two years divided by the number of that journal’s papers within the previous two years. A higher impact factor means that the journal has articles that are more frequently cited by other journals, thus suggesting its higher scholarly value. This ranking system defines high-impact journals as those whose impact factors are ranked in the top 5% of the total journals within a specific subject category. With high-impact journal lists derived from JCR, this ranking system is able to count the numbers of each university’s articles published in high-impact journals by subject.
References
Hirsch, J. E. (2005). An index to quantify an individual's scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572.
Huang, M. H. (2008). Application of H-index for Research Evaluation at University Level. Evaluation in Higher Education, 1(2), 29-50.
Huang, M. H., & Chi, P. S. (2010). A Comparative Analysis of the Application of H-index, G-index, and A-index in Institutional-Level Research Evaluation. Journal of Library and Information Studies, 8(2), 1-10.