By John O’Leary, QS academic Advisory Board
This week sees the end of the international rankings season, with QS publishing the first-ever comparisons of Latin American universities and Times Higher Education (THE) issuing the second edition of its global rankings with Thomson Reuters.
The moment provides an opportunity to take stock of the main rankings before yet more organisations join the field. The European Commission, for example, may soon publish the first results from its U-Multirank project, while the OECD is still piloting its Assessment of Higher Education Learning Outcomes (AHELO) initiative, which tests students in different countries in a range of subjects from economics to engineering. Probably the most significant development of 2011 was the publication by QS of the first rankings by individual subject.
The 26 tables are the initial response to a demand from prospective students for more granular information on the university departments in which they will actually study. There will be considerable interest in the academic community this week in the changes in methodology made by THE. The magazine’s attempt to broaden the focus of international rankings was welcomed by many of its readers, but the flaws in its original methodology underlined the difficulties inherent in such an approach.
A global survey of teaching quality based on academic opinion attracted particular criticism for purporting to measure something that statisticians and researchers in a series of discussions have dismissed as impracticable. Professor Malcolm Grant, the Provost of University College London, in a speech at the British Council’s Going Global conference in Hong Kong, questioned whether any academic present could judge how the quality of teaching at their university compared with teaching in Moscow or Melbourne.The same point has been made by academics in debates held in Stockholm, Montreal and Singapore, among others, yet the survey remains unchanged and heavily weighted in THE’s rankings.
A number of changes have been announced by THE, notably to the system used to measure “research influence”, which saw Alexandria University ranked above Harvard and Cambridge on that measure last year. The extent of the changes is such that THE has warned that this year’s results are not comparable with those for 2010. Among them is the addition of a new measure of international outlook, which gives additional credit for research papers that are jointly authored with an overseas partner.
Commentators will also look at the impact of exchange rate fluctuations on the measures of income used in the THE rankings. Other international ranking organisations steer clear of indicators based on income and expenditure, partly because changes in position may reflect the performance of currencies rather than universities.
In a period of dramatic flux on the currency markets, it will be interesting to see whether these fears have been borne out. QS’s own institutional rankings showed increased stability this year, with unchanged indicators and weightings, but there was greater representation of institutions in the Middle East and Latin America. The size of the company’s reputational surveys increased significantly, with almost 34,000 academics and 17,000 employers responding.
The Academic Ranking of World Universities, published in Shanghai, is even more stable, to the point where its results have attracted rather less media coverage in the last two years. The Shanghai Rankings organisation has followed QS into regional rankings, producing a ‘Greater China’ table that includes Taiwan and Hong Kong. The inaugural table was headed jointly by Tsinghua University and the National Taiwan University, and took account of factors such as the percentage of graduate students, the university’s budget and the proportion of staff with PhDs (but did not attempt to measure teaching quality.) The more specialist Webometrics ranking of universities’ internet activity has also continued to grow in scope. It attracts particular interest in countries where universities are underrepresented in other international rankings.
It is clear that international rankings remain a hive of development. By the time that the next rankings season ends, there will be more players, but also the beginnings of the audit system promised by IREG, the International Observatory on Academic Ranking and Excellence.