Ranking organisations must be as transparent as possible about the data and methods they use to compare universities, Ben Sowter, the head of the QS Intelligence Unit which produces the company’s rankings, says in a new book published by Unesco.
The book, entitled Rankings and Accountability in Higher Education: Uses and Misuses, showcases the views of 20 academics and rankers. It includes debate on the principles of global ranking, as well as descriptions of current and proposed methodologies.
In a chapter entitled Issues of Transparency and Applicability in Global University Rankings, Sowter argues that global university rankings have become an invaluable tool in the increasingly complex process of choosing where to study, although they can never replace individual preferences. With their growing importance to students and universities comes an obligation to be open to scrutiny.
Sowter acknowledges that no rankings currently offer complete transparency in the sense that the reader is able to recreate the exercise. “In the case of QS, this is not down to a philosophical or commercial objective against transparency, but more to do with technical and resourcing constraints in preparing all the necessary material and keeping it all fully up to date,” he explains.
QS publishes extensive documentation on its methodology, final indicator scores, detailed definitions of the data requested, means and standard deviations for each indicator, specific weightings and demographic breakdowns of survey responses. The level of openness displayed by QS and some other ranking organisations has been a force for greater transparency in higher education generally, he argues.
Sowter writes: “QS is a business entirely grounded in higher education,and feels the pressure to be transparent and responsible – much more so than an institution, government agency or media organization.”
Mmantsetsa Marope and Peter Wells, two of the three editors of the book, say in their introduction: “The explosion of university rankings perhaps signals the reality that we live in a compared and ranked world. The twenty-first century is increasingly compared and ranked along a myriad of dimensions.”
The two editors, both former members of Unesco’s Higher Education Division, add: “Universities are among our canonical twenty-first century institutions. In and of themselves, they are standard setters for how other aspects of our ‘worlds, countries and institutions’ are compared and ranked. It therefore seems inevitable that universities would themselves be subjected to comparisons and rankings. However, being complex institutions and being part of complex systems, it seems equally inevitable that comparisons and rankings of universities would be anything but polemical.”
Sowter writes: “QS is in complete agreement that current aggregate global rankings do not present a sufficiently comprehensive picture of the performance of universities. Going further, QS believes that aggregate global rankings will never be able to provide a complete picture regardless of how sophisticated data collection mechanisms may become.”
University selection decisions are deeply personal and potentially crucial, he writes. “Rankings results, as published, are just one (perhaps expert) interpretation of the data – a little like a film critic stating that one film is better than another. The critic may know far more about film and may be able to justify his viewpoint using a near scientific formulae, but his ultimate proclamation may bear no resemblance to the viewpoint of any given audience member.”
The book also contains articles by Profesor Nian Cai Liu, the founder of Shanghai’s Academic Ranking of World Universities, and Phil Baty, rankings editor of Times Higher Education magazine. Both are mainly descriptions of their own methodologies with acknowledgments that no single ranking can reflect adequately the various qualities of universities around the world.
Rankings and Accountability in Higher Education: Uses and Misuses can be downloaded at
