Ranking: what are they?

The press and media are increasingly reporting news on international university rankings and, in recent years, all Italian universities have given them increasing importance.
But what are rankings, who draws them up and what effects do they induce? 
 

A FEW WORDS: RANKING AND RATING, INDICATORS, METHODOLOGY

Rankings  are ordered rankings, where the final score that determines the position in the ranking is attributed following a comparison between universities using parameters also called indicators.
The indicators can be quantitative and concern the size of the population (e.g. number of students, teachers), teaching (e.g. number of courses of study or PhD), bibliometric (number of scientific publications), economic; All these dimensions are quantified numerically.
Indicators can also be used that provide qualitative assessments, i.e. not translatable into a number, of the impact on society and the environment of the actions of the universities and the services provided.

Some rankings identify the university that obtains the maximum score in all indicators and use it as a reference for all the others. Other rankings, on the other hand, attribute absolute scores. 

The collection of information takes place by the agency or body that carries out the analysis, through a methodology, i.e. a set of definitions and rules. 
Each ranking has its own methodological rules and similar indicators can be given a different weight: for this reason, the same university can have very different evaluations and rankings. 
Knowing how to read the results of a ranking also means knowing and deepening its methodology, which is not always public and transparent.

Some data may be collected independently by the agencies (e.g. public data on financial statements or those present in databases of scientific publications), Other information is provided by the universities participating in the rankings. Many agencies also carry out surveys interviewing professors, students, and companies to assess the reputation of a university.

Another very common term when it comes to evaluation is the rating, sometimes mistakenly used as a synonym for ranking.  Ratings do not produce a ranking, but assign judgments to universities according to whether they have (or not) certain characteristics. An example is the periodic accreditation operated by ANVUR, for which the University of Padova has been evaluated as "Class A"

PILLS DI HISTORY OF RANKINGS 

The rankings of universities and, more generally, of educational institutions were created in Anglo-Saxon countries to support students (and their families) in the choice of college and university, through an evaluation of the scientific success and prestige of teachers and alumni who have graduated from that institution. University  rankings have then expanded to include comparisons with universities around the world and have acquired increasing importance in the last fifty years.

Let's mark some temporal references, without claiming to be complete.

In 1983 , U.S. News & World Report  published  the first national ranking of American universities.

In 1999, in Italy, the first publication by  Censis of the survey on Italian universities took place. 

In 2003 Shanghai Jiao Tong University published the "Academic Ranking of World Universities" (better known as ARWU) which can be considered the first example of a ranking worldwide.
ARWU was born with political purposes: the Chinese government wanted to quantify the distance between national universities and the best universities in the world, in order to place at least one Chinese university among the first in the world.
Today, the ARWU ranking is no longer drawn up by Jiao Tong University, but by a specialized agency

In 2004 the Times Higher Education Supplement (THES, later THE) published a worldwide university ranking in collaboration with the Quacquarelli-Symonds agency (QS) This ranking was born as a real supplement to the Times newspaper. The then director John O'Leary, sensing the potential publishing success, hoped for a more reliable and methodologically solid ranking than the ARWU ranking.

In 2010 the paths of THE and QS separated. 

THE first started a new collaboration with Thomson Reuters (now Clarivate Analytics) and then became autonomous in the methodological definition and development of the survey. Today THE - Times Higher Education  is an agency in its own right and with a global operational scope.

QS - Top Universities continues to work with its methodology, publishes its rankings and expands its portfolio of services over time, becoming one of the most important university rankings in the world and the agency that has the greatest media impact. The main website received 149 million visits in 2019, and in the same year, QS was mentioned in around 94,000 media references.

In 2014, thanks to funding from the European Community, the first implementation of the U-Multirank project was launched: like the other rankings , it aims to guide the choice of university but instead of defining a ranking, it provides a list of universities with the corresponding performance value based on the criteria chosen by the user himself.

In recent years, we have witnessed the emergence of numerous other international rankings such as CWTS Leiden Ranking, CWUR, Nature Index, which are mainly based on bibliometric indexes of universities.  

UNIVERSITIES AND SUSTAINABILITY

UI GreenMetric World University Ranking is an initiative of the Indonesian University and was launched in 2010. This ranking was the first in the world to focus on sustainability and environmental issues. It is aimed at universities around the world and assesses the commitment and policies that universities adopt regarding sustainability and climate change issues. In particular, in addition to climate change, GreenMetric turns its attention to the waste of water and electricity, waste recycling, energy efficiency of buildings, "green" transport policies, etc.

LIMITS AND OPPORTUNITIES 

For at least a decade there have been international studies and publications on rankings, sometimes with a critical view that highlights the limits and the negative effect of excessive competition.  Such criticisms are also increasingly read in the press.

Rankings by their nature are tools that stimulate international competition. The parameters used by the main rankings are effective for measuring the "exclusive" context of Anglo-Saxon universities and all those new universities that are inspired by the same model: these are universities that receive substantial funding from private individuals and companies, free to recruit prestigious researchers and professors, able to strongly select those enrolled and to enrol a limited number of students, imposing high enrolment fees.

The Italian context, on the other hand, is inclusive, it values the right to study, universities are mostly public, recruitment and budgets are subject to regulatory (and bureaucratic) limitations; these are all fundamental aspects to correctly interpret the results. From a cultural point of view, moreover, performance evaluation is a relatively young and much debated topic in the academic field. 

To learn more: 

AA.VV., ItaliaDecide, 2021: Italy and its reputation: The University 

M.Degli Esposti, G.Vidotto, CRUI Foundation, 2020: The CRUI Working Group on International Rankings: Activities, Results and Perspectives 2017 - 2020 

 S.Rider, M. A. Peters, M. Hyvönen, T. Besley, Springer, 2022:  World Class Universities - A contrasted concept

Page edited by:

Ranking Unit
Statistical Data and Ranking Coordination Sector
UPCS - Strategic Planning and Control Office

ranking@unipd.it