Sunday, September 13, 2020

More Relivant Higher Educaiton Ranking Systems Needed

Australian education consultancy Studymove has pointed out that the higher the QS World University Ranking for an Australian university, the higher the international tuition fees charged. The correlation applies across fields and levels of study, but is least for education studies. The consultants speculate that high unemployment may result in students selecting institutions and programs with higher raking for employment outcomes. However, I expect many students don't look past the overall ranking of a university to examine individual measures. The current ranking schemes are weighted towards academic quality and research output, although these are not relevant to most students.

Like many such ranking systems, QS is heavily skewed to academic reputation. In the case of QS, the largest proportion of the overall measure is based on a survey of academics. Other raking systems use research papers published as a measure of academic quality. However, the research at a university has little effect on the quality of education a student receives. Most students are not planning to be researchers and few who complete research degrees end up in research jobs. Researchers don't make particularly good teachers anyway and this emphasis on research may be resulting in students making poor education choices.

Webometrics uses measures of openness, as well as quality, to provide a more relevant ranking of universities. This relies on readily available information, rather than surveys, which has the advantage of allowing inclusion of many more institution. In particular Webometrics includes thousands of vocational institutions which are excluded from most university ranking systems. Australian has only 43 universities, but Webometrics lists another 150 non-university institutions. 

Despite the different measures used, Australian universities outrank the non-university institutions on the Webometrics scoring and the raking of universities is not very different from QS. This suggests it should be possible to create new rankings of universities at low cost, using a similar approach to Webometrics.

The Webometrics methodology, like that of QS, is heavily weighted towards academic quality. Overall, 35% of the measure is based on papers cited in research publications which perhaps explains why universities are at the top of the list, outranking vocational institutions. 

The ordering of the universities is different to QS, but the same universities feature near the top of both lists. Near the bottom of the universities in the Webometrics ranking are a few large state government vocational intuitions: TAFE NSW, TAFE Queensland, Adelaide Institute of TAFE and Canberra Instituter of Technology. Just one private for-profit vocational provider also features: Open Colleges, which has a 100 year history in correspondence education.

I pasted the Webometrics list of Australian universities to a spreadsheet, and reweighed the scores by deleting the excellence measure. The leading universities remained at the top of the list, but a few vocational institutions crept up a little: Open Colleges from 41 to 39th place, TAFE NSW from 43 to 40, TAFE Queensland 46 to 44, Adelaide Institute of TAFE 45,  Kangan Institute 49. Canberra Instituter of Technology dropped one place from 49 to 50.

No comments:

Post a Comment