Data Science Ranking Universities

In "Data science can fix ranking briar patch" (The Australian, 14 March 2018), Tim Dodd suggests that data science could be used to rank universities, but this has been done for years with the "Ranking Web of Universities".  Their methodology emphasizes the quality and quantity of information universities provide online (which I think is a good thing). This produces slightly different rankings for Australian universities, to measures emphasizing research output behind pay-walls.

For its top ten Australian institutions the QS World University Rankings has: ANU, Melbourne, UNSW, Queensland, Sydney, Monash, UWA, Adelaide, UTS and Newcastle. On the Ranking Web of Universities, ANU slips from first to fifth place, while UTS and Newcastle displace Curtin, and Macquarie in the bottom two places.

Australian Web ranking	World Web Rank	University	QS Australian Rank	Presence Rank*	Impact Rank*	Openness Rank*	Excellence Rank*
1	55	University of Melbourne	2	179	108	53	26
2	62	University of New South Wales	3	166	87	85	52
3	63	University of Queensland	4	152	114	72	37
4	74	University of Sydney	5	626	172	94	30
5	75	Australian National University	1	122	72	75	131
6	93	Monash University	6	571	206	97	57
7	129	University of Adelaide	8	531	143	128	149
8	136	University of Western Australia	7	581	189	171	115
9	235	Curtin University of Technology	15	1069	227	245	276
10	271	Macquarie University	12	553	266	196	341

Webometrics Ranking of Ten Thousand Universities

The "Webometrics Ranking of World Universities" by Spanish researchers has advantages over the better known ranking systems and produces a few surprises. Webometrics ranks more than ten thousand universities around the world, which is many more than other measures provide. For Australia 199 institutions are listed, whereas other rankings mostly cover the 43 accredited as universities, not colleges which also issue Higher Education qualifications.

Webometrics uses four measures: Presence, Impact, Openness and Excellence. While the weighted score of these measures gives a similar result to other measures, there are some surprises. For example, many ranking system give a similar result to the the Times Higher Education University Rankings with the "Group of Eight" first:

1. University of Melbourne
2. Australian National University
3. University of Queensland
4. University of Sydney
5. Monash University
6. University of New South Wales
7. University of Western Australia
8. University of Adelaide

Webometrics lists the same top Australian universities , but in a different order:

1. University of Melbourne
2. University of Queensland
3. University of New South Wales
4. Australian National University
5. University of Sydney
6. University of Western Australia
7. University of Adelaide
8. Monash University

The ANU slips two places due to low Presence and Excellence scores. Monash drops three places to eight position, due to a very low Presence score.

The Presence measure is unusual in university ranking systems, in that it measures the quality of the university's website. Ranked by presence, the top eight Australian universities are:

1. University of Queensland
2. University of Melbourne
3. University of New South Wales
4. Australian National University
5. RMIT University
6. Edith Cowan University
7. University of Western Australia
8. Macquarie University

Sydney, Adelaide and Monash are replaced by RMIT, Edith Cowan and Macquarie. This effect of the Presence is more pronounced with world rankings, where the usual US and UK prestige institutions rank first in Webometrics, but for presence the University of São Paulo and  Fundação Getúlio Vargas (Brazil), at 6 and 7 in the world, outrank Clatech and the University of Oxford. 

Another interesting result is that universities with "open" in their names do not rate highly on the Webometrics openness scale (the Open University UK ranks highest at 403). Similarly those with "virtual" in their name do not rate highly for web presence (Tamil Virtual Academy, is highest at 1172).

The institutions I have studied at most recently rank as follows on Webometrics:

Rank	Institution	Presence	Impact	Openness	Excellence
78	ANU	300	70	69	131
738	USQ	1438	658	718	1200
1242	Athabasca University	1256	1169	1362	1879
8175	CIT	4961	6352	8635	5778

This is much as I would expect, with ANU being a leading research university, USQ and Athabasca teaching universities and CIT a vocational college. However both USQ and Athabasca score poorly compared to ANU on presence and openness, despite their emphasis on e-learning and access to education. USQ claims a "commitment to open education" and Athabasca describes itself as "Canada's Open University", but neither rates in the top one thousand.

Reduce Number of Australian Universities to Improve Rankings

The latest Times Higher Education's World Reputation Rankings has only three Australian universities in the top 100:  University of Melbourne, Australian National University and University of Sydney. These rankings are based on the opinions of academics. As Asian countries increase in wealth they can afford to spend more on their universities and so their rankings will increase and those of other countries will continue to decline.

The rankings are not very meaningful for academics, as it is the team for a particular sub-discipline which is important, not the institution overall. However, the university reputation is important in terms of attracting students and funding.

One way Australia could improve its rankings is to have fewer universities. Currently Australia has 43 universities for 1.3 million students, which is about 30,000 students per university.  Research by Garrett (2016) indicates that larger institutions (100,000 to 500,000 students) were growing, whereas smaller ones were declining. This suggests Australia should have about 10 universities, each with about 130,000 students. This need not require a radical restructuring or closing of campuses, just merging of university brands.

Garrett's study was of on-line universities and it might be argued that this is not applicable to Australia's campus based institutions. However, a radical restructuring of universities is happening in Australia, brought on by adoption of e-learning. This will happen regardless of the size of Australian universities and will happen regardless of what they do. Currently this restructuring is being largely undertaken by stealth under the label of "blended learning". However, whatever it is called, if Australian universities don't restructure for e-learning and have a sufficient size to make them viable, they will be put out of business by overseas institutions offering education on-line to Australian students.

References

Garrett, R. (2016). The State of Open Universities in the Commonwealth: A perspective on performance, competition and innovation. Retrieved from: http://dspace.col.org/bitstream/handle/11599/2048/2016_Garrett_State-of-Open-Universities.pdf?sequence=1&isAllowed=y

Are Malaysian Universities Performing Poorly?

Murray Hunter asks "Why Malaysian universities are performing poorly" (OnLine Opinion, 11 January 2016). He notes that Malaysian universities do not rank well in regional or global scales, such as QS World University Ranking. I checked the latest rankings and found the top ranking Malaysian institution was Universiti Malaya at 146, then another four in the top 400 and two in the top 1,000. This does not appear such a bad result to me. Singapore has only two, although they rate higher that Malaysia (NUS at 12 and NTU at 13).

Hunter dismissed a suggestion by Dr. Kamarudin Hussin, VC of Universiti Malaysia Perlis (Unimap) that the rankings favor established universities. I checked Malaysia's ranking on a slightly different and more progressive ranking system, the Raking Web of Universities (RWU) This lists 118 Malaysian institution (many are "colleges" rather than universities and also campuses of overseas institutions). University of Malaya comes top, as it did in QS. The RWU lists 11,898 institution,whereas the QS stops ranking at 701. UM's ranking of 461 in RWU is there a good result. The RWU places more emphasis on factors such as the university's web presence, which perhaps reflects Malaysian institution's better ranking than on QS.

Hunter suggests that the poor performance of Malaysian universities  is due to a lack of academic freedom and expenditure on non-academic items. However, he had already commented that universities in countries with authoritarian governments had done well in the QS rankings. Also allegations of lavish entertaining and trips by university staff are not confined to Malaysia.

Hunter suggests Malaysian universities are "dominated by vice chancellors who are intent on micromanaging their universities". This is not a new complaint about universities world wide, nor is his proposed solution: "re-organize Malaysian public universities from the top down". However, if the problem is micromanaging from the top, then I suggest any top down approach will likely make matters worse, rather than better.

Universities are not top-down organizations and VCs do not really run them. Universities are made up of semi-autonomous units which do the real work. A university is similar to a corporation with multiple business units, or a country which is a federation of states (as Malaysia is). There is a delicate balancing act, as to what functions are administrated centrally at a university and what is left to a policy which the parts administer themselves. This also applies at the national level, with government administering some aspects of universities centrally and leaving other aspects to policy. An example is the way quality of research and teaching is set, either through direct setting of standards, or encourager through grants.

Malaysian Universities QS Rankings

146	Universiti Malaya (UM)
289	Universiti Sains Malaysia (USM)
303	Universiti Teknologi Malaysia
312	Universiti Kebangsaan Malaysia (UKM)
331	Universiti Putra Malaysia (UPM)
551	International Islamic University Malaysia (IIUM)
701+	Universiti Teknologi MARA - UiTM
701+	Universiti Utara Malaysia (UUM)

Malaysia in Raking Web of Universities

Ranking	World Rank	University	Det.	Presence Rank*	Impact Rank*	Openness Rank*	Excellence Rank*
1	461	University of Malaya		601	1158	250	380
2	517	Universiti Teknologi Malaysia		94	1471	164	526
3	522	Universiti Sains Malaysia		984	1090	569	422
4	589	Universiti Putra Malaysia		173	1684	726	484
5	703	Universiti Kebangsaan Malaysia / National University of Malaysia		771	1863	677	527
6	881	Universiti Teknologi MARA / MARA University of Technology		120	2141	441	1063
7	1186	Universiti Tenaga Nasional		1750	1377	2217	1610
8	1372	Universiti Tun Hussein Onn Malaysia		1431	2283	1970	1543

Rate University Teaching Quality On-line

In "The hunt for Australia's best teaching uni" (The Age, 9 March 2015), Gary Newman asks how to rank the quality of teaching at universities. Using the measures which have been used he lists Australia's top universities for teaching as: Victoria, Griffith, Deakin, Central Queensland, Tasmania, Edith Cowan, Canberra, Federation, Bond, Notre Dame, Sunshine Coast, New England and Western Sydney. That seems a reasonable list, with the regional universities not known for research having to focus on teaching. Also many of these institutions started as technical and teachers colleges devoted to education and now have an emphasis on e-learning (which requires a more professional approach to teaching).

As he points out the current ranking systems are based largely on research quality which is not necessarily related to the quality of teaching. He says a 2013 study provides some data (this appears to be "Factors Associated with Job Satisfaction Amongst Australian University Academics and Future Workforce Implications" by Peter Bentley, Hamish Coates, Ian Dobson, Leo Goedegebuure and Lynn Meek).

One interesting variation in rankings is the Ranking Web of Universities (Webometrics Ranking), which bases it assessment on the university's web presence. This produces a result similar to other measures but is easier to compile. Perhaps that could be applied to provide a measure of teaching quality, not just for Australian institutions, but world wide.

This, of course assumes that students are interested in teaching quality. It may be that students select a university based on its research reputation, as that is how the quality of their qualification will be seen.

University Ranking

The Global Employability University Survey 2014 has ANU as the top ranking Australian university (20 in the world), followed by Monash University (33), UNSW (55) and University of Melbourne (50).

This compares with the Times Higher Education World University Rankings 2014-2015 with the top Australian institution, University of Melbourne (33), then ANU (45), Sydney University (60), University of Queensland (65), Monash University (83), UNSW (109).

The university ranking system I find most interesting  is Webometrics. This is based on an analysis of the university's web pages. With measures for presence, impact, openness and excellence. On these measures, Australia has first the University of Melbourne (82), then University of New South Wales (96), University of Queensland (98),  Australian National University (101) and Monash University (114).

Ranking Web of Universities

Various rankings of universities have been published. These use a combination of research publications, status of university staff and some measures of teaching quality. This can influence a student's choice of an institution and so is taken very seriously by university administrators. One interesting variation is the Ranking Web of Universities (Webometrics Ranking), which bases it assessment on the university's web presence. Apart from making the process more automated, this more closely mimics a world where academics get information on-line, not from traditional publications. Interestingly the results from Webometrics are not so different from the ranking produced by more labor intensive and traditional methods.

In the latest Webometrics, the top ranking Australian university is the Australian National University (ANU) at 76 in the world. The overall rank is compued from three components: Presence Rank: 335, Impact Rank: 96, Openness Rank: 110, Excellence Rank: 131.

ANU is followed in the Webometrics list for Australia by: University of Melbourne, University of Sydney, Monash University, University of Queensland. The Times Higher Education World University Rankings  has University of Melbourne before ANU,  then University of Sydney, University of Queensland and University of New South Wales, with Monash University relegated to sixth place. The Academic Ranking of World Universities has the University of Western Australia displacing Monash for fifth place.

While universities would need to make large investments in research over many years to improve their ranking in traditional systems, it should be much simpler and cheaper for university administrators to improve the Webometrics rank. This is because web "presence" and "openness" are two of the four criteria used.

Administrators may not be able to get researchers to do research any quicker or to write better papers, but they can improve the university's web site and access to on-line publications. The web sites can be improved by using web accessibility guidelines, so that web pages are easy to access. Also universities can pay the additional publication fees to have journal papers made open access, so that readers don't need to pay a subscription to read them.

Often marketing and graphic design staff produce complex, hard to access web designs, in the mistaken belief these will appeal more to readers. What in fact happens is that web search engines can't index the content and people, particularly using mobile devices can't read the documents. It is better to use simple web formats.

Researchers will choose the most prestigious journals to publish their papers. However, these publications tend not to be "open access" and require a subscription to read the articles. Some journals offer to make individual papers open access if an additional fee is paid by the author. But academic are reluctant to pay these fees, unless there is a requirement from their institution (and a special grant) to do so. As a result these closed access papers tend to be read and cited less than open access ones, simply because they are harder to get on-line.