Australian Universities Rank Well for Impact Globally

Australian universities have come out well in The Times Higher Education Impact Rankings 2021, having 4 in the top 10 and 17 in the top 100.

Times Higher Education assesed universities against the UN Sustainable Development Goals (SDGs). Out of 1,115 universities, the University of Manchester (UK) comes first, followed by three Australians: University of Sydney, RMIT and La Trobe. University of Wollongong comes in sixth. 

On the SDG goal of "Quality of Education" (central to the mission of universities), Australia did not do so well, with only one in the top 10: University of Canberra at 5 and only 12 in the top 100.

The SDG goal of "Gender Equality" gave an interesting result, with a Saudi Arabian institution coming out on top: Princess Nourah bint Abdulrahman University, followed by three from Australia: La Trobe University, Western Sydney University and Charles Darwin University.

Australia had no universities in the top ten for "Industry Innovation and Infrastructure", with University of British Colombia (Canada) on top. This is well deserved, I visited UBC in 2014 to look at education and innovation.

Of course not too much should be read into these type of rankings. Those universities without the resources to devote to filling in the information requests from ranking organisations will tend to not do so well. They might instead want to devote resources to helping with development. In particular thousands of training focused institutions, which are providing most of the practical assistance with development, don't appear in these rankings. The "Webometrics Ranking of World Universities" includes about 9,000 more institutions than Times. 

Unfortunately some universities choose to release misleading statements based rankings. As an example CQ University previously claimed "CQUNI RANKS IN TOP 25 PER CENT OF UNIS WORLDWIDE FOR SUSTAINABILITY" (23 April 2020). However, only about 1,000 institutions are included in the rankings. So we don't know how CQU performs against the many more thousands not included. While it might be assumed these institutions would not rank well against leading universities in terms of research output, that may not be the case in terms of quality of education or sustainability. Australian universities should refrain from making such misleading claims as it could damage their reputation and that of the nation. In 2017 the UK Advertising Standards Authority issued guidelines to stop UK universities making similar misleading claims.  Perhaps similar guidelines are needed in Australia.

ps: I made a submission to the Australian Senate on the United Nations Sustainable Development Goals in 2018, on "Education for Sustainable Development".

Usable and accessible online courses

Greetings from the seminar room of Australian Centre on China in the World (CIW)at the Australian National University where I am taking part in "How usable and accessible are your on-line courses?" by Professor Denise Wood, Central Queensland University, and Dr Sheila Scutter, University of South Australia. We were first introduced to the principles of accessible web design and the more general Universal Design for Learning. They are designing a MOOC for teaching this. Jeremy Smith from ANU received an acknowledgement for his contribution.

The project is not just about making course materials readable for people with a disability, it is also  about accessibility more broadly. Also it is about making course accessible on-line for completely remote delivery, or in blended mode.

Some of these are issues of basic course design, which will benefit all students. For example, how to how to contact the instructor and what course materials are available are issues which are not well addressed in traditional classroom based courses and can be improved with on-line support.

One tool mentioned was the Student Usability Scale (SUS). Also the use of software installed on the user's computer to record a test session for the usability of a course was discussed. The Flexible Learning for Open Eduction (FLOE)  tool was demonstrated.

I am familiar with some of these accessibility issues having been an expert witness in the Sydney Olympics 2000 Accessibility case and invited to advise the Beijing Olympics.

Wood and Scutter gave an excellent seminar on well run research into teaching. Unfortunately, this appears to be another of the Office of Learning and Teaching (OLT) funded research projects which is addressing the wrong question. There are decades of research on on-line course design and accessibility. There are tools and training courses based on this research in routine use in the vocational education and training (VET) sector for developing accessible courses. The question is why Australian universities do not implement what has been shown to work in the VET sector.

For anyone teaching across the vocational and university sectors (as I do), the differences are obvious. In the vocational sector (and in distance/open universities) the courses are designed by a team and then delivered by instructors. All the courses for a program use the same template, with course components laid out in the same way. The instructor has limited, or no, ability to change the template or the course materials (even to correct spelling errors or broken links). As a result, accessibility is designed in to the program, at the cost of flexibility.

Universities, particularly research orientated universities, allow their academics more freedom in course design and delivery than the vocational sector. The academic can teach what they want, how they want. But researchers are trained to conduct research and are understandably reluctant to take the extra time needed to learn how to design courses and teach. This problem is not unique to Australia. Last year I talked to staff at Cambridge University about how to help the graduate students with on-line courses. One option would be to incorporate basic teaching skills in postgraduate research programs, bringing early career academics up to the standard of VET teachers. Bryant and Richardson (2015) found that university lecturers with teacher training tended to have fewer failing students.

Academics producing a course content need to only work on the content, using the skills of specialists to worry about how it is presented. As an example text can be written using the default formatting features of a word-processor, so that it will inherent the style set up for the institution's template. I have used this approach for the course "ICT Sustainability", enabling the course content to restyle automatically when the institution's template is updated. This has also allowed the course to be ported between three very different higher education institutions (vocational, open and research orientated) and also adapted for mobile devices with minimal additional work required.

Scriven describes how from the 1970s he became increasingly disenchanted with evaluating US education programs based on the goals of the programs, leading to Goal-Free Evaluation (GFE)  (p. 56, 1991). A program designed to improve the usability and accessibility of courses is likely to be successful by its own internal logic: those academics who are motivated to take part are likely to produce better courses. However, using GFE, the bigger question becomes apparent: how do we motivate Australian university academics to improve courses? I suggest this is the area needing research.

References

Bryant, D., & Richardson, A. (2015). To be, or not to be, trained. Journal of Higher Education Policy and Management, 37(6), 682-688. DOI: 10.1080/1360080X.2015.1102818

Scriven, M. (1991). Prose and cons about goal-free evaluation. Evaluation Practice, 12(1), 55-63. Retrieved from http://dx.doi.org/10.1016/0886-1633(91)90024-R

Marketing Universities

Universities are all selling the same products (research and education), so it is not surprising if their marketing is similar. Prestige institutions use their research reputation to market education ("We have a Nobel Prize winner!"), while those without a research record use vocational relevance ("We will get you a job!"). Australian universities seem to be going through a spate of re-branding at the present.

The University of Western Sydney (UWS), a vocationally orientated institution with a good reputation, is re-branding itself "Western Sydney University" (WSU or "W" for short). This seems a reasonable change to make, but there has been some criticism that a large part of the $20M marketing budget will be spent on this. See "University of Western Sydney to get new name and logo", by Melanie Kembrey, Sydney Morning Herald, August 7, 2015.

 A more questionable case of re-branding is that Central Queensland University (CQU), a modest university (in, as the name suggests, central Queensland), is reported to be changing to "Australian National Regional University" (ANRU"). CQU is a regional university in Australia, but the problem is that the new name would be very close to that of the "Australian National University" (ANU), one of the country's leading research universities (where I am an Adjunct Lecturer). See "University games: protect ANU from CQU's rebranding overreach" by Ross Fitzgerald, The Age, 6 August 2015.

Multiple Choice Questions Have a Role In Education

In its flexible learning strategy, the University of British Colombia identified issues for universities, the most important of which I would summarise as

1. Increasing focus on vocational education,
2. Online competition,
3. Demand for mid-career education.

The problem for higher education institutions is to provide courses which are accessible (preferably online), meet the vocational needs of students, but are also academically sound. In announcing the decision of Central Queensland University (CQU) to discontinue the use of multiple choice questions for examinations, Pro Vice-Chancellor, Professor Rob Reed, discussed the history of educational psychologist Frederick Kelly's use of the tests in WW1 US Army intelligence testing. CQU was concerned that multiple choice questions were not suitable for assessing real world skills. 

Anderson and Dron (2013) categorised three generations of distance education pedagogy based on the learning theories underlying them. However while being an improvement on an analysis simply based on the technology used, this still assumes that only one approach to learning can (and should) be applied in a course (or program). Students need to have a basic knowledge of a topic, before learning advanced skills. Rather than adopt one teaching and assessment approach for a whole course, these should suit the particular material and level. It is proposed that a combination of techniques can be used to teach basic and advanced material to the same students using the same learning technology.

 Multiple choice questions (MCQ) are a useful way to provide a student with feedback on their progress. This should not be seen as an alternative to assessment of deeper learning, but as a supplement. By banning MCQ for examinations, it is not clear if they have prohibited their use for other forms of assessment.
A search of the CQU Handbook found many courses for semester 3 2014, which have on-line quizzes as part of their assessment. An example is Aviation Theory IV (AVAT12004), which has seven on-line quizzes making up 70% of the assessment and "Written Assessment" for the remainder. It may be that these on-line quizzes use no multiple choice questions and all require numeric or short form text answers. However, that would not appear to be a particuarly useful assesment strategy, as it would take more time and staff effort, while not appreciably improve the quality of the assessment.