Judging Universities by Papers Published and Grants Got is a Really, Really, Bad Idea

The Australian Tertiary Education Quality and Standards Agency (TEQSA) is considering how to measure the quality of research undertaken at a university. This is important, as Australian law does not allow for teaching only universities. If an institution doesn't have quality research, it can't be a university, regardless of the quality of the education provided. 

TEQSA's draft  proposed legislative instrument includes the volume of citations in peer-reviewed journal papers. Ironically, there is a considerable body of peer-reviewed literature to say citations are not a good measure of research quality. 

Australia's first university was established  to advance "... religion and morality and the promotion of useful knowledge ..." and  provide professionals with "... proficiency in literature, science and art ..." (Sydney University Act, October, 1850). The aims of universities to the present day are similar, for cultural and useful knowledge, plus the training of professionals. Publications in peer reviewed journals may go some way to do some of this, but I suggest measures of how useful the research carried out at universities is and how widely its is disseminated beyond the academic community.

The COVID-19 pandemic has shown the value of the professional training provided by university, with medical researchers and professionals key to combating the disease. However, this has also shown limitations. The paper review process has been unable to respond quickly enough to need, with un-reviewed papers being relied on. Also the dangers of scientific research being misunderstood, and in some cases deliberately misrepresented has been seen.

Global warming is another instance where scientific publications have been of limited value. Researchers have show a lack of ability, and willingness, to engage publicly, thus making their research of far less value.

I suggest that universities need to be judged on the value of their research output to the community and on the effectiveness of their researchers to communicate the results. The quality and quantity of research graduates should also be part of this measure.

ps: Any discussion of research quality quickly gets philosophical, putting me in mind of Pirsig. ;-)

Zen and the Art of Motorcycle
Maintenance, Robert M. Pirsig, 2006

"... the real university exists not as the physical campus, but as a body of reason within the minds of students and teachers ..." From Chapter 13, Zen and the Art of Motorcycle Maintenance, Robert M. Pirsig, 2006