Governance at Australian Higher Education Providers

The Senate Education and Employment Legislation Committee is holding an inquiry into the "Quality of governance at Australian higher education providers". I made a submission on 3 March. The Committee has not approved the publication of my submission yet, but it was along the following lines:

1. TEQSA Powers Adequate: The Tertiary Education Quality and Standards Agency (TEQSA) can investigate corporate governance issues at Australian higher education providers, where relevant to the quality of education provided. Compliance with workplace, employment practices, executive remuneration, and the use of consultants are covered by other agencies.  
2. Introduce Real Time Monitoring: TEQSA could use real-time analytical techniques to detect unusual behavior in an organization in days, rather than having to wait for an annual report.
3. Focus on more important issues: Australian universities face challenges adapting to online education and competition from offshore providers. These are more pressing issues than governance.

Senate Inquiry into the Quality of Governance at Australian Higher Education Providers

The Senate Education and Employment Legislation Committee is holding an inquiry into the "Quality of governance at Australian higher education providers". Submissions are invited by 3 March 2025, with a report due 4 April 2025. 

 The terms of reference of the inquiry are:

"The adequacy of the powers available to the Tertiary Education Quality and Standards Agency to perform its role in identifying and addressing corporate governance issues at Australian higher education providers, with particular reference to:

a. The composition of providers' governing bodies and the transparency, accountability and effectiveness of their functions and processes, including in relation to expenditure, risk management and conflicts of interest;

b. The standard and accuracy of providers' financial reporting, and the effectiveness of financial safeguards and controls;

c. Providers' compliance with legislative requirements, including compliance with workplace laws and regulations;

d. The impact of providers' employment practices, executive remuneration, and the use of external consultants, on staff, students and the quality of higher education offered; and

e. Any related matters."

Some points in the submissions so far

So far five submissions have been published:

1	Dr Fiona Martin (PDF 60 KB)
2	Mr Ian Gray (PDF 1310 KB)
3	Mr Robert Heron (PDF 56 KB)
4	Dr Raffaele Ciriello (PDF 93 KB)
5	Emeritus Professor William Maley (PDF 328 KB)

Dr Martin expressed concern about universities being more about business than education and research and high Vice Chancellor salaries. Mr Gray is concerned about TEQSA's powers to do their job, lower entry and pass standards at universities & a lack of reporting & auditing of research grants. Mr Heron proposes public reporting on accredited courses & is concerned about conflicts of interest where academics require students to purchase the books they wrote. Dr  Ciriello is concerned by "a managerial elite that prioritises profits over academic integrity".  and proposes adopting the European model, where university executive are elected by faculty. Dr Maley expresses concern about the decay of ‘faculty governance’, with decision making concentrated in Deputy and Pro-Vice-Chancellors, and Chief Officers. 

TEQSA Masterclass Workshop on Detecting Contract Cheating

Penny Wheeler, Amanda White, &
Tom Worthington, at the TEQSA Masterclass

Greetings from the TEQSA Masterclass Workshop on Contract cheating detection and deterrence, at the National Museum of Australia in Canberra. This builds on the online Detecting Contract Cheating course from TEQSA. The good point of the course and workshop is that they raise uncomfortable questions for university academics. But I am not so sure the answers presented are the correct ones. Rather than a "Law & Order - Cheating Investigation Unit", I suggest universities should focus on quality assessment to deter cheating, and structure it to prevent cheating students from ever graduating.

I had some concerns about the legalistic approach in the online module leading to discrimination against vulnerable students, and even more so in the face to face workshop. Also TEQSA clearly envisages having centralized cheating investigation units. 

There is also discussion of what are plea bargains, where students receive a lesser penalty, for admitting cheating. However, if there are to be professional investigators, I suggest they will need to be formally qualified. Also those making the decisions will have to have formal training how to make administrative decisions (similar to the Administrative Decision Making course offered by the Australian Public Service Academy). It does not seem reasonable to deny a degree to a student, based on evidence collected and decided by people who do not even have a relevant TAFE certificate.

Having done an online course, and a face to face workshop, with no form of assessment is not sufficient. Government investigators at least have a vocational certificate (such as a Certificate IV in Government Investigations offered by the Canberra Institute of Technology), and something similar is required for university investigators. Similarly, those serving on tribunals have to have some form of training. 

I suggest that universities should look to their vocational education colleagues, who are much more used to dealing with nationally standardized regulations. 

My preference would be to decriminalize cheating, much the way is being done with illicit drug use in Canberra. Students would be given a failing grade where they cheated, and given compulsory training.

A related issue which came up during the workshop is educational institutions complicity in visa fraud. This where someone obtains a student visa, in order to be able to work in Australia. Those who accept such visas are at risk to being exploited by organised crime. It came as a surprise when I discovered the universities were processing visa applications on behalf of government. It would be tempting to return the function to government staff. But those staff may not have the specialized knowledge of education. It may be that educational institutions, particularly smaller ones, should form consortia and pool staff.

The investigation processes illustrated by TEQSA depended on technical means involving Learning Management System logs, document metadata, Internet Protocols and the like. Such procedures will only catch relatively unsophisticated students, and commercial cheating services. It might be useful to build technical means into the university systems to deter such activities. As an example, the tool could ask the student when they login from a different IP address to explain where they are. Similarly when submitting a document, the system could check the metadada on the document for consistency. The contract cheating company could, of course, coach the students to provide plausible answered, but otherwise only the students who can't afford to pay for good cheating will be caught. 

The course and workshop focused on contract cheating. They were designed before generative AI came to public attention. It is unfortunate TEQSA did not anticipate what was coming and so was required to instead react. They are now trying to catch up. An AI topic which has not yet been addresses is AI for teaching and assessment. I took part in a workshop on building an AI tutor chatbot in 2018, which had some interesting implications.

Does speaking a language other than English really make you cheat more on assignments?

I am doing the "Detecting Contract Cheating" course from TEQSA. In Module 2 "Supply and demand" under "who are cohorts likely to cheat" lists "Students who speak a language other than English". That is, if you speak a language other than English, the claim is you probably cheat. But isn't that most of the people in the world? I find it hard to believe that English speakers are inherently honest, or not speaking English makes you dishonest. The course cites Bretag et. al (2019), but my reading of it doesn't support that conclusion.

However, the course, so far, is mostly good. In particular I like the section on how to deter contract cheating, by providing the students with the help they need. I had to give this some thought, because it is easy to say there is no excuse for cheating. However, if your family, or your entire village, have invested everything they own in your education, if you are having difficulty with the language, you have limited time for study away from family or job (or both), if the universality does offer you any help getting up to speed with the topic, it is very easy to be tempted to cheat.

The course consists of some short videos, which like a typical student I play at high speed and read the subtitles. Curiously, the videos kept referring to "expectations" for what higher education intuitions would do about cheating. This is odd as TEQSA are overseeing legally mandated practices, which I would have thought were "requirements", not expectations.

The course contains so odd phrases, such as "collective competence", to describe a team of people who, between them, have the necessary skills to deal with academic misconduct. This appears designed for individuals to avoid responsibility. The example of a surgical team is used. But every member of such a team is tested for individual competence. No hospital would allow a team to operate where it was not clear that every member had the required skills.

The course also contained a weird video explaining the usefulness of subject matter experts. It was as if universities normally use random individuals to do assessment, and using experts is an innovation. This was followed by a video about the value of "investigators". This is a term I learned a few months ago. It is something unlikely to be familiar to most who teach at university, and needs more background, explanation, and justification. Also the full title of the role should be used, not just "investigator", as this is also a term used in academic research (I have been a chief investigator). While not mentioned in the course, I assume investigators are required to be qualified, to a similar level to those who carry on investigations in Commonwealth agencies, with a Certificate IV in Government Investigations (PSP40416).

The course has some views on the seriousness of plagiarism charges I don't agree with. As an example a finding of plagiarism resulting in a 10% grade penalty is classified as low severity. However, as a student I found the one and only time I was accused of plagiarism as very serious. Fortunately this was a mistake by my instructor, as they had accepted Turnitin's text matching report. Turnitin matched my assignment with the published paper I produced from the assignment (I hadn't expected the paper to be published so quickly, so hadn't mentioned it in the assignment).

Some of the legal advice in the course may need checking. As an example, it is suggested if an institution receives a report from a contract cheating service that one of their students is using it this may be blackmail, and so the institution should report this to TEQSA, but that cheating at an institution is not a crime. I am no legal expert, but thought blackmail is a crime, to be investigated not by TEQSA, but by the police (or in some states under the jurisdiction of a corruption commission). Also cheating at an institution may be a crime, where the student profits from it. Also the course argues that the saftey and well being of the students should be the priority. However, if cheating is resulting in many incompetent people being licenced to carry out dangerous procedures, the public interest may outweigh the student's interests.

Some of the approaches proposed by the course will throw suspicion on innocent studnts, and unfairly discriminate against specific groups. As an example, those with a disability who use accessibility tools will likely have the metadata stripped from their documents. If the investigator, as suggested here, looks for the name of the student and the time they took to edit to be consistent will suspect cheating when all the happened as when student prepared their answer in their accessibility tool, then pasted it in. I have this problem with some online systems, which do not allow me to spell check. I have to compose offline, then paste the answer in. That can look like I am getting someone else to write it.

Another approach which might cause unfair suspicion is bibliographic forensics. The course suggests checking that the references a student uses match their expected level of knowledge. However, I was a student in a topic I had two decades experience in, but the rules required me to do the introductory units. To avoid the tutors saying (as they occasionally did "Tom what are you doing in this class?"), I would hind my knowledge of the topic and pretend to be an ordinary student. But using the techniques in this course that could make me look like I was cheating.

The course suggests LMS logs as a source of evidence, but doesn't discuss under what conditions this, or other, information can be used against the student. Do investigators need a reasonable suspicion, or can they simply sift through the LMS logs looking for any wrongdoing? To use an extreme analogy, they could also use face recognition on security cameras in student dorms to look for collusion, but a court would likely find that an unreasonable invasion of privacy.

There are some power imbalances in the procedures suggested in the course. As an example, the student is required to provide proof of identity at an interview, but the interviewer is not. It is a well worn cliche of police procedurals that investigators show their identification to suspects. Another example is that it is suggested that a support person not be permitted to speak extensively to the student in a language the interviewer does not understand. Unless it is slowing down the interview, what right does the interviewer have to prevent the student and their support person communicating in whatever language they are most comfortable with?

The course suggests having studnts bring their device to the interview. I find it disturbing that investigators would conduct a search of a student's device, as this is likely to contain a lot of personal, private, and sensitive information, both their own, that of family and friends, and of their employer. If asked to do this, my first thought would be to wipe the device clean of all records and logs beforehand to protect myself, family, and clients. That act in itself might then be considered suspicious by the investigator.

It is a little annoying as when I stopped to fill out the workbook, I ended back at the start of the module, and had to fast forward through the video again. The modules seem far too large to each do at one sitting. But if I stop, I risk having to go back to the start.

The workbook is in PDF. To enter my work I had to import it into the word processor, where the formatting went askew. I have to create a text box for each answer, and hope it will be readable in the final result.

Also I received a warning I had an unreliable Internet connection, and my progress may not be saved. That seems odd, as I am sitting in my university office with a very high speed internet connection. Most of the time everything loaded quickly, although the final reflection video stuttered. 

Overall the course was of some use for learning more about how to investigate cheating, but was based on an approach using formal investigators. I would have liked some background as to why and how a decision had been made to introduce this role, and if all universities are required to implement them (if so, when?). Also I signed up for the course so I could go to a workshop. But I now can't see any details of the workshop, and it took me so long to do the course (about 7 hours, whereas it was supposed to take 3), that I have forgotten where I saw the workshop details. In hunting around I found a Situational Judgement Test which was taken before the course (and I had forgotten about).

Curiously I was not prompted to take the test as part of the course, but had to stumble over it. Finding what I might have to do next is difficult as I have the text enlarged to make it readable, so all I see is "Modules Situational Situational", not what the modules are, or what "situational" is.

References

1. Tracey Bretag, Rowena Harper, Michael Burton, Cath Ellis, Philip Newton, Pearl Rozenberg, Sonia Saddiqui & Karen van Haeringen (2019) Contract cheating: a survey of Australian university students, Studies in Higher Education, 44:11, 1837-1856, DOI: 10.1080/03075079.2018.1462788
2. 
   

Essay Mill Website Blocked Under Australia Law

Essay Mill Website Before Ban

Websites which offer to write assignments for students are now banned under Australian law. The first example of such an essay mill, I have seen blocked was offering anything from a one page undergraduate paper to a complete PhD thesis, for about $20 a page. The site now displays in Australia 'The service is unavailable in Australia under the "Tertiary Education Quality and Standards Agency Amendment" law.'. Curiously this does not give the full title of the legislation, leaving out "Prohibiting Academic Cheating". So I think this may be action by the website provider, to avoid prosecution, than by the relevant government agency. 

It is easy enough to get around this for the desperate student. The website provider can they try to deny any wrongdoing. However, the student is likely to be disappointed when, despite assurances of the supplier, their deception is detected, and they are subject to academic disciplinary procedures. Students who are studying for entry into a profession need to keep in mind that there may be very severe legal consequences if they cheat and so are not competent to do their job.

Judging Universities by Papers Published and Grants Got is a Really, Really, Bad Idea

The Australian Tertiary Education Quality and Standards Agency (TEQSA) is considering how to measure the quality of research undertaken at a university. This is important, as Australian law does not allow for teaching only universities. If an institution doesn't have quality research, it can't be a university, regardless of the quality of the education provided. 

TEQSA's draft  proposed legislative instrument includes the volume of citations in peer-reviewed journal papers. Ironically, there is a considerable body of peer-reviewed literature to say citations are not a good measure of research quality. 

Australia's first university was established  to advance "... religion and morality and the promotion of useful knowledge ..." and  provide professionals with "... proficiency in literature, science and art ..." (Sydney University Act, October, 1850). The aims of universities to the present day are similar, for cultural and useful knowledge, plus the training of professionals. Publications in peer reviewed journals may go some way to do some of this, but I suggest measures of how useful the research carried out at universities is and how widely its is disseminated beyond the academic community.

The COVID-19 pandemic has shown the value of the professional training provided by university, with medical researchers and professionals key to combating the disease. However, this has also shown limitations. The paper review process has been unable to respond quickly enough to need, with un-reviewed papers being relied on. Also the dangers of scientific research being misunderstood, and in some cases deliberately misrepresented has been seen.

Global warming is another instance where scientific publications have been of limited value. Researchers have show a lack of ability, and willingness, to engage publicly, thus making their research of far less value.

I suggest that universities need to be judged on the value of their research output to the community and on the effectiveness of their researchers to communicate the results. The quality and quantity of research graduates should also be part of this measure.

ps: Any discussion of research quality quickly gets philosophical, putting me in mind of Pirsig. ;-)

Zen and the Art of Motorcycle
Maintenance, Robert M. Pirsig, 2006

"... the real university exists not as the physical campus, but as a body of reason within the minds of students and teachers ..." From Chapter 13, Zen and the Art of Motorcycle Maintenance, Robert M. Pirsig, 2006

Designing Out Contract Cheating While Designing In Quality Education

Phillip Dawson

The Australian Tertiary Education Quality and Standards Agency (TEQSA) has an Experts advice hub which is particularly useful with the move to online education. One useful piece of advice is "The prevention of contract cheating in an online environment" by Associate Professor Phillip Dawson, Deakin University. At first I found this a bit annoying, as it starts with 3 myths about contract cheating: it is rare, it can be designed out, and is impossible to detect. After reading these I concluded that TEQSA wants me to give up on project based progressive assessment and just set one exam at the end of semester.
But reading on Professor Dawson does suggest assessment design approaches to help reduce the prevalence of cheating: reflections on practical work, individual work, and in-class tasks. Assessment which it is suggested most attracts cheating are ones with lots of marks, and short deadlines, such as take-home exams.

One of the key problems I suggest causing contract cheating is the lack of time and effort academics think they should put into assessment design and delivery. Like most, my first exposure to assessment was being asked to set exam questions and mark assignments, with no prior training. Completing a couple of courses on how to design and deliver assessment was a revelation. Much of what I had been doing turned out to be, at best, irrelevant, and in some cases counter productive. Also the amount of time needed to do assessment properly was sobering.

I had assumed that assessment was an afterthought tacked onto a course. Much of the frustration of academics perhaps comes from this assumption. Assessment should take up about half the staff time in a typical course. Once you accept this, it is less frustrating how much time it takes, as you expect this.

As the guide suggests, many small assessment tasks, with generous deadlines, place less pressure on students to cheat. Practical work, where each student has a different project and where they have to reflect on what they did makes cheating harder. Where the student has to explain what they did to the assessor this also helps. However, these all take much more work to design, administer and grade. These also take skills which the average academic doesn't have, because it was not part of their teacher training (assuming they received any teacher training).

As an example, I once sat at a course planning meeting where we discussed the assessment of reflective e-portfolios. As the discussion progressed, I realized that of the dozen tutors and lecturers there I was the only one who had ever completed a reflective e-portfolio as a student, and the only one with any formal training in how to assess them. Without that training ans experience, tutors were assessing the e-portfolios as if they were project assignments.

If you set out to design the assessment for a course, allocating marks to small tasks, thinking about the time the student, and the assessor, will need, it is possible to design most cheating out. However, I suggest emphasizing the benefits for the students, and for their teachers, of more realistic, better planned assessment.

One think academics need to decide is what they are doing assessment. If a graduate needs particular skills and knowledge to undertake a professional role, then there is no need for more than a pass/fail test. If the assessment is to identify those who will undertake further advanced studies, then more fine grained assessment is needed. However, these two approaches can be mixed in the one course. If you don't want students asking for extra marks on every little assessment task, then have these small tasks just count for a pass (or whatever is considered the minimum acceptable level). Reserve the fine grain marking for the important tests.

More Flexibility from Accreditation Bodies During Pandemic

Five organizations with members who train medical and other professionals have called on accreditation bodies to be more flexible during the COVID-19 pandemic. The Australian Council of Professions (ACoP), Universities Australia (UA), the Independent Tertiary Education Council Australia (ITECA), the Australian Collaborative Education Network (ACEN) and the Independent Higher Education Australia (IHEA) are concerned that accreditation procedures may slow the supply of trained staff when they are needed most.

The problem is that front line professionals need hands-on training, supervision and testing, but this is difficult to provide due to lock-downs and stringent processes at places of employment. With the Tertiary Education Quality Standards Agency (TEQSA), they are supporting creation of principles for professional accreditation during the COVID-19 pandemic. The aim is to maintain quality while there are fewer reduced availability of Professional Placements available, and with measures such as online assessments required.

However, the first of the proposed principles I don't agree with: 

a. Professional accreditation bodies recognise that during the current pandemic higher education providers must attend to educational delivery and student needs first.

In a pandemic, the need of the community for intimidate support comes first, with education of students a lower priority. All professional codes of ethics state that the needs of the community override those of the individual professionals and their organizations. Trained staff are needed now to save lives, so training programs may be shortened, and students asked to volunteer for font-line duty, sooner than they otherwise would. As future professionals, this is a valuable lesson for students: my needs do not come first.

One way I suggest accreditation bodies can help, without compromising standards, is though online working and allowing online training and assessment. The need to physically visit and meet with educators should be replaced with an online option, and not hours of tedious Zoom meetings: a genuine online, efficient process.

The use of online forms of reporting by and of students in placement should be encouraged. Much of the tedious paperwork which takes up everyone's time can be put online, streamlined, and in some cases eliminated all together. Some years ago I helped the University of Queensland develop a system for online reporting of occupational therapy students in professional practice placements. This took several attempts and revisions, to get something practical for use in a clinical environment. This proved a lot harder than it first looked.

To a large extent, the need for reporting can be replaced with monitoring of the online logs already kept routinely in the workplace. The Australian National University provides an example of this with its TechLauncher program. Students undertake team projects for real clients, as part of the requirements for Australian Computer Society professional accreditation. Students use online tools to collaborate. A byproduct of this is that the tools keep a log of what every student did, stamped with the time and date they did it. Examiners have access to all the logs, so they do not have to rely on what students, and their clients, say they did. Students, clients and instructors, can also report on the quality of student work, as it is happening, rather than waiting until long after the event.

Professional Accreditation Needs to Move Online

The Department of Education and Training issued the report "Professional Accreditation: Mapping the territory" in September 2017, but this does not appear to have attracted much attention. The report for the Department by PhillipsKPAPty Ltd, looked at the way  100 professional associations set requirements for higher education programs to be suitable for their members. 

The report points out that accreditation processes are similar between professional bodies: with a public document published specifying competencies, or a body of knowledge, which graduates are to have.Most are aligned with Australian post-secondary requirements, such as TEQSA, AQF, and Higher Education Standards Framework.

Of 100 accrediting bodies, all but 14 were self-regulating. The exceptions were for health professionals, under the National Registration and Accreditation Scheme. Half the accrediting agencies belonged to Professions Australia, Australian Health Professions Accreditation Councils Forum or another umbrella body. The authors expressed concern about smaller and newer professional bodies lacking resources for effective accreditation and not drawing on the experience of other groups.

Most accreditation is now national, but some is still state based, with inconsistencies between states. Accreditation of teacher education was identified by the authors as a problem area with a whole chapter (5) devoted to the topic. State authorities were interpreting national teaching requirements inconsistently and adding their own criteria. the authors made the extrondary statement:

"The high political and industrial stakes surrounding initial teacher education confound investigation and resolution of the apparent difficulties in this report, and exceed by far the terms of reference of this overview."

The engineering and computing professions come in for positive comment, with the authors noting that Engineers Australia was an original signatorys to the International Engineering Alliance’s Washington Accord in 1989 and the Australian Computer Society a signatory to the similar Seoul Accord. These accords recognize accreditation processes internationally. As a Certified Professional member of the ACS I benefit from this.

One problem noted, particularly for health professionals, was a requirement for training to undertaken in Australia or by Australian registered professionals. Another issue is programs accredited by multiple professional bodies. One way around this, I suggest, are joint accreditations. As an example, I have been on a panel accrediting a program for both accounting and computing bodies. Even if there are two sets of paperwork to complete, it helps if the educational institution has to deal with just one visit by one panel.

One problem the report identifies are accreditation of programs using capstones, research projects, work-placed learning and reflective journals. These are useful learning techniques, but require specialist skills and it helps if the teacher has been trained using these techniques (I undertook an e-portfolio capstone for my MEd).

The issue of accreditation of online programs does not receive as much attention as it deserves in the report:

• 'In the case of new mixed mode delivery technologies and paradigms such as MOOCs the current approach is to put the onus on the educational provider to provide the evidence that assessment of learning outcomes is rigorous. Some providers express frustration with the lack of familiarity with these methods represented in review panels who tend to prefer traditional face to face approaches to classroom teaching. Some providers are beginning to invite accreditation panel members to log into their learning management systems so they can “experience some aspects of what it is like to be a student.”'

Australian higher education has already passed a tipping point: students now receive more of their instruction on-line than in face-to-face classrooms. Within a few years almost all university education will be undertaken online in Australia. I suggest accreditation bodies need to prepare for this reality, rather than treating it as a novel exception.

One recommendation in the report which may be contentious is:

"Mutual recognition of online and on campus programs could be considered to avoid duplication of content where mode of delivery is the only difference." 

The  topic of how those who accredit programs is briefly covered, with the recommendation:

• "Develop more efficient ways to train assessors – online, collaborative inter-professional, inter-agency training."

The authors appear to have missed the obvious similarity between evaluation of programs within institutions and accreditation by outside bodies. Education specialists are trained in evaluation and I suggest these skills could also be applied to accreditation.

TEQSA Guidance Note on Work-Integrated Learning

TEQSA have issued a Guidance Note on Work-Integrated Learning (v 1.1, 25 Aug 2017). This eight page document discusses how the Higher Education Standards Framework applies to students learning in the workplace, including clinical placements, online projects, internships, and workplace projects.

TEQSA  say that educational institutions are required have a "tenable rationale" for including Work Integrated Learning (WIL), and to monitor the student's experience and learning outcomes, "against defined, expected outcomes".

TEQSA provide an example of non-compliance where the educational institution does not assure the suitability of placements and safety of the students, doesn't provide feedback to the students and just has a single assessment item at the end of the placement unrelated to employability skills.

One approach I suggest could be used is on-line learning management systems and e-portfolios for  monitoring of student's progress and feedback for students and their host. I was on the project management committee for an on-line system developed at the University of Queensland to evaluate students on occupational therapy professional practice placements.

This was originally a paper based process, using the "Student Practice Evaluation Form "SPEF-R" (Turpin, Fitzgerald, and Rodger, 2011) used widely by universities around Australia.  It was converted to a mobile device compatible web based system using bespoke software. However, the same function could now be implemented using off-the-shelf- learning systems.

References

Turpin, M., Fitzgerald, C. and Rodger, S. (2011), Development of the Student Practice Evaluation Form Revised Edition Package. Australian Occupational Therapy Journal, 58: 67–73. doi: 10.1111/j.1440-1630.2010.00890.x Retrieved from http://onlinelibrary.wiley.com/doi/10.1111/j.1440-1630.2010.00890.x/full

Ensuring Quality in University Programs

Yesterday, I commented on Professor Jim Barber's call for changes to Australian law governing the regulation of Australian universities to allow for lower cost on-line courses. Professor Barber proposed the regulations look at the quality of the educational outcomes, rather than measure the input. Today I attended a seminar on how to ensure quality of assessment for courses. This was putting the case for output based measures of quality, applying processes from manufacturing industry.

It may seem inappropriate to treat students as units of production, but if society demands consistent results within and between institutions, then there would seem to be no way to avoid some measurement and comparison of students. The concept of Quality is written into the "Tertiary Education Quality and Standards Agency Act 2011":

                     (a)  to provide for national consistency in the regulation of higher education; and

                     (b)  to regulate higher education using:

                              (i)  a standards‑based quality framework; and

                             (ii)  principles relating to regulatory necessity, risk and proportionality; and

                     (c)  to protect and enhance:

                              (i)  Australia’s reputation for quality higher education and training services; and

                             (ii)  Australia’s international competitiveness in the higher education sector; and

                            (iii)  excellence, diversity and innovation in higher education in Australia; and

                     (d)  to encourage and promote a higher education system that is appropriate to meet Australia’s social and economic needs for a highly educated and skilled population; and

                     (e)  to protect students undertaking, or proposing to undertake, higher education in Australia by requiring the provision of quality higher education; and

                      (f)  to ensure students undertaking, or proposing to undertake, higher education, have access to information relating to higher education in Australia ...

The Tertiary Education Quality and Standards Agency (TEQSA) oversees this quality process. The Act mentions "quality assurance practice", but does not define what this is. The Act defines a quality framework as a series of standards decided by the Minister on the advice of a Higher Education Standards Panel. Some institutions can self-accredit their courses, whereas others are accredited by TEQSA.

TEQSA's "Higher Education Standards Framework (Threshold Standards) 2011"  is very general and non-prescriptive. For example:

The higher education provider has sufficient appropriately qualified personnel to manage and to provide academic leadership for the higher education provider’s higher education operations.

 The framework doesn't define "sufficient" or "appropriately qualified". This would vary greatly between disciplines and methods of teaching. Strictly speaking, such an input measure has no place in a quality framework, unless it can be shown to directly relate to the product produced. As an example, there is little research to show that the research record of university staff improves the quality of teaching. What does improve teaching is training teachers in how to teach.

Another example of confusing input with output is class size. One university might limit tutorial sizes to fifteen students, while another has classes of 100 students. Research shows that class size has no effect on student outcomes.

The framework is also prescriptive in referring to "locations", which assumes that students attend a class on a campus. This can be easily interpreted for an on-line course, where students should also be safe. But, it would be better if the framework did not have an inbuilt bias.

Developing Nations Formally Adopting Quality Standards

The vague references to quality in
TEQSA's documents contrast with the approach of the proposed by Basir (2012) for  Malaysian universities to use of the ISO 9000 Quality Management Standards (ISO 2008) . It is suggested this could lift the quality of Malaysian coursework programs to western standards. While that may seem ambitious, the Japanese car industry used similar standards to overtake western car makers within a few decades. By use of on-line technology, the same may be possible in education in less than a decade. However, the Malaysian Qualifications Agency does not apprise to have mandated this strict us of standards.

Quality and Higher Education in India

In "Building the Links Between Funding and Quality in Higher Education: India's Challenge"(Lindsay Daugherty, Trey Miller, Rafiq Dossani, Megan Clifford, RAND Corporation, 2013) describe quality measures introduced by the Indian government. The authors suggest suggest seven policy actions to link funding to quality.

Many of the measures proposed for India are already in place in Australia, such as a student financial aid system tied to accreditation. However, others are yet to be implemented in Australia (or USA), such as: "... a quantitative data system to measure quality of higher education institutions...".

Government Rating Universities Not a Good Idea

The US Government is reported to be planning to rate colleges and using this to apportion funding ("Obama’s Plan Aims to Lower Cost of College", by TAMAR LEWIN, New York Times, August 22, 2013). If government is funding colleges, then it is reasonable there is a government mandated minimum standard. But a government rating one college as better than another is more problematic. Apart from the problem of having a reliable system, it has to be asked what is the purpose of the government rating and if it will be effective.

Australia has the Tertiary Education Quality and Standards Agency (TEQSA), which is a government agency, but relatively free of political interference, with academics setting the standards.

The Australian Government provides the "MyUniversity" website with details of all Australian institutions. This details the qualifications of the staff and how many have awards for teaching, which is relatively uncontroversial.

More at issue are student survey results for each subject area. I have recently undertaken tertiary teacher training and so am comfortable with the idea of being rated by the students and have designed a course which rates highly. But some of my colleagues have difficulty with the concept and practice of designing courses which meet external standards and are popular with students.

There is the risk of a race to the bottom, with courses which just meet minimum standards and are designed to be easy and therefore popular. But I find that students value a course which challenges them. Also while meeting external standards is a useful discipline for the course designer.

Problems with US Higher Education

The US Government plans to rate colleges are reported to be related to a number of perceived problems with US higher education: low completion rates of students, and defaults on student loans. Some solutions proposed are already in routine use in Australia, such as competency-based assessment, used in the Australian vocational system, where the emphasis is on demonstrating the required skills, not the length of study. The Australian system of student loans is tied to the income of the student, lessening the hardship for the student (but not reducing the need for policy to encourage vocationally relevant courses).

While the New York Times article mentions MOOCs as a way to  reduce costs. However, someone has to pay for the development of MOOCs. Without correcting flaws in the US education policy, MOOCs will accentuate problems. It is likely there will be new startup educational institutions offering MOOC based programs of questionable value, temping established institutions to follow the same path to compete. This is likely to lead to a collapse of institutions, much as happened at the turn of the last century with the DOT.Com crash of companies offering free web products with no underlying business case.