Report Finds Most Satisfied with Education Provided Despite Pandemic Restrictions

"Experience and views on education during the COVID-19 pandemic"  (Biddle, Edwards and Gray) reports the results of a November 2020 poll with 3,029 respondents in Australia, of whom 671 were an adult student in secondary school, further education, or  university. This found 88% were very or somewhat satisfied with how their their child’s educational institution handled teaching during COVID-19. 85% were very or somewhat satisfied with their own adult education.

Abstract

"COVID-19 has resulted in disruptions to schooling for the vast majority of Australian school children. Universities and other post-secondary education providers have also seen widespread shifts to remote learning, and considerable impacts on school funding. While there have undoubtedly been negative impacts of the COVID-19 pandemic on education institutions, students and their families, the crisis has at the same time created an opportunity to reflect on the role of education in a society like Australia’s. 

In this paper we provide a summary of survey data on the experiences of students and their families during the pandemic, as well as attitudes of the entire Australian population to the role of schools and universities. We found 47.8 per cent, or almost one-in-two Australians were very satisfied with their child’s educational institution, while 40.2 per cent were somewhat satisfied. Only a small percentage of the population were not satisfied with their child’s education, with a slightly higher per cent of adult learners not being satisfied with their own education. Despite this high level of satisfaction, the paper also shows that a large number parents or adult learners were concerned about their own learning or their children’s learning."

Top Five Factors Associated with Student Success

The top five factors associated with student success identified by meta-meta-analysis by Schneider and Preckel (2017), were: 1. peer assessment, 2. self efficacy, 3. teacher preparation, 4. teacher clarity, 5. grade goals. The first of is reasonably clear, if difficult to implement. Having students peer asses is not difficult to set up, particularly if you have a Learning management System, such as Moodle, to look after the administrative details. However, it can be difficult get some students, and many staff, to accept that students can provide quality assessment.

Self efficacy is easy to recommend, but how do you build students’ belief in their ability? The obvious ways are to state clearly what you want them to do, and give them steps along the way (scaffolding). Teacher preparation and clarity seem obvious, but I see many cases where teachers have produced overly complex lessons, and then compound the problem with long complex explanations of the lessons.

I don't quite understand the point on grade goals. Presumably students are attempting to pass their courses, or where some higher level of achievement is needed, to reach that level. What does worry me is where students have set unnecessarily high goals. As an example, some students will ask for a regrade on the basis they are aiming to get a high distinction (80%) for all assessment items. There are ways to counter this with the assessment scheme, for example having just a pass/fail grade for some tasks.

Reference

Schneider, M., & Preckel, F. (2017). Variables associated with achievement in higher education: A systematic review of meta-analyses. Psychological bulletin, 143(6), 565. URL https://www.uni-trier.de/fileadmin/fb1/prof/PSY/PAE/Team/Schneider/SchneiderPreckel2017.pdf

Australian Employers Are Satified With Graduates

The "2018 Employer Satisfaction Survey" (QILT, 10 January 2019), indicates employers are very happy with the graduates they get. The Quality Indicators for Learning and Teaching (QILT) team, was funded by the Australia Government. An analysis of 5,300 survey results from graduate supervisors showed 84.8% were satisfied with their graduates. The least satisfied were in creative arts. It may be that the more vocationally aligned STEM degrees are a closer match to jobs.

The Group of Eight "Leading Universities" did not do so well, with only half of them scoring above average for employer satisfaction. There is no clear relationship between university size, focus or location, and employer satisfaction. Perhaps it is just that Australia has high standards, and we don't have any "bad" universities. This is good news for employers, but does it indicate Australia is over investing in university education?

ps: Bond University topped the 2018 Employer Satisfaction Survey, so it should be worth listening to their Professor Keitha Dunstan, Deputy Vice- Chancellor (Academic), 26th February, 2019. The event is in Canberra, although Blackboard's web page confusingly says "Sydney".

What Makes for a Good Student Experience?

The Australian 2017 Student Experience Survey (QILT, 2018), shows three non-government universities with much higher undergraduate student satisfaction than other institutions. It would be easy to conclude that private institutions do better, but is that the case? It may be what they teach and who they teach, as much as how.

The good news is that while some Australian universities rate better than others on student satisfaction, they all rate well. Universities are tending to move more of the teaching online. This provides students with better access to learning, but may result in overall lower student satisfaction results.

Most Australian universities scored between 70% to just above 80% on educational experience for undergraduate students in 2017. Three exceptions were Bond University, The University of Notre Dame Australia, and the University of Divinity, all over 90%. However, not all non-government universities did well. Torrens University Australia, the only private for-profit university in Australia, had a score of only 77.6%, below many government universities.

Torrens is essentially an online university and these tend to rate poorly on student satisfaction. The University of Southern Queensland, which also has many online students also rated poorly at 73.6% (I have completed two online courses at USQ and thought they were good).

Absent from the QILT report is Open Universities Australia (OUA), a consortium of universities offering online courses. OUA would be likely to have a low student satisfaction score, similar to USQ and Torrens. It would be easy to conclude that these institutions are doing something wrong. But they accept students who would otherwise not be able to study at all. Given a choice, I am sure these students are more satisfied being able to study, than not.

The QILT report mention of online study in the Executive summary (page iv):

"The largest variation was that external/distance students were less likely to respond positively about their Learner Engagement, 22 per cent and 63 per cent respectively. Older students also rated Learner Engagement less positively than younger students, but this difference is most likely associated with the prevalence of external or internal study modes in these age groups.

Large differences in results by study mode for Learner Engagement continues to suggest that this scale may be performing differently for internal/mixed mode students and external mode students. The Department of Education and Training is currently considering a review
of the Learner Engagement scale. As an interim measure, the QILT website, which reports SES results at the institution by study area level, will continue to exclude external mode responses for the Learner Engagement focus area. This report, however, which reports SES results at national and aggregate levels, includes external mode responses in all Learner Engagement results. ..."

Also section 3.3 "The postgraduate coursework student experience by study area":

"The widest range in focus area results was for Learner Engagement, with 34 percentage points separating the study areas with the highest and lowest results, Rehabilitation at 72 per cent, and Nursing at 38 per cent, which may be associated with the relative proportion of online or distance learning associated with the various study areas."

Extend Teaching Skills for IT Profession

The SFIA Foundation has invited suggestions for enhancements to the Skills Framework for the Information Age (SFIA). I suggest defining learning and teaching skills at levels 3 and perhaps also at level 2.

As part of my Master of Education studies I did a quick analysis of education skills for IT professionals. SFIA was very useful for this, providing a concise set of IT teaching skills:

SFIA Category Skills and ACS Membership Level

Subcategory	Skill	Code	Levels
* SFIA Version 6 definitions from ACS (2015)
Skill management	Learning and development management	ETMG	--34567
	Learning assessment and evaluation	LEDA	--3456-
	Learning design and development	TMCR	---456-
	Learning delivery	ETDL	--3456-
	Teaching and subject formation	TEAC	----56-
People management	Professional development	PDSV	---456-

However, some skills are only defined at higher levels.  ETMG, LEDA and ETDL start at level 3, but TMCR and PDSV start at 4 and TEAC at 5.

The entry level vocational qualification for the IT profession in Australia is "Diploma" in the Australian Qualifications Framework (AQF Level 5). The Australian Computer Society has identified this as being SFIA Level 3. It would therefore be useful to have the SFIA learning/teaching skills defined at SFIA Level 3.

The entry level qualification for those conducting training in Australia is a Certificate IV in Training and Assessment at AQF Level 4. So it may be useful to also have the training skills defined at SFIA Level 2.

It should be noted that the qualifications required for those teaching in Australia are inversely related to the level at which they teach: the higher the level of the teaching, the lower the qualification required. School teachers are required to have a degree (soon to be masters), vocational teachers only a certificate and university lecturing requires no teaching qualification at all.

ps: The SFIA Foundation are one of the more responsive standards setting bodies. In 2008 I was commissioned by the ACS to design a course in Green IT. There were no SFIA skills defined for this at the time, so I sent my draft course to the SFIA Foundation. Skills in sustainability were subsequently added to SFIA and matched my course syllabus reasonably well. I later changed the course descriptions to reference SFIA. The course is currently offered by Athabasca University (Canada).

Quality Indicators for Learning and Teaching in Australian Higher Education

The Australian Government funds the QILT Quality Indicators for Learning and Teaching website. One point to note is if your browser has an ad-blocker, such as AdBlockPlus you may not see any institutions listed (I didn't until I disabled the ad-blocker for the QILT site). QILT lists 55 institutions offering Teacher Education (including Vocational Education & Training, and Higher Education and Special Education). There are 44 institutions offering programs at the postgraduate level (39 Universities and 9 Non-Universities). 

One limitation with the QILT site is that there is no option to select institutions offering on-line courses. The assumption seems to be that the potential student will select a state and then look at institutions in that state.

The site requires the student to select no more than six institutions to compare. There is no way, for example, to list all the institutions in Australia, by quality ranking and then select the top six. The student would have to look at each institution and manually note and sort the top six.

There are  5 ACT Universities with Postgraduate courses in Teacher Education listed. So I selected all five, plus Montessori Institute (a non-university postgraduate provider, which, not surprisingly trains Montessori teachers). Only four institutions were shown, due to too small a number of students for the with two. The four institutions were around 80%, on "Overall satisfaction" and with confidence intervals so wide that this which not be of use in selecting between institutions. On the "Teaching scale"
University of Canberra was 10% above the others.  On the "Skills scale" University of New South Wales was slightly ahead.

What is not clear is how useful this information is for students selecting programs. The differences between institutions are minimal. A better approach might be to put resources into ensuring all institutions meet an acceptable minimum standard, so a student knows that if they enroll locally they will receive an acceptable quality education.

Also with the transition to e-learning the quality of legacy classroom based courses will be of little relevance.Students are increasingly likely to follow the approach I took in looking for a program in 2013. First I looked in the city where I lived. After finding nothing suitable in my city, I think looked internationally. Once beyond commuting distance, it makes little difference if the institution is 1,000 km or 10,000 km away. In the last five years I have been a student of ANU and CIT (within 10 km of home), USQ (1,000 km) and Athabasca University (14,000 km).

m-Learning in Sydney

The 15th World Conference on Mobile and Contextual Learning (mLearn 2016), was held at the University of Technology in Sydney, 24 to 26 October. There were five parallel sessions in the MLearn2016 Program. The mLearn 2016 Proceedings are available as a PDF file. Here is a summary of my notes from the event:

UTS Bachelor of Technology and Innovation

Professor Shirley Alexander, UTS Deputy Vice-Chancellor, welcomed the delegates and talked about the UTS Bachelor of Technology and Innovation (BTI), being introduced next year. This is a general degree in technology, suitable for those going into business. I am skeptical of the idea of a BTI as the 21st Century replacement for the arts degree. BIT graduates may end up like arts graduates: well educated, but not qualified for any real job.

Mobile Learning and Indigenous People

Marguerite Koole Assistant Professor, Curriculum Studies at University of Saskatchewan described applying a frame model of learning (Koole & Ally, 2006).

Kevinwâsakâyâsiw Lewis, then talked about the teaching of teachers of indigenous languages in the University of Saskatchewan's Certificate in Indigenous Languages. Asked about the relevance of song in language learning, Kevin pointed out that chant and song enables the learner to stop worrying about how they pronounce. Also the repetition in the song helps learning, as well as being culturally significant. This reminded me of Dr McComas Taylor, at ANU's Teaching Sanskrit online with chanting.

mLearning for Aboriginal Pre-service Teachers in Remote Communities

Philip Townsend talked on "A Theory of Enhancement of Professional Learning for Aboriginal and Torres Strait Islander Pre-service Teachers in Very Remote Communities through Mobile Learning". He pointed out that Aboriginal and Torres Strait Islanders have a much lower completion rate than the general population. The Community Based ITE program, specifically for students in remote areas, has an even lower completion rate than other forms of education. Philip has devoted his PHD research to finding if mobile devices can improve completion rates. Also he mentioned the Cooperative Research Centre for Remote Economic Participation. I suggest that addressing the needs of remote students can assist all students. If we make courses flexible enough for the remote students, these courses will also be better for city and campus based students. Philip was awarded top paper at the conference.

Mobiles for TEQSA Regulatory Course Compliance

Anthony Chung from mobileLearning gave one of the more practical and less academic (in a good way) sessions on "How does a Mobile Platform Address TEQSA and Other Regulatory Compliance for Online Courses?". Anthony went through some of the TEQSA requirements and how these could be facilitated with mobile devices. What was interesting about mobileLearning's approach is that they provide  mobile interface to the institution's native educational applications, rather than replace them. The idea is to make it easier for students and staff to do what is educationally useful. One example is to encourage the student to engage with the learning. This can be difficult where a Learning Management System (LMS) design for static desktop pages is used. The mobile interface can make it easier to access the materials and interact.

IamLearn AGM

The AGM of the International Association for Mobile Learning (IamLearn), was held in conjunction with the conference. Because I registered for the conference I received a year's membership. One initiative is that a new website will allow members to add content directly, rather than emailed to an officeholder who then puts it up. This is something other organizations might like to try

Telstra on Personalised mEducation

 Susi Steigler-Peters, from Telstra Corporation talked on "Personalised Learning, mEducation and Partnerships". Susi spent most of the time describing research on education Telstra has funded over the last few years. This research sounds well designed, but does not tell us, as educators, anything new. So I asked Susi how Telstra could help improve education. In response she cited something called "Telstra Smart Learning" and presented a use case of a student named "Ferris" having an "enriched learning experience". Susi also mentioned a center to be established in Sydney for education. It might be useful for Telstra to instead showcase some of the work in their muru-D startup accelerator.

Online Tutorials for Higher Education

Norman Wildberger and Joshua Capel presented an excellent live demonstration of "Higher Education Practice Online Tutorials and GeoGebra as Mobile Learning Tools" used for teaching mathematics at UNSW. However,what seemed to be missing was a business model to support the high up-front investment to produce such material.

UNSW mLearning Physics

Elizabeth Angstmann talked on "Higher Education Practice Using a mobile Moodle app in an online physics course". This shows an approach to education which focuses on ensuring that e-quality education can provide quality education, backed up by solid research.

Identity and the Mobile Learner

Professor John Traxler, University of Wolverhampton gave the last keynote on "The Role of Education in Identity Transformation and Acculturation". Professor Traxler challenged the assumption that m-learning (and education in general) was a culturally neutral boon for developing nations. Rather than telling us a canned answer, this was a presentation asking questions, which was refreshing.

Professor Traxler is concerned about a Western European view of education and technology being culturally imposed on others. However, this might also be a product of that culture.

Online Education for Prisoners

Louise Patching, USQ, talked on "Using Offline Personal Devices to Enable Access to Higher Education in Prisons" for the Making the Connection Project. Louise pointed out that lack of Internet access is only one factor limiting e-learning in prisons; also there is limited access to hardware, limited prior education of the students. She explained that USQ had decades of experience in teaching prisoners. USQ has a special server for providing materials to prisons and a way for materials to be loaded to portable devices.

Some of what USQ has done would be useful, more generally for students. For example, a problem with webinars is where students have inadvertent left their microphones turned on. I suggest that the off-line features could be built into Moodle and the Linux operating system, allowing the approach taken by USQ to be made available world-wide at no cost.

Encouraging Faculty Development

 Lisa O’Neill, York University, talked on "Encouraging Faculty Development Through Micro-Credentialing". This presented a good overview of the mechanics of recognizing small units of learning with "badges". However, what is also needed is a way to motivate university academics to learn to teach and undertake activities which award such badges.

I suggest that university educators should follow the approach used by nurses and computer professionals in Australia to enhance their recognition of their profession. This could include draft policy for universities and government to require and recognize teaching qualifications of university academics. Also they could suggest changes to university ranking schemes to give teaching quality the same weighting as research output.

References

Koole, M., & Ally, M. (2006, April). Framework for the rational analysis of mobile education (FRAME) model: Revising the ABCs of educational practices. In International Conference on Networking, International Conference on Systems and International Conference on Mobile Communications and Learning Technologies (ICNICONSMCL'06) (pp. 216-216). IEEE. Retrieved from http://dx.doi.org/10.1109/ICNICONSMCL.2006.103

Townsend, P., Halsey, J. R., & Guenther, J. (2016). Mobile Learning Congruencies with Aboriginal and Torres Strait Islander Cultural Philosophies. In Publishing Higher Degree Research (pp. 25-32). SensePublishers. Retrieved from http://link.springer.com/chapter/10.1007/978-94-6300-672-9_3 

Encouraging Faculty Development

Yesterday at mLearn 2016 in Sydney, Lisa O’Neill, York University, talked on "Encouraging Faculty Development Through Micro-Credentialing". This presented a good overview of the mechanics of recognizing small units of learning with "badges". However, what is also needed is a way to motivate university academics to learn to teach and undertake activities which award such badges.

I suggest that university educators should follow the approach used by nurses and computer professionals in Australia to enhance their recognition of their profession. This could include draft policy for universities and government to require and recognize teaching qualifications of university academics. Also they could suggest changes to university ranking schemes to give teaching quality the same weighting as research output.

Australian universities commonly have a 40/40/20 expectation of academics: teaching 40%, research 40%, and engagement/administration 20%. So it would be reasonable to have teaching weighted equally with research, when ranking universities and university academics.

UK Teaching Excellence Framework

The UK Government has proposed a Teaching Excellence Framework (TEF) for the higher education sector. There is a consultation "green" paper" called "Fulfilling our potential: teaching excellence, social mobility and student choice" (Cm 9141 PDF, 2.15MB, 105 pages, 6 November 2015). Comments close 15 January 2016. Unfortunately I could not find anything about teaching in the document, or about ways to improve teaching. It appears to be about setting up a government agency to publish unspecified ratings of universities, which would have little to do with teaching. The UK government and UK universities seem to want to avoid the obvious first step in improving the quality of teaching at universities: ensure that those teaching are educated and qualified to do so. Before implementing complex schemes for incentives for excellent teaching, government and universities could simply ensure the teachers are trained, at least to the level of school teachers.

The proposal aims to:

• "drive up teaching standards and give students more information through a new Teaching Excellence Framework that will encourage a greater focus on high quality teaching and graduate employment prospects
• widen participation for students from disadvantaged backgrounds and encourage providers to increase focus on supporting all students through their course and into employment or further study. A new Social Mobility Advisory Group would report to the Universities Minister with a plan to meet the Prime Minister’s ambitions to increase the proportion of disadvantaged students entering higher education and increase the number of BME students by 20% by 2020
• enable students to choose from a wider range of high-quality higher education providers by making it less bureaucratic to establish a new university through faster access to Degree Awarding Powers and University Title
• establish a new Office for Students to promote the student interest and value for money, and reduce the regulatory burden on the sector ...

Through the Teaching Excellence Framework (TEF) there will be stronger incentives for excellent teaching and students will have more information about the type of teaching they can expect and their likely career paths after graduation. The TEF will use measures such as student satisfaction, student retention rates and graduate job prospects. Higher education institutions providing high quality teaching would be able to increase tuition fees in line with inflation. Those that fail to meet expectations would risk losing additional fee income."

From: Student choice at the heart of new higher education reforms, Press release, Jo Johnson MP, UK Department for Business, Innovation & Skills, 6 November 2015

Rate University Teaching Quality On-line

In "The hunt for Australia's best teaching uni" (The Age, 9 March 2015), Gary Newman asks how to rank the quality of teaching at universities. Using the measures which have been used he lists Australia's top universities for teaching as: Victoria, Griffith, Deakin, Central Queensland, Tasmania, Edith Cowan, Canberra, Federation, Bond, Notre Dame, Sunshine Coast, New England and Western Sydney. That seems a reasonable list, with the regional universities not known for research having to focus on teaching. Also many of these institutions started as technical and teachers colleges devoted to education and now have an emphasis on e-learning (which requires a more professional approach to teaching).

As he points out the current ranking systems are based largely on research quality which is not necessarily related to the quality of teaching. He says a 2013 study provides some data (this appears to be "Factors Associated with Job Satisfaction Amongst Australian University Academics and Future Workforce Implications" by Peter Bentley, Hamish Coates, Ian Dobson, Leo Goedegebuure and Lynn Meek).

One interesting variation in rankings is the Ranking Web of Universities (Webometrics Ranking), which bases it assessment on the university's web presence. This produces a result similar to other measures but is easier to compile. Perhaps that could be applied to provide a measure of teaching quality, not just for Australian institutions, but world wide.

This, of course assumes that students are interested in teaching quality. It may be that students select a university based on its research reputation, as that is how the quality of their qualification will be seen.