Thursday, June 28, 2018

Learning to Teach and Assess at the ANU Early Career Academic Forum

The Australian National University is holding an "Early Career Academic Forum", 24 to 25 July 2018. This is run by the ANU Network of Early Career Teachers, Academics and Researchers ("NECTAR"). This is aimed at new staff starting their career and Phd students considering a career. I will be facilitating a discussion of Improving Teaching and Learning at ANU.
"Program Highlights:
Tuesday, 24 July
  • Workshop: Indigenous perspective inclusive/sensitive pedagogy
  • World Cafe style Morning Tea with discussions around Learning to Teach and Assess, PDRs and promotions, challenges faced by Early Career Academics and many more.
  • Roundtable with Marnie Hughes-Warrington, DVC (Academic), and Grady Venville, PVC (Education).
  • Workshop: ANU Teaching and Learning Vision with Grady Venville
  • Research Presentation on ECA Employability
  • Panel: Job Security. Confirmed panel members: Sue Thomas (CEO, ARC), Mike Calford (Provost, ANU), Denise Ferris (Head, School of Art & Design) and Lyndall Strazdins (National Centre for Epidemiology and Population Health, ANU).
Wednesday, 25 July:
  • Panel: Overturning the obstacles of sexual misconduct, harassment, hostility and practices of inequity within the university. Confirmed panel members: Dean of Staff, Alyssa Shaw (PARSA President), Renee Hamilton (Universities Australia) and Jamiyl Mosley (Head of Burton and Garran Hall).
  • Roundtable with VC Brian Schmidt.
  • AGM
  • Workshop: “What to do when things go wrong” with Gail Frank (Adviser to staff, ANU)."
From: Early Career Academic Forum, NECTAR, 2018
ps: An interesting future option for ANU is microcredentialing, as mentioned by the VC at a Senate inquiry recently. 

Wednesday, June 27, 2018

Review of Australian Education Qualifications Framework

The Australian government today announced a review Australia’s education qualifications framework. This will be headed by Professor Peter Noonan, Victoria University. A Contextual Research for the Australian Qualifications Framework Review paper has been prepared by PhillipsKPA. The paper compares the AQF with other nations qualifications frameworks. One positive finding was more employer engagement in accreditation, than in Europe. A point against the AQF is regulation of the use of Recognition of Prior Learning (RPL) and micro-credentials. The authors suggest removing administrative detail from the AQF where this is covered by regulation and legislation elsewhere.

Less usefully, the authors suggest moving to "credit points" as a measure of the volume of learning rather than years of study. Apart from helping international comparisons, this would make no practical difference, as the notional years can easily be converted to notional hours.

What is ResearchOps?

Greetings from the workshop on Research Ops Hosted by Ruth Ellison at the Digital Transformation Agency in Canberra (there are other locations around the world). This is about refining the design of user interfaces for web based applications, based on user input. This overlaps with existing user interface, systems analysis and industrial designs disciplines.

Students taking part in a  Lego Serious Play exercise at the Australian National University in Canberra
One of the issues discussed was how to get a multidisciplinary team working. I mentioned a Lego Serious Play session run by Dr Stephen Dann for the ANU TechLauhcher program. In this the new student team members used Lego to help draw out issues about what they saw as their role and what their project was about. This could be useful for ResearchOps teams.

Tuesday, June 26, 2018

ANU Social Enterprise Bootcamp

Greetings from the ANU Social Enterprise Bootcamp, at the Australian National University (ANU) in Canberra. This is hosted by the ANU Technology Transfer Office. The workshop is being run by Cindy Mitchell, Social Impact Strategist at the University of Canberra and CEO of the Mill House Social Enterprise Accelerator. Cindy has adapted the tools usually used to start-up a new for-profit business to the not-for-profit sector.

There do not appear to be many formal qualifications in this area. Macquarie University has a Master of Social Entrepreneurship and LSE an MSc in Social Innovation and Entrepreneurship.

We had to each complete a "Design for Impact for Scale Canvas".
For the purpose of the exercise I looked at how we might help early career academics and professionals  teach part time at university. The idea would be to have them see themselves as professional educators. They would be offered ways to become qualified in eduction without having to attend class or pay a lot of money. The emphasis would be on what saves time and frustration and gets them contracts, not education theory. This is a path I have been on over the last few years,but I would hope to make it a bit easier.

Monday, June 25, 2018

ANU Calls for Report on the Disappearance of Prime Minister

No,  the current Prime Minister isn't missing. The Australian National University's central library suffered a flood on in February 2018. ANU has launched a flood appeal, asking not only for money, but have listed the books they need. There are thousands of books on the list and make fascinating reading. One which caught my eye was the Report by the Commonwealth and Victoria police on the disappearance of the Prime Minister the Right Honourable Harold Holt, Cheviot Beach, Portsea, Victoria, Sunday 17 December 1967.

Sunday, June 24, 2018

Online Language Learning

The Australian National University is now offering online language courses through Open Universities Australia, with , , , , and available. These are full fee, for-credit university courses. This is supported by the ANU Digital Education Services (DES) team, lead by Grazia Scotellaro.

Free online language ebooks are also available from ANU Press:

The Joy of Sanskrit: A first-year syllabus for tertiary students by McComas Taylorand Grazia Scotellaro

Modern Japanese OnlineModern Japanese Online:  The first course to mastering modern Japanese, by Naomi Ogi and Duck-Young Lee

Saturday, June 23, 2018

Online Learning is Just Learning

Vallance and Wilson-Keates have taken exception to Clow, and Kolomitro's characterization of online education. But I think they have all missed the point: online education is not very much different to campus based, especially now most university courses make use of the Internet for at least part of the delivery. It is a little like arguing an electric car is better than internal combustion, while the industry is moving to hybrids, which combine the two forms of power. There are very few students studying purely online or on-campus, most mix both formats. The educational techniques needed for each are not as different as these authors suggest.

Clow, and Kolomitro give online learning faint praise, describing its benefits "in theory": inclusivity,  reach, pace and flexibility.  Then then present the case against, with limited student-student-faculty-institution interaction, isolation and lack of community. They describe a "one-size-fits-all" online education model. Clow, and Kolomitro present a picture of an in-person classroom with collaboration, engagement.

Strangely, Vallance & Wilson-Keates start their rebuttal of the attack on online learning by disagreeing with the one thing Clow, and Kolomitro said in its favor: that it provides for those otherwise excluded from higher education. Vallance & Wilson-Keates go further to specifically claim that Athabasca University (AU) students don't enroll because they were excluded from conventional institutions. This is a odd claim to make, given that AU says on its web page "we are Canada's Open University" (an open university being one without the entry requirements of other institutions_. I know at least one AU student who felt excluded from other universities: me. It is a little odd that after making me feel so welcome, AU would want to deny this.

Vallance & Wilson-Keates provide no evidence for the claim that "The online educational context in the Canadian landscape is no longer regarded as an inferior experience...". Having read research on online education as part of my MEd in Distance Education, I consider it is very much still the case that online education is seen as a second best option, in Canada, and throughout the world. The research shows that online graduates are every bit as good as campus based ones and the students are happy with the experience, but online is still seen as second best.

Vallance & Wilson-Keates claim that interaction in an online course is up to the instructor. However, this denies that the nature of the possible interaction is limited by the technology. As an online student I did feel isolation at times. However, I also felt isolation as an on-campus student, and much more isolation when a part-time night-course student.

We need to stop this "how many angels could dance on the head of a pin" debate over online versus classroom education. The question is not if online is better than face-to-face, but what mix of the two provides the most cost effective, quality education. The students have already voted with their feet and are leaving conventional lecture theaters for on-line learning. But they will still come to class, provided they are given quality, affordable, convenient experience. I know this because I studied this topic, as a graduate student, in a classroom in Canberra, and online at Athabasca, while teaching in a classroom in Canberra and online  (also I designed one of AU's courses).


Vallance, Jeff & Wilson-Keates, Barbara.  (, 18 June). Online education in Canada provides learners with a flexible, inclusive and quality educational experience. Athabasca University. URL

Clow, Erin & Kolomitro, Klodiana. (2018, 2 May). Online learning isn’t as inclusive as you may think. University Affairs. URL

Report on Australian Vocational Education and Training Regulation

The Australian Government today released the review of  Vocational Education and Training (VET) regulation, by Professor Valerie Braithwaite (2018). Professor Braithwaite made 23 recommendations the to government on 31 January 2018. The government has announced it will progress 10 recommendations, and "support in-principle" another 11. However, is not clear what the government doesn't support, or why it has taken almost five months to release the report.

The report provides useful recommendations (collected below) for improving the legislation under which VET operates. However, what is needed is a broader review of vocational education, as an industry and as a service to the community. Also the relationship between VET and the university sector. There are calls for universities to provide more job ready skills for graduates and more responsive to industry needs, which up until now has been the role of VET.

Professor Braithwaite begins by pointing out what has been achieved with VET regulation in Australia, with national regulation introduced in 2011. A patchwork of state oversight was replaced by the Australian Skills Quality Authority (ASQA). No major deficits were found preventing  ASQA from regulating VET, However, the report recommends "... a higher bar for gaining and maintaining registration as a VET provider". 
The report notes that 81.4% of VET students are part time, with business services the most popular program. Students may be at a large TAFE campus, small RTO, or online. Unfortunately the report does not detail what proportion are at each. While there have been concerns over VET quality, the report notes in 2017 that 87% of students were satisfied with their provider.
"Recommendation 1: ASQA develop and implement processes to enhance its capabilities and opportunities to proactively engage in regulatory conversations with students, teachers, RTOs, industry and other interested stakeholders. The desired outcomes are to improve the value of the student-focused regulatory approach and involve the sector in developing the regulatory culture that drives ASQA’s use of its legislative powers.

Recommendation 2: In order to enhance transparency and consistency in the use of the legislative framework, ASQA should build on its regulatory conversations and practice reflections to develop and clearly articulate to the regulatory community the principles applied to the interpretation of legislation and the use of powers.

Recommendation 3: ASQA works with RTOs to develop positive assurance flags to include in the ASQA risk matrix and develop a mutually agreed method of communicating this information publicly without increasing the compliance burden on RTOs.

Recommendation 4: The Australian Government amends the legislative framework to ensure that entrants to the registered training market be required to clearly demonstrate educational commitment and knowledge of how to provide best practice support to students. This statement of commitment should be required as a condition of registration and include quality performance objectives, which, if breached, could lead to sanctions and ultimately de‑registration.

Recommendation 5: The Australian Government strengthens the fit and proper person requirements and change notification requirements under the NVETR legislation and where appropriate aligns them with TEQSA and ESOS Act provisions and any other relevant legislation.

Recommendation 6: The Australian Government amends the legislative framework to ensure greater scrutiny of new providers to:
  • provide that where an RTO without reasonable justification does not commence providing training within 12 months of being registered, or during its registration ceases to provide training for a 12-month period, its registration automatically lapses, meaning that it would no longer be registered.
  • prevent RTOs changing the scope of the courses they deliver where an RTO has been operating for less than 12 months.

Recommendation 7: The legislative framework be revised to require an RTO to assess the quality of its teaching workforce and develop teacher quality improvement actions, which must be submitted to ASQA annually as a part of the Quality Indicator Annual Summary report.

Recommendation 8: The Training and Education Training Package be reviewed with the purpose of creating a career path for teaching excellence in vocational education and training.

Recommendation 9: The Australian Government leads a process to raise the standards of teaching and training excellence and professionalism in the sector through creation of the role of Master Assessor. A Master Assessor would be placed at the pinnacle of the VET teacher/trainer career path with the responsibility to mentor through professional development programs and assess the quality of an RTO’s next cohort of graduating students.

Recommendation 10: The legislative framework be amended to increase the frequency of data provision to the National Centre for Vocational Education Research to quarterly for all RTOs.

Recommendation 11: The Australian Government prioritises the improvement of policies and systems that allow for transfer of real-time data for timely use by other agencies with regulatory responsibilities for identifying and responding to emerging sectoral and provider‑based issues.

Recommendation 12:
  1. The Australian Government and the National Centre for Vocational Education Research explore ways to increase student response rates to the Student Outcomes Survey, and
  2. The National Centre for Vocational Education Research, ASQA, and the sector identify a module of questions that directly addresses the quality of the student journey in the Student Outcomes Survey.

Recommendation 13: The legislative framework be amended to enable the National Centre for Vocational Education Research to make the RTO level data it holds publicly available and identifiable.

Recommendation 14: The Australian Government explores ways to strengthen the regulatory framework by expanding the circle of dialogue around improving the quality of the student journey pre- and post-audit to include all stakeholders who could contribute to future improvement in an RTO’s performance.

Recommendation 15: The National Vocational Education and Training Regulator Act 2011 be amended to require ASQA to publicly release audit reports.

Recommendation 16: The legislative framework be amended to require RTOs to publish nationally consistent consumer information that is accessible and meaningful to students and meets the basic needs for decision making (for example, course entry requirements, course length, employment outcomes, and fees, including subsidies and course cancellation fees).

Recommendation 17: The legislative framework be amended to strengthen ASQA’s ability to take action under a general prohibition against misleading or deceptive conduct which reflects Australian Consumer Law requirements.

Recommendation 18: The legislative framework be amended to require RTOs to strengthen consumer protection in student enrolment agreements through the adoption of contracts that avoid unfair terms as defined in Australian Consumer Law.

Recommendation 19: The legislative framework be amended to require RTOs to keep electronic records showing a minimum of student completions of units, courses and qualifications over the life of the RTO, preferably using an AVETMISS-compliant student management system.

Recommendation 20: The Australian Government investigates ways in which, in cases of administration and liquidation, priority is given to the timely provision of student records to ASQA and the protection of students’ investment in their education.

Recommendation 21: The legislative framework be amended to explicitly address student safety and wellbeing in alignment with the Higher Education Standards Framework (Threshold Standards) 2015.

Recommendation 22: The Australian Government considers strengthening tuition assurance by assuming responsibility for the operation of all tuition assurance and protection arrangements and ensuring that the scope of these arrangements protects all VET students.

Recommendation 23: The Australian Government establishes a national Tertiary Sector Ombudsman."
From Braithwaite, 2018


Valerie Braithwaite, All eyes on quality: Review of the NationalVocational Education and Training Regulator Act 2011 report, (Canberra: Australian Government, 2018). URL

Thursday, June 21, 2018

AltEd: Australian Non-Accredited Education Industry Group

Six non-government training companies have formed AltEd, an Australian Non-Accredited Education Industry Group, aiming to be leading providers of quality digital skills education. Australia has a system where both government TAFEs, private not-for-profit and for profit Vocational Educaiton and Training (VET) organizations are accredited under a unified national system. However, AltEd's focus is on training outside this government regulated system.

There have been recent instances of misuse of the regulated system by accredited for-profit providers and questions over the quality of some government providers. The subsequent tightening of regulation has brought complaints from providers that this makes it difficult to provide education.

Rather than seeking a change in the regulations, AltEd is arguing that not being accredited provides more flexibility, and so better education. This is a bold move as non-accredited courses will not be eligible for subsidized government student loans or inclusion in government subsidized international education marketing.

AltEd's founding members are MasterlyPlato Project, General Assembly, QLC, Zambesi, and School of Design Thinking. Student loans are available for General Assembly and Plato Project, trough the company Study Loans. 

It should be noted there is no requirement for training institutions to be government accredited. However, it is not clear to me how not being accredited will, of itself, improve the quality of vocational courses. Admittedly, becoming accredited involves complex and time consuming processes. However, if an educational institution already has a systematic process for designing learning and assessment, with quality control and evaluation, then the additional regulatory burden should not be too high.

Wednesday, June 20, 2018

Conference on Future Academic Workforce

The University of Technology Sydney is hosting a free one day conference on "The Future of Academic Work", 5 December 2018. This will report on research on how successful the role of "Scholarly Teaching Fellow (STF) has been at Australian universities. A quick search of LinkedIn showed fewer than 100 people with this job description. These were all at five Australian universities: Monash (11), Macquarie (8), La Trobe (6), Sydney (15), and Technology Sydney (15).
"In the changed university what is the role of academics? This one-day conference seeks to initiate a debate about the changing nature of academic work in universities, and beyond. ... the new Scholarly Teaching Fellow (STF) role was introduced into Australian universities in 2013. The positions were aimed at creating a more stable teaching workforce, while also addressing growing concerns about the injustices of academic casualisation. The STF positions aim to offer a career path for casual academics, and have had an important impact on the sector-wide debate about the relationship between teaching, scholarship and research. ... 100 in-depth interviews conducted across several universities with STFs, managers and stakeholders will be presented and debated at the conference. This research reflects on the implementation and experience of the new STF positions, and the opportunities and challenges of changing academic work. The resulting report will be pre-circulated; conference workshops will debate aspects of the report; keynotes will address wider conceptual and policy challenges."

From "The Future of Academic Work: a Deliberative Conference", UTS, 2018.

Monday, June 18, 2018

Conferences, in Sri Lanka, Malaysia or Singapore September/October?

I am scheduled to speak at the National IT Conference in Sri Lanka (NITC 2018), 2 to 4 October 2018, organized by the Computer Society of Sri Lanka (CSSL). Are there any other computer, education or environmental events I could attend in in Sri Lanka, or in the week before, or after on the way through Malaysia or Singapore?

On a previous visit to Colombo I talked about "Mobiles and e-learning for PandemicFlu Response" to members of the Sahana Software Foundation, at Virtusa's Colombo Office. In 2015 I stopped off for the Second International Conference on Open and Flexible Education (ICOFE 2015) in Hong Kong on my way to the 10th International Conference on Computer Science & Education (ICCSE 2015) in Cambridge.

ANU Multi-purpose Halls

ANU Culture and Events Building,, multi-purpose hall floor plan, with retractable stadium seating
The Australian National University (ANU) is constructing a "Culture and Events Building", planned to be open in 2019. This building has two multi-purpose halls with retractable stadium seating for large events (500 seats) and small events (220 seats). This is can be used for example, where there is are conference presentations during the day in stadium format, then the seats retracted and tables put out on the flat floor for the conference dinner.

Thursday, June 14, 2018

University Training for Casual Academics

As one of the "permanent casuals" of Australian academia, I found Baik, Naylor and Corrin's work on how to support us of great interest (2018). I was a reluctant student of education, but when on to complete the ANU Graduate Certificate in HE. As the Authors point out, most university teaching is carried out by non-permanent staff. This has its advantages, as experts from industry can share their current experience with students. Also those with more recent experience as a student (such as graduate students) make better teachers. The Authors focus on a Framework for
sessional teachers at the University of Melbourne. 
It was not clear how much Professional Development the Baik, Naylor and Corrin had in mind for staff. There is mention of a "paid induction program as well as two hours of paid ongoing PD". However, it is not clear how extensive the induction program is, or if it is part of a formal educational qualification.
One area where I don't agree with the authors is on the role of online and face-to-face training for teaching. They suggest an online option should be provided, but only as a fall-back for those who can't come to class. I suggest this policy is outdated. Most students in Australian university courses are now blended and students don't come to class if there is an online alternative. Those teaching need experience in what it is like to be an online student, as that is what most of their students are. They best way to learn to teach online is by being an online student.

ps: In my own case, I like that as an Adjunct I am considered part of the academic staff, but I mostly don't have to go to meetings. However, the lack of permanent employment can have a corrosive effect on early career academics. When asked about postgraduate programs I suggest to prospective students they look at masters coursework programs and professional doctorates, which will qualify them for careers outside academia. With a secure job in industry they then have the option of being part time academics. However, those who instead choose the PhD path have only a very slim chance of ever getting a secure job as an academic and will not be as well qualified for industry.


Chi Baik, Ryan Naylor & Linda Corrin (2018) Developing a framework for university-wide improvement in the training and support of ‘casual’ academics, Journal of Higher Education Policy and Management, DOI: 10.1080/1360080X.2018.1479948

OpenCon 2018, November 2-4, Toronto, Travel Scholarships

Travel scholarships are being offered for students and early career academics to "OpenCon 2018",  November 2-4 in Toronto. Normally I don't bother applying for such things because the paperwork is so tedious. But this one requires a relatively simple web form.
"OpenCon is the conference and community for students and early career professionals interested in advancing Open Access, Open Education and Open Data. OpenCon 2018 will be held on November 2-4 in Toronto, Canada. Each year, OpenCon brings together a diverse, representative, and engaged group of participants, with travel scholarships available to most participants. For this reason, attendance at OpenCon 2018 is by application only."

On the Future of Work and Workers

On 4 June I talked to an Australian Senate Committee on the Future of Work and Workers, at Parliament House in Canberra, alongside Rob Fitzpatrick, CEO of the Australian Information Industry Association (AIIA), and just after the ANU VC. The committee had my short written submission, but here is the Question and Answer:
"Mr Worthington : I'm an honorary lecturer in computer science at the Australian National University where I teach computer professional skills, and I'm a member of the professional education governance committee of the Australian Computer Society, where I help set the education standards for computer professionals, but I'm making this submission in my own individual capacity as an educator.
You've heard from others about how technology is going to eliminate many jobs, and I would agree with the estimates of somewhere between 10 to 40 per cent of jobs. Some of my colleagues in the IT profession in Australia are effectively helping that happen. Workers will need better soft and technical skills. They will also need regular retraining because they will have multiple careers. I guess I'm an example of that. I started as a computer programmer with the government, working for the defence department. I've been retrained with a second skill as an educator, being trained in Australia and in North America.
Australia already has an education system which could provide work-ready learning across schools, vocational and higher education, and I suggest that we just need to tweak it a bit to make it more flexible. We need to strengthen the vocational education and training—VET—sector, so that it will blend between secondary education and universities. We need to fix up the conditional loans schemes so that they can be applied to the VET sector more freely.
We need to make university more flexible where the assumption isn't that someone does a three-year degree full-time and the exceptions are part-time people. The assumption should be that people will do smaller subdegree qualifications. The Vice-Chancellor of ANU mentioned microcredentials. They'll be doing those part-time and mostly online, not on campus, in response to the needs they have in the workplace. Actually most university courses are already blended. They're already mostly online and partly on campus as we speak, but that's not reflected in government policy or the rhetoric of universities mostly. 
We have a particular problem with international students. Regulations require them to be enrolled full-time all the time and that imposes a burden on people doing the education because, if a student fails or drops one subject, they have a problem. So I think we need to make that more flexible and allow them to do part of that online and allow them to do less than a full-time load.
We need to address soft skills more in our university system. I help do that at the ANU with the TechLauncher program where we have teams of students working on real problems for real clients in Canberra. These are skills that can be taught, but the problem is that the people teaching them—university academics—generally don't have the skills to do the teaching. We need to ask our university lecturers to be formally qualified in teaching I believe, which is a problem because university professors like to make other people do courses but they don't like to do them themselves. So I think that will need some encouragement via funding and perhaps some regulation in the system. That's about all I had to say. I welcome questions.
CHAIR: Thanks, Mr Worthington. At this point I might exit and hand over to Senator Patrick. I hope to be back before you finish up.
ACTING CHAIR ( Senator Patrick ): Thank you, Chair. Perhaps I'll start in reverse order. Mr Worthington, you describe your own situation as having changed throughout your career. I myself have managed a number of engineers over time. There is a progression we currently have in engineering where you might be a computer scientist then perhaps either go off in a different direction to management or broaden yourself from computer science to hardware to then algorithm development and some other disciplines, so you become multidisciplined. So is there really a change that is taking place or are you simply leveraging off your past knowledge and steering your career perhaps to a higher level?
Mr Worthington : I think there is a change. You need to learn new skills in a new area. The problem comes when you need to learn them relatively quickly. One of the examples given in the previous evidence was about cybersecurity. I help teach some people who are going to go into that area and that involves for technical people issues of human behaviour, international relations, the law and all sorts of things that they weren't previously familiar with. So I think this not just a matter of degree but a matter of learning new things. 
It can also be that they're not necessarily at a higher level of abstraction—for example, to teach in the Australian vocational education sector I had to go to TAFE and learn TAFE teaching, even though I was already giving lectures at universities. I had to learn a different way to approach it. I could've said: 'That's TAFE teaching. That's a low level. Why do I have to get that?' But I had to learn a different way of teaching, so I went to the appropriate place to learn those types of skills. I think we'll see people having a grab bag of little qualifications and different skills they get from different forms of institutions.
ACTING CHAIR: I guess I was suggesting that as you move along in your career and you do make changes there are, really, additional skills that you're learning which complement the next thing that you're doing. You don't end up giving away some of those past skills that you have learnt.
Mr Worthington : I wouldn't agree with that. For example, I tutor teams of students writing computer software and I have to say to them, 'The commuter programming languages I learnt stopped being used before you were born, so I can't help you with the technical aspects of your code. I can help you with how to talk to the client, how to manage the team, how to plan and strategise, but for the technical elements of this I am no longer confident in this area.' 
ACTING CHAIR: I think there is a place for people who can install a program in assembly language. It's still okay!
Mr Worthington : Yes, but they've stopped using some of the processes I used as well.
Mr Worthington : I don't think it's just a matter of adding to your skills. There'll be many skills that are no longer relevant. AI was mentioned. There'll be things that used to be seen as a very important technical skill. Now the computer does it, and what was your main bread and butter nobody really wants. 
ACTING CHAIR: I noticed you were in my line of sight when I was asking about the 10 per cent versus 40 per cent. You seemed to be nodding on the 10 per cent.
Mr Worthington : It's somewhere in between 10 and 40 per cent for jobs disappearing. Nobody really knows. I think there's a view out there that all this technological development happens in a smooth, planned way. But, having seen it from the inside, we're making it up as we go along and we don't really know, when we produce some new technology, whether it's going to take off or not, whether people will really accept automated self-driving processes or not. So, to a large extent, we don't know. 
ACTING CHAIR: That's the cruise-control example I was using: a safe way to get, eventually, to automation but it may not happen quickly, or maybe there's some point where it suddenly takes off. I guess these things are hard to predict.
Mr Worthington : Yes. I have people come up to me every week and say, 'Hey, look, we've invented a new' something. And I go, 'Yes, and what can you use it for, and will anybody want it?' They're the really hard questions. 
ACTING CHAIR: Yes. I ran an R&D cell. I had 30 ideas pop before me per week and every once in a while one of them had a market, so I know where you're coming from. You talk about the development of new technology as being something that Australians want to have happen. Noting that, from my experience, you need to do the R&D stage, then you need to move to prototyping, then to manufacturing, and get the feedback loop into the development cycle, are we in some sense handicapped because of our shrinking manufacturing sector? ..."

From: Select Committee on the Future of Work and Workers 04/06/2018, Senate Hansard, Australian Parliament 

Wednesday, June 13, 2018

SIGPLAN Empirical Evaluation Checklist

The ACM Special Interest Group on Programming Languages (SIGPLAN) have provided a revised "SIGPLAN Empirical Evaluation Checklist". This provides criteria to help decides what is suitable in a research paper for publication. This might be of interest more widely, but emphasizes quantitative research, rather than qualitative.

Here are the criteria by Berger, Blackburn, Hauswirth, and Hicks (2018):
Clearly stated claims
  • Explicit Claims
  • Appropriately-Scoped Claims
  • Acknowledges Limitations 
Suitable Comparison
  • Appropriate Baseline for Comparison 
  • Fair Comparison
Principled Benchmark Choice
  •  Appropriate Suite
  • Non-Standard Suite(s) Justified
  • Applications, Not (Just) Kernels
Adequate Data Analysis
  • Sufficient Number of Trials
  • Appropriate Summary Statistics
  • Report Data Distribution
Relevant Metrics
  •  Direct or Appropriate Proxy Metric
  • Measures All Important Effects
Appropriate and Clear Experimental Design
  • Sufficient Information to Repeat
  • Reasonable Platform
  • Explores Key Design Parameters
  • Open Loop in Workload Generator
  • Cross-Validation Where Needed
Presentation of Results
  • Comprehensive Summary Results
  • Axes Include Zero
  • Ratios Plotted Correctly
  • Appropriate Level of Precision
Adapted from Berger, Blackburn, Hauswirth, and Hicks (2018).


 E. D. Berger, S. M. Blackburn, M. Hauswirth, and M. Hicks (June 2018). SIGPLAN Empirical Evaluation Checklist, ACM SIGPLAN. URL

Charles Darwin University in Top 4% Globally

Charles Darwin University claims to be "... ranked in the top two percent of universities worldwide", based on the THE World University Rankings. However, CDU is ranked 301–350 out of 1,000 institutions, placing it in the top 30%, not 2%. In 2017 the UK Advertising Standards Authority issued guidelines to stop UK universities making similar misleading claims.  Perhaps similar guidelines are needed in Australia.

The Webometrics Ranking of World Universites places CDU 1,165 out of 27,000 institutions. So it would be reasonable for CDU to claim to be ranked in the top four percent of universities worldwide, on that evidence.

Tuesday, June 12, 2018

Artificial Intelligence will take over most teaching

Professor Neil Selwyn,
Monash University
Professor Neil Selwyn, of Monash University, argues Artificial Intelligence (AI) will never take over from human teachers (2018). I suggest AI will never take over completely from human teachers, but it will supplement their role for many routine tasks. At the Australian Senate Committee hearing on the Future of Work and Workers last week, I was asked what proportion of jobs AI would replace. I suggested more like 40% than 10% but no one really knows.

The teaching profession doesn't "face" an impending change, that change has already started happening. University teaching has already flipped from being campus based to mostly online. However, most university lecturers will not admit this to themselves or to others. The higher levels of school will follow over the next few years. This is without AI, just using decades old e-learning technology.

Much of education can already be reliably provided by machines. This requires fewer, but more highly trained, teachers. What AI will not be able to do, at lest not well, is handle the exceptional situations.

Like human teachers, AI learns (that is why it is called "artificial intelligence"). Software is used to mimic human learning. Unlike a human, AI can learn from millions of cases very quickly. However, the results still need to be checked by a human as they can be unpredictable.

AI can minimc a human very effectively. The ELIZA natural language program of the 1960s was able to mimic a human in a conversation. It does not take much to do this, if the topic is limited to a narrow field, such as a course.

Like human teachers, some AI can explain its chain of reasoning, allowing the student to learn not just what to think, but how to.

AI can use a virtual face and body on screen, but in most cases this is not necessary. Most university students now learn online using text based materials. Where there are videos they watch them at high speed so any person visible is little more than a blur.

This is not to say AI will, or should, replace all human teachers. But AI will be used alongside other tools, such as writing and books, for teaching. It is a long time since anyone argued seriously that students should not write notes as they would then not be able to memorize, or that students should not read books, only listen to the teacher. In a few years time arguments against AI will seem as quaint as those against writing and books.


Neil Selwyn  (201, June 12). Six reasons Artificial Intelligence technology will never take over from human teachers, Edu Research Matters (Blog). Australia Association for Research in Education. URL

Monday, June 11, 2018

International Students Appreciate Active Learning

Marrone,  Taylor and Hammerle (2018) found that active learning techniques work for large lectures, particularly for international students. This is a significant finding, as it has been assumed by some that international students just want to passively be told what to learn. The authors also suggest better results could be obtained with small group activities, where international students feel more comfortable. The research was conducted with 413 accounting students at an Australian university.


Marrone, M., Taylor, M., & Hammerle, M. (2018). Do International Students Appreciate Active Learning in Lectures?. Australasian Journal of Information Systems, 22. doi:

Saturday, June 9, 2018

Temporary Computer Based Examination Room at ANU

Room in the Computer Science and Information Technology Building, Australian National University, set up with temporary screens and computers for a computer based examination.  Screen design  by Bob Edwards. Photo by Tom Worthington
At the end of each semester, the Research School of Computer Science at the Australian National University (ANU) sets up as temporary computer based examination rooms. Computer labs are already equipped with desktop computers, tutorial rooms have all-in-one desktop computers added to the existing movable desks. Students own devices (BYOD) are not used, as advanced computing students have the skills to circumvent the restricted access controls on such devices.

Two rows of desks are arranged so that students can sit in two facing rows. Six all-in-one desktop PCs are placed in two rows on the desks. The computer screens are large enough to prevent the student seeing the work of the student opposite.

Underside of a temporary screen used between students sitting computer based examinations in the Computer Science and Information Technology Building, Australian National University. The two screws in the underside stop the panel being slide too far onto the desk. Design  by Bob Edwards. Photo by Tom Worthington
Particleboard screens are placed on the desks, between each two students. The screens are self standing, and protrude about 150 mm off the desk, to prevent the student seeing their neighbor's work. Two screw-heads underneath the screen base prevent it being pushed further onto the desk. The screens were designed by Bob Edwards.

Improving retention and completion for students in higher education

The report "Improving retention, completion and success in higher education" (Australian Department of Education and Training, 6 June 2018) makes 18 recommendations to improve student retention.  Many of the recommendations seem a little pointless, such as the first, suggesting institutions give students "the best chance to complete their studies". However, some are more significant, such as recommending nested courses and allowing for trimester structures. Unfortunately the report is of little practical value as it concentrates on internal on-campus students, whereas most Australian students are now studying externally online.
The report notes that the attrition rate for external students is about two and a half times that of internal students and the number of such students is rising. What the report does not point out that this is partly a correlation, not a causal relationship. The factors which cause students to enroll externally also result in the dropping out at a higher rate.
The report suggests that "Curricula for external courses must be designed with the external student in mind and must utilise the benefits of contemporary technology in course design". However, the report doesn't address the change which has already taken place in Australian higher education: most students now study externally online off-campus. Most of these students are officially enrolled as internal on-campus students, but don't come to lectures and only attend where required. 

Rather than address the needs of these students, government policies and some university practices are trying to force these students to attend when they do not need to or want to. Instead, I suggest the curricula for all courses needs to be designed with the external students in mind. Universities and government policy makers need to accept that students stopped coming to class years ago and change policy and practice accordingly.


Expectations of completion in the current context

  1. As a first priority, institutions should ensure students who have the capacity to succeed in higher education are given the best chance to complete their studies through the appropriate provision of academic and other support services as required of them by the Higher Education Standards Framework.

Supporting students to make the right choices

  1. School students and mature-age people need better access to effective career advice. The National Career Education Strategy, due to be released in 2018, should be closely monitored to identify improvements in the area of student career advice, including study options and pathways, and information about the post school learning environment. This strategy should also be expanded to include mature-age students or a separate strategy should be initiated for this cohort.
  2. Career advice cannot be left to schools. Every higher education institution should ensure that their students are given the opportunity for career planning and course advice on entry to the institution and as they require it throughout their studies.
  3. Where and how student success, completions, retention and attrition data is made accessible to students should form part of considerations by the Department of Education and Training in the establishment of a new online information platform.

Supporting students to complete their studies

  1. Every institution should have its own comprehensive student-centred retention strategy, which is regularly evaluated. These strategies could include institutional retention benchmarks and, as appropriate, processes for entry and exit interviews, the integration of data-based risk analytics and targeted support interventions, a suite of support services and a means to re-engage with students who have withdrawn.
  2. Institutions should automatically review the enrolment of all students who have not engaged in their studies to an agreed level by the census date.
  3. Institutions should pay particular attention to ensuring their support services are meeting the needs of external students who are not regularly attending campus because these students are identified as at risk of not completing their studies.
  4. Every institution should have an institution-wide mental health strategy and implementation plan.
  5. Institutions should increasingly offer nested courses, which are appropriate and compliant with the Australian Qualifications Framework, to provide students with a greater range of exit options with meaningful qualifications.

Sharing best practice

  1. There is already a wide variety of approaches to sharing best practice within the higher education sector. However, these approaches are not always scalable or frequently evaluated. Peak bodies should collaborate to develop streamlined processes to collect and disseminate best practice, with support from the Department of Education and Training. A dedicated website could be established for this purpose.

Clarity of definitions and enhancing transparency

  1. The higher education community should work together with the Department of Education and Training to ensure a greater understanding and clarity of definitions in attrition, retention, success and completions data. The Department should continue to measure and publish adjusted attrition, retention, student success and completions data.
  2. At present some institutions have a trimester structure of teaching and this can lead to different timings for assessment, graduation and reporting. As a result, students who complete Semester 1 and 2 and enrol in Semester 3 but not Semester 4 are recorded as not completed. Consequently, the definition of attrition should be changed to reflect the trimester teaching structure.
  3. The adjusted attrition rate should be the primary measure of attrition published for domestic commencing bachelor students.
  4. The Department of Education and Training should further develop and publish the calculation of attrition rates that take into account key student characteristics so as to better reflect institutional differences.
  5. The Department of Education and Training should report attrition among non-university higher education providers on a similar basis to its reporting of Table A and B universities.
  6. The Department of Education and Training should publish attrition data at more disaggregated levels, for example, by institution, by study area and by student characteristics.
  7. The Department of Education and Training should establish a common student identifier to better understand student pathways across tertiary education with a view to working with State and Territory Governments to establish a common student identifier across all levels of schooling.

Accountability and regulation

  1. TEQSA already has sufficient powers in relation to provider compliance with the Higher Education Standards Framework in terms of the identification and tracking of students at risk with support strategies in place, analysis of student performance and evidence on reasons for attrition. TEQSA should continue to take account of every institution’s retention performance in assessing whether these standards are being met."
    From:  "Improving retention, completion and success in higher education" (Australian Department of Education and Training, 6 June 2018)

ANU TechLauncher Call for Projects, Mentor, Tutors and Guest Speaker

Students taking part in a  Lego Serious Play exercise at the Australian National University in Canberra
The Australian National University (ANU) runs the
ANU TechLauncher program, where teams of computer science and engineering students work work on a real project for a business, government or academic client A call for projects and participation has been issued for  Semester 2, 23 July to 1 November.

Athabasca University Should Continue with Open Source Software

Athabasca University (AU) is receiving CAD$4.9M from its Canadian Alberta provincial government, with part for IT. This is not a lot of money, so I suggest AU make a virtue of necessity and focus on the use of free open source software. AU already uses the Moodle Learning Management System (from Australia), Mahara e-portfolio (from New Zealand)  and Open Journal Systems  for e-publications (from Canada). Apart from reducing the cost of software acquisition (they are all free), this open source software tens to be less demanding in its use of computing power and network access. AU could add other such software to its suite.

I was a student at AU (virtually) from 2013 to 2017 and my course in green computing is offered by AU. During this time I have used the online services of AU extensively. Last week I attended the EduTech Australia annual conference looking at the latest in educational software for universities. Noting I saw changed my view that open access software is a good option for universities.

Friday, June 8, 2018

Best of EduTECH Sydney, Higher Ed Leaders Congress, Day 2

EduTECH Sydney starts this morning. I am on 2:20pm today speaking about e-portfolios in the Higher Ed Leaders Congress. Apart from that, here is my best of:
9:10 Distant learning through AR: How AR has helped paramedic students 'VISUALISE' skills
  • Learn how AR can be used to provide distant learning
  • Discover how to provide more opportunity for distance students through AR and mixed-media visualisations
  • Anatoli Kovalev
  • Build your student's confidence level through digital engagement
9:30 UNSW Hero Program case study: Student-driven innovation for universities
  • Discover the first program in Australia that gets students and their professional work experience to design their own learning experience and environment
  • How universities can work with students to design and develop internal innovation projects at your university
  • Lessons learned from project, and how you can successfully achieve great collaboration with students, operational and educational staff
9:50 Critical success factors of the implementation of a National eLearning Platform
  • See how Singapore’s National eLearning Platform has supported the teaching and learning of 96,000 students and 3,500 staff within a 6 month project time frame
  • Learn how to effectively collaborative through inter-institutional teams to work towards a common goal
  • Discover how to provide students and industry with appropriate skillsets through the National eLearning Platform
10:10 Blending learning pedagogy, technology and space in a connected world: Implications for institutions, teachers and students
  • Discover how learning and teaching has changed in a blended world and blended teaching has changed in a connected world
  • Learn how to use technology to support effective assessment and enhance student learning
  • Discover how teachers and students can thrive in a connected world
  • See how project-based learning can create active life-long learning

Shirley Alexander
11:30 How to prepare work-ready graduates
  • Learn how to integrate technology in the classroom to deliver enhanced teaching and learning experiences
  • See how strong collaborative partnerships can aid educational institutions to prepare for digital disruption
  • Discover how virtual and physical education infrastructure can drive innovation in Australian education
Pauline Farrell
Pauline Farrell,
Former Director, 
Swinburne University

11:50 Education's response to prepare students for the future of work
  • Discover how AI will change the workforce and how we can prepare our students
  • Learn how online learning innovation can be used to respond to changes in workforce
  • See where in the education cycle you can be proactive in driving student engagement with digital learning
12:10 Digital inclusion, moving from chalk & talk to eEducation
  • Discover a vision for an educational future: Transforming education through technology
  • Learn how to use online and mobile technologies to improve skills
  • Create a framework for assessing and considering readiness of universities to provide access to students
12:30 Lunch
Jo Mithen
Monash College
14:00 Industry partnerships to create better opportunities for students
  • Insights into Monash Talent: learning how to solve skills challenges facing industries
  • Using technology to find the right graduates to the industry
  • Discover how to match services to make sure right candidate aligned to right role
14:20 Using an ePortfolio to capture students’ skill sets to align to workplace
  • Discover innovative education techniques to teach students how to better communicate with employers and people
  • Learn how to build your students confidence and capture their skills set to become more job ready
  • Discover how to provide formal postgraduate education to students in their workplaces via mobile devices
15:20 Afternoon tea