Paper Written by ChatGPT Get 2.6 out of 5 from Reviewers

Roger Clarke recently referred me a paper for review, knowing my interest in military matters, and AI. Apart from the topic, and how it was generated, the paper was also of interest as it was submitted to a public review forum, Qeios.

The paper was bland, much like other AI generated text. It was not really publishable as an academic work, as it didn't have anything new to say. But it was very much like many internal documents I had read, written by committee at the Australian Department of Defence. 

The other problem with the document was attribution. I suggested the author's name be changed to "Anonymous", as the content comes from a large corpus of online work by many authors. The person listed as author I suggested should be acknowledged as an editor. This would be a way, I suggest to acknowledge the contribute on people who generate such work.

Online Assessment with Portfolios in Response to the Coronavirus, 8am Wednesday

I will talk on "Online Assessment with Portfolios in Response to the Coronavirus" Wednesday 12 August, 8-9 am, AEST (Canberra Time) as part of the Microlearning Series curated by Manisha Khetarpal at Maskwacis Cultural College in Canada. This is the third of six weekly sessions.  Register now.

Higher Education After COVID-19

1. Responding to the Coronavirus Emergency with e-Learning
2. Open Content for e-Learning in Response to the Coronavirus
3. Online Assessment with e-Portfolios in Response to the Coronavirus
4. Tools used to engage students in online delivery
5. Mentoring student group work onlin.
6. Higher education after COVID-19: Not business as usual

These are online, open to all and free, but please  register now. Suggestions are welcome.

Part 3: Online Assessment with Portfolios in Response to the Coronavirus

Slides and notes (2 Mbytes PDF)

Due to the risk of COVID-19, universities are using online invigilated examinations in place of examination rooms. However, there are much better ways to assess students online. Assessment can be an afterthought when delivering a face to face course: after all, it comes after the learning, doesn't it? Perhaps not. Assessment can be used to provide the student and the teach information on what they already know, to help plan what to learn and how. Assessment can be used during learning to see what more needs to be done. Assessment can also be integrated into what the student works on, in real world tasks, or simulations of them. A portfolio can be used for students to collect evidence of the skills and knowledge they have gained in formal courses and co-curricular, to show they have the real world skills needed to graduate. All of these approaches to assessment require more skill of the educator, than a final examination, but may not take any extra time. There are tools which can be used online, such as Moodle (Quizzes, forums, and peer assessed assignments) and Mahara (e-portfolios), but these require a knowledge of pedagogy, as well as the technology.  This third talk in the series and also I will cover some requests from the second talk.

See also:

1. Assessment for ICT Sustainability-Assessment and Strategies for a Low Carbon Future, (ANU COMP7310, Athabasca University COMP 635,  ACS Green Technology Strategies), Worthington, 2016
2. Assessment for the Learning to Reflect module for the ANU TechLauncher program, (COMP3500, COMP3550, COMP3710, COMP4500, and COMP8715), Worthington, 2019

3. Capstone e-Portfolio for the Masters of Education, Athabasca University, Tom Worthington (2016)
   

Don't Replace Long Paper Examinations with Long Electronic Ones

• Online invigilation allows paperless home exams
• ProctorU at Athabasca University,  Proctorio at ANU
• Work okay, but do we need a long exam?
  

In the first half of the year I tried a couple of demo exams with one of the leading online exams products. There were some good points : the compatibility test beforehand warned my CPU was too slow, but it worked okay anyway. I have a relatively under-powered laptop by today's standards and slow Internet connection. This is not just because I am a cheapskate, I am doing what Ken Mattingly did for Apollo 13: make sure that if it works on my equipment, it will work for those out in the void.

The business of panning the camera around the room, to make sure no one else was present was a bit of light relief.

Having tried this process twice, I could image taking a short test this way. There is the comfort of a familiar environment, rather than an exam room, but help is further away. If it was for a small amount of my grade (perhaps 20%) and short (around 30 minutes), this might be tolerable, just. But if for the majority of the grade in a course and for hours, there is no way I would ever contemplate sitting such a test.

In my view, any exam should be relatively short, more like a take-home examination, than end of semester one. One issue is when the students can do it. I worry that some course designers are assuming all the students take the test at precisely the same time. This will not work well at home. There should be a period of a day, or days, over which students can choose to start their exam.

Academics and university administrators may be using online examinations in the mistaken belief that some law, policy or rule requires examinations. Even before COVID-19, there was flexibility with how assessment could be done. Assessment rules generally have a lot of detail about how to do a traditional examination, but then include a get-out clause saying that some other form of assessment can be used. Other forms of assessment take skill to design and resources to administer. But if you have been trained in how to design assessment (as I have), this can be done reliably, at reasonable cost.
While just a practice, these exams brought back a tinge of the terrors I suffered as a student with assessment. As an instructor, I would not use such a system for long examinations. Such examinations are not a good way to assess the knowledge, and even less so for assessing skills needed for real work tasks. In addition such examinations increase the risk to the health and safety of students. Students were already at risk of mental illness and self harm before COVID-19. They are now at higher risk, and there are better, safer, ways to carry out assessment. Online invigilated tests should have only a small part in assessing students.

See also:

1. Don't Replace Long Paper Examinations with Electronic Ones, May 24, 2020
2. Make Online Exams More Like the Real World, May 21, 2020
3. Don't Postpone Examinations: Replace Them With Realistic Tests, March 13, 2020

Use Small Quizzes and Short Exercises For Building Confidence

Example quiz questions in Moodle

 

Multiple choice and short free form questions can be used to build the student's confidence and prompt them to study the material. A small number of marks can be allocated to encourage students to actually do the quiz, which can then prompt them to study the material.

Typically I will set a half dozen or fewer multiple choice questions per week and two discussion questions. These would normally make up about 20% of the final grade. To avoid this making it too easy for bright students, I set the grading scheme so these tests do not count for more than a mid range grade (a Credit, not a Distinction or High Distinction). 

Multiple choice and some other types of quiz question can be marked automatically. They can also have questions drawn at random from a bank, to make cheating harder. But the object of the exercise is not to test the student's know under exam conditions, but to help them learn.

Free form text answers can't be automatically marked, but they can be peer assessed, also creating an additional learning opportunity. Students learn from having to read an answer from another student and make a useful comment about it. Of course some students will try using boilerplate phrase and give everyone a high mark, but that behavior is easily corrected by giving them zero for the exercise.

This type of assessment doesn't need fine grained marking, it can be a two, three or five point scale.

Use Projects for More Advanced Skills

"In the weekly forums and the two major assignments, you are asked to write about the ICT sustainability of an organization your are familiar with. It may be where you work, w

here you have worked, or an organization you can obtain information on from public sources. For this first assignment, describe the organization and how it uses ICT. Outline how you plan to obtain the information needed to report on the carbon footprint and materials use of the ICT operations of the organization."

From Assessment: ICT Sustainability

The way to provide students with real world skills is to have them working on projects as close to real world as possible. This is easiest to do for students who are already employed in a relevant job, where they can do their project in the workplace. This is common in engineering, computing and teaching. As an example, I encourage the ICT Sustainability students to approach their boss and get permission to estimate the carbon emissions of their organization. If students don't have a suitable job, I suggest they pick somewhere they would like to work and do the project based on open source information. 

Some courses are built around an internship or a group project for a client (ANU has these for computing students). However, this approach requires a considerable investment of staff time and skills of dealing with industry.

Use an e-Portfolio for a Capstone

Hot Air Balloon Over Canberra,
Photo by Tom Worthington CC-by 3.0 1996

 

“Three years ago I set out to make the transition, from a computer professional, to a designer and teacher of computing courses. My original learning goal was to acquire the skills needed to undertake quantitative research in education. Instead, I have learned enough theory, balanced with practical skills, to help with the transition to e-learning in Australian universities. ...International tensions could disrupt the flow of students to Australia very quickly ... "

From: Capstone e-Portfolio for the Masters of Education, Athabasca University, Tom Worthington (2016)

 

 

In place of a conventional masters thesis, I completed a Capstone e-Portfolio using Mahara for my MEd at Athabasca University. This presented five artifacts, covering six competency areas, in a process described by Hoven (2015). With this the student prepares a collection of samples of their work, and explains how this is evidence that they have the skills and knowledge required. It can be used for a small part of a course or for an entire degree. 

 

What is key is not having an e-portfolio software package (just about any document construction tool will do), but a process to help the student build the portfolio, piece by piece. Hoven describes how students were having difficult preparing a portfolio and this was delaying their degree completion. I was one of the students in the transition period, who could choose to do this as a course, with an instructor and with step by step feedback from peers. This process works well, but it takes the student a few attempts to understand what is happening. I had to undertake several of these portfolio experiences before I had the hang of it. This can be used for anything from a vocational short course to a postgraduate degree.

References:

 Capstone e-Portfolio for the Masters of Education, Athabasca University, Tom Worthington (2016)

Hoven, D. (2014). ePortfolios in Post-Secondary Education: An Alternate Approach to Assessment. UAE Journal of Educational Technology and eLearning, 1, 11-24. URL https://web.archive.org/web/20160210160342/http://ejournal.hct.ac.ae/wp-content/uploads/2014_Article2_Debra.pdf

Disguise a Reflection Exercise as a Job Application

"Select a real position to apply for. Prepare a cover letter ... CV ... statement addressing the selection criteria ... Supplementary material ... and a copy of the advertisement you are responding to. ...

After submitting, each student will be allocated four assignments from other students to provide feedback on. ... The reviews are double-blind peer reviewed ... Twenty percent of your mark will be for the quality of your feedback."

 From: Learning to Reflect for ANU COMP3500/3550/3710/4500/8715, Tom Worthington, 2019.

 

Educators are fond of asking students to "reflect on their learning" in the e-portfolio. However, as a STEM student I had difficulty understanding what was expected. The ANU TechLauncher students  are equally confused by such a task. So several years ago the reflective exercise using Mahara was replaced with one where the student has to write a job application submitted as one PDF document. The student selects a real job to apply for, or application to undertake a PhD, apply for start-up funding or similar. In their application the student must present evidence of what they did, what they learned and how they learned it, in their course. This is, in effect, a capstone reflective e-portfolio exercise in disguise. It is much easier for the student to understand as they are (hopefully) about to graduate and be looking for a job (or a PhD program, or startup funding). 

For the ANU TechLauncher program I wrote a blended module to assist the students. The essential point here is no so much the course content, but the exercises to help students and provide feedback. Last year I included an online contingency as envisaged in my MEd portfolio. This was activated for COVID-19, with relatively little difficulty.

 

References

 

Learning to Reflect for ANU COMP3500/3550/3710/4500/8715, Tom Worthington, 2019. URL http://www.tomw.net.au/technology/education/learning_to_reflect/learning_to_reflect_2_1.shtml

 

Worthington, T. (2020, June). Blend and Flip for Teaching Communication Skills to Final Year International Computer Science Students. Paper accepted for the IEEE International Conference on Teaching, Assessment and Learning for Engineering (TALE), 10-13 December 2019, Yogyakarta, Indonesia. URL https://openresearch-repository.anu.edu.au/handle/1885/204833

Designing in an On-line Learning Option, Tom Worthington, Higher Educaiton Whisperer, March 14, 2020 URL https://blog.highereducationwhisperer.com/2020/03/designing-in-on-line-learning-option.html

Use e-Portfolios Throughout a Program?

Mahara SmartEvidence Example

1. identify their study goals and licensing requirements
2. inventory existing skills and knowledge
3. obtain exemptions from courses
4. plan study
5. record evidence of achieving each goal
   

 

ePortfolios are something usually used at the end of a degree program as part of a capstone. The student may be told earlier they have to do this, but if it is only due at the end, then that is when the student will worry about it. I suggest e-portfolios could be used through a program of study, first to help the student identify their study goals (and specifically any program or industry licensing requirements they must meet), inventory their existing skills and knowledge, obtain exemptions from courses, plan their study, record evidence of the achievement of each goal.

What the student is seeking entry to a profession, the template of requirements could be loaded into the e-portfolio of them to complete. The student could plan courses to meet these requirements, but they could also check which could be met from prior experience, by co-curricular activities, work experience and projects. The aim should be for the average student to only undertake formal study for one third to one half their program. 

 

For a university degree, plus multiple concurrent industry licensing requirements,  there may be hundreds of requirements to meet, each requiring evidence to be kept and certification to the required standard recorded. Mahara SmartEvidence and  Moodle Competencies have some of the features needed to be able to record student's achievements in a fine grained way needed for a whole of program e-Portfolio. 

References

Assessing Competencies with Mahara and Moodle, Higher Educaiton Whisperer, August 5, 2016. URL https://blog.highereducationwhisperer.com/2016/08/mahara-annotation-function.html

Worthington, T. (2014, August). Chinese and Australian students learning to work together online proposal to expand the New Colombo Plan to the online environment. In 2014 9th International Conference on Computer Science & Education (pp. 164-168). IEEE. URL https://openresearch-repository.anu.edu.au/bitstream/1885/11724/1/Worthington%20Chinese%20and%20Australian%20students%20learning%202014.pdf

 

Virtual Hackathon on Fighting Pandemics 

Remo Conference Example

Last weekend I talked at the Virtual Hackathon on Fighting Pandemics. This is organized by the Australian National University Humanitarian Innovation Society (ANU HISoc), with the Clinton Global Initiative University and IBM. We have had an introduction from the organizers at HISoc, and a design thinking workshop from IBM. I then talked about better logistics. The opening sessions used Remo Conference, and later ones Zoom video conferencing.

Remo Conference takes the "conference" part of video conferencing more literally than other products, with an interface which looks like a conference room floor plan. Participants can be pre-allocated to tables, with a tiny picture of them, labeled with their initials, around the table as if seated. Spare space at the table is indicated by empty chairs. Those at a table can see and hear each other with the video windows appearing at the top of the screen. Participants can move tables by double clicking on another. Some tables may be reserved for MCs and a help desk. The MC can call all participants to a presentation, at which point the table conversations shut down and a large window opens for the presenter. In contrast, when in presentation mode Remo Conference looks much like any other webinar product, with a presentation window, chat, Q&A and Raise-Hand functions.

While there are some technical problems with Remo, it shows potential for a way to provide a classroom-like experience.

 

 

Reference

 

Hacking the Virtual Hackerthon,  Higher Education Whisperer, August 8, 2020 URL https://blog.highereducationwhisperer.com/2020/08/hacking-virtual-hackerthon.html

Discussion Questions

1. Who are you teaching?
2. How are they assessed?
3. How do you record the evidence? 
4. Does this work online?

What worked well at the hackerthon was that I did not present too much or too complex information. I followed this up with four questions for the participants, which worked well in getting a discussion going. So I have prepared four questions for this week.

Reference

Better Logistics for Fighting Pandemics, Higher Education Whisperer, July 31, 2020. URL https://blog.highereducationwhisperer.com/2020/07/better-logistics-for-fighting-pandemics.html

 

Open Content for e-Learning in Response to the Coronavirus, 8am Wednesday

I will talk on "Open Content for e-Learning in Response to the Coronavirus" 5 August, 8-9 am, AEST (Canberra Time) as part of the Microlearning Series curated by Manisha Khetarpal at Maskwacis Cultural College in Canada. This is the second of six weekly sessions.  Register now.

Higher Education After COVID-19

1. Responding to the Coronavirus Emergency with e-Learning
2. Open Content created
3. Assessments in online delivery
4. Tools used to engage students in online delivery
5. Mentoring student group work onlin.
6. Higher education after COVID-19: Not business as usual

These are online, open to all and free, but please  register now. Suggestions are welcome.

Part 2: Open Content for e-Learning in Response to the Coronavirus

Slides and notes (2 Mbytes PDF)

Openness in education can apply to the way education is provided  as well as the course materials. This is not just about using free stuff to save money. The Open Source movement in computing and the Wikipedia show a way of contributing, as well as using, content. This second talk in the series is based on the chapter "Use of Open Education Resources" in my free e-book "Digital Teaching In Higher Education". Also I will cover some requests from the first talk.

Openness in Education

1. Open University
2. Open Educational Resources
3. Open Business Models
4. OER Example: ICT Sustainability
5. Questions from last week:
   1. Is elearning more time consuming?
   2. Hackerthons & Group Project Work Converging

Open University

• UK Open University established in 1969
• Provide access for disadvantaged students
  
• Confronted today's issues decades before COVID-19

The UK Open University (OUUK) was established in 1969. Walter Perry, the first Vice Chancellor's 1976 account of the formation of the intuition is a must read for those interested in online university education in the face of COVID-19 today. Many of the issues now being addressed with learning during COVID-19 came up 50 years ago.

"Open" in OUUK terms is about providing access to a university education to those who do not have access to traditional university, due to a lack of formal education, money, location, gender or membership of a minority group. Open universities have used whatever communication technology is available: paper mail, radio and TV broadcast, video and audio tapes, now computers, the Internet, and smart phones.

The Open Universities Australia (OUA) consortium has been delivering on-line courses from some of Australia's most respected universities for decades. Australia's University of New England (UNE) was one of the models for OUUK and there are hundreds of open universities around the world.

I find it frustrating when I hear of concerns expressed about remote learning today as if these are new and novel: problems identified, and for which solutions were found, decades ago.

References: 

Perry, W. (1976). Open University: a personal account. Milton Keynes: Open University Press. 

Remove Barriers to Australian On-line Universities (Blog Post),  Tom Worthington, September 21, 2013, The Higher Education Whisperer

Open Educational Resources

"Think free as in free speech,
not free beer."
Richard Stallman
^{(quoted by Lessig, Lawrence 2006).
Photo} Galuel / CC BY-SA

"digitised materials offered freely and openly for educators, students and self-learners to use and reuse for teaching, learning and research"

From OECD, Centre for Educational Research and Innovation, 2007 (in CETIS, 2008).

MIT's OpenCourseWare can be modified for reuse, unlike most MOOCs.


 
The Microlearning Series has had many presentations about the concept of Open Learning, so I will not go into much detail here. The idea is appealing, but the practice is difficult.

Two examples of OER are MIT's OpenCourseWare  and more recent Massive Open Online Courses (MOOCs). An important distinction is that MIT allowed their content to be modified and reused. Many other providers don't allow modification, and some limit reuse.

References:

CETIS, J. (2008). Open educational resources-Opportunities and challenges for higher education. http://wiki.cetis.ac.uk/images/0/0b/OER_Briefing_Paper.pdf

Open Business Models

• Apache Software Foundation
• Wikimedia Foundation

The major failing in many OER projects has been how to sustain them. Projects are usually funded by a government, an institution or a corporation in a burst of autistic enthusiasm, but the funding runs out and the project folds. There has to be a way for OER as to provide a reward to those undertaking it, so they can keep doing it.

The Open Source movement in computing provides one model. Individuals, corporations, governments and universities work on software which can be modified and used for free. They do this partly for the common good, but also so they can make use of the resulting software, including commercial companies for profit.

An alternative approach is where a not-for profit organization can solicit ongoing donations to cover the cost, while making use of mostly donated labor. While not working fr profit, The Wikimedia Foundation, which runs the Wikipedia, and the Apache Software Foundation, which provides much of the software for the Internet, are sophisticated organizations with significant revenue.

OER Example: ICT Sustainability

1. Graduate online course
2. Run by ACS and ANU in 2009
3. Also offered through Open Universities Australia
4. Adapted for Athabasca University by Brian Stewart
5. Moodle file, e-book under a CC BY-SA open license.

The Australian Computer Society (ACS) me to design an e-learning course in “Green ICT Strategies” in 2008, to train computer professionals in how to estimate and reduce CO₂ emissions. The ACS allowed me to retain the intellectual properly, so I released it under a Creative Commons license. It was run as part of the ACS graduate level certificate in February 2009, and the Australian National University (ANU) Masters of Computing from July 2009. ACS also offered the course through Open Universities Australia. A North American version of the course was developed by Brian Stewart for Athabasca University (Canada) in 2011 and is still running as "Green ICT Strategies". 

The course was developed in Moodle, with an e-book of notes, under a CC BY-SA open license. A traditional distance education asynchronous approach was used with quizzes, discussion questions and assignments.

Originally ACS created the course for public benefit. Each time I ran the course for ANU I would make improvements, which could be used by others.

Reference:

Worthington, T., "A Green computing professional education course online: Designing and delivering a course in ICT sustainability using Internet and eBooks," Computer Science & Education (ICCSE), 2012 7th International Conference on , vol., no., pp.263,266, 14-17 July 2012 URL: http://dx.doi.org/10.1109/ICCSE.2012.6295070  Preprint available at: http://hdl.handle.net/1885/9013

Is e-learning more time consuming?

• Not if you know what you are doing, as a student or teacher.
• Don't waste time talking at students while they passively listen.
• Budget staff and student time on what is important: student activities and assessment.
• OER and Open Access Software can help

Last week one of the questions asked in the forum was: "Is e-learning more time consuming". If you have learned how to learn, or teach, online and a disciplined in what you do, then the answer is "no". However, there can be much frustration, and time-wasting, where teachers try to replicate a classroom experience and students expect that.

A common complaint by students (one I read in a Facbook post today) is that the teachers are not teaching them online, but instead leaving them to work out what to do themselves.

The teachers job is to guide the student's learning, not tell them what to do in detail. Live "lectures" online, or in a classroom, are about the least useful form of instruction for this. A tutor talking to a small group of students is also of little educational value. Instead the teacher needs to focus on getting students to do things, and reflect on what they have done.

OER and Open Access software can help. These can not only save time, but also provide useful pointers to ways of working better. Rather than prepare materials yourself, alone, work with others. Rather than studying alone, work with others. Decide in advance to share your results, acknowledging everyone's contribution.

Hackerthons & Group Project Work Converging

ANU TechLauncher,
Team Formation Exercise, 2018

Australian & NZ Defence Forces Logistics Hackathon: by ACS, 72 hours in May, 70 mentors and 500 participants online.

ANU TechLauncher team formation: 300 students and 40 clients for two hours online.

Same tools and techniques used: Slack, Zoom and GitLab.



One way to get students to engage with their studies is with projects. However, group projects can impose a large administrative overhead for teachers. For years I have been supervising formal university group projects, usually as part of a capstone: required to be completed before the student can graduate. These projects take 12 or 24 weeks, with students in class with a tutor for two hours a week.

At the same time I have been mentoring teams in hacker competitions, which typically take 72 hours, run at a venue 12 or 24 hours a day, usually over a weekend. The hackerthon participants are mostly students, but these are are extra-curricular with no course credit, but some prizes.

The photo shows how these large scale exercises used to be done: in a large room over many hours at a time (Worthington, 2020). Some large hackerthons took over whole university buildings, or floors of corporate headquarters. There is considerable excitement generated by the close physical proximity and fast pace.

With COVID-19 both the project courses and the hackerthons have moved online using the same project management and communication tools. There is the possibility to use hackerthons as a form of group teaching technique. This approach emphasizes the participants organizing their own work and those who participated previously running later events.

Steve Nouri organized a Secure Supply Chains COVVID-19 Hackerthon for the Australian and NZ Defence Forces, hosted by the Australian Computer Society in May 2020. I was one of the Lead Mentors, providing advice to 70 mentors, who in turn helped 500 participants, over 72 hours. 

Last week I helped with the two hour team formation exercise for ANU Techlauncher. There are about 300 students, who formed teams of about seven students each live online after listing to pitches from clients. The students will now spend one or two semesters working on projects for about forty clients. Each team has a tutor who they meet with weekly, but the emphasis is on students working togehter.

Both the hackerthon and Techlauncher used Slack, Zoom and other communication tools, plus project tools, such as GitLab. There is potential to make more use of OER templates for such group work and Open Source software.


References:

Lessons Learned From Executing Successful Virtual Hackathons, Steve Nouri, Forbes, Jun 11, 2020. URL https://www.forbes.com/sites/forbestechcouncil/2020/06/11/lessons-learned-from-executing-successful-virtual-hackathons/

Online real time student project team formation with Zoom and Slack,  Tom Worthington (Blog post), The Higher Educaiton Whisperer, 1 August 2020. URL https://blog.highereducationwhisperer.com/2020/08/online-real-time-student-project-team.html

Worthington, T. (2020, June). Blend and Flip for Teaching Communication Skills to Final Year International Computer Science Students. Paper accepted for the IEEE International Conference on Teaching, Assessment and Learning for Engineering (TALE), 10-13 December 2019, Yogyakarta, Indonesia. URL https://openresearch-repository.anu.edu.au/handle/1885/204833

Misconceptions and Challenges with Open Educaiton Resources

Dave Towey,
University of Nottingham

Greetings from Yogyakarta, at the IEEE TALE 2019 engineering education conference, where Dave Towey, University of Nottingham is speaking on  "OER: Six Perspectives on Global Misconceptions and Challenges". He identified the issue of costly US textbooks as one of the sources of interest, but also misunderstandings, with Open Educaiton Resources.

"Abstract—Open Educational Resources (OERs) are freely accessible materials for teaching, learning, and research that have been made available such that they can be freely used, modified, and shared. Prompted by the potential positive impact that OERs can have, a growing community of OER enthusiasts
has been advocating for their adoption in higher education, and elsewhere. The growing demand for computer science education and training (especially in areas related to artificial intelligence, machine learning, and big data) has led to speculation as to how OERs may be best leveraged to support such teaching and learning. The recent occasion of an international OER workshop at the 2019 IEEE Conference on Computers, Software and Applications (COMPSAC) allowed for six OER enthusiasts, all with extensive OER experience, to reflect on the status of OER adoption and understanding. These reflections, addressing misconceptions and challenges, form the basis for this paper."

The six misconceptions are (this is my interpretation, not the author's words):

1. MOOCs are open
2. If its on the Internet, it's open
3. If its open it's free
4. Lazy educators use open stuff
5. OER reduces cost
6. OER is low quality

At question time I suggested one way to promote OER is to apply the techniques provided in the start-up entrepreneurial centers at universities. OER done as an ad-hoc cottage industry will not be able to compete with for-profit professional publishing output. So I suggest applying the entrepreneurial approach used to set up new industries. This would look at how to produce a product which appeals to educators, how to obtain funding to grown the business. This is does not necessarily require a for-profit corporation, it could be a not-for-profit organization.

Openness and Reproducibility in Science Workshop

Greetings from the Openness and Reproducibility in Science Workshop at the Australian National University in Canberra. Rosalind Attenborough (University of Edinburgh) talked on her PHD research on "Scientific openness: a new epistemic virtue?". The talk was a lot more accessible that the title of her research would indicate. She is interviewing researchers to get tier vies on what openness is, its perceived benefits and problems. 

Rosalind briefly mentioned open source software as a progenitor for open science. So I asked her why she did not look at the history of open source software. It seems to me that many of the issues with openness, particularly the reward structures for people working in it and resourced had already been solved by the software community. Open source software is now produced by everyone from individuals, to academics, government agencies, and multi-billion dollar corporations. We routinely teach computing students about the tools and techniques of open access, including how to make a career out of it.

Programme

9.00-9.30	Arrival, coffee
Session 1: Ideas of openness
9.30-9.40	Welcome and introduction	Prof Joan Leach (ANU)
9.40-10.30	Talks: mapping the landscape of open science	A/Prof Sujatha Raman (ANU) Ros Attenborough (Edinburgh)
10.30-11.00	Coffee break
11.00-12.30	Panel discussion: Why openness? What does it mean and what is it for?	Prof Rachel Ankeny (Adelaide) Prof Ginny Barbour (QUT, AOASG) Dr Chris Cvitanovic (ANU) Prof Kiaran Kirk (ANU) [TBC] Prof Cameron Neylon (Curtin) [TBC] Prof Adrian Mackenzie (ANU) [TBC]
12.30-13.30	Lunch
Session 2: Practices of openness
13.30-13.50	Talk: "What's So Open About Plant Pathology?"	A/Prof Adam Sparks (USQ) https://www.openplantpathology.org/
13.50-14.40	Intro, followed by open discussion: "What are preprints and how do they accelerate science communication?"	Dr Sarvenaz Sarabipour (Johns Hopkins) via Zoom
14.40-15.00	Talk: “Building a local community: R-Ladies in Canberra"	Dr Alice Richardson (ANU, R-Ladies Canberra) Dr Petra Kuhnert (Data61, CSIRO, R-Ladies Canberra)
15.00-15.30	Coffee break
15.30-15.50	Talk: "How to facilitate a global open science early career research community with local impact"	Dr Benjamin Schwessinger, eLife ambassador program, reproducibility for everyone
15.50-16.10	Talk: “For reproducibility, we need the methods behind the data.”	Dr Lenny Tetylman (CEO of protocols.io) via Zoom
16.10-16.20	Wrap-up
16.20-17.00	Poster session and socializing (posters on different topics related to reproducibility and rigour)	Posters organised by Dr Diep Ganguly (post-doc, ANU)

Future of Scholarly Publishing

The EU has released "Future of Scholarly Publishing and Scholarly Communication", by a panel of experts, including from the Bill and Melinda Gates Foundation, Wellcome Trust, Elsevier, and Springer Nature. Not surprisingly, Springer Nature and Elsevier did not agree to the rest of the experts' recommendation to move to open access publishing as soon as possible.

I don't agree with one of the panel's unanimous recommendations: not to use journal-based metrics for hiring, promotion and tenure, and funding decisions. In my view, metrics are essential, but the right metrics.

Several recommendations urge balanced and diverse representation (gender, geography and career stage). Using blind reviews, not just for papers, would go some way to achieving this.

The expert group proposes a set of principles for scholarly communication:

"... accessibility, maximum usability, and accommodating an expanding range of scholarly contributions (data, software, new documentary forms, etc.).  ... a distributed infrastructure based on open standards to ensure access and interoperability. ... equity, diversity and inclusivity, and to the need for community building. ... quality and the integrity of scholarly contributions ... flexibility and innovations while also retaining its focus on cost effectiveness."

From "Future of Scholarly Publishing andScholarly Communication" (EU, P. 6, 2019).

Recommendations to researchers and research communities:

1. "When participating in research assessment, for example in hiring, promotion and tenure, and funding decisions, focus on the merits and impact of a researcher's work and refrain from the use of metrics - particularly journal-based metrics - as a proxy. In particular, they should incorporate the recommendations from DORA and the Leiden Manifesto into the assessment process.

2. Take responsibility for ensuring that all research contributions are made openly available, discoverable, and reusable according to agreed community standards (including the FAIR principles).

3. Increase awareness of, and sense of responsibility for, implications of choices and actions in roles as authors, reviewers and members of decision-making groups.

4. Strive for a balanced and diverse representation (in terms of gender, geography and career stage) when seeking collaborations, organizing conferences, convening committees, and assigning editors and peer-reviewers, and building communities such as learned societies.

5. Work towards increased recognition and appreciation of peer-review work as core research tasks. To this end, support greater transparency, including the publishing of signed reports. Support better training and inclusion, and focus on quality of the research in peer review.

6. In the case of communities of researchers, such as learned societies, develop policies and practices that support modes of scholarly communication in line with the vision outlined above. Along with universities, learned societies and other research communities need to alert and train their researchers to the importance and the responsibilities of communicating knowledge, either formally, through publishing, or through other means."

Universities and research institutions should:

1. "Develop policies and practices to ensure that all research contributions are made openly available, discoverable, and reusable according to agreed community standards (including the FAIR principles).

2. Promote and implement the recommendations of DORA and the Leiden manifesto to ensure that research assessment takes into account a wide range of scholarly contributions including research articles, preprints, datasets, software, patents and materials (e.g. in hiring, tenure, and promotion decisions).

3. In deciding which infrastructures to use, support, and contribute to, choose platforms using free or open source software, offering open data via an open license, and leveraging open standards where possible. Acting in this fashion will also reinforce researcher-led initiatives that aim to facilitate scholarly communication and publishing.

4. Strive for a balanced and diverse representation including, but not limited to, gender, geography and career stage) when hiring, seeking collaborations, when organizing conferences, when convening committees, and when assigning editors and peer-reviewers, and building communities such as learned societies.

5. In negotiations with service-providers refuse non-disclosure clauses and include clauses which enable cost and price control, and compliance monitoring. Strive to facilitate collective action with other institutions by e.g. sharing cost and price data through joint initiatives (e.g. OpenAPC)."

Research funders and policy-makers should:

1. "Develop policies - along with appropriate funding mechanisms - to ensure all research contributions arising from their funding are available to everyone, everywhere, without any barriers to access or restrictions on reuse.

2. When evaluating researchers, ensure that a wide range of contributions (scholarly publications, but also data, software, materials etc) and activities (mentoring, teaching, reviewing etc) are considered, and that processes and criteria of evaluation are both appropriate to the funder’s research programme, and transparent.

3. Develop funding mechanisms to support the development of open, interconnected and distributed scholarly publication infrastructures, and for their maintenance over the long term.

4. Consider how funding policies affect diversity and inclusivity of research on a global scale. In particular, funders should work to ensure that review boards, committees, panels, etc., are diverse - in terms of gender, geography, and career stage.

5. Work with the other actors in the scholarly communications ecosystem to ensure that the total costs of enabling research to be openly available to everyone, everywhere, without barrier or restriction, be also open and transparent."

Publishers and other service providers:

1. "Develop and publicly announce transition plans to move as soon as possible to comprehensive open access (Springer Nature and Elsevier have differing views with respect to this recommendation, a result of extensive disscusions in the expert group.)

2. Develop, use, and support interoperable tools (including open source software wherever possible) and services not only to facilitate access and reuse of scholarly outputs, but also to facilitate innovative interventions of new entrants.

3. Strive for balanced diversity (including, but not limited to, gender, geography and career stage) among authors, reviewers, and editors who work with publications.

4. Foster transparency and accountability in peer review, for example by publishing peer review reports and author responses alongside the published articles

5. Make all publishing charges public (including special pricing and waivers), and provide full descriptions of services provided, in order to enable the development of a transparent and cost-effective marketplace designed to support the open communication and reuse of all scholarly contributions.

6. Experiment with new approaches to the evaluation and communication of research outputs, and share the outcomes so that a body of evidence can help to optimise future systems."

Practitioners, educators, and other societal groups:

1. "Organize and advocate for free access to, and right to reuse of, publicly funded research results.

2. Reach out to funders, research institutions, and policy makers in order to develop new communication channels, new forms of co-creation and co-planning of research, and new forms of funding in response to needs, concerns and issues emanating from the population at large.

3. Look for opportunities to engage with research topics / results that are of interest to societal groups and their communities.

4. Bring forward research topics/questions that are mis- or underrepresented (e.g. by contacting relevant researchers, attracting the attention of other actors in the science system, or mobilising action in organised interest groups)."

Placing the Student at the Centred of Open Education

Dr. Rajiv Jhangiani, Professor of Psychology at Kwantlen Polytechnic University, presented a USQ webinar on "Student-centred learning with open education" today. This was inspiring stuff, with open educational practices (OEP) providing better quality student centered learning, as well as lowering cost. But usually with such presentations I ask "How can I do this?" and there is no answer or it involves spending a lot of money. Not in this case, Dr. Jhangiani's book "The Philosophy and Practices that are Revolutionizing Education and Science" is available free online. He also recommended Elizabeth Mays' "A Guide to Making Open Textbooks with Students".

Academic Open Access Publishing Needs Education and Auditing

A move to open access research publishing will need, I suggest, a well resourced information campaign, and a well staffed anti-fraud unit. European and UK research funding bodies have decided to move to open access publishing by 2020. As a condition of receiving funding researchers will be required to publish papers from that research where they are available free. The researchers will be provided with finding to pay for publication. However, it is likely that some academics and publishers will attempt to game the new system, placing public money and the integrity of research at risk.

In 2014 the Australian Government expanded student loans from universities to non-government  vocational and educational training (VET) colleges. A 2015 Parliamentary inquiry heard "... harrowing and concerning evidence of misconduct by private VET providers ...". Education providers, and brokers, offered students inducements such as laptops to sign up for courses. This came at a cost of hundreds of millions of dollars to the Australian community and harmed the reputation of Australian vocational education.

Similar scams are likely with open access publishing, unless controls are in place from the start of the scheme. Academics are under pressure to publish and will be tempted to accept inducements from publishers and brokers, to place their business with them. We will likely see offers of free equipment and trips in return for publishing. We are also likely to see offers to the heads of research organizations for all research by their staff to be directed to one publisher in return for favorable treatment.

It should be made clear to publishers and researchers from the outset that offering, or accepting, cash and gifts in return for publishing is a crime. It should be made clear there are trained criminal investigators looking for such behavior and those responsible with be prosecuted.

OpenCon 2018, November 2-4, Toronto, Travel Scholarships

Travel scholarships are being offered for students and early career academics to "OpenCon 2018",  November 2-4 in Toronto. Normally I don't bother applying for such things because the paperwork is so tedious. But this one requires a relatively simple web form.

"OpenCon is the conference and community for students and early career professionals interested in advancing Open Access, Open Education and Open Data. OpenCon 2018 will be held on November 2-4 in Toronto, Canada. Each year, OpenCon brings together a diverse, representative, and engaged group of participants, with travel scholarships available to most participants. For this reason, attendance at OpenCon 2018 is by application only."

Failure of UK Research Open Access Policy

Dr Danny Kingsley, Head of Scholarly Communication at Cambridge University Library has concluded that the UK's policy for free and open access to publicly-funded research has failed: "Manipulation of embargo periods, confusing information, and a graduated charging system for different licenses all work towards ensuring a second income stream. Far from moving to an open access future we seem to be trapped in a worse situation than we started." (Kingsley, 2017).

I was intrigued by Danny's inclusion of "scheduled tweets" to accompany the presentation. 

To answer the question at the end of the presentation "How do we get out of this mess?", I suggest a modestly funded initiative to encourage new Open Access publishers. This could use the start-up infrastructure already established around universities, such as Cambridge. 

Academic staff and students could be trained and funded to set up new companies to provide OA publishing in competition with existing closed-access ones. These new companies would aim to be profitable, in the long term, while providing publications without a fee to the subscriber. The main question to be answered by any such start-up would not be publishing infrastructure (which there is plenty of free-open-access software for), or revenue streams (which are feasible), but what incentives could be provided to induce academics to choose a new, unproven publisher.

Reference

Kingsley, D. (2017). So did it work? Considering the impact of Finch 5 years on [Presentation file]. https://www.repository.cam.ac.uk/handle/1810/269913

Open Educational Resources: Short free online course starts Monday from ANU

A free on-line course "Open Educational Resources: A Continuum for Practice" is being offered by the Australian National University (ANU), starting Monday. This is in the ANU Online Coffee Course Series, where the student spends fifteen minutes a day studying for a week. There are no tests, just on-line discussion.

"In this upcoming course, you’ll explore some of the deeper issues surrounding open education; specifically, practical decisions and questions that should be asked when developing, reusing, and collaborating on open education activities. Each day the facilitators will provide stimulus for reflection and group discussion, as well as a synthesis of the previous days’ discussion."

Modules

1. From Resources to Practice: why ‘mere access’ isn’t enough.
2. ‘I’ll know it when I see it’, or, usefully defining quality in openness.
3. Assessment in an open world: beyond the disposable assignment.
4. Designing for reuse: PDF and the open practitioner.
5. ‘Nothing new’ or radical reality? Putting openness into practice.

Facilitators

Adrian Stagg, Manager (Open Educational Practice) for the University of Southern Queensland.

Emma Power, research assistant for the Pro-Vice Chancellor’s Office (SILS), at the University of Southern Queensland.

 

Open Access Policy Draft from Australian Research Council

The Australian Research Council (ARC) have released a draft Open Access Policy for comment. The current policy was adopted in 1 January 2013. The policy requires researchers receiving a grant to make their research papers available in open access format within a year of publication, or tell the ARC why they can't. What is missing from the review are any statistics on how successful the current policy is. I suggest the new policy require institutions to publish a list of all publications on their web site, indicating which are open access and an annual count of the number open and closed, along with the amount of public funding received for those open and closed. The ARC should then use this information to produce statistics on openness of Australian research, in terms of numbers of publications and proportion of funding, along with the full list of institutions ranked by openness.

Some Recent Open Access Books on Higher Education

The Directory of Open Access Books (as mentioned in The Imaginary Journal of Poetic Economics) indexes several thousand scholarly books. As with DOAJ for journals, this is useful if you are looking for readings for a course and you want students to be easily able to obtain them. Not only can the students access the papers without having to worry about paying a fee, but they also tend to be in formats which are easier to read. In many cases, book chapters are available as separate downloads, so a students does not have to get the entire book, just for the few pages you want them to read. Interestingly, unlike the journal papers, which tend to be in open access journals produced by non-profit groups, there are open access books from conventional for-profit publishers (who also sell paper, and in some cases electronic, copies of the same free books).

A search for book on Higher Education returned 48, of which here are a few recent ones which look interesting:

Becoming a World-Class University: The case of King Abdulaziz University

Authors: Osama Tayeb --- Adnan Zahed --- Jozef Ritzen
ISBN: 9783319263793 9783319263809 Year: 2016 Pages: 190 Language: English
Publisher: Springer
Subject: Education

The Dream Is Over: The Crisis of Clark Kerr’s California Idea of Higher Education

Author: Marginson ,Simon
ISBN: 9780520292840 9780520966208 Year: 2016 Pages: 260 Language: English
Publisher: University of California Press
Subject: Education

Open Education: International Perspectives in Higher Education

Author: Blessinger , Patrick (Editor); Bliss, TJ (Editor)
ISBN: 9781783742783/9781783742806 Year: 2016 Volume: 1 Pages: 378 Language: English
Publisher: Open Book Publishers
Subject: Education

Higher Education Reforms in Romania: Between the Bologna Process and National Challenges

Authors: Adrian Curaj --- Ligia Deca --- Eva Egron-Polak --- Jamil Salmi
ISBN: 9783319080536 9783319080543 Year: 2015 Pages: 226 Language: English
Publisher: Springer
Subject: Education

Knowledge Production and Contradictory Functions in African Higher Education

Authors: Nico Cloete --- Peter Maassen --- Tracy Bailey
Book Series: African Higher Education Dynamics Series ISBN: 9781920677855 Year: 2015 Volume: 1 Pages: 310 Language: English
Publisher: African Minds
Subject: Education

Mergers and Alliances in Higher Education: International Practice and Emerging Opportunities

Authors: Adrian Curaj --- Luke Georghiou --- Jennifer Casenga Harper --- Eva Egron-Polak
ISBN: 9783319131344 9783319131351 Year: 2015 Pages: 307 Language: English
Publisher: Springer
Subject: Education

Talent Development in European Higher Education: Honors Programs in the Benelux, Nordic and German-Speaking Countries

Author: Dr. Marca V.C. Wolfensberger
ISBN: 9783319129181 9783319129198 Year: 2015 Pages: 335 Language: English
Publisher: Springer
Subject: Education