- Responding to the Coronavirus Emergency with e-Learning
- Open Content for e-Learning in Response to the Coronavirus
- Online Assessment with e-Portfolios in Response to the Coronavirus
- Tools used to engage students in online delivery
- Mentoring student group work onlin.
- Higher education after COVID-19: Not business as usual
Part 3: Online Assessment with Portfolios in Response to the CoronavirusSlides and notes (2 Mbytes PDF)
Due to the risk of COVID-19, universities are using online invigilated examinations in place of examination rooms. However, there are much better ways to assess students online. Assessment can be an afterthought when delivering a face to face course: after all, it comes after the learning, doesn't it? Perhaps not. Assessment can be used to provide the student and the teach information on what they already know, to help plan what to learn and how. Assessment can be used during learning to see what more needs to be done. Assessment can also be integrated into what the student works on, in real world tasks, or simulations of them. A portfolio can be used for students to collect evidence of the skills and knowledge they have gained in formal courses and co-curricular, to show they have the real world skills needed to graduate. All of these approaches to assessment require more skill of the educator, than a final examination, but may not take any extra time. There are tools which can be used online, such as Moodle (Quizzes, forums, and peer assessed assignments) and Mahara (e-portfolios), but these require a knowledge of pedagogy, as well as the technology. This third talk in the series and also I will cover some requests from the second talk.
- Assessment for ICT Sustainability-Assessment and Strategies for a Low Carbon Future, (ANU COMP7310, Athabasca University COMP 635, ACS Green Technology Strategies), Worthington, 2016
- Assessment for the Learning to Reflect module for the ANU TechLauncher program, (COMP3500, COMP3550, COMP3710, COMP4500, and COMP8715), Worthington, 2019
Capstone e-Portfolio for the Masters of Education, Athabasca University, Tom Worthington (2016)
- Online invigilation allows paperless home exams
- ProctorU at Athabasca University, Proctorio at ANU
- Work okay, but do we need a long exam?
The business of panning the camera around the room, to make sure no one else was present was a bit of light relief.
Having tried this process twice, I could image taking a short test this way. There is the comfort of a familiar environment, rather than an exam room, but help is further away. If it was for a small amount of my grade (perhaps 20%) and short (around 30 minutes), this might be tolerable, just. But if for the majority of the grade in a course and for hours, there is no way I would ever contemplate sitting such a test.
In my view, any exam should be relatively short, more like a take-home examination, than end of semester one. One issue is when the students can do it. I worry that some course designers are assuming all the students take the test at precisely the same time. This will not work well at home. There should be a period of a day, or days, over which students can choose to start their exam.
Academics and university administrators may be using online examinations in the mistaken belief that some law, policy or rule requires examinations. Even before COVID-19, there was flexibility with how assessment could be done. Assessment rules generally have a lot of detail about how to do a traditional examination, but then include a get-out clause saying that some other form of assessment can be used. Other forms of assessment take skill to design and resources to administer. But if you have been trained in how to design assessment (as I have), this can be done reliably, at reasonable cost.
While just a practice, these exams brought back a tinge of the terrors I suffered as a student with assessment. As an instructor, I would not use such a system for long examinations. Such examinations are not a good way to assess the knowledge, and even less so for assessing skills needed for real work tasks. In addition such examinations increase the risk to the health and safety of students. Students were already at risk of mental illness and self harm before COVID-19. They are now at higher risk, and there are better, safer, ways to carry out assessment. Online invigilated tests should have only a small part in assessing students.
- Don't Replace Long Paper Examinations with Electronic Ones, May 24, 2020
- Make Online Exams More Like the Real World, May 21, 2020
- Don't Postpone Examinations: Replace Them With Realistic Tests, March 13, 2020
Use Small Quizzes and Short Exercises For Building Confidence
|Example quiz questions in Moodle|
Multiple choice and short free form questions can be used to build the student's confidence and prompt them to study the material. A small number of marks can be allocated to encourage students to actually do the quiz, which can then prompt them to study the material.
Typically I will set a half dozen or fewer multiple choice questions per week and two discussion questions. These would normally make up about 20% of the final grade. To avoid this making it too easy for bright students, I set the grading scheme so these tests do not count for more than a mid range grade (a Credit, not a Distinction or High Distinction).
Multiple choice and some other types of quiz question can be marked automatically. They can also have questions drawn at random from a bank, to make cheating harder. But the object of the exercise is not to test the student's know under exam conditions, but to help them learn.Free form text answers can't be automatically marked, but they can be peer assessed, also creating an additional learning opportunity. Students learn from having to read an answer from another student and make a useful comment about it. Of course some students will try using boilerplate phrase and give everyone a high mark, but that behavior is easily corrected by giving them zero for the exercise.
This type of assessment doesn't need fine grained marking, it can be a two, three or five point scale.
Use Projects for More Advanced Skills
here you have worked, or an organization you can obtain information on from public sources. For this first assignment, describe the organization and how it uses ICT. Outline how you plan to obtain the information needed to report on the carbon footprint and materials use of the ICT operations of the organization."
The way to provide students with real world skills is to have them working on projects as close to real world as possible. This is easiest to do for students who are already employed in a relevant job, where they can do their project in the workplace. This is common in engineering, computing and teaching. As an example, I encourage the ICT Sustainability students to approach their boss and get permission to estimate the carbon emissions of their organization. If students don't have a suitable job, I suggest they pick somewhere they would like to work and do the project based on open source information.
Some courses are built around an internship or a group project for a client (ANU has these for computing students). However, this approach requires a considerable investment of staff time and skills of dealing with industry.
Use an e-Portfolio for a Capstone
|Hot Air Balloon Over Canberra, |
Photo by Tom Worthington CC-by 3.0 1996
“Three years ago I set out to make the transition, from a computer professional, to a designer and teacher of computing courses. My original learning goal was to acquire the skills needed to undertake quantitative research in education. Instead, I have learned enough theory, balanced with practical skills, to help with the transition to e-learning in Australian universities. ...International tensions could disrupt the flow of students to Australia very quickly ... "
From: Capstone e-Portfolio for the Masters of Education, Athabasca University, Tom Worthington (2016)
Capstone e-Portfolio for the Masters of Education, Athabasca University, Tom Worthington (2016)
Disguise a Reflection Exercise as a Job Application
"Select a real position to apply for. Prepare a cover letter ... CV ... statement addressing the selection criteria ... Supplementary material ... and a copy of the advertisement you are responding to. ...
After submitting, each student will be allocated four assignments from other students to provide feedback on. ... The reviews are double-blind peer reviewed ... Twenty percent of your mark will be for the quality of your feedback."From: Learning to Reflect for ANU COMP3500/3550/3710/4500/8715, Tom Worthington, 2019.
Designing in an On-line Learning Option, Tom Worthington, Higher Educaiton Whisperer, March 14, 2020 URL https://blog.highereducationwhisperer.com/2020/03/designing-in-on-line-learning-option.html
Use e-Portfolios Throughout a Program?
|Mahara SmartEvidence Example|
- identify their study goals and licensing requirements
- inventory existing skills and knowledge
- obtain exemptions from courses
- plan study
- record evidence of achieving each goal
What the student is seeking entry to a profession, the template of requirements could be loaded into the e-portfolio of them to complete. The student could plan courses to meet these requirements, but they could also check which could be met from prior experience, by co-curricular activities, work experience and projects. The aim should be for the average student to only undertake formal study for one third to one half their program.
Assessing Competencies with Mahara and Moodle, Higher Educaiton Whisperer, August 5, 2016. URL https://blog.highereducationwhisperer.com/2016/08/mahara-annotation-function.html
Virtual Hackathon on Fighting Pandemics
|Remo Conference Example|
Last weekend I talked at the Virtual Hackathon on Fighting Pandemics. This is organized by the Australian National
University Humanitarian Innovation Society (ANU HISoc), with the Clinton Global Initiative University and IBM. We have had an introduction from the organizers at HISoc, and a design thinking workshop from IBM. I then talked about better logistics. The opening sessions used Remo Conference, and later ones Zoom video conferencing.
takes the "conference" part of video conferencing
more literally than other products, with an interface which looks like a
conference room floor plan. Participants can be
pre-allocated to tables, with a tiny picture of them, labeled with their
initials, around the table as if seated. Spare space at the table is
indicated by empty chairs. Those at a table can see and hear each other
with the video windows appearing at the top of the screen. Participants
can move tables by double clicking on another. Some tables may be
reserved for MCs and a help desk. The MC can call all participants to a
presentation, at which point the table conversations shut down and a
large window opens for the presenter. In contrast, when in presentation mode Remo
Conference looks much like any other webinar product, with a
presentation window, chat, Q&A and Raise-Hand functions.
While there are some technical problems with Remo, it shows potential for a way to provide a classroom-like experience.
- Who are you teaching?
- How are they assessed?
- How do you record the evidence?
- Does this work online?