Wednesday, February 12, 2020

Alterative Assesment Processes for Students Not on Campus Due to COVID-19 Coronavirus

Yesterday I was asked about how to assess students who are unable to get to campus due to the COVID-19 Coronavirus. Many Australian universities still have an on-campus end of semester paper-based examination. These cannot be administered to students who are isolated for public health reasons.

Athabasca University (where I studied last), has proctored online examinations for undergraduate courses, via a commercial service, ProctorU. There were no examinations for the postgraduate program I took, this having assignments, quizzes, peer assessment, and a live online capstone presentation with Q&A.

It should be kept in mind that an examination is not required at the end of every course, nor is it a good way to assess real-world skills. Many Australian universities, supplement assessment with non-supervised assessment under the category of "take home examinations".

The usual approach with online course is to have regular small, assessment tasks. This is to keep the students engaged, provide feedback, and allow staff to check student progress. Some think this assessment should not count towards the final result and some do. I lean towards the latter, but the small tests should not count for much. It is possible to use an assessment scheme which encourages, or requires, students to do the small stuff, but so that it either doesn't count to their final result, or much. For example, you can make this a hurdle: the student has to do the small stuff, but it doesn't change their final grade.

For some versions of the course ICT Sustainability I set small assessment tasks which only count for a Credit, not a Distinction or High Distinction. This is described in the course notes (but keep in mind that is a graduate course):

With online courses there is always the worry that students had someone else undertake the assessment for them. That worry can be lessened by having assessment spread out through the course, making it harder for students to simply contract out a few big tests. An approach frequently also used is to have the student give a presentation and answer questions. The presentation can be face-to-face, or live online.

Monday, February 10, 2020

Reflecting on Reflecting on Learning

Learning to Reflect" blended module. This is the last assessed task for students undertaking the ANU Tech Launcher program before they graduate. Last semester the students read notes, watched videos, did an online quiz, participated in an on-line forum (with peer assessment), then a face-to-face workshop (run by Tempie Archer, from ANU Careers). They then submitted and assignment (with peer feedback), and repeated the process. That all worked well, but was seem by the students, and my colleagues, as too complex, and too much work. There was too much to read, too many assessment items, and too much peer assessment. Also the tutors, who the students have for face-to-face tutorials, felt excluded from the process.





Resources for New Tutors

I have been a preoccupied providing advice on how to quickly provide courses for online delivery, for students who are unable to get to class due to the Novel Coronavirus. But I need to get back to tutor training. I am finding the online course Contemporary Approaches University Teaching has some good material, but would take some pruning, as it is a 12 week, 24 hour course, not something a tutor can do in a few hours. Some useful resources for students I have found, mostly from ANU's Coffee Courses:
  1. LinkedIn recently predicted that the most in-demand soft skills with employers for 2020 will be creativity, persuasion, collaboration, adaptability, and emotional intelligence.Tutoring can help you refine these skills and provide evidence to a prospective employer you are leadership material.

    See: The Most In-Demand Hard and Soft Skills of 2020, Bruce Anderson, LinkedIn Talent Blog, January 9, 2020. URL https://business.linkedin.com/talent-solutions/blog/trends-and-research/2020/most-in-demand-hard-and-soft-skills
  2. "... authentic real life assessment tasks should contain the challenges of a real life work context.
    From: Principles of authentic assessment, from Assessment and Feedback, Jill Lyall and Mandy Tutalo, ANU Coffee Course, 2019 URL https://anuonline.weblogs.anu.edu.au/2019/04/30/day-2-principles-of-authentic-assessment/
  3. In Small Group Teaching: From: Good practice examples in Module 3: Teaching practice, Enhancing Student Wellbeing, 2016. URL http://unistudentwellbeing.edu.au/teaching-practice/examples/
  4. Seven Learning Concepts
    1. Deep vs Surface Learning,
    2. Extrinsic vs Intrinsic motivation
    3. Taxonomies of knowledge and learning
    4. Characteristics of Adult Learning
    5. Constructivism
    6. Student-centred learning
    7. Active learning
    From: Why learning theory? Seven Key Concepts for University Teaching and Learning, Jill Lyall, ANU Coffee Courses, 2018. URL http://anuonline.weblogs.anu.edu.au/2018/11/14/day-1-why-learning-theory/

Saturday, February 8, 2020

Ten Tips for Educaitonal Instutions Coping with Off-campus Students

Yesterday I was asked for some tips on how to provide courses for online delivery, for students who are unable to get to class due to the Novel Coronavirus. Here are ten tips, added to those previously provided.
  1. This is Not About China: Procedures should not refer to "for China". Instead "students who can't attend campus". By referring to China, this will stress the students and may make officials less cooperative. Also, it may well be that other countries will be affected, as well as domestic students.
  2. Don't stress funding implications: While there may be a cost to educational institutions, the emphasis should be on helping students, not ensuring cashflow.
  3. Educational standards still apply: Government regulators may have indicated that some rules for international students are being waved. However, principles of quality education still apply, as do professional ethics and the law. A university can't graduate someone who the public depends on in a life and death situation, until that person is fully trained, and tested. The training and testing may be carried out differently but must meet the same standards.
  4. Distance Education techniques are available: There is no need to make stuff up. Some educators have been refining techniques for education remotely over the decades. You can draw on this experience, tools, and techniques. There are likely people at your institution trained and experienced in how to do this.
  5. Contact your overseas colleagues: This is not just a problem for Australia, but educational institutions around the world. You may be able to offer help to those in areas impacted, as well as request assistance from others. This would also be useful for reassuring students, family, friends, and government officials in those countries, that your institution is respecting their local customs and laws.
  6. Do not attempt to circumvent national security restrictions: Nations restrict what services work on-line across their borders. It is tempting to circumvent these restrictions to allow your usual online tools to work for remote students. However, this may be considered a crime. Check for alternatives.
  7. Don't handle this as a "special consideration": Large numbers of students are unlikely to be able to attend campus for an extended period. So you need to design courses, materials, and assessments for this. It is not practical, and is unfair, to have each student apply for special consideration of their circumstances. There will still be a need for such special consideration, in many cases, but educational institutions need to be able to offer an alternative routine for most students.
  8. Deliver clear messages: Students, staff and parents will be stressed. This makes it more difficult for them to understand instructions. Also, rumors will spread. Put out simple clear messages. Have senior staff appear, to provide a human face to the institution. This doesn't need videos, a text statement with a photo is sufficient.
  9. Keep domestic students informed: Domestic students need to feel their education is not being neglected, and excessive resources, and special consideration given to international students. Tell all students what is happening, and offer all the new flexible education options, such as online lessons, to them. Offer virtual mentor and tutor programs to domestic students to help their international colleagues on-line.
  10. Look after yourself and colleagues: Online students expect instant replies from administrative and teaching staff 24 hours a day, and can be scathing in their criticism. Give students reasonable expectations of how quickly they will receive a reply and ensure there is sufficient staff to do this. Try to answer common questions via a forum, rather than to each student individually. Do not reply to intemperate language from a student in haste, or anger.

Friday, February 7, 2020

Ten Tips for Quickly Converting Courses for Online Delivery

Yesterday I was asked for some tips on how to provide courses for online delivery, for students who are unable to get to class due to the Novel Coronavirus. Here are ten tips, added to those previously provided.
  1. Don't panic: Courses have been delivered online by universities around the world for decades. This includes the Green Computing Course from Computer Science at ANU since 2009 (also offered by Athabasca University in Canada). ANU has a series of short Coffee Courses on how to do this.
  2. Focus on student communication, not content: I spent three years as a postgraduate international online student, learning how to provide online education from Australia to China and India. While there were frustrations with ebooks and videos, the major problem was the crushing loneliness. Being an online student amplifies all the worries students have at the best of times, and these are not the best of times. We need to give students the sense there is someone out there worrying about them. Also , we need to encourage them to communicate with other students. You can build such teacher-to-student and student-to-student communication in as part of a course.
  3. Learning To Reflect VideoIt is Not About Video:  The easiest, but least important thing you can do, is to convert face-to-face lectures to recorded video. Students like having recorded video lectures available. However, don't waste time making high-quality videos: it makes no difference to learning. Provided students see a still image of you occasionally, you do not have to appear in the video lectures: powerpoint slides with audio are fine. Unedited recordings of live lectures are also okay. For the "Learning to Reflect" module of ANU TechLauncher last year, I used video automatically created from the course notes, with a synthetic voice.
  4. eBook for Green Computing Course
    Text-Based Notes Are Key: Provide students with notes detailing what they have to do, and when they have to do it. Ideally, provide this all at the beginning of the course. You will be sending them reminders and updates, but students feel more secure if they have everything at the start. My preference is an e-book, but the format doesn't matter as much as the student being able to download everything, easily.
  5.  Target Smartphones: Students are increasingly using smartphones for study, so make sure your material is suitable. Learning Management Systems, such as Moodle, now supports smartphones, but make sure the formatting you use does not stop this working.
  6. Prompt students to study: Students will be distracted and so you need to tell them what to do, and when to do it. Don't assume the student can find the relevant material, or resource: give them a link directly to what you want them to look at. You can start with subtle nudges, such as "Thank you to the 49 students who have already completed the quiz", them more direct: "You have until 1 pm Canberra time, to complete the quiz, for 1% of your grade". Peer pressure, and marks, are very effective ways to motivate students.
  7. Rationalize assessment: Consider what assessment you can reliably deliver online, and in what size chunks. Are you providing the assessment for formative purposes, to help with study, for summative purposes (the final grade), or both? Consider simple short assessment items and make them easier to mark. Make use of the automated assessment delivery and marking options built into the Learning Management System. I particularly like short regular quizzes (with questions drawn from a bank at random, to make it harder to cheat).
  8. Provide Asynchronous Communication, Supplemented by Synchronous:  I was an online graduate student of education for seven years. Of thousands of hours of study, only a couple of hours were real-time (synchronous) communication with a teacher or other students. Most communication for online courses is asynchronous: you post a message and someone reads it later. This is partly due to the difficulties of getting people together at the same time. It is also due to inevitable problems with online communication. So focus on the asynchronous posts to forums. These can be supplemented with short video recordings. If you have the resources, then add some live webinars, but record these sessions, for those who can't attend.
  9. Use the tools your colleagues use:  Educational institutions provide learning management systems, video recording, webinar, and other tools for online teaching. There may also be other tools in use by your colleagues. These may not be the best tools, but you can get help with them, so it should be your first choice.
  10.  Look after yourself and colleagues: Burnout of online teachers is very common. Online students expect instant replies from staff 24 hours a day, and can be scathing in their criticism. Give students reasonable expectations of how quickly they will receive a reply and ensure there are sufficient staff to do this. Try to answer common questions via a forum, rather that to each student individually. Do not reply to intemperate language from a student in haste, or anger.

Thursday, February 6, 2020

TARDIS to Record for Off-campus Students

ANU One-Button-Studio
If educational institutions need to record audio podcasts or videos, due to the Novel Coronavirus, they can use a TARDIS, as ABC Radio refer to their small soundproof recording booths. Educational institutions can purchase such booths in kit form, or build one in a small store room, lined with sound absorbing panels. The booth may have recording equipment installed, with easy to operate controls, for a One-Button-Studio: insert a flash drive, push the button, and start talking. But before building a booth, or studio, look around your campus and see if there is one already set up.

Live Video for Tutorials and Laboratory Sessions Remotely

If educational institutions need to conduct tutorials and laboratory sessions remotely for off-campus students due to the Novel Coronavirus, there are video conferencing and webinar tools for this. Before going out and purchasing a tool, first check if your institution already has one, either bundled with your Learning Management System, or from your network suppler. Examples are Echo360, Adobe Connect, Zoom and Big Blue Button.

These tools do much the same thing. You use a web browser, or downloaded app to send live video from a web camera, and whatever is on your computer screen to students. The students can use a text chat window to reply. There can also be quizzes. All of this can be recorded for later replay. Students can also talk, be seen, and perhaps send what is on their screen, but that gets complicated), especially for large classes. You will want to at least have a moderator to look after the chat, while someone else does the talking.

I suggest using these synchronous tools sparingly, as a supplement to asynchronous online learning. That is, students should be able to watch a recorded video, read some notes, do some sort of exercise step, by step, in their own time. There can then be some synchronous (real-time) activities to help them.

What also helps are activities to connect students to each other online and carefully worded announcements from staff, to the class, to give the sense someone is out there. This is all part of the conventional approach developed for distance education over the last few decades.

The "Learning to Reflect" module I designed  is an example of this approach. The students read the notes, watch the videos, do the quizzes, post to forums, and reply to other students, before the live part. The student gets all the notes at the start of the course. There are also suggested regular posts for the tutor.

Online Exams for Students Off-campus Due to Novel Coronavirus

If educational institutions need to conduct examinations for off-campus students due to the Novel Coronavirus, there are tools for this virtual invigilation, or remote online proctoring. The student has to sit in front of their computer with a web camera pointed at them, while they undertake the test. I have not used such a product, but Athabasca University use ProtcorU. I see the occasional grumble from students on the AU support group, but overall it seems to work.

Monday, February 3, 2020

The Role of the PhD in the Modern World

Greetings from the Canberra Innovation Network for the launch of
"PostAc", to help PhD students into non-academic research careers. Professor Keith Nugent ANU Deputy Vice-Chancellor for Research and Innovation, explained that the ANU was redefining the PhD for wider roles outside academia. Professor Elanor Huntington, ANU Dean of Engineering and Computer Science, pointed out she had a Masters of Computer Science as well as a PhD in the more esoteric area of physics. Also she pointed out that the PostAc service will also collect information on PhDs and jobs. Dr Mewburn mentioned they were open to offers from companies, such as LinkedIn, to purchase PostAc.

One option I suggested to Dr Mewburn that PostAc might consider is auto searches. The student may not know what to search for.  Perhaps the tool could read the student's thesis and suggest a job based on that, and whatever else is publicly available on them. This might include hobbies: an academic in a job interview once told me that had no leadership skills, but then mentioned the lead the university alpine climbing team (if you can get people up a mountain, that shows leadership).

PostAcc is a clever hack to overcome a public policy failure. Unfortunately none of the speakers directly addressed this policy failing: universities produce many more PhDs than there are research jobs for. While someone who has an advanced research degree might be able to find a non-research job with PostAcc, it would be better if they enrolled in a degree which suited those jobs. In most cases a coursework masters is a suitable postgraduate qualification for a job. If more specialized skills are needed, then there is the option of a professional doctorate. These are at the same academic level as a PhD (with the title "Doctor"), but the focus is on skills for industry, rather than just research. PostAcc can provide a palliative for the failure to direct students to these more useful degrees, but not cure it.

Reference


Pitt, R., & Mewburn, I. (2016). Academic superheroes? A critical analysis of academic job descriptions. Journal of Higher Education Policy and Management, 38(1), 88-101. URL https://www.tandfonline.com/doi/pdf/10.1080/1360080X.2015.1126896

Sunday, February 2, 2020

Top Five Factors Associated with Student Success

The top five factors associated with student success identified by meta-meta-analysis by Schneider and Preckel (2017), were: 1. peer assessment, 2. self efficacy, 3. teacher preparation, 4. teacher clarity, 5. grade goals. The first of is reasonably clear, if difficult to implement. Having students peer asses is not difficult to set up, particularly if you have a Learning management System, such as Moodle, to look after the administrative details. However, it can be difficult get some students, and many staff, to accept that students can provide quality assessment.

Self efficacy is easy to recommend, but how do you build students’ belief in their ability? The obvious ways are to state clearly what you want them to do, and give them steps along the way (scaffolding). Teacher preparation and clarity seem obvious, but I see many cases where teachers have produced overly complex lessons, and then compound the problem with long complex explanations of the lessons.

I don't quite understand the point on grade goals. Presumably students are attempting to pass their courses, or where some higher level of achievement is needed, to reach that level. What does worry me is where students have set unnecessarily high goals. As an example, some students will ask for a regrade on the basis they are aiming to get a high distinction (80%) for all assessment items. There are ways to counter this with the assessment scheme, for example having just a pass/fail grade for some tasks.

Reference


Schneider, M., & Preckel, F. (2017). Variables associated with achievement in higher education: A systematic review of meta-analyses. Psychological bulletin, 143(6), 565. URL https://www.uni-trier.de/fileadmin/fb1/prof/PSY/PAE/Team/Schneider/SchneiderPreckel2017.pdf