Monday, November 30, 2020

Turnitin’s Gradescope Workshop at ASCILITE 2020 Conference

Greetings from the ASCILITE 2020 Conference Online, where I am taking part in a Turnitin Gradescope Workshop. Two papers I helped with are being presented tomorrow, but first I am brushing up my grading skills. Anu had TurnItIn and I have used their GradeMark tool, but that is much the same as online assignment marking with Moodle (apart from integration with TurnItIn's plagiarism detector). Hopefully Gradescope does more than this.

I worry that online and semi-automated grading tools may make the fixation students and teachers have with grades worse. These tools look objective and analytical, which grading is not. I am reasonably confident that I can decide that a piece of work is below, at, or above the required standard. However, I am not sure I can really say where the work sits on a seven point scale, let alone a mark out of 100.

What would be useful would be a way to evaluate the quality of grading, especially where multiple graders are used.

Despite being a vendor provided workshop, there was a minimum of sales pitch. However, it would have been useful to compare Gradescope to commonly used online assessment tools, such as the Quiz, Assignment and Workshop tools in Moodle.

Gradescope has something called a "dynamic rubric". This allows the grader to create a rubric during the marking. I have some difficulties with this idea. First of all conventional wisdom, and natural justice, say that the student should be provided with the rubric before the assessment, so they know how they will be graded. If the rubric is made up during marking, the student can't have had it before. Next, this removes the discipline from the assessment designer to think carefully about what they are trying to assess and how it will be marked. There could also be a lack of consistency if the rubric changes during the grading. However, if the grader has not been provided with any rubric, this would be better than nothing. It would also be useful where a problem is found with a question, and the expected answer is wrong.

Gradescope provides statistics overall and on individual questions, much the same as other online marking tools. What I had hoped for were ways for evaluating the quality of marking, between multiple graders, but there doesn't seem anything for this. There was also mention of the use of AI, but no details in the workshop. Overall Gradescope seems to have the features I would expect in any grading tool.

Before creating a complex marking scheme, educators need to stop and consider the purpose of the assessment and the cost and suffering they are causing students. In most cases school and university assessment is intended to find if the student's skills and knowledge are to a required standard. We want citizens and workers who can function in a modern society. There is no need to rank all the students, or mark them to within 1%, for this: pass/fail is sufficient.

Universities might like an easy way to rank applicants, but is that really the job of schools and are school assessments really much use for that? Similarly, university academics may want to use assessment to select students for advanced research courses, but few students will be undertaking these advanced courses, so is it worth the cost of administering this assessment for all students? Outside university, no one much cares about university grades: as long as you passed, that is all that matters.

Saturday, November 28, 2020

Mobile Learning Special Interest Group


The ASCILITE Mobile Learning Special Interest Group has produced an eight minute video to explain what they do for the ASCILITE 2020 Conference next week. This was made in one take via Zoom. It is a bit more lively that the version I made by pasting the text from the website into a text to speech system:



Friday, November 20, 2020

ASCILITE 2020 Conference Online 30 November

The ASCILITE 2020 Conference will be the organisation's first online. This is hosted 30 November and 2 December by UNE. This will be my first ASCILITE and I am an author on two joint papers: 


  1. A collaborative design model to support hybrid learning environments during COVID19 by Cochrane, Birt, Cowie, Deneen, Goldacre, Narayan, Ransom, Sinfield & Worthington (Day 2, Session 4, Stream A, 11:30AM).
  2. A mobile ecology of resources for Covid-19 learning by Narayan, Cochrane, Cowie, Birt, Hinze, Goldacre, Deneen, Ransom, Sinfield and Worthington (Day 2, Session 4, Stream C, 11:30AM).
These have been scheduled for parallel sessions at the same time at the conference. Normally this would involve a sprint down the corridor (in  some conference centers this might be half a kilometre and up several floors). Online it should be a couple of clicks, or as there are so many authors we can just divide up).

This is my first ASCILITE (I did attend a concurrent program at ASCILITE 2019 Singapore). The one and only time I submitted a paper previously, the reviews were so witheringly condescending that I gave up the whole idea of submitting a paper, or attending. The reviews of the joint papers this time were much more positive, indicating the fault was not with the reviewers, but my previous authorship. ;-)

The practice at face to face conferences is to keep on to time, not start early, so people can catch a specific presentation. However, I like the idea that you pick a session and have to stick with it. With four parallel steams this can be difficult. Here are my picks:

1:30 — 3:00 pm Breakout Zoom Session 1

Session 1 - Stream A


Bend me, stretch me: connecting learning design to choice - Carmen Vallis and Courtney Shalavin
ID: 32


Rising to the occasion: Exploring the changing emphasis on educational design during COVID-19 - Amanda Bellaby, Michael Sankey and Louis Albert
ID: 64


Excellence in design for online business - Annora Eyt-Dessus and Leonard Houx
ID: 81


Session 1 - Stream B


A spectrum of assessments - Rina Shvartsman and Stephen Abblitt
ID: 35


Learning from COVID-19 to futureproof assessment in Business Education - Sandra Barker, Harsh Suri, Brent Gregory, Audrea Warner, Amanda White, Vivek Venkiteswaran and Una Lightfoot
ID: 70


Modelling the impact of alternative educational qualifications on the New Zealand higher education system Stephen Marshall
ID: 73


Session 1 - Stream C


Dealing with Diversity: Factors discouraging participation of Māori and Pacifica females in ICT education - Scott Morton, Petrea Redmond and Peter Albion
ID: 4


Active learning in the time of the pandemic: Report from the eye of the storm - Iwona Czaplinski, Christine Devine, Martin Sillence, Andrew Fielding, Oliver Gaede and Christoph Schrank
ID: 11


Investigating links between students’ agency experiences in digital educational interactions, participation and academic performance - Maria Hvid Stenalt
ID: 82


Session 1 - Stream D


Pecha Kucha

D1 3 Alagu Sundaram 

Nurturing Self-Directedness

in Learners in a Fully Online

Module


3:00 — 3:30 pm— Break —3:30 — 5:00 pmBreakout Session 2

Session 2 - Stream A


Reimagining IL teaching and learning during the COVID-19 pandemic:  Research and evidence-based practice skills training redesigned for online delivery. - Fiona Jones, Abigail Baker, Raymond A'Court and Jo Hardy
ID: 56


Learning from a rapid transition to emergency remote teaching: Developing a typology of online business education designs - Elaine Huber, Celina McEwen, Peter Bryant, Matthew Taylor, Natasha Arthars and Henry Boateng
ID: 77


Promoting student engagement and preparation in flipped learning’s pre-class activities – A systematic review - Jessica Shan Mei Yang
ID: 9


Session 2 - Stream B


The benefits of creating open educational resources as assessment in an online education courseEseta Tualaulelei
ID: 16


(e)Portfolio: a history - Orna Farrell
ID: 13


Assessing esport candidacy for critical thinking education - Ger Post and James Birt
ID: 20


Collaborative approach and lessons learnt from transitioning to remotely invigilated online examinations - Vinh Tran, Justin Chu and Jasmine Cheng
ID: 48


Session 2 - Stream C


A rationale for using interactive and immersive technology to support practical skill development of online OHS education - Elise Crawford, Frank Bogna, Aldo Raineri and Ryan L. Kift
ID: 59


The post-pandemic blended university in the time of digitisation Philip Uys and Mike Douse
ID: 6


Predictors of students’ perceived learning in off-campus learning environment: Online interactions are not enough - David Kwok
ID: 23


Emergency responses to teaching, assessment and student support during the COVID-19 pandemic - Alison Reedy, Kalie Carmichael and Oriel Kelly
ID: 67


Session 2 - Stream D


Pecha Kucha

For a full list of Pecha Kucha presentations click here



Session 3 - Stream A


Maybe It's Us: Imagining Organisational Learning Design - Sarah Thorneycroft
ID: 41


Three arrows models in the developing of new digital learning experiences - Pablo Riveros Riveros, Mika Tamura Tamura and Jin Tanaka Tanaka
ID: 51


Should we care about what the students do? Challenging how we design for online learning Bettina Schwenger
ID: 52


The Value of Design Patterns in Designing Teaching in Online Settings - Steven Warburton and Mark Perry
ID: 55


Session 3 - Stream B


Development of a rubric to assess student participation in an online discussion board - Elizabeth Ware
ID: 58


Online Supervised Exams: Entering the 4th Year at UNE Jennifer Lawrence and Kylie Day
ID: 63


Transforming Assessment – Critical reflections around resolving tensions between assessment for learning and of learning - Sabina Cerimagic and Priya Khanna
ID: 78


Using FeedbackFruits to enhance student learning: Scaling for transformative implementation - Chris Campbell, Lenka Borer and Sheila McCarthy
ID: 88


Session 3 - Stream C


Role of Social Interactions during Digital Game-based Learning in Science Education: A Systematic Review - Pey-Yng Low
ID: 18


Investigating the characteristics of MOOCs: A case study - Jennifer W.M. Lai, Matt Bower, Yvonne Breyer and John De Nobile
ID: 24


Revisiting the Intelligent Book: Towards Seamless Intelligent Content and Continuously Deployed Courses - William Billingsley
ID: 79


Session 3 - Stream D


Pecha Kucha

For a full list of Pecha Kucha presentations click here


11:00 — 11:30 am— Break —
11:30 — 1:00 pmBreakout Zoom Session 4

Session 4 - Stream A


Making engaging online videos: What can higher education teachers learn from YouTubers? - Neil Cowie and Keiko Sakui
ID: 7


Process not product: negotiating innovative interdisciplinary honours outcomes - Grant Ellmers and Chris Moore
ID: 19


A collaborative design model to support hybrid learning environments during COVID19Thomas Cochrane, James Birt, Neil Cowie, Chris Deneen, Paul Goldacre, Vickel Narayan, Lisa Ransom, David Sinfield and Tom Worthington
ID: 36


The implementation of H5P in an open-access first-year human physiology subject to improve student engagement and perception of learning. - Deanna Horvath, Amy Larsen, Stuart James and Victor Renolds
ID: 61


Session 4 - Stream B


How do we value academic time? Mark Schier
ID: 37


Strategies for improving use of text-matching software by staff Miriam Sullivan and Miela Kolomaznik
ID: 46


Rapid response to supporting learning and teaching: A whole of university approach - Chris Campbell and Simone Poulsen
ID: 54


Session 4 - Stream C


An Interactive Virtual Reality Physics Instructional Environment based on Vygotskian Educational Theory - Michael Cowling and Robert Vanderburg
ID: 25


A mobile ecology of resources for Covid-19 learning Vickel Narayan, Thomas Cochrane, Neil Cowie, James Birt, Meredith Hinze, Paul Goldacre, Chris Deneen, Lisa Ransom, David Sinfield and Tom Worthington
ID: 40


Applications for Virtual Reality Experiences in Tertiary Education - Ghaith Zakaria and Sonia Wilkie
ID: 68


Creating a digital learning ecosystem to facilitate authentic place-based learning and international collaboration – a coastal case study - Elisa Bone, Richard Greenfield, Gray Williams and Bayden Russell
ID: 84


Session 4 - Stream D


Community Mentoring Program presentations

1:00 — 2:30 pm— Lunch + extra activities (SIGs - Special Interest Groups and Zoomba) —
Read extra activity information here
2:30 — 4:00 pmBreakout Zoom Session 5

Session 5 - Stream A


Understanding Learning Analytics Indicators for Predicting Study Success - Dirk Ifenthaler and Jane Yin-Kim Yau
ID: 1


Content analytics for curriculum review: A learning analytics use case for exploration of learner context - Leah P. Macfadyen
ID: 2


Peering into the crystal ball of the disengaged: What happens to students that do not submit an early assessment item? - Kelly Linden, Neil Van Der Ploeg, Ben Hicks, Madeline Wright and Prue Gonzalez
ID: 44


Widening the net to reduce the debt: Reducing student debt by increasing identification of completely disengaged studentsNeil van der Ploeg, Kelly Linden, Ben Hicks and Prue Gonzalez
ID: 45


Session 5 - Stream B


On effective technology integration in accounting and business education - Marina Thomas and Valeri Chukhlomin
ID: 10


Supporting the transition to online teaching through evidence-based professional development - Darci Taylor and Joanne Elliott
ID: 22


Immersive professional learning to foster technology-enabled peer-review - Gayani Samarawickrema and Olga Gavrilenko
ID: 57


Writing Analytics Across Essay Tasks with Different Cognitive Load Demands - Eduardo Oliveira, Rianne Conijn, Paula de Barba, Kelly Trezise, Menno Van Zaanen and Gregor Kennedy
ID: 39


Session 5 - Stream C


2019 Innovation, 2019 Emerging Scholar and Community Fellow Award Winner presentations




Wednesday, November 18, 2020

Have students provide a photo to help instructors feel good online

There has been much written about the stresses caused for students having to study online this year due to COVID-19. There has been less on the effect on teaching staff. One of my students is researching how to make online learning better and came across some work indicating students were happier with video conferencing than those teaching them. It occurred to me that some of the techniques online teachers are trained in to make them appear more human to the students might also be used to make the students appear more present to their teacher.

One of the basic techniques is to have a profile photo. This appears in a video conference if the camera is off. I use a still image taken by my web camera showing me dressed typically for a video conference with the usual background. When not speaking in a video conference, I suspect than many people don't notice I switch from live video to the still image. 

Teachers express frustration about talking to a Zoom screen showing rows of black rectangles. It is tempting to tell students to turn on their cameras, but there are many reasons they may be unwilling or unable to. I use a wireless broadband connection which sometimes can't support video. Sometimes I am in a shared office with people who do not want to be on camera. So I understand why students do not want to turn a camera on. 

On a visit to Cambridge University, some years ago, I came across research which suggested people relate to a computer as if it was human, if it has a face. This did not need to be high quality video, a still image, or even a smile face would do :-) 

So I suggest that having students include profile photos on their video would help teachers feel they are talking to people. This could be implemented by asking students, or by having the Learning Management System provide the official student photos to the video conference system by default.

Blend and Flip for Teaching Communication Skills to Final Year International Computer Science Students

My paper "Blend and Flip for Teaching Communication Skills to Final Year International Computer Science Students" has just been published. This was presented at a conference in Jogjakarta last year, but I guess COVID-19 held publishing up. In this I describe using a "chunky" approach to blended learning. The notes for the presentation are also available.

Abstract: In addition to technical knowledge, graduates in computing and engineering disciplines are expected to have communication skills, and the ability to undertake lifelong learning. These skills are difficult to acquire using conventional lecture and tutorial based teaching. Final year international graduate computer science students at the Australian National University, College of Engineering and Computer Science, were found to have particular difficulty when asked to write about their learning. In response, lectures were replaced with online exercises, group workshops in a new purpose-built flat floor classroom, and peer-assessed progressive assessment. This approach was trialed with eighty students in 2019. Preliminary results indicate students performed at least as well as with conventional lecture-based instruction.

What I did not mention in the paper, but turned out very useful, was that in 2018 I had designed this to allow rapid conversion from blended to pure online mode, if students were unable to get to campus (which I identified as an issue in 2016). So when COVID-19 struck in early 2020, it was only necessary to replace the face-to-face workshops with video conferences. Everything else, including assessment, was already online and so needed no changes. 

The difficult question will be what to do when some students return to campus. It would be possible to again offer face to face workshops. However, not all the students are ever going to return to campus, so should they be able to participate in hybrid or parallel modes? That is, should the face to face class be linked online to the remote students, or a separate class run for the online students?

In the past I assumed that the hybrid mode would disadvantage the remote students. However, this week I attended a face to face class where the instructor kept apologizing because their materials and presentation technique were for online delivery. It may be the students in the room who are disadvantaged.

Reference

Worthington, T. (2019, December). Blend and Flip for Teaching Communication Skills to Final Year International Computer Science Students. In 2019 IEEE International Conference on Engineering, Technology and Education (TALE) (pp. 1-5). IEEE. https://doi.org/10.1109/TALE48000.2019.9225921

Tuesday, November 17, 2020

Designing University Courseware for Micro-credentials and Courses So Students Do the Work

The Australian government has encouraged universities to offer shorter qualifications to provide vocational skills which are in demand, in areas such as cyber security. This includes certificates (taking about 12 weeks full time study) and microcredentials (around 3 weeks, or less). This requires getting students focused on study quickly, as there is not time to waste on an orientation week, or students wandering around finding out what to do. Also learning designers can't afford to spend as much time on a short program as a lone one.

When I first tried teaching online, a decade ago, I tried to emulate a face to face lectures and tutorials. This is a mistake many first time online teachers make, as these are the only formats they are familiar with. Lectures and tutorials are not a very effective teaching techniques, offline or online. 

The approach I developed over ten years teaching online was to design first for pure asynchronous online delivery, then add synchronous components, which could alternatively be delivered face to face. This approach could be extended to micro-credentials which could then be used for conventional courses in a certificate.

What every teacher has to keep in mind is that the task is not to work very hard, it is to get the student to work very hard. Some teachers respond to students who don't seem to be learning by talking at them more, which makes the situation worse. Instead they have to set goals for the students. The only way I have found to do this is with assessment: if you want a student to do something, give them marks for it.

So we first have to set learning objectives for the microcredential or course (preferably based on external widely recognized standards, not something developed at the university), then translate these into assessment tasks, then provide activities for students to learn the skills and knowledge required and lastly provide materials to help. Most of the budget of student learning time should be taken up with individual and group activities, not time spent in "class". Most of the budget for instructor time should be for helping student with their assessed tasks, not giving presentations or leading discussions.

Friday, November 13, 2020

Designing Micro-credentials for an Australian University

Greetings from the Designing Micro-credentials and Professional Short Courses at the Marie Reay Teaching Centre at the Australian National University. This is my first time in a classroom since March, either as a teacher or student, due to COVID-19. 

It was exciting to be in a room with colleagues (a very large room with a lot of space and other COVID precautions). The instructor took some time to get used to teaching in a classroom, using online terminology and techniques (which was amusing).

The ANU introduced a Micro-credentials Procedure (5 October 2019). The term has no set definition so ANU has defined the specific way it sees these short qualifications being used, for postgraduate studies which offer credit to a degree, over days or weeks, not months. ANU will be using the same Moodle platform as used for courses, by customized for micro-credentials.

A micro-credential could be delivered in a classroom, blended, or fully online. However, the assumption is that most will be at least partly, if not entirely online. The coursers will use a mix of synchronous and asynchronous approaches (that is some in real time with the students in the room or a video conference online at the same time, but mostly the student studying materials in their own time).

The key issue I see for success with micro-credentials is avoiding attempting to duplicate a traditional university campus academic experience. This is because, in part, that experience is a myth and much of that which is real is not suitable for mature students who would undertake a micro-credential. Traditional university teaching has lectures and tutorials, where the teaching staff do most of the talking and students spend their time listening (or in fact not listening). In reality most students did not attend the lectures and if required to attend tutorials do not actively participate. What a micro-credential, or any good university course, needs to do is get students working

At the workshop we were shown several online tools designed to get studnts active and communicating with each other. Examples were a simulated posit note wall using mural.co , a quiz using kahoot.it,  and voting using menti.com. I did not find any of these particularly useful or engaging. These will be particularly unappealing to more mature learners, assuming they can get the interface to work, and their poor accessibility features do not prevent use. One of the participants at my table mentioned the use of Slack.com, and I suggest that is more the type of application which will be of value for advanced learners. The emphasis should be on students interacting with each other on substantial projects over time, not quick exercises driven by an instructor.

The particular micro-credentials I have in mind are tech-teaching. This would provide an introduction to teaching for those in technology fields, such as computing. These could be university tutors, or those doing small amounts of training in a workplace. This would be aligned with the Skills Framework for the Information Age SFIA, such as Learning delivery ETDL.

One point is that with this type of educational offering meeting other course participants is as important as the content. This may require more than the usual icebreaker activities. 

The ANU staff introduced aspects of the micro-credentials policy, as well as how to go about designing a course, with leaning objectives for a target audience. How to design a course I am familiar with, having been formally trained to do this. However, micro-credentials have their own requirements, being short and for a very specific audience. It occurs to me that the ideation process used in product development could be of use. 

Micro-credentials introduce restrictions on the educational design and delivery process, beyond those of conventional courses. My practice has been to design pure online courses, and then add face to face elements. I will try adding a third step: first designing a set of microcredentials, which can be combined to create a conventional course, run in blended, or online mode. This should be easier than starting with a conventional course and then trying to modify it to be a set of microcredentials.

One area where I suggest more guidance is needed on micro-credentials is timing and structure. A typical Australian university student would be expected to do about 40 hours study a week for 12 weeks. Microcredentials are meant to be short, however, university lecturers have a tendency to try to cram too much into a course.

It may take several attempts for any organisation to come up with a workable strategy for micro-credential. I suggest treating the first version as a pilot.

Thursday, November 12, 2020

What is Learning?

Greetings from the IEEE Standard for Learning Metadata Meeting (P2881) being held online. This is my third meeting. The group is still working its way through what the new version of the standard needs to do, which other standards about learning don't already do. This can be very frustrating, defining the definition of definitions, and asking questions such as "What is learning?", "Is a competency a learning Object?". But also some more practical questions: "How much are existing metadata standards actually used?". These are important questions, as he world's students have been forced to use e-learning this year due to COVID-19 and there is a need to bring some order to the resulting chaos.

However, the mechanics of the meeting process still cause me some difficulties. With summer time, the meeting has moved from 4:30am to 6:30am in Canberra, which is an improvement. But I am still not at my best with metaphysical questions at this time in the morning. ;-)

The meeting is using WebEx, which I have not used for years and again had to dial in for the audio. There does not appear to be a Linux client, so I am using the web interface. This is far slower and less responsive than Zoom. It is like using video conferencing from ten years ago.