Thursday, July 11, 2013

Higher Education Whisperer on Assessment

For the second year running I received high student feedback scores for the course "ICT Sustainability" I designed and run at the Australian National University. Other lecturers have asked how I did this and for tips on improving their courses. I didn't think I was doing anything exceptional with course design or assessment. But then realized I had spent a year studying for a Graduate Certificate in Higher Education and four years refining on-line course design and delivery. In that time I guess I learned a thing or two. So I thought I would offer some tips on how to improve courses.

The title "Higher Education Whisperer" is inspired by Dr Inger Mewburn's "Thesis Whisperer" blog. The term whispering is from horse whispering: training based on a rapport with the animal, observing behavior and having respect. I have found the same applies to students: you need to see things from the student's point of view, look at what they actually do and treat them as people. A teacher who assumes their students are stupid, lazy and dishonest should not be surprised if they do not inspire effort from the students and get poor feedback scores.

The best way I have found to see things from the student's perspective is to enroll in a course. The obvious course to enroll in is one on teaching, where you can look at the research available on student behavior. It can be difficult for a teacher to come to terms with student behavior, even when confronted by the evidence. The most obvious case is with assessment. Assessment is very important to students, which should be obvious to teachers, but many design courses with assessment as an afterthought.

The first thing to look at to improve a course is the assessment. In a recent case I looked at the assessment for a course which was not popular with students. The first obvious problem was that there was a 60% final examination. Such examinations are very stressful for students (and for staff making them), not useful aid to learning and not an effective way to assess what students have learned. So I suggested reducing the exam to 30%.

Also I suggested moving more of the assessment to the first half of the course (up from 20% to 44%). Staff complain that students don't study until just before the final exam, but if you design the assessment that way, what can you expect? Increasing an early assignment from 7% to 20% provided more reward for the student's effort and the opportunity for this to be a learning exercise not just final assessment. For simplicity I suggested also increasing the second assignment from 13% to 20% to match the first.

The course already had 15% allocated for small weekly assessment items. This is a good way to keep students working and also having them pay attention to feedback provided (as it has a mark attached). But the assessment scheme confusingly had the top 10 of 12 items assessed. I suggested a simple sum of all 12 weeks, increased to 24% (2% per week).

Also it is important that students get their mark and feedback promptly each week for the previous week. This is particularly important early in a course so that students who are not doing well (or not doing anything) get the message: "Shape up or ship out".

The final suggested scheme was:

  1. Weekly Work: 24% (+9%) 
  2. Assignment 1: 20% (+13%)
  3. Mid Semester Exam: 10% (+5%)
  4. Assignment 2:20% (+7%)
  5. Final Exam: 26% (-34%)
Obviously the assessment scheme could be revised further, but these changes should greatly reduce the stress on students (and on staff).

Another change is to make deadline firm. Teachers make the mistake of thinking if they provide "flexibility" with assessment it will be appreciated by students. But if you have rubbery deadlines you will cause confusion, stress and a perception of unfairness. If there is a deadline for an assignment, then make it firm: students who do not submit on time get zero marks. Obviously there needs to be provision for special circumstances, such as due to illness. But this should be exceptional, not routine.

Particularly when training professionals, on whom the lives of the community depends (such as doctors, engineers and computer programmers)  deadlines matter. A professional who does not learn to deliver work on time is a danger to the community.

ps: I will be speaking on "ICT trends in Education" at the Australian Computer Society in Canberra, 12 November 2013.

No comments:

Post a Comment