Reframing Teaching to Use Flexible Classrooms

Last Monday I attended "Work Integrated Learning - a disciplined approach" at University of Canberra. This was based on a recent paper by Ruge and McCormack (2017) on constructive alignment of the assessment for a course at University of Canberra. Two recent case studies in EduCause point to the need for new classroom designs, plus course design and the training of staff in how to do this (Morrone, Flaming,  Birdwell, Russell, Roman & Jesse, 2017).

New ANU Buildings
(artists' impression).

Universities are building new flexible learning classrooms (such as in the new teaching building at ANU). These typically have a flat floor, and tables for small student groups, with display screens on all walls. However, to effectively use these spaces course redesign and teaching staff training is required. There is no point in moving out of lecture theaters if the "lecturer" gives hour long non-interactive monologues in the new "flexible" room. There is no point in putting the students in groups and link them with tech, if the primary assessment is an end of semester individual paper based examination.

The "lecturer" needs to be trained how to teach without lecturing and the assessment redesigned to use group and project based work. New classrooms can then be used effectively and the learning aligned to vocational requirements beyond the classroom. For more on this, see my free e-book "Digital Teaching"

References

Morrone, A., Flaming, A., Birdwell, T., Russell, J., Roman, T., & Jesse, M. (2017, December 4) Creating Active Learning Classrooms Is Not Enough: Lessons from Two Case Studies, EduCause. URL https://er.educause.edu/articles/2017/12/creating-active-learning-classrooms-is-not-enough-lessons-from-two-case-studies

Ruge, G., & McCormack, C. (2017). Building and construction students’ skills development for employability–reframing assessment for learning in discipline-specific contexts. Architectural Engineering and Design Management, 1-19. http://www.tandfonline.com/doi/abs/10.1080/17452007.2017.1328351

Using Machine-Learning to Identify Which Work is Not by a Student

Amigud et al. (2017) tested using machine-learning to identify what work is not by a student. The AI system is "trained" using work submitted by the students and then can spot work which does not match the writing style of an individual. They claim 93% accuracy, compared to human instructors being able to identify work not by the student only 12% of the time. This approach has the advantage that it can identify where work was contracted out to someone else, as well as where it was copied from a public source. 

When faced with the suspicion that some students are cheating, it is tempting to ask for a supplementary face-to-face examination for that student. However, I suggest designing assessment so that all students are required to present a consistent body of work throughout their course.

Large assessment items can be replaced by multiple interlinked, scaffolded tasks. This will help students learn, while testing they have acquired required skills and knowledge. As a byproduct, this will make cheating much more difficult, as a student can't just contract out one major assignment as it will not be consistent with the student's other work.

As an example, in "ICT Sustainability", students have to answer weekly questions. The student then uses the answers in the two major assignments, which are each in two parts. This makes it difficult for a student to contract out one assignment, as the content has to be consistent with their prior work on the same topic. There are also weekly automated quizzes on basic knowledge, drawn from a question bank (to make cheating harder). To be eligible to pass, the student must get at least 50% for the weekly work and 50% for the assignments, so they can't pass by doing well in just one assignment.

Also I use progressive assessment to identify those students in the first few weeks who are having difficulty, so they can be offered help and counseling. These students will be less likely to cheat, as they can get help and they know they are under closer observation every week. I designed and refined this approach to assessment as part of my graduate studies in education.

Reference

Amigud, A., Arnedo-Moreno, J., Daradoumis, T., & Guerrero-Roldan, A. (2017). Using Learning Analytics for Preserving Academic Integrity. The International Review Of Research In Open And Distributed Learning, 18(5). doi:http://dx.doi.org/10.19173/irrodl.v18i5.3103

Australian University Students Allegedly Cheating with iPads

I was interviewed briefly by Caroline Winter at ABC Radio's The World Today, about university students allegedly using tablet computers to cheat in examinations: "University investigates allegations of cheating" (19 Nov 2013, 2:44pm AEDT). This was a follow-up to the item "Medical students allegedly cheated on Adelaide University exam about human reproductive health" (19 November 2013, 8:01am AEDT).

Students are alleged to have screen captured from an examination paper and left them for the next group of students. Apparently the iPads were supplied by the university and not reset between examination candidates. This seems a curious way to administer a computer based examination and such an examination seems an odd way to do assessment.

For the last year I have been assisting another university to convert a paper based clinical assessment process from paper to computer. In that case the doctors supervising students in hospital use the system to report on student progress. This uses HTML 5 based web forms so that it will function with any type of tablet computer (Google Android, Apple iPad, Microsoft Surface or any web browser).

There are also ways to administer examinations on a computer, but you obviously need to restrict the student's access to material not permitted in the examination and have a real, or virtual, invigilator. You can't simply give the student any old computer (or their own) and let them sit the examination unsupervised. The exception would be "take home" examinations which make up a small proportion of the assessment, or none at all and are formative to help with learning, rather than summative.

There are ways to make cheating harder with screen based tests, such as choosing questions at random from bank and altering quantities in numerical ones.

How, What and When of Improving Student Satisfaction

Recently I was asked for the how, what and when of improving the student's satisfaction with courses:

1. How: Focus on assessment, course materials and feedback. Student activities are important, lectures are not.
2. What: Small regular assessment items, at least two large assignments, minimal exams. On-line course materials, with optional lectures. Feedback with assessment results.
3. When: Assessment every week or two, all course notes and assessment tasks provided before the course starts, mark and feedback within a week of small assessment submission.

SUGGESTIONS FOR A TYPICAL COURSE

Here are some suggestions I provided for one course (some details omitted, to protect the course designer's feelings):

Better course materials and more progressive assessment would improve course completion rates. It does not matter if a Learning Management System (LMS), or something else, is used to deliver course materials, provided they are well designed. Ordinary web pages and Streams are a viable alternative to Wattle. Using Wattle for all courses would not be a significant improvement.

The assessment, course notes and staff contact details should be most prominent on course websites. Details of lecture times should not be so prominent, as students place a low value on lectures and if staff try to make students attend they sound desperate. All lecture notes and assessment items should be provided to student at the start of a course and no changed made, except where errors are found.

Just as there is an examiners meeting at the end of semester, I suggest a meeting at the beginning of the semester, where the person in charge of each course presents their course material and assessment design to their peers. There would then be peer pressure to produce clear, complete and consistent work.

Some points:

* TRIM COURSE DESCRIPTION:

The Course Description is a little long. I suggest cutting it by one quarter to just two paragraphs. The Learning Outcomes are far too long and I suggest cutting them by half to about four points. They should be derived from some accepted skills standard, such as (for ICT) the ACS Body of Knowledge, Skills Framework for the Information Age, or similar and be explicitly covered in parts of the course notes, workshops and assessment.

The Indicative Assessment description is misleading. It says there is almost an equal weighting between Continuous Assessment and Examinations. But the assessment page indicates a large end of semester examination a small mid semester exam and a couple of small assignments.

An alternative assessment strategy would be 20% for each of the mid and end of semester exams, mid and end of semester assignments and 20% total for a short lab exercise or quiz each week. However, consideration should be given to eliminating the conventional examinations entirely, as they have little value as an assessment tool.

I suggest having firm deadlines for all assessment: late submission should result in a mark of zero, unless special consideration is granted. Students should be encouraged to submit a draft of each assignment at least three days in advance, with failure of the student's computer system (or the institution's computer system) on the day the assignment is due, not being  grounds for special consideration (nor would an illness lasting less than three days).

Students who have a propensity to submit work late would quickly learn this was not going to be tolerated and change their ways (I find this very effective in my Green ICT course).

* MORE ON STAFF AND ASSESSMENT ON COURSE WEB PAGE:

The web page for the course has a photo which appears to have nothing to do with the topic of the course. Students like to know who their teachers are, so I suggest replacing this with ones of the teaching staff, with links to their bios. In preference to email, provide Moodle Dialogue or similar. This keeps all correspondence with the student in one place, so that all teaching staff can see it.
The web pages emphasize attending lectures, but these are the least useful part of a course (and if you have to beg students to attend lectures you sound desperate). I suggest providing more details of the assessment up front, which is what the students consider important.

Labs are a useful educational technique, so I suggest communicating this to the students by making all labs assessed. Students can submit material on-line and be marked for it.

* COURSE NOTES NOT LECTURE SCHEDULE:

The lecture schedule should be replaced with a course ebook. This would contain all the notes for the course, provided to the students at the start of the course. The subtitle "to be adapted and refined throughout the course" should be removed, as this says to the students that the course has not been properly designed.

Lectures could continue to be held, but it should not be assumed that students attend. Consideration could be given to using a workshop format with min-lectures, but this would require a TEAL type room, large enough to hold all students.

* COURSE BOOK FOR WEEKLY LABS:

I suggest the Weekly laboratories material be inserted in the course notes under the appropriate topic headings. Having many separate documents creates confusion.

* ONLINE PEER ASSISTED LEARNING:

Peer Assisted Learning (PAL) sounds like a good idea, but I could not understand from the web page how it was supposed to work. Also this appears to be only a face-to-face group. I suggest it be extended with on-line forums.

* CLARIFY DIFFERENCES WITH POSTGRADUATE COURSES:

I had difficulty understanding how the postgraduate version of the same course was supposed to work. It seems to add more work for postgraduates. That suggests that either the undergraduate version has less than the usual amount of work or postgraduate has too much.