Implementing Large Scale Electronic Examinations at ANU

Greetings form the famous room N101 at the Australian National University in Canberra, where Professor Steve Blackburn is speaking on "Electronic Exams" (video now available).

Steve pointed out that the ANU Research School of Computer Science have been conducting on-line examinations for a decade. This has focused on a Linux environment for testing computer programming, but Steve suggests the procedures could be applied more broadly for other disciplines.

Steve pointed out that Moodle had also been used for on-line examinations. I conducted such an examination in a RSCS lab in 2007 for a short course on Electronic Document Management. This was a small group and one invigilator was used to ensure the students only used the specified websites. As Steve pointed out, securing Moodle for a large scale examination is problematic.

UTas/UQ have a project for electronic examinations on BYOD devices "Transforming Exams". Steve mentioned some of the complexities in this approach.

Steve decided to use a browser based electronic examination which mimics the look of a paper based examination. The web pages use persistent storage, javascrpit, CSS and other standard web features.

A sample of the exam is available. The examination web page has the color and layout of a paper examination form at ANU. Interestingly, while intended for a specific desktop environment, because the web page uses standard and straightforward code, it displays reasonably well on a mobile device. Also an accessibility test reported only one known problem (title missing). This suggests the examination would be suitable for those with a disability.

One interesting point is that the 2015 electronic examination was designed based on the 2014  paper one. Steve commented that students complained that students found the electronic version too hard and too long, compared to the paper equivalent. Also students had lower marks for the electronic than paper examination. He suggests this may be because students would sketch on paper, whereas they are encouraged to immediately work on the detail in the electronic form. One of the audience commented that the expectations of examiners may be higher with electronic examinations, as the limitations of handwriting are not there. Steven suggests this needs some rethink of question design and practice examinations.

Steve commented that this was being presented as an example of practice not as something claiming to be groundbreaking research. Clearly the audience did not agree with this and suggested it was worthy to be shown at the Australasian Computer Science Week of Conferences (ACSW 2016), being hosted at ANU, 2 February 2016.


One issue electronic examinations raises is: why have examinations at all? If the students are using on-line systems day-to-day, then why not use this for assessment? One issue raised by the audience are the accreditation requirements of external agencies. However, I suggest there are ways to satisfy these requirements and in some cases the assumed impediments are not real. As an example, in 2009 when I set out to design a fully on-line course at ANU, some commented that this would not meet the university rules for accession, as every course had to have an "examination".  A check of the rules showed this was not the case: courses do not have to have examinations and where there are examinations these need not be on paper.