Penny Wheeler, Amanda White, & Tom Worthington, at the TEQSA Masterclass
Greetings from the TEQSA Masterclass Workshop on Contract cheating detection and deterrence, at the National Museum of Australia in Canberra. This builds on the online Detecting Contract Cheating course from TEQSA. The good point of the course and workshop is that they raise uncomfortable questions for university academics. But I am not so sure the answers presented are the correct ones. Rather than a "Law & Order - Cheating Investigation Unit", I suggest universities should focus on quality assessment to deter cheating, and structure it to prevent cheating students from ever graduating.
I had some concerns about the legalistic approach in the online module leading to discrimination against vulnerable students, and even more so in the face to face workshop. Also TEQSA clearly envisages having centralized cheating investigation units.
There is also discussion of what are plea bargains, where students receive a lesser penalty, for admitting cheating. However, if there are to be professional investigators, I suggest they will need to be formally qualified. Also those making the decisions will have to have formal training how to make administrative decisions (similar to the Administrative Decision Making course offered by the Australian Public Service Academy). It does not seem reasonable to deny a degree to a student, based on evidence collected and decided by people who do not even have a relevant TAFE certificate.
Having done an online course, and a face to face workshop, with no form of assessment is not sufficient. Government investigators at least have a vocational certificate (such as a Certificate IV in Government Investigations offered by the Canberra Institute of Technology), and something similar is required for university investigators. Similarly, those serving on tribunals have to have some form of training.
I suggest that universities should look to their vocational education colleagues, who are much more used to dealing with nationally standardized regulations.
My preference would be to decriminalize cheating, much the way is being done with illicit drug use in Canberra. Students would be given a failing grade where they cheated, and given compulsory training.
A related issue which came up during the workshop is educational institutions complicity in visa fraud. This where someone obtains a student visa, in order to be able to work in Australia. Those who accept such visas are at risk to being exploited by organised crime. It came as a surprise when I discovered the universities were processing visa applications on behalf of government. It would be tempting to return the function to government staff. But those staff may not have the specialized knowledge of education. It may be that educational institutions, particularly smaller ones, should form consortia and pool staff.
The investigation processes illustrated by TEQSA depended on technical means involving Learning Management System logs, document metadata, Internet Protocols and the like. Such procedures will only catch relatively unsophisticated students, and commercial cheating services. It might be useful to build technical means into the university systems to deter such activities. As an example, the tool could ask the student when they login from a different IP address to explain where they are. Similarly when submitting a document, the system could check the metadada on the document for consistency. The contract cheating company could, of course, coach the students to provide plausible answered, but otherwise only the students who can't afford to pay for good cheating will be caught.
The course and workshop focused on contract cheating. They were designed before generative AI came to public attention. It is unfortunate TEQSA did not anticipate what was coming and so was required to instead react. They are now trying to catch up. An AI topic which has not yet been addresses is AI for teaching and assessment. I took part in a workshop on building an AI tutor chatbot in 2018, which had some interesting implications.
I am doing the "Detecting Contract Cheating" course from TEQSA. In Module 2 "Supply and demand" under "who are cohorts likely to cheat" lists "Students who speak a language other than English". That is, if you speak a language other than English, the claim is you probably cheat. But isn't that most of the people in the world? I find it hard to believe that English speakers are inherently honest, or not speaking English makes you dishonest. The course cites Bretag et. al (2019), but my reading of it doesn't support that conclusion.
However, the course, so far, is mostly good. In particular I like the section on how to deter contract cheating, by providing the students with the help they need. I had to give this some thought, because it is easy to say there is no excuse for cheating. However, if your family, or your entire village, have invested everything they own in your education, if you are having difficulty with the language, you have limited time for study away from family or job (or both), if the universality does offer you any help getting up to speed with the topic, it is very easy to be tempted to cheat.
The course consists of some short videos, which like a typical student I play at high speed and read the subtitles. Curiously, the videos kept referring to "expectations" for what higher education intuitions would do about cheating. This is odd as TEQSA are overseeing legally mandated practices, which I would have thought were "requirements", not expectations.
The course contains so odd phrases, such as "collective competence", to describe a team of people who, between them, have the necessary skills to deal with academic misconduct. This appears designed for individuals to avoid responsibility. The example of a surgical team is used. But every member of such a team is tested for individual competence. No hospital would allow a team to operate where it was not clear that every member had the required skills.
The course also contained a weird video explaining the usefulness of subject matter experts. It was as if universities normally use random individuals to do assessment, and using experts is an innovation. This was followed by a video about the value of "investigators". This is a term I learned a few months ago. It is something unlikely to be familiar to most who teach at university, and needs more background, explanation, and justification. Also the full title of the role should be used, not just "investigator", as this is also a term used in academic research (I have been a chief investigator). While not mentioned in the course, I assume investigators are required to be qualified, to a similar level to those who carry on investigations in Commonwealth agencies, with a Certificate IV in Government Investigations (PSP40416).
The course has some views on the seriousness of plagiarism charges I don't agree with. As an example a finding of plagiarism resulting in a 10% grade penalty is classified as low severity. However, as a student I found the one and only time I was accused of plagiarism as very serious. Fortunately this was a mistake by my instructor, as they had accepted Turnitin's text matching report. Turnitin matched my assignment with the published paper I produced from the assignment (I hadn't expected the paper to be published so quickly, so hadn't mentioned it in the assignment).
Some of the legal advice in the course may need checking. As an example, it is suggested if an institution receives a report from a contract cheating service that one of their students is using it this may be blackmail, and so the institution should report this to TEQSA, but that cheating at an institution is not a crime. I am no legal expert, but thought blackmail is a crime, to be investigated not by TEQSA, but by the police (or in some states under the jurisdiction of a corruption commission). Also cheating at an institution may be a crime, where the student profits from it. Also the course argues that the saftey and well being of the students should be the priority. However, if cheating is resulting in many incompetent people being licenced to carry out dangerous procedures, the public interest may outweigh the student's interests.
Some of the approaches proposed by the course will throw suspicion on innocent studnts, and unfairly discriminate against specific groups. As an example, those with a disability who use accessibility tools will likely have the metadata stripped from their documents. If the investigator, as suggested here, looks for the name of the student and the time they took to edit to be consistent will suspect cheating when all the happened as when student prepared their answer in their accessibility tool, then pasted it in. I have this problem with some online systems, which do not allow me to spell check. I have to compose offline, then paste the answer in. That can look like I am getting someone else to write it.
Another approach which might cause unfair suspicion is bibliographic forensics. The course suggests checking that the references a student uses match their expected level of knowledge. However, I was a student in a topic I had two decades experience in, but the rules required me to do the introductory units. To avoid the tutors saying (as they occasionally did "Tom what are you doing in this class?"), I would hind my knowledge of the topic and pretend to be an ordinary student. But using the techniques in this course that could make me look like I was cheating.
The course suggests LMS logs as a source of evidence, but doesn't discuss under what conditions this, or other, information can be used against the student. Do investigators need a reasonable suspicion, or can they simply sift through the LMS logs looking for any wrongdoing? To use an extreme analogy, they could also use face recognition on security cameras in student dorms to look for collusion, but a court would likely find that an unreasonable invasion of privacy.
There are some power imbalances in the procedures suggested in the course. As an example, the student is required to provide proof of identity at an interview, but the interviewer is not. It is a well worn cliche of police procedurals that investigators show their identification to suspects. Another example is that it is suggested that a support person not be permitted to speak extensively to the student in a language the interviewer does not understand. Unless it is slowing down the interview, what right does the interviewer have to prevent the student and their support person communicating in whatever language they are most comfortable with?
The course suggests having studnts bring their device to the interview. I find it disturbing that investigators would conduct a search of a student's device, as this is likely to contain a lot of personal, private, and sensitive information, both their own, that of family and friends, and of their employer. If asked to do this, my first thought would be to wipe the device clean of all records and logs beforehand to protect myself, family, and clients. That act in itself might then be considered suspicious by the investigator.
It is a little annoying as when I stopped to fill out the workbook, I ended back at the start of the module, and had to fast forward through the video again. The modules seem far too large to each do at one sitting. But if I stop, I risk having to go back to the start.
The workbook is in PDF. To enter my work I had to import it into the word processor, where the formatting went askew. I have to create a text box for each answer, and hope it will be readable in the final result.
Also I received a warning I had an unreliable Internet connection, and my progress may not be saved. That seems odd, as I am sitting in my university office with a very high speed internet connection. Most of the time everything loaded quickly, although the final reflection video stuttered.
Overall the course was of some use for learning more about how to investigate cheating, but was based on an approach using formal investigators. I would have liked some background as to why and how a decision had been made to introduce this role, and if all universities are required to implement them (if so, when?). Also I signed up for the course so I could go to a workshop. But I now can't see any details of the workshop, and it took me so long to do the course (about 7 hours, whereas it was supposed to take 3), that I have forgotten where I saw the workshop details. In hunting around I found a Situational Judgement Test which was taken before the course (and I had forgotten about).
Curiously I was not prompted to take the test as part of the course, but had to stumble over it. Finding what I might have to do next is difficult as I have the text enlarged to make it readable, so all I see is "Modules Situational Situational", not what the modules are, or what "situational" is.
References
Tracey Bretag, Rowena Harper, Michael Burton, Cath Ellis, Philip Newton, Pearl Rozenberg, Sonia Saddiqui & Karen van Haeringen(2019)Contract cheating: a survey of Australian university students,Studies in Higher Education,44:11,1837-1856,DOI: 10.1080/03075079.2018.1462788
Offering to write a university students' work for them has been illegal in
Australia since 2020. However, when I asked Facebook to take down a post promoting such a service to Canberra students, they refused.
Perhaps it is time for TEQSA, who administer the law, to prosecute
Facebook. The financial penalty is only $100,000 but the possibility of
a two year jail sentence might get the attention of Facebook executives.
What Facebook replied:
"We didn't take down ***'s post
We know that this is not what you wanted, and we thought it might help
if we explain how the review process works.
Our technology helps us review reports first. This means that we can
find content that goes against our Community Standards quickly and reply
to people in a reasonable period of time. Some reports, such as those
that might contain child exploitation, are prioritised for review by our
team.
Our technology reviewed your report and, ultimately, we decided not to
take the content down. If you think that we've made a mistake, you can
request another review. We'll use what you've sent us to improve the
technology and the reporting experience.
We understand that the content may be offensive or hurtful. Facebook is
a global community, and people express themselves differently, but we
only take down content that goes against our standards. We review and
update our standards regularly, with the help of experts.
Thank you for helping to keep Facebook safe and welcoming for everyone."
Just attended the ASCILITE seminar "A positive approach to academic integrity and the potential future of artificial intelligence", with Ishpal Sandhu & Patrick Lynch,from RMIT University. They discussed the potential and pitfalls of AI with issues such as student plagiarism. One of the audience asked how they could see the tools used, without signing up to a contract cheating site. So I did a quick search and found an interface to an AI system. I asked it a few questions, and got answers which seem very plausible, at a casual glance:
Q: How can we stop students cheating?
A: There is no one answer to this question as the reasons why students cheat vary. Some possible solutions include increasing the consequences for cheating, providing more opportunities for academic help, and increasing the difficulty of assignments.
A very plausible answer, although the first sentence could apply to anything. As one person at the seminar commented, it is the sort of answer a politician gives: vague on details.
Q. How successful was Online Learning during the COVID-19 Pandemic?
A: There is no one-size-fits-all answer to this question, as the success of online learning during the COVID-19 pandemic depends on a variety of factors, including the type of online learning platform used, the quality of the online learning content, the students' prior experience with online learning, and the students' level of motivation. However, overall, online learning appears to have been successful for many students during the COVID-19 pandemic.
The start of this answer seems to use the same formula as thew previous one, to say there are many factors. It has that same vague quality to the balance of the answer.
Q: Who is Tom Worthington?
A; Tom Worthington is an Australian information technology academic, consultant and commentator. He is a Fellow of the Australian Computer Society and a Visiting Fellow in the Research School of Computer Science at the Australian National University.
A bit out of date: I am now an Honorary Senior Lecturer, and the School is now just "Computing".
Websites which offer to write assignments for students are now banned under Australian law. The first example of such an essay mill, I have seen blocked was offering anything from a one page undergraduate paper to a complete PhD thesis, for about $20 a page. The site now displays in Australia 'The service is unavailable in Australia under the "Tertiary Education Quality and Standards Agency Amendment" law.'. Curiously this does not give the full title of the legislation, leaving out "Prohibiting Academic Cheating". So I think this may be action by the website provider, to avoid prosecution, than by the relevant government agency.
It is easy enough to get around this for the desperate student. The website provider can they try to deny any wrongdoing. However, the student is likely to be disappointed when, despite assurances of the supplier, their deception is detected, and they are subject to academic disciplinary procedures. Students who are studying for entry into a profession need to keep in mind that there may be very severe legal consequences if they cheat and so are not competent to do their job.
I have been looking at which online tools could be used for students to create a logbook toShow-Your-Work. The Mahara Journal looked promising, especially as it is usually installed alongside Moodle. However, this would still be an additional tool students and staff would have to become familiar with. An alternative which looks promising is the Moodle Wiki. This can be set up so each student gets their own. I have created a logbook template which can be provided in the Wiki.
My template has a paragraph of explanatory text, then sections for the student to fill in. The student could create extra pages if they have a lot of content. But I expect one page will do for a typical student. They just click on "edit" the heading for a week, and put in some content.
The fill in the blanks sentences are adapted from James (p.43, 2005). The topics for each week are from the Australian National University's Techlauncher program (Awasthy, Flint, and Sankaranarayana, 2017). Questions for the Work Portfolio (Weeks 4 and 8) were suggested by Tempe Archer, ANU Careers. The idea here is to provide a prompt for the student each week to start writing and avoid presenting then with a confronting blank page. The students are asked to write about the activity set for that week and a specific aspect of it.
Jacques, Ouahabi and Lequeu (2020) refer to the use of logbooks in Google Drive for French first year first year engineering students learning online, but unfortunately give no more details. Kumar, Silva and Prelath, R. (2020) mention not having a project logbook as a problem for studio based learning in a Malaysian course, but again provide no more details.
Reference
Awasthy, R., Flint, S., & Sankaranarayana, R. (2017, April). Lifting the constraints—Closing the skills gap with authentic student projects. In 2017 IEEE Global Engineering Education Conference (EDUCON) (pp. 955-960). IEEE. URL https://doi.org/10.1109/EDUCON.2017.7942964
James, Alisa, "Journaling as an Assessment Option" (2005). Kinesiology, Sport Studies and Physical Education Faculty Publications. 78. https://digitalcommons.brockport.edu/pes_facpub/78
Jacques, S., Ouahabi, A., & Lequeu, T. (2020). Remote Knowledge Acquisition and Assessment During the COVID-19 Pandemic. International Journal of Engineering Pedagogy (iJEP), 10. URL https://www.thierry-lequeu.fr/data/JACQUES-04.pdf
Kumar, J. A., Silva, P. A., & Prelath, R. (2020). Implementing studio-based learning for design education: a study on the perception and challenges of Malaysian undergraduates. International Journal of Technology and Design Education, 1-21. URL https://link.springer.com/content/pdf/10.1007/s10798-020-09566-1.pdf
Chicago
Logbook
You can use this logbook to keep notes, plans, and drafts of your work. This logbook is not assessed and is not visible to other students. However, it can be used as supporting evidence that the work you submitted for assessment was your own. All entries are logged and timestamped by the system, so the examiner will be able to see when you made notes and prepared drafts of your work.
Logbook
Your Name:
Student ID:
Week 1: Team Formation
Date:
Entry: The project is to ___ . My role is to ___.
Week 2: Orientation
Date:
Entry: To successfully undertake my role I will need to learn to ___ . To do that I plan to ___.
Week 3: First Audit
Date:
Entry: The team managed to ___. To help with that I ___.
Week 4: First WPP Workshop
Date:
Entry: The key points from the WPP workshop were ___. The job I am considering is __ . So I will need to work on my ___ .
Week 5
Date:
Entry: One of the areas I need to work on with the teamwork is my ___ behavior. To address this I intend to ___ .
Week 6: Second Audit
Date:
Entry: The hardest part of the audit was ___ . This is because ___ .
Week 7
Date:
Entry: One of the skills I need to work on is ___ . To improve, outside the course I will ___ .
Week 8: Second Work Portfolio Workshop
Date:
Entry: Here is my unpacking of a job advertisement for the WPP workshop: ___
Week 9: Project Showcase Video
Date:
Entry: The main aspects to developing the video were to ___. My role was ___ . I found this ___.
Week 10: Third Audit
Date:
Entry: I was able to help the team with the audit by ___ . This could be useful in my career ___ .
Week 11
Date:
Entry: The team uses different forms of communication for different purposes. This includes ___ to ___ and ___ to ___. It could be improved by ___ .
I have been looking at how to use tools such as Microsoft Word in Office 365, Mahara Journals, or Git for students to have STEM students Show-Your-Work. There has been much written about the use of reflective e-portfolios for assessment. I have been trained to write, teach and assess reflectively. However this is something the average STEM student and teacher finds very difficult. So I propose a form of non-reflective, non-assessed student journal: the student is prompted to record what they did.
James (2005) sets out how to guide physical educations students through preparing an assessed journal. They are referring to paper journals, but the technique applies also to electronic ones. First the student should be encouraged to be prepared with a suitable journal personalized to their tastes. With an e-journal I suggest a preliminary exercise where the student ensures they have access to the e-journal system, and enter some information about themselves.
Ensuring students don't lose an e-journal is less of a problem than for the physical ones James discusses (p.42, 2005). An e-journal system used by an institution should obviously be protected from hacking and based up. However, it would nevertheless be useful to guide students through taking a copy of their journal, and reminding them not to edit old entries (as that then updates the time stamps ruining their value as evidence).
Writing prompts are more important for an e-journal used with an online course, than for the face to face classes James (2005) discusses. As the student will be mostly studying asynchronously, the teacher is not there and then saying "put a copy of that in your journal now". This has to be explicitly stated in course materials, ideally as part of a assessed task, so the student has an incentive. This can also be a good point. I suggest to mark a point in the course, an approach of synchronization of asynchronous learning (Worthington, 2013).
Writing prompt suggested by James (p.43, 2005) include:
"Write a paragraph about a ... goal you would like to reach. Explain why you want to reach that goal.
Write a paragraph about what you did today that helped you to be successful in today's activities.
Write a paragraph about your ... behavior for the day. What things might you do to demonstrate more sporting behavior in the future?
What was the hardest thing for you to do today? Why was it hard?
Write a paragraph that includes the cues of striking that we learned today. What will you do outside of school to practice these cues?
Write a paragraph that includes the main aspects one should consider when developing a fitness program. Hint: Remember the FIT principle
Write a paragraph that describes activities that you can do in your community that promote cardiovascular endurance .
Write a paragraph that explains the difference between verbal and nonverbal communication. Why is it important to use both in cooperative activities? Be sure to give specific examples of both verbal and nonverbal communication." From James p.43, 2005, numbering added.
These are ordered from more to less reflective. The first four are about the individual student goals, success, behavior, difficulties. The next three are about future plans of the student to learn skills. The last is a more traditional study question about the course material.
These questions are not that different to ones which might be asked as study aids in any course. One of my frustrations as a student was that my answers to such questions were never looked at by anyone, let alone count towards assessment. In theory they help with learning, but in practice, like any student, I would tend to focus on what got looked at and especially marked. The e-journal gets around this problem by having answers go somewhere, perhaps be looked at and help me at least pass the course.
James (p.44, 2005) points out It is important assess journals, but not overdo it. entries. The author suggests the use of rubrics and checklists. However, they are unrealistic in expecting students to be frank in their entries. If the e-journal is part of the assessment, then students will write what they want the examiner to read.
For STEM students it can be especially difficult to find a comfortable voice to write about themselves. They may not understand the reason for writing this way and have had years of training which emphasized a neutral, impersonal way of writing. Also asking students to write for their e-journal is an extra task, as is assessing this. So I suggest a less personal approach, where the student places copies of what they are doing for their assigned tasks in their journal and comments on these. In this way the student is commenting on the task and their work, not themselves.
James, Alisa, "Journaling as an Assessment Option" (2005). Kinesiology, Sport Studies and Physical Education Faculty Publications. 78. https://digitalcommons.brockport.edu/pes_facpub/78
Worthington, T. (2013). Synchronizing Asynchronous Learning: Combining Synchronous and Asynchronous Techniques. In Proceedings of 2013 8th International Conference on Computer Science & Education (ICCSE), 26 Apr - 28 Apr 2013 , Sri Lanka. URL: http://dx.doi.org/10.1109/ICCSE.2013.6553983 Preprint available at: http://hdl.handle.net/1885/9556
"It appears to me that students can "game" assignment deadlines by creating incomplete Journal entries before the assignment deadline, then editing them after the deadline, since the Journal entry time stamps don't update after editing..." From forum: https://mahara.org/interaction/forum/topic.php?id=6470&offset=0&limit=10
This was later fixed with each journal entry displayed showing both a "Posted on" and "Last updated" timestamp. I tried this out on Mahara and it seems to work. But is anyone using this? In practice, do students understand what to do and in particular not to make updates to journal entries (so it looks like they did all the work just before the deadline).
STEM professionals and in many other disciplines are expected to keep detailed records of their work. This is for intellectual property protection, accountability, and well as normal management of activities. Occasionally I am asked to be an expert witness for a court case when a computer project fails. I trawl through all these records to see what when wrong, and importantly, when. Some disciplines have specialized tools, such as Git for computing and Electronic lab notebooks, such as SciNote for medical research.
So it would be reasonable to require students to keep records, as part of projects and assignments. This should not require additional work for the students as they should be doing this as part of their normal work. If students are not doing this they are not doing their work professionally and so can expect a lower grade (all the way down to zero). This is apart from any penalty if it is shown the work submitted was not their own.
For students not using specialized tools, e-portfolios may be sufficient, such as Mahara Journals. It would still be possible for a student to contract cheat, by having someone else prepare the journal entries for them to post. However, this would require a level of forethought and planning. The student could also provide their password to the contractor to make the entries, but that would open the student and contractor to charges of criminal conspiracy, as well as misuse of a computer system. This would be in addition to legal penalties for the contract cheating.
Recently I reported a Facebook post which was promoting a contract cheating service. Facebook replied that this did not go against their "Community Standards". I asked for a review and Facebook confirmed it met their standards. However, contract cheating is prohibited by Australian law, with a penalty of up to two years gaol and $100,000 fine. The service does not have to be provided from within Australia for the law to apply. The law applies to advertising the service, as well as providing it. It would be up to a court to determine if by knowingly profiting from the promotion of such services Facebook is in breech of Australia law and if its executives should serve gaol time.
Like many, it appears my students will be studying away from the face to face classroom again in 2021, at least for the first part of the year, due to COVID-19. So I have been considering how to improve online delivery, and in particular how to improve the experience of assessment for students while deterring misconduct. One way may be to require students to "Show Your Work" in assessments.
Some Literature
As Nguyen, Keuseman and Humston (2020) note online assessment was suddenly a major concern due to COVID-19. Some techniques they explored for chemistry students were short answers and multiple-choice tests to investigate higher-order thinking. They also looked at more frequent lower-stakes tests. This is possible as they are easier to administer online than face to face. The frequency, the authors suggest, may help with student learning and the lower stakes reduce the temptation to cheat. Interestingly the frequency is also claimed to reduce procrastination, as students are forced to study to pass the frequent small tests, rather than wait until a large test.
Nguyen, Keuseman and Humston (2020) also briefly looked at the use of an academic integrity pledge. However, they concluded this was only effective at an institution that has a culture of academic integrity. This seemed curiously circular reasoning: an institution that did not have a culture of academic integrity presumably would not have an integrity pledge.
To identify misconduct in online STEM courses, Sangalli, Martinez-Muñoz, and Cañabate (2020) analyzed the log from a learning management system of students undertaking exercises. They found co-occurrence of responses to exercises was an indicator of collusion, with pairs of students answering the same questions at the same time. Of course, this might give a false positive for students who are studying together.
Catalena (2020) was able to use the pattern of submission of student's coding assignments to identify plagiarism in a computer course. As students can submit repeatedly to a system that validates their code, it is possible to distinguish those who make incremental progress, from those who submit completed work. However, while submission of completed work without intermediate steps may indicate the student cheated by obtaining a correct answer from someone else it may be they used some other system for refining their answer, or they are exceptionally talented.
Pribela and Pracner (p. 99, 2017) propose building a system for students to create computer code, limiting the student's ability to copy from outside the system. This makes it possible to check the consistency of the student's work. It also prevents students from forgetting to include some element in their final assignment submission, as everything is in the system and accessible to the examiner. This seems a heavy-handed way to try to prevent misconduct and one which would stop students from using outside legitimate tools. The authors produced their temporal file system, based on Git, however, most of the benefit, I suggest, could be obtained by just using Git and allowing students import to it.
Dalziel (2008) suggests students prepare an ePortfolio, including the plans, notes, and comments to deter plagiarism. They point out that if entries are timestamped it is possible to see when entries are added. The author points out that the PebblePad ePortfolio tool has a link to the TurnItIn plagiarism detection tool. However, they don't report any results of using this approach.
Discussion
Students can be reluctant to start work on an assignment. Having a looming deadline, the student can be tempted to use old work of their own or someone else's, a form of poor academic practice or misconduct.
Students are urged to undertake their work methodically. However, they are only asked to submit the final product, not the steps taken to get there. This signals to the student that preparatory work is not of value. Students are therefore tempted to simply try to produce their final product, skipping steps in its production. Students may also become frustrated not understanding why they can't produce a polished result instantly, not realizing they are not the only ones who have to work through draft after draft.
An examiner competent in the discipline assisted by automated tools can look for signs of misconduct. However, this can be a time-consuming process which is stressful for staff as well as the accused student. Rather than look for poor practice, I suggest having students provide evidence of good practice.
A similar problem occurs with online examinations, where remote proctoring tools, such as ProctorU and Proctorio, have been employed. Those tools work reasonably well, but my colleagues at ANU Computer Science developed an alternative approach of self-invigilation for online examinations. Each student is encouraged to make recordings of themselves undertaking the exam, similar to those created by proctoring tools. The difference is the student makes the recording, rather than have software imposed on them. If the examiners raise concerns about who sat the exam, the student can present the video as evidence.
I suggest this self-invigilation process could be extended to assignments through a Show-Your-Work process, as Dalziel (2008) suggests. Rather than just submit their final completed assignment, the student would make available to the examiner all drafts, notes, and other work that was used in preparing the assignment. If there was doubt as to the author of the final work, the examiner could look to see if there was a consistent body of work by the student supporting it.
The student would be required to use a system that time stamped their work, and tracked any changes, not only of drafts of the final work but all notes used. It would then be possible to look for a consistent pattern of work over days or weeks.
This Show-Your-Work approach is already used in some group project-based courses in computer science at ANU. These require student teams to use project management software, such as GitLab. The examiner can see when work was uploaded to the system and who uploaded it, to see if this supports the team's claims.
Just as students can be videoed while undertaking an exam, a video record of all the time each student spent working on an assignment (or all the time they were studying) is technically feasible. However, this would be cumbersome to use, intrusive, and not necessary. An approach of logging all the work done on an assignment should be sufficient, as this is something the student should be doing anyway. Students in any field of study should be doing so in a methodical way, STEM, computer science, and engineering in particular.
While it would be possible for a student to fake the assignment preparation process, it would be cumbersome. The student would have to take the working notes of someone else (or commission these to be written) and submit these to the online system over several days or weeks. If the student tried to enter all the working notes just before the deadline, this will be evident from the timestamps.
The student could alternatively not only get the work from someone else, but have them submit it for them. This would require handing over their ID and password to upload the material. Using two-factor authentication might help deter this, where the student would be required to enter a code sent to the mobile phone. Also, the possibility of a jail term for conspiracy and computer crime may deter some students.
Asking students to show the notes and drafts of their assignment is much the same as telling students on an assignment question to "show their work". Where a student doesn't show how they derived the final result, the examiner can reduce the grade. This doesn't require accusing the student of plagiarism.
If students are asked to show their work, they will need to be guided as to what is a reasonable quantity of work over what period. During my MEd studies, I created a journal using the institution's Mahara e-portfolio system, in which I recorded my impressions of the program. Also, I created a journal for each course, and one for my capstone e-Portfolio (in place of a thesis). In these, I made notes on references found, answers to study questions, and appended drafts of assignments. This was mostly to aid me in my work, and to have material to draw on for my reflective e-portfolio. However, I also kept this as evidence, if I was ever to be charged with academic misconduct. Fortunately, that never occurred, and my journals remain read only by me. I made an average of 100 postings per course, a total of 100,000 words. This was in addition to the assignments and capstone e-portfolio.
Based on my experience as a student, a reasonable guide would be 50 words of supporting material per percent of assessment. So an assignment worth 20% of the course assessment undertaken over two weeks would require 1,000 words of supporting notes in 10 posts. This would be in addition to ten drafts of the assignments, over at least five days of the two weeks.
References
Catalena, K. A. (2020). Mining Student Submission Information to Refine Plagiarism Detection (Doctoral dissertation). https://hdl.handle.net/1969.1/191583
Nguyen, J. G., Keuseman, K. J., & Humston, J. J. (2020). Minimize Online Cheating for Online Assessments During COVID-19 Pandemic. Journal of Chemical Education, 97(9), 3429-3435. URL https://pubs.acs.org/doi/pdf/10.1021/acs.jchemed.0c00790
Dalziel, C. (2008). Using ePortfolios to combat plagiarism. Hello! Where are you in the landscape of educational technology? Proceedings ascilite Melbourne 2008. URL https://ascilite.org/conferences/melbourne08/procs/dalziel.pdf
Pribela, I., & Pracner, D. (2017, January). A Temporal File System for Student's Assignments in The System Svetovid. In SQAMIA. URL https://perun.pmf.uns.ac.rs/sqamia/2017/download/sqamia2017-proc.pdf#page=99
Sangalli, V. A., Martinez-Muñoz, G., & Cañabate, E. P. (2020, April). Identifying Cheating Users in Online Courses. In 2020 IEEE Global Engineering Education Conference (EDUCON) (pp. 1168-1175). IEEE. URL https://doi.org/10.1109/EDUCON45650.2020.9125252
Professor Dawson surprised me by being mildly positive about the use of online invigilated examinations, while pointing out that the
companies which provide these services were reluctant to have them
independently tested. He was amused by my suggestion that a chromakey bodysuit could hide a helped in the background. ;-)
The
Australian Tertiary Education Quality and Standards Agency (TEQSA) has an Experts advice hub which is particularly useful with the move to online education. One useful piece of advice is "The prevention of contract cheating in an online environment" by Associate Professor Phillip Dawson, Deakin University. At first I found this a bit annoying, as it starts with 3 myths about contract cheating: it is rare, it can be designed out, and is impossible to detect. After reading these I concluded that TEQSA wants me to give up on project based progressive assessment and just set one exam at the end of semester.
But reading on Professor Dawson does suggest assessment design approaches to help reduce the prevalence of cheating: reflections on practical work, individual work, and in-class tasks. Assessment which it is suggested most attracts cheating are ones with lots of marks, and short deadlines, such as take-home exams.
One of the key problems I suggest causing contract cheating is the lack of time and effort academics think they should put into assessment design and delivery. Like most, my first exposure to assessment was being asked to set exam questions and mark assignments, with no prior training. Completing a couple of courses on how to design and deliver assessment was a revelation. Much of what I had been doing turned out to be, at best, irrelevant, and in some cases counter productive. Also the amount of time needed to do assessment properly was sobering.
I had assumed that assessment was an afterthought tacked onto a course. Much of the frustration of academics perhaps comes from this assumption. Assessment should take up about half the staff time in a typical course. Once you accept this, it is less frustrating how much time it takes, as you expect this.
As the guide suggests, many small assessment tasks, with generous deadlines, place less pressure on students to cheat. Practical work, where each student has a different project and where they have to reflect on what they did makes cheating harder. Where the student has to explain what they did to the assessor this also helps. However, these all take much more work to design, administer and grade. These also take skills which the average academic doesn't have, because it was not part of their teacher training (assuming they received any teacher training).
As an example, I once sat at a course planning meeting where we discussed the assessment of reflective e-portfolios. As the discussion progressed, I realized that of the dozen tutors and lecturers there I was the only one who had ever completed a reflective e-portfolio as a student, and the only one with any formal training in how to assess them. Without that training ans experience, tutors were assessing the e-portfolios as if they were project assignments.
If you set out to design the assessment for a course, allocating marks to small tasks, thinking about the time the student, and the assessor, will need, it is possible to design most cheating out. However, I suggest emphasizing the benefits for the students, and for their teachers, of more realistic, better planned assessment.
One think academics need to decide is what they are doing assessment. If a graduate needs particular skills and knowledge to undertake a professional role, then there is no need for more than a pass/fail test. If the assessment is to identify those who will undertake further advanced studies, then more fine grained assessment is needed. However, these two approaches can be mixed in the one course. If you don't want students asking for extra marks on every little assessment task, then have these small tasks just count for a pass (or whatever is considered the minimum acceptable level). Reserve the fine grain marking for the important tests.
This week I had a curious email from an essay writing service proposing to pay me to promote their activities through this blog. From reading the email, and a casual read of their website it appears a legitimate business helping students. But on closer examination they offer to write whole assignments for students, admissions essays, and take multiple choice tests for them: "In need of a top-quality custom essay? Let our experts handle it! Our academic essay writers in Australia are waiting to help you with: Custom Essay ... Admission Essay... Multiple Choice Questions ...". Such cheating, plagiarism, and colluding, is dishonest and provides an unfair advantage. Unfortunately, while it is against the rules of educational institutions for students to make use of such services, it is not yet illegal in Australia for companies, or individuals, to offer or provide them. Perhaps a useful training exercise for cyber security students to find out as much as they can about those who provide these services, and a means of shutting them down.
"My name is ****, I work as a marketing assistant and outreach expert at ****.
I'm writing to ask if you would be interested in mutually beneficial business cooperation.
I would like you to consider two advertising options:
1) Sponsored
post contribution, meaning we write a unique SEO-friendly article which
corresponds to your blog theme and overall writing style;
2) Link insertion, meaning we pay for our link to be added in one of the existing articles related to our niche.
Let me your current rates for content publishing on tomw.net.au and content requirements (if any).
Looking forward to working with you,
****"
The Australian Minister for Education, Dan Tehan MP, released a draft law "Prohibiting Academic Cheating Services", 7 April 2019. This would make it a criminal offense to provide or advertise academic cheating services, with up to
two years imprisonment, or more than $100,000 fine. Students who use the services will not be subject to this law, with any academic penalty left to their institution. The law will apply to services provided outside Australia for those in Australia. Tertiary Education Quality and Standards Agency (TEQSA) will undertake monitoring, and can ask a court for an order to have web sites providing services blocked. There is a summary overview of the draft Bill available. The Australian Department of Education has asked for comments by 28 June 2019.
It is not clear to me how well this law will work where services are being provided from outside Australia. Also it is not clear if this also applies to students in Australia enrolled in courses outside Australia.
In my view educators need to accept that students do not see cheating as a serious issue. Giving students stern warnings has proved ineffective. Making it something like a crime requires educational institutions to have complex slow processes, which students can use to avoid penalties. Instead I suggest treating cheating as a learning experience.
Students should be trained and tested on study skills, including how to write assignments. Most students will respond to this. Assessment can be designed so the few students who persist with attempting to cheat never graduate.
Rowland, Slade, Wong and Whiting (2018) looked at what attracts students to websites offering to write their assignments for them (so called contract cheating websites). The authors found that contract cheating websites had similar features to a typical website for a hotel. They found differences with the contract cheating websites emphasizing quality: qualifications of their authors, quality of the results, security, price, timeliness, confidentiality and satisfaction. Rather than a hotel, perhaps the authors should have used a dating website for comparison, where more trust is required by the client.
The way for institutions to counter the attraction of contract cheating, I suggest, is to use similar marketing on student support websites. Institutions might want to set up, or sponsor, semi-independent help sites, so students feel it is not more of the same from the institution.
It would also be useful for institutions to design courses using on-line distance education techniques. The idea is to provide students a course in small increments, with feedback and support. This way a student sees what they have to do and receive small rewards, in the form of grades, for doing it. Students who do not do the small exercises on-time and to the required standard can be quickly identified by instructors and offered help, before a major hurdle is reached. This way a student will not be left languishing at the back of the class for most of a semester, then be tempted to cheat to pass a major assessment exercise.
The student who does try to cheat on a major assignment will be easier to identify as this will not be consistent with their past work. Students will know that their instructor knows, from the feedback they get and so they will know they are likely to be caught cheating. I use this approach in the course "ICT Sustainability".
Reference
Rowland, S., Slade, C., Wong, K. S.,
& Whiting, B. (2018). ‘Just turn to us’: the persuasive features of
contract cheating websites. Assessment & Evaluation in Higher Education, 43(4), 652-665. URL https://doi.org/10.1080/02602938.2017.1391948
