Thursday, March 23, 2023

Allow Students to use AI Generated Text in an Assignment Structured like a Systematic Review?

Does anyone else find it ironic that universities are using AI to make sure the students don't? ;-) More seriously, one of my students asked if they could use ChatGPT to write a report on their work experience. I checked the institutional rules, and found I had discretion as to what to do. I said "no", on the basis they would not be the author of the work, just an editor of it. But then their supervisor gave me a call to say they had suggested ChatGPT. This puts it in a new light, if this is an accepted, and encouraged, work practice.  So what I am now thinking is have the student prepare a report much like one used for a Systematic Review. 

The student framed their request in terms of using ChatGPT as a grammar checker. The problem is that ChatGPT does much more than simply correct grammar, it selects facts and concepts, drawn from a large collection of work from numerous authors, not always correct, or attributed. 

Universities provide advice and assistance with writing. Professor Inger Mewburn at ANU (the Thesis Whisperer), recommends Grammarly, and the free alternative Typely. I used Grammarly as a graduate student, and found it very useful (English is my first, and only, language, but I only barely passed the subject at school).

My experiences in using Chat GPT, and its predecessor GPT3, and other AI, have been mixed. Recently I reviewed a paper generated by Chat GPT. I gave the paper one out of five, and suggested the name of the author should be "Anonymous" with the person who submitted it listed as an editor, not author. But not everyone agreed, with an average review of 2.6 out of 5.

Using an approach similar to a systematic review, the student would set out what they are attempting to accomplish, what queries they tried, what the results they got were, how they selected the most appropriate result, what manual changes they made to the automated result, a critique of the result, and recommendations for improvements to the process. The idea would be the student would demonstrate skill in using a tool to produce a result, and value add to the automated output.

No comments:

Post a Comment