AI for Andragogy Testing Tests with Simulated Students

 

Andragogy for Impact (AnfI) 2026

Greetings from Singapore, where at the first of two  workshops at Andragogy for Impact (AnfI) 2026, by the Singapore University of Social Sciences. The first is AI-Enabled Learning and Teaching for Adult Learners by Dr Lye Che Yee, and a gaggle of SUSS staff. The first challenge posed was how student will learn if AI just gives them the answer. One difference noted was that previous search systems required the student to provide keywords & search commands, whereas AI can understand natural language. I suspect this is an illusion. With early programming languages, such as COBOL, it was claimed business people could use it, programmers were not required. This was not the case, nor with later generation languages, when the same claims were made. Training was needed to understand the problem, express it well, document the process, & test the answers. 

Dr Yee suggests that learners will need to learn a set of essential skills, even if AI does the hack work. These include communication skills, which is something one of my team's of ANU students system told me was lacking in my CV. One tricky skill which will be needed is judgement.

Associate Professor Wong showed how they redesigned a take home assignment to incorporate use of AI. The student has to explain how it was used with the techniques in the course. They argued it was not necessary to teach how to use LLMs, just as it is unnecessary to teach use of office software. I suggest this is incorrect. At primary school we were trained to make an emergency phone call. Just because people have technology, they do not necessarily know how to use it effectively. In fact Professor Wong's assessment design appears to be teaching how to use a LLM with detailed instructions. The key question for me is the students are trained in how to do this, or just given the assignment and expected to work that out themselves. 

One incidental point which came up in discussion is that SUSS still subscribes to TurnItIn for checking student assignments, but has turned of the AI score as it is "nonsense".

We then moved on to adaptive drift, which I didn't quite get. But that might be because I got caught up chatting at morning tea, and was late back. 

Back on track we were asked to try an application which used a simulated student to submit answers to typical questions, to be able to see how well LLMs could assess. The point here was that it is hard to simulate a weak student, as the LLM has so much knowledge. This is not an unusual problem, academics who have not been trained to teach have difficulty with struggling students. This suggests that software test techniques could be applied to reviewing assessment tasks. With software development it is routine to run an automated set of tests each time software is changed. The same could be done with assessment: the assignment changes and a series of tests are triggered, with simulated students submitting answers, which are then evulated.