Making sense of Learning Designer skills

Greetings from ASCILITE 2023, where Assoc Prof Gilmore from RMIT is talking on an international analysis of qalifications and skills asked for Learning Designers. These are people who help academics and academics produce courses. This is relatively new in school and university systems, but more common in vocational education.

Analytics for Lifelong Learning

Simon Buckingham Shum,
UTS Connected Intelligence Centre

Greetings from the Marie Reay Teaching Centre, at the Australian National University in Canberra, where Professor Simon Buckingham Shum from the UTS Connected Intelligence Centre, is presenting a seminar on "Analytics for Lifelong Learning Competencies: Aligning Pedagogy, Human-Centred Design & University Strategy". He presented a hierarchy starting with the student focus, then learning design, ending with the organizational view. I suggest a few more layers are needed, for the industry/discipline view, national view, and international. University educators need to think about what is good not just for the student and the institution, but more widely outside.

Professor Buckingham Shum showed examples of tools to assist nurses in a clinical setting, students learning reflective writing and more traditional academic writing. However, I was a little skeptical as the examples all seemed to be for very narrowly focused STEM discipline learning. If you are teaching students how to do a specific task in a well-defined job, in a standardized regulated discipline, such as medicine, engineering or computing, then using analytics is relatively easy.  However, there was then an example of helping law students with writing an argument, undertaken with Dr Philippa Ryan, who is now at ANU.  But Dr Ryan is not your average non-STEM academic (she was on the ACS Blockchain Committee with me). Can university academics who are experts in research aspects of their discipline, and are not experts in computing, analytics, or education, cope with this?

As Professor Buckingham Shum pointed out, it would not be very useful to bolt a sophisticated analytics system onto old fashioned education.   system. Also, few of those involved in university education will have all the skills in the discipline, analytics, and education. An exception is some in the computing discipline, where analytics is part of their discipline, and they have training in education. His suggested solution is something like the UTS Connected Intelligence Centre.

One practical outcome could be that the UTS AcaWriter software tool for academic and reflective writing could be further developed by ANU's TechLauncher computing project students, for teaching reflection.

Research Finds Adjunct Staff Better for Students than Full Time Professors

The paper "Are Tenure Track Professors Better Teachers?" by David N. Figlio, Morton O. Schapiro and Kevin B. Soter (NBER, September 2013) concludes that part time adjunct staff make better university teachers than full time professors. This is a welcome finding for those of us who are adjunct lecturers and hardly surprising. However, I suspect it has more to do with adjuncts having more, and more up-to-date, training in teaching, than anything to do with full/part-time status. Adjuncts are likely to be newer to the institution, so more likely to have been through the teacher-training program which institutions run and also less able to avoid doing it (as Professors can). The paper does not appear to address the teaching qualifications of the staff in the research.

There may also be an element of priorities in the difference. When I suggested to one professor that improved teacher training for staff would improve student outcomes, they explained to me that staff were selected and promoted based on their research publication record: quality of teaching was not important. I was a little shocked by this, but if that is what the system is telling staff is important, they can't be blamed for acting accordingly. This may change as student feedback on individual courses and for institutions becomes more common.

Learning Analytics

Greetings from the Australian National University in Canberra, where a workshop on "Learning analytics: Building evidence based practice" by Dr Shane Dawson, UniSA, is being hosted. Learning analytics is about analysis of data about students to improve courses. This has come to prominence with Learning Management Systems, as these record detailed information about what students do and when they do it. However, there are risks in misinterpreting statistics, in particular confusing a correlation with a causal relationship. As an example, there is a well known correlation between a student's participation in on-line course forums and their final result. But it is not necessarily the case that forcing students to participate will improve their results. The other question this raises is what teachers did the past: just hope that the way the educated worked?

One of the hidden agendas with learning analytics is that it works better with large amounts of data, from large courses. If you are teaching only a few dozen, or few hundred students, then some statistical techniques do not work well. There is then an impetus to have thousands, hundreds of thousands or millions of students.

One question I have which learning analytics could help with is the differences between classroom and on-line courses. I did do a quick check to see if the students who do my on-line course get different results from their classroom course: they don't (there is about 0.8 correlation for on-line and course results of the same students). But are the students who undertake on-line courses different in some way? Why do some students withdraw early on? Is there something I can do to reduce the withdrawal rate. It might be sufficient to know what program the student is enrolled in. But the central universality database has more student details, such as if they are a domestic or international student, previous results and where they previously studied.

Some of this is just a matter of good course design, assessment and normal teaching. As an example the Research School of Computer Science built itself a database years ago, with all student results in it. This is used in the end-of-semester examiner's meeting, where the results of each course are compared. If necessary, the results from previous years can be compared.

One thing which struck me about the discussion of learning analytics is the emphasis on student performance, but what about teacher performance?  The same analysis done of what students do and their results can also be applied to staff: what they do and how this effects outcomes.

Shane's paper "Informing Pedagogical Action: Aligning Learning Analytics With Learning Design" has more detail. Shane's workshop is being hosted at University of Tasmania 19 July.

This workshop, sponsored by the Higher Education Research and Development Society of Australasia and ANU Online as part of an Office for Learning and Teaching project, aims to:

• Provide an overview of the current state of learning analytics
• Illustrate how learning analytics can provide direct evidence of student learning
• Explore the diversity of tools and methods associated with learning analytics
• Provide practical ideas to apply learning analytics for evaluating and improving teaching practice.

Participants are expected to bring a laptop to enable hands-on involvement.
The high growth in adoption of education technologies such as learning management systems (LMS) across the education sector has resulted in alternate and more accessible data on learning and teaching practice. As with most online systems, student interactions with course activities are captured and stored. These digital footprints can be ‘mined’ and analysed to establish patterns of learning behaviour and teaching practice, a process described as learning analytics. Tracking the patterns of student interactions can provide detailed insight into the learning process and allows for rapid evaluation of the impact of specific learning activities. In essence, learning analytics empowers both instructor and student to make informed decisions about their learning and teaching processes, through the interpretation of educational data from both learner and teacher orientations.
This workshop provides an initial overview of the current state of learning analytics and illustrates how the implementation of multi-analytic lenses can provide direct evidence of student learning. In so doing, Dr Dawson will discuss the value of learning analytics and in particular social network analysis (SNA) as a methodology for visualizing curriculum and peer networks that are established through course relationships such as student engagement and course progression. Following this overview, participants will explore the diversity of data sources at their disposal drawing on analytic tools and dashboards such as SNAPP and LMS sources.
Dr. Shane Dawson is the Deputy Director of the Learning and Teaching Unit, and Associate Professor of Technology Enhanced Learning at the University of South Australia. His research activities focus on learning analytics and social networks to inform teaching and learning theory and practice. Shane’s research has demonstrated the use of learner interaction and network data to provide lead indicators of student sense of community, academic success and course satisfaction. Shane has also been involved in developing pedagogical models for enhancing creative capacity in undergraduate students. He is a co-founder and executive member of the Society for Learning Analytics Research and was co-chair of the 2012 Learning Analytics and Knowledge conference in Vancouver, Canada. ...

Funded by HERDSA and the Australian Government Office for Learning and Teaching.