Usable and accessible online courses

Greetings from the seminar room of Australian Centre on China in the World (CIW)at the Australian National University where I am taking part in "How usable and accessible are your on-line courses?" by Professor Denise Wood, Central Queensland University, and Dr Sheila Scutter, University of South Australia. We were first introduced to the principles of accessible web design and the more general Universal Design for Learning. They are designing a MOOC for teaching this. Jeremy Smith from ANU received an acknowledgement for his contribution.

The project is not just about making course materials readable for people with a disability, it is also  about accessibility more broadly. Also it is about making course accessible on-line for completely remote delivery, or in blended mode.

Some of these are issues of basic course design, which will benefit all students. For example, how to how to contact the instructor and what course materials are available are issues which are not well addressed in traditional classroom based courses and can be improved with on-line support.

One tool mentioned was the Student Usability Scale (SUS). Also the use of software installed on the user's computer to record a test session for the usability of a course was discussed. The Flexible Learning for Open Eduction (FLOE)  tool was demonstrated.

I am familiar with some of these accessibility issues having been an expert witness in the Sydney Olympics 2000 Accessibility case and invited to advise the Beijing Olympics.

Wood and Scutter gave an excellent seminar on well run research into teaching. Unfortunately, this appears to be another of the Office of Learning and Teaching (OLT) funded research projects which is addressing the wrong question. There are decades of research on on-line course design and accessibility. There are tools and training courses based on this research in routine use in the vocational education and training (VET) sector for developing accessible courses. The question is why Australian universities do not implement what has been shown to work in the VET sector.

For anyone teaching across the vocational and university sectors (as I do), the differences are obvious. In the vocational sector (and in distance/open universities) the courses are designed by a team and then delivered by instructors. All the courses for a program use the same template, with course components laid out in the same way. The instructor has limited, or no, ability to change the template or the course materials (even to correct spelling errors or broken links). As a result, accessibility is designed in to the program, at the cost of flexibility.

Universities, particularly research orientated universities, allow their academics more freedom in course design and delivery than the vocational sector. The academic can teach what they want, how they want. But researchers are trained to conduct research and are understandably reluctant to take the extra time needed to learn how to design courses and teach. This problem is not unique to Australia. Last year I talked to staff at Cambridge University about how to help the graduate students with on-line courses. One option would be to incorporate basic teaching skills in postgraduate research programs, bringing early career academics up to the standard of VET teachers. Bryant and Richardson (2015) found that university lecturers with teacher training tended to have fewer failing students.

Academics producing a course content need to only work on the content, using the skills of specialists to worry about how it is presented. As an example text can be written using the default formatting features of a word-processor, so that it will inherent the style set up for the institution's template. I have used this approach for the course "ICT Sustainability", enabling the course content to restyle automatically when the institution's template is updated. This has also allowed the course to be ported between three very different higher education institutions (vocational, open and research orientated) and also adapted for mobile devices with minimal additional work required.

Scriven describes how from the 1970s he became increasingly disenchanted with evaluating US education programs based on the goals of the programs, leading to Goal-Free Evaluation (GFE)  (p. 56, 1991). A program designed to improve the usability and accessibility of courses is likely to be successful by its own internal logic: those academics who are motivated to take part are likely to produce better courses. However, using GFE, the bigger question becomes apparent: how do we motivate Australian university academics to improve courses? I suggest this is the area needing research.

References

Bryant, D., & Richardson, A. (2015). To be, or not to be, trained. Journal of Higher Education Policy and Management, 37(6), 682-688. DOI: 10.1080/1360080X.2015.1102818

Scriven, M. (1991). Prose and cons about goal-free evaluation. Evaluation Practice, 12(1), 55-63. Retrieved from http://dx.doi.org/10.1016/0886-1633(91)90024-R

Save the Office for Learning and Teaching Petition

The Office for Learning and Teaching (OLT) is an Australian Government body which funds research to improve teaching at universities. The Australian government plans to close the OLT in mid 2016. There was a report "A New National Institute for Learning and Teaching" calling for a new body, based at an Australian university, but this has not progressed. In response Associate Professor Angela Carbone, Monash University has set up a petition "Save the Office for Learning and Teaching" calling on the government to either continue the OLT or proceed with the new center.

While I support work to improve university teaching, OLT has not been very effective at doing that. OLT funds research, whereas what is needed is training. The problem is not that there is a lack of research on how to teach, just a lack of incentive to apply it in universities.

Teaching Better Judgement

Greetings from the Australian National University in Canberra, where I am taking part in Professor Lambert Schuwirth's  Better Judgement Workshop. This is part of a Flinders University, funded by the Office for Learning and Teaching (OLT). This is about how to better train assessors for professions such as medicine.

This is about assessment of skills in the workplace. But something which struck me was that I feel very comfortable sitting in a classroom. I have been teaching and doing only  on-line courses for the last year and there is something familiar about being back in a classroom, with rows of desks, a lectern at the front and fellow students around me.

The next thing which struck me was that the workshop seemed to be about workplace assessment for competence, an area I was trained in as part of a Certificate IV in Training and Assessment. With this you first decide what the job requires, then what behavior you can observe of the student to asses if they can do what is required. This could be the student actually doing the job,or doing a simulation of the job. Only if that is not feasible would you give them an "examination" as such, as that is not a very good way to assess what some can do.

Also the Vocational Education and Training (VET) sector, unlike universities, requires formal training and assessment of assessors. Having been trained in how to design and administer assessment makes me more confident in what to do and, hopefully, better at it.

This brings up a problem with the OLT's strategy for improving the quality of terracing at Australian universities. OLT funds research on teaching and learning issues. The researchers are required to not only produce the usual academic papers, but also disseminate their results to academics. The usual way is to present results at workshops on campuses across Australia. The problem with this is that, by its nature, research has to be on something new. However, the day to day problems with teaching in Australian universities already have known solutions. The need is not for research, but for training of staff in how to teach.

Assessment Quality in Higher Education

Greetings from the Inspire Center at the University of Canberra where Professor Geoff Scott is conducting a "Valid Assessment and Assuring Assessment Quality in Higher Education" workshop, sponsored by the Office of Learning and Teaching. But so far what he seems to be telling us is assessment 101: have assessment aligned to the learning outcomes and tell the students all this at the start of the course. This is something I learned in USQ Assessment, Evaluation and Learning (EDU5713) and discussed some of in "Australian University Education in 2025". Hopefully we will get into the promised high level policy matters soon.

Professor Scott went on to say that there should be peer review, multiple reference points to validate program reference outcomes. He used engineering as an example, but I suggest that like computer science, these disciplines have clear vocational requirements and there has been close cooperation between education institutions and the industry for decades. What I would like to know is what to do in the other discipline areas which are not so vocationally focused.

Professor Scott also recommended the book "Search Inside Yourself: The Unexpected Path to Achieving Success, Happiness (and World Peace)" by Chade-Meng Tan. He characterized this as a mindfulness program developed by Google, but it was not clear to me what this had to do with the topic of assessment quality in Higher Education. Also I am not sure that this program is central to Google (not one I have talked to at Google has mentioned it).

Professor Scott is a dynamic speaker and he has an interesting project. But I had assumed we would hear about the results of his work and something new an innovative. So far this seems to be a moderately useful summary of the issues and challenges, but any competent senior academic will know this already (or could could find and read it in a few minutes on-line).

Professor Scott's workshop was promoted as being for "... particularly, Heads of School, Associate Deans (L&T) and Heads of Program (or Course)", but so far he seems to have only defined the problem (which is already well known from many studies) and has not yet started work on the solution. In conversation Professor Scott mentioned he had published research on the topic but did not cite this in his presentation. It may be that he needs to modify his presentation to suit a more research orientated audience.

At one point Professor Scott mentioned there was material from a book and the reference was in the notes. Unfortunately the references on screen and in the notes handed out are illegible, being in very small multi-colored text.

The Office of Learning and Teaching funds some excellent initiatives, but perhaps they need to set some standards for the way the results of the work are communicated. An example of a well run event was the IMSITE (Inspiring Mathematics and Science in Teacher Education) Dissemination forum.

While I can't recommend Professor Scott's workshop, he will be speaking again at University of Adelaide (23 April),  Victoria University (24 April) and Queensland University of Technology (27 April).

ps: One of the most useful outcomes of the event was to remind me of how good the Inspire Centre's teaching facilities are.  We are in the TEAL room which allows a blend of lecture and group style work seamlessly.

pps, 24 March: I found a copy of Professor Scott's slides from a 2014 workshop "Assuring the quality of achievement standards in H.E. - Educating capable graduates not just for today but for tomorrow". Here is the further reading and resources which had hypertext links (without the red text on a blue background, which makes the original unreadable):

1. AAHE (1996); AAHE principles of good practice in assessing student learning
2. Campbell. S (2008): Assessment reform as a stimulus for quality improvement in universityL&T: an Australian case study Outcomes of HE conference
3. Hanover Research (2013): Best and innovative practices in HE Assessment, April 2013
4. HEA (2012): A marked improvement: transforming assessment in HE, HEA, Newcastle, UK
5. Krause, K.L & Scott, G (2014): A sector-wide model for assuring final year subject andprogram achievement standards through inter-university moderation, Office of Learning &
   Teaching, Australian Government, Sydney.

STEMing the Decline in Science and Maths Teaching

Greetings from the Shine Dome in Canberra, where I am taking part in an IMSITE (Inspiring Mathematics and Science in Teacher Education) Dissemination forum. This is sponsored by the Office of Learning and Teaching to reverse the decline in science and maths teaching in schools. This adversely effects student's interest in, and ability to undertake, technology and engineering programs at university. This is an area being researched in Australia, as well as other countries.

The event is is being chaired by the always enthusiastic, Dr Judy-Anne Osborn, Lecturer, School of Mathematical and Physical Sciences, University of Newcastle. She described the "online-ificaiton" of some of their MATH2910 course and MATH2920 invited other institution to collaborate in the teaching of this.

University of Wollongong is part of IMSITE. They offer a teaching minor for science degree students and are hosting a STEM Teachers Conference 25 July 2015

It occurs to me that there are some obvious ways to  foster teaching and learning maths and science. One would be to use the link to the exciting careers, which are the other two letters in the STEM acronym: Science, Technology, Engineering and Mathematics. It is the areas of technology and engineering which have the career prospects.

A way to improve the teaching would be to use on-line education, both for teacher training and, for direct student education. It seems to me that teacher educators are making their job harder than it needs to be, by not using e-learning techniques. It seems that educators don't think computers can be used for teaching in schools and so should not be used for teaching teachers. I suggest "flipping" the approach to place e-learning at the center of education. As I explained to one of the attendees, you can make your on-line materials backward compatible with paper and classroom teaching, whereas it is much more difficult to design for paper and the classroom, then adapt for on-line.

My rule of thumb is that the average university student needs to spend about 20% of their time in class (one day a week), with the rest on-line. Extending this to school, might see the youngest students starting at 80% in class, 50% for primary students, and about 30% for high school students.

This is not to say students will not be actively engaged in learning, just that this need not be in a formal classroom setting. The least useful activity is to have students sitting passively listening to a teacher, they should be doing something, mostly with other students. Where students are required to "attend" school, they can spend most of their time in a learning center (what used to be called a "library"). Only a few teacher-librarians are needed to watch over several hundred students.

Bring-Your-Own-Device (BYOD) e-Examinations

Today I attended a fascinating webinar “Bring-Your-Own-Device (BYOD) e-Examinations” hosted by Ascilite. The first speaker described Transforming Exams platform, developed with an Office for Learning and Teaching grant by University of Queensland and University of Tasmania. The system provides an off-line electronic exam environment. The examination is prepared in a format compatible with Moodle and then loaded onto USB flash drives. The drives include the Ubuntu Linux operating system and when the computer is booted it restricts the user to the examination system.

This could provide a very useful transition for institutions which have paper based examinations. It allows an environment which emulates the paper system, but provides some benefits of electronic delivery.

While the Transforming Exams system is ingenious, I found the idea of having examinations a bit dated. I design online courses for students, who may be anywhere in the world and so use of an examination venue would be cumbersome, even without the paper. From the point of view of pedagogy, I don't think an examination is a realistic way to assess real skills. As a student I would not consider enrolling in a course which had a high stakes examination.

Providing Culturally Sensitive Feedback to Higher Degree Students

Yesterday I attended one of a series of workshops being given by Dr Sylvia Alston and Dr Elke Stracke, as part of the Office of Learning and Teaching  (OLT) funded project "Giving and receiving written feedback in HDR supervision: enhancing supervisors' and candidates' skills in a cross-cultural context".

The attendees were divided into groups of about five, who undertook a series of activities. These generally involved first filling in a one page paper activity sheet and then discussing the results. First there were two questionnaires, one for supervisors of Higher Degree Research (HDR) and one for students. The workshop activities were then on topics such as the effect the medium used for student-supervisor communication has (such as oral face to face versus email) and the possible tone of supervisor feedback. One topic which came up in discussion was dealing with students (and supervisors) who have English as a second language.

What struck me was that while this discussion was interesting, none of it was new. This was covered in the Graduate Certificate in Higher Education I undertook last year. Some of this was covered in the course I took on supervision (requiring preparation of a "Statement of Supervisory Expectations"), but also the skills on teaching generally are applicable. It is not clear what it is intended to accomplish with these workshops, beyond what is already provided in coursework.

Also the workshop was presented as a curiously old fashioned way, with no on-line resources and paper handouts. At the end we were handed a sheet of references, which might be very useful, but why do I have to type in the URLs from a sheet of paper? The workshop would be improved by applying 20th century techniques, having on-line resources before, during and after the workshop.

Workshop facilitated by Elke Stracke, Sylvia Alston and Joelle Vandermensbrugghe ...

Description: This interactive workshop examines traps to avoid and offers tips to ensure successful feedback practice in HDR supervision across cultures.

Feedback lies at the heart of any learning experience, and giving and receiving feedback is an important and integral part of postgraduate supervision practice. However, its complexity is not always recognized and the practice not discussed by candidates and supervisors. Students tend to be concerned by a lack of clear directions, while supervisors tend to complain about the lack of take-up of feedback. Often, cross-cultural differences between supervisors and candidates can add to such mismatches.

Find out more about feedback in this interactive workshop that draws on recent research in the field. In this 2-hour workshop you will
- explore how to give and receive oral and written feedback;
- examine supervisors' written language use and candidates'reactions;
- develop critical awareness of potential conflict due to language use and cross-cultural differences;
- increase your skills (as supervisor or candidate) in giving and receiving feedback in a cross-cultural context.

This workshop provides an opportunity to think about how you, as supervisor or HDR candidate, go about giving, receiving and responding to feedback, and to find out what works best for you. ...

This workshop is part of the OLT-funded project - Giving and Receiving Written Feedback in HDR Supervision: Enhancing Supervisors' and Candidates' Skills in a Cross-Cultural Context - led by Associate Professor Elke Stracke (UC, Faculty of Arts and Design) with Dr Theresa Winchester-Seeto (Macquarie University, Learning and Teaching Centre), Dr Joelle Vandermensbrugghe (UC, Graduate Research Office), and Dr Sylvia Alston (UC, Faculty of Arts and Design)

Tips for Getting a Teaching Grant

Rosalind James and Cathryn McCormack's "Grant Application Handbook for Office of Learning and Teaching Grants: Tips and tricks for grant application preparation" (OLT, 2013) is more than just a set of tips and tricks. At 142 pages this document is large enough to be the notes for a 12 week course. It is also a large 6.8 Mbyte PDF download. The promised "TIPS AND ADVICE FOR WRITING A PROPOSAL" section is in part 9, half way through the manual.

The handbook suffers from a verbose style, as an example the introductory "Who this is written for" does not get around to saying who the manual is for, until the second last paragraph. There is a useful summary at the end of the section, which should be at the beginning:

You may find this handbook useful if you are:

• preparing a teaching and learning grant for submission to the (OLT) for the first time,
• refining a previously rejected application for resubmission,
• developing an Expression of Interest into a full application,
• a successful grant applicant who is keen to improve a following application, or
• supporting or mentoring grant applicants within your university.

While as the manual says it would be useful for applying for many sorts of grants, it does not get around to explaining what an Office for Learning and Teaching (OLT) grant is until part 2. This should be up front so as not to waste the reader's time.

The manual duplicates much of the detail of grant processes available from the OLT website. This information will likely change frequently and it would be better left out of the manual. This would then be a shorter, more readable, more general purpose manual.

The manual is published under a Creative Commons License (CC BY-NC-SA 3.0 AU), so it would be possible to edit the document, removing the superfluous material, cleaning up the slightly jokey writing style and produce a more usable manual about one third the size.

Table of Contents

PART 1. INTRODUCTION

1. WHO THIS IS WRITTEN FOR
2. USING THIS HANDBOOK
3. ABBREVIATIONS
4. ABOUT THE AUTHORS

PART 2. OVERVIEW OF OLT GRANTS

1. WHAT ARE OLT GRANTS?
2. FUNDING RANGE AND PROJECT DURATION
3. ELIGIBILITY
4. RESEARCH STATUS OF OLT-FUNDED PROJECTS
5. AIM OF THE OLT GRANTS PROGRAMS
6. OLT ASSESSMENT PROCESS

PART 3. HOW READY ARE YOU?

1. ARE YOU READY?
2. INSTITUTIONAL REQUIREMENTS
3. PROPOSAL EVALUATION CRITERIA

• Project outcomes and rationale
• Approach
• Value/Need for Project.
• Project Management
• Budget

PART 4. APPLICATION PREPARATION

1. THE PROCESS OVERALL
2. IDENTIFYING INSTITUTIONAL PROCESSES
3. SUBMISSION DATES 
4. RESPONSIBILITIES
5. IDENTIFYING SUPPORT AND RESPONSIBILITIES

PART 5. GENERATING AND DEVELOPING IDEAS

1. GETTING IDEAS THAT GET FUNDING
2. DON’T HAVE AN IDEA?
3. DEVELOPING YOUR IDEA
4. NEW IDEA?
5. BEFORE YOU START WRITING
6. CONCEPT MAPPING

1. PART 6. IMPLEMENTING YOUR IDEA: PROJECT PLANNING
2. WHAT WILL YOUR PROJECT ACHIEVE?
3. PROJECT OUTCOMES/DELIVERABLES
4. WHAT ARE YOU GOING TO DO?
5. DISSEMINATION
6. PROJECT EVALUATION

PART 7. BUILDING A TEAM AND ORGANISING COLLABORATORS

1. PROJECT PERSONNEL
2. COLLABORATIONS/PARTNERSHIP AND OTHER ROLES

PART 8. PREPARING YOUR BUDGET

1. DISCLAIMER
2. INTRODUCTION
3. RECOMMENDED APPROACH
4. PREPARING YOUR BUDGET
5. ADEQUACY AND ACCURACY

PART 9. THE NITTY-GRITTY OF WRITING

1. TIPS AND ADVICE FOR WRITING A PROPOSAL
2. ORGANISING COLLABORATIVE WRITING
3. PROJECT ABSTRACT
4. WRITING THE GRANT APPLICATION: LEADERSHIP FOR EXCELLENCE IN LEARNING TEACHING; INNOVATION AND DEVELOPMENT.
5. WRITING THE GRANT APPLICATION: SEED PROJECTS
6. SUGGESTED PROPOSAL STRUCTURE: EXTENSION GRANTS

PART 10. COLLATING FOR SUBMISSION (FINALLY!)

1. DOCUMENT FORMAT
2. THE APPLICATION CHECKLIST
3. STRUCTURE OF PROPOSAL
4. FORMATTING TO REDUCE LENGTH
5. TEXT FORMATTING
6. PRESENTATION
7. PROJECT TITLE AND ABSTRACT
8. BUDGET
9. PUTTING IT ALL TOGETHER

• APPENDIX A: OLT PROJECT CONCEPT FORM
• APPENDIX B: SUPPORT LETTER TEMPLATES AND EXAMPLES
• APPENDIX C: BUDGET WORKBOOK
• APPENDIX D: BUDGET DEVELOPMENT—WORKED EXAMPLE
• APPENDIX E: EXAMPLES OF BUDGET JUSTIFICATION
• APPENDIX F: EXAMPLES OF TIMELINE FORMATS
• APPENDIX G: HOW TO AVOID 10 ‘FATAL’ MISTAKES