Wednesday, July 31, 2013

Alternatives to Email for e-Learning

Recently one of the institutions I teach at outsourced its email system to a "cloud" provider. This involved the transfer of tens of thousands of email accounts from about 100 mail servers. Much of the benefit of this project came from tracking down all the email servers and accounts (the outsourcing is not necessarly a good idea).

The email rehosting went reasonably smoothly, but one of the surprises was how much email people keep and the inappropriate uses it is put to. Email systems are intended for communication between people, not for long term storage of official records. Large and important documents should be held in corporate management systems, not scattered throughout the email. There are other more appropriate communication channels for specialized tasks, such as teaching and collaboration.

For communication with a group there are many products which will let you collaborate. Some of these are design for specific purposes, such as Moodle's forums, for student discussion. There also also web based tools for one-to-one communication with students. These can seem more cumbersome than email, but all the communication is neatly kept in the Learning Management System, indexed by student, where another teacher can pick it up if needed.

Large amounts of email traffic is generated sending around drafts of documents. This can cause problems when you loose track of what is the latest version, or if an email goes astray and a draft gets into the wrong hands. A better approach is to upload the document to a central management system and have it send everyone an alert. They then log into the central system to get the latest version. If an email goes astray, no harm is done as an unauthorized person will not have access to the central system.

Preparing a Poster for a Conference

A poster is a short form presentation at an academic conference. The process is much simpler, and often used to introduce new postgraduate students to conferences. Unlike a full refereed paper, the author just submits a short proposal, here is my poster "International Graduate Level Sustainable ICT Course" for CCA-EDUCAUSE Australasia 2011 (now called THETA Australasia).

Consider a Poster and Paper


Even if your full paper has been accepted for a conference you may still want to prepare a poster as well. Academic conferences have multiple streams, so every delegate can't attend every session. There is usually a poster time, where the author standards next to their poster and answers questions, giving you a chance to talk to people one-to-one, rather than the sometimes adversarial question and answer sessions in the paper sessions.

Unlike a full paper, which has to be submitted months beforehand, the author brings the physical poster with them to the conference. The poster is usually a paper document (possibly laminated with plastic for strength). Printed on one side of the page in colour. The poster might include a small sample object, or even a computer screen. But usually no power is provided and limited security, so any valuable or breakable object on the poster may be lost, stolen or broken.

The poster is placed on panels in an area of the conference venue, usually for the full period of the conference. The poster can include contact details for the author, so a delegate can arrange to meet. The author can use a computer for a presentation, but will have to prop it up as best they can, with no desk provided. Therefore a hand held tablet computer might be best for a demonstration.

Include a QR Code

It should be noted that many delegates at conferences, particularly at high tech ones, will have a smart phone or tablet computer. If the poster has a web address and a QR Code (a 2 dimensional barcode), this will allow them to obtain a copy of the poser, additional text, a video presentation and submit a question for the author. The Google URL Shortener will create a QR code from a web address.

Posters vary in size, but are usually 750 mm by 1200 mm  in the USA and A0 elsewhere (841 mm x 1189 mm). Large posters are usually in landscape mode (long side horizontal), but portrait mode is better for smaller ones.

Make it Bold

There may be hundreds of posters at a conference, with less than ideal lighting and positioning. Therefore the graphics used have to be bold and text kept short, in a large readable format. Many authors make the mistake of putting the text of a whole paper on the poster. The result is not readable from a comfortable distance. It is better to think of the poster as the slides for a short presentation.

"Advice on designing scientific posters" (Colin Purrington, 2011) provides a good overview of what to put in a poster. This suggests a person should be able to read the poster in at most 10 minutes, equivalent to a 15-minute talk.That sounds like too much content to me. A sample poster is provided by ANU CECS has about 1600 words (22,000 characters), which seems more than is desirable.

The conference organizer might provide a template for the conference, or your organisation may do so. ANU CECS have a LaTeX template (Zip) and a PowerPoint template (PPT). There are also specialist products, such as PosterGenius. Usually the poster will be created as a PDF file and then printed using a large format ink jet printer. When creating the PDF file, keep in mind that the images will be much larger than on a normal printed page and so need to be at a suitably high resolution.

A typical poster layout will have a header and footer one eighth the height of the page. The title will be at the top center. The organization name at the bottom. The main area of a large poster may be divided into four columns. Images may extend over more than one column. Usually dark text is used on a light background for the main area (sanserif font). Some designs have background images or patterns, but this can make the text hard to read.

An interesting variation on the academic poster is the Quad Chart format, as used for the NSF 2011 SBIR/STTR Phase II Grantee Conference. This divides the page into just four quadrants, each on a specific topic. It is popular for use by US military and government research funding bodies to obtain very brief descriptions of research proposals. The Georgia Institute of Technology Systems Realization Laboratory uses these topics for quad charts:
  1. Motivation & Objectives
  2. Approach
  3. Key Results
  4. Summary & Conclusions
The quad chart is in effect four slides for a very brief pitch presentation.

Consider a Smaller Poster

This year I was attending an international conference (ICCSE 2013) and wanted to display a poster, in addition to my presentation. But I was traveling extensively and did not want to be burdened with a large poster tube. I fond that an A2 poster (one quarter the area of the usual A0 poster) would fit in an airline carry-on bag. Printed on white polyester and rolled into a tight 10 mm cylinder, it resisted crushing well (the polyester looks like glossy paper, but is much stronger and thinner).

Two columns of text in A2 portrait worked well. The simplest way to format this was with the word processor set for an A4 page, which was then enlarged to A2 while printing. I used 10 point for regular text and 12 point for headings, but this was still a bit small when enlarged. 11.5 point text with 14 point headings might be better. Also I included a QR code, which people scanned with their mobile phone for more details on-line.

Perhaps academic conferences should switch to A2 size posters with QR codes as standard. This would allow twice as many posters in the available display space. It would be much more convenient for authors and organizers. It would also force authors to be more succinct in what they say on the poster. In addition A2 posters are much cheaper to have printed (a consideration for students).

Tuesday, July 30, 2013

Green IT/IS Education and Training

Ellen England and Summer Bartczak produced the very useful paper "Where Can Green IT/IS Education and Training Be Found Today?" (Journal of Sustainability Education, 19 March 2012). They conducted a literature review and Internet search, looking for courses and training on sustainable computing.

England and Bartczak found that certification and training courses outnumbered college (university) courses.

Degree Programs in Sustainability

There were only two complete degree programs in green computing, both in the UK:
  1. University of Bradford Sustainable Computing MSc (in development), with core modules in Sustainable Computing Technology, Critical Contexts: Computing for the Environment, Developments for Sustainable Computing, Frameworks for Sustainable Computing, and a Research Dissertation and Project. Unfortunately this program does not appear to have proceeded past the planning stage.
  2. Leeds Metropolitan University MSc/PGDip/PGCert in Green Computing, was offered in 2012/2013. The MSc appears to have been renamed Sustainable Computing (in line with other such programs) and has core modules of Eco-Engineering, Green Computing Strategies, Green Computing Technologies, ICT & Environment, plus Project Management, Research Practice and a Dissertation.

Certificates

One program not listed by England and Bartczak  is the Box Hill Institute (BHI) Vocational Graduate Certificate in ICT. This includes  Units of Competency (what the vocations sector call "courses") in
Principles Of Sustainability (VBN762), 
Manage Improvements in ICT Sustainability (VPAU779), Use ICT To Improve Sustainability Outcomes (VPAU780). BHI is unusual as it is a vocational institution which offers postgraduate programs.I visited HI in 2009 to discuss the proposed green computing offerings and drop off a copy of my course notes. I was impressed with the institution.

University/College Courses/Modules

Many of the courses mentioned by England and Bartczak appear to be no longer offered, such as "Principles of Green IT for Sustainability" from UC Berkeley Extension. The materials for University of Massachusetts' Cmpsci 691GC: Green Computing Seminar do not appear to have been updated since 2009. But Arizona State University's CSE 591: Topics in Green Computing and Communication was being offered in Spring 2013 (with Week 1 Slides by Sandeep Gupta available). Link√∂ping University offer TDDD50 Green Computing. , covering Global ICT footprint, life cycle, Power-aware computing, plus tools and metrics.

Boston University Metropolitan College are one of the few to list textbooks with their MET CS-504 Green Information Technology by David Shirley:
  1. Green IT: Reduce Your Information System’s EnvironmentalImpact While Adding to the Bottom Line, Toby J. Velete, Anthony T. Velete, and Robert Elsenpeter, McGraw-Hill, 2008
  2. The Greening of IT: How Companies Can Make a Difference for the Environment, John Lamb, IBM Press; 1 edition (May 7, 2009) 
Other recommended texts include:
  1. Green Project Management, Richard Maltzman and David Shirley, CRC Press, 2010 
  2. Foundation of Green IT: Consolidation, Virtualization, Efficiency, and ROI in the Data Center, Marty Poniatowski, Prentice Hall, 2009.

Boston University Metropolitan College's course also requires the student to purchase a Kill A Watt Electricity Usage Monitor and use the Blackboard Learning Management System (LMS).

Prof. Fred Chon provides extensive notes for his course CS 290N: Green Computing at University of California, Santa Barbara, but the materials were last updated in 2010 and it not clear the course is still running. The textbook is The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines, Urs Hoelzle and Luiz Andre Barroso, published by Morgan and Claypool, 2009.

Saint Xavier University offers Green Computing and Technology (CMPSC 107)as part of a suite of "Environmental and Sustainability Studies Courses".

Swarthmore College offers Socially Responsible Computing  (CS91) by  Professor Douglas Turnbull, with extensive notes provided on-line.

England and Bartczak also mention my course at the Australian National University, COMP7310: ICT Sustainability (previously "Green Information Technology").

England and Bartczak don't mention the green ICT courses offered through Open Universities Australia (OUA), a consortium of Australian bricks-and-motar higher education instutions who pool their online courses. Install and configure virtual machines for sustainable ICT (ICANWK402A) from Polytechnic West is offered through OUA. Also available thorough OUA is "Green Technology Strategies" from ACS (the same course I also teach at ANU).

Non-academic Certificate Programs

England and Bartczaklist certificate programs not offered through an academic institution. The most prominent of these is the British Computer Society's BCS Certificate in Green IT. The certificate consists of a Syllabus, with Accredited Training Providers who each to the syllabus and an examination. There is also the textbook Green IT for Sustainable Business Practice: An ISEB Foundation Guide" by Mark O'Neill (British Informatics Society Ltd, 2010).

RapidStart Pte Ltd in Singapore offer Green IT Certification at three levels:
  1. Certified Green IT Associate (CGIA)
  2. Certified Green IT Specialist (CGIS)
  3. Certified Green IT Professional (CGIP)
While RapidStart describes itself as an accredited training provider, it is not clear if the green IT courses have any form of external accreditation.

The Green Computing Initiative (GCI), offers three self-study courses and corresponding examinations:
  1. Understanding and Utilizing Green Computing Technologies (UUGCT) for the Certified Green Computing User Specialist examination (CGCUS03),
  2. Strategizing, Designing and Optimizing Green Computing Technologies (SDOGCT) for the Certified Green Computing Architect  examination (CGCA01),
  3. Implementing, Managing and Optimizing Green Computing Technologies (IMOGCT)  for the Certified Green Computing Professional examination (CGCP02).
Many of the certificates listed by
England and Bartczaklistappear to have been discontinued.

ACS/ANU/OUA/AU Green ICT Course


England and Bartczaklist list the Australian Computer Society's Green Technology Strategies module for the ACS Computer Professional Education Program, which I designed and teach. This module is also offered through OUA as "Green Technology Strategies" and uses the same course content as the Australian National University, masters course COMP7310: ICT Sustainability. (the ACS and ANU versions of the course have slightly different assessment). My book of course notes used for all three courses is available free on the web:
"ICT Sustainability: Assessment and Strategies for a Low Carbon Future" (2011). As well as Kindle, iPad, ePub, PDF eBook and Paperback Editions.

A North American version of the course was developed by ne of my former students, Brian Stewart, at Athabasca University (Canada) as Green ICT Strategies (COMP 635).

More on the rationale behind the development of the course, which has been running since February 2009, is available in the conference paper and notes for "A Green Computing Professional Education Course Online" (ICCSE 2012).

This course predated the BCS green certificate and so may be the first such course, as well as the oldest still running and only one offered by four institutions internationally.

Demilitarizing the Education of Australian Military Officers

Major General Michael Smith, (Australian Army, Retired) has proposed that Australian military officers should attend civilian universities (ABC Radio National Sunday Profile, 28 July 2013 12:30PM). Currently officer cadets attend the Australian Defence Force Academy (ADFA), with an academic program provided by the University of New South Wales. General Smith suggests changing this would produce a better culture towards women in the ADF (although civilian universities have a higher rate of sexual assault than ADFA).

General Smith's proposal implies that the current ADFA campus in Canberra would be closed down and the students sent to civilian campuses across Australia. The cadets would then receive separate military training. I suggest a simpler option would be to re-designate the ADFA campus a civilian university.

Currently the ADFA campus is run by the Defence Organization, with UNSW staff providing academic education programs. The students are in uniform and live in dormitories on campus under military discipline.

If it was felt necessary to have officer cadets attend a civilian campus, this could be done at ADFA relatively simply. The signs could be changed from "ADFA" to "UNSW", the students not required to wear uniforms and the accommodation run under university college rules. Military students could be allowed to live off campus and civilian students invited to stay on-campus.

More responsibility for running the campus would be taken on by the civilian Rector (the UNSW Canberra equivalent of Vice-Chancellor) and the Commandant ADFA would concentrate on the separate military training program the cadets would receive.

ADFA already has post-graduate civilian students. It is likely that UNSW would take the opportunity to recruit more civilian undergraduate and postgraduate students.

Monday, July 29, 2013

Getting Paid for Publishing Educational Material Online

Checking my bank account today I noticed a deposit from "Cal - Trust Copyright". It took me some time to work out this was from Copyright Agency Limited (CAL). This is an Australian not-for-profit organization which collects copyright payments and distributes them to publishers and authors. They also collect
and distribute payments under statutory licenses taken out by education and government organizations. The idea is that you say on your document that the
statutory licenses apply and then receive a payment when organizations use them. This doesn't stop you selling copies of the same materials to others, or even giving the material away on-line to other users.

The rules on who holds the rights to material and how the license fees are distributed is bewilderingly complex. But if you prepare some educational material yourself (not as an employee of a educational institution), then you may be entitled to a payment.

Sunday, July 28, 2013

Government ICT Apprentices & Cadets

The Australian Public Service (APS) is currently recruiting ICT Apprentices and ICT Cadets. The Apprentices work full-time in an Australian Government agency and study part-time for a Certificate IV or Diploma in ICT at a vocational institution. The agency pays the tuition fees and provides paid study leave to attend class. The Cadets work part-time in an agency, while studying for an undergraduate ICT degree at university. The program operates in Canberra, Sydney, Melbourne, Brisbane and Adelaide. Perhaps the APS could expand the scheme Australia wide, with the student studying on-line and being supervised on-line.

Team Science Lessons for Team Teaching

"Team Science" refers to an approach to research using an interdisciplinary group. It might be interesting to extend the discussion of "Team Science" to e-learning. Universities commonly use an approach with one academic in charge of a course, assisted by tutors (much like one researcher with research assistants). Large  on-line courses require a team approach, with subject matter experts, educational designers and content creators.

I mentored a group for Shayne Flint's "Unravelling Complexity" course at ANU last semester. It was interesting to see a group made up of science, law and arts students tackle a problem. My group was working on what to do about e-waste. It is easy to underestimate the diffiulty with a cross-disciplinary team, each member with their own way of working and modes of communication.

MOOCs Like Budget Airlines or Strip Mining

In "Clay Shirky Says MOOCs Will Matter, but Worries About Corporate Players" Jeffrey R. Young writes that on-line courses will be a boon for students and parents worried by high education fees (July 25, 2013, 2:24 pm). I expect that on-line courses, including MOOCs, will result in a disaggregation of fees, rather than a simple reduction. Budget airlines have a low ticket price, but then charge for extras which used to be included, such as baggage, blankets, drinks and food. The MOOC may be free, but then you pay if you want assessment, textbooks, or a human tutor.

As an example of this, I paid for two on-line courses at USQ, but at the end found I had to pay an extra fee to get a copy of my results. It had not occurred to me when I enrolled that my results were an optional extra.

MOOCs may also be seen as an efficient way for rich countries to pillage developing nations for their intellectual capital. MOOCs provide an efficient way to identify the brightest students to offer a scholarship to and extract them from developing nations, much as minerals are extracted.

ps: My MOOCs with Books Webinar is 17 September.

Friday, July 26, 2013

Higher Education Academy Professional Recognition Scheme

The UK based Higher Education Academy (HEA) runs the HEA Professional Recognition Scheme. This is intended to lift the professionalism of university teaching. The scheme is aligned to the UK Professional Standards Framework (UKPSF). There are four levels of recognition (and typical job descriptions):
  1. AFHEA – Associate Fellow of the Higher Education Academy (Graduate Teaching Assistant)
  2. FHEA – Fellow of the Higher Education Academy (Lecturer)
  3. SFHEA - Senior Fellow of the Higher Education Academy (academic leader or middle management)
  4. PFHEA – Principal Fellow of the Higher Education Academy (senior leader)
The HEA is promoting their scheme internationally, with the Australian National University the first Australian subscribing institution. The HEA is more general than the Certified Membership Scheme run by UK Association for Learning Technology (ALT), which was also extended to Australia last year. The way the HEAPRS/UKPSF is structured is similar to schemes run for other professions, such as ACS CP/IP3/SFIA for computer professionals.

UK Professional Standards Framework

The UK Professional Standards Framework (UKPSF) consists of:

Framework 

An Introduction.

Descriptors 

Four categories of level of teaching responsibility, corresponding to the four levels of HEA recognition:
  1. Associate
  2. Fellow
  3. Senior Fellow
  4. Principal Fellow

Dimensions of Practice 

Three dimensions of practice:
  1. Areas of activity:
    1. Design and plan learning activities and/or programmes of study
    2. Teach and/or support learning
    3. Assess and give feedback to learners
    4. Develop effective learning environments and approaches to student support and guidance
    5. Engage in continuing professional development in subjects/disciplines and their pedagogy, incorporating research, scholarship and the evaluation of professional practices
  2. Aspects of Core knowledge:
    1. The subject material
    2. Appropriate methods for teaching and learning in the subject area and at the level of the academic programme
    3. How students learn, both generally and within their subject/disciplinary area(s)  
    4. The use and value of appropriate learning technologies
    5. Methods for evaluating the effectiveness of teaching
    6. The implications of quality assurance and quality enhancement for academic and professional practice with a particular focus on teaching
  3. Professional values:
    1. Respect individual learners and diverse learning communities   
    2. Promote participation in higher education and equality of opportunity for learners
    3. Use evidence-informed approaches and the outcomes from research, scholarship and continuing professional development
    4. Acknowledge the wider context in which higher education operates recognising the implications for professional practice

User Experience Designer Jobs at Universities

The University of Canberra recently had a vacancy for a "Lead User Experience Designer" to produce courses. It would be interesting to see if there are any differences in roles compared to an "Educational Designer", "Course Designer" or "Instructional Designer".
... The Lead User Experience Designer is responsible for the development of new models of teaching and learning with an emphasis on course design and delivery that drives Flexibility, Innovation, Retention and Engagement. ...

Thursday, July 25, 2013

University Television Production

Greetings from the SMPTE 2013 Exhibition, Darling Harbor in Sydney. The most interesting exhibit was Charles Sturt University's outside broadcast van,  producing a live stream from the exhibition floor.

Wednesday, July 24, 2013

International Standard for Higher Education via Professional Boides

Recently I was asked about the desirability and feasibility of international standards for higher education. Experience from professional bodies in standardising professional skills shows this is feasable and desirable. This also already provides a level of standardisation between national educational systems

Some programs in higher education are accredited as being suitable for membership of professional bodies, providing a form of standardization between institutions nationally, and in some cases internationally. In the computing profession, a commonly used standard in the USA, and elsewhere, is the ACM/IEEE CS Computer Science Curricula. A new 2013 ACM/IEEE CS Computer Science Curricula is under development.

The Australian Computer Society (ACS) accredits computing degree courses at Australian universities as being suitable for membership and has its own ICT Professional Body of Knowledge. Each university's curriculum, facilities and staff are checked by ACS before accreditation.

There are some standards for computer professional skills agreed between national bodies, thus bringing some level of international standardization. One standard from the UK and also used in Australia is the Skills Framework for the Information Age (SFIA).

Tuesday, July 23, 2013

Death of The Book in the Digital Age

Greetings from the Australian National University in Canberra, where Professor Alan Liu, UCSB, is speaking on "This is not a book: Long forms of attention in the digital age". He commented that the lectern provided in the ANU seminar room was designed for paper, not for his laptop and mouse, which tend to slide off (perhaps he needs a tablet computer). Open Journal Systems was mentioned as one of the lively formats which while mimicking traditional book publishing are much more lively.

Professor Liu mentioned "Twitter fiction: 21 authors try their hand at 140-character novels" by the Guardian, which includes a "book" of 140 characters by Ian Rankin and smallplaces by Nick Belardes, with a book  as Tweets.

Professor Liu then discussed several projects to use a computer to analysis the documents which a group of people read. One example was "Making Visible the Invisible" (2005-2014) which displays on a screen at the Seattle Central Library what is borrowed from the collection. Professor Liu mentioned the Research-oriented Social Environment (RoSE) at the University of California, Santa Barbara. This harvests books from online sources and creates Facebook-like pages for the authors (Facebook for the dead). Then the historical authors are linked to living authors. He in effect suggested that this collection of information is the book.
Professor Liu argued that the book is a long form of attention work, intended for permanent standard and authoritative use. But the historical "book" was not necessarily read in a linear fashion, nor fixed or authoritative. Presumably it was only with the invention of the printing press with mass production of identical copies bound together would make the book standardized. The Wikipedia suggests the word book is derived from "block of wood", which suggests permanence and authority (as you do not carve a block of wood without thinking about it carefully).

In my view Professor Liu missed the point that electronic documents, plus on-line forums are not emulating just books, but the process of scholarly dialogue. This then resembled a symposium, where scholars present their work and then discuss it. Books, e and paper, are just part of this.

Professor Liu is the author of  "The Laws of Cool: Knowledge Work and the Culture of Information", which is a book. ;-)

UCSB Research-Oriented Social Environment

Professor Alan Liu, UCSB, will speak on "This is not a book: Long forms of attention in the digital age" at the Australian National University in Canberra,  3:00 PM 23 July 2013. He will talk about the Research-oriented Social Environment (RoSE) at the University of California, Santa Barbara. Professor Liu is the author of  "The Laws of Cool: Knowledge Work and the Culture of Information".
Seminar

This is not a book: Long forms of attention in the digital age

A common response to an electronic book or other digital media is that, while it may be better or worse than a book, “this is not a book.” But digital media has the uncanny effect of making us realize that physical books themselves were never truly books--if by “book” we mean a long form of attention designed for the permanent, standard, and authoritative communication of human thought or experience. Recent research in book history also suggests that there was nothing sacred about the physical book as the carrier of enduring meanings and values important for a culture, nation, or people. This talk outlines methods for discovering and tracking socially repeatable and valued “long forms of attention” whether in books or other constellations of materials, in the past or the digital present. The talk concludes with a look at the RoSE (Research-oriented Social Environment) being created by a team at the University of California, Santa Barbara, directed by Liu.

Alan Liu is Professor in the English Department at the University of California, Santa Barbara, and an affiliated faculty member of UCSB’s Media Arts & Technology graduate program.

He began his research in the field of British romantic literature and art. His first book, Wordsworth: The Sense of History (Stanford University Press, 1989), explored the relation between the imaginative experiences of literature and history. In a series of theoretical essays in the 1990s, he explored cultural criticism, the “new historicism,” and postmodernism in contemporary literary studies. In 1994, when he started his Voice of the Shuttle Web site for humanities research, he began to study information culture as a way to close the circuit between the literary or historical imagination and the technological imagination. In 2004, he published his The Laws of Cool: Knowledge Work and the Culture of Information (University of Chicago Press). In 2008, he also published from University of Chicago Press his Local Transcendence: Essays on Postmodern Historicism and the Database.

Liu is currently working on books about the digital humanities and the relationship between media and history.
Venue: Theatrette, SirResearch-oriented Social Environment (RoSE) Roland Wilson #120, McCoy Crt, ANU Date: Tuesday, 23 July 2013
Time
: 3:00 PM - 4:30 PM
Enquiries
: Leena Messina on 6125 4357

Sunday, July 21, 2013

Digital Technologies Job Available Teaching Governance for Melanesia

Recently I was asked if there were any jobs available in higher education at ANU. So I looked at the Current Vacancies at The Australian National University web page. There are currently sixteen positions available: eleven Academic Vacancies and five General Vacancies. One which caught my attention is a Digital Technologies Research Fellow in the State, Society and Governance in Melanesia Program (SSGM). This involves the use of social media and new technology for teaching.

Ranking Web of Universities

Various rankings of universities have been published. These use a combination of research publications, status of university staff and some measures of teaching quality. This can influence a student's choice of an institution and so is taken very seriously by university administrators. One interesting variation is the Ranking Web of Universities (Webometrics Ranking), which bases it assessment on the university's web presence. Apart from making the process more automated, this more closely mimics a world where academics get information on-line, not from traditional publications. Interestingly the results from Webometrics are not so different from the ranking produced by more labor intensive and traditional methods.

In the latest Webometrics, the top ranking Australian university is the Australian National University (ANU) at 76 in the world. The overall rank is compued from three components: Presence Rank: 335, Impact Rank: 96, Openness Rank: 110, Excellence Rank: 131.

ANU is followed in the Webometrics list for Australia by: University of Melbourne, University of Sydney, Monash University, University of Queensland. The Times Higher Education World University Rankings  has University of Melbourne before ANU,  then University of Sydney, University of Queensland and University of New South Wales, with Monash University relegated to sixth place. The Academic Ranking of World Universities has the University of Western Australia displacing Monash for fifth place.

While universities would need to make large investments in research over many years to improve their ranking in traditional systems, it should be much simpler and cheaper for university administrators to improve the Webometrics rank. This is because web "presence" and "openness" are two of the four criteria used.

Administrators may not be able to get researchers to do research any quicker or to write better papers, but they can improve the university's web site and access to on-line publications. The web sites can be improved by using web accessibility guidelines, so that web pages are easy to access. Also universities can pay the additional publication fees to have journal papers made open access, so that readers don't need to pay a subscription to read them.

Often marketing and graphic design staff produce complex, hard to access web designs, in the mistaken belief these will appeal more to readers. What in fact happens is that web search engines can't index the content and people, particularly using mobile devices can't read the documents. It is better to use simple web formats.

Researchers will choose the most prestigious journals to publish their papers. However, these publications tend not to be "open access" and require a subscription to read the articles. Some journals offer to make individual papers open access if an additional fee is paid by the author. But academic are reluctant to pay these fees, unless there is a requirement from their institution (and a special grant) to do so. As a result these closed access papers tend to be read and cited less than open access ones, simply because they are harder to get on-line.

MOOCs with Books: Technology Plus Traditional Teaching for an On-line Education Revolution

I will be speaking on "MOOCs with Books: Technology Plus Traditional Teaching for an On-line Education Revolution" in a webinar hosted by the National VET e-Learning Strategy, 1pm AEST 17 September (12:30pm Adelaide time). There will be a face-to-face version of the event hosted by the Australian Computer Society in Canberra, 4:45pm 12 November 2013.
TOPIC: MOOCs with Books: Synchronisation of large scale asynchronous e-learning
PRESENTER:  Tom Worthington,  Adjunct Lecturer, Australian National University

A quiet revolution is taking place in Australia's schools, TAFEs and universities, with education moving on-line. Award winning education designer Tom Worthington will provide an overview of the trends and its implications for education:
  1. Massive Open Online Courses (MOOCS)
  2. Social media for education
  3. Open Source e-Portfolio software
  4. Cloud based Learning Management Systems (LMS)
  5. e-Book textbooks
  6. Portable course-ware formats
See also:
  1. Synchronizing Asynchronous Learning: Combining Synchronous and Asynchronous Techniques
  2. A Green Computing Professional Education Course Online
  3. On-line Professional Education For Australian Research-Intensive Universities in the Asian Century

Saturday, July 20, 2013

Role of Examiners Meetings

Recently I was asked about the assessment process by a lecturer who was uncomfortable with a rigid mechanical marking process, where students they though should pass had been failed for not getting the required 50% So I described the process I was used to taking part in at the end of each semester: the "examiners meeting", which I assumed every university used. Each examiner has to justify the grades of all their students to their peers. Usually the marks of the top students and those failing are looked at most closely. It is common for one of the meeting to ask to see the results for other courses for a student, if they think that student should not be failed on one course. Examiners are frequently sent off to review marks where they appear to be inconsistent.

Digital Education Capability Maturity Model

Mal Lee and Roger Broadie are working on  the "Evolutionary Stages of Schooling: Key Indicators". They have identified six stages:
  1. Traditional Paper Based
  2. Early Digital Stage
  3. Digital Stage
  4. Early Networked Stage
  5. Networked Stage
  6. Digital Normalization
The last stage is where education is distributed between school and home on-line.

This is reminiscent of various capability maturity models (CMM) which attempt to measure how far along a path of development an organization is. The best known, at least in computing and engineering, is from the Carnegie Mellon  the Software Engineering Institute (SEI). There have been attempts to apply a similar methodology to education and it may be feasable to produce a "Digital Education Capability Maturity Model". But perhaps education, at least in its traditional form, doesn't support this.

A CMM is an attempt to apply industrial process concepts to other fields. The idea is that if you have a documented, repeatable process, then you can produce a consistent quality product. But this concept has problems when applied to any creative process which relies on individuals, be it computer programming or learning. Just because an organization has had a group of people produce some computer software, or some successful students, is no guarantee a different set of people can produce the same result.

Also it is interesting that Lee and Broadie place an emphasis on "schools". In a networked digital learning environment, what is a "school"? Also they refer to "student’s digital functionality" and the workings of the technology being a student responsibility, not of the school. But there is little mention of what digital skills the teacher will require, nor how the students and teachers will acquire these skills.

It seems to me that discussion of "digital" or on-line education is a passing fad. In using on-line techniques for teaching postgraduate students over the last five years, the novelty of the technology has worn off. What I concentrate on is teaching students, suing the technology as a communications medium. Stripped of the techno-jargon, on-line pedagogy does not differ from classroom teaching.

What I see particularly at universities is on-line technology being used as a way to address teacher and student skills. Rather than directly address a lack of effective teaching skills of teachers and learning skills of students, this is disguised with the introduction of on-line technology. However, just providing some web based services is not going to, in itself, change the behavior of teachers or students.

I suggest what is needed to truly have an education revolution, is the use of the most powerful tool available in education: which is teaching itself. We need to bootstrap from the current situation, by teaching teachers and students new learning techniques, by using the available technology. The best way to teach someone to do something is to get them to actually do it.


Master of Education in Knowledge Networks and Digital Innovation

Having completed the Australian National University (ANU) Certificate in Higher Education, my intimidate task is to apply what I have learned to help individual academics and universities with e-learning. This new blog, Higher Education Whisperer, is part of that.

Also I am looking for a Masters in Education to undertake. Unfortunately ANU discontinued theirs before I could enroll. But Charles Sturt University (CSU) have a new one which looks interesting, in "Master of Education (Knowledge Networks and Digital Innovation)". Courses include:
  1. INF532 Knowledge Networking for Educators
  2. INF533 Literature in Digital Environments
  3. INF535 Information Flow and Advanced Search
  4. INF536 Designing Spaces for Learning
  5. INF541 Game Based Learning
  6. ETL411 ICT for Innovative Practice
  7. ETL402 Literature in Education
  8. ETL523 Digital Citizenship in Schools
  9. INF506 Social Networking for Information Professionals
  10. INF443 Digital Preservation
  11. EER500 Introduction to Educational Research
  12. ESC515 Classroom Technologies

Friday, July 19, 2013

Innovation in Higher Education

Greetings from the Startup Demo/Pitch Session at the Entry 29 Canberra Co-working Space, in Canberra. Entry 29 opened in May and was set up with the help of ANU to provide a place where staff, students and the Canberra community could get together to set up new business ventures. Entry 29 is similar to "Conceptnursery.Com", which I visited at the  Sri Lanka Institute of Information Technology (SLIIT) in Colombo and Fishburners in Sydney. I am talking to the group on how I set up "Higher Education Whisperer" as a commercial venture to help academics.As well as the website, I am considering software, a book and consulting to universities to help put courses on-line. What I have found it there is a large unmet demand for advice to academics.

On-line Tertiary Teaching for Research-Led Education

Tom Worthington receiving a Graduate Certificate in Higher Education from Professor Gareth Evans, Chancellor of the Australian National University, 19 July 2013.
Greetings from the conferring ceremony at Lewellyn Hall, the Australian National University in Canberra, where I am to receive a Graduate Certificate in Higher Education, from the Chancellor, the Hon Professor Gareth Evans AC QC. It was July 2011 when I started looking at postgraduate education qualifications and August when I selected the ANU, for Learning On-line Tertiary Teaching for Research-Led Education. The courses I selected in consultation with my education adviser, were: Assessment, Evaluation and Learning EDU5713 and Online Pedagogy in Practice EDU8114, online at USQ, followed by "Research Supervision" and a capstone course at ANU. The intention was to develop a form of e-learning which suits a research oriented university, like ANU. I believe I have achieved that (you can read my postings about "" along the way, but in the process re-learned an appreciation for how much hard work it is to be a student.

The graduation ceremony is a little bit Hogwarts, with the participants dressed in an assortment of academic gowns and hats (mine is giving me a headache). We all had to have our colored sashes carefully positioned.

ps:  In a 1993 fictional future history "Canberra 2020: World Information Capital" I made Mr. Evans the UN Secretary General, which caused him some amusement.

Thursday, July 18, 2013

Audio Recording Options for an On-line Course

Recently I was asked about recording options for on-line courses. The simplest recording option when creating an on-line course is not to use any audio or video. I created a whole 12 week course for which I recorded no audio and no video. I do refer the students to relevant video and audio produced by others, but I only supply text myself. This course, "ICT Sustainability", is popular with students and got me an award from my peers.

When I am producing audio I use a USB Headset. This puts the microphone close to the mouth and the audio is digitized in the headset, which improves the quality. MP3 is fine for the audio format.

If you are recording with a live audience, see if your institution has a built in recording option. The Australian National University has lecture theaters equipped with a recording system which takes the audio from the room PA system and the video from the main screen. This saves having to worry about a separate microphone. I just have to remember to press "Record" at the start and "Stop" at the end and a podcast is produced. The ANU system is called the Digital Lecture Delivery Service and many universities have similar facilities.

Journal of Australian Educational Computing

Australian Educational Computing (AEC) is a free open access e-journal from the Australian Council for Computers in Education (ACCE). The latest edition [Vol 28, No 1 (2013)] has two articles I found of particular interest: "Integrating Facebook into a University cohort to enhance student sense of belonging" (McGuckin, T., & Sealey, R.) and "Who you know and what you know: Student interaction in online discussions" (Stevens, T.).

However,  AEC is produced in a confusing array of formats. The refereed journal papers are distributed using an Open Joirnal System (OJS) website. This is an excellent way to distribute scholarly work, but ACCE have chosen to publish the individual papers as Microsft Word files, rather than easier to read HTML (web pages). The papers are also distributed as part of an iBook and PDF magazine. The web address for these is gven on the OJS site, but while I was able to highlight the address it was not a hypertext link and when I copied and pasted the link I was still not able to find the iBook or PDF. Also the magazine version includes extra non-refereed articles. It is not clear why ACCE don't simply include these other items on the one web site. The OJS system has a feature for indicating which are refereed papers and which are not.

Wednesday, July 17, 2013

Masters by Coursework Student Projects

Recently I created a student project on "Software for Synchronised Asynchronous Constructivist e-Learning" and put it in the list for Individual Project Courses in the ANU Research School of Computer Science. The project is based on my "paper for ICCSE 2013" and a frustration with current e-learning tools. The Master of Computing students have the option of a double Project Course COMP87*0. There are two assessment options for implementation and research orientated projects. There is also a proforma for an Independent Study Contract, for the student and supervisor to work out.

Learning Analytics

Greetings from the Australian National University in Canberra, where a workshop on "Learning analytics: Building evidence based practice" by Dr Shane Dawson, UniSA, is being hosted. Learning analytics is about analysis of data about students to improve courses. This has come to prominence with Learning Management Systems, as these record detailed information about what students do and when they do it. However, there are risks in misinterpreting statistics, in particular confusing a correlation with a causal relationship. As an example, there is a well known correlation between a student's participation in on-line course forums and their final result. But it is not necessarily the case that forcing students to participate will improve their results. The other question this raises is what teachers did the past: just hope that the way the educated worked?

One of the hidden agendas with learning analytics is that it works better with large amounts of data, from large courses. If you are teaching only a few dozen, or few hundred students, then some statistical techniques do not work well. There is then an impetus to have thousands, hundreds of thousands or millions of students.

One question I have which learning analytics could help with is the differences between classroom and on-line courses. I did do a quick check to see if the students who do my on-line course get different results from their classroom course: they don't (there is about 0.8 correlation for on-line and course results of the same students). But are the students who undertake on-line courses different in some way? Why do some students withdraw early on? Is there something I can do to reduce the withdrawal rate. It might be sufficient to know what program the student is enrolled in. But the central universality database has more student details, such as if they are a domestic or international student, previous results and where they previously studied.

Some of this is just a matter of good course design, assessment and normal teaching. As an example the Research School of Computer Science built itself a database years ago, with all student results in it. This is used in the end-of-semester examiner's meeting, where the results of each course are compared. If necessary, the results from previous years can be compared.

One thing which struck me about the discussion of learning analytics is the emphasis on student performance, but what about teacher performance?  The same analysis done of what students do and their results can also be applied to staff: what they do and how this effects outcomes.

Shane's paper "Informing Pedagogical Action: Aligning Learning Analytics With Learning Design" has more detail. Shane's workshop is being hosted at University of Tasmania 19 July.


This workshop, sponsored by the Higher Education Research and Development Society of Australasia and ANU Online as part of an Office for Learning and Teaching project, aims to:
  • Provide an overview of the current state of learning analytics
  • Illustrate how learning analytics can provide direct evidence of student learning
  • Explore the diversity of tools and methods associated with learning analytics
  • Provide practical ideas to apply learning analytics for evaluating and improving teaching practice.
Participants are expected to bring a laptop to enable hands-on involvement.
The high growth in adoption of education technologies such as learning management systems (LMS) across the education sector has resulted in alternate and more accessible data on learning and teaching practice. As with most online systems, student interactions with course activities are captured and stored. These digital footprints can be ‘mined’ and analysed to establish patterns of learning behaviour and teaching practice, a process described as learning analytics. Tracking the patterns of student interactions can provide detailed insight into the learning process and allows for rapid evaluation of the impact of specific learning activities. In essence, learning analytics empowers both instructor and student to make informed decisions about their learning and teaching processes, through the interpretation of educational data from both learner and teacher orientations.
This workshop provides an initial overview of the current state of learning analytics and illustrates how the implementation of multi-analytic lenses can provide direct evidence of student learning. In so doing, Dr Dawson will discuss the value of learning analytics and in particular social network analysis (SNA) as a methodology for visualizing curriculum and peer networks that are established through course relationships such as student engagement and course progression. Following this overview, participants will explore the diversity of data sources at their disposal drawing on analytic tools and dashboards such as SNAPP and LMS sources.
Dr. Shane Dawson is the Deputy Director of the Learning and Teaching Unit, and Associate Professor of Technology Enhanced Learning at the University of South Australia. His research activities focus on learning analytics and social networks to inform teaching and learning theory and practice. Shane’s research has demonstrated the use of learner interaction and network data to provide lead indicators of student sense of community, academic success and course satisfaction. Shane has also been involved in developing pedagogical models for enhancing creative capacity in undergraduate students. He is a co-founder and executive member of the Society for Learning Analytics Research and was co-chair of the 2012 Learning Analytics and Knowledge conference in Vancouver, Canada. ...

Funded by HERDSA and the Australian Government Office for Learning and Teaching.

Tuesday, July 16, 2013

Canberra Innovation Startup Demo/Pitch Session

A Startup Demo/Pitch Session will be held at the Entry 29 Canberra Co-working Space, in Canberra, Friday, 19 July 2013. Entry 29 opened in May and was set up with the help of ANU to provide a place where staff, students and the Canberra community could get together to set up new business ventures. Entry 29 is similar to "Conceptnursery.Com", which I visited at the  Sri Lanka Institute of Information Technology (SLIIT) in Colombo and Fishburners in Sydney.

There is also the annual "Innovation ACT" program sponsored by the ACT Government to encourage higher education students to take their ideas further.On the program students prepare business plans and practice their pitch. This provides an interesting counterpoint to the sort of documents and presentations which academics are trained to give.

How to Break Into University Teaching

Just had a message from someone in the IT industry: "I'm interested in exploring an opportunities that may exist to lecture at ANU ... Do you know how I would go about making this happen?". The easiest way is to know someone and have an existing track record. In this case the person has a Masters from ANU in the relevant discipline. This makes it easier as they not only know the topic, they know people in the Department. I suggested they contact the relevant Professor and say they were available. The other thing to do is to sign up for some teacher training. University departments run short training sessions in tutoring, as well as longer courses on education. Unlike the vocational education sector and schools, there are no formal educational qualifications required for lecturing, but it does help. Universities generally run an introductory course (equivalent to one semester unit) and a postgraduate certificate course (equivalent to one semester full time).

Monday, July 15, 2013

What Social Media can do for Universities

The topic for the next ANU Collegiate Lunch in Canberra on 7 August 2013, is What can the 'social media' do for us?", with myself leading the discussion. The "us" in this case is academia. For those at university, the idea of virtual communities should not be a new one.

Academia is a Virtual Community

Academics have used publications, correspondence and meetings to keep in touch with remote colleagues. Use of the Internet and the web has made this scholarly communication easier. At the same time the cheaper and easier access to information has challenged the role of the university and of the academic.

In the past academics were rated on their ability to get information published, but now anyone can publish and read for almost free on-line. You don't have to visit a university to read a journal and don't have to have the institution behind you to have your work published.

Creating and maintaining virtual communities still required effort and also new skills. I am frequently asked how "young people", "students" and "early career academics" can be encouraged to take part in events. The concern is that these people are not coming to old fashioned lectures, seminars, symposium, meetings and conferences.

Flip the Symposium


My preferred solution to the problem of a lack of attendance is to "flip" the event. The term comes from the "flipped classroom", where students study the material on their own and then get together in a classroom for group activities. The idea is that group time is too valuable to be wasted on everyone just passively listening to one person talking.

Research shows that student learn much better when actively engaged in doing something. Presumably this also applies to academic discourse. Rather than just sitting in the conference listening, they can be doing something active.

A new mode for conferences is to provide the papers and presentations in advance. The face-to-face time of the conference (if there is one) can then be mostly devoted to discussion. An example of this is the "Global Conference on Research Integration and Implementation". This will be held at the ANU in Canberra, 8 to 11 September 2013, but posters for the event are already up on-line.

Even when there is a formal presentation, the audience can be discussing the topic on-line at the same time. Some of the "Bar Camp" style events I attend have a twitter hash tag for the conference and encourage delegates to use it during presentations. This can be confronting for the presenter, who may have a display of the stream of discussion appearing behind them, as they speak. But this does make for a more lively format.

Social Media is a Skill to be Learned


It need to be kept in mind that using social media requires some practice and considerable effort. It is a mistake to assume you can simply create a Twitter hash-tag and the job is done. Also the aim and measures of success need to be adjusted. Having an on-line discussion may attract people to a face-to-face event, but some will be happy with just the on-line component and not turn up.

The rapid pace of on-line discussion can be frightening for those used to the leisurely pace of paper correspondence and annual conferences. If you advise something is available on-line, you had better make sure it is, or there will be complaints. It is very easy to leave the discussion on a Friday afternoon and find on Monday morning that chaos has broken out over the weekend.

The use of social media is something that university students should learn, along with how to write a paper, as part of their basic degree education. Those intending to have a career in research, university teaching or industry will need to know how to collaborate and mentor on-line. For that I have proposed Incorporating Professional Skills in the ANU Master of Computing” and this could be extended to all ANU degree programs.

Current Use of Social Media at ANU

Keep in mind that using "social media" does not necessarily require the use of Twitter, Facebook, or some other commercial product. You might want to have your students in a "walled garden" where their information is confined to the institution or just the class.

ANU Alliance is a forum for ANU staff and students implemented using Sakai open source software. Also forums are provided in the Moodle Learning Management system for students to discuss course topics. For more general discussion Yammer is used by staff.

It can be difficult to convince academics that they need to learn how to communicate, so I have proposed we get them while they are young and teach professional on-line skills to postgraduate students.

Propose Crowd-Sourcing ANU Policy


Social media can be used to help run a university, as well as in teaching. Universities have a tradition of consulting their staff, students and the community on what they do and how they do it. However, as universities get larger this consultation becomes more difficult. The ANU Vice Chancellor, Professor Ian Young, is reported to have recently said:
"The University has hundreds of policies, and to create a formal consultation process in the development of such policies would be a huge administrative overhead that would slow down the process and add significant cost." From: ANU revises investment policy, Matthew Raggatt, Canberra Times, 18
However, I suggest the use of social media would allow staff, students and the community to be consulted with minimal administrative overhead, quickly and at low cost. In this way policy can be in part "crowd-sourced".

This is not to say that every decision will be put to a public vote, but as was done recently over budget cuts, the wider university community can be consulted. Rather than set up an ad-hoc process for each consultation, the ANU could implement a system for routine and ongoing consultation, via the Internet.

An example of where such consultation would be useful is the contentious issue of ANU's investment policy.The group Fossil Free ANU proposes a move away from investing in fossil fuel. However, more would be needed for a ethical investment policy and corporate social responsibility.

Pia Waugh at the Department of Finance initiated two short courses for public servants on "Online engagement". I talked at one of these and the same tools and techniques are applicable to universities.

ANU offers courses covering social media, including "Science Communication and the Web" (SCOM8012), "Online Research Methods" (DEMO8087) and "Environmental Policy and Communications" (EMDV8007).
Charles Sturt University (CSU) offer a course in "Social Networking for Information Professionals" (INF506), as part of their Master of Education in Knowledge Networks and Digital Innovation (which I am considering enrolling in).

Recent Research on Social Media in Education

For some recent Australian research on social media for education, see:
  1.  McGuckin, T., & Sealey, R. (2013). Integrating Facebook into a University cohort to enhance student sense of belonging: a pilot program in Sport and Exercise Science.. Australian Educational Computing, 28(1). Retrieved from http://ojs.acce.cc.com.au/index.php/AEC/article/view/13
  2. Stevens, T. (2013). Who you know and what you know: Student interaction in online discussions. Australian Educational Computing, 28(1). Retrieved from http://ojs.acce.cc.com.au/index.php/AEC/article/view/6

Student dual citizenship in digital and physical worlds

In 2012 the Australian Computer Society (ACS) commissioned a study into school student's use of computer technology and their views on a career in technology. The results have been published by the University of Canberra as "Digital Technology and Australian Teenagers: Consumption, Study and Careers" (by Karen Macpherson, May 2013).

Macpherson reports that young people have a type of “dual citizenship”: in physical space and digital space, with different language and mores. Educators then have to help students bridge these cultures. I suggest this might be though of in a similar way to overseas students who have English as a second language. In a way this has always been the case, as student learn the language and culture of higher education.

In teaching university students via the Internet, I find that they know how to use a computer (these are computer science students), but not necessarily how to use it to have a structured professional discussion.

Dr Karen Macpherson will be speaking about her research at the Australian Computer Society Branch Forum in Canberra,  6 August 2013. I will be speaking at the 12 November ACS ICT trends in Education" (I am a member of the ACS and teach in the ACS Certified Computer Professional Program).

Sunday, July 14, 2013

Secrets of Perfect Student Feedback

One of the most common mistakes I see university lecturers make is to write too much feedback and then delay providing it to the student, to the point where it is no longer of use. If feedback is being given on weekly work, then it needs to be given that week, or at the latest the next week. Feedback on major assessment items needs to be given well before the student starts work on the next major assessment.

My course "ICT Sustainability" at the Australian National University received a top student feedback score last semester (5 out of 5). The students also rated the feedback they received during the course in support of learning at 5 out of 5.

One way a teacher can improve their student's learning is to think like an Intelligent Tutoring System (ITS). An ITS is a a computer program which provides prompt and individual feedback to students. Such a system keeps track of what each student is doing and how they are doing. The educator has to design exercises for the students, broken into detailed steps. They indicate not only what is the correct response at each step but the likely incorrect responses, so the can be programmed to steer the student back on track.

ITS can be useful where there is a set process for the student to learn. But it can result in a very process driven form of learning. Also ITS takes considerable resources to program.

But even without the use of ITS, any educational designer can consider how the learning task can be broken up and what are the desired responses from the student and likely mistakes they will make. The teacher can keep track of where each student is up to and provide timely feedback to each student.

Feedback accompanied by a mark is more likely to be read by the student (formative feedback accompanied by summative assessment). The mark doesn't need to be very large (one or two percent is enough to get a student's attention). The feedback only needs to be a sentence for two, or a few ticks on a standard table of responses (a rubric).
Feedback has to tell the student what they did wrong (preferably after telling them something they did right) and how they can do it better. Also there has to be the opportunity for the student to do better on a following assessable item.

There is no point in providing extensive feedback on a final assessment item for a course, which the student gets back after the course is over. All this does is to invite the student to appeal their final grade, as that is the only way for them to improve their assessment for the course.

Thursday, July 11, 2013

Higher Education Whisperer on Assessment

For the second year running I received high student feedback scores for the course "ICT Sustainability" I designed and run at the Australian National University. Other lecturers have asked how I did this and for tips on improving their courses. I didn't think I was doing anything exceptional with course design or assessment. But then realized I had spent a year studying for a Graduate Certificate in Higher Education and four years refining on-line course design and delivery. In that time I guess I learned a thing or two. So I thought I would offer some tips on how to improve courses.

The title "Higher Education Whisperer" is inspired by Dr Inger Mewburn's "Thesis Whisperer" blog. The term whispering is from horse whispering: training based on a rapport with the animal, observing behavior and having respect. I have found the same applies to students: you need to see things from the student's point of view, look at what they actually do and treat them as people. A teacher who assumes their students are stupid, lazy and dishonest should not be surprised if they do not inspire effort from the students and get poor feedback scores.

The best way I have found to see things from the student's perspective is to enroll in a course. The obvious course to enroll in is one on teaching, where you can look at the research available on student behavior. It can be difficult for a teacher to come to terms with student behavior, even when confronted by the evidence. The most obvious case is with assessment. Assessment is very important to students, which should be obvious to teachers, but many design courses with assessment as an afterthought.

The first thing to look at to improve a course is the assessment. In a recent case I looked at the assessment for a course which was not popular with students. The first obvious problem was that there was a 60% final examination. Such examinations are very stressful for students (and for staff making them), not useful aid to learning and not an effective way to assess what students have learned. So I suggested reducing the exam to 30%.

Also I suggested moving more of the assessment to the first half of the course (up from 20% to 44%). Staff complain that students don't study until just before the final exam, but if you design the assessment that way, what can you expect? Increasing an early assignment from 7% to 20% provided more reward for the student's effort and the opportunity for this to be a learning exercise not just final assessment. For simplicity I suggested also increasing the second assignment from 13% to 20% to match the first.

The course already had 15% allocated for small weekly assessment items. This is a good way to keep students working and also having them pay attention to feedback provided (as it has a mark attached). But the assessment scheme confusingly had the top 10 of 12 items assessed. I suggested a simple sum of all 12 weeks, increased to 24% (2% per week).

Also it is important that students get their mark and feedback promptly each week for the previous week. This is particularly important early in a course so that students who are not doing well (or not doing anything) get the message: "Shape up or ship out".

The final suggested scheme was:

  1. Weekly Work: 24% (+9%) 
  2. Assignment 1: 20% (+13%)
  3. Mid Semester Exam: 10% (+5%)
  4. Assignment 2:20% (+7%)
  5. Final Exam: 26% (-34%)
Obviously the assessment scheme could be revised further, but these changes should greatly reduce the stress on students (and on staff).

Another change is to make deadline firm. Teachers make the mistake of thinking if they provide "flexibility" with assessment it will be appreciated by students. But if you have rubbery deadlines you will cause confusion, stress and a perception of unfairness. If there is a deadline for an assignment, then make it firm: students who do not submit on time get zero marks. Obviously there needs to be provision for special circumstances, such as due to illness. But this should be exceptional, not routine.

Particularly when training professionals, on whom the lives of the community depends (such as doctors, engineers and computer programmers)  deadlines matter. A professional who does not learn to deliver work on time is a danger to the community.

ps: I will be speaking on "ICT trends in Education" at the Australian Computer Society in Canberra, 12 November 2013.