Technical Standards for Learning Objects

The issue of Instructional Design technical standards,  compliance, SCORM, metadata tests and W3C HTML validation came up in the course I am doing. This is an area best avoided, unless someone is paying you to do it, as it is frustrating, time consuming and never ending. The Australian VET E-standards is a reasonable overview.

One general point: Most learning objects are a ZIP file which contains folders, the folders have web format  files for the content in them (HTML, CSS, JPEG, PNG). There are also some XML files with metadata (cataloging information) and quiz questions. E-books use a similar format.

The standards issue comes up with which version of file formats are used (HTML 4, XHTML, HTML 5 ...), what the folder structure is and what metadata is included. Vendors of products make claims as to what they support and educational institutions get stuck with particular products.

What I do is try to avoid using anything more than basic web formats for my educational content, so that this will more easily convert from one system to another. As an example, I use default formatting for headings and text. I do not specify the font, color, or size of text, so it will appear using the default of the system it is imported into. The result can look dull, but at least the student is likely to be able to read it and I don't have to spend hours fixing the formatting.

Just to show off, if you look through "IEEE Standard for Learning Object Metadata" (1484.12.1-2002) you will see my name it (I was on the balloting group, contributed one comment and then voted "yes"). ;-)

Review of a Vocational Learning Object

Currently I am a student of Instructional Design, learning to use the "Learning Object Review Instrument" (LORI). The Edutech Wiki entry for LORI had a simple explanation, listing the nine criteria, rated on a scale of one to five: Content Quality, Learning Goal Alignment, Feedback and Adaptation, Motivation, Presentation Design, Interaction Usability, Accessibility, Reusability, and Standards Compliance. A more scholarly source is Akhavan & Arefi (2014). They traced the development of LORI and then consulted experts on e-learning quality criteria using the Delphi technique, but do not appear to have significantly improved on LORI.

LORI seems to be from elera.net but I was unable to get that website to work (the Internet Archive had the latest capture at 15 December 2011). The recommended method to have a group rate an object is with the "convergent participation model" (Nesbit and Belfer, 2004). Evaluators first assess the object independently, they then compare, discuss and adjust their assessments (which can be supported with online tools). While Nesbit and Belfer (2004) do not mention it, the implication is that the goal is consensus by the group.

In such activities in the past I have found it useful to introduce an intermediary step where the individual scores are combined in some way (such as by averaging) to produce a proposed group position. Most often the group is happy with this result eliminating the need for a discussion, saving time and the risk of "group think", where a suboptimal result is selected for the sake of consensus.

The LORI manual is only 12 pages long (Nesbit, Belfer & Leacock, 2003). The manual shows a star rating system for each of the nine criteria, a paragraph describing a what a five star rating would be, a paragraph describing the (lowest) one star rating and an example paragraph. There is a sample scoring sheet.

Here is what I came up with for my LORI example:

Learning Object: "Undertake business planning" (Australian National Training Authority, 2014)

Reviewer: Tom Worthington

General Remarks: The module provides Topics for a simple why what and how of a business plan. There are multiple choice Self tests, which provide feedback. The Activities provide simple scenarios for the student to discuss. A detailed Example business is provided.

 Scores are: 1 Low to 5 High

1. Content Quality: Veracity, accuracy, balanced presentation of ideas, and appropriate level of detail: 4 [Minimal theory is provided, as suits a Certificate IV in the Australian Vocational Education and Training (VET) System]

2. Learning Goal Alignment: Alignment among learning goals, activities, assessments, and learner characteristics: 4 [The object is for the VET Training Package "Business Services", level Certificate IV in Small Business Management (BSB40407), competency "Undertake small business planning" (BSBSBM404A). The goals, activities, assessment are aligned with the needs of someone setting up a business for the first time.]

3. Feedback and Adaptation: Adaptive content or feedback driven by differential learner input or learner modeling: 2 [The only adaptive content are hints in response to incorrect questions in the multiple choice quizzes]

4. Motivation: Ability to motivate and interest an identified population of learners: 3 [The case studies used are likely to be relevant to a student seeking to set up a small business or gain employment in one. However, there is no system of "badges" or other learner rewards to keep them studying, just the quiz results.]

5. Presentation Design: Design of visual and auditory information for enhanced learning and efficient mental processing: 5 [Clear text and simple diagrams are used, as suits the web based IMS Content Package format.]

6. Interaction Usability: Ease of navigation, predictability of the user interface, and quality of the interface help features: 4 [The web based IMS Content Package format is used, with its standard interface. This has some limitations depending on the Learning Management System used to host the package (such as Moodle).]

7. Accessibility: Design of controls and presentation formats to accommodate disabled and mobile learners: 2 [The web based IMS Content Package format allows for disabled and mobile users, however the standard Moodle interface does not accommodate Mobile devices well. The use of Flash for the quizzes limits the accessibility of the system).]

8. Reusability: Ability to use in varying learning contexts and with learners from differing backgrounds: 2 [The content is designed for Australian, English speaking students who have completed school. This would not suit those for other backgrounds and different education levels (in particular may not be suitable for university degree program use). But that is not the intended purpose of the object.]

9. Standards Compliance: Adherence to international standards and specifications: 4 [The module is implemented as an IMS Content Package (and claims SCORM 1.2), using web standard formats for text (HTML 4.01 Transitional), scripting (CSS and Javascript) and images (GIF and JPEG), thus conforming to standards. PDF is used for some documents, however proprietary RTF and XLF formats are used for others (not Open Document Format 1.2, or simply HTML). Also the text appears compatible with web accessibility standards (W3C WCAG  2.0). The content, apart from the quizzes appear broadly compatible with mobile devices (W3C mobileOK Scheme 1.0), despite problems with Moodle's mobile compatibility. However, the major problem in terms of standards (and accessibility) is the use of  Shockwave Flash for quizzes. ]

References

Akhavan, P., & Arefi, M. F. (2014). Developing a Conceptual Framework for Evaluation of E-Content of Virtual Courses: E-Learning Center of an Iranian University Case Study.

Arefi (2014), Developing a Conceptual Framework for Evaluation of E-Content of Virtual Courses: E-Learning Center of an Iranian University Case Study, Interdisciplinary Journal of E-Learning and Learning Objects, 10(1), 53-73. Retrieved from: http://poseidon01.ssrn.com/delivery.php?ID=809117069101031088086104127087029074000085037059021024124065101030009029090088008110060097037059060026020097006127017116122102025086030014047009024118073002002120004005033009093106120067119029115088127103&EXT=pdf&TYPE=1

Australian National Training Authority. (2014). Undertake business planning. Retrieved from https://nationalvetcontent.edu.au/share/page/document-details?nodeRef=workspace://SpacesStore/3ae72d24-e122-4b43-af5e-d2ae15d08740

Nesbit, J. C., & Belfer, K. (2004). Collaborative evaluation of learning objects. Online education using learning objects, 138-153. Retrieved from: https://web.archive.org/web/20080128191654/http://www.elera.net/eLera/Home/Articles/ColEvaluation.htm

Nesbit, J., Belfer, K., & Leacock, T. (2003). Learning object review instrument (LORI) manual 1.5. Retrieved from: https://web.archive.org/web/20111113192806/http://www.elera.net/eLera/Home/Articles/LORI%201.5.pdf

Australian Technical Standards for Online Course Materials

The E-standards Expert Group has issued updated standards for online course materials for the vocational education and training sector in Australia: "VET E-standards for Training" (Department of Industry, 10 April 2014). While aimed at TAFEs and non-government Registered Training Organisations (RTOs), these standards are equally applicable for schools and universities. The standards recommended are not controversial, being commonly used on the web.

What might be less familiar are the formats for packaging educational material, such as IMS Content Packaging (Version 1.1.4) and SCORM 1.2. There will still be difficult decisions to be made, for example, if you have some content, should you package it as an EPUB ebook, IMS or SCORM?

In many cases these standards will not be so important to individual course designers and teachers, as they will have to use what the software provided by their institution supports. However, educators should keep the standards in mind so that their content can be ported more easily from system to system.

In practice I try to use simply formatted web pages for course content. I use HTML in preference to PDF, Powerpoint and DOCX files. This way the content can be converted to other formats as required and can be assembled into e-books and modules, using whichever packaging standard is appropriate.