Currently I am a student of Instructional Design, learning to use the "Learning Object Review Instrument" (LORI). The Edutech Wiki entry for LORI had a simple explanation, listing the nine criteria, rated on a scale of one to five: Content Quality, Learning Goal Alignment, Feedback and Adaptation, Motivation, Presentation Design, Interaction Usability, Accessibility, Reusability, and Standards Compliance. A more scholarly source is Akhavan & Arefi (2014). They traced the development of LORI and then consulted experts on e-learning quality criteria using the Delphi technique, but do not appear to have significantly improved on LORI.
LORI seems to be from elera.net but I was unable to get that website to work (the Internet Archive had the latest capture at 15 December 2011). The recommended method to have a group rate an object is with the "convergent participation model" (Nesbit and Belfer, 2004). Evaluators first assess the object independently, they then compare, discuss and adjust their assessments (which can be supported with online tools). While Nesbit and Belfer (2004) do not mention it, the implication is that the goal is consensus by the group.
In such activities in the past I have found it useful to introduce an intermediary step where the individual scores are combined in some way (such as by averaging) to produce a proposed group position. Most often the group is happy with this result eliminating the need for a discussion, saving time and the risk of "group think", where a suboptimal result is selected for the sake of consensus.
The LORI manual is only 12 pages long (Nesbit, Belfer & Leacock, 2003). The manual shows a star rating system for each of the nine criteria, a paragraph describing a what a five star rating would be, a paragraph describing the (lowest) one star rating and an example paragraph. There is a sample scoring sheet.
Here is what I came up with for my LORI example:
Learning Object: "Undertake business planning" (Australian National Training Authority, 2014)
Reviewer: Tom Worthington
General Remarks: The module provides Topics for a simple why what and how of a business plan. There are multiple choice Self tests, which provide feedback. The Activities provide simple scenarios for the student to discuss. A detailed Example business is provided.
Scores are: 1 Low to 5 High
1. Content Quality: Veracity, accuracy, balanced presentation of ideas, and appropriate level of detail: 4 [Minimal theory is provided, as suits a Certificate IV in the Australian Vocational Education and Training (VET) System]
2. Learning Goal Alignment: Alignment among learning goals, activities, assessments, and learner characteristics: 4 [The object is for the VET Training Package "Business Services", level Certificate IV in Small Business Management (BSB40407), competency "Undertake small business planning" (BSBSBM404A). The goals, activities, assessment are aligned with the needs of someone setting up a business for the first time.]
3. Feedback and Adaptation: Adaptive content or feedback driven by differential learner input or learner modeling: 2 [The only adaptive content are hints in response to incorrect questions in the multiple choice quizzes]
4. Motivation: Ability to motivate and interest an identified population of learners: 3 [The case studies used are likely to be relevant to a student seeking to set up a small business or gain employment in one. However, there is no system of "badges" or other learner rewards to keep them studying, just the quiz results.]
5. Presentation Design: Design of visual and auditory information for enhanced learning and efficient mental processing: 5 [Clear text and simple diagrams are used, as suits the web based IMS Content Package format.]
6. Interaction Usability: Ease of navigation, predictability of the user interface, and quality of the interface help features: 4 [The web based IMS Content Package format is used, with its standard interface. This has some limitations depending on the Learning Management System used to host the package (such as Moodle).]
7. Accessibility: Design of controls and presentation formats to accommodate disabled and mobile learners: 2 [The web based IMS Content Package format allows for disabled and mobile users, however the standard Moodle interface does not accommodate Mobile devices well. The use of Flash for the quizzes limits the accessibility of the system).]
8. Reusability: Ability to use in varying learning contexts and with learners from differing backgrounds: 2 [The content is designed for Australian, English speaking students who have completed school. This would not suit those for other backgrounds and different education levels (in particular may not be suitable for university degree program use). But that is not the intended purpose of the object.]
9. Standards Compliance: Adherence to international standards and specifications: 4 [The module is implemented as an IMS Content Package (and claims SCORM 1.2), using web standard formats for text (HTML 4.01 Transitional), scripting (CSS and Javascript) and images (GIF and JPEG), thus conforming to standards. PDF is used for some documents, however proprietary RTF and XLF formats are used for others (not Open Document Format 1.2, or simply HTML). Also the text appears compatible with web accessibility standards (W3C WCAG 2.0). The content, apart from the quizzes appear broadly compatible with mobile devices (W3C mobileOK Scheme 1.0), despite problems with Moodle's mobile compatibility. However, the major problem in terms of standards (and accessibility) is the use of Shockwave Flash for quizzes. ]
References
Akhavan, P., & Arefi, M. F. (2014). Developing a Conceptual
Framework for Evaluation of E-Content of Virtual Courses: E-Learning
Center of an Iranian University Case Study.
Arefi (2014), Developing a Conceptual Framework for Evaluation of
E-Content of Virtual Courses: E-Learning Center of an Iranian
University Case Study, Interdisciplinary Journal of E-Learning and
Learning Objects, 10(1), 53-73. Retrieved from: http://poseidon01.ssrn.com/delivery.php?ID=809117069101031088086104127087029074000085037059021024124065101030009029090088008110060097037059060026020097006127017116122102025086030014047009024118073002002120004005033009093106120067119029115088127103&EXT=pdf&TYPE=1
Australian National Training Authority. (2014). Undertake business planning. Retrieved from https://nationalvetcontent.edu.au/share/page/document-details?nodeRef=workspace://SpacesStore/3ae72d24-e122-4b43-af5e-d2ae15d08740
Nesbit, J. C., & Belfer, K. (2004). Collaborative evaluation of learning objects. Online education using learning objects, 138-153. Retrieved from: https://web.archive.org/web/20080128191654/http://www.elera.net/eLera/Home/Articles/ColEvaluation.htm
Nesbit, J., Belfer, K., & Leacock, T. (2003). Learning object review instrument (LORI) manual 1.5. Retrieved from: https://web.archive.org/web/20111113192806/http://www.elera.net/eLera/Home/Articles/LORI%201.5.pdf
No comments:
Post a Comment