Assessing Competencies with Mahara

SmartEvidence overview page

Last week I attended a meeting in Canberra discussing how new features in Moodle and Mahara might be used for assessing the competencies of university students. Mahara is introducing SmartEvidence later in the year and Moodle already has "Competencies".

Mahara SmartEvidence seems to be more suited to the reflective development of higher-order skills, while Moodle Competencies may be useful for low level skills. SmartEvidence might be used across a program), while Competencies would be used within courses).

SmartEvidence is being introduced in Mahara version 16.10 (due for release 31 October 2016). There is a brief overview of Mahara SmartEvidence in the manual and detailed technical specification. The tool is intended to allow a "competency framework" with prescribed "competencies" (also called "skills" or "attributes") to be defined. A student can then nominate which pages in their e-portfolio they believe provides evidence of the competences. An assessor can then record if the evidence partly or fully meets the requirements. Without SmartEvidence a student can use Mahara to present their e-portfolio, but they and the assessor then has to manually tabulate the competencies.

SmartEvidence emulates the paper based process traditionally used for competency based evaluation. Competencies can be grouped into clusters of competences to make it easier to deal with a large assessment process. As an example, Athabasca University's MEd has six clusters, with a total of 47 competencies (Hoven, 2015).

SmartEvidence may not provide all the features needed to replace the on-line processes now used with Mahara and Moodle. As an example, SmartEvidence allows a student to nominate a page to support multiple competences. However, there is no way to point to a specific part of the page and the assessor is left to manually look though material seeing what may match.

SmartEvidence appears to have no provision for multiple levels of assessment. In particular there is no provision for peer review of e-portfolios by other students, which I have found to be a very useful part of a program Capstone (both as a reviewer and someone being reviewed).

In addition the competency frameworks for SmartEvidence are assumed to be centrally managed and fixed. The assessor cannot make up their own framework and cannot modify a provided framework. This makes sense where there is a centralized approach used, with a division between assessment designers and implementors, however, it limits flexibility.