Assessing Competencies with Mahara and Moodle

The Annotation tools in the Mahara ePortfolio is being developed to allow students to easily indicate which evidence supports particular competencies and then have an assessor verify this. There is a video “SmartEvidence: Why did it come about, and what is it?”  (and slides) by Shane Nuessler (University of Canberra) and Kristina Hoeppner (Catalyst IT) from 16 April 2015.

When I undertook the  Certificate IV in Training and Assessment (TAE40110) in 2013, the Canberra Institute of Technology (CIT) used a multi-page paper based form, with a long list of competences. For each competency there would be a page or two of evidence required. Where multiple competences were from the same evidence, there was a cross reference notation needed and much flipping back and forwards between pages. This resulted in a folder full of paper, which is presumably stored in a basement at CIT.

At the moment I am preparing my own e-Portfolio for my Master of Education in Distance Education, using Mahara. I was going to list the competences on the front page and then provide the assessor with a hypertext link to the page with the evidence. But this will require me to manually hypertext each competency to the relevant artifact page. This will be tedious to do and to maintain. The  annotation feature seems to be intended to do this automatically. But I am not sure there is enough implemented yet to be usable.

Mahara Version 15 implemented basic annotations. You add an annotation box to each artificat. This is just a text box, where you type in what competencies you are claiming. The assesor then comments and when it is approved the page is locked. There are also tags, which might be useful. More features are planned for release in October.

As it is the annotation tool might be used by tagging each annotation with the relevant competencies. The tag cloud generated by Mahara would then supply a cross reference of which pieces of evidence are relevant to which competences.

Moodle 3 Competencies

Moodle 3's Competencies may help. But appear more suited to Vocational Education and Training (VET) than university assessment. With the VET approach the student is presented with a long list of small tests: do a quiz, or submit some evidence. The assessor then gives each a tick, one after another in sequence (I had to do this for the Cert IV in Training and Assessment).

But what you will typically have with university is a small number of large pieces of evidence to support a large number of competences. As an example the capstone for the Master of Education I am doing requires 5 artifacts to support 47 competencies.

Taking the VET approach I would end up with 47 competences to fill in, submitting the same 5 artifacts on average 10 times each. The alternative is that I list the 5 artifacts, then each competency it covers. But then I have to manually provide a cross-reference table to show all competences in order and where they are covered in the e-portfolio.

What it would be useful for Mahara/Moodle to do that cross reference automatically. The student would put the e-portfolio in Mahara, with competences claimed for each artifact. A tool would then scan the eportfolio and place the cross-referenced list of competences in Moodle.

Learning reflective writing

As well as the mechanics of using an e-portfolio, there is the larger problem of learning what to write. Many courses and programs just seem to give the students a tool like Mahara, perhaps with some training on how to use the tool, but no training in reflective writing. This is something which students need to practice all the way through a course or program.

Zawacki-Richter, Baecker and Hanft (2010) divide a sample of e-portfolios into four competence classes:

1. SCC (socio-communicative competence),
2. MPC (methods and professional competence),
3. PLC (personal competence), and
4. AAC (activity and action competence).

I suspect I fall into the MPC class and was a little offended, on behalf of my class, by the way we are described.

MPCs are characterized as seeing group work as a “means to an end” and not crediting other group members in their e-portfolio. These comments appear to be meant as criticisms. However, group work is a means to an end, otherwise why would any rational person do it? Also I avoid mentioning others by name in e-portfolio entries (using *fellow student* and *instructor*), as otherwise it would be a breech of privacy.

Reference

Zawacki-Richter, O., Baecker, E., & Hanft, A. (2010). Validation of competencies in e-portfolios: A qualitative analysis. The International Review Of Research In Open And Distributed Learning, 12(1), 42-60. Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/893/1671