QUT innovation space

Looking for more content on innovation, I came across a report on a entrepreneurship education (EE) program run by QUT (Collet & Roberts, 2014).

The report notes the need to:

1. "Evaluate the context of the type of EE program to be delivered and the student demand for the skills training ...
   
2. Create a community that builds on three dimensions: a physical space, a virtual environment and a network of mentors and partners.
   
3. Supplement the community with external partnerships that aid in delivery of skills training materials.
   
4. Ensure discovery of the community through the use of external IT services to deliver advertising and networking outlets.
   
5. Manage unrealistic student expectations of billion dollar products.
   
6. Continuously renew and rebuild simple activities to maintain student engagement.
   
7. Accommodate the non-university end-user group within the community.
   
8. Recognise and address the skills bottlenecks that serve as barriers to concept progression; in this case, externally provided IT and programming skills.
   
9. Use available on-line and published resources rather than engage in constructing project-specific resources that quickly become obsolete.
   
10. Avoid perceptions of faculty ownership and operate in an increasingly competitive environment.
    
11. Recognise that the continuum between creativity/innovation and entrepreneurship is complex, non-linear and requires different training regimes during the different phases of the pipeline. One small entity, such as the QIS, cannot address them all." From Collet and Roberts (2014, p. 5).
    

The most popular events were:

1. "Ideas & IP (workshop)
2. Pitching Practice (workshop series)
3. Thirsty Thursday (events)
4. Student Start-Up Night (events)
5. Design Thinking with Deloitte (workshops)
6. Story of a Start Up (guest speaker series)
7. Pitching Competitions (events/workshops)
8. Five day business course (course)
9. Innovation Boot Camp (two day introduction to entrepreneurship)
10. Open House (event/workshop/speaker series)"

From Collet and Roberts (2014, p. 39).

What is perhaps more interesting is exploring the activities which were not popular, as these could be ones which would be appropriate for an on-line course which complements the face-to-face activities. Detailed instructional design allows for the student to be taken through a learning experience which is not fun, like a workshop, but may be necessary for them.

 

Reference

Collet, C., & Roberts, J. (2014). The QUT innovation space: a trans-disciplinary learning environment for entrepreneurship education: final report. Retried from http://eprints.qut.edu.au/72840/

TechLauncher Team Formation Day

Greetings from the "TechLauncher Team Formation Day" at Canberra Innovation Network (CBRIN). This is for Australian National University computing students to produce real software, with the option of setting up a company to sell it.

There is a team of five students working on my project "Better Webinar Tool For Teaching". The idea is to combine the features of a Learning Management System (like Moodle), real time Webinar software.  Also I gave a talk to high school students about it (they had useful suggestions, including simultaneous language translation and looking at what Khan Academy already offers). One of the Angels (a person who finds start-up businesses) then got the students working on my project to "pitch" it (even though they had only been working on it for 20 minutes).

It was exciting to see the students enthusiastic about working on project, not just because they get credit for their studies (and might make their fortune), but because they wanted to create something of lasting value.

Future researchers, writing the Canberra equivalent to "The Cambridge Phenomenon" may trace a change in Canberra's economy this event. The ANU campus is located next to Canberra's CBD (blandly named "Civic"). The CBRIN office is in the zone next to the campus which was where the legal and insurance offices of the city are, but is becoming a hi-tech startup area. Within a few years this may take over the whole of the CBD, occupying former government offices and becoming Canberra's third most significant industry (after Government and Education).

Google Wants Coders Who Can Communicate

Dr Will Uther, from Google Sydney, talked to the PHD students at the Australian National University this afternoon about "Some things a PhD student needs to know before applying at Google".  One point he made was that PHDs need to practice their coding skills for Google job interviews. Also he suggested trying some of the problems in Cracking the Coding Interview: 150 Programming Questions and Solutions (Gayle Laakmann McDowell, 2011), for practice.

Soft Skills Review of Australian ICT Degrees

ACS is reported to be reviewing their ICT Profession Core Body of Knowledge (CBOK) and are particularly interested in what employers want in terms of "soft skills" of graduates. See "ICT tertiary courses under review", by David Swan (The Australian, 26 February 2015). I discussed how a university could use e-portfolios for teaching soft skills in "Proposal for Incorporating Professional Skills in the ANU Master of Computing".

ps: Dr Will Uther, from Google Sydney, is turning up here at ANU 3pm, to tell us what they really want from graduates.

Designing an Innovation Course: Part 2 Some Definitions

In Part 1 I looked at "Needs Assessment and Proposal Development" for a course in "Innovation, Commercialisation and Entrepreneurship in Technology". With a structure, now I need some content to teach. So I started an open access search for materials I could use for a course. The first requisition to answer is "What is Innovation?".

The first item to pop up was the Oslo Manual (OECD/Eurostat, 2005). This is for collecting statistics about firms. This at least provides a useful definition of:

"An innovation is the implementation of a new or significantly improved product (good or service), or process, a new marketing method, or a new organisational method in business practices, workplace organisation or external relations." From OECD/Eurostat (2005, p. 46) emphasis added.

"Innovation activities are all scientific, technological, organisational, financial
and commercial steps which actually, or are intended to, lead to the implementation of innovations. Some innovation activities are themselves innovative, others are not novel activities but are necessary for the implementation of innovations. Innovation activities also include R&D that is not directly related to the development of a specific innovation." From OECD/Eurostat (2005, p. 47) emphasis added.

 OECD/Eurostat (2005, pp. 47-51) identify four types of innovations (emphasis added):

1. A product innovation is the introduction of a good or service that is new or significantly improved with respect to its characteristics or intended uses. This includes significant improvements in technical specifications, components and materials, incorporated software, user friendliness or other functional characteristics.

2. A process innovation is the implementation of a new or significantly improved production or delivery method. This includes significant changes in techniques,
   equipment and/or software.

3. A marketing innovation is the implementation of a new marketing method involving significant changes in product design or packaging, product placement,
   product promotion or pricing.

4. An organisational innovation is the implementation of a new organisational method in the firm’s business practices, workplace organisation or external relations.

While the OECD likes to collect statistics for these neat categories, in order to innovate iot is likely that you will need to undertake several. As an example an organizational innovation which gives more workplace autonomy for decision making might be implemented through software and an on-line forum, making it a product innovation as well.
One point of difference is that OECD/Eurostat (2005, p. 56) argues that "It is not an innovation to stop doing something, even if it improves a firm’s performance.". However, removing unnecessary work practices and processes can be a significant innovation.

Novelty versus diffusion

OECD/Eurostat (2005), points out that innovation requires some degree of novelty and distinguishes three categories:

1. New to the firm: "A product, process, marketing method or organisational method may already have been implemented by other firms, but if it is new to the firm (or in case of products and processes: significantly improved), then it is an innovation for that firm." From OECD/Eurostat (2005, p. 57)
2. New to the market: "Innovations are new to the market when the firm is the first to
   introduce the innovation on its market. The market is simply defined as the
   firm and its competitors and it can include a geographic region or product line." From OECD/Eurostat (2005, p. 58)
3. New to the world.: "An innovation is new to the world when the firm is the first to introduce the innovation for all markets and industries, domestic and international. New to the world therefore implies a qualitatively greater degree of novelty than new to the market." From OECD/Eurostat (2005, p. 58).

 Disruptive innovation

Disruptive or Radical innovation is "an innovation that has a significant impact on a market and on the economic activity of firms in that market."  (OECD/Eurostat, 2005, p. 58). The impact of innovation may be great even where it is not very novel. As an example, the effect of the use of on-line social media tools have on the way organizations are managed is still being worked out today, even though such tools are not new. Schubert and Williams (2013) trace the use of social media in business over the last six years and comment that so far 'the most popular usage scenarios of the software are not very “social” but support
people in their daily joint work with a focus on getting the job done', so there is scope for further disruptive innovation.

The innovative firm

(OECD/Eurostat, 2005, p. 58) Define the innovative firm as one which "... has
implemented at least one innovation ...". They go on to characterize innovation activities for a period as being:

● "Successful in having resulted in the implementation of an innovation
(although the innovation need not have been commercially successful).
● Ongoing, for work in progress which has not yet resulted in the
implementation of an innovation.
● Abandoned before the implementation of an innovation., while a product or process innovator is defined as a firm that has implemented either a product or a process
innovation." From OECD/Eurostat (2005, p. 58)

References

OECD/Eurostat (2005), Oslo Manual: Guidelines for Collecting and Interpreting Innovation Data, 3rd Edition, The Measurement of Scientific and Technological Activities, OECD Publishing, Paris.
DOI: http://dx.doi.org/10.1787/9789264013100-en

Schubert, P., & Williams, S. P. (2013). The Concept of Social Business: Oxymoron or Sign of a Changing Work Culture?. Proceedings of the 26th Bled eConference, 1-14. Retrieved from http://www.researchgate.net/profile/Petra_Schubert/publication/260517948_The_Concept_of_Social_Business_Oxymoron_or_Sign_of_a_Changing_Work_Culture/links/00b495317ddc07721c000000.pdf

Digital Students at the Australian Defence Academy

Greetings from the "Red Room" at the Australian Defence Force Academy (ADFA) in Canberra (aka UNSW Canberra), where Dr Jason Mazanov is speaking on "The Digital Student: Insights from Building an On-line Masters Course". He described how ADFA moved from paper based correspondence courses to their own bespoke on-line system "Olive" and then to Moodle. More importantly he described the path from just using message boards to more sophisticated use of social media to build an on-line community of inquiry (Garrison & Anderson, 2010). He pointed out that the military on-line learning culture is didactic with completely self contained materials and students who will look after each other.

Dr. Mazanov showed one of his Moodle course designs on screen. He uses peer feedback for students and some live online (synchronous) sessions. He also uses Twitter for "come and do stuff".

Dr. Mazanov has 75 to 86 students in his masters course. One problem found from student feedback surveys  was that students though they had to get 15/15 for their online engagement. It seems to me that this part of the assessment should perhaps be competency based. He found some improvement in learning for digitally literate and non-literate students.

Some other stud net feedback was they liked discussions, but not Wikis or Twitter. Students liked on-line pre-recorded videos. Students preferred asynchronous formats. Twitter was too short for an academic discussion and they were not skilled at threading. One interesting comments from a student was that they wanted a distance eduction course, not one which replicated a campus experience (a military student cannot necessarily connect regularly when deployed).

Dr. Mazanov provided very useful examples of how to use on-line leaning for a course and credible research to back this up. However, the bigger questions remain of how universities incorporate on-line leaning in their programs and their right balance of teaching and research skills for their staff.

Dr. Mazanov pointed out the typical ADFA masters student is a middle aged male military officer, who is not a "digital native". The same might apply to other masters students from middle management positions in the public service and private industry. These students need training in how to use the Internet.

The lack of digital literacy by Australia's middle management should be of concern to their organizations. Apart from lowering the ability of these staff to educate themselves, they are  vulnerable to Phishing attacks, placing their companies, government agencies and the ADF at risk.

I suggest that improving the digital skills of officers is the cheapest way to improve the effectiveness of Australia's military forces. Western forces are, at best, halting the advance of ISIS in the Middle East. While they have superior weapons and conventional military training, the western forces, including the ADF, are deficient in digital skills and as a result are losing the information war. Training by UNSW Canberra/ADFA could help win that war and those to come, which will be fought on-line.

ps: The Red Room looks like a TEAM learning room to me with an almost square room with screens around the walls and round tables for students and the presenter walking around (Cabaret style).  One problem is that LCDs screens are used, which I found too small to read. These are also in front of windows with light coming though from behind (ADFA has much better seminar rooms).

References

Garrison, D. R., Anderson, T., & Archer, W. (2010). The first decade of the community of inquiry framework: A retrospective. The Internet and Higher Education, 13(1), 5-9. Retrieved from http://dx.doi.org/10.1016/j.iheduc.2009.10.003

Why Do Your Expect an Improvement in Learning with iPads?

Jay Ashcroft asks "Why so many schools fail to get impact from iPad?" (Learn-Maker, 23 February 2015). But how does Jay expect that just providing an iPads, or any other tablet device, on its own to improve education? My primary class was the first at my school to use ball point pens. This was a step up from pencils, with color! But I doubt that the teacher, or anyone else, would expect this change to improve student's learning. Similarly, if you give a student an iPad, but leave the teaching methods unchanged, you can't expect an improvement in leaning (I would expect a drop in learning). What is needed is to use the Internet and the iPads to teach teachers how to teach using iPads.  Apart from being slow and expensive, the old fashioned approach of train-the-trainer sessions misses the point about the change taking place in how education is delivered.

There is a new One Laptop per Child tablet/laptop in development (the XO-infinity). It is specifically designed for school students and has interesting modular features (I would like to see USB dock and mSata docs added). But the previous OLPC devices were not shown to have improved learning and I do not expect this device to either. Jay suggests several days training for each school, with a core team of teaching doing the training. But curiously they don't suggest using the iPad itself, or the Internet, as part of the training. Insrtead this is all face-to-face teaching, which I suggest is missing the point. 

ps: The Xo_infinity is intended to have user upgradeable modules. This is a good idea in theory  (several of the laptops I have had included this feature). But I suggest basing the modules on existing interface standards and form factors, to lower the production cost and widen the choice. In particular, I suggest having:

1. USB dongle bay, : This would be a recess in the base of the case with a removable cover and containing widely spaced USB sockets. Memory sticks, WiFi, 4G, TV and other USB devices could be plugged in.
2. mSata Bay:  Make a plastic case which clips around a standard size mSata board, so this then can be slid into a slot in the computer case. This way custom electronics does not need to be manufactured, just the plastic case.

STEM Education Underpins Our Prosperity

Dr Nicholas Gruen, CEO of Lateral Economics writes "STEM: part culture war, part cargo cult" (Online Opinion, 17 February 2015). STEM is short for "Science, Technology, Engineering and Maths" and as Dr Gruen points out there are numerous calls for more of it in schools and universities. But I suggest there are good reasons for this, and don't agree it is a "cargo cult" (I could hardly do that when I teach Computer Science and Net Technology). Australia's economic wellbeing and much of its society, depends on STEM. Our agricultural and mining sectors depend on science and engineering. The systems in our cities which keep the citizens fed and healthy also depend on science and engineering. Our fourth biggest export industry is education, which is  dependent on technology and scientific research.

Dr Gruen claims that "old-fashioned entrepreneurialism" has been the dominant input to the success of Silicon Valley. This is not quite true, as what has emerged is a sophisticated blending of STEM and entrepreneurialism. Computer coding is not the only contribution of STEM to Silicon Valley, there is also engineering of hardware, product design and sophisticated project management approaches which make large and complex projects possible.

There are practitioners who've taught themselves, but they are making use of billions of dollars of government investment in STEM, which is a legacy of the cold war and the space program. Even now the US government and US military fund research which enterprises can then commercially exploit.

As Dr Gruen points out, there are free resources online. But there is little future for a country which relies on the education offered free by the government and companies of another.

Australia can hope that a few crumbs from the US entrepreneurial table will fall on the floor for us, or we can invest in our own future. One way to do this is to teach entrepreneurial skills to STEM students. I am helping do this trough the Australian Computer Society. My class of "New Technology Alignment" students are up to week three of their on-line course. They first have to find a business opportunity and then propose technology for that opportunity. Those working in an organization have the opportunity to do their project, for their boss, in the workplace. At the Australian National University I am helping with TechLauncher where students have to option to set up a new company as part of their for-credit project. I will be discussing how to teach innovation, at CSIRO in Canberra, 27 April 2015.

We do need STEM in-service training for existing teachers. This can be on-line, but needs to be more than just telling them to go a look at a web video. What is required is properly designed on-line education programs for teachers and for stud nets. Australia has had such programs, particularly for rural and remote students. Those programs could be broadened for all students..

Google Scholar Anticipates My Search Request

I was about to use Google Scholar to search for research on interaction for distance education courses. But before entering any search terms I saw Google was already recommending two useful papers:

Li, J., & Greenhow, C. (2015). Scholars and social media: tweeting in the conference backchannel for professional learning. Educational Media International, (ahead-of-print), 1-14. 

Anderson, T., Upton, L., Dron, J., Malone, J., & Poelhuber, B. (2015). Social Interaction in Self-paced Distance Education. Open Praxis, 7(1), 7-23.

Australian Graduates Can Learn to Innovate at Universities, Government and Industry

Something is terribly wrong when a specialist writer on small companies and entrepreneurs doesn't understand that a government department can be more innovative than a retail computer store. Tony Featherstone in "I don't want to work at Apple" (SMH, February 19, 2015) worries that Australian university graduates would prefer to work for the government than for Apple. But as far as I know, Apple Computer does not carry out R&D in Australia, they just sell computers imported from China, running software imported from the USA. In contrast, Australian government agencies, often in conjunction with Australian universities, have been producing home grown innovation for decades.

Mr. Featherstone referred to our university students wanting to be "auditors, bank managers and bureaucrats". Currently I am teaching students how to be auditors at the Australian National University. But they are going to audit carbon emissions, not money. After checking the emissions, they then have to come up with creative ways to reduce energy use, save Australian company's money and help save the planet from global warming.

As a public servant I helped write the public policy needed to make the Internet available across Australia. We were helped in this by Australian universities. Along the way the Australian Defence Force was reequipped with new digital communications to better fight wars and deal with natural disasters.

But not all graduates want to join a large government agency. Some go to work at Google's Sydney R&D office, building better digital mapping systems. Other students join CEA in Canberra, to program the world's most advanced military radars.

Mr. Featherstone is correct that Australian universities need more incubator programs. The ACT is taking a regional approach with the major universities joining with local government to create the CBR Innovation Network. Under one roof, the center of Canberra now has a facility brining together students, investors and advisors. The Australian National University has also started TechLauncher, with the option of students working for up to two years on their start-up, for credit.

Entrepreneurship and innovation is something to teach to students. I am running a course in "New Technology Alignment" on-line at the moment, for the Australian Computer Society. But this is not just a matter of a few Powerpoint slides and then let 'em loose.

Designing an Innovation Course: Part 1 - Needs Assessment and Proposal Development

My students in the Australian Computer Society's "New Technology Alignment" (NTA) on-line postgraduate course are now up to week 3 (and doing well). At the same time I have been looking at designing a more general course, provisionally called "Innovation, Commercialisation and Entrepreneurship in Technology", to be offered on-line, initially for students in the Australian Capital Territory (ACT), Canberra. I am doing the design as as assessment for a course Instructional Design. This series of postings are excerpts of some of that material (which is therefore a little more verbose and academic than otherwise required for a workplace design exercise). Previously I looked at existing Canberra courses in "Commercialisation and Entrepreneurship in Technology Course Proposal".

Steps in Instructional Design

First a needs assessment will be carried out, followed by a proposal for what is to be developed. These first two phases will be followed by creation of one of the learning objects for the course.
An "Innovation ACT" competition was established at the Australian National University (ANU) in Canberra in 2008 (Blackhall, n.d.). The competition, now supported by the University of Canberra and the local Canberra government (the “ACT Government”), has the aim of providing):

• “Entrepreneurial education via seminar sessions ran parallel to a university semester
  
• Entrepreneurial experiences within a competition environment that allows students to test their ideas.” From Innovation ACT (2014a), emphasis added.
  

The Innovation ACT competition provides students with handbooks and templates, students attend presentations, prepare their proposals with the help of a mentor and then pitch their ideas to a panel of judges (InnovationACT, 2014b). Prasad (2014) discusses the history and educational role of such enterprise competitions and categorizes it as an “action learning” pedagogy.
While popular with students and having an educational role, the Innovation ACT competition is not part of a formal educational program and so is not evaluated as to its educational effectiveness and students do not receive credit for participation towards their studies. This document discusses how to design an on-line course which students could take in conjunction with Innovation ACT and similar competitions, as part of a university degree program. The course would be designed to fit with postgraduate certificate and degree programs in the computing discipline, as that is to author's discipline area.
Kakouris (2009, p. 231) argues for an ADDIE model (Anallise, Design, Develop, Implement and Evaluate) is suitable for providing on-line entrepreneurial education, emphasizing guidance, communication and peer support. They argue that the DE teaching material can be used to implement Gagné's Nine Events of Instruction: Gain Attention, Inform Learners of Objectives, Stimulate Recall of Prior Learning, Present the Content, Provide Learning Guidance, Elicit Performance (Practice), Provide Feedback, Assess Performance, Enhance Retention and Transfer to job” (Gagné (1965) cited in Kakouris (2009, p. 233)).

1. Is a Human Tutor Needed?

Kakouris (2009, p. 233) assert that the DE system can act as a “virtual educator” without a human tutor. More recently replacing the tutor with an automated system has been attempted with a Massive Open Online Course (MOOC) on entrepreneurship. Al-Atabi and DeBoer (2014) report on an entrepreneurship MOOC conducted with 1600 online student in 115 countries, plus 60 on-campus students. The online students formed teams and under took group projects. In addition to videos, the students received points from peers and badges to provide them with feedback on progress. Al-Atabi and DeBoer (2014) noted that the completion rate for online students was 25%, which is higher than a typical MOOC, but was far lower than the 90% completion rate for the students undertaking the same course on-campus.
Neck, Greene and Brush (2014) point out the role for the instructor to “facilitate engagement in creative processes” for higher level skills. As the course under development here is intended to be part of a conventional university degree program, a completion rate of 25% is unacceptably low. It is therefore proposed to take a middle path, having on-line materials, but facilitated by a human tutor, to achieve a completion rate comparable to face-to-face courses.

Part 1: Needs Assessment

2. Needs assessment approach

Smith and Ragan, (2005, pp. 43) suggest that an ID needs assessment should first establish if there is a need at all. They outline a cycle of needs assessment, design, production, implementation and evaluation (Smith and Ragan, 2005, pp. 44), while advocating the evaluation plans actually be constructed during the needs assessment phase. This may be unrealistic where the need for a course has not yet been established and so work on evaluation would be wasted if the course is never run.
Smith and Ragan, (2005, pp. 44) divided needs assessments into three model types:

1. Problem Model: As Smith and Ragan note, it is necessary to determine if there really is a “problem” and if a cause it the best way to solve it. In the case of an unsolicited new course on innovation, the problem model is not as applicable, as there is no current group of employees to canvas. As the aim is to have students go out and create new companies, there are also not employers to consult. The Innovation ACT competition is part funded by the local government, which has in an economic development strategy to foster new industries, a “culture of entrepreneurship” and encourage startup firms to provide employment (ACT Government 2013). The ACT Government might therefore be consulted about the problem of educating innovators.
   

2. Innovation Model: This approach looks for changes in the students, the education system or the environment. The innovation model would seem apt for a course in innovation: students are less likely to want to simply get a job in a corporation and instead want to set up their own company. The approach of involving students in an innovation competition working on a real world project is not new in the Australian education system (along with e-learning and e-portfolio systems which make it easier to offer such education), but not widely used. However, the reduction in the available of jobs for life has required graduates to be more entrepreneurial, being able to take on new roles and even invent a job for themselves. Thompson and Kwong (2013) found that “enterprise education”, designed to develop entrepreneurial skills, in UK schools had a “direct positive relationship with entrepreneurial activities and intentions”. This indicates that students will respond positively to such education and an introductory course on innovation with lead to the student doing more such work.
   
3. Discrepancy Model: The discrepancy model, as described by Smith and Ragan, (2005, pp. 45) , does not start with a new need, but as a check to see if an existing course is meeting the already established requirements. This applies to an innovation course, as some such courses already exist, along with externally set skills requirements. The analysis to be carried out therefore incorporates some elements of the discrepancy model, at least to say what is wrong with existing courses and so why a new course is required. With this the requirements will be listed and how well these are met with courses, to determine the gap.
   

3. Scope and extent of the need

a. Who to query. As there is a question over the popularity of the existing course, the first group to survey are potential students. It would be simpler to have access to students currently enrolled in a program of study (as they are easy to access). But it may be worthwhile contacting those who have not been attracted to programs, perhaps via a professional body, such as the Australian Computer Society and Engineers Australia. These bodies could also asked as to the need. Innovation organizations, such as the various “co-working” offices and “hacker” competition providers may be of use. In addition experts in the field can be consulted, as Dr. Lachlan Blackhall, founder of Innovation ACT (Blackhall, n.d.), who has worked on engaging students with real world problems (Smith, Brown, Blackhall, Loden & O’Shea, 2010).
 
b. How to Query. An on-line survey instrument could be used to collect information from potential students. This could use multiple choice and rating questions. Interviews could be used with organizational representatives. However, they are unlikely to agree to a formal social science style of interview and a more informal approach may need to be used.
 
c. Type of questions. Reimers-Hild and King, (2009) proposed six questions for entrepreneurial leadership and innovation in the context of distance education. These could be applied more generally for questioning potential students and employers about innovation courses:

1. “How entrepreneurial is your organization?  On a scale of 1-5, would you classify your organization as a 1 (not at all entrepreneurial) or a 5 (extremely entrepreneurial)?

2. How are administrators, instructors and learners in your organization learning to be more entrepreneurial? 

3. Developing a global mindset throughout an organization characterized by risk taking, innovation and change should be encouraged, not discouraged. …

4. Is innovation a priority?  On a scale of 1-5, would you classify your organization as a 1 (not at all innovative) or a 5 (extremely innovative)?...

5. In what ways can your leaders share the vision ... Can they use both face-to-face and online methods?  Can they use both individual and large group settings?...

6. How can you institutions connect employees and learners with their passions and their personal vision of the future?...

7. What is your organization doing to develop and leverage the human and social capital of its administrators, instructors and students?  …”

From Reimers-Hild and King , 2009 (emphasis added) .

d. Other data sources. While the sources discussed above may be of some use, the primary source of information will be preexisting skills definitions and syllabuses. In particular Australian computer science degrees are accredited by the Australian Computer Society (ACS, 2014). The Society promotes the use of an internationally standardized skills framework and courses are required to be “aligned” with the framework (IP3, 2015). It would therefore be appropriate to based the course on the most appropriate skills definitions in that framework. McEwan (2013) discusses the use of SFIA skills definitions (SFIA Foundation Ltd, 2015) for university courses and note it is particularly useful for fast developing new job categories (SFIA is also part of the ACS/IP3 framework). McEwan proposed the use of SFIA level 5/6 for Masters-level courses and 4/5 for Honors-level. They also found that one SFIA skill was insufficient for a typical university course and used two. In this case McEwan (2013) aligned a course with skills “Emerging Technology Monitoring” (EMRG) and “Innovation” (INOV).

Alongside the university system, Australia has a system of Vocational Education and Training (VET), which as Mazzarol (2014) points out, has been active in offering courses in entrepreneurship for small business. Some universities have associated VET Registered Training Organizations (RTOs) to deliver such courses, at a lower qualification level than their degree programs.
The VET system has a national database of standardized skills sets, make up of units of competency (Australian Department of Industry, 2013) and a database of preprepared learning objects (National VET E-learning Strategy, 2013). A search of the training database for “innovation” found a “Managing Innovation Skill Set” BSBSS00014, “Innovation Leadership Skill Set” (BSBSS00008) with units of competency “Establish systems that support innovation” (BSBINN501A), “Foster leadership and innovation” (PSPGOV604A). However, of the ten courses listed in the training database with the word “entrepreneurship” in the title, only two are currently offered, a Graduate Certificate and a Diploma of “Entrepreneurship for Food and Wine”. This indicates that perhaps the demand for such courses at the VET level is not a strong as Mazzarol (2014) suggests. A search of the database of learning objects found three relevant entries: “Communicate information and ideas”, “Plan for change” and “Manage emerging challenges and opportunities”. These may be of some limited value in the innovation course for low level skills, as may the units of competency.
The intention is that students can optionally undertake an innovation competition, in particular “Innovation ACT” alongside the course. Therefore the content of “Innovation ACT” will provide more detail as to the need.

4. Need and Causes

The Australian Computer Society already offers an on-line innovation course: "New Technology Alignment" NTA, (ACS, 2013). However, NTA is intended for employees of corporations to identify innovations within the organization. The need is to address the aspirations of students to set up their own enterprises working on their own products, rather than working for a corporation. This is in part by an innate wish to innovate and partly due to the difficulty in finding worthwhile (or any) employment in a corporation. Segal Quince & Partners (1985) argue that the growth of start-up high technology businesses around Cambridge from the 1980s was in part due to students who wanted to maintain the Cambridge lifestyle and, with the lack of alternative employment, were forced to setup their own business. The ACT Government (responsible for Canberra), is implementing a similar strategy by funding Innovation ACT, to encourage students to stay in Canberra and set up a business, rather than move away after graduation. The proposed course would teach the students skills needed to set up a business in Canberra.
What is Available: The University of Canberra (UoC) and ANU both offer innovation courses in Canberra. UoC have courses as part of the Bachelor of Entrepreneurship and Innovation (University of Canberra, 2012b). A typical unit is “Managing Change and Innovation” (University of Canberra, 2012a), offered in blended mode (on-line content with on campus attendance of up to thirty nine hours). ANU has "innovation" courses in business and engineering programs: “Entrepreneurship and Innovation” MGMT3027 (ANU.n.d. b), “Innovation and Commercialisation” MGMT7165 (ANU.n.d. c), “Engineering Innovation” ENGN3230 (ANU.n.d. a), “Technology and Innovation Management and Strategy” MGMT7106 (ANU.n.d. d). However, these are courses have largely the format of a conventional lecture and examination based university program, are not integrated with an innovation competition and not aligned with external skills standards.
What is desired: A course which is delivered on-line, can be used alongside an innovation competition to provide the student with more hands on-experience and aligned with external skills standards to provide an industry relevant and preferably global qualification.
Cause of the Needs Gap: The Canberra university courses are designed to fit within conventional classroom teaching techniques and program structures. The student is assumed to undertake their academic study at the university, receive a university qualification and then move to employment, most likely at a corporation or institutional setting.

5. Potential solutions

One solution to the problem of including innovation in a university technology program is the “New Venture Design” course for engineering and business students at UBC (Kruchten, Lawrence, Dahl & Cubbon, 2011). Since 2003 UBC's final year engineering and business students have had the option to work in mixed teams on an entrepreneurial venture. Teams of six UBC students produce a prototype and business plan. The students are provided with conventional lectures and lab work activities. Teams can optionally enter external innovation completions in Vancouver, or elsewhere. This approach solves the problem of providing students with academic credit for innovation competition, but duplicates the activities of the competition, increasing resource requirements and student effort. Also the use of conventional lectures and labs limits the course to on-campus students.
a. Instructional solutions: On-line course materials and forums for students to help form their teams can be provided. Quizzes and large assessment items, which follow the sequence and content of Innovation ACT (and similar competitions) can be provided. One issue concerns scheduling. Ideally students would be able to commence the course at any time, to suit the competition they were intending to enter. However, as such competitions depend on the students forming a team and this could be difficult to schedule within the course. An alternative strategy would be for the students to undertake the course self paced, with or without, their team. Another alternative is to have the course in a set program term and not closely align it with the competition. This would cause difficulties where the student submits competition materials as part of their assessment, but the competition and assessment deadlines do not align.
 
b. Non-instructional solutions: An alternative to a full course would be to rely on the competition materials and process to provide the entire learning experience and have the student submit evidence for assessment, as a form of Recognition of Prior Learning (RPL). This would require the student and/or the assessor to check the competition covered the syllabus (and used the same terminology) and the materials produced were suitable for assessment. Also there would be the difficulty that innovation competitions are almost always entered by teams of competitors, and are assessed exclusively on the team product. It is therefore not possible to know what contribution an individual team member made. This might be overcome by having the student keep a diary with their contributions during the competition (submitted via an e-portfolio system) and with some form of test (such as on-line quiz).

Part 2: Proposal Development

6. Components of the course

Course description: "Innovation, Commercialisation and Entrepreneurship in Technology" is a new course to for students to develop the capability to identify and develop new technology based business ideas. Students will learn to identity strategic uses for information technology, applying systematic investigation, analysis, review and documentation to take an idea through the stages of development and proposal. Students are encouraged to take part in Innovation ACT, or a similar innovation competition, and submit their competition materials for assessment.
 
Learning Outcomes
After successful completion of this subject students will be able to :

1. Investigate a strategic application of IT.

Propose new ways of conducting business using IT.

Skills Alignment:

1. SFIA Version 5, Level 6: Business analysis BUAN, (SFIA Foundation Ltd, 2015)
   
2. SFIA Version 5, Level 6: Innovation INOV, (SFIA Foundation Ltd, 2015)

Course components: The major topics (based on Innovation ACT, 2014) are:

1. Business Model Thinking
   
2. Stakeholder Engagement
   
3. Concept Generation
   
4. Value Capture
   

Activities are:

1. Contributions to on-line forums/exercises for ten weeks (assessed at 2% per week for 12 weeks, with the best 10 counted),

Mid semester assignment: “Investigation of a strategic application of IT”. Individual work of 2,000 words, plus references 40%

End of course deliverables: A business proposal. Students are encouraged to undertake the work as part of Innovation ACT, or another innovation competition. However, the activity must take during the semester. May be performed in a group of up to six with all receiving the same mark. Up to 2,000 words, plus references, 40%.

The Innovation ACT Business proposal consists of:

1. Business Model Canvas: One page diagram of the business model, using the IACT Business Model Canvas template, or similar (about 5% to 6%).

Executive Summary: One page text summary of the business model (300 words, about 5% to 6%).

Canvas Report: Five to eight page report on development of plan (this is equivalent to 1,500 to 2,400 words of assessment, about 30% to 50% of the assessment)

Continuation report: Detailed plan outlining funding requirements and proposed expenditure (Assuming 5 pages, that is 25% to 30% of the assessment).

Pitch: Notes and visual materials for a five minute presentation. A video of the presentation can also be provided, but for academic purposes, the assessment will be based on the notes for the presentation, not the presentation itself (assessment 5% to 10%).

7. The learner population

The learner population for this course would have a degree in computer science, information technology, software engineering or other technology discipline. The students would typically be enrolled in a postgraduate certificate or degree program (usually a Masters by coursework) at an Australian higher education institution. Students would be in a city where they have access to an innovation competition (such as Innovation ACT in Canberra). Students would be expected to have experience at using a computer and the Internet to be able to undertake an on-line program. They would require sufficient communication skills in the language of instruction (English) and to work in teams. As Blair and Hoy (2006) point out “... an online community doesn’t happen by sheer virtue of creating discussion forums and requiring weekly postings ...”. Also the innovation course will be a form of group Problem Based Learning (PBL), which can be subject to dysfunctional group interaction and high cogitative load (Hung, 2011). However, it is not clear if any additional skills can be asked of the students, beyond written and computer literacy.

8. How the course will be delivered

The course will be delivered as in online DE format as a 12 weekly units using non-real-time (asynchronous) delivery. Course notes will be provided an an e-book, with additional readings (and videos) and weekly exercises to complete. Forums will be provided for student interaction with each other in groups and with an instructor, via a Learning Management System (LMS) such as Moodle. An e-portfolio system (such as Mahara) will be provided for students to collate their project material. No real-time (synchronous) activities will be provided, due to the difficulty of supporting these and of students in different time zones participating.
Students will be expected to undertake team activities, and optional participation in an innovation competition, without further support from the course (organizing their own meetings and any telecommunications required). Materials and exercises will be designed in accessible web formats suitable for mobile devices. Suwantarathip and Orawiwatnakul (2015) report success delivering small exercises to students using mobile devices.
Feedback for low level tasks will be provided by small automated weekly quizzes, to assist students with terminology. Van der Kleij, Feskens and Eggen (2015) note the importance of feedback in a computer based course. Students will also receive weekly feedback and a mark from the instructor, but this will be of necessity a brief few sentences. The mark will be based on an average of peer assessment collected by the LMS and then vetted by the instructor.

9. Example Content

What is Innovation?: Unit 1 (What is Innovation?) provides some terminology and concepts, before the students start to think about what project they would like to work on. The primary reading is Moore (2012). Students are also introduced to the on-line discussion forums at this point and do an icebreaker exercise to become antiquated so they can then form teams in the next unit to work on a project. This fits in the course's aim of bridging theory and practice, individual student study and group project work.

Conclusion

This document provides the first two steps in instructional design (ID) for a new course "Innovation, Commercialisation and Entrepreneurship in Technology" to be offered on-line, initially for students in Canberra, Australia. The aim is to provide a formal masters level course to complement competitions, such as "Innovation ACT", catering for students who have ambitions of becoming entrepreneurs. A needs assessment, after Smith and Ragan, (2005, pp. 44) was provided and the limitations of existing courses discussed. An on-line course is discussed (noting the limitations in the fixed term based course format). A brief description of one unit is provided. This will be fowled in a separate document with creation of one learning objects for the course.

References

ACS. (2013). New Technology Alignment. Retrieved from https://www.acs.org.au/__data/assets/pdf_file/0006/8889/Subject-outline_New-Technology-Alignment.pdf
ACS. (2014). Application Guidelines: Professional Level Courses. Retrieved from https://www.acs.org.au/__data/assets/pdf_file/0010/24499/ACS-Accreditation-Document-2-Application-Guidelines-28-March-2014V0-1.pdf
ACT Government. (2013). Growth, Diversification and Jobs - A Business Development Strategy for the ACT. Retrieved from http://www.business.act.gov.au/resources_and_networks/business_development_strategy/business_development_strategy#innovinit
Al-Atabi, M., & DeBoer, J. (2014). Teaching entrepreneurship using Massive Open Online Course (MOOC). Technovation, 34(4), 261-264.
Australian Department of Industry. (2013). About the National Register on Vocational Education and Training (VET) in Australia. Retrieved from https://training.gov.au/Home/About
Australian National University. (n.d. a). Engineering Innovation ENGN3230. Retrieved from http://programsandcourses.anu.edu.au/course/ENGN3230
Australian National University. (n.d. b). Entrepreneurship and Innovation MGMT3027. Retrieved from http://programsandcourses.anu.edu.au/course/MGMT3027
Australian National University. (n.d. c). Innovation and Commercialisation MGMT7165. Retrieved from http://programsandcourses.anu.edu.au/course/MGMT7165
Australian National University. (n.d. d). Technology and Innovation Management and Strategy MGMT7106. Retrieved from http://programsandcourses.anu.edu.au/course/MGMT7106
Blackhall, L. (n.d.) LinkedIn [Profile page]. Retrieved January 22, 2015 from http://au.linkedin.com/in/lachlanblackhall
InnovationACT. (2014a). Innovation ACT: History. Retrieved January 25, 2015, from http://web.archive.org/web/20131030170036/http://www.innovationact.org/about/history/
InnovationACT. (2014b). Innovation ACT: Resources. Retrieved January 25, 2015, from http://web.archive.org/web/20131030170036/http://www.innovationact.org/resources/

Blair, K., & Hoy, C. (2006). Paying attention to adult learners online: The pedagogy and politics of community. Computers and Composition, 23(1), 32-48. Retrieved from http://www.sciencedirect.com/science/article/pii/S8755461505000824
Gagné, R. (1965). The Conditions of Learning, New York: Holt, Rinehart & Winston, 1965. Cited in Kakouris, A. (2009). Online platforms for entrepreneurship education: an instructional design approach. In 4th European Conference on Entrepreneurship and Innovation, Academic Conferences Limited., Reading, UK, September (p. 233).
Hung, W. (2011). Theory to reality: a few issues in implementing problem-based learning. Educational Technology Research and Development, 59(4), 529-552. Retrieved from http://link.springer.com/content/pdf/10.1007%2Fs11423-011-9198-1.pdf
IP3. (2015). IP3 Professional IT Standards. Retrieved from http://ipthree.org/gain-ip3-accreditation/ip3-accreditation-program/it-professional-standards/
Kakouris, A. (2009). Online platforms for entrepreneurship education: an instructional design approach. In 4th European Conference on Entrepreneurship and Innovation, Academic Conferences Limited., Reading, UK, September (pp. 230-235).

Kruchten, P., Lawrence, P., Dahl, D., & Cubbon, P. (2011). New Venture Design–Interdisciplinary Capstone Projects at UBC. Proceedings of the Canadian Engineering Education Association. Retrieved from http://library.queensu.ca/ojs/index.php/PCEEA/article/download/3637/3650
Mazzarol, T. (2014). How do Australia's universities engage with entrepreneurship and small business?. Centre for Entrepreneurial Management and Innovation (CEMI) Discussion Paper, (1401). Retrieved from http://dx.doi.org/10.2139/ssrn.2428008
McEwan, T. (2013, October). Commercial competency and computing students: Using the skills framework for the information age in higher education. In Frontiers in Education Conference, 2013 IEEE (pp. 286-292). IEEE. Retrieved from http://ieeexplore.ieee.org/xpls/icp.jsp?arnumber=6684833
Moore, A. (2012). Correia, A. P. (2012). Defining Innovation. In A. P. Correia (Ed.), Breaking the Mold: An Educational Perspective on Diffusion of Innovation. Retrieved from http://en.wikibooks.org/wiki/Breaking_the_Mold:_An_Educational_Perspective_on_Diffusion_of_Innovation/Defining_Innovation
National VET E-learning Strategy. (2013). National VET Content. Retrieved from https://nationalvetcontent.edu.au/
Neck, H. M., Greene, P. G., & Brush, C. G. (Eds.). (2014). Teaching entrepreneurship: A practice-based approach. Edward Elgar Publishing (p. 65).
Prasad, T. (2014). Developing Enterprise Culture among the Students through Intercollegiate Competitions: A Case of Student Enterprise Competition (SEC) 2007. Developments in Business Simulation and Experiential Learning, 35. Retrieved from https://absel-ojs-ttu.tdl.org/absel/index.php/absel/article/download/399/365

Reimers-Hild, C., & King, J. W. (2009). Six questions for entrepreneurial leadership and innovation in distance education. Online journal of distance learning administration, 12(4). Retrieved from http://www.westga.edu/~distance/ojdla/winter124/reimers-hild124.html
Segal Quince & Partners (1985). The Cambridge phenomenon : the growth of high technology industry in a university town. Segal Quince & Partners, Cambridge
SFIA Foundation Ltd, (2015). The purpose of SFIA. [online] Skills Framework for the Information Age. Retrieved from https://www.sfia-online.org/v501/en/index.html
Smith, J., Brown, L., Blackhall, L., Loden, D., & O’Shea, J. (2010, September). New Partnerships Linking Universities and NGO's on Education for Development Engineering: Case Study from Engineers Without Borders Australia'. In Joint International IGIP-SEFI Annual Conference (Trnava, Slovakia). Retrieved from http://www.sefi.be/wp-content/papers2010/papers/1248.pdf
Smith, P. L. & Ragan, T. J. (2005). Instructional design (3rd ed.). Hoboken, NJ: Wiley Jossey-Bass Education.
Suwantarathip, O., & Orawiwatnakul, W. (2015). Using Mobile-Assisted Exercises to Support Students' Vocabulary Skill Development. TOJET, 14(1). http://www.tojet.net/articles/v14i1/14118.pdf
Thompson, P., & Kwong, C. (2013) Compulsory School Based Enterprise Education as a Gateway to an Entrepreneurial Career. Retrieved from http://www.isbe.org.uk/content/assets/Enterprise_Education-_Caleb_Kwong.pdf
University of Canberra. (2012a). Managing change and innovation 7776.3. Retrieved from http://www.canberra.edu.au/coursesandunits/unit?unit_cd=7776&version_number=3&rownum=3165
University of Canberra. (2012b). Bachelor of Entrepreneurship and Innovation 208JA.1. Retrieved from http://www.canberra.edu.au/coursesandunits/course?course_cd=208JA&version_number=1&location_cd=BRUCE
Van der Kleij, F. M., Feskens, R. C., & Eggen, T. J. (2015). Effects of Feedback in a Computer-Based Learning Environment on Students’ Learning Outcomes A Meta-Analysis. Review of Educational Research, 0034654314564881. Retrieved from http://rer.sagepub.com/content/early/2015/01/08/0034654314564881.full.pdf

Technical Standards for Learning Objects

The issue of Instructional Design technical standards,  compliance, SCORM, metadata tests and W3C HTML validation came up in the course I am doing. This is an area best avoided, unless someone is paying you to do it, as it is frustrating, time consuming and never ending. The Australian VET E-standards is a reasonable overview.

One general point: Most learning objects are a ZIP file which contains folders, the folders have web format  files for the content in them (HTML, CSS, JPEG, PNG). There are also some XML files with metadata (cataloging information) and quiz questions. E-books use a similar format.

The standards issue comes up with which version of file formats are used (HTML 4, XHTML, HTML 5 ...), what the folder structure is and what metadata is included. Vendors of products make claims as to what they support and educational institutions get stuck with particular products.

What I do is try to avoid using anything more than basic web formats for my educational content, so that this will more easily convert from one system to another. As an example, I use default formatting for headings and text. I do not specify the font, color, or size of text, so it will appear using the default of the system it is imported into. The result can look dull, but at least the student is likely to be able to read it and I don't have to spend hours fixing the formatting.

Just to show off, if you look through "IEEE Standard for Learning Object Metadata" (1484.12.1-2002) you will see my name it (I was on the balloting group, contributed one comment and then voted "yes"). ;-)

On-line courses can support social interaction

Professor Jason Potts from RMIT, writes "Why MOOCs will fail – they're not dating sites" (Online Opinion, 11 February 2015).

Fixed: a team of students I mentored last year has built "Bringpeer", a social network just for university students, just for university students. It has been released first for Australian National University students in Canberra. ;-)

On-line courses can support social interaction

More seriously, as a student, designer and teacher of on-line university courses, one of the things I have noticed is the isolation the students can feel. On-line tutors are trained to counter this by getting the students to first introduce themselves. This is not compulsory, but each student is provided with a bio page, where they can put their photo and a little about themselves. Many advanced on-line courses also require students to form teams to do project work (I am meeting on-line tomorrow with a team from Kenya, Norway, Japan and Canada). Some institutions also provide a way for students to meet up face-to-face to study (but this requires some safety protocols).

Many MOOCS do not cover these social aspects well, not because the technology can't support it, but because the courses have not been well designed by competent instructional designers.

The case for MOOCs

The case for MOOCs is not, as Professor Potts suggests "they can deliver the same educational services, but at a fraction of the cost". Distance education (DE) courses, of one sort or another, have existed for more than one hundred years. Replacing paper with broadcasts, video tapes or the Internet has not made much difference. Such DE courses don't provide the same education and they are not necessarily that much cheaper. These courses provide access to a different form of education (in some ways better) for people who would otherwise not have access.

MOOCs are not cheap or easy

Professor Potts seems to think that MOOCs are cheap and easy. However, very few of the people who sign up for a free course ever finish it. Doing one MOOC will not get you the equivalent of a university degree. Some of the MOOC providers offer a "professional" program, but if you add up the cost of all the courses required, it is not necessarily cheaper than a conventional higher education program (especially as MOOCs have a lower completion rate).

Be it on-line or face to face, what I have learned as a student of higher education is that it is very hard work being a student.

But are MOOCs more effective ways of dating?

Given that many people now find their partners via an on-line service, there is no reason to believe that on-line eduction programs which have a social component can't be equally effective as face-to-face campuses.

On-line courses are going to challenge their business model

The real story is that on-line tools are already being used to supplement most university programs in Australia. It would be difficult to find a university course which does not provide some services to the students on-line. Many are now "blended" with the on-line component being an essential part of the course.

Very few courses and programs are offered entirely on-line. However, this will change in the next few years, so that the norm will be on-line courses, supplemented with classes. This will be a challenge for Australian universities, as they will then have to compete with those from around the world. Torrens University Australia is part of Laureate International Universities, which already provides education on-line to 800,000 students.

Even today there are enough on-line programs for a student who wants this option. When selecting a postgraduate program in higher education, I was able to select from several purely on-line programs offered in Australia (although I eventually decided to enroll in North America).

The argument about the viability of on-line courses seems a little outdated, like discussing the benefits of email was fifteen years ago. There were some people who "got it" and the rest had a few years to catch up. I started delivering on-line university courses in 2009 and, apart from the occasional guest lecture, have not been back in front of a class since then. The idea that myself and my students would have to arrange to assemble in one geographic location somewhere in the world, just so that I could talk at them for an hour now sounds weird (and research shows it is not a very effective teaching technique).

Innovations in teaching innovation at CSIRO

I will be speaking on "Innovations in teaching innovation", at the CSIRO ICT Centre, Center, Australian National University in Canberra, 4pm, 27 April 2015 (draft presentation available).

Innovations in teaching innovation

Tom Worthingon (ANU Research School of Computer Science and ACS Virtual College)

CSIRO ICT IR and friends

DATE: 2015-04-27
TIME: 16:00:00 - 17:00:00
LOCATION: CSIRO seminar room, S206, ANU Computer Science and Information Technology Building, North Road, Acton, ACT
CONTACT: paul.thomas@csiro.au

ABSTRACT:
Tom takes over teaching the Australian Computer Society's "New Technology Alignment" (NTA) postgraduate course from January 2015. This offered on-line directly by the ACS Virtual College and through Open Universities Australia. In late 2014 he attended a conference in Canada on computer science education and met with Canadian academics to discuss flexible learning and teaching innovation. Tom discusses how this might be done in Australia, by blending on-line formal courses with face-to-face competitions.

Web Based eBook Interface for Young Students?

Another "learning object" I have had to review as as part of a course on instructional design, is an e-book for young readers: "Sk8 for Jake". The content of the book seemed suitable for the intended audience, but the web interface has some problems. Perhaps this would be better as an ePub e-book.

Some routine tests run on the interface:

1. W3C Markup Validation Service: 32 Errors, 12 warnings.
2.  W3C mobileOK Checker: Score of 0 out of 5 (lowest possible score). Failures per severity: critical 4, severe 2, medium 2, low 5.
3. Achecker Accessibility Review for WCAG 2.0 (Level AA) Guidelines: Known Problems (22), Likely Problems (0), Potential Problems (187).

These are some easily fixed problems with the interface. However the major difficulty is that the content of the book appears to be presented as one large image per page. There is no text which could be converted to Braille, or synthetic speech. This makes the entire book inaccessible.
There is an audio option, which has very clear narration and has the option of setting the speed. But I had difficulty operating the audio interface and it does not appear to exactly match the text in the book.

NZ Moodle video

As part of a course on instructional design, I had to review a video from HRDNZ on "Moodle Quiz Questions". I loved the New Zealand accent, but the automated Close Captions system had difficulty understanding what they were saying. For example "False" comes out as "Fells". ;-)

More seriously the video and sound were very clear and well structured. But it all whizzed past a bit quickly for me to keep up with. The background music is so loud that it may create difficulties for those with a hearing impairment. Also it was a little lacking in context, in that I was not sure what sort of students this was for, if it was part of a course and if there is a workbook somewhere. That might be on the title screen but it whizzed past so quickly I could not read it.

Obviously as this is a video it will not be suitable for those who can't see (or do not have enough bandwidth to get the video).

One problem I have with such instructional videos is that they tell you how to do something (in this case create a quiz) but not why. In Australian universities, the use of multiple choice tests is treated with suspicion, being seen as something which can only test very basic skills and which the students can easy cheat at.  Before this how video, there would need to be one on why it is a good idea.

Review of a Vocational Learning Object

Currently I am a student of Instructional Design, learning to use the "Learning Object Review Instrument" (LORI). The Edutech Wiki entry for LORI had a simple explanation, listing the nine criteria, rated on a scale of one to five: Content Quality, Learning Goal Alignment, Feedback and Adaptation, Motivation, Presentation Design, Interaction Usability, Accessibility, Reusability, and Standards Compliance. A more scholarly source is Akhavan & Arefi (2014). They traced the development of LORI and then consulted experts on e-learning quality criteria using the Delphi technique, but do not appear to have significantly improved on LORI.

LORI seems to be from elera.net but I was unable to get that website to work (the Internet Archive had the latest capture at 15 December 2011). The recommended method to have a group rate an object is with the "convergent participation model" (Nesbit and Belfer, 2004). Evaluators first assess the object independently, they then compare, discuss and adjust their assessments (which can be supported with online tools). While Nesbit and Belfer (2004) do not mention it, the implication is that the goal is consensus by the group.

In such activities in the past I have found it useful to introduce an intermediary step where the individual scores are combined in some way (such as by averaging) to produce a proposed group position. Most often the group is happy with this result eliminating the need for a discussion, saving time and the risk of "group think", where a suboptimal result is selected for the sake of consensus.

The LORI manual is only 12 pages long (Nesbit, Belfer & Leacock, 2003). The manual shows a star rating system for each of the nine criteria, a paragraph describing a what a five star rating would be, a paragraph describing the (lowest) one star rating and an example paragraph. There is a sample scoring sheet.

Here is what I came up with for my LORI example:

Learning Object: "Undertake business planning" (Australian National Training Authority, 2014)

Reviewer: Tom Worthington

General Remarks: The module provides Topics for a simple why what and how of a business plan. There are multiple choice Self tests, which provide feedback. The Activities provide simple scenarios for the student to discuss. A detailed Example business is provided.

 Scores are: 1 Low to 5 High

1. Content Quality: Veracity, accuracy, balanced presentation of ideas, and appropriate level of detail: 4 [Minimal theory is provided, as suits a Certificate IV in the Australian Vocational Education and Training (VET) System]

2. Learning Goal Alignment: Alignment among learning goals, activities, assessments, and learner characteristics: 4 [The object is for the VET Training Package "Business Services", level Certificate IV in Small Business Management (BSB40407), competency "Undertake small business planning" (BSBSBM404A). The goals, activities, assessment are aligned with the needs of someone setting up a business for the first time.]

3. Feedback and Adaptation: Adaptive content or feedback driven by differential learner input or learner modeling: 2 [The only adaptive content are hints in response to incorrect questions in the multiple choice quizzes]

4. Motivation: Ability to motivate and interest an identified population of learners: 3 [The case studies used are likely to be relevant to a student seeking to set up a small business or gain employment in one. However, there is no system of "badges" or other learner rewards to keep them studying, just the quiz results.]

5. Presentation Design: Design of visual and auditory information for enhanced learning and efficient mental processing: 5 [Clear text and simple diagrams are used, as suits the web based IMS Content Package format.]

6. Interaction Usability: Ease of navigation, predictability of the user interface, and quality of the interface help features: 4 [The web based IMS Content Package format is used, with its standard interface. This has some limitations depending on the Learning Management System used to host the package (such as Moodle).]

7. Accessibility: Design of controls and presentation formats to accommodate disabled and mobile learners: 2 [The web based IMS Content Package format allows for disabled and mobile users, however the standard Moodle interface does not accommodate Mobile devices well. The use of Flash for the quizzes limits the accessibility of the system).]

8. Reusability: Ability to use in varying learning contexts and with learners from differing backgrounds: 2 [The content is designed for Australian, English speaking students who have completed school. This would not suit those for other backgrounds and different education levels (in particular may not be suitable for university degree program use). But that is not the intended purpose of the object.]

9. Standards Compliance: Adherence to international standards and specifications: 4 [The module is implemented as an IMS Content Package (and claims SCORM 1.2), using web standard formats for text (HTML 4.01 Transitional), scripting (CSS and Javascript) and images (GIF and JPEG), thus conforming to standards. PDF is used for some documents, however proprietary RTF and XLF formats are used for others (not Open Document Format 1.2, or simply HTML). Also the text appears compatible with web accessibility standards (W3C WCAG  2.0). The content, apart from the quizzes appear broadly compatible with mobile devices (W3C mobileOK Scheme 1.0), despite problems with Moodle's mobile compatibility. However, the major problem in terms of standards (and accessibility) is the use of  Shockwave Flash for quizzes. ]

References

Akhavan, P., & Arefi, M. F. (2014). Developing a Conceptual Framework for Evaluation of E-Content of Virtual Courses: E-Learning Center of an Iranian University Case Study.

Arefi (2014), Developing a Conceptual Framework for Evaluation of E-Content of Virtual Courses: E-Learning Center of an Iranian University Case Study, Interdisciplinary Journal of E-Learning and Learning Objects, 10(1), 53-73. Retrieved from: http://poseidon01.ssrn.com/delivery.php?ID=809117069101031088086104127087029074000085037059021024124065101030009029090088008110060097037059060026020097006127017116122102025086030014047009024118073002002120004005033009093106120067119029115088127103&EXT=pdf&TYPE=1

Australian National Training Authority. (2014). Undertake business planning. Retrieved from https://nationalvetcontent.edu.au/share/page/document-details?nodeRef=workspace://SpacesStore/3ae72d24-e122-4b43-af5e-d2ae15d08740

Nesbit, J. C., & Belfer, K. (2004). Collaborative evaluation of learning objects. Online education using learning objects, 138-153. Retrieved from: https://web.archive.org/web/20080128191654/http://www.elera.net/eLera/Home/Articles/ColEvaluation.htm

Nesbit, J., Belfer, K., & Leacock, T. (2003). Learning object review instrument (LORI) manual 1.5. Retrieved from: https://web.archive.org/web/20111113192806/http://www.elera.net/eLera/Home/Articles/LORI%201.5.pdf