Why Languages Suck

Greetings from the famous room N101 at the Australian National University, where Professor Steve Blackburn of the Research School of Computer Science is speaking on "Why languages suck (and what we might do about it)". He started with the example of the "Patriot Missile Software Problem", which killed 28 U.S. soldiers 1991. A second example was "Hospitals' medication software likely to kill patients within month: Queensland Health report". This should be of concern to ACT residents as the same Metavision Intensive Care software is used at Calvary Hospital Canberra.

Professor Blackburn then gave apparently simple examples of code in PHP which give unexpected results. Steve went on to discuss the cost of inefficient computer programming languages which use more processing power and result in increased carbon emissions and global warming. He claimed that problems with languages occur because they involve difficult concepts and are difficult to engineer. Decisions made in language design get "baked in" to the system. This did not seem to me to be a new insight, when designing any new system trade-offs need to be made and have to be lived with.

An example is the Contact Wire System for the Gotthard Base Tunnel. At the time the tunnel was designed an over-head wire had to be used to supply power to the trains. Today an alternative technology is available which would allow the tunnel to be smaller diameter, saving many millions of dollars. But at the time the tunnel was designed the new technology was not proven and so could not be used.

ABSTRACT:
This talk looks at problems faced by many of today's languages, how those problems arose, what we could do to address those problems and what we could do to avoid history repeating itself. The basic question is why is it that languages like JavaScript, PHP, and Python are so important and yet have such conspicuous shortcomings, both in their semantics and their performance? The talk is part sociological (how do we as a community end up making such mistakes?) part big picture (what are the technological solutions to such problems and why is it hard?). The talk will include a brief description of the micro vm weare designing and building, (microvm.org) and some recent research that tackles the major barrier to high performance garbage collection in many managed language implementations.

Commercialisation and Entrepreneurship in Technology Course Proposal

For some years I have been considering how to offer the students undertaking the "Innovation ACT" competition credit as part of a university degree. Inspired by my visit to UBCin Vancouver, I started work on such a course. Rather than approach universities and professional bodies in the usual way, I thought I would release the idea and see who is interested. The concept is reasonably simple: students would enroll in an on-line course and study the theory of innovation. They would then take part in an innovation competition, such as Innovation ACT, to experience the practice. Half of the students assessment would be for theory and half for the practical materials they prepare as part of the competition. To pilot this I would need six students to make a minimum student cohort (this could be two teams of three students, for example). Anyone interest, please let me know:

Proposal for a Course: Commercialisation and Entrepreneurship in Technology

Tom Worthington 23 October 2014
This is a preliminary outline of a Technology Innovation course. Students would learn how to take an idea and turn it into a business proposal. They would learn theory online and then undertake practical work, using the format successfully applied in my course ICT Sustainability (Worthington, 2011). Students would have the option of undertaking their practical work as part of an innovation competition, such as Innovation ACT.

Background

In Canberra, University of Canberra and ANU already run a number of innovation courses. However, none of these are designed to be run in conjunction with innovation competitions. Also current courses expect attendance at classes in person, and tend to emphasize use of lectures and examinations, which do not suit the teaching of innovation.

Current Canberra Based Innovation Courses

University of Canberra Courses

University of Canberra (UoC) and ANU both offer innovation courses. UoC have courses as part of the Bachelor of Entrepreneurship and Innovation. A typical unit is “Managing Change and Innovation” (7776.3), offered in blended mode (on-line content with on campus attendance of up to thirty nine hours).

Learning Outcomes

1. understand critically a range of theories and practices of change management.

2. demonstrate the applicability of organizational change practices in different circumstances.

3. appreciate the complexities and challenges inherent in planning and managing organizational change.

4. be familiar with the nature of innovation and how to implement it

5. research, identify, organize and present relevant materials and arguments in a range of modes.

There is a detailed fifteen page unit outline of the course provided for the Bhutan campus students. This includes:

Text:

Graetz, F., Rimmer, M., Smith, A. and Lawrence, A. 2010. Managing Organisational Change, Australasian edition. Milton, Qld: John Wiley & Sons.

Recommended:

Goffin, K. and Mitchell, R. 2010 Innovation and Management: Strategy and Implementation Using the Pentathlon Framework, 2^nd edition. Houndmills: Palgrave Macmillan. Chapter 4. 

There is also an extensive list of further reading:

Beckhard, R. and Harris, R. 1987. Organizational Transitions: Managing Complex Change. Reading, Mass.: Addison-Wesley.

Beer, M. and Eisenstat, R. 2000. Breaking the Code of Change. Boston, Mass.: Harvard Business School Press.

Beitler, M. 2006. Strategic Organizational Change, 2^nd edition.

Bolman, L.G. and Deal, T.E. 1997. Reframing Organizations: Artistry, Choice and Leadership. San Francisco: Jossey-Bass.

Boonstra, J. ed. 2002. The Psychological Management of Organizational Change. London: Wiley.

Bridges, W. 1991. Managing Transitions: Making the Most of Change. Reading, Mass.: Addison-Wesley.

Bunker, B. and Alban, B. 1996. Large Group Interventions: Engaging the Whole System for Rapid Change. San Francisco: Jossey-Bass.

Burns, T. and Stalker, G. 1961. The Management of Innovation. London: Tavistock.

Burnes, B. 2000. Managing Change: A Strategic Approach to Organisational Dynamics. Harlow: Pearson.

Cameron, E. and green, M. 2009. Making Sense of Change Management. London: Kogan Page.

Carnall. C. 2003. Managing Change in Organizations, 4^th edition. Harlow: FT Prentice Hall.

Cummings, T. and Worley, C. 2001. Organizational Development and Change, 7^th edition. Cincinnati, Ohio: West.

Clark, J. 1995. Managing Innovation and Change. London: Sage.

Demers, C. 2007. Organizational Change Theories: a Synthesis. Los Angeles: Sage Publications.

Dunphy, D., Griffiths, A. and Benn, S. 2007. Organizational Change for Corporate Sustainability. London: Routledge.

Hayes, J. 2007. The Theory and Practice of Change Management, 2^nd edition. Houndmills: Palgrave.

Jones, G. 2006. Organizational Theory, Design and Change, 5^th edition. Englewood Cliffs: Prentice Hall.

Kanter, R. (1983) The Change Masters. London: George Allen and Unwin.

Kantor, R., Stein, B. aand Jick, T. 1992. The Challenge of Organizational Change. New York: Free Press.

Kaplan, P. and Norton, D.P. 1996. The Balanced Scorecard. Boston, Mass.: Harvard Business School Press.

Klewes, J. and Langen, R.eds. 2010. Change 2.0: Beyond Organisational Transformation. Heidelberg: Springer.

Kotter, J.P. Leading Change. Boston, Mass.: Harvard Business School Press.

Kotter, J.P. 1990. A Force for Change: How Leadership Differs from Management. New York: Free Press.

Kouzes, J.M. and Posner, B.Z. 1995. The Leadership Challenge. San Francisco: Jossey-Bass.

Mabey, C. and Mayon-White, B. eds. 1993. Managing Change. London: Paul Chapman Publishing.

Mills, J., Dye, K. and Mills, A. 2009. Understanding Organizational Change. London: Routledge.

Nadler, D., Shaw, R. and Walton, A. eds. 1995. Discontinuous Change: Leading Organizational Transformation. San Francisco: Jossey-Bass.

Nanus, B. 1992. Visionary Leadership: Creating a Compelling Sense of Direction for Your Organization. San Francisco: Jossey-Bass.

Palmer, I., Dunford, R. and Akin, G. 2006. Managing Organizational Change: A Multiple Perspective Approach. Boston: McGraw-Hill Irwin.

Senior, B. 2002. Organizational Change, 2^nd edition. London: Financial Times and Prentice Hall.

Senge, P.M. 1990. The Fifth Discipline: The Art and Practice of the Learning Organization. New York: Doubleday/Currency.

Smale, G. 1998. Managing Change Through Innovation. London: The Stationery Office.

Tichy, N. and Devanna, M. 1986. The Transformational Leader. Chichester: Wiley.

Weick, K. and Sutcliffe, K. 2001. Managing the Unexpected: Assuring High Performance in and Age of Complexity. San Francisco: Jossey-Bass.

Whiteley, A. 1995. Managing Change: A Core Values Approach. South Melbourne: Macmillan Education Australia.

Wilson, D. 1992. A Strategy for Change. London: Routledge.

 

Assessment

1. Case study 30%
2. Essay 40%
3. Examination 30%

ANU Innovation Courses

ANU has several "innovation" courses in business and engineering programs:

Entrepreneurship and Innovation MGMT3027

Learning Outcomes

• define, explain and illustrate theories of business innovation and entrepreneurship, the evolution of industries and economies, and the roles of entrepreneurs;
• develop a comprehensive and well structured business plan for a new venture;
• present a persuasive business plan to potential investors or to internal stakeholders and effectively answer probing questions on the substance of the plan; and,
• work effectively in multidisciplinary, cross-cultural teams, communicating, negotiating and contributing shared contributions towards the development of a team project.

Assessment

• New venture business plan 30%
• Business plan presentation 10%
• Case study responses 20%
• Final examination 40%

Workload

• 3 contact hours per week plus private study time. 

There is a detailed eighteen page "Course Outline" for  MGMT3027. This outlines the study methods and texts.

Text:

Allen, K. R. (2011). Launching new ventures: an entrepreneurial approach. Cengage Learning.

Also recommended:

Osterwalder, A., & Pigneur, Y. (2010). Business Model Generation: A Handbook For Visionaries, Game Changers, And Challengers Author: Alexander Osterwalder, Yves.

Weeks of the course:

1. Introduction and expectations, Entrepreneurs and opportunities, Economy-wide context
2. Teams, shareholders, the nature of opportunities and the process of business planning
3. Product and process innovation, Researching markets and customers
4. Business model innovation, Designing operations, process flow, capturing and building on customer and process learning
5. Resource planning and budgeting, Alliances, partnerships, networks and organisation design, Legal considerations
6. Marketing plan, distribution, start-up and financing
7. Growth, change, harvest and exit
8. Trial pitch session
9. Corporate entrepreneurship, open innovation and applications beyond entrepreneurship
10. Review of the six integrating themes
11. Review of the six integrating themes
12. Team business plan pitches
13. Course recap, review, revision, final Q&A

Six Integrative Themes

1. Opportunities, drivers and processes of entrepreneurship and innovation
2. Entrepreneurship and innovation at an economy-wide level
3. Entrepreneurial and innovation processes at the level of the firm
4. Business models, business model innovation and entrepreneurship
5. Entrepreneurial teams and stakeholder interactions
6. Resource acquisition, leverage, valuation and negotiation

Innovation and Commercialisation MGMT7165

The Preliminary Briefing Note (2014) provided ten pages of details:

Learning Outcomes

• Describe the process involved in different types of innovation, and the role that commercialization plays in this process in diverse organizational contexts.
• Analyze an innovation project, identifying drivers for success and factors leading to the risk of failure.
• Diagnose and select frameworks, tools and techniques for the management of innovation projects in different types of organization.
• Develop an innovation strategy for an organization., including input from a range of internal and external stakeholders. 

Assessment:

1. Innovation Labs 25% (5% x 5 labs) 
2. Team Presentation 20%
3. Case Study Analysis 15%
4. Write-Up of Discussion Insights (15%)
   Individual Report 25%

Engineering Innovation ENGN3230

Learning Outcomes

• Identify the need for innovative engineering; and generate and evaluate innovative concepts
• Understand the basic elements of innovation, innovation management and commercialization; and be able to plan and schedule activities in accordance with standard practice.
• Be conversant with Intellectual Property (IP) law; and evaluate, exploit and manage Intellectual Property.
• Understand decision making responsibilities at the interface between business and innovation.
• Understand the dynamics of collaborative teams and how to work effectively within a team to accomplish tasks within given deadlines.
• As an entrepreneur, understand start-up company pathways and develop a business pitch for funding

Assessment

• Quizzes (10%);
• Tutorials and case studies (20%);
• Technical reports and presentation (40%);
• Exam (30%).

Workload

• Lectures (30hrs);
• Tutorials (10hrs);
• Tutorial case studies (20hrs);
• Assessment activity and self-directed learning (70hrs).

Also the Lecture 1 slides are available.

Technology and Innovation Management and Strategy MGMT7106

1. Stimulate and inform a strategic perspective on the role of innovation, and in particular to increase understanding of:

• The characteristics of innovation processes and the factors that shape and drive innovation;
• The potential roles of incremental and disruptive innovation in creating and sustaining firm competitiveness

1. Understand the sources of innovation competence in firms and how these competences are developed, and in particular to increase understanding of:

• Why some firms are more successfully innovative than others;
• The many different sources of knowledge and capability used for innovation and the strategies for accessing them.

1. Understand the major tools that are used increasingly to assist innovation management, both at the project level and at the level of organizational development.
2. Understand the central role of learning in innovation and in innovation management.

Assessment

1. Case Note 1: Presentation of Innovation Survey 10%
2. Case Note 1: 3M or Medronic 10%
3. Case Note 2: Lenovo or Hyundi 10%
4. Case Note 3: Innovation Audit 10%
5. Peer Assessment 10%
6. Essay/Assignment/Exam 50%

Workload

• 5 x 4 hour classes,
• 2x 8 hour classes

Other University Innovation Courses

UBC run a two semester interdisciplinary capstone project (APSC 496) for business and engineering students (Kruchten, Lawrence, Dahl & Cubbon, 2011). The students are required to work in teams to produce a business plan. This course is much more extensive than envisaged for ANU CS. However, the textbooks used may be applicable:

1. K. T. Ulrich and S. D. Eppinger, Product Design and Development, 4th ed., McGraw Hill, Boston, 2008.
2. J. A. Timmons and S. Spinelli, New Venture Creation--Entrepreneurship for the 21st Century, 7th ed., McGraw-Hill Irwin, Boston, 2007.
3. S. Birley and D. Muzyka, Mastering Entrepreneurship, Pearson Education, Harlow, UK, 2000.

There is also a related book, which may be of use:

• New venture creation : entrepreneurship for the 21st century : a Pacific Rim perspective / Jeffry A. Timmons, McGraw-Hill Education, 2010 

Open Access Materials

As might be expected, textbooks about innovation can be expensive. I found one free open access book:

• Correia, A.-P. (2012) (Ed.). Breaking the Mold: An Educational Perspective on Diffusion of Innovation. Wikibook/textbook, available at: http://en.wikibooks.org/wiki/Breaking_the_Mold:_An_Educational_Perspective_on_Diffusion_of_Innovation

 

Problems with Current Innovation Courses

Innovation courses offered in Canberra have largely the format of a conventional lecture and examination based university course, which is not suited to the topic of innovation. It is proposed to overcome this by flipped classroom: students learn the theory online, discuss it with their peers in online forms and then are required to apply it.
Also courses are not aligned with the requirements for accreditation by bodies such as the Australian Computer Society. To meet accreditation requirements, the new course would be aligned with SFIA Version 5:

SFIA Skill "Innovation" INOV Level 6:

The capability to recognise and exploit business opportunities provided by IT, (for example, the Internet), to ensure more efficient and effective performance of organisations, to explore possibilities for new ways of conducting business and organisational processes, and to establish new businesses.

Recognises potential strategic application of IT, and initiates investigation and development of innovative methods of exploiting IT assets, to the benefit of organisations and the community. Plays an active role in improving the interface between the business and IT.

SFIA Skill “Business analysis” BUAN Level 6

The methodical investigation, analysis, review and documentation of all or part of a business in terms of business functions and processes, the information used and the data on which the information is based. The definition of requirements for improving processes and systems, reducing their costs, enhancing their sustainability, and the quantification of potential business benefits. The creation of viable specifications and acceptance criteria in preparation for the construction of information and communication systems.

Takes full responsibility for business analysis within a significant segment of an organisation where the advice given and decisions made will have a measurable impact on the profitability or effectiveness of the organisation. Establishes the contribution that technology can make to business objectives, defining strategies, validating and justifying business needs, conducting feasibility studies, producing high-level and detailed business models, preparing business cases, overseeing development and implementation of solutions, taking into account the implications of change on the organisation and all stakeholders. Guides senior management towards accepting change brought about through process and organisational change.

What would be in an online innovation course?

The new course will develop the capability to identify and develop new computing based business ideas. Students will learn to identity strategic uses for information technology, applying systematic investigation, analysis, review and documentation to take an idea through the stages of development and proposal. Students are encouraged to take part in the Innovation ACT, or a similar innovation competition, and submit their competition entry for assessment.
Learning Outcomes
After successful completion of this subject students will be able to :

1. Investigate a strategic application of IT.
2. Propose new ways of conducting business using IT.

Skills Alignment:

1. SFIA Version 5, Level 6: Business analysis (BUAN)
2. SFIA Version 5, Level 6: Innovation (INOV)

Workload

• 12 Weekly Online Modules.
• Tutorials: 12 Weekly online text based discussion forums (assesble item at the end of each week).  

Timing

Second half of the year for alignment with Innovation ACT (the course could be offered to fit with other university systems and innovation completions world wide). Timetable to suit ANU Semester 2, 2015

Week	Date	Course topic	Assessment %	Innovation ACT
1	20 July 2014		2%
2	27		2%
3	3 August		2%
4	10		2%	Launch & Pitch
5	17		Business Model Thinking 2%	Team Registration Opens Workshop 1: Learning to Build an Innovative Venture - Business Model Thinking: Using a Business Model Canvas, for development of a venture concept.
6	24		Stakeholder Engagement 2%	Workshop 2: Relationships with Users and Partners - Stakeholder Engagement: engage with a stakeholders, to develop value network. Team Registration Closes
7	31		Concept Generation 2%	Workshop 3: Developing Prototype Solutions: Test venture to deal with uncertainty.
	Mid semester break		Investigation of a strategic application of IT 40%
8	21 September		Value Capture 2%	Workshop 4: Getting Returns, Support, and Funding_ Value Capture: find ways to capture value for the team, investors and supporters.
9	28		2%
10	5 October		2%	Submission of deliverables
11	12		2%	Success and Failure: Journey as Reward
12	19		2%	Pitch Night
13	26			Awards Night

Deliverables

Progress items:

1. Business Model Thinking
2. Stakeholder Engagement
3. Stakeholder Engagement Mentor Progress Score

(0, 1 or 2)

4. Concept Generation
5. Concept Generation Mentor Progress Score

(0, 1 or 2)

6. Value Capture
7. Value Capture Mentor Progress Score

(0, 1 or 2) Final deliverables:

1. Business Model Canvas: One page diagram of the business

model, using the IACT Business Model Canvas template , or similar (about 5% to 6%).

1. Executive Summary: One page text summary of the business model (300 words, about 5% to 6%).
2. Canvas Report: Five to eight page report on development of plan (this is equivalent to 1,500 to 2,400 words of assessment, about 30% to 50% of the assessment)
3. Continuation report: Detailed plan outlining funding requirements and proposed expenditure (Assuming 5 pages, that is 25% to 30% of the assessment).
4. Pitch: Notes and visual materials for a five minute presentation. A video of the presentation can also be provided, but for academic purposes, the assessment will be based on the ntoes for the presentation, not the presentation itself (assessment 5% to 10%).
5. Literature review: This would be in addition to the Innovation ACT deliverables, added to balance the assessment in the first half of the course and provide some academic grounding to the practical aspects (Two pages, 10%).

7. Assessment

The assessment would be made up of:

1. 20% for contributions to forums/exercises for ten weeks (2% per week for 12 weeks, with the best 10 counted)
2. Mid semester assignment: “Investigation of a strategic application of IT”. Individual work of 2,000 words, plus references 40%
3. End of course deliverables: Business proposal. Students are encouraged to undertake the work as part of Innovation ACT, or another innovation competition. However, the activity must take during the semester. May be performed in a group of up to six. 40%.

Students must achieve at least 10% for contributions to forums, 20% mid-semester and 20% final items to be eligible to pass the course (overall pass mark for course may be set lower, or higher than 50%).

Item	Date Due	Marks
Weekly contributions (2% per week, best 10 weeks of 12)	Due at the end of each week.	20
Investigation of a strategic application of IT	Mid Semester	40
Business Proposal:
Business Model Canvas		5
Executive Summary		5
Canvas Report		15
Continuation report		10
Pitch		5
Total		100

Materials

The Innovation ACT program provides a useful set of materials:

Program Information

• Team Registration Form (PDF)
• Participant Handbook (PDF)

• Mentor Handbook (PDF)

• Business Model Handbook (PDF)

Deliverables Templates

• Executive Summary template (PDF)
• Business Model Canvas template (PDF)
• Canvas Report template (PDF)
• Continuation Plan template (PDF)

Workshop 1 Materials – “Learning to Build an Innovative Venture”

• Team Hunting activity (PDF)
• Problem Finding activity (PDF)
• Business Model Thinking activity (PDF)

Workshop 2 Materials – “Relationships with Users and Partners”

• Presentation by Murray Rankin (PDF)
• Stakeholder Trading Cards activity (PDF)
• Pitch Outline activity (PDF)
• Mentor Matching activity (PDF)

Workshop 3 Materials – “Developing Prototype Solutions”

• User Journey activity (PDF)
• Business Model Evolution activity (PDF)

Workshop 4 Materials – “Getting Returns, Support and Funding”

• Costs and Suppliers activity (PDF)
• Customers and Revenues activity (PDF)

Relevant Skills and Experience of the Designer

Tom Worthington has trained in the design of blended and online courses and assessment, as part of a Graduate Certificate in Higher Education (ANU 2013) and Certificate IV in Training and Assessment (CIT 2013). He received the ACS Canberra ICT award 2010 for the design of the course “ICT Sustainability), which has been run by ANU and Athabasca University (Canada), since 2009. Tom was the Defence Project Manager for the DBQ Windows Project, federally funded to develop a graphical user interface for Australian database software (Worthington, 1994). He has evaluated applications for innovation grants for the Department of Industry, acted as a consultant for patent applications and provided advice on web design for the Beijing Olympics. Tom has been a speaker and mentor for Innovation ACT, on the organising team for GovHack and GovCamp Canberra and a judge for Random Hacks of Kindness (Sydney).

Contact the Course Designer

Institutions and organizations interested in offering this course can contact:
Tom Worthington FACS CP, TomW Communications Pty Ltd. http://tomw.net.au/admin/ tom.worthington@tomw.net.au

References

Innovation ACT, Participant Handbook, 2014. URL: http://www.innovationact.org/iact2013/wp-content/uploads/iact140813_doc_participanthandbook.pdf
Kruchten, P., Lawrence, P., Dahl, D., & Cubbon, P. (2011). New Venture Design–Interdisciplinary Capstone Projects at UBC. Proceedings of the Canadian Engineering Education Association. URL: http://library.queensu.ca/ojs/index.php/PCEEA/article/download/3637/3650
Worthington, Tom (1994, March, 18). DESIGNING A GRAPHICAL USER INTERFACE FOR AN AUSTRALIAN DATABASE PRODUCT: BACKGROUND TO THE DEVELOPMENT OF DBQ WINDOWS. retrieved August 5, 2014, from Canberra Branch Conference, Australian Computer Society Web Site: http://www.tomw.net.au/itinpdpj.txt
Worthington, T. (2011) ICT Sustainability: assessment and strategies for a low carbon future / Tom Worthington  Tomw Communications, Belconnen, A.C.T. Retrieved from: http://www.tomw.net.au/ict_sustainability/

Thesis Examination at ANU

Greetings from the Australian National University where Margaret Kiley iS speaking on "Everything you ever wanted to know about thesis examination, but were too afraid to ask". She explained that the thesis is just the end of a process which starts with the entry process. Previously at least a first class honors, or equivalent, was required for a PHD. There are now students who are ready to learn, but may not have this background.

ANU has "Research Theses Submission and Examination: Information for Higher Degree Research Students". The engineering college of ANU is an example of a detailed step by step process for a PHD, with Progress Milestones.ANU also allows a "Thesis by Compilation".

Dr. Kiley described a European approach where the candidate has to publicly defend their work, with anyone able to ask a question. Under the UK system students are asked questions in a "viva", with external experts asking questions in a closed group. Canada has an external expert assessing the thesis, followed by a small group asking questions  (I am a Canadian distance education student, where the questions are asked on-line). In Australia only the written thesis is examined by two or three anonymous examiners. I asked Margaret f there were any universities which conduct a double blind process (the examiners do not know who the candidate is, as well as students not knowing the candidate), she didn't and argued that the examiners knowing who the candidate is a good thing.

One aspect which worries me is that the examiner is paid only a token few hundred dollars to examine a thesis, when the work required would cost thousands of dollars. Margaret explained that the examiner does the job not for the money, but as a reciprocal arrangement and because they then have access to detailed new research. It would seem to me the result will be that only salaried academics (and full time researchers) will therefore be able to be examiners. It would be prohibitively expensive for someone who is paid by the hour to spend a week examining a thesis for a few hundred dollars. Also this process of favors may not meet the requirements of ethical standards. Universities in several Australian states are subject investigation against commissions against corruption. The situation where academics trade flavors with supposedly independent examiners may be seen as corrupt.

ANU currently has no set period for examiners to return their comments (four to six weeks looks reasonable).  One enhancement which could be made to the ANU system would be for the degree to be conferred electronically as soon as the student has met all requirements. This would allow the student to receive their degree weeks, or months, earlier. The award ceremony with gowns and pieces of paper could still be held, but this should not hoold up awarding of the degree.

Event Details

The examination process is one of the least discussed aspects of research candidature. Candidates and supervisors are often so busy getting research done they spare little time for thinking about the examination process until the very end. There is a surprising amount of literature on how examiners approach and respond to thesis text which can be a rich source of information about how to write one in the first place. In this session Margaret Kiley, author of the seminal paper It's a PhD, not a Nobel Prize discusses her research on the issue and the subsequent research that has been conducted.
How does the Australian examination process work and how does the examination of ANU PhDs stand up to international best practice?
Why do people fail or asked to do major corrections?
How can supervisors better prepare candidates for thesis submission and the examination process and what can ANU do to help?

Mat and Buck Excellent Adventure in Computer Learning

Today I attended a presentation by Matthew Alger and Buck Shlegeris, students at the Australian National University, on "Reward-modulated inference". The topic sounds dry and technical, but Matthew and Buck's two person hyperactive performance was very entertaining, in the style of "Bill & Ted's Excellent Adventure". While not qualified to comment on the technical detail, it stuck me that there are further insights for human learning which can be gained from machine learning. These include finding the right combination of supervised and unsupervised learning, rewards for reinforced learning, the value of preprocessing, deep learning in neural networks and advanced in extreme learning.

Extraordinary Meeting of Australian Vocational Educators About Scams

The Australian Council for Private Education and Training (ACPET) has called a meeting for Thursday this week to address misleading advertising, poor training and inappropriate soliciting of students (see media release) by some education providers. ACPET has 1205 members, including the Australian Catholic University, ANU College (Study Group Australia PTY Ltd), Australian Computer Society, Carnegie Mellon University, Department of Education and Training International (Queensland), University of Notre Dame Australia, University College London.

The ABC reported that companies have been offering free training courses with a free laptop for the student. After enrolling the student finds the course is not suitable for them and then have a large student loan debt ("Private training college watchdog urged to crack down on 'spruikers' misleading potential students", by Alison Branley and Norman Hermant, ABC, 19 Oct 2014).

Australian Council for Private Education and Training 

Media release, Tuesday 28 October 2014

Training sector gets tough on unscrupulous operators

The Australian Council for Private Education and Training has called together the leaders from Australia’s private higher education and training sector and industry to meet this week to develop an industry-led response to ensuring the integrity and quality private tertiary education.

The extraordinary meeting between the Australian Council for Private Education and Training (ACPET), English Australia, International Education Association of Australia, Australian Chamber of Commerce and Industry and the Chair of the Vocational Education and Training Advisory Board was convened by the new CEO of ACPET, Rod Camm, in response to reports about providers distributing misleading advertising, delivering poor training and soliciting students for courses they are possibly not suited to.

Mr Camm said private higher and vocational education providers had a proud history of delivering high quality and flexible programs.

“Our industry shares the concerns about inappropriate behaviour and it is vital that the sector takes a tough approach to poor practice, which undermines all of the good work that is happening across government, industry and the tertiary education sector”.

“Nobody is more frustrated or disappointed by poor practice than our quality members,” Mr Camm said.

“When unscrupulous operators do the wrong thing, they tarnish the reputation of our entire sector, which is overwhelmingly comprised of hardworking, reputable training providers that deliver high quality courses and achieve great outcomes for their students,” he said.

“When the leaders of the private peak bodies and industry meet this week, we will develop a response to poor behaviour - put simply they are not welcome.”

Mr Camm said ACPET recently expelled a member for misleading advertising, and has introduced a quality support service to improve performance across its membership. This will be the foundation for further changes and other industry peak bodies were doing similar things to ensure quality standards. He said it was vital that the industry work in a coordinated way, alongside state and federal regulatory authorities, to identify any weaknesses in the current approach.

“We need to eliminate poor performance so that it doesn't undermine the sector’s reputation in the middle of much needed reforms that improve the flexibility, competitiveness and industry relevance of both higher and vocational education. Students and industry overwhelmingly support greater choice and diversity, but they need to feel confident that the sector is committed to upholding the highest standards,” he said.

“Regulation is an important element, but so to is an industry-led approach. Our members are overwhelming committed to those high standards, but we also have a responsibility to weed out those that are not.”

The meeting of the sector’s leaders will take place in Sydney on Thursday (October 30).

Media contacts:
Ben Eade, Ten Colours, 0406 641 881
Rod Camm, ACPET, 0409 484 0511

Designing University Degrees

Last night at a professional association meeting I was asked by a member about their study options at university. Today I find myself at a meeting discussing the design of degrees. One thing that strikes me is the need for flexibility. I am a student in a program which offers almost all courses in three terms each year. Compared to this, in more conventional programs courses are only offered once a year.

Student Run Study Groups for Students

An interesting aspect of learning is the role of fellow students. One of the activities the Australian National University's ANU  Computer Science Students’ Association (CSSA) provides are "study groups". These are sessions run by students to help other students with their studies.

Adjusting to the Australian Qualifications Framework

The Australian Qualifications Framework (AFQ) has set standards for Australian higher education qualifications, which is a good thing. But this requires some adjustments by universities. One issue is how much teaching of research methods there is for honors students. There is a "AQF specification for the Bachelor Honours Degree", however honors is a hybrid between undergraduate and postgraduate. Honors is an extra year on a Bachelor degree, "to qualify individuals who apply a body of knowledge in a specific context to undertake professional work and as a pathway for research". Amongst the difficult questions this raises is how much about research do the students need to know and how they gain these skills. With just a year there is not a lot of time for formal research methods courses and then to "... plan and execute project work and/or a piece of research and scholarship with some independence".

Attrition and Completion in Doctoral Education: A Wicked Problem?

Nigel Palmer  will speak on "Attrition and Completion in Doctoral Education: A Wicked Problem?" at the Australian National University in Canberra, 5:00 PM 5 November 2014:

"Is doctoral attrition a wicked problem? This seminar examines time to completion and completion rates for international and domestic doctoral candidates by gender, and explores opportunities for developing better strategies for engaging and supporting candidates through to the successful completion of their degree. 

Better information on degree completions for doctoral candidates is essential if strategies for addressing student attrition are to be effective. This seminar reports findings from a scoping study supported by the Gender Institute and The Research Training unit at The Australian National University aimed at informing future research and engagement on patterns of participation and completion in doctoral degrees at the ANU, and in the Australian higher education sector more broadly. ..."

But in my view, doctoral completion rates only appear a wicked problem, if you take students ill suited to undertaking a thesis, for whom a thesis is not vocationally relevant and then have untrained supervisors attempt to supervise them.

There are well researched and proven techniques to improve doctoral completion rates. These involve having supervisors who have a formal qualifications in teaching and with training in how to supervise. The students, including research students, undertake a carefully designed program of coursework which emphasizes teamwork.

It needs to be recognized that only a small proportion of doctorates should be what the Australian Quality Framework calls "Doctoral Degree (Research)". Most doctorates should be Doctoral Degree (Professional), producing original research, but relevant to a job and with more coursework. Of the small number of research doctorates, almost all should be by "publication". The thesis should be reserved for the tiny fraction of the population who are capable of undertaking this most difficult form of education. I suggest less than 1% of directorates should be by thesis.

If doctoral degrees have well structured education programs, designed by a qualified educator and if the student has a supervisor formally trained and credentialed in how to supervise, problems are manageable.

Australia produces about 7,000 PHDs a year. Assuming 4,000 of these PHDs remain in Australia, that is about ten times the number the nation needs for research positions. 90% of the graduates will be in vocational positions and so need a professional doctorate. So a reasonable division for domestic students would be 10% Doctoral Degree Research (400 students) and 90% Doctoral Degree Professional. Of the Doctoral Degree Research students, I suggest 10% would be by a thesis (1% of the total doctoral degrees, or just 40 students).

This problem can be made even simpler by not artificially dividing students into different categories before the commence their studies. Instead of having to pick those students most suited to research in advance of seeing them do anything, the should enroll in a general postgraduate program and then discover which way their talents and interests are. In this way the less than 1% of doctoral students destined for a thesis could be identified. More importantly, the 99% of students not destined for a thesis need not feel they are failures.

edX Professional MOOCs Not Cheaper Than University

Massive Open Online Courses (MOOCs) have been promoted as a way to provide low cost education. However, I noticed that edX is now offering "edX Professional Education" courses which are considerably more expensive than previous edX MOOCs and comparable in cost to regular university courses.



Rice University is offering a course in "Basics of Energy Sustainability" for US$495. This is a four week course requiring two to four hours study a week (eight to sixteen hours total). That works out to between $31 and $62 per hour of study.

The course "ICT Sustainability" I run at ANU costs about $3,000. It is a typical twelve week long university course and requires eight to ten hours a week study (96 to 120 hours total). So this costs about $25 to $31 per hour, about the same, or less than the edXRice course. This is an on-line course which typically has twenty students and has a human tutor to assist.

Review of the Australian School Curriculum Misses Importance of e-Learning

The Australian Government has released the Final Report of a  "Review of the Australian Curriculum" by Professor Ken Wiltshire AO and Dr Kevin Donnelly.  Available are:

1. Final Report of the Review of the Australian Curriculum: (PDF 3.27 MB or DOCX 11.82 MB)
2. Initial Australian Government Response Review of the Australian Curriculum
3. Supplementary Material for the Review of the Australian Curriculum
4. Fact Sheet on the Review of the Australian Curriculum
5. Frequently Asked Questions on the Review of the Australian Curriculum

One concern I have is that sufficient weight is not being given to the way the Internet is changing how education is provided. The report mentions "Internet" only twice in 294 pages (on  Page 147).

In the first reference the authors comment that just having information via the Internet is not sufficient for understanding. However this fails to mention decades of research and practical experience showing it is possible to provide quality education on-line:

"The argument that the disciplines are changing so rapidly that it is impossible to identify them with any certainty or precision and, as a result, that all students need to do is to access the internet when wanting information, is misleading. Information is not knowledge and understanding is not wisdom. Education, while dealing with information and understanding, is primarily concerned with knowledge and wisdom that while evolving and open to debate has stood the test of time."

The second mention of the Internet claims that  "many" complained about a lack of hard copy of the curriculum materials. It is not clear how these many complaints were received, given that the review used an an online submission process (those who sent a submission must have had Internet access). Also, as the authors state, not all households have Internet access, but the ABS says 83% do. There are more than one million householders without Internet access, if many of had complained (in this context "many" would be tens of thousands of people).

"The bulk of parents seem to have been unhappy about the token involvement of parents in development of the curriculum, and report frustration in being able to gain independent access to curriculum documents. Many complain about the lack of hard copy, pointing out that not all households have internet access (265); nor are all parents computer literate. As in so many other aspects of government service delivery in Australia, purely web-based delivery is not adequate."

The technologies curriculum, that is teaching ICT is better covered in the report, than the use of ICT for teaching:

"There is also strong support for its inclusion in
the Australian Curriculum – particularly from professional bodies associated with computers and technologies – and a belief that it appropriately captured the critical elements of the learning area and provided a sound curriculum foundation which could accommodate future instances of digital technology. The Australian Computer Society (ACS) says:

The ACS strongly endorses the creation of the digital technologies subject and notes the
important distinction of this subject from the role of ICT as a general capability. Both aspects
are critically important in the education of students, but the distinction between them is vital
for individual students and for Australia as a nation.414" Page 208

Recommendation 15 of the report is likely to be most contriversial and with which I have most difficulty with:

"ACARA revise the Australian Curriculum to place more emphasis on morals, values and spirituality as outlined in the Melbourne Declaration, and to better recognise the contribution of Western civilisation, our Judeo-Christian heritage, the role of economic development and industry and the
democratic underpinning of the British system of government to Australia’s development." Page 246

It is not clear to me why the many Australian citizens who do not have a British Judeo-Christian heritage should have their taxes spent indoctoranating their children in someone else's values. Discussion of "Western civilisation" belongs in history courses, but alongside the other civilisations which have contributed to today's Australia.

An emphasis on morals and values would be appropriate for schools, along the lines of the  Primary Ethics, program developed by St James Ethics Centre for NSW primary schools. However, this should discuss Judeo-Christian concepts alongside other religions and moral systems of Australians.

Reducing a University's Carbon Emissions

In the article  "ANU is top carbon emitter among unis" (AFR, 10 Oct 2014) Tim Dodd reports that the Australian National University has higher carbon emissions per student than other Australian universities.

Carbon emission figures are published by the Clean Energy Regulator. The way these figures are calculated and how to reduce them is discussed in my book "ICT Sustainability: Assessment and Strategies for a Low Carbon Future" and the ANU course COMP7310.

Education and research organisations are well down on the list of emitters. For scope 2 emissions (use of electricity and other indirect emissions), the top emitter is CSIRO, in 98th place. The CSIRO and ANU have similar intensity of emissions per staff member, both being research organisations which also provide facilities to researchers at other institutions.

As has been pointed out ("Belted from pillar to post",  Sinclair Davidson, October 10, 2014), ANU hosts a
supercomputer used by researchers in government agencies (ironically to predict climate change from carbon emissions). During planning for the facility, I was asked what measures could be taken to reduce its energy use. With an administrative data centre you can lower power use at limes of low demand. But with a super computer there is only so much you can do to reduce energy use. A super-computer runs at full speed much of the time (that is what makes it "super") and so uses a lot of energy.

As discussed in my book and course, there are ways to reduce energy use of administrative computers and use computers to reduce emissions from other activities. One way is by Dematerialisation, which is not an exotic technology from Star Trek, but simply replacing paper with electronic processing. An example is the ANU Digital Transformation Project is to replace paper administrative forms. As well as the saving in energy used to make the paper, the energy needed to transport it around and store it (in some cases for decades) is saved. Of course the computer system used to replace the paper forms also uses energy and so it needs to be checked that it is run efficiently.

Other forms of dematerialisation are blended learning and e-learning, where part, or all of the classroom teaching is replaced with on-line equivalents. Blended learning reduces the classrooms needed, or allows existing classrooms to be used more intensively. E-learning eliminates classrooms entirely and then there are additional emissions savings from students not having to travel to class. But there is increased energy used by the computer systems at the university to provide the on-line courses and students will be using a computer more, which cuts into any overall savings.

Sustainability, Ethics and the University

Professor Ian Young, Vice-Chancellor of the Australian National University, announced on 9 October 2014 that
the university would divest itself of stocks in seven companies. This followed adoption of a Socially Responsible Investment (SRI) Policy and advice from the Centre for Australian Ethical Research (CAER), which rated the stocks on Environmental, Social and Governance (ESG) performance.

The ANU's decision has come in for criticism from mining industry representatives and the Australian Government for doing too much as well as from environmental activists for not doing enough. This range of criticism supports my own view, which is that the ANU has got Socially Responsible Investment Policy about right.

Some commentators suggest that ANU should leave any decisions on environmental effects of investment to the Australian Government and invest purely on financial criteria. However, if anyone should be investing with a view to the future, it is a university. I teach ANU students "ICT Sustainability" and professional ethics. The ANU's approach fits with what I teach and might make a case study for future courses.


Some commentators have criticised the ANU for not consulting the companies before deciding to divest their shares and others criticised it for not making all the details of the decision process public. However, shareholders are not required to consult a company before selling shares, they may need to inform the market if this could effect the market overall, which ANU has done. ANU has been more transparent in its investment decision than other institutions.

The ANU will have further difficult decisions to make over other investments. The seven companies divested so far are only about 5.1% per cent of the University’s Australian equity holdings (1% of total investments). The university can't avoid all environmental harm, or it would not be able to operate at all. Also it must consider the short term financial aspects as well as long term social ones.

As Professor Young, a respected scientist, said, climate change is "the most serious issue ever to have faced humanity". Combating global warming will inevitably adversely effect export of Australian fossil fuels, particularly coal. I suggest it would be preferable for Australia to start to adjust to this reality now, rather than wait until changes are imposed on it later.

As an Adjunct Lecturer at ANU (not a salaried employee), I don't have a financial interest in ANU's investment policy. However, I do contribute to a private sector ethical superannuation investment fund, which has a Socially Responsible Investment Policy. This has provided a good financial return on my investments (so far), as well as helping Australia adjust to a low carbon future.

Multiple Choice Questions Have a Role In Education

In its flexible learning strategy, the University of British Colombia identified issues for universities, the most important of which I would summarise as

1. Increasing focus on vocational education,
2. Online competition,
3. Demand for mid-career education.

The problem for higher education institutions is to provide courses which are accessible (preferably online), meet the vocational needs of students, but are also academically sound. In announcing the decision of Central Queensland University (CQU) to discontinue the use of multiple choice questions for examinations, Pro Vice-Chancellor, Professor Rob Reed, discussed the history of educational psychologist Frederick Kelly's use of the tests in WW1 US Army intelligence testing. CQU was concerned that multiple choice questions were not suitable for assessing real world skills. 

Anderson and Dron (2013) categorised three generations of distance education pedagogy based on the learning theories underlying them. However while being an improvement on an analysis simply based on the technology used, this still assumes that only one approach to learning can (and should) be applied in a course (or program). Students need to have a basic knowledge of a topic, before learning advanced skills. Rather than adopt one teaching and assessment approach for a whole course, these should suit the particular material and level. It is proposed that a combination of techniques can be used to teach basic and advanced material to the same students using the same learning technology.

 Multiple choice questions (MCQ) are a useful way to provide a student with feedback on their progress. This should not be seen as an alternative to assessment of deeper learning, but as a supplement. By banning MCQ for examinations, it is not clear if they have prohibited their use for other forms of assessment.
A search of the CQU Handbook found many courses for semester 3 2014, which have on-line quizzes as part of their assessment. An example is Aviation Theory IV (AVAT12004), which has seven on-line quizzes making up 70% of the assessment and "Written Assessment" for the remainder. It may be that these on-line quizzes use no multiple choice questions and all require numeric or short form text answers. However, that would not appear to be a particuarly useful assesment strategy, as it would take more time and staff effort, while not appreciably improve the quality of the assessment.