University Ranking

The Global Employability University Survey 2014 has ANU as the top ranking Australian university (20 in the world), followed by Monash University (33), UNSW (55) and University of Melbourne (50).

This compares with the Times Higher Education World University Rankings 2014-2015 with the top Australian institution, University of Melbourne (33), then ANU (45), Sydney University (60), University of Queensland (65), Monash University (83), UNSW (109).

The university ranking system I find most interesting  is Webometrics. This is based on an analysis of the university's web pages. With measures for presence, impact, openness and excellence. On these measures, Australia has first the University of Melbourne (82), then University of New South Wales (96), University of Queensland (98),  Australian National University (101) and Monash University (114).

Demonstrating the Value of Education

Geoff Irvine writes in "The Walking Dead in Higher Ed" (Inside Higher Ed, December 19, 201) that universities  "are struggling to meaningfully demonstrate the true value of their institution for students, educators and the greater community because they can't really prove that students are learning". The suggested solutions are to "Disrupt" by demanding short term change, "Get Expertise" to help with staff development outcomes/competency-based assessment. While I would benefit personally from such a move (having sent the last few years skilling myself to provide e-learning advice to universities), I urge caution: any attempt at a quick fix may risk an institution's reputation.

Irvine also suggests to "Rally the Movers and Shakers", by this he means the "innovators and the early adopters" I suggest this would be a better approach than being in outside consultants: have those already on the campus who can demonstrate how the new techniques they use produce results.

New Technology Course for New Entrepreneurs

The Australian Computer Society has an on-line course which might be of interest to new entrepreneurs with a technical background: "New Technology Alignment", starting 19 January 2015 (I am the tutor). This may be of interest to  former public servants taking part in the Public Sector Landing Pad program (PSLP) at Entry 29 co-working space in Canberra, and those in other co-working spaces around the world.

Course content

New Technology Alignment has an emphasis on opening your mind to new technical trends, and how they align with business today.

It includes four modules, which explore:

1. Frameworks for measuring the impacts of technology on business performance
2. Fostering innovation and encouraging adoption
3. Technology assessment and integration
4. Promotion and realising benefits.

Learning outcomes

• Knowledge and understanding of emerging technologies.
• Ability to identify new and emerging information technology trends and assess their relevance and potential value to the organisation.
• Ability to strategise for and promote emerging technology awareness among staff and management. 

Stopping Cheating in Higher Education

"An Australian guide to the risk management of VET online e-assessment"  has been released, this followed an enquiry ., which included a survey of stakeholders. This work is aimed at the VET sector but is also applicable to universities. This takes a very practical and systematic look at assessment. One important point is that plagiarism detection software is not a solution.

References:

1. Morris, T. (2014). An Australian guide to the risk management of VET online e-assessment: a companion document to the research report into the veracity and authenticity concerns of stakeholders. from http://e-standards.flexiblelearning.net.au/documents/2014_3_Australian-GUIDE-risk-management-of-VET-online-e-assessment.docx
2. Morris, T. (2014). An Australian enquiry into the veracity and authenticity of VET online e-assessment: a risk management approach to stakeholder concerns. from http://vital.new.voced.edu.au/vital/access/services/Download/ngv:65804/SOURCE202
3. Morris, T. (2014). Assessment e-Risk Survey of key stakeholders 2014: an Australian enquiry into VET online e-assessment: support document. from http://ngt.flexiblelearning.net.au/documents/2014_3_Assessment-e-Risk-Survey.docx

Australia Research Information Network

Greetings from an Australia Open Data Teleconference about the
the Australia Research Information Network (AURIN). The meeting was called by Steven Adler from IBM in NY. AURIN provides access to research data about the urban environment, such as demography, housing, transport, energy and water.

Knowledge super corridors in Southeast Asia

Professor Chun Kwong Han, Visiting Fellow, Crawford School of Public Policy, ANU talked on "Knowledge super corridors in Southeast Asia". He recommended

the "Cambridge Handbook of Strategy as Practice" (by Damon Golsorkhi, Linda Rouleau, David Seidl and Eero Vaara, 2010). Also I see he has published the paper "Economic Transformation in a Southeast Asian Country: Being Critical for Success", 2011. However, the presentation only briefly touched on knowledge super corridors and was a general theoretical analysis of the governance of government and non-government projects. He also mentioned Australia was a leader in e-commerce and noted the work of Pia Waugh at the Australian Department of Finance. As far as I could work out the "Knowledge super corridor" is similar to the Australia/Japan Multi Function Polis (MFP) of 1987, which did not live up to initial expectations.

Developing countries in Asia are in the process of transitioning from a production economy to a knowledge-based economy (k-economy). Various new knowledge and information communications technology (ICT) mega-projects are being designed and executed at the international, national, state and industry levels to sustain competitiveness. The structures and processes by which these so-called ‘knowledge super corridors’ are developed and implemented are complex economic-social-political decisions.

A sophisticated model is developed from critical and practice theories, whereby the new critical practice lens generates knowledge for understanding, evaluation and action. In this talk, Professor Han will illustrate the value of the model with two case studies on a k-economy blueprint and a knowledge portal in emerging k-economies in Southeast Asia.
He will also highlight the digital economy and ICT initiatives currently transforming Australia.

Your future is Australia's future

The Australian Government is running a TV advertising campaign asking the public to search for "Your future is Australia's future: The facts about higher education". This is the title of a web page, discussing the share of of course fees paid by the federal government, the quality standards from the Tertiary Education Quality and Standards Agency and the HECS student loans scheme. This then links to a "Changes to Higher Education" page, which says "The Australian Government wants to reform our higher education system, opening more pathways, providing more support and offering more choices.". Curiously this does not appear to mention the key aspect of the government's HE reform, which is to allow universities to set their own fees. It will be interesting to see if this advertising campaign will change the views of cross-bench Senators, who up until now have declined to support the government's proposals. However, in my view the proposed reforms and much of the public debate is missing the most important question which needs to be addressed: Will Higher Education Reforms Position Australian Universities to Compete On-line? 

ps: The Your future web page has some technical problems: 65 HTML Errors, a MobileOK Rating of only 19 out of 100 and 24 Accessibility problems.

The graphic for the Your future web page, is similar to that showing two people at Domainname.edu.au.

Innovations in Teaching Innovation in Canberra

I will be speaking on "Innovations in Teaching Innovation" at the first Australian Computer Society e-Learning Special Interest Group meeting in Canberra for 2015, on Wednesday, 11 February:

Tom takes over teaching the Australian Computer Society's "New Technology Alignment" (NTA) postgraduate course from January 2015. This offered on-line directly by the ACS Virtual College and through Open Universities Australia. Earlier in the year he attended a conference in Canada on computer science education and met with Canadian academics to discuss flexible learning and teaching innovation. Tom discusses how this might be done in Australia, by blending on-line formal courses with face-to-face competitions. 

Topics and speakers for the next four meetings are needed:

• Wednesday, 6 May
• Wednesday, 5 August
• Wednesday, 11 November

All will be held in the Graduate Lounge of University House, Australian National University, from 5pm for 5:30pm.

The meetings for 2014 were:

1. Brenda Aynsley, President of the Australian Computer Society,  on "Teaching with Technology", 16 July 2014.
2. Tom Worthington,Tutor at the ACS Virtual College and Adjunct Lecturer at ANU Research School of Computer Science, on "Teaching Students to Work Together Online", 3 September 2014.
3. Jill Andrew, Future Learning Team, on "Integrating Technology into the Australian School Curriculum", 8 October 2014
4. Dr McComas Taylor, Head, South Asia Program, ANU,  "The Joy of ePub", 5 November 2014

US Energy Expert Awarded Honorary Doctorate by ANU

Greetings from the Great Hall of the Australian National University in Canberra, where Dr. Steven Chu, Professor of Physics and Molecular & Cellular Physiology and former U.S. Secretary of Energy was just awarded an honorary Doctorate for "Science in the Service of Society". Dr. Chu was the first energy secretary who was a scientist and was charged with increasing renewable energy use. Last week he addressed the Light, Energy and the Environment Congress and I was impressed with the way he combined knowledge of the science and politics of climate change. ANU's conferring of this award could be seen as a subtle criticism of the Australian Government's "direct action" climate change policy, as he oversaw a very different policy as part of the US Obama administration. Dr. Chu is also be speaking at the ANU ECI Energy Update, tomorrow, 9 December 2014.

Economic Impact of Canberra's Universities

The report "Higher learning:Economic and social impact of the major universities in the ACT" (Deloitte Access Economics, July 2014), provides a detailed, if flawed, analysis of Canberra's students and the benefits they bring to the city. The report was commissioned by the ANU and University of Canberra and so concentrates on their students, but nevertheless brings together very useful information about the students.

According to the report, Education and Training makes up $2.3B, or 7.7%, of Canberra's GDP. This is the fourth biggest area of the local economy, after Public Administration and Safety (34%), Professional, Scientific and Technical Services (11%) and Construction (11%).

An interesting chart on page 14 of the report shows employment rates and wages for those with different qualifications (Sourced from the Productivity Commission). Not surprisingly, those with higher education qualifications have a higher chance of being employed and get paid more. But surprisingly, there is little difference between a VET Certificate and a university degree.

Table 5.2 on page 45 shows Education (Tertiary, Preschool and School Education) is the third highest category of employment for ANU graduates (10% domestic students and 15% International). However, Table A.3 (Page 53), shows that only 1% of ANU students are studying Education, compared to 10.9% at University of Canberra. This suggests ANU could expand its Education program and acquire about another 2,000 students as a result.


The report discusses possible future expansion of university's contribution to the Canberra economy. However, the report fails to mention the changes taking place in the way universities teach students. Teaching is moving from lecture based, to on-line and blended modes. I could not find the word "Internet" in the report at all. The only mention of "online" was for tools for treating mental disorders. The report points out "online programs are scalable and can be used en masse", but does not make the link between this and education generally, just for mental health treatment.

Education is changing so that students will not need to be on-campus for most of their studies. As a result, students will not need to be in the same city (or country) as their institution, most of the time. While this may sound far fetched, the newspaper and retail industries have been radically altered by the Internet. That change is now occurring in education. Canberra's universities could choose not to offer an on-line education option, in an attempt to keep the students on campus, but this is likely to instead result in students choosing a more flexible institution in another city (or country). Canberra needs to provide something for the students to do, such as a job, or working on a start-up company, to keep them in Canberra, while they are studying.

Will Higher Education Reforms Position Australian Universities to Compete On-line?

The debate over changes to Australian Higher Education has failed to address the major challenges which Australian universities face. Australian higher education policy needs to change to accept the reality that Australian students need vocationally orientated education and this can be delivered most effectively on-line.

The 2014-15Australian Federal Budget included reforms to the funding of higher education, most significantly, from 2016, the removal of the government imposed limit on the fees universities could charge domestic students (Australian Department of Education, 2014). The intention was to introduce this change from 2016, but the measures have been held held up in the Australia Senate, where the government does not have a majority. Negotiations with minor parties might see some form of legislation pass during 2015 (Knott,2014). 

However, proposals for universities to set their own fees, structural adjustment fund to help universities with the change and a scholarship scheme for regional students will not address the needs of Australian students. This policy misdirection is similar to ineffective government efforts to restructure the Australian motor vehicle industry, with all manufacturers announcing they will cease production by the end of 2017.

Hannah Forsyth's "A History of the Modern Australian University" pointed out that Australian universities were created to deliver practically orientated education in areas important to the nation, such as mining. The most effective and efficient to provide vocationally orientated education now is on-line. This opens the Australian education market to international competition, as happened with the motor vehicle industry.

The debate which needs to take place is over how to optimize higher education, using a combination of universities and vocational education institutions. Providing subsidies to prop up obsolete university campuses will not assist the students and will condemn the Australian education industry to a similar fate as the  motor vehicle industry (although the collapse of the education industry will be much quicker).

Apart from courses where students travel to another country, or study on-line at an overseas instution, Australian universities now face competition from an international on-line university in Australia.  Laureate International Universities, is an international for-profit provider of on-line university education. It is now accredited to deliver courses from its international network, through an Australian arm, Torrens University Australia (TEQSA, 2012). In addition, Australia has signed a number of free trade agreements with other nations, which cover services. Education providers in those countries can argue that Australian students studying at their campuses must receive the same Australian government funding support as students at Australian campuses.

The Australian organization which seems to have the best idea of what is happening with Higher Education, and has positioned itself for the change, is not a university. Open Universities Australia (OUA) is a consortium of universities delivering on-line education. This started as a brokering service where OUA got the students, who then studied at the member universities. But in the last few years OUA has acquired companies and set up new initiatives to deliver learning services and courses directly, including in the VET sector.

Earlier in the year I discussed the issue of the future of higher education with academics in Vancouver, but have not seen a similar discussion happening in Australia. It would be unfortunate if the the "reforms" to Australian Higher Education failed to address the important questions.

The debate over universities setting their own fees is reminiscent of the Australian Wool Reserve Price Scheme. This scheme put a floor under the price Australian wool producers were paid and was designed to smooth out fluctuations. However, when the price of wool dropped for an extended period, the only buyer was the scheme, until it ran out of funds. Similarly universities are deluding themselves if they think they can set the prices for courses. The price for courses will be set by the world market. A few Australian universities with a prestigious reputation will be able to charge a premium, but most universities will not.

Canberra Computing Education Conventicle

Greetings from the famous room N101 at the Australian National University where the "Canberra Computing Education Conventicle" just started. This is a small, informal conference (Conventicle) where new work, not ready for publication is presented. There was such an event in Melbourne last year. As this is new work I can't point to published papers on these specific topics, but will refer to previous work:

• Chair's welcome by Chris Johnson ANU
• Curriculum drift, Lynette Johns-Boast, ANU. Lynette suggested a mechanism for more frequent adjustment of th curriculum. One of the participants suggested assessment be at a program level, not course by course. This then lead to a discussion of external examination systems in India and in OxBridge.
• Stumbling Around Trying to Attract the Attention of Millennials, Tim Turner, UNSW Canberra. Dr. Turner pointed out many of todays university students have never known a time without a computer. He is concerned that computers are therefore not as inherently interesting to these students and constant exposure to the Internet has reduced their attention span (I am not sure this is true). He pointed out that making video takes new skills which academics may not have (I did a course in training video production, but am an exception). In my view the idea that today's students have a shorter attention span and are not really interested in learning for its own sake is nonsense (and something which academics all the way back to Aristotle). We now have about 50 years of experience in teaching vocationally orientated students and several decades of teaching students on-line. There are techniques for designing courses which have been tested and proven to work. The solution is for academics to enroll in training courses to learn how to teach. 
• The Three Faces of Quality, Craig Macdonald, Canberra
• Multiple-choice vs free=text code-explaining examination questions, Simon,    Newcastle 
• ANU Measuring Success: Varying Intention and Participation, Kim Blackmore,  ANU. Kim commented that the with Understanding India edX MOOC, the most popular topic was the role of mobile phones in story telling. The course had the same drop off in student numbers as other MOOCs, but what struck me was that about 70% of the students who were still participating at week three completed the course. This is about the same as for conventional 12 to 13 week courses, where students can drop out without penalty. Perhaps the ration of completions to the number participating one quarter of the way through the course would be a useful metric for comparison with conventional courses. One interesting result was that the Understating India course was popular with the Indian diaspora. What worries me about MOOCs is that this is a very expensive way to teach university academics how to design and deliver on-line courses. It would be much quicker, cheaper and more reliable to have academics who have to teach on-line to be formally qualified to do so (there are many good courses available to learn to teach on-line). It is not difficult to get academic to do training, if they are suitably rewarded.
   
• Computer Science Curriculum and Schools: Opportunities and Obstacles, Bruce Fuda, Gungahlin College/InTEACT. Bruce poitned out that Canberra's school based curriculum allowed them to have already been implementing the national Digital Technologies curriculum. University can expect to see Canberra students who have undertaken the school program within two years and NSW students within four years. Bruce pointed out that there were few ICT qualified teachers in schools and so those who are teaching the digital technologies program could benefit from external assistance. Bruce pointed out the curriculum included vocational skills in multimedia (which I don't think is a bad thing, I studied that at TAFE), but revision of the curriculum could have more ICT. Given the course topic is ICT and there is a shortage of trained teachers, it occurs to me that these courses could be run on-line with remote expert tutors, to assist local teachers, as well as the students. Teaching materials could also be shared with the VET sector for the advanced courses.
• Mobile Learning in Context, Chris Johnson ANU. Chris proposes to instrument mobile devices to see how students use them for learning. This sounds a useful idea, but it would be useful to first ask teachers and students how they use the technology. As an example, I envision that students would use a small mobile device for the discussion forums of courses I run: the posts are short enough for a small screen and can be read in small chunks. I would expect, at the other extreme, the student would need a larger device and more time, to write a 2,000 word assignment. Also I suggest the research needs to take into account that students are likely to be using multiple devices at the same time. It seems to me that school teachersare the most advanced in the use of mobile devices for learning and it would be better to look at what they are doing than school teachers.
• Flipping introduction to Computer Systems, Eric McCreath, ANU. Eric described how he converted a conventional course into a flipped one. What was most impressive about this was that it was not done with large amounts of additional resources. Eric recoded presentations with a web-cam in their own time and a weekly quiz to get them to watch the videos. Interestingly Eric used paper based multiple choice quizzes scanned with the standard university Multi-Function Unit, rather than requiring specialized "clicker" devices, or the student's mobile devices. The flipped format receives similar student feedback to the non-flipped version of the course.

The 2015 Canberra Computing Education Conventicle will be hosted by Tim Turner, at UNSW Canberra.

Experiments with users and sample size

Greetings from the Australian National University in Canberra, where Diane Kelly, University of North Carolina, is speaking on "Statistical power analysis for sample size estimation and understanding risks in experiments with users". Having struggled through a course in research methods, I was relieved to hear that there is no perfect sample size. Diane looked at some of the constraints on sample size, such as budget and time.

ABSTRACT:
One critical decision that researchers must make when designing experiments with users is how many participants to study. In our field, the determination of sample size is often based on heuristics and limited by practical constraints such as time and finances. As a result, many studies are underpowered and it is common to see researchers make statements like "With more participants significance might have been detected," but what does this mean? What does it mean for a study to be underpowered? How does this effect what we are able to discover about information search behavior, how we interpret study results and how we make choices about what to study next? How does one determine an appropriate sample size? What does it even mean for a sample size to be appropriate? In this talk, I will discuss the use of statistical power analysis for sample size estimation in experiments. Statistical power analysis does not necessarily give researchers a magic number, but rather allows researchers to understand the risks of Type I and Type II errors given an expected effect size. In discussing this topic, the issues of effect size, Type I and Type II errors and experimental design, including choice of statistical procedures, will also be addressed. I hope this talk will function as a conversation starter about issues related to sample size in experimental interactive information retrieval.

Blow up the MOOC

Yesterday the Australian National University hosted the last of this year's series of events about the future of education, focusing on Massive Open On-line Courses (MOOCs). A Podcast of the event is available. The event was called "Blow up the lecture?", but it is MOOCs have outlived their usefulness.

The panel was made up of and experienced MOOC maker from the USA,  Professor Armando Fox, Faculty Advisor to the UC Berkeley MOOCLab; an ANU professor developing a new MOOC at ANU, Gabriele Bammer from the National Centre for Epidemiology & Population Health; an ANU student who has taken some MOOCs
Benjamin Niles, President of the ANU Postgraduate and Research Students' Association (PARSA); and Year 8 student from a local school who has studied some MOOCs.

All the panelists were enthusiastic about the potential of large scale low cost or free courses to broaden education. While I am equally enthusiastic about making education broadly available, MOOCs are not the way to do it. Decades of experience in developing on-line courses, with hard lessons as to what works and what does not, are being ignored by MOOC enthusiasts. Also the recent evidence that MOOCs are not delivering claimed benefits is being ignored.

Fortunately most universities have only made a modest investment in MOOCs. There will be little financial or reputation loss if MOOCs fall out of favor over the next few months. Universities can, and should, continue to provide free and low cost extension courses, as they have done for hundreds of years. However, universities should not be distracted from their main role in degree education.

The idea that developing MOOCs will some how improve the overall quality of education at universities is flawed. This thinking needs to be flipped back to the traditional university approach, where materials from degree courses are adapted for short and extension courses. University degree programs are changing from a face-to-face mode to blended and on-line. Some of the materials and techniques used for teaching degree students on-line could be  adapted for the broader population. This is essentially the approach one of the panelists, Professor Armando Fox, . He did not get the chance to speak at length but discussed it in a previous presentation "How MOOCs Can Reinvigorate Classroom Teaching". Note that Professor Fox uses the term "Residential Education" to refer to face-to-face teaching in this presentation (not students living in dormitories),

In addition, universities need to keep in mind that there are other educational institutions tasked, and better equipped, to provide broad, low cost education for the community. The most enthusiastic proponents of MOOCs have tended to be prestigious western research universities. These institutions tend to have the least experience in on-line education. The less prestigious teaching universities, which have developed expertise in on-line education over decades, have tended to be sidelined in the MOOC hype, but continue to turn out well educated graduates each year.  Australia's TAFEs and Registered Training Organizations are set up to deliver unglamorous, but educationally sound, low cost on-line education.

Attrition and Completion in Doctoral Education: A Wicked Problem?

Greetings from the Australian National University in Canberra, where Nigel Palmer, honorary research fellow with the Centre for the Study of Higher Education at the University of Melbourne is speaking on "Attrition and Completion in Doctoral Education: A Wicked Problem?". He has been researching completion rates for doctoral candidates by international/domestic status and gender (and produced papers on education). He is looking at how to better support candidates, to improve completion times and rates completion of their degree.

Nigel argued that even if universities are full fee paying, students dropping out is still a problem. He then discussed if it was a wicked problem. He suggests it is because of the difficulty of measuring attrition as well as the management of attrition itself.

One reason for worrying about attrition is equity of access to education.  Nigel showed a graph showed the increasing number of women commencing a doctoral program over time. However, he then had a second graph showing that women were over represented in educaiton and helath, but many fewer in IT and Engineering.  One interesting trend was that in education and health participation was higher in undergraduate degrees than postgraduate ones, while it was revered in IT and engineering.

Nigel then showed a series of graphs showing international versus domestic students for the group of eight universities. Unfortunately all the colored likes look blue to me, so I could not see what the graphs were showing.

Nigel suggested that some measures to lessen attrition, such as research methods courses,  will also improve the graduate outcomes (courses for doctoral students is a hot topic at ANU at present). 

Nigel described a "crude" research doctoral degree completion rates, stupid statistic. This is calculated from the number of students completing in one year divided by the number there were the year before. This is not a very useful statistic because students normally take longer than a year to complete a degree. There have been attempts to come up with better measures, using longer periods and techniques such as a three year moving average (three years being more typical for degree completion). Unfortunately as Nigel points out such calculations are skewed by recent growth of international enrolments in Australia, which makes the relative competion rate look low.

The US PHD  Completion project found completion rates of 49% for humanities to 65% for engineering. Similar studies for ANU showed a completion rate of about 80% for full time students. Niegl's research is showing a similar high completion rate, but it is too early to report definitive results. The figures show an interesting S curve, with the rate of completions increasing rapidly for about the first five years and then tapering off. Nigel pointed out that there were a significant number of completions from five to ten years.

It is surprising how difficult it is to get an agreed measure and then actual statistics for completion rates for degrees. Universities are in the business of producing graduates (and this is a very large export industry for Australia). It would seem reasonable to have agreed measures for university output.

If there were more reliable statistics, it would be possible to consider measures to aid completion rates. But will this really have a useful effect on measures. As an example, if domestic male students complete at a lower rate than international female students, would it be appropriate to set a higher entry standard, or resources provided, on the basis of gender?

Nigel concluded that doctoral attrition was not a wicked problem and this this might engender a sense of helplessness.

I suggest some easy measures to address attrition would be to provide more flexible programs. As an example, Australian postgraduate students could be offered programs which allow for a mix of coursework and research. As is usual in North America, students would start with coursework and then, depending on their interests and aptitudes, focus on coursework or research. Also the option of the student completing a qualification lower than a doctorate, without feeling they have in some way failed, would be useful. In addition it would be useful to have accessible support services, which are available on and off campus, would be useful. Also services for monitoring supervisors in the way teachers are, would be useful.

Sample size risks in experiments with users

Diane Kelly, University of North Carolina, will speak on "Statistical power analysis for sample size estimation and understanding risks in experiments with users" at CSIRO IR & Friends at the Australian National University in Canberra, 4pm, 1 December 2014.

ABSTRACT:
One critical decision that researchers must make when designing experiments with users is how many participants to study. In our field, the determination of sample size is often based on heuristics and limited by practical constraints such as time and finances. As a result, many studies are underpowered and it is common to see researchers make statements like "With more participants significance might have been detected," but what does this mean? What does it mean for a study to be underpowered? How does this effect what we are able to discover about information search behavior, how we interpret study results and how we make choices about what to study next? How does one determine an appropriate sample size? What does it even mean for a sample size to be appropriate? In this talk, I will discuss the use of statistical power analysis for sample size estimation in experiments. Statistical power analysis does not necessarily give researchers a magic number, but rather allows researchers to understand the risks of Type I and Type II errors given an expected effect size. In discussing this topic, the issues of effect size, Type I and Type II errors and experimental design, including choice of statistical procedures, will also be addressed. I hope this talk will function as a conversation starter about issues related to sample size in experimental interactive information retrieval.

Innovations in Teaching Innovation

From January 2015 I am taking over teaching of the Australian Computer Society's "New Technology Alignment" (NTA) on-line postgraduate course. This is offered directly by ACS as part of its postgraduate certificate (register now for courses starting 19 January 2015) and through Open Universities Australia. The course was designed by

Professor Doug Grant and I will be just making some minor updates to the material. But suggestions would be welcome.

In August I met Philippe Kruchten at UBC Electrical and Computer Engineering at the Universality of British Colombia (UBC) in Vancouver, Canada. UBC's  "New Venture Design" course (APSC 486),  has engineering and business students learn to produce a business plan for a product. The students are encouraged to enter a innovation competition or program as part of the course. After looking at courses on innovation in Canberra, I created "Commercialisation and Entrepreneurship in Technology" which can be undertaken by students as a special topic.

Selling Social Science Impact

Greetings from the Australian National University where Professor Peter Davis, University of Auckland, is speaking on “Valuing the social sciences: An agenda for hard times". He argues that social sciences needs to makes its case as a useful field better and also apply more quantitative measures. He gave the example of driver education in schools, where the common suggests this is useful, but research shows it is not.
Professor Davis made the point that social scientists made a bigger impact that hard science, but outside scholarly publications.

This all seems very sensible and makes me wonder what social scientists normally spend their time doing. Peter commented that many social science students do not undertake any research methods course, which I found difficult to believe. The reason was that staff worry the statistic could drive students away from social science completely (which I can believe).  One quarter of the Master of Education I am doing is research methods, with courses on general, quantitative and qualitative methods (another chunk of the program is devoted to applying and communicating the results). Despite having a mathematics and computing background, I found statistics a very difficult subject and likely would have given up studies all together if that was what I first encountered. As it was, the statistics was something I knew I had to do.

I asked Professor Davis if social scientists could work more with the hard sciences, giving the example of climate change, where there is overwhelming evidence that global warming is real, but little action has been taken. He replied that the "nudge" theory, used in public health, could be applied. The idea is that people could be helped to make small changes to their behavior, rather than large, difficult,  lifestyle changes.

One recently example of where social science can help is the evaluation of social welfare policy. The Australian government introduced "income management", initially targeting aboriginal communities. The idea was that government payments would be made in a way which prevented the recipient from buying alcohol. Also requirements for recipients sending their children to school were made. However, a review of the program shows that it does not reduce alcohol consumption or increase school attendance. Unfortunately, it seems likely the government will ignore the evidence and expand the program anyway, for reasons of political ideology.

Making Stealth Paint for Aircraft

Greetings from the Australian National University where Alan W. Weimer from the Colorado Centre for Biorefining and Biofuels,, University of Colorado  is speaking on "Lab Curiosity to Commercial Process- what it takes". He started by talking about his experience of working on how to create boron carbide in industrial quantities. Alan commented on the problem of working with "white glove scientists" who concentrated on the purity of the final product, not the cost of the production process. The key to an efficient process was rapid heating. The problem was to produce a reactor vessel which could tolerate the high temperature. The solution was a transport flow, where the particles fall through a gas. However, the problem then was to find a market large enough to justify using the process. A new market was found in cutting tools for electronics production. Alan then changed topics and discussed the coating of particles with anti-corrosion materials for use on stealth aircraft.

ICT Sustainability Australia Briefing

I will be on a panel at the "Fujitsu ICT Sustainability Executive Breakfast Briefing" in Canberra, 4 December 2014. This is for the release of the Fujitsu Australia  "ICT Sustainability: Australian Benchmark 2014" report. Also speaking will be Lee Stewart, Head of Sustainability, Fujitsu Australia, William Ehmcke, CEO, Connection Research & Director of Foundation for IT Sustainability and Neale Rowe, Distinguished Services Engineer, Cisco. You can register for this free event.

It is an interesting time to be looking at computers and energy saving, with the G20 Energy Efficiency Action Plan, released last week, includes a strategy to reduce energy consumption for networked devices. How to progress the G20's energy efficiency plans would be a good project for my "ICT Sustainability" students.

Benchmarking with the Sessional Staff Standards Framework

Greetings from a workshop on "Benchmarking with the Sessional Staff Standards Framework", presented by Marina Harvey, of Macquarie University. This is part of the BLASST Project, which has produced a Sessional Staff Standards Framework, BLASST Guide. and an accompanying interactive tool (B-BIT).

Sessional staff are casual, non-perminat university teachers (called "Hourly Paid Lecturers" or "HPL" in the UK). Marina cited the work by May, Peetz, and Strachan (2013) on the numbers of sessional staff in Australia, doing about 50% of the teaching. This is of great interest in the higher education system, with a fear that teaching is becoming "casualised", with full time tenured staff being replaced with those employed for a few hours at a time. The worry is that students will receive a lower quality education as a result, as these staff may not receive training or be involved in the activities of the university.

Teach Students to Teach

As part of the BLASST workshop we were asked to come up with initiatives to further sessional staff development. An initiative I suggested was for universities to offer students a course in teaching, as part of their degree. This could be an expanded version of the type of tutor training students normally receive. This could be aligned with the VET Training and Assessment curriculum, so students could obtain a qualification (especially where the university has a RTO associated with it).

As an example, the ANU College of Engineering and Computer Science (CECS) runs a CECS Teaching Quality Program (TQP). Tutors are paid to attend the training, as well as for tutoring. An alternative approach would be to offer students teacher education as part of their program. Students would undertake a semester of tutoring as part of their education. Students would not be paid for this, as it would be part of their education. However, as well as normal course credit, students could be awarded a vocational qualification through ANU College, allowing students to teach in Registered Training Organizations. Such an initiative would likely be cost neutral: the money saved by not paying the students would likely cover the increased cost of the more extensive education and certification.

Some sessional staff could be offered multi-year year contracts, so they maintain their access to university resources between teaching periods. These staff could also be offered a process to obtain the same teaching qualifications offered to students in their degrees, as part of this (most likely by Recognition of Prior Learning). As an example the staff member could have an e-portfolio which was suitable for submission for certification.

Professors Do Not Make the Best Teachers

Universities like to promote the image of the tenured full time professor who conducts research and then passes on their decades of wisdom to the students. There are some such people,  but researchers do not necessarily make good teachers. Advanced students make very good tutors for less advanced students. The design of courses also requires specialist skills which are not part of academic training.

Students undertaking a degree, even the highest level doctoral degrees, need to understand that they have little chance of a full time career in academia. Universities turn out many times as many PHDs as needed for academia. Most graduates will be working outside the university and can hope for, at best, a part time casual role at university. Casual staff in any field have to take responsibility for their own training and development, including paying the cost of courses themselves and doing the training on their own time.

While the Blasst Project has produced useful material, the emphasis on sessional staff is not useful. If standards for teaching at universities are required, these should apply to all staff, regardless of their employment status. To say that sessional staff must meet teaching standards, while permanent staff do not, makes little sense. The Blasst Project materials would work just as well with the word "sessional" changed to "teaching".

A Plan for Teaching in The Internet Age

The BLAST project seems to be using a pre-Internet paradigm, with the assumption that teaching will be undertaken on campus, with staff and students physically present at the same time on the campus. With university teaching transitioning to the on-line environment, academic thinking needs to get past this last century paradigm. Most teaching will be done with students and their teachers not physically on campus. My rule of thumb is that a typical student will need to spend about 20% of their time on campus (one day a week for a full time student). Arrangements for staff, including training and "meetings" need to take account that most of them will not be on campus much of the time.

References

May, R., Peetz, D., & Strachan, G. (2013). The casual academic workforce and labour market segmentation in Australia. Labour & Industry: a journal of the social and economic relations of work, 23(3), 258-275. http://www.tandfonline.com/doi/full/10.1080/10301763.2013.839085#.VG0g-IXRWgY

Australian Government Open Data Repository Focusing on Real Data

Greetings from  the "Big data, big opportunity" conference, at the Australian National University in Canberra, where Pia Waugh, Director of Coordination and Gov 2.0, Australian Government is explaining the open data strategy. She emphasized that real data for data.gov.au is needed, not electronic copies of government documents. A good example is that the budget data was provided as computer readable files, as well as human readable documents. Not all government data can be made public as it contains confidential information about people and organizations. I noticed Pia is the only one of the presenters talking about information access how actually has a computer in front of them.

ps: Recently the Australian Prime Minister said "Coal is good for humanity ...", perhaps we can get Malcolm Turnbull , Minister for Communications to say: "Data is good for humanity, data is good for prosperity, data is an essential part of our economic future, here in Australia, and right around the world". ;-)

Climate Governance

Greetings from  the "Big data, big opportunity" conference, at the Australian National University in Canberra, where Eliza Murray is speaking on "Could order and ambition emerge from the fragmented climate governance complex?". She is using network analysis and demonstrated this using Gephi software. The idea is to look at the connections between national and international institutions involved with climate change. She estimates there are about 1,000 institutions to research, which therefore requires automated analysis. She intends to mine the website of climate change organizations and look at the overlapping memberships. She expects to find that the organizations have become decentralized by not fragmented.

Eliza commented that she initially stated her research on Australian organizations, but found these were not indicative of the international situation. When interviewing people in other countries she is asked "Why are these weird things going on in Australia with climate change?

I suggested Eliza contact Paul Thomas and the people at CSIRO researching information retrieval.

Citizen Mapping in Indonesia

Greetings from  the "Big data, big opportunity" conference, at the Australian National University in Canberra, where Christina Griffin is speaking on "Open access spatial data for effective disaster risk reduction". She is emphasizing the use of open street map for dealing with disasters in developing nations. While crowd sourced mapping data has limitations, it is better than no data at all. She is studying vulnerability to disasters in central Java, Indonesia. It happens I helped with the deployment of Sahana open source software for the 2006 Yogjakarta Earthquake in Indonesia.

New Zealand Educational Entrepreneurs

Greetings from the Griffin Room, overlooking Lake Burley Griffin at the Australian National University in Canberra, where Steve Thomas is speaking on ‘Putting a Value On It’. The value that New Zealand educational entrepreneurs plan to create. He started by defining social entrepreneurship, in terms of innovation, revenue generation for improving welfare. He said there was not much research on this.

He is studying "Partnership Schools" (Kura Hourua)  (PSKH). These allow more flexibility in teaching, with different teaching hours and unregistered teachers. Examples included military style schools, farm based and Mauri and pacific inland culture orientated schools. Some schools plan to use Wraparound Services to address health and low socio-economic status.

Steve pointed out that some of these services are not new, but are being delivered in new ways. But it is too early to show this works.

I asked Steve about use of e-learning in NZ schools, given the NZ Education Department developed the Mahara e-portfolio software. He commented that several of the people interviewed had commented they were looking at IT use, but did not seem to be clear on how to do this.

Scholarships for Study in Beijing

Greeting from the Australian National University where I am attending a presentation on "Schwarzman Scholarship".  This is a new program intended to be the Chinese equivalent of the Rhodes Scholarship at University of Oxford, England. The Schwarzman Scholarship funds a one-year master’s program at Tsinghua University in Beijing. The student doesn't need to have any prior Chinese language skills, but will be required to study basic Mandarin. Applications will be open from April 2015, with studies starting in June 2016.

Improve Student Writing with Teaching and Testing

Dr McComas Taylor, is reported to have said that some international students were "functionally illiterate" ("Higher education caught between integrity and fees", Amy McNeilage and Lisa Visentin, Newcastle Herald, 12 November 2014). That has not been my experience. University students do have difficulty with writing, but this can be addressed by teaching and testing communication skills. We can deter plagarism by teaching students how to write and testing them continuously throughout their studies.

In my paper "A Green Computing Professional Education Course Online" for the 7th International Conference on Computer Science & Education (2012), I noted written reports can be difficult for students, particularly those with English as a second language. One option I considered was increasing the English Language Competency score required for enrolling students. But this would then deny students the opportunity to improve their English while undertaking a course. Instead one approach I use in teaching "ICT Sustainability" (ANU COMP7310) is to give the students small writing tasks each week. This builds their confidence. Also it allows me to build a picture of the student's writing style and makes it harder for them to have someone else write an assignment for them undetected.

In the first two weeks of the course I have the students have use formal referencing, so I can identify those who need help with this. Some students, even graduate students, don't know how to cite sources and so are at risk of being accused of plagiarism. When faced with having to write, some students withdraw from the course (without penalty), but after some extra tuition most can meet the required standard.

One useful initiative is that students of the ANU Master of Computing now have to undertake two courses in professional communications (COMP8701 & COMP8705). Rather than just have subject matter tutors in each course dealing with writing problems, it makes sense to have specialist educators teaching communication.

Reference

Worthington, T. (2012, July). A Green computing professional education course online: Designing and delivering a course in ICT sustainability using Internet and eBooks. In Computer Science & Education (ICCSE), 2012 7th International Conference on (pp. 263-266). IEEE. DOI 10.1109/ICCSE.2012.6295070

Big Data Policy Conference in Canberra

There are still some free tickets left for the "Big data, big opportunity" conference, at the Australian National University in Canberra, 19 November 2014. This features Pia Waugh, Director of Coordination and Gov 2.0, Australian Government. This is a PHD conference, where the program is mostly research students presenting their work in short snappy presentations (along with some keynotes by celebrities).

The Big Data, Big Opportunity conference will examine opportunities presented by effectively harnessing big data, and in particular will look at how open data (across the Academy, government and industry) can enhance research, shape policy development, and impact on innovation. The conference will provide a forum for PhD students, academics, policymakers, and industry to jointly discuss the implications and challenges of moving towards open access data.
The conference will run the following topic streams:
• Economics/economic policy
• Public policy and governance
• Environment, development and resource management

My picks for the day

Time	Session
0845	Registration
0915	Welcome: Arjuna Mohottala, President, Crawford PhD Conference 2014 Organising Committee, Molonglo Theatre
0920	Keynote: John McMillan, Australian Information Commissioner, Molonglo Theatre
1010	Morning tea.
1030	Griffin Room: ‘Putting a Value On It’. The value that New Zealand educational entrepreneurs plan to create, Steve Thomas
1100	Lennox Room: Applying reinforcement learning to single and multi-agent economic problems, Neal Hughes Discussant: Akshay Shanker
1130	Griffin Room: Where big data meets no data, Belinda Thompson
1200	Griffin Room: Facing our demons: Do mindfulness skills help people deal with failure at work? James Donald
1230	Lunch.
1300	Griffin Room: Giving rights to nature: A new institutional approach for overcoming social dilemmas? Julia Talbot-Jones
1330	Molonglo Theatre: Small states, big effects? Oil price shocks and economic growth in small island developing states, Alrick Campbell. Discussant: Arjuna Mohottala
1400	Griffin Room: Open access spatial data for effective disaster risk reduction, Christina Griffin
1430	Lennox Room: Could order and ambition emerge from the fragmented climate governance complex? Eliza Murray
1500	Afternoon tea.
1520	Panel session: The successes, challenges, and future potential of big data, Molonglo Theatre Chair: Jenny Gordon, Principal Adviser Research Canberra, Productivity Commission • Andy Heys, Software Architect, IBM Australia • Greg Laughlin, Principal Policy Adviser, Australian National Data Service • Duncan Stone, Senior Manager, Open Innovation, Price waterhouse Coopers • Pia Waugh, Director of Coordination and Gov 2.0, Australian Government
1645	Wrap-up: Arjuna Mohottala, President, Crawford PhD Conference 2014 Organising Committee
1700	Close

Twenty Five Students in a Class Says Brain Research

At the moment I am studying educational psychology. Alongside the set readings, I have been studying "Towards a Theoretical Neuroscience: from Cell Chemistry to Cognition" by a Canadian, Andrew Coward (2013). This is an easier read the textbook I have been assigned.

It happens I have known Andrew for years. I caught up with him on a recent visit to Vancouver and he mentioned the book. Andrew is a retired telephone engineer, who decided to apply his knowledge of engineering complex telecommunication systems to explain the working of the human brain. I thought this nuts (and told him so), but the medical researchers he has been collaborating with seem to think there is something in it.

One of Andrew's concepts is that the brain uses multiple levels of abstraction for understanding. This is something directly applicable to learning, which you can present students with a broad concept and then teach more detailed ideas (or alternatively present details first and the big picture later).

One point I cam across was a discussion of the  maximum size of a group of people. Andrew quotes sources to say that the maximum size of a group of Pleistocene hunters was 25. But later settlements had 200 people. It is interesting that 25 is about the maximum size of a tutorial group in an Australian university (with one teacher per 24 students at Australian universities).

La Trobe University's Teaching and Learning Sapces Design Guidelines define a "Medium" Large Tiered Lecture Theatre as having 150 to 250 seats and being the largest size which can "facilitate an engagement between the lecturer and the assembled student audience through strategically positioned aisle ways". Andrew comments in his book that settlements 12,000 years ago had 200 residents. The lecture theatre with its aisle ways might be seen as a modern knowledge village.

APEC Education Mobility Data Collection Survey

The "APEC Education Mobility Data Collection Survey" was announced at the meeting in Chin. I was not able to find any more detail, but Richardson (2014) provided a Discussion Paper on "Promoting Regional Education Services Integration: APEC University Associations" at a Cross-Border Education Cooperation Workshop in May 2014.

The obvious way to provide students with access to education in other countries is by distance education with on-line courses via the Internet.  Richardson (2014) discusses Distance Education, Open Educational Resources (OER), Coursera and edX Massive Open Online Courses (MOOCs). However, the free and low cost online courses have been seen as a second class form of education and this has been further confused with the advent of free and low cost MOOCs. At the the 2014 IEEE International Conference on Computer Science and Education (ICCSE 2014) in Vancouver a few mo ths ago I preed a paper on "Chinese and Australian Students Learning to Work Together Online - Proposal to Expand the New Colombo Plan to the Online Environment". I was surprised this was not enthusiastically received by my Chinese colleagues, but this may be because of the perception that high quality courses have to be face-to-face.

In support of APEC’s student mobility target, the United States, with support from Australia, launched the first phase of a five-year project, APEC Education Mobility Data Collection Survey and Report, in July 2014.  For each member economy, the survey responses will be compiled into economy-level reports that will 1) identify key stakeholders in international education mobility, 2) document domestic data collection methodologies and indicators, and 3) show current trends in in-bound and out-bound education mobility.  Furthermore, the information collected will be used to establish a baseline of APEC economies’ currenthttp://www.whitehouse.gov/the-press-office/2014/11/11/fact-sheet-fulfilling-leaders-instructions-quality-higher-education education data collection efforts, which could then inform potential capacity building efforts to support APEC economies in tracking mobility.  Many economies have national data collection organizations, but the data collection process currently varies widely from economy to economy in terms of methodology, frequency of data collection, key definitions, and scope.  By aligning regional data collection methodologies and practices, APEC members can utilize that foundation to identify gaps in student mobility and opportunities to further promote regional educational exchanges in the lead up to 2020.  From "Fulfilling Leaders’ Instructions on Quality in Higher Education", The White House, USA,

November 11, 2014