My Last Review

I have decided to stop reviewing papers for academic journals (I will still do for conferences I am involved with). The reason is that the academic publishing system is exploitative. Reviewers don't get paid, or any other form of compensation, for reviewing. In theory this is something you do, as you will then have your papers reviewed. But in practice there are many free riders. My gesture is a tiny one, but then I remember when I decided to stop giving lectures and that gained traction.

Authors submit papers and expect others to review them, but are not required to review. The authors receive a benefit from published papers, and a financial benefit when this results in a job, or research grant. Some publications are for profit and the published receives revenue from subscribers. The ones missing out from this are reviewers, who get no credit, or payment. So I will stop doing this. 

The system could be easily fixed. Reviewers could receive a voucher for each four papers they review, entitling them to priority processing for one paper submitted. For for-profit publications, they could simply be paid.

Publications could also invest in automated tools to take some of the drudgery out of reviewing. This would check for plagiarism (especially self plagiarism, where autoes submit the same paper, or ones with just a few changes, to multiple publications). Systems could also check references in papers. 

Accelerating and Democratizing Scientific Research Lifecycle

Greetings from the ANU School of Computing, where Qingyun Wang from University of Illinois Urbana-Champaign is speaking on Accelerating and Democratizing Scientific Research Lifecycle. Much has been written about more research of lower quality being published. Rather than bemoaning the role of AI in making this worse, here the idea is to use "AI for Scientists" (AI4Scientist) to address it. The first part of this is to automatically analyze papers to provide structured data. This can be used to explain the paper to the reader and also fact check it.

This can be taken further to expand the generation of hypotheses, but so far in very restricted well defined fields, such as biomedical. AI could also be used to seek out "hidden treasures" in the scientific literature. This would be a boon, as there is a lot of papers which simply repeat previous research, with slight variations. 

What I find most interesting is the possibility to use this to teach students to write better papers. It could also be used to help working researchers write more readable papers, with fewer errors. It would be interesting to see how well this would work for non-STEM disciplines where the format of papers is not so constrained. A particular problem for human readers, and likely for AI, are interdisciplinary research. 

Academic Comics Accessible?

Helen Kara has written an academic journal article in comic form. While innovative, as the text is part of the image it is not searchable, nor can it be turned into synthetic speech for someone who is blind. How can alternate text be provided for such an article? Do academic ethics require this? Does the law require it? Who's responsibility is it: the publisher, the author, or both? 

Future of Scholarly Publishing

The EU has released "Future of Scholarly Publishing and Scholarly Communication", by a panel of experts, including from the Bill and Melinda Gates Foundation, Wellcome Trust, Elsevier, and Springer Nature. Not surprisingly, Springer Nature and Elsevier did not agree to the rest of the experts' recommendation to move to open access publishing as soon as possible.

I don't agree with one of the panel's unanimous recommendations: not to use journal-based metrics for hiring, promotion and tenure, and funding decisions. In my view, metrics are essential, but the right metrics.

Several recommendations urge balanced and diverse representation (gender, geography and career stage). Using blind reviews, not just for papers, would go some way to achieving this.

The expert group proposes a set of principles for scholarly communication:

"... accessibility, maximum usability, and accommodating an expanding range of scholarly contributions (data, software, new documentary forms, etc.).  ... a distributed infrastructure based on open standards to ensure access and interoperability. ... equity, diversity and inclusivity, and to the need for community building. ... quality and the integrity of scholarly contributions ... flexibility and innovations while also retaining its focus on cost effectiveness."

From "Future of Scholarly Publishing andScholarly Communication" (EU, P. 6, 2019).

Recommendations to researchers and research communities:

1. "When participating in research assessment, for example in hiring, promotion and tenure, and funding decisions, focus on the merits and impact of a researcher's work and refrain from the use of metrics - particularly journal-based metrics - as a proxy. In particular, they should incorporate the recommendations from DORA and the Leiden Manifesto into the assessment process.

2. Take responsibility for ensuring that all research contributions are made openly available, discoverable, and reusable according to agreed community standards (including the FAIR principles).

3. Increase awareness of, and sense of responsibility for, implications of choices and actions in roles as authors, reviewers and members of decision-making groups.

4. Strive for a balanced and diverse representation (in terms of gender, geography and career stage) when seeking collaborations, organizing conferences, convening committees, and assigning editors and peer-reviewers, and building communities such as learned societies.

5. Work towards increased recognition and appreciation of peer-review work as core research tasks. To this end, support greater transparency, including the publishing of signed reports. Support better training and inclusion, and focus on quality of the research in peer review.

6. In the case of communities of researchers, such as learned societies, develop policies and practices that support modes of scholarly communication in line with the vision outlined above. Along with universities, learned societies and other research communities need to alert and train their researchers to the importance and the responsibilities of communicating knowledge, either formally, through publishing, or through other means."

Universities and research institutions should:

1. "Develop policies and practices to ensure that all research contributions are made openly available, discoverable, and reusable according to agreed community standards (including the FAIR principles).

2. Promote and implement the recommendations of DORA and the Leiden manifesto to ensure that research assessment takes into account a wide range of scholarly contributions including research articles, preprints, datasets, software, patents and materials (e.g. in hiring, tenure, and promotion decisions).

3. In deciding which infrastructures to use, support, and contribute to, choose platforms using free or open source software, offering open data via an open license, and leveraging open standards where possible. Acting in this fashion will also reinforce researcher-led initiatives that aim to facilitate scholarly communication and publishing.

4. Strive for a balanced and diverse representation including, but not limited to, gender, geography and career stage) when hiring, seeking collaborations, when organizing conferences, when convening committees, and when assigning editors and peer-reviewers, and building communities such as learned societies.

5. In negotiations with service-providers refuse non-disclosure clauses and include clauses which enable cost and price control, and compliance monitoring. Strive to facilitate collective action with other institutions by e.g. sharing cost and price data through joint initiatives (e.g. OpenAPC)."

Research funders and policy-makers should:

1. "Develop policies - along with appropriate funding mechanisms - to ensure all research contributions arising from their funding are available to everyone, everywhere, without any barriers to access or restrictions on reuse.

2. When evaluating researchers, ensure that a wide range of contributions (scholarly publications, but also data, software, materials etc) and activities (mentoring, teaching, reviewing etc) are considered, and that processes and criteria of evaluation are both appropriate to the funder’s research programme, and transparent.

3. Develop funding mechanisms to support the development of open, interconnected and distributed scholarly publication infrastructures, and for their maintenance over the long term.

4. Consider how funding policies affect diversity and inclusivity of research on a global scale. In particular, funders should work to ensure that review boards, committees, panels, etc., are diverse - in terms of gender, geography, and career stage.

5. Work with the other actors in the scholarly communications ecosystem to ensure that the total costs of enabling research to be openly available to everyone, everywhere, without barrier or restriction, be also open and transparent."

Publishers and other service providers:

1. "Develop and publicly announce transition plans to move as soon as possible to comprehensive open access (Springer Nature and Elsevier have differing views with respect to this recommendation, a result of extensive disscusions in the expert group.)

2. Develop, use, and support interoperable tools (including open source software wherever possible) and services not only to facilitate access and reuse of scholarly outputs, but also to facilitate innovative interventions of new entrants.

3. Strive for balanced diversity (including, but not limited to, gender, geography and career stage) among authors, reviewers, and editors who work with publications.

4. Foster transparency and accountability in peer review, for example by publishing peer review reports and author responses alongside the published articles

5. Make all publishing charges public (including special pricing and waivers), and provide full descriptions of services provided, in order to enable the development of a transparent and cost-effective marketplace designed to support the open communication and reuse of all scholarly contributions.

6. Experiment with new approaches to the evaluation and communication of research outputs, and share the outcomes so that a body of evidence can help to optimise future systems."

Practitioners, educators, and other societal groups:

1. "Organize and advocate for free access to, and right to reuse of, publicly funded research results.

2. Reach out to funders, research institutions, and policy makers in order to develop new communication channels, new forms of co-creation and co-planning of research, and new forms of funding in response to needs, concerns and issues emanating from the population at large.

3. Look for opportunities to engage with research topics / results that are of interest to societal groups and their communities.

4. Bring forward research topics/questions that are mis- or underrepresented (e.g. by contacting relevant researchers, attracting the attention of other actors in the science system, or mobilising action in organised interest groups)."

Papers, papers everywhere, but not a lot to read

Dolores Añón Higón

I spent the morning sifting though one hundred papers published in the last year on "ICT Sustainability". I found three of interest: "ICT and environmental sustainability"  (Hign, Gholami and Shirazi, 2017), "ICT Reduces National Carbon Emissions?" (Wagner, 2017) and  "Online Educaiton Reduces Carbon Emissions" (Versteijlen, Salgado, Groesbeek and Counotte, 2017).

There were about a dozen more papers which looked interesting, but I could not locate the full text of, despite having access to the library at a world leading university. There were many other papers where the grammar of the article was so hard to follow it was difficult to work out what the author was trying to say. Then there were the bulk of papers which did not seem to be saying much at all, at least nothing new.

One thing which would help such a search would be the ability to limit it to open access publications. Unfortunately, Google Scholar does not appear to have this option. Adding the phrase "open access" to the search doesn't really help: some papers are open access, but others are about open access and others are from a publisher which mentions other papers are (but no the one cited). A search using a specialized open access search tool "OAIster"  resulted in only eighteen papers on ICT sustainability for 2017. Of these eighteen papers there were five of interest, but three had no link to download and two were behind paywalls of for-profit publishers (thus not open access). The publications which appear to be most readily available are Masters theses from European open universities.

Reference

An Hign, D., Gholami, R., & Shirazi, F. (2017). ICT and environmental sustainability. Telematics and Informatics, 34(4), 85-95. URL https://doi.org/10.1016/j.tele.2017.01.001

Versteijlen, M., Salgado, F. P., Groesbeek, M. J., & Counotte, A. (2017). Pros and cons of online education as a measure to reduce carbon emissions in higher education in the Netherlands. Current Opinion in Environmental Sustainability, 28, 80-89. URL https://doi.org/10.1016/j.cosust.2017.09.004

Wagner, K. C. (2017). Towards a Green Global Golden Age?: ICT enabled cornucopian sustainability and a suggestion for its reform. Master Thesis Series in Environmental Studies and Sustainability Science. URL http://lup.lub.lu.se/student-papers/record/8912373

Not the Academic Book of the Future

Publisher Palgrave Macmillan are offering a free 134 page ebook on "The Academic Book of the Future" by Rebecca E. Lyons and Samantha Rayner (2015). But if this is the future of academic publishing, it is a very dull place. As an example, in chapter 12 Craig Dadds writes that campus bookstores are "essential". It is obvious that bookstores do add to campus life, but how they are to survive competition from on-line sales of paper and e-books?

My suggestion is that salvation for the campus bookstore will come with more students living on campus. Bookstores will turn into department stores, which also sell books. The UBC bookstore in Vancouver is a good example of this. When I was there last year I noticed that most space was taken over for selling clothing, bedding, small appliances and electronics (all things students need), along with textbooks.

More exciting than the content are the formats Lyons and Rayner's book is offered in. As well as the usual PDF, it is also in ePub version, which is impressive for a free book. The Kindle version mentioned is not really a Kindle version, just the PDF converted.

The ePub is only 683Kbytes, whereas the PDF is 7.2 MBytes. About 5 Mbytes is taken up by the cover image of the book being incorrectly formatted for PDF and the rest seems to be from incorrect encoding of fonts.

The book has the most open "by" license, so it would be very simple to produce your own version from the HTML in the ePub edition.

Open Scholarship

Open Scholarship seems an odd term to me: when was scholarship "closed"?

Willinsky (2009) offers a division of publishers into Independent Journals, Scholarly Societies and Commercial Publishers. They then trace open access back to the 1990s. But what might have been more useful would be to look at when academic publishing started to be closed. I would assume this might be around the time of the industrialization of scientific research in WW2.

Weller (2011) looks at scholarship from a more points of view than Willinsky (2009), looking at application and teaching as two scholarly activities which benefit from open access. Curiously neither author seems to worry much about scholarly discourse, which I thought was the whole point of the academic exercise. That is, you are not so much trying to set something in stone by publishing an academic paper, but continuing a discussion.

I am a member of the US based IEEE, who are a major publisher of engineering standards and papers. While IEEE is a scholarly society, their publishing arm is so big it is more like a commercial publisher, although they have a tradition of also supporting quirky little journals, which are more like independents.

The IEEE Open Access Publishing Options gives an interesting insight into how a publisher sees open access. Essentially, in IEEE's world "open access" is synonymous with "author pays". That is, the question from the publisher's point of view with open access is "who pays us?". If the reader is not paying for publication, then the next best option for the publisher is to get the author to pay.

The IEEE divides its open access publishing into:

1. Topical: Only three of IEEE's thousands of journals are fully open access, the first of which, IEEE Photonics Journal, was only started in 2012.
   
2. Hybrid: Authors have the option of paying to make content in otherwise closed journals open. The advantage for the author is that they avoid their paper being ghettoized in an "open" publication. IEEE point out that papers in high impact factor subscription based journals can be made open.
   
3. Mega: IEEE has created new "rapid-decision, open access mega journal" called IEEE Access. In some ways this is an attempt to reverse the trend which has seen increasingly specialized journals. The most interesting point with IEEE Access is the emphasis on rapid publication (4 weeks for review). However as a paper costs the author US$1,750.00 to publish, this is not open, in the sense of being available to the majority of authors (who could not afford the fee).
   

What seems curious is that, as far as I can tell, the review process for IEEE's papers is still unpaid, even where the author pays for rapid publication. Why should I, as a reviewer, give one of these papers priority, when I am not getting paid more (or anything at all) for this work?

Normally academia operates on an informal system of favors: do this for me and I will do something for you later. But if a commercial publisher, or quasi-commercial publisher (like IEEE), is charging a premium up-front fee for a rush service, what is the incentive for me to cooperate?

References

Willinsky, J. (2009). The stratified economics of open access. Economic Analysis and Policy, 39(1), 53-70. Retrieved from http://www.sciencedirect.com/science/article/pii/S0313592609500434/pdfft?md5=03632c0fbabd6d1dbb70133d756ff05a&pid=1-s2.0-S0313592609500434-main.pdf

 

Weller, M. (2011). The digital scholar: How technology is transforming scholarly practice. A&C Black. Retrieved from https://www.bloomsburycollections.com/book/the-digital-scholar-how-technology-is-transforming-scholarly-practice/ch4-the-nature-of-scholarship

ANU Reporter Electronic Edition

Last night I attended the launch of the Australian national University’s ANU Reporter app. Available for Apple iPad and Google Tablets, this is a new electronic edition of the university publication first published on paper in 1970. However, I was disappointed in the print and electronic editions of the magazine, both in terms of design and content. This is essentially a dull print publication converted to a dull downloadable format. I suggest ANU should flip its publishing process, producing for the web first, with downloadable (Apps) as a second priority and then providing print editions last of all. ANU can then target readers with content they are likely to want to read.

The first paper edition of the new ANU Reporter has a grey cover and mostly grey interior, with a few pages with some muted colours. The paper edition is on heavy matt paper. The overall effect is of a dull, serious, scholarly work, perhaps worthy, but not interesting.

The App edition of the new ANU Reporter mimics the paper edition, with the same grey tones and page based layout, with just a few videos added. Unfortunately this is essentially a dull paper magazine format converted to an electronic edition, not taking advantage of the electronic features.

ANU Reporter does not appear to be available online as ordinary web pages. As a result it will be difficult to find the content with a web search and to read it on an ordinary web browser.

The layout of the magazine seems to change from article to article. This reminds me of some student produced publications, where a committee could not agree on a style.

ANU Reporter is trying to be someone thing for everyone, with articles for students, Alumni, donors and others. However, as a result it is likely to appeal to no one. When the ANU Reporter was produced on paper it was necessary to limit the size and make difficult decisions on what should be in each edition to appeal to a wide audience. But this is not necessary in the electronic age: ANU can produce any number of different electronic publications for different readers and publish them as often as needed (from every few months, to every few hours). Excerpts from those publications can be published on paper, for the few people who do not read online publications (and to impress high-net-worth donors).

Providing online and then adapting for other delivery methods is also a technique ANU is yet to adopt for teaching.I will be discussing this in "Teaching Students to Work Together Online", at ANU, 3rd September 2014

Preparing a Poster for a Conference

A poster is a short form presentation at an academic conference. The process is much simpler, and often used to introduce new postgraduate students to conferences. Unlike a full refereed paper, the author just submits a short proposal, here is my poster "International Graduate Level Sustainable ICT Course" for CCA-EDUCAUSE Australasia 2011 (now called THETA Australasia).

Consider a Poster and Paper

Even if your full paper has been accepted for a conference you may still want to prepare a poster as well. Academic conferences have multiple streams, so every delegate can't attend every session. There is usually a poster time, where the author standards next to their poster and answers questions, giving you a chance to talk to people one-to-one, rather than the sometimes adversarial question and answer sessions in the paper sessions.

Unlike a full paper, which has to be submitted months beforehand, the author brings the physical poster with them to the conference. The poster is usually a paper document (possibly laminated with plastic for strength). Printed on one side of the page in colour. The poster might include a small sample object, or even a computer screen. But usually no power is provided and limited security, so any valuable or breakable object on the poster may be lost, stolen or broken.

The poster is placed on panels in an area of the conference venue, usually for the full period of the conference. The poster can include contact details for the author, so a delegate can arrange to meet. The author can use a computer for a presentation, but will have to prop it up as best they can, with no desk provided. Therefore a hand held tablet computer might be best for a demonstration.

Include a QR Code

It should be noted that many delegates at conferences, particularly at high tech ones, will have a smart phone or tablet computer. If the poster has a web address and a QR Code (a 2 dimensional barcode), this will allow them to obtain a copy of the poser, additional text, a video presentation and submit a question for the author. The Google URL Shortener will create a QR code from a web address.

Posters vary in size, but are usually 750 mm by 1200 mm  in the USA and A0 elsewhere (841 mm x 1189 mm). Large posters are usually in landscape mode (long side horizontal), but portrait mode is better for smaller ones.

Make it Bold

There may be hundreds of posters at a conference, with less than ideal lighting and positioning. Therefore the graphics used have to be bold and text kept short, in a large readable format. Many authors make the mistake of putting the text of a whole paper on the poster. The result is not readable from a comfortable distance. It is better to think of the poster as the slides for a short presentation.

"Advice on designing scientific posters" (Colin Purrington, 2011) provides a good overview of what to put in a poster. This suggests a person should be able to read the poster in at most 10 minutes, equivalent to a 15-minute talk.That sounds like too much content to me. A sample poster is provided by ANU CECS has about 1600 words (22,000 characters), which seems more than is desirable.

The conference organizer might provide a template for the conference, or your organisation may do so. ANU CECS have a LaTeX template (Zip) and a PowerPoint template (PPT). There are also specialist products, such as PosterGenius. Usually the poster will be created as a PDF file and then printed using a large format ink jet printer. When creating the PDF file, keep in mind that the images will be much larger than on a normal printed page and so need to be at a suitably high resolution.

A typical poster layout will have a header and footer one eighth the height of the page. The title will be at the top center. The organization name at the bottom. The main area of a large poster may be divided into four columns. Images may extend over more than one column. Usually dark text is used on a light background for the main area (sanserif font). Some designs have background images or patterns, but this can make the text hard to read.

An interesting variation on the academic poster is the Quad Chart format, as used for the NSF 2011 SBIR/STTR Phase II Grantee Conference. This divides the page into just four quadrants, each on a specific topic. It is popular for use by US military and government research funding bodies to obtain very brief descriptions of research proposals. The Georgia Institute of Technology Systems Realization Laboratory uses these topics for quad charts:

1. Motivation & Objectives
2. Approach
3. Key Results
4. Summary & Conclusions

The quad chart is in effect four slides for a very brief pitch presentation.

Consider a Smaller Poster

This year I was attending an international conference (ICCSE 2013) and wanted to display a poster, in addition to my presentation. But I was traveling extensively and did not want to be burdened with a large poster tube. I fond that an A2 poster (one quarter the area of the usual A0 poster) would fit in an airline carry-on bag. Printed on white polyester and rolled into a tight 10 mm cylinder, it resisted crushing well (the polyester looks like glossy paper, but is much stronger and thinner).

Two columns of text in A2 portrait worked well. The simplest way to format this was with the word processor set for an A4 page, which was then enlarged to A2 while printing. I used 10 point for regular text and 12 point for headings, but this was still a bit small when enlarged. 11.5 point text with 14 point headings might be better. Also I included a QR code, which people scanned with their mobile phone for more details on-line.

Perhaps academic conferences should switch to A2 size posters with QR codes as standard. This would allow twice as many posters in the available display space. It would be much more convenient for authors and organizers. It would also force authors to be more succinct in what they say on the poster. In addition A2 posters are much cheaper to have printed (a consideration for students).