Drafting proposals in the open – sketching out project ideas

[also posted in http://www.science3point0.com/evomri/2011/05/03/drafting-proposals-in-the-open-sketching-out-project-ideas/] This post is licenced under a CC0

Recapitulation

“Professionally our methods of transmitting and reviewing the results of research are generations old and by now are totally inadequate for their purpose.” Vannevar Bush, 1945

As announced last week, we – Fabiana Kubke and Daniel Mietchen – are currently participating in the Getting your CC project funded course at Peer-to-Peer University, and have decided to draft our proposal collaboratively and in the open. Part of our motivation is our (and others’) perceived need for making scientific information more useful by positioning it where it can be easily found, used, linked to, repurposed, and updated.

The introductory meeting of the course took place on April 26 (UTC) via Skype. We have since incorporated some of the feedback we got so far, and in this post – which Claudia Koltzenburg helped us draft – we will outline the next steps in the hope to entice others to get involved as well.

The course is scheduled around a series of workshops, each focusing on a different aspect of the proposal-writing process:

• May 3, Workshop 1: How do we move from having an idea to realizing those ideas in terms of having aims, and goals?
• May 6, Workshop 2: Which funding bodies are there that can give financial support, and how do we find appropriate sponsors for our project?
• May 10, Workshop 3: How can we structure our proposed work in terms of tasks and how do we make a realistic timeline?
• May 17, Workshop 4: Knowing how long time we estimate, and the resources we need, how do we put together a reasonable budget?
• May 24, Workshop 5: What happens after the funding period is over? How do we make the project sustainable?
• May 31, Workshop 6: What would we look at when reviewing another proposal.

The grant proposals are to be drafted in parallel to these workshops until June, when the proposals produced will be peer-reviewed, and professional feedback will be provided to increase the chances of getting funded. In preparation for today’s workshop, we will use this post to explore the aims and goals of our project(s).

The candidate projects in a nutshell

The ideas submitted as part of the application for the course all center around what could be thought of as an Encyclopaedia of original research, which shall therefore be the default focus of the grant proposal (as idea 1). Two smaller projects (ideas 2 and 3) build on idea 1 but are more specific and could thus be integrated into a proposal about idea 1, or developed independently, whereas idea 4 is wider in scope than idea 1. We expect the final scope of our grant proposal to be defined more precisely before attending Workshop 3.

“In the academy [..] we need to recognise an ethical obligation [...] which is at the core of our mission which is universal access to knowledge.” Larry Lessig, in the video embedded below, which is CC-BY-licensed.

The Architecture of Access to Scientific Knowledge from Lessig on Vimeo.

Idea 1: The primary aim of the Encyclopaedia of original research (henceforth EOR) is to arrange the existing scientific literature in a way that allows it to become dynamic in nature. The primary goal is to develop a platform that is able to capture and archive the open scientific literature such that the original work is being preserved (like at arXive or PubMed Central) but becomes dynamically and collaboratively editable (like at OpenWetWare). By way of such a platform, scientists and others could share their knowledge more effectively than through papers: Work on related matters could be more easily identified and conceptualized, and so could gaps in knowledge. Besides the possibility for direct editing, facilities for annotation, commenting and other ways of interaction with the community of researchers in the field would ensure the widest possible peer review.

The Encyclopaedia of original research combines two of the principles for open science that have been put forward by Science Commons: it takes the “Open Access” literature and recognizes that it too – like data – is a lost opportunity “without structure and annotation”. The characteristics of the platform that would contain the encyclopaedia are complex: it needs to track individual contributions to enable proper attribution, the content needs to be granular enough to be able to be cite individual elements within an entire piece, individual pieces of works or elements within it need to be able to be dynamically linked, indexed and contextualized, and the metadata needs to be structured to enhance discoverability, an attribute that is essential for reuse. The user interface also needs to be suitable for the different technological levels of knowledge or levels of comfort appropriate for individual scientists, so that technology is not a barrier for adoption and/or contribution.

We expect that the encyclopaedia will benefit science by helping to avoid duplication of research efforts (and related funding), providing a faster means of updating information otherwise delayed by prevalent publication cycles or not deemed “worthy” of formal publication (practical example case) and promoting the open discussion of research findings in light of new evidence.

The approach, though initially focused on contemporary literature, could likewise be applied to legacy literature – a start in this direction has already been made, as discussed here.

Idea 2: The aim here is to take advantage of such a repository to facilitate the delivery of scientific and health-related information to remote areas where this information may not be readily available but where access to it is essential for the well-being of those communities. What we imagine is that the above EOR could incorporate (or lead to) lay summaries (similar to AcaWiki of the scientific literature or such as those that already exist in blogs) as part of its knowledge base and both the original research as the lay summaries can be translated to local languages. Specific content that is relevant to specific world regions (e.g., malaria in Africa, Chagas in South America) can be bundled in formats that are compatible with existing local technology. One way to at least partially achieve this goal is to bundle region-relevant information so that it can take advantage of ongoing deployments associated with the One Laptop Per Child (OLPC) project and where it can reach the communities that would benefit most from that research.

Idea 3: The aim here is to take advantage of the infrastructure of the EOR (and part of the information contained within it) to complement (or support) digital collections not typically considered part of the scientific “literature”, e.g. from museums or databases. Take, for example this artifact from the Matapihi digital collection. The interaction of the user with the digital object could be enhanced by linking it to different representations of the same specimen (say, an MRI scan), or to relevant scientific information pertaining to similar specimens. It could further be brought back to life by linking it to other cultural artifacts: for instance, ‘Have specific works of poetry or music been inspired by these types of specimens?’, or ‘Are there local traditions or myths that are associated with the artifact?’, or ‘Is there a personal notebook of the individual who brought this specimen to where it is? As an example, this other digital object from the New Zealand National Library presents not just the artifact but the cultural context of what the object represents and how it relates to the local cultural heritage.

The fate of these ideas will depend on how the grant writing develops; the project as a whole could be shrunk to either of these projects (or similar ones), or these projects could be spinned off or retired.

Idea 4: The project could in principle also be expanded in scope, e.g. to test the efficiency of open versus traditional science. However, in order to produce a competitive grant on this big issue, we would require considerable support from beyond our current team of three.

The next steps

By May 10, we will need to identify in a first instance the type (or types) of funding bodies that would be suitable (at least in principle) to fund and/or sponsor the project. We would like to invite feedback and suggestions for that part of the process as well. For that purpose, we have set up a page on Wikiversity were we will be aggregating the relevant feedback we receive, and draft the next blog post in this series.

We would also like to invite feedback on which platform would be most suitable for the drafting of the full proposal. Different wiki spaces seem to be appropriate, as are Google Docs, but the idea of drafting it on GitHub is also on the table.

Science lessons from 8 year old children

[Cross posted from Talking Teaching]

Ed Yong in Not Exactly Rocket science alerted me to an article published in Biological Letters Biology Letters from the Royal Society. I will not discuss the content of the article, Ed Yong has (as usual) done a wonderful job. I would like instead to share the ‘concept’ of the article.

The article reports on some research that shows that bumble-bees use both colour and spatial relationship in their foraging behaviour. But enough about that. What is unique about this article is that the research was conducted by a group of school children. It is also unique in that it is written by a group of school children (in their language). And the icing on the cake are the figures: pencil coloured; no fancy graphic software.

This is, in my opinion, authentic teaching at its best. And authentic learning. And while we are at it, authentic publishing.

So what have I learned from this group of children? That, as they say, science is fun. And that teaching science, whatever the student age group, can be made fun and authentic and can get children motivated.

The background reads:

Although the historical context of any study is of course important, including references in this instance would be disingenuous for two reasons. First, given the way scientific data are naturally reported, the relevant information is simply inaccessible to the literate ability of 8- to 10-year-old children, and second, the true motivation for any scientific study (at least one of integrity) is one’s own curiosity, which for the children was not inspired by the scientific literature, but their own observations of the world.

I could not agree more. I love biology because I ‘played’ with biology as a child. I was fortunate enough to have a father who never answered my question with ‘I don’t know’ without following that up with ‘but lets try to find out’. As a child my father valued my questions and my curiosity, more so about things he didn’t have an answer for. And I will always be grateful to him for that. For my teachers, well, that was a different issue: rather annoying having a pupil in the class that just refused to overcome the ‘why?’ stage.

And these children have been given a great gift by being it let known that their thoughts and ideas have value. And that, once that barriers that have to do with the specific language of the scientific literature are withdrawn, their ideas and thoughts can bring about new knowledge.

These children will also grow up having learned a few fundamental things about science: How an idea is brought into shape, how scientific questions are narrowed, and the hard work and discipline that is needed to see an experiment through. Oh yes, and that no matter how good an idea may be, reviewers may still reject your grant.

None of this they could have learned from a science textbook.

The editors of the Royal Society should also be commended for not requiring that the manuscript adjust to the traditional publishing formats and allowing the authentic voice of the children to come through. This paper should become obligatory reading in science classes. If nothing else, children will recognise their own voices and curiosity in the reading, and, who knows, other groups of children with innovative teachers may teach us (adult scientists) another thing or two.

Citation:
P. S. Blackawton, S. Airzee, A. Allen, S. Baker, A. Berrow, C. Blair, M. Churchill, J. Coles, R. F.-J. Cumming, L. Fraquelli, C. Hackford, A. Hinton Mellor1, M. Hutchcroft, B. Ireland, D. Jewsbury, A. Littlejohns, G. M. Littlejohns, M. Lotto, J. McKeown, A. O’Toole, H. Richards, L. Robbins-Davey, S. Roblyn, H. Rodwell-Lynn, D. Schenck, J. Springer, A. Wishy, T. Rodwell-Lynn, D. Strudwick and R. B. Lotto (2010) Blackawton bees. Biology Letters DOI:10.1098/rsbl.2010.1056

[Open] Science Sunday – 15.08.10

News hit the stands about a new research collaboration to find biological markers for Alzheimer’s disease (read the stories in the New York Times and the Wall Street Journal). (HT @atreolar on Twitter). One thing that sets this collaboration apart was that the work being done would have researchers

“share all the data, making every single finding public immediately, available to anyone with a computer anywhere in the world.”

The advantages of sharing data were made clear with respect to this project in the article:

“Different people using different methods on different subjects in different places were getting different results, which is not surprising. What was needed was to get everyone together and to get a common data set.”

And this is a very strong argument for data sharing. But as interesting as the story itself is, I find more interesting some of the issues it identified with respect to scientists sharing data at such a wide scale. Specifically this paragraph brought back some things to mind:

“At first, the collaboration struck many scientists as worrisome — they would be giving up ownership of data, and anyone could use it, publish papers, maybe even misinterpret it and publish information that was wrong. “

This (with different grammatic construction) is the argument floating around. We (scientists) may all see the advantage of data sharing, but are we willing to ‘give it up’?

If you ask scientists many of us would probably say that we do science for a specific purpose: try to help find a cure for a disease, solve some environmental problem, to contribute to human culture through the creation of knowledge. Data sharing makes us put our money where our mouths are.

But is it that easy?

I would argue it isn’t. Even when we may be willing to put our data out there, to have others use it and interpret it, there is a reality we still need to face: our hiring and promotion committees. And these look at our scientific output as ‘papers published’.

There has been a lot of chatter on what the values of the papers are: should impact factor matter? Should we be looking at article level metrics? But either still look at the papers. Should we stop valuing papers and start valuing datasets?

I brought this issue up at the Data Matters MoRST meeting I attended. The current PBRF system is incompatible with data sharing. It still measures ‘output’ as individual papers. And whether I like it or not, my University’s funding (and my ability to survive in the system) depends on me satisfying these criteria. So to promote data sharing, this too needs to change.

I wonder what would happen next time I apply for promotion if instead of listing my publications on my CV I were to list my ‘datasets’: This is the data I have generated (and made public), and this is how it has been used by me and by others. Wouldn’t that be a real measure of the impact of my work? Does it really matter ‘who’ used the data to advance knowledge? Or in other words, has the time come for ‘Data Level Metrics’?

Perhaps if we gave data the same hierarchy as papers when it comes to evaluating performance, people may quickly learn that by putting the data out there the impact of our work may be easily increased (and measured). And we may be quicker to put it out.

On other news:

The Open Science Summit‘s opening session are now online thanks to ForaTV. It was a great opening session to be at, and I am glad I managed to make it there. Unfortunately I wasn’t able to stay for the rest of the meeting.

At the same time that this was happening, the government of New Zeland released its Open Access and Licencing Framework (NZGOAL). You can read about it on the Open Knowledge Foundation website, which has links to all of the documents. This is indeed good news for data sharing in New Zealand. And when I returned from my trip I found an email from The Creative Commons Aotearoa New Zealand informing me that I had been selected as a member of the CCANZ Advisory Panel.

I want to thank CCANZ for allowing me to be part of this panel, it is indeed an honour and I look forward to the good things that promise to come out of it.

[Open] Science Sunday – 18.04.10

Opening content by traditional Toll Access journals

At about the time that I had made the personal commitment to only contribute to Open Access publishing (as service on editorial boards or peer review) I was contacted by Georg Striedter asking me to join the Editorial Board of Brain Behavior and Evolution. After hearing from him his views about the new direction he was planning to take the journal, I could not refuse. Georg Striedter took the Editorial position for the journal starting this year and a big change ensued. The journal now has a new section called Highlights and Perspectives in Neuroscience, and articles in this section have been made free to access. I have been quite impressed with this section, and the quality of the discussions there. As an example, you can go and look at Mark Changizi’s piece “Neuroscientis’s Embarassment: Artificial Intelligence’s Opportunity” and Anat Barnea’s piece “Wild Neurogenesis”. For the latter, I recommend reading first a great summary of the article around which the discussion centres which was posted by NeuroDojo when the article came out. Mark Changizi’s piece is self contained but you might also want to check this awesome discussion around brain size.

Many journals are opening up some of their content for free, and this is a good move. For example, on the 26^th of March, Nature Publishing broke the news that the Nature News content free of charge. This is a great section and it is wonderful to have that content available to the general public.

Don’t mess with technology, what about BioTorrents?

On the 10^th of April I participated in the Public Acta meeting in Wellington. One of the statements of the Wellington Declaration said that

“[Technological Protection Measures] should not infringe on or limit the rights of users to use or access copyright material in a manner that would be permitted without the TPM”

One of the arguments raised that day, is that technology is sometimes attacked when it is can be used to infringe copyrighted works, but that restricting such technologies may infringe on the ability of accessing material that is otherwise legally available.

This week saw the publication of a paper in PLoS One by Morgan GI Langille and Jonathan A Eisen. BioTorrents: A file sharing service for Scientific Data. As Tim O’Reilly said on his Tweet, this is a great use of the BitTorrent technology. Here is a technology that has valuable applications and should, too be protected as such.

One may ask what prompted me to attend the PublicACTA meeting. The answer is simple: Most scientific information is behind the copyright that as authors we often transfer to the journals where our work is published (for journals outside the Open Access model). Education, Health and Science rely heavily on having good access to this information. Any decision to regulate copyright will inevitably have an impact on Education, Health, Science and Technology. So ACTA cannot be framed around the protection of recording artists and the film industry and not consider its implications for these other areas of public good. The text of ACTA will be made public next week, at which time I hope scientists, educators and health professionals will collaborate in making sure the implications for their fields are taken into account.

Thoughts on Hunter’s statement on Science, Climate Change and Integrity

Professor Keith A Hunter has published a statment on ‘Science, Climate Change and Integrity‘ in the Royal Society of New Zealand website. His position with respect to the controversies surrounding climate change issues are made clear, as is his call for a re-examination of attitudes on both sides of the argument.

The controversies surrounding the science of climate change underscore the need for a more open approach in the reporting of scientific data, and Professor Hunter’s statement makes a strong argument towards moving in that direction. A while back Cameron Neylon [1] wrote in his blog in reference to the CRU email leaks that

[...] scandal has exposed the shambolic way that we deal with collecting, archiving, and making available both data and analysis in science, as well as the endemic issues around the hoarding of data by those who have collected it.

There are many arguments in favour of making scientific data openly available. In a recent commentary on Science, Jean-Claude Bradley is quoted as saying:

“It’s sort of going away from a culture of trust to one of proof,” Bradley says. “Everybody makes mistakes. And if you don’t expose your raw data, nobody will find your mistakes.”

Along those same lines Hunter argues that

“Science is a rational endeavour that is based on logical and critical analysis of scientific theories in the light of actual evidence. It follows that scientific information, including a transparent description of how the data has been processed and tested against hypotheses, must be publically available, especially when it has been publicly funded [...]”

Without this open approach, the validity of scientific information has to be entrusted to the peer review system, but even Professor Hunter echoes what are concerns of the scientific community at large, that

“while we place great faith in the peer review process to weed out ideas that are wrong, peer review is not perfect and can be abused by both sides.”

And further argues that:

“If the intensity of the personal attacks on climate scientists over recent months are to have any positive effect, it will be the adoption of a more transparent approach to the dissemination of information.”

Two sides of the coin: Public/Open access vs Open Data:

Although the issues surrounding open data and open access can be seen to sit under the same umbrella, they really deal with two slightly different issues regarding the dissemination of information. While public or open access to published data is now a requirement by many public funding agencies, and is important for the public dissemination of information,  it does not in itself solve issues such as those raised around climategate.

I am a strong supporter of Open Access publishing — whereby the public has access to the published information — but unfortunatley it shares some of the same shortcomings with toll access publishing when it comes to the review process: the process is not fail-proof. The Public Library of Science has to be commended for opening the post-publication discussion of the work they publish and making it possible to highlight both shortcomings and strengths in the published material, and in this way allowing to correct any errors associated with the peer-review process. Yet even within this open model the criticisms are raised about the published material itself, and this solution falls short of solving the kinds of issues raised by the opponents of climate change: The raw data is not published by default (although it can be requested and it is PLoS policy that it should be made available by the authors).

Open Data on the other hand makes the raw data available: the analysis can be checked, rechecked, rehashed and reanalysed by other people. And as Jonathan Eisen is quoted as saying in the Science article, people do find mistakes.

Opening the data allows those mistakes to be found and to be corrected, and that can only be good since the ultimate goal of science is not to defend one’s pet theory but to keep one’s mind open to find the answer that is most consistent with the data. As Lawrence Krauss said in his lecture:

“I would argue that the definition of open-mindedness is forcing our beliefs to conform to reality, and not the other way around.”

Opening the data will inevitably lead to agreed upon interpretations that conform with reality: It allows the conversation to centre around scientific facts rather than around personal attacks to the scientific community or to the groups of vocal skeptics. And, ultimately, finding the best answer is the one common ground that is shared by both groups.

So what next?

Hunter states that:

[...] it is only fair to expect the critics of the mainstream scientific views [...] to adopt an equally transparent approach with their own information, and with their own use and re-analysis of data entrusted to the public domain. They should also subject their findings to rigorous peer review. Opinion, however forthrightly expressed, will not change the laws of basic science.

As far as I know, there is still an Open Access Mandate to be had in New Zealand’s public research funding agencies. Let alone one on Open Data. So it was not without surprise that I read Hunter’s statment that

“In this regard, the Royal Society of New Zealand intends to play its part by developing a Code of Practice for Public Dissemination of Information that it hopes will assist the various New Zealand science organisations in improving their practices.”

This approach is needed if common ground is to be found between scientists and between scientist, society and policy makers. Disagreement is perhaps the strongest force that moves the interpretation of scientific data within the bounds of the most likely explanation. But disagreement can only move in a positive direction when all parties involved have equal access to information. In science that is called the data.

[1] Cameron Neylon’s blog has now moved here.

[2] Disclaimer: I receive and have received research funding from the Royal Society of New Zealand and am an Academic Editor of PLoS One.