Workshop on ‘Open science’, Tel Aviv, January 24th, 2018
NEURON is fully committed to the highest possible standards of quality assurance, including working to nationally and internationally recognized best practices. Within a series of specific thematic workshops the issue of reproducibility in biomedical research was addressed at a workshop among NEURON members in Oslo, January 2017. This year’s workshop focused on the issue of Open Science.
NEURON’s implementation of standards in the design of animal research and full implementation of the 3Rs reflects contemporary good practice for all research using animals, and reinforces that these standards are important for ethical reasons and to obtain the best possible scientific results. On the lines of transparency, reproducibility and independent verification in biomedical preclinical research the next possible step is to share all data, and results of all analyses as well as – in ultimate consequence – any relevant syntax or code. The NEURON workshop on open science in January 2018 in Tel Aviv focused on open access to research including publication and data management plan policies. Everyone agrees that there are good reasons for having open data because it speeds research by enabling others to build promptly on results, and it improves reproducibility by allowing scientists to test whether claims in a paper truly reflect the whole data set, and it is useful in identifying incorrect data, and, not least it improves the attribution of credit to the data’s originators.
Open Access is the practice of providing online access to scientific information (publications and data) that is free of charge to the end-user. Motivation: Scientists and citizens should be provided with uncomplicated access to (publicly funded) research results.
Generally there are two models of Open Access to publications:
- Green Open Access: self-archiving of publications by authors (social sciences and humanities within 12 month, natural sciences within 6 months).
- Gold Open Access (= „Open Access Publishing“): the immediate publication of scientific information in the „Open Access”-mode (relevant journals and repositories). The EU funds both models.
Open Access is one very prominent part of Open Science (the global process of transformation and opening up science and research through Information and Communication Technologies), which is itself a key ingredient of the transition towards a data-driven economy and society. Open access to research is an indispensable, however, one of several elements of open science. Historically this started with the 2002 Budapest Open Access Initiative, and the 2003 Berlin Declaration on Open Access to Knowledge in the Sciences and Humanities. More than 475 Research Funding Organisations have signed the declaration until today.
The European Commission sees Open Access not as an end in itself ’…..but as a tool to facilitate and improve the transparency and circulation of scientific information in Europe and ultimately, to produce even more high quality science and contribute to better policy making.’ Almost all member states have set up legal and administrative rules to support ‘Open Access’ to scientific publications and some are also promoting Open Access to data. ‘Publicly funded knowledge must be available for researchers and the private sector to enhance the knowledge base, reduce regional disparities in terms of research, promote new technologies and products, and produce innovative solutions to societal challenges. Unrestricted and free of charge access to publications is backed by a growing number of universities, research centers and funding agencies across Europe.’ (Open Innovation, Open Science, Open to the World – a vision for Europe. EUROPEAN COMMISSION, Directorate-General for Research and Innovation, Directorate A – Policy Development and Coordination, Unit A1 – Internal and external communication. Brüssel, 2016)
With Horizon 2020 the EU mandates Open Access to all scientific publications resulting from Horizon 2020 funding. Accordingly all publications – after the peer-review process – must be available for reading, printing and download at no cost in an online- repository.
The EU did start the Pilot on Open Research Data to support scientists. Two factsheets with a number of useful links and a FAQ section aim to support researchers, project coordinators and administrators.
However, online and open access publishing bears some risks by having bred so called Predator Journals: The digital publishing is based on the principle that the author pays charges for the publication of their paper. The article is then available at no-charge to the end-user. This open access model however attracted profiteers and fraud. For a few hundred dollars ‘handling fee’ they offer to publish ANY scientific paper in journals with euphonious names (World Journal of International Research) – without any quality control. American librarian Jeffrey Bealls, University of Colorado collected over the years thousands of dubious pseudo journals in a list. However, since January 2016 the list is not further developed after the editors of those journals had threatened to sue the University.
Dr. David Mellor (Center for Open Science, Virginia, USA) introduced the Transparency and Openness Promotion (TOP) Guidelines that are promoted by the Center for Open Science. These describe eight standards that can be implemented over three levels of increasing rigor summarized as: Disclose, Require, Verify.
Pre-registration of the study design is one of the standards defined in the TOP Guidelines. It was highlighted how pre-registration helps improving the study design, mitigate the publications bias and clearly delineates exploratory from confirmatory research. The registered reports publication format offers an approach to the scientific community that emphasizes the importance of the research question and the quality of the methodology prior to data collection. The format requires authors of empirical studies to pre-register their study protocols including hypotheses, details on the sample size and sampling methods, and the analysis plan.
The submission of an empirical study outline is followed by a standard peer review process determining the quality of research methodology. If the study is accepted as a registered report, publication is granted to the authors irrespective of the outcomes.
At the Center of Open Science detailed information on the process is provided, and in particular a list of participating journals (currently 80) of which around 10 can be accounted to the neuroscientific area.
The Center for Open Science is a non-profit organization whose mission is to increase the reproducibility and transparency of science. To achieve that mission a number of measures are installed:
- Meta-science and replication studies to evaluate the barriers to reproducibility;
- Policy and training to reduce bias and increase transparency and the Preregistration Challenge https://cos.io/prereg and https://cos.io/rr), and
- A common infrastructure to enable open sharing of projects and preprints (https://osf.io and https://osf.io/preprints/).
Prof Dr Christine Winter (Charité, Germany) and Dr Idan Menashe (Ben-Gurion University, Israel) acknowledged the benefit of open access however, both addressed some obstacles that yet prevent widespread open access. The positive effect of open access publication is obvious. The benefits include that it enables data-driven replication, may inspire new research, and supports utilization of synergies and prevention of unnecessary duplication. For scientists this ensures enhancing the efficient use of resources. What makes open access attractive for researchers is an efficient publishing mechanism, fast processing and publication, access to a broad audience, and the option to publish data that could otherwise not be published. Why then do researchers only publish a smaller portion of their research in open access? In the current system, the scientific reputation is tied to high impact (journal) publications and data is a powerful resource that, if shared without exploitation, loses its exclusive value in a very competitive and self-referential system for reputation. Hence, traditional journals are attractive because they offer an established publication system that goes hand in hand with peer-recognition. In contrast, open access and data sharing are often not rewarded appropriately. Fraud in open access publications and “predator journals” fuel skepticism towards open access publication. It remains difficult for the researchers to assess the quality of open access journals.
Prof Dr Anton Bespalov (PAASP GmbH Heidelberg, Germany and Pavlov Medical University, Russia) stressed transparent reporting as essential element of Open Science. Both, exploratory and confirmatory research are necessary and decision-making on academic, industrial and funders levels requires research where various measures are taken to control selection, detection, attrition, performance, and reporting bias. Suggestions as to what can be done are on the funder/research authority level to make scientists aware of better research practices BEFORE the studies are done; a better study proposal likely translates into more reliable results. On the Journal/Publisher level the suggestion is to implement research rigor checklists to accompany every manuscript submission.
The visions of funding agencies were explored in several presentations. The aim of SNSF (Switzerland), presented by Dr Ayşim Yılmaz is to achieve 100% open access publication by 2020. Also, the MRC (UK) presented by Mrs. Geraldine Clement-Stoneham, and ZonMW (The Netherlands) presented by Dr Margreet Bloemers are at the forefront of a development towards Open Science. MRC even mandates open access publication and all publications must be archived in Europe PubMed Central plus.
Dr. Sascha Helduser pointed out NEURON‘s perspective in the Strategic Research Agenda that „Open access data policy should be a prerequisite for funding in the NEURON framework“ for the optimal use of data and resources, the reduction of unnecessary duplication, and the improvement of research success and efficiency.
Funders are indeed in a position to initiate change in the academic system. However, the change process needs time and must take place on all levels of the academic system. Therefore, ZonMW stressed that funders need to allow committed stakeholders to keep pace. To this end, ZonMW negotiated commitments of relevant boards of science organizations to open access, optimizing data for reuse, and supporting Open Science. MRC pointed out that there is no “one-size-fits-all-solution” and advocated a pragmatic approach. In addition, it is evident that funds need are necessary to support Open Science and need to be made available by funding organizations. This includes funding of open access publication as well as data repositories. For example, both MRC and SNSF allocate block grants for open access publication that researchers can apply for even after the runtime of their studies. This is important as most results are only ready to be published after termination of the project runtime. In addition, SNSF provides funding for data preparation and services to make data available through public repositories conforming to the FAIR principles. According to SNSF’s policies, service of repository providers has to be non-commercial. In addition, other ways of publishing could be encouraged like preprints in bioRxiv.
Digital repositories are new and integrated resources for information and present a research-supporting strategic infrastructure. However, this infrastructure require substantial organizational efforts, e.g. quality of repository and contents, and long-term storage and sustainability. Zenodo is a catch-all repository that enables researchers, scientists, EU projects and institutions to:
- share research results in a wide variety of formats including text, spreadsheets, audio, video, and images across all fields of science;
- display their research results and get credited by making the research results citable and integrating them into existing reporting lines to funding agencies like the European Commission, and
- easily access and reuse shared research results and, integrate their research outputs with the OpenAIRE portal.
The European Strategy Forum on Research Infrastructures (ESFRI) measure ELIXIR curates a list of recommended Deposition Database for experimental data: https://www.elixir-europe.org/platforms/data/elixir-deposition-databases. The list will expand over time.
Mandating Data Management Plans (DMP) was presented as another important element of quality assurance and transparency in science. A DMP describes procedures for collection and utilization of data throughout the entire project and its anticipated impact. EC and several funders request a DMP with grant applications.
According to experiences by MRC and SNSF, who require DMP with grant applications, such measure enforces the researchers to think thoroughly about their (to be produced) data including the necessary infrastructure demands to process and analyze the data (e.g. installing server, communicate with IT services, etc.). SNSF implements DMP in that this plan is formally required with the grant application. Researchers have the option to revise and up-date DMP throughout the runtime of their project, and the DMP is published at the end of the funding period on SNSF’s public database. Similarly, MRC requires DMP as part of the review process, and applicants may adapt DMP following reviewer’s comments.
NEURON strives to implement feasible measures for the Joint Transnational Calls to further support the funded projects for excellent research results.