Foundations of a Framework for Peer-Reviewing the Research Flow
Traditionally, peer-review focuses on the evaluation of scientific publications, literature products that describe the research process and its final results in natural language. The adoption of ICT technologies in support of science introduces new opportunities to support transparent evaluation, thanks to the possibility of sharing research products, even inputs, intermediate and negative results, repetition and reproduction of the research activities conducted in a digital laboratory. Such innovative shift also sets the condition for novel peer review methodologies, as well as scientific reward policies, where scientific results can be transparently and objectively assessed via machine-assisted processes. This paper presents the foundations of a framework for the representation of a peer-reviewable research flow for a given discipline of science. Such a framework may become the scaffolding enabling the development of tools for supporting ongoing peer review of research flows. Such tools could be “hooked”, in real time, to the underlying digital laboratory, where scientists are carrying out their research flow, and they would abstract over the complexity of the research activity and offer user-friendly dashboards.
KeywordsOpen peer review Digital science Open Science
This work is partially funded by the EC project OpenUP (H2020-GARRI-2015-1, Grant Agreement: 710722). The content of this work reflects the views of the author(s). The European Commission is not responsible for any use that may be made of the information it contains.
- 1.European Commission: Validation of the results of the public consultation on Science 2.0: Science in Transition [report]. Brussels: European Commission, Directorate-General for Research and Innovation (2015). http://ec.europa.eu/research/consultations/science-2.0/science_2_0_final_report.pdf
- 2.European Commission’s Directorate-General for Research & Innovation (RTD): Open Innovation, Open Science and Open to the World (2016). https://ec.europa.eu/digital-single-market/en/news/open-innovation-open-science-open-world-vision-europe
- 3.FOSTER: Open Science Definition. https://www.fosteropenscience.eu/foster-taxonomy/open-science-definition
- 5.Smagorinsky, P.: The method section as conceptual epicenter in constructing social science research reports. Writ. Commun. 25, 389–411 (2008). http://journals.sagepub.com/doi/pdf/10.1177/0741088308317815CrossRefGoogle Scholar
- 6.Teytelman, L.: We’ve been itching to share this! Integration of GigaScience and protocols.io is an example of how science publishing should work. Protocols.io news (2016). https://www.protocols.io/groups/protocolsio/news/weve-been-itching-to-share-this-integration-of-gigascience
- 8.Center for Open Science: Registered Reports: peer review before results are known to align scientific values and practices. https://cos.io/rr/
- 9.FORCE11: Guiding Principles for Findable, Accessible, Interoperable and Re-usable Data Publishing Version B1.0 (2014). https://www.force11.org/fairprinciples
- 15.Protocols.io team: How to make your protocol more reproducible, discoverable, and user-friendly (2017). http://dx.doi.org/10.17504/protocols.io.g7vbzn6
- 16.Tang, A.: ArrayExpress at EMBL-EBI - quality first! Repositive blog (2017). https://blog.repositive.io/arrayexpress-at-embl-ebi-quality-first/
- 18.Shanahan, D.: A peerless review? Automating methodological and statistical review (2016). https://blogs.biomedcentral.com/bmcblog/2016/05/23/peerless-review-automating-methodological-statistical-review/
- 21.Kraker, P., Bachleitner, R., et al.: Deliverable D4.1 – Practices evaluation and mapping: methods, tools and user needs (2017). http://openup-h2020.eu/wp-content/uploads/2017/01/OpenUP_D4.1_Practices-evaluation-and-mapping.-Methods-tools-and-user-needs.pdf