Foundations of a Framework for Peer-Reviewing the Research Flow

  • Alessia BardiEmail author
  • Vittore Casarosa
  • Paolo Manghi
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 988)


Traditionally, peer-review focuses on the evaluation of scientific publications, literature products that describe the research process and its final results in natural language. The adoption of ICT technologies in support of science introduces new opportunities to support transparent evaluation, thanks to the possibility of sharing research products, even inputs, intermediate and negative results, repetition and reproduction of the research activities conducted in a digital laboratory. Such innovative shift also sets the condition for novel peer review methodologies, as well as scientific reward policies, where scientific results can be transparently and objectively assessed via machine-assisted processes. This paper presents the foundations of a framework for the representation of a peer-reviewable research flow for a given discipline of science. Such a framework may become the scaffolding enabling the development of tools for supporting ongoing peer review of research flows. Such tools could be “hooked”, in real time, to the underlying digital laboratory, where scientists are carrying out their research flow, and they would abstract over the complexity of the research activity and offer user-friendly dashboards.


Open peer review Digital science Open Science 



This work is partially funded by the EC project OpenUP (H2020-GARRI-2015-1, Grant Agreement: 710722). The content of this work reflects the views of the author(s). The European Commission is not responsible for any use that may be made of the information it contains.


  1. 1.
    European Commission: Validation of the results of the public consultation on Science 2.0: Science in Transition [report]. Brussels: European Commission, Directorate-General for Research and Innovation (2015).
  2. 2.
    European Commission’s Directorate-General for Research & Innovation (RTD): Open Innovation, Open Science and Open to the World (2016).
  3. 3.
  4. 4.
    Bechhofer, S., et al.: Why linked data is not enough for scientists. Future Gener. Comput. Syst. 29(2), 99–611 (2013). ISSN 0167-739XCrossRefGoogle Scholar
  5. 5.
    Smagorinsky, P.: The method section as conceptual epicenter in constructing social science research reports. Writ. Commun. 25, 389–411 (2008). Scholar
  6. 6.
    Teytelman, L.: We’ve been itching to share this! Integration of GigaScience and is an example of how science publishing should work. news (2016).
  7. 7.
    Cotos, E., Huffman, S., Link, S.: A move/step model for methods sections: demonstrating rigour and credibility. Engl. Specif. Purp. 46, 90–106 (2017). ISSN 0889-4906CrossRefGoogle Scholar
  8. 8.
    Center for Open Science: Registered Reports: peer review before results are known to align scientific values and practices.
  9. 9.
    FORCE11: Guiding Principles for Findable, Accessible, Interoperable and Re-usable Data Publishing Version B1.0 (2014).
  10. 10.
    Wilkinson, M.D., et al.: The FAIR Guiding Principles for scientific data management and stewardship. Sci. Data 3, 160018 (2016). Scholar
  11. 11.
    Candela, L., Castelli, D., Manghi, P., Tani, A.: Data journals: a survey. J. Assoc. Inf. Sci. Technol. 66, 1747–1762 (2015). Scholar
  12. 12.
    Assante, M., Candela, L., Castelli, D., Tani, A.: Are scientific data repositories coping with research data publishing? Data Sci. J. 15, 6 (2016). Scholar
  13. 13.
    Mayernik, M.S., Callaghan, S., Leigh, R., Tedds, J., Worley, S.: Peer review of datasets: when, why, and how. Bull. Amer. Meteor. Soc. 96, 191–201 (2015). Scholar
  14. 14.
    Carpenter, T.A.: What Constitutes Peer Review of Data: a survey of published peer review guidelines (2017). arXiv preprint arXiv:1704.02236.
  15. 15. team: How to make your protocol more reproducible, discoverable, and user-friendly (2017).
  16. 16.
    Tang, A.: ArrayExpress at EMBL-EBI - quality first! Repositive blog (2017).
  17. 17.
    De Roure, D., Goble, C., Stevens, R.: The design and realisation of the myExperiment Virtual Research Environment for social sharing of workflows. Future Gener. Comput. Syst. 25(5), 561–567 (2009). Scholar
  18. 18.
    Shanahan, D.: A peerless review? Automating methodological and statistical review (2016).
  19. 19.
    Di Leo, A., Risi, E., Biganzoli, L.: No pain, no gain… What we can learn from a trial reporting negative results. Ann. Oncol. 28(4), 678–680 (2017). Scholar
  20. 20.
    Borgman, C.L.: Big Data, Little Data, No Data: Scholarship in the Networked World. MIT Press, Cambridge (2015)CrossRefGoogle Scholar
  21. 21.
    Kraker, P., Bachleitner, R., et al.: Deliverable D4.1 – Practices evaluation and mapping: methods, tools and user needs (2017).

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Institute of Information Science and Technologies - CNRPisaItaly

Personalised recommendations