Abstract
One of the pillars of the scientific method is reproducibility – the ability to replicate the results of a prior study if the same procedures are followed. A lack of reproducibility can lead to wasted resources, false conclusions, and a loss of public trust in science. Ensuring reproducibility is challenging due to the heterogeneity of the methods used in different fields of science. In this article, we present an approach for increasing the reproducibility of research results, by semantically describing and interlinking relevant artifacts such as data, software scripts or simulations in a knowledge graph. In order to ensure the flexibility to adapt the approach to different fields of science, we devise a template model, which allows defining typical descriptions required to increase reproducibility of a certain type of study. We provide a scoring model for gradually assessing the reproducibility of a certain study based on the templates and provide a knowledge graph infrastructure for curating reproducibility descriptions along with semantic research contribution descriptions. We demonstrate the feasibility of our approach with an example in data science.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
References
Estimating the reproducibility of psychological science. Science 349(6251), aac4716 (2015). https://doi.org/10.1126/science.aac4716, https://www.science.org/doi/abs/10.1126/science.aac4716
Buys, C.M., Shaw, P.L.: Data management practices across an institution: survey and report. 3(2), 1225 (2015). https://doi.org/10.7710/2162-3309.1225
Chang, A.C., Li, P.: Is economics research replicable? Sixty published papers from thirteen journals say “usually not” (2015). https://shorturl.at/jlpxQ
Chen, X.: Open is not enough. Nat. Phys. 15, 7 (2019)
Dewald, W.G., Thursby, J.G., Anderson, R.G.: Replication in empirical economics: the journal of money, credit and banking project. 76(4), 587–603 (1986). https://www.jstor.org/stable/1806061
Feger, S.S., Dallmeier-Tiessen, S., Woźniak, P.W., Schmidt, A.: The role of HCI in reproducible science: understanding, supporting and motivating core practices. In: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, CHI EA 2019, pp. 1–6. Association for Computing Machinery, New York (2019). https://doi.org/10.1145/3290607.3312905
Feger, S.S.: Interactive tools for reproducible science - understanding, supporting, and motivating reproducible science practices, p. 221 (2020)
Feger, S.S., Woźniak, P.W.: Reproducibility: a researcher-centered definition. 6(2), 17 (2022). https://doi.org/10.3390/mti6020017, https://www.mdpi.com/2414-4088/6/2/17
Figueiredo Filho, D., Lins, R., Domingos, A., Janz, N., Silva, L.: Seven reasons why: a user’s guide to transparency and reproducibility. 13(2), e0001 (2019). https://doi.org/10.1590/1981-3821201900020001, http://www.scielo.br/scielo.php?script=sci_arttext &pid=S1981-38212019000200400 &tlng=en
Freese, J., Peterson, D.: Replication in social science. Annu. Rev. Sociol. 43(1), 147–165 (2017). https://doi.org/10.1146/annurev-soc-060116-053450
Goodman, S.N., Fanelli, D., Ioannidis, J.P.A.: What does research reproducibility mean? Sci. Transl. Med. 8(341), 341ps12–341ps12 (2016). https://doi.org/10.1126/scitranslmed.aaf5027, https://www.science.org/doi/abs/10.1126/scitranslmed.aaf5027
Howison, J., Herbsleb, J.D.: Scientific software production: incentives and collaboration. In: Proceedings of the ACM 2011 Conference on Computer Supported Cooperative Work, CSCW 2011, pp. 513–522. Association for Computing Machinery, New York (2011). https://doi.org/10.1145/1958824.1958904
Hoy, M.B.: Big data: an introduction for librarians. Med. Reference Serv. Q. 33(3), 320–326 (2014). https://doi.org/10.1080/02763869.2014.925709. pMID: 25023020
Hussein, H., Oelen, A., Karras, O., Auer, S.: KGMM - a maturity model for scholarly knowledge graphs based on intertwined human-machine collaboration. In: Tseng, Y.H., Katsurai, M., Nguyen, H.N. (eds.) ICADL 2022. LNCS, vol. 13636, pp. 253–269. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-21756-2_21
Key, E.M.: How are we doing? Data access and replication in political science. PS: Polit. Sci. Polit. 49(2), 268–272 (2016). https://doi.org/10.1017/S1049096516000184
Knublauch, H., Kontokostas, D.: Shapes constraint language (SHACL). W3C Candidate Recommendation 11(8) (2017)
Krawczyk, M., Reuben, E.: (Un)available upon request: field experiment on researchers’ willingness to share supplementary materials. 19(3), 175–186 (2012). https://doi.org/10.1080/08989621.2012.678688
Leek, J.T., Peng, R.D.: Reproducible research can still be wrong: adopting a prevention approach. Proc. Natl. Acad. Sci. 112(6), 1645–1646 (2015). https://doi.org/10.1073/pnas.1421412111, https://www.pnas.org/doi/abs/10.1073/pnas.1421412111
Lucas, J.W., Morrell, K., Posard, M.: Considerations on the ‘replication problem’ in sociology. 44(2), 217–232 (2013). https://doi.org/10.1007/s12108-013-9176-7
Markowetz, F.: Five selfish reasons to work reproducibly. 16(1), 274 (2015). https://doi.org/10.1186/s13059-015-0850-7
Munafò, M.R., et al.: A manifesto for reproducible science. 1, 0021 (2017). https://doi.org/10.1038/s41562-016-0021
Nosek, B.A., et al.: Promoting an open research culture. Science 348(6242), 1422–1425 (2015). https://doi.org/10.1126/science.aab2374, https://www.science.org/doi/abs/10.1126/science.aab2374
Peng, R.D.: Reproducible research in computational science. 334(6060), 1226–1227 (2011). https://doi.org/10.1126/science.1213847
Savage, C.J., Vickers, A.J.: Empirical study of data sharing by authors publishing in PLoS journals. 4(9) (2009). https://doi.org/10.1371/journal.pone.0007078, https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0007078
Stocker, M., et al.: FAIR scientific information with the open research knowledge graph. 1(1), 19–21 (2023). https://doi.org/10.3233/FC-221513, https://content.iospress.com/articles/fair-connect/fc221513
Vines, T.H., et al.: The availability of research data declines rapidly with article age. 24(1), 94–97 (2014). https://doi.org/10.1016/j.cub.2013.11.014, https://www.cell.com/current-biology/abstract/S0960-9822(13)01400-0, publisher: Elsevier
Wilkinson, M.D., et al.: The FAIR guiding principles for scientific data management and stewardship. Sci. Data 3, 1–9 (2016). https://doi.org/10.1038/sdata.2016.18
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Hussein, H., Farfar, K.E., Oelen, A., Karras, O., Auer, S. (2023). Increasing Reproducibility in Science by Interlinking Semantic Artifact Descriptions in a Knowledge Graph. In: Goh, D.H., Chen, SJ., Tuarob, S. (eds) Leveraging Generative Intelligence in Digital Libraries: Towards Human-Machine Collaboration. ICADL 2023. Lecture Notes in Computer Science, vol 14458. Springer, Singapore. https://doi.org/10.1007/978-981-99-8088-8_19
Download citation
DOI: https://doi.org/10.1007/978-981-99-8088-8_19
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8087-1
Online ISBN: 978-981-99-8088-8
eBook Packages: Computer ScienceComputer Science (R0)