Open Science in Software Engineering
- 712 Downloads
Open science describes the movement of making any research artifact available to the public and includes, but is not limited to, open access, open data, and open source. While open science is becoming generally accepted as a norm in other scientific disciplines, in software engineering, we are still struggling in adapting open science to the particularities of our discipline, rendering progress in our scientific community cumbersome. In this chapter, we reflect upon the essentials in open science for software engineering including what open science is, why we should engage in it, and how we should do it. We particularly draw from our experiences made as conference chairs implementing open science initiatives and as researchers actively engaging in open science to critically discuss challenges and pitfalls and to address more advanced topics such as how and under which conditions to share preprints, what infrastructure and licence model to cover, or how do it within the limitations of different reviewing models, such as double-blind reviewing. Our hope is to help establishing a common ground and to contribute to make open science a norm also in software engineering.
We want to thank all the members of the empirical software engineering research community who are actively supporting the open science movement and its adoption to the software engineering community. Just to name a few: Robert Feldt and Tom Zimmermann, editors in chief of the Empirical Software Engineering Journal, are committed to support the implementation of a new Reproducibility and Open Science initiative1—the first one to implement an open data initiative following a holistic process including a badge system. The steering committee of the International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE) supported the implementation of an open science initiative from 2016 on. Markku Oivo, general chair of the International Symposium on Empirical Software Engineering and Measurement (ESEM) 2018, has actively supported the adoption of the CHASE open science initiative with focus on data sharing for the major Empirical Software Engineering conference so that we could pave the road for a long-term change in that community. Sebastian Uchitel, general chair of the International Software Engineering Conference (ICSE) 2017, further supported an initiative to foster sharing of preprints, and Natalia Juristo, general chair of ICSE 2021, further actively supports the adoption of the broader ESEM open science initiative to our major general software engineering conference. Finally, we want to thank Per Runeson, Klaas-Jan Stol, and Breno de França for their elaborate comments on earlier versions on this manuscript.
- Arxiv (2019a) arxiv license information. https://arxiv.org/help/license. Archived: http://web.archive.org/web/20190410151011/https://arxiv.org/help/license. Accessed 10 Apr 2019
- Arxiv (2019b) arxiv license information. https://arXiv.org/licenses/nonexclusive-distrib/1.0/license.html. Archived: http://web.archive.org/web/20190410165523/https://arxiv.org/licenses/nonexclusive-distrib/1.0/license.html. Accessed 10 Apr 2019
- Auer S, Bizer C, Kobilarov G, Lehmann J, Cyganiak R, Ives Z (2007) DBpedia: a nucleus for a web of open data. Springer, Berlin, pp 722–735Google Scholar
- BOAI (2002) Budapest open access initiative. https://www.budapestopenaccessinitiative.org/read
- Bolam JP, Foxe JJ (2017) Transparent review at the European journal of neuroscience: experiences one year on. Eur J Neurosci 46(11):2647–2647. https://onlinelibrary.wiley.com/doi/abs/10.1111/ejn.13762 CrossRefGoogle Scholar
- Childs S, McLeod J, Lomas E, Cook G (2014) Opening research data: issues and opportunities. Rec Manag J 24(2):142–162Google Scholar
- FOSTER (2019) Open science taxonomy. https://www.fosteropenscience.eu/taxonomy/term/7
- Ginsparg P (2011) It was twenty years ago today… Preprint. arXiv:1108.2700Google Scholar
- Gómez O, Juristo N, Vegas S (2012) Replication types in experimental disciplines. In: Proceedings of the 2010 ACM-IEEE international symposium on empirical software engineering and measurement, pp 1–10Google Scholar
- Graziotin D (2019) How to disclose data for double-blind review and make it archived open data upon acceptance. https://ineed.coffee/5205/. Archived: https://web.archive.org/web/20190410141340/https://ineed.coffee/5205/. Accessed 10 Apr 2019
- Koehler W (2003) A longitudinal study of web pages continued: a consideration of document persistence. Inf Res 9(2). http://www.informationr.net/ir/9-2/paper174.html
- Lambert C (2006) The marketplace of perceptions. Harv Mag 108(4):50Google Scholar
- Nagappan M, Robbes R, Kamei Y, Tanter É, McIntosh S, Mockus A, Hassan A (2015) An empirical study of goto in C code from GitHub repositories. In: Proceedings of the 2015 10th joint meeting on foundations of software engineering. ACM, New YorkGoogle Scholar
- O’Connor R (2011) The ACM and me. http://r6.ca/blog/20110930T012533Z.html. Archived: http://web.archive.org/web/20190410153103/http://r6.ca/blog/20110930T012533Z.html. Accessed 10 Apr 2019
- Ross-Hellauer T (2017) What is open peer review? A systematic review [version 2; peer review: 4 approved]. F1000Research 6:588. https://doi.org/10.12688/f1000research.11369.2
- Schimmer R, Geschuhn KK, Vogler A (2015) Disrupting the subscription journals’ business model for the necessary large-scale transformation to open access. http://pure.mpg.de/pubman/item/escidoc:2148961
- Stallman RM, McGrath R, Smith P (2001) GNU make, CiteseerGoogle Scholar
- Tennant JP, Dugan JM, Graziotin D, Jacques DC, Waldner F, Mietchen D, Elkhatib Y, Collister LB, Pikas CK, Crick T, Masuzzo P, Caravaggi A, Berg DR, Niemeyer KE, Ross-Hellauer T, Mannheimer S, Rigling L, Katz DS, Tzovaras BG, Pacheco-Mendoza J, Fatima N, Poblet M, Isaakidis M, Irawan DE, Renaut S, Madan CR, Matthias L, Kjær JN, O’Donnell DP, Neylon C, Kearns S, Selvaraju M, Colomb J (2017) A multi-disciplinary perspective on emergent and future innovations in peer review [version 3; peer review: 2 approved]. F1000Research 6:1151. https://doi.org/10.12688/f1000research.12037.3
- Tennant J, Beamer JE, Bosman J, Brembs B, Chung NC, Clement G, Crick T, Dugan J, Dunning A, Eccles D et al (2019) Foundations for open scholarship strategy development. https://osf.io/preprints/metaarxiv/b4v8p
- Ushey K, McPherson J, Cheng J, Atkins A, Allaire J (2018) packrat: a dependency management system for projects and their R package dependencies. R package version 0.5.0. https://CRAN.R-project.org/package=packrat
- Van den Eynden V, Corti L, Woollard M, Bishop L, Horton L (2011) Managing and sharing data; a best practice guide for researchers. Retrieved from the University of Essex Data Archive: http://repository.essex.ac.uk/2156/1/managingsharing.pdf. Accessed 31 Mar 2020Google Scholar
- van Deursen A (2016) Green open access FAQ. https://avandeursen.com/2016/11/06/green-open-access-faq/. Archived: https://web.archive.org/web/20190410141222/https://avandeursen.com/2016/11/06/green-open-access-faq/. Accessed 10 Apr 2019
- Wikimedia (2013) Consequences, risks and side-effects of the license module “non-commercial use only”. OpenGLAM. https://openglam.org/2013/01/08/consequences-risks-and-side-effects-of-the-license-module-non-commercial-use-only/
- Woelfle M, Olliaro P, Todd MH (2011) Open science is a research accelerator. Nat Chem 3:745 EPGoogle Scholar
- Xie Y (2015) Dynamic documents with R and knitr, 2nd edn. Chapman and Hall/CRC, Boca Raton. ISBN 978-1498716963. https://yihui.name/knitr/
- Xie Y, Allaire J, Grolemund G (2018) R Markdown: the definitive guide. Chapman and Hall/CRC, Boca Raton. ISBN 9781138359338. https://bookdown.org/yihui/rmarkdown
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.