Abstract
Research software is often developed by individual researchers or small teams in parallel to their research work. The more people and research projects rely on the software in question, the more important it is that software updates implement new features correctly and do not introduce regressions. Thus, developers of research software must balance their limited resources between implementing new features and thoroughly testing any code changes.
We present the processes we use for developing the distributed integration framework RCE at DLR. These processes aim to strike a balance between automation and manual testing, reducing the testing overhead while addressing issues as early as possible. We furthermore briefly describe how these testing processes integrate with the surrounding processes for development and releasing.
A. Weinert—Authors are listed in alphabetical order.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Afzal, W., Alone, S., Glocksien, K., Torkar, R.: Software test process improvement approaches: a systematic literature review and an industrial case study. J. Syst. Softw. 111, 1–33 (2016). https://doi.org/10.1016/j.jss.2015.08.048
Ammann, P., Offutt, J.: Introduction to Software Testing. Cambridge University Press (2016). https://doi.org/10.1017/9781316771273
Anand, S., et al.: An orchestrated survey of methodologies for automated software test case generation. J. Sys. Softw. 86(8), 1978–2001 (2013). https://doi.org/10.1016/j.jss.2013.02.061
Beck, K., et al.: Manifesto for agile software development (2001)
Beyer, D.: Software verification: 10th comparative evaluation (SV-COMP 2021). In: TACAS 2021, pp. 401–422 (2021). https://doi.org/10.1007/978-3-030-72013-1_24
Boden, B., Flink, J., Först, N., Mischke, R., Schaffert, K., Weinert, A., Wohlan, A., Schreiber, A.: RCE: An integration environment for engineering and science. SoftwareX 15, 100759 (2021). https://doi.org/10.1016/j.softx.2021.100759
Boden, B., Mischke, R., Weinert, A., Schreiber, A.: Supporting the composition of domain-specific software via task-specific roles. In: ICW 2020. ACM (2020). https://doi.org/10.1145/3397537.3399576
Brett, A., et al.: Research Software Engineers: State Of The Nation Report 2017 (2017). https://doi.org/10.5281/ZENODO.495360
Böhme, M., Pham, V.T., Nguyen, M.D., Roychoudhury, A.: Directed greybox fuzzing. In: CCS 2017. ACM (2017). https://doi.org/10.1145/3133956.3134020
EasyMock Developers: EasyMock. https://easymock.org/, repository at https://github.com/easymock/easymock
Flink, J., Mischke, R., Schaffert, K., Schneider, D., Weinert, A.: Orchestrating tool chains for model-based systems engineering with RCE. In: Aerospace Conference 2022. IEEE (2022)
Gurock Software GmbH. TestRail https://www.gurock.com/testrail/
Hampel, B.: Automatisierte Anwendung von Chaos Engineering Methoden zur Untersuchung der Robustheit eines verteilten Softwaresystems. Bachelor Thesis, Leipzig University of Applied Sciences (2021)
Jenkins Developers. Jenkins. https://www.jenkins.io/, repository at https://github.com/jenkinsci/jenkins
JUnit Developers. JUnit https://junit.org/, repository at https://github.com/junit-team/
Li, J., Zhao, B., Zhang, C.: Fuzzing: a survey. Cybersecurity 1(1), 1–13 (2018). https://doi.org/10.1186/s42400-018-0002-y
MantisBT Developers. MantisBT https://mantisbt.org/, repository at https://github.com/mantisbt/mantisbt
Maven Developers. Apache Maven https://maven.apache.org/, repository at https://gitbox.apache.org/repos/asf/maven-sources.git
Norman, D.A.: Human-centered design considered harmful. Interactions 12(4), 14–19 (2005). https://doi.org/10.1145/1070960.1070976
Oliveira, D., Rosenthal, M., Morin, N., Yeh, K.C., Cappos, J., Zhuang, Y.: It’s the psychology stupid: how heuristics explain software vulnerabilities and how priming can illuminate developer’s blind spots. In: ACSAC 2014. ACM (2014). https://doi.org/10.1145/2664243.2664254
Orso, A., Rothermel, G.: Software testing: a research travelogue (2000–2014). In: FOSE 2014. ACM (2014). https://doi.org/10.1145/2593882.2593885
Preston-Werner, T.: Semantic versioning 2.0. https://semver.org/
Risse, K.: Virtual Product House-Integration and Test Centre for Virtual Certification, https://www.dlr.de/content/en/articles/aeronautics/aeronautics-research/virtual-product-house.html. Accessed 4 Feb 2022
Risueño, A.P., Bussemaker, J., Ciampa, P.D., Nagel, B.: MDAx: Agile generation of collaborative MDAO workflows for complex systems. In: AIAA AVIATION 2020 FORUM. American Institute of Aeronautics and Astronautics (2020). https://doi.org/10.2514/6.2020-3133
Health, R.T.I.: Social, and Economics Research: The Economic Impacts of Inadequate Infrastructure for Software Testing. Tech. rep, NIST (2002)
SmartBear Software Inc. Cucumber. https://cucumber.io/, repository at https://github.com/cucumber/
SmartBear Software Inc. Gherkin https://cucumber.io/docs/gherkin/
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Mischke, R., Schaffert, K., Schneider, D., Weinert, A. (2022). Automated and Manual Testing in the Development of the Research Software RCE. In: Groen, D., de Mulatier, C., Paszynski, M., Krzhizhanovskaya, V.V., Dongarra, J.J., Sloot, P.M.A. (eds) Computational Science – ICCS 2022. ICCS 2022. Lecture Notes in Computer Science, vol 13353. Springer, Cham. https://doi.org/10.1007/978-3-031-08760-8_44
Download citation
DOI: https://doi.org/10.1007/978-3-031-08760-8_44
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-08759-2
Online ISBN: 978-3-031-08760-8
eBook Packages: Computer ScienceComputer Science (R0)