Abstract
Today’s software systems like cyber-physical production systems or big data systems have to process large volumes and diverse types of data which heavily influences the quality of these so-called data-intensive systems. However, traditional software testing approaches rather focus on functional behavior than on data aspects. Therefore, the role of data in testing has to be rethought, and specific testing approaches for data-intensive software systems are required. Thus, the aim of this chapter is to contribute to this area by (1) providing basic terminology and background on data-intensive software systems and their testing and (2) presenting the state of the research and the hot topics in the area. Finally, the directions of research and the new frontiers on testing data-intensive software systems are discussed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Ammann, P., & Offutt, J. (2016). Introduction to software testing. Cambridge: Cambridge University Press.
Auer, F., & Felderer, M. (2018). Current state of research on continuous experimentation: A systematic mapping study. In 2018 44th Euromicro Conference on Software Engineering and Advanced Applications (SEAA) (pp. 335–344). Piscataway, NJ: IEEE.
de Bayser, M., Azevedo, L. G., & Cerqueira, R. (2015). Researchops: The case for devops in scientific applications. In 2015 IFIP/IEEE International Symposium on Integrated Network Management (IM) (pp. 1398–1404). Piscataway, NJ: IEEE
Bourque, P., & Dupuis, R. (Eds.) (2014). Guide to the software engineering body of knowledge version 3.0 SWEBOK. Piscataway, NJ: IEEE. http://www.computer.org/web/swebok
Casale, G., Ardagna, D., Artac, M., Barbier, F., Nitto, E. D., Henry, et al.: (2015). Dice: quality-driven development of data-intensive cloud applications. In Proceedings of the Seventh International Workshop on Modeling in Software Engineering (pp. 78–83). Piscataway, NJ: IEEE Press
Corbin, J. M., & Strauss, A. (1990). Grounded theory research: Procedures, canons, and evaluative criteria (pp. 3–21). Berlin: Springer.
Dersin, P. (2014). Systems of systems. Piscataway, NJ: IEEE.
Du, X., Xie, X., Li, Y., Ma, L., Zhao, J., & Liu, Y. (2018). Deepcruiser: Automated guided testing for stateful deep learning systems. arXiv preprint arXiv:1812.05339.
Fagerholm, F., Guinea, A.S., Mäenpää, H., Münch, J. (2014). Building blocks for continuous experimentation. In Proceedings of the 1st International Workshop on Rapid Continuous Software Engineering (pp. 26–35). New York, NY: ACM.
Felderer, M., Büchler, M., Johns, M., Brucker, A. D., Breu, R., & Pretschner, A. (2016). Security testing: A survey. In: Advances in Computers (Vol. 101, pp. 1–51). Amsterdam: Elsevier.
Foidl, H., & Felderer, M. (2015). Research challenges of industry 4.0 for quality management. In International Conference on Enterprise Resource Planning Systems (pp. 121–137). Berlin: Springer.
Gadler, D., Mairegger, M., Janes, A., & Russo, B. (2017). Mining logs to model the use of a system. In 2017 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2017 (pp. 334–343). Piscataway, NJ: IEEE.
Hummel, O., Eichelberger, H., Giloj, A., Werle, D., & Schmid, K. (2018). A collection of software engineering challenges for big data system development. In: 2018 44th Euromicro Conference on Software Engineering and Advanced Applications (SEAA) (pp. 362–369). Piscataway, NJ: IEEE.
ISO. (1994). ISO 8402:1994 Quality Management and Quality Assurance—Vocabulary. Tech. rep., ISO.
ISO/IEC. (1994). Information Technology – Open Systems Interconnection – Conformance Esting Methodology and Framework. International ISO/IEC multi–part standard No. 9646.
ISO/IEC. (2008). ISO/IEC 25012:2008 Software Engineering – Software Product Quality Requirements and Evaluation (Square) – Data Quality Model. Tech. Rep., ISO.
ISO/IEC. (2011). ISO/IEC 25010:2011 Systems and Software Engineering – Systems and Software Quality Requirements and Evaluation (Square) – System and Software Quality Models. Tech. Rep., ISO.
ISTQB. (2012). Standard Glossary of Terms Used in Software Testing. Version 2.2. Tech. Rep., ISTQB.
Kim, J., Feldt, R., & Yoo, S. (2019). Guiding deep learning system testing using surprise adequacy. In: Proceedings of the 41st International Conference on Software Engineering (pp. 1039–1049). Piscataway, NJ: IEEE Press
Kleppmann, M. (2017). Designing data-intensive applications: The big ideas behind reliable, scalable, and maintainable systems. Sebastopol, CA: O’Reilly.
Mattmann, C. A., Crichton, D. J., Hart, A. F., Goodale, C., Hughes, J.S., Kelly, S., et al. (2011). Architecting data-intensive software systems. In: Handbook of data intensive computing (pp. 25–57). Berlin: Springer.
Nagy, C. (2013). Static analysis of data-intensive applications. In 2013 17th European Conference on Software Maintenance and Reengineering (pp. 435–438). Piscataway, NJ: IEEE.
Otero, C. E., & Peter, A. (2015). Research directions for engineering big data analytics software (pp. 13–19). Piscataway, NJ: IEEE.
Pei, K., Cao, Y., Yang, J., & Jana, S. (2017). Deepxplore: Automated whitebox testing of deep learning systems. In Proceedings of the 26th Symposium on Operating Systems Principles (pp. 1–18). New York, NY: ACM.
Radatz, J., Geraci, A., & Katki, F. (1990). IEEE Standard Glossary of Software Engineering Terminology. Tech. Rep., Piscataway, NJ: IEEE.
Russo, B., Succi, G., & Pedrycz, W. (2015). Mining system logs to learn error predictors: a case study of a telemetry system (pp. 879–927). Berlin: Springer.
Segura, S., Fraser, G., Sanchez, A. B., & Ruiz-Cortés, A. (2016). A survey on metamorphic testing (pp. 805–824). Piscataway, NJ: IEEE.
Shull, F. (2013). Getting an intuition for big data (pp. 3–6). Piscataway, NJ: IEEE.
Tian, Y., Pei, K., Jana, S., & Ray, B. (2018). Deeptest: Automated testing of deep-neural-network-driven autonomous cars. In: Proceedings of the 40th International Conference on Software Engineering (pp. 303–314). New York, NY: ACM.
Tonella, P. (2019). 2019–2023 ERC project: Precrime: Self-assessment Oracles for Anticipatory Testing Retrieved 2018, January 7th, 2019, from www.pre-crime.eu
Utting, M., & Legeard, B. (2007). Practical model-based testing: A tools approach. San Francisco, CA: Morgan Kaufmann Publishers.
Wang, S., Wan, J., Li, D., & Zhang, C. (2016). Implementing smart factory of industrie 4.0: An outlook (p. 3159805). London, UK: Sage Publications.
Weyuker, E. J. (1982). On testing non-testable programs (pp. 465–470). London, UK: The British Computer Society.
Xie, X., Ho, J. W., Murphy, C., Kaiser, G., Xu, B., & Chen, T. Y. (2011). Testing and validating machine learning classifiers by metamorphic testing (pp. 544–558). Amsterdam: Elsevier.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Felderer, M., Russo, B., Auer, F. (2019). On Testing Data-Intensive Software Systems. In: Biffl, S., Eckhart, M., Lüder, A., Weippl, E. (eds) Security and Quality in Cyber-Physical Systems Engineering. Springer, Cham. https://doi.org/10.1007/978-3-030-25312-7_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-25312-7_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-25311-0
Online ISBN: 978-3-030-25312-7
eBook Packages: Computer ScienceComputer Science (R0)