Abstract
Going data intensive requires much effort not only in the design, but also in system/infrastructure configuration and deployment; most of these activities still happen via heavy manual fine-tuning and often costly trial-and-error experimentation.
This book chapter introduces the field of data engineering; sets out to list the key desiderata of modern-day, data-intensive applications and AI/ML analytics software; and argues the necessity of novel methods and techniques, including MLOps. All topics will be further elaborated in the remaining chapters of this first module on data engineering.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Artac, M., Borovảąal, T., Di Nitto, E., Guerriero, M., Perez-Palacin, D., & Tamburri, D. A. (2018). Infrastructure-as-code for data-intensive architectures: A model-driven development approach. In 2018 IEEE International Conference on Software Architecture (ICSA) (pp. 156–15609).
Ciavotta, M., Krstic, S., Tamburri, D. A., & van den Heuvel, W.-J. (2019). Hyperspark: A data-intensive programming environment for parallel metaheuristics. In E. Bertino, C. K. Chang, P. Chen, E. Damiani, M. Goul, & K. Oyama (Eds.), BigData Congress (pp. 85–92). IEEE.
Desjardins, J. (2019). How much data is generated each day?
Guerriero, M., Tamburri, D. A., & Di Nitto, E. (2018). Defining, enforcing and checking privacy policies in data-intensive applications. In Proceedings of the 13th International Conference on Software Engineering for Adaptive and Self-Managing Systems, SEAMS ‘18 (pp. 172–182), New York, NY, USA. Association for Computing Machinery.
Perez-Palacin, D., Merseguer, J., Requeno, J. I., Guerriero, M., Di Nitto, E., & Tamburri, D. A. (2019). A uml profile for the design, quality assessment and deployment of data-intensive applications. Software Systems Modeling, 1–38.
Senthil Kumar, S., & Kirthika, M. V. (2017). Big data analytics architecture and challenges, issues of big data analytics. International Journal of Trend in Scientific Research and Development, 1(6), 669–673.
Sun, P., & Wen, Y. (2019). Scalable architectures for big data analysis. In S. Sakr & A. Y. Zomaya (Eds.), Encyclopedia of big data technologies. Springer.
Sun, Z., Strang, K. D., & Li, R. (2018). Big data with ten big characteristics. In ICBDR (pp. 56–61). ACM.
Susanto, H., Leu, F.-Y., Rosiyadi, D., & Kang, C. C. (2019). Revealing storage and speed transmission emerging technology of big data. In L. Barolli, M. Takizawa, F. Xhafa, & T. Enokido (Eds.), AINA (Advances in intelligent systems and computing) (Vol. 926, pp. 571–583). Springer.
Tamburri, D. A. (2020). Design principles for the general data protection regulation (GDPR): A formal concept analysis and its evaluation. Information Systems, 91, 101469.
van den Heuvel, W.-J., & Tamburri, D. A. (2020). Model-driven ml-ops for intelligent enterprise applications: Vision, approaches and challenges. In B. Shishkov (Ed.), Business modeling and software design (pp. 169–181). Springer International Publishing.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Tamburri, D., van den Heuvel, WJ. (2023). Big Data Engineering. In: Liebregts, W., van den Heuvel, WJ., van den Born, A. (eds) Data Science for Entrepreneurship. Classroom Companion: Business. Springer, Cham. https://doi.org/10.1007/978-3-031-19554-9_2
Download citation
DOI: https://doi.org/10.1007/978-3-031-19554-9_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-19553-2
Online ISBN: 978-3-031-19554-9
eBook Packages: Business and ManagementBusiness and Management (R0)