Towards an Automatized Way for Modeling Big Data System Architectures

Conference paper
Part of the Lecture Notes in Business Information Processing book series (LNBIP, volume 389)


Although the term of big data and related technologies received lots of attention in recent years, many projects are less successful than anticipated. One of the most crucial steps in the planning of a system includes the modeling of the underlying architecture. However, as of now, no standardized approach exists that facilitates the modeling of big data system architectures (BDSA). In this research, a systematic approach is presented that delivers a foundation towards a standard for the modeling of BDSA. Further, a prototype is introduced that automatizes the creation of those models reducing the required effort and simultaneously increasing the maintainability.


Big data System architecture Modeling Deployment diagram Prototype Literature review Design science research 


  1. 1.
    ISO/IEC/IEEE 42010:2011(E): Systems and software engineering—architecture description. IEEE Computer SocietyGoogle Scholar
  2. 2.
    Golden, B.: A Unified Formalism for Complex Systems Architecture (2013)Google Scholar
  3. 3.
    Chaudron, M.R.V., Heijstek, W., Nugroho, A.: How effective is UML modeling? Softw. Syst. Model. 11, 571–580 (2012)CrossRefGoogle Scholar
  4. 4.
    Desic, S., Gvozdanovic, D., Kusek, M., Huljenic, D.: Advantages of UML-based object-oriented system development. In: MIPRO Meeting (2011)Google Scholar
  5. 5.
    NIST Big Data Interoperability Framework, vol. 1, definitions, version 2. National Institute of Standards and Technology, Gaithersburg, MD (2018)Google Scholar
  6. 6.
    Volk, M., Bosse, S., Bischoff, D., Turowski, K.: Decision-support for selecting big data reference architectures. In: Abramowicz, W. (ed.) 22nd International Conference, BIS (Business Information Systems) 2019, pp. 3–17 (2019)Google Scholar
  7. 7.
    Staegemann, D., Volk, M., Nahhas, A., Abdallah, M., Turowski, K.: Exploring the specificities and challenges of testing big data systems. In: 15th International Conference on Signal Image Technology and Internet based Systems, SITIS, Italy (2019)Google Scholar
  8. 8.
    Geerdink, B.: A reference architecture for big data solutions introducing a model to perform predictive analytics using big data technology. In: ICITST (International Conference for Internet Technology and Secured Transactions), vol. 8, pp. 71–76 (2013)Google Scholar
  9. 9.
    Hruschka, S., Herrero, V., Romero, O., Abelló, A., Franch, X., Vansummeren, S., Valerio, D.: A software reference architecture for semantic-aware big data systems. Inf. Softw. Technol. 90, 75–92 (2017)CrossRefGoogle Scholar
  10. 10.
    Hevner, A.R., March, S.T., Park, J., Ram, S.: Design science in information systems research. MIS Q. 28, 75–105 (2004)CrossRefGoogle Scholar
  11. 11.
    Peffers, K., Rothenberger, M., Tuunanen, T., Vaezi, R.: Design science research evaluation. In: DESRIST (International Conference on Design Science Research in Information Systems), pp. 398–410 (2012)Google Scholar
  12. 12.
    Webster, J., Watson, R.T.: Guest editorial: analyzing the past to prepare for the future: writing a literature review. MIS Q. 26, xiii–xxiii (11 p.) (2002)Google Scholar
  13. 13.
    Levy, Y., Ellis, T.J.: A systems approach to conduct an effective literature review in support of information systems research. Inform. Sci. J. 9, 181–212 (2006)Google Scholar
  14. 14.
    Tan, C., Sun, L., Liu, K.: Big data architecture for pervasive healthcare: a literature review. In: ECIS (European Conference on Information Systems), vol. 23 (2015)Google Scholar
  15. 15.
    Gölzer, P., Cato, P., Amberg, M.: Data processing requirements of industry 4.0 - use cases for big data applications. In: ECIS (Conference: European Conference on Information Systems), vol. 23 (2015)Google Scholar
  16. 16.
    Burmeister, F., Drews, P., Schirmer, I.: Towards an extended enterprise architecture meta-model for big data - a literature-based approach. In: AMCIS (Americas Conference on Information Systems), vol. 24 (2018)Google Scholar
  17. 17.
    Goes, P.B.: Big data - analytics engine for digital transformation: where is IS? In: AMCIS (Americas Conference on Information Systems) (2015)Google Scholar
  18. 18.
    Chen, H.-M., Kazman, R., Garbajosa, J., Gonzalez, E.: Big data value engineering for business model innovation. In: HICSS (Hawaii International Conference on System Sciences), vol. 50, pp. 5921–5930 (2017)Google Scholar
  19. 19.
    Passlick, J., Lebek, B., Breitner, M.H.: A self-service supporting business intelligence and big data analytics architecture. In: Wirtschaftsinformatik 2017, pp. 1126–1140 (2017)Google Scholar
  20. 20.
    Schwarz, C., Schwarz, A., Black, W.C.: Examining the impact of multicollinearity in discovering higher-order factor models. CAIS 34(1), 62 (2014)Google Scholar
  21. 21.
    Le Dinh, T., Phan, T.-C., Bui, T.: Towards an Architecture for big data-driven knowledge management systems. In: SIGODIS (Intelligence And Intelligent Systems), pp. 1–10 (2016)Google Scholar
  22. 22.
    Alshboul, Y., Nepali, R., Wang, Y.: Big data lifecycle: threats and security model. In: SIGSEC (Information Systems Security, Assurence and Privacy) (2015)Google Scholar
  23. 23.
    Persico, V., Pescapé, A., Picariello, A., Sperlí, G.: Benchmarking big data architectures for social networks data processing using public cloud platforms. Future Gener. Comput. Syst. 89, 98–109 (2018)CrossRefGoogle Scholar
  24. 24.
    Yuan, J., Chen, M., Jiang, T., Li, T.: Complete tolerance relation based parallel filling for incomplete energy big data. Knowl.-Based Syst. 132, 215–225 (2017)CrossRefGoogle Scholar
  25. 25.
    Song, J., Guo, C., Wang, Z., Zhang, Y., Yu, G., Pierson, J.-M.: HaoLap: a Hadoop based OLAP system for big data. J. Syst. Softw. 102, 167–181 (2015)CrossRefGoogle Scholar
  26. 26.
    Hsu, H.-H., Chang, C.-Y., Hsu, C.-H. (eds.): Big Data Analytics for Sensor-Network Collected Intelligence. Academic Press, London (2017)Google Scholar
  27. 27.
    Campos, J., Sharma, P., Gabiria, U.G., Jantunen, E., Baglee, D.: A big data analytical architecture for the asset management. CIRP 64, 369–374 (2017)CrossRefGoogle Scholar
  28. 28.
    Ahmad, A., Babar, M., Din, S., Khalid, S., Ullah, M.M., Paul, A., Goutham Reddy, A., Min-Allah, N.: Socio-cyber network: The potential of cyber-physical system to define human behaviors using big data analytics. Future Gener. Comput. Syst. 92, 868–878 (2019)CrossRefGoogle Scholar
  29. 29.
    Ahmad, A., Khan, M., Paul, A., Din, S., Rathore, M.M., Jeon, G., Choi, G.S.: Toward modeling and optimization of features selection in big data based social Internet of Things. Future Gener. Comput. Syst. 82, 715–726 (2018)CrossRefGoogle Scholar
  30. 30.
    Babar, M., Rahman, A., Arif, F., Jeon, G.: Energy-harvesting based on internet of things and big data analytics for smart health monitoring. Sustain. Comput.: Inf. Syst. 20, 155–164 (2018)Google Scholar
  31. 31.
    Blazquez, D., Domenech, J.: Big data sources and methods for social and economic analyses. Technol. Forecast. Soc. Change 130, 99–113 (2018)CrossRefGoogle Scholar
  32. 32.
    Mistrík, I. (ed.): Software Architecture for Big Data and the Cloud. MK an imprint of Elsevier, Cambridge (2017)Google Scholar
  33. 33.
    Spangenberg, N., Wilke, M., Franczyk, B.: A big data architecture for intra-surgical remaining time predictions. Procedia Comput. Sci. 113, 310–317 (2017)CrossRefGoogle Scholar
  34. 34.
    Chen, H.-M., Kazman, R., Garbajosa, J., Gonzalez, E.: Toward big data value engineering for innovation. In: BIGDSE (International Workshop on Big Data Software Engineering), vol. 2, pp. 44–50 (2016)Google Scholar
  35. 35.
    Emmanuel, I., Stanier, C.: Defining big data. In: BDCA (International Conference on Big Data and Advanced Wireless Technologies), pp. 1–6 (2016)Google Scholar
  36. 36.
    Guerriero, M., Tajfar, S., Tamburri, D.A., Di Nitto, E.: Towards a model-driven design tool for big data architectures. In: BIGDSE (International Workshop on Big Data Software Engineering), vol. 2, pp. 37–43 (2016)Google Scholar
  37. 37.
    Khan, N., Alsaqer, M., Shah, H., Badsha, G., Ahmad Abbasi, A., Salehian, S.: The 10 Vs, issues and challenges of big data. In: ICBDE (International Conference on Big Data and Education), pp. 52–56 (2018)Google Scholar
  38. 38.
    Klein, J., Buglak, R., Blockow, D., Wuttke, T., Cooper, B.: A reference architecture for big data systems in the national security domain. In: BIGDSE (International Workshop on Big Data Software Engineering), vol. 2, pp. 51–57 (2016)Google Scholar
  39. 39.
    Nielsen, F.Å.: A new ANEW: evaluation of a word list for sentiment analysis in microblogs. In: MSM (Workshop on ‘Making Sense of Microposts’), pp. 47–51 (2011)Google Scholar
  40. 40.
    Ptiček, M., Vrdoljak, B.: Big data and new data warehousing approaches. In: ICCBDC (International Conference on Cloud and Big Data Computing), pp. 6–10 (2017)Google Scholar
  41. 41.
    Sebaa, A., Nouicer, A., Chikh, F., Tari, A.: Big data technologies to improve medical data warehousing. In: BDCA (international Conference on Big Data, Cloud and Applications), vol. 2, pp. 1–5 (2017)Google Scholar
  42. 42.
    Seref, B., Bostanci, E.: Opportunities, threats and future directions in big data for medical wearables. In: BDAW (International Conference on Big Data and Advanced Wireless Technologies), pp. 1–5 (2016)Google Scholar
  43. 43.
    Zafar, M.N., Azam, F., Rehman, S., Anwar, M.W.: A systematic review of big data analytics using model driven engineering. In: ICCBDC (International Conference on Cloud and Big Data Computing), pp. 1–5 (2017)Google Scholar
  44. 44.
    Sang, G.M., Xu, L., Vrieze, P.D.: A reference architecture for big data systems. In: SKIMA (International Conference on Software, Knowledge, Information Management and Applications), vol. 10 (2016)Google Scholar
  45. 45.
    Chen, H.-M., Kazman, R., Haziyev, S.: Agile big data analytics development: an architecture-centric approach. In: HICSS (Hawaii International Conference on System Sciences), vol. 49, pp. 5378–5387 (2016)Google Scholar
  46. 46.
    Darwish, T.S.J., Abu Bakar, K.: Fog based intelligent transportation big data analytics in the Internet of vehicles environment: motivations, architecture, challenges, and critical issues. IEEE Access 6, 15679–15701 (2018)CrossRefGoogle Scholar
  47. 47.
    Boci, E., Thistlethwaite, S.: A novel big data architecture in support of ADS-B data analytic. In: ICNS (Integrated Communication, Navigation and Surveillance Conference), pp. C1-1–C1-8 (2015)Google Scholar
  48. 48.
    Gohar, M., Hassan, S.A., Khan, M., Guizani, N., Ahmed, A., Rahman, A.U.: A big data analytics architecture for the Internet of small Things. IEEE Mag. 56, 128–133 (2018)CrossRefGoogle Scholar
  49. 49.
    Haroun, A., Mostefaoui, A., Dessables, F.: A big data architecture for automotive applications: PSA group deployment experience. In: CCGRID (International Symposium on Cluster, Cloud and Grid Computing), vol. 17, pp. 921–928 (2017)Google Scholar
  50. 50.
    Twardowski, B., Ryzko, D.: Multi-agent architecture for real-time Big Data processing. In: IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT), vol. 3, pp. 333–337 (2014)Google Scholar
  51. 51.
    Kiran, M., Murphy, P., Monga, I., Dugan, J., Singh Baveja, S.: Lambda architecture for cost-effective batch and speed big data processing. In: Big Data (IEEE International Conference on Big Data), pp. 2785–2792 (2015)Google Scholar
  52. 52.
    Pavlikov, R., Beisembekova, R.: Architecture and security tools in distributed information systems with Big Data. In: AICT (International Conference on Application of Information and Communication Technologies), vol. 10 (2016)Google Scholar
  53. 53.
    Koley, S., Nandy, S., Dutta, P., Dhar, S., Sur, T.: Big data architecture with mobile cloud in CDroid operating system for storing huge data. In: CAST, pp. 12–17 (2016)Google Scholar
  54. 54.
    Din, S., Ghayvat, H., Paul, A., Ahmad, A., Rathore, M.M., Shafi, I.: An architecture to analyze big data in the Internet of Things. In: ICST (International Conference on Sensing Technology), vol. 9, pp. 677–682 (2015)Google Scholar
  55. 55.
    Wang, H., Wang, Q., Liu, P., Sun, L.: Big data and intelligent agent based smart grid architecture. In: ICA (IEEE International Conference on Agents), pp. 106–107 (2017)Google Scholar
  56. 56.
    Kashlev, A., Lu, S.: A System architecture for running big data workflows in the cloud. In: IEEE International Conference on Services Computing, pp. 51–58 (2014)Google Scholar
  57. 57.
    Agrawal, R., Imran, A., Seay, C., Walker, J.: A layer based architecture for provenance in big data. In: Big Data (International Conference on Big Data), pp. 29–31 (2014)Google Scholar
  58. 58.
    Liu, D.: Big data analytics architecture for internet-of-vehicles based on the spark. In: ICITBS (International Conference on Intelligent Transportation, Big Data and Smart City), pp. 13–16 (2018)Google Scholar
  59. 59.
    Martinez-Mosquera, D., Lujan-Mora, S., Recalde, H.: Conceptual modeling of big data extract processes with UML. In: INCISCOS (International Conference on Information Systems and Computer Science), pp. 207–211 (2017)Google Scholar
  60. 60.
    Munar, A., Chiner, E., Sales, I.: A big data financial information management architecture for global banking. In: FiCloud (International Conference on Future Internet of Things and Cloud), vol. 2, pp. 385–388 (2014)Google Scholar
  61. 61.
    Costa, C., Santos, M.Y.: BASIS: a big data architecture for smart cities. In: SAI (SAI Computing Conference), pp. 1247–1256 (2016)Google Scholar
  62. 62.
    Siriweera, T.H.A.S., Paik, I., Kumara, B.T.G.S., Koswatta, K.R.C.: Intelligent big data analysis architecture based on automatic service composition. In: IEEE International Congress on Big Data, pp. 276–280 (2015)Google Scholar
  63. 63.
    Sergeevich, K.A., Ovseevna, A.M., Petrovich, S.I.: Web-application for real-time big data visualization of complex physical experiments. In: SIBCON (2015)Google Scholar
  64. 64.
    Viana, P., Sato, L.: A proposal for a reference architecture for long-term archiving, preservation, and retrieval of big data. In: International Conference on Trust, Security and Privacy in Computing and Communications), vol. 13, pp. 622–629 (2014)Google Scholar
  65. 65.
    Canito, A., Fernandes, M., Conceição, L., Praça, I., Marreiros, G.: A big data platform for industrial enterprise asset value enablers. In: DCAI (International Conference on Distributed Computing and Artificial Intelligence), vol. 15, pp. 145–154 (2018)Google Scholar
  66. 66.
    Koren, O., Binyaminov, M., Perel, N.: The impact of distributed data in big data platforms on organizations. In: FTC (Proceedings of the Future Technologies Conference), pp. 1024–1036 (2018)Google Scholar
  67. 67.
    Lu, Y., Xu, X.: Cloud-based manufacturing equipment and big data analytics to enable on-demand manufacturing services. Robot. Comput.-Integr. Manuf. 57, 92–102 (2019)CrossRefGoogle Scholar
  68. 68.
    Shakhovska, N., Duda, O., Matsiuk, O., Bolyubash, Y., Vovnyanka, R.: Analysis of the activity of territorial communities using information technology of big data based on the entity-characteristic mode. In: CSIT (International Conference on Computer Science and Information Technologies), pp. 155–170 (2018)Google Scholar
  69. 69.
    Narain Singh, K., Kumar Behera, R., Kumar Mantri, J.: Big data ecosystem: review on architectural evolution. In: IEMIS (Emerging Technologies in Data Mining and Information Security), vol. 2, pp. 335–345 (2018)Google Scholar
  70. 70.
    Singh, P.K., Verma, R.K., Krishna Prasad, P.E.S.N.: IoT-based smartbots for smart city using MCC and big data. In: SIST (Smart Intelligent Computing and Applications), pp. 525–534 (2018)Google Scholar
  71. 71.
    Woo, J., Shin, S.-J., Seo, W., Meilanitasari, P.: Developing a big data analytics platform for manufacturing systems: architecture, method, and implementation. Int. J. Adv. Manuf. Technol. 99, 2193–2217 (2018)CrossRefGoogle Scholar
  72. 72.
    Billot, R., Bothorel, C., Lenca, P.: Introduction to Big Data and Its Applications in Insurance, Chap. 1, pp. 1–25 (2018)Google Scholar
  73. 73.
    Assunção, M.D., Calheiros, R.N., Bianchi, S., Netto, M.A.S., Buyya, R.: Big Data computing and clouds: Trends and future directions. J. Parallel Distrib. Comput. 79–80, 3–15 (2015)CrossRefGoogle Scholar
  74. 74.
    Borodo, S.M., Shamsuddin, S.M., Hasan, S.: Big data platforms and techniques. IJEECS 1, 191–200 (2016)CrossRefGoogle Scholar
  75. 75.
    Chen, H.-M., Kazman, R., Haziyev, S.: Agile big data analytics for web-based systems: an architecture-centric approach. IEEE Trans. Big Data 2, 234–248 (2016)CrossRefGoogle Scholar
  76. 76.
    Chen, H.-M., Kazman, R., Haziyev, S.: Strategic prototyping for developing big data systems. IEEE Softw. 33, 36–43 (2016)CrossRefGoogle Scholar
  77. 77.
    Demchenko, Y., Ngo, C., Membrey, P.: Architecture Framework and Components for the Big Data Ecosystem (2013)Google Scholar
  78. 78.
    Pääkkönen, P., Pakkala, D.: Reference architecture and classification of technologies, products and services for big data systems. Big Data Res. 2, 166–186 (2015)CrossRefGoogle Scholar
  79. 79.
    Philip Chen, C.L., Zhang, C.-Y.: Data-intensive applications, challenges, techniques and technologies: a survey on big data. Inf. Sci. 275, 314–347 (2014)CrossRefGoogle Scholar
  80. 80.
    Ullah Rathore, M.M., Paul, A., Ahmad, A., Chen, B.-W., Huang, B., Ji, W.: Real-time big data analytical architecture for remote sensing application. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 8, 4610–4621 (2015)Google Scholar
  81. 81.
    Medvidovic, N., Taylor, R.N.: A classification and comparison framework for software architecture description languages. IIEEE Trans. Softw. Eng. 26, 70–93 (2000)CrossRefGoogle Scholar
  82. 82.
    OMG: Unified Modeling Language, v 2.5.1, pp. 1–796 (2017)Google Scholar
  83. 83.
    Bettini, L.: Implementing domain-specific languages with Xtext and Xtend. Learn how to implement a DSL with Xtext and Xtend using easy-to-understand examples and best practices. Packt Publishing (2016)Google Scholar
  84. 84.
    Volk, M., Staegemann, D., Pohl, M., Turowski, K.: Challenging big data engineering: positioning of current and future development. In: Proceedings of the 4th International, pp. 351–358 (2019)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Otto-von-Guericke-University MagdeburgMagdeburgGermany

Personalised recommendations