A Framework for Distributed Data Processing

  • José Kadir Febrer-HernándezEmail author
  • Vitali Herrera SemenetsEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11896)


Nowadays, the data generated in the telecommunications networks tend to grow exponentially leading to a Big Data challenges, which makes it necessary to discover different ways to safely process this data. The reported strategies aim to provide reliable and flexible services for asynchronous data exchange. The parallel and distributed processing of large volumes of data plays a fundamental role in scenarios that require a response as soon as possible, such as detecting fraud in telecommunications services or carrying out security controls. In this paper, we present a strategy that allows to distribute data and manage several instances of the same application, which are executed in a distributed way. An aspect to be highlighted is that heterogeneity is not required in the computational units, that is, both conventional PCs and blade clusters can participate. Another important advantages of this tool are its flexibility and its adaptability. The data are distributed depending on the workload of the different application instances. Finally, a case study is presented for the distributed processing of the Windows Operating System logs.


Distributed systems Parallel processing Data stream 


  1. 1.
    Basanta-Val, P., García-Valls, M.: A distributed real-time Java-centric architecture for industrial systems. IEEE Trans. Industr. Inf. 10(1), 27–34 (2014)CrossRefGoogle Scholar
  2. 2.
    Bruyninckx, H.: The real-time motion control core of the Orocos project. In: International Conference on Robotics and Automation (ICRA 2003), vol. 2, pp. 2766–2771 (2003)Google Scholar
  3. 3.
    Cortes, C., Pregibon, D.: Signature-based methods for data streams. Data Min. Knowl. Disc. 5(3), 167–182 (2001)CrossRefGoogle Scholar
  4. 4.
    Hapner, M., Burridge, R.: Java message service. Technical report, Sun Microsystems Inc. (2002)Google Scholar
  5. 5.
    He, W., Da Xu, L.: Integration of distributed enterprise applications: a survey. IEEE Trans. Industr. Inf. 10(1), 35–42 (2014)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Henning, M., Spruiell, M.: Distributed programming with ice. Technical report, ZeroC Inc. (2003)Google Scholar
  7. 7.
    Kambatla, K., Kollias, G., Kumar, V., Grama, A.: Trends in big data analytics. J. Parallel Distrib. Comput. 74(7), 2561–2573 (2014)CrossRefGoogle Scholar
  8. 8.
    OMG: The common object request broker (CORBA): architecture and specification. Technical report, Object Management Group (OMG) (1995)Google Scholar
  9. 9.
    OMG: Data distribution service for real-time systems. Version 1 edn. Object Management Group (OMG) (2007)Google Scholar
  10. 10.
    Valls, M.G., Val, P.B.: Comparative analysis of two different middleware approaches for reconfiguration of distributed real-time systems. J. Syst. Architect. 60(2), 221–233 (2014)CrossRefGoogle Scholar
  11. 11.
    Wu, X., Zhu, X., Wu, G.Q., Ding, W.: Data mining with big data. IEEE Trans. Knowl. Data Eng. 26(1), 97–107 (2014)CrossRefGoogle Scholar
  12. 12.
    Habeeb, R.A.A., Nasaruddin, F., Gani, A., Hashem, I.A.T., Ahmed, E., Imran, M.: Real-time big data processing for anomaly detection: a survey. Int. J. Inf. Manag. 45, 289–307 (2018)CrossRefGoogle Scholar
  13. 13.
    Gupta, S., Kar, A.K., Baabdullah, A., Al-Khowaiter, W.A.: Big data with cognitive computing: a review for the future. Int. J. Inf. Manag. 42, 78–89 (2018)CrossRefGoogle Scholar
  14. 14.
    Teixeira, F.A., Pereira, F.M., Wong, H.C., Nogueira, J.M., Oliveira, L.B.: SIoT: Securing Internet of Things through distributed systems analysis. Future Gener. Comput. Syst. 92, 1172–1186 (2019)CrossRefGoogle Scholar
  15. 15.
    Malik, M., et al.: Big vs little core for energy-efficient Hadoop computing. J. Parallel Distrib. Comput. 129, 110–124 (2019)CrossRefGoogle Scholar
  16. 16.
    Meng, X., et al.: MLlib: machine learning in apache spark. J. Mach. Learn. Res. 17(1), 1235–1241 (2016)MathSciNetzbMATHGoogle Scholar
  17. 17.
    Iqbal, M.H., Soomro, T.R.: Big data analysis: apache storm perspective. Int. J. Comput. Trends Technol. 19(1), 9–14 (2015)CrossRefGoogle Scholar
  18. 18.
    Carbone, P., Katsifodimos, A., Ewen, S., Markl, V., Haridi, S., Tzoumas, K.: Apache Flink: stream and batch processing in a single engine. Bull. IEEE Comput. Soc. Tech. Committee Data Eng. 36(4) (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Advanced Technologies Application Center (CENATAV)HavanaCuba

Personalised recommendations