Big Data Analytics in IOT: Challenges, Open Research Issues and Tools

  • Fabián Constante Nicolalde
  • Fernando Silva
  • Boris Herrera
  • António Pereira
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 746)


Terabytes of data are generated day-to-day from modern information systems, cloud computing and digital technologies, as the increasing number of Internet connected devices grows. However, the analysis of these massive data requires many efforts at multiple levels for knowledge extraction and decision making. Therefore, Big Data Analytics is a current area of research and development that has become increasingly important. This article investigates cutting-edge research efforts aimed at analyzing Internet of Things (IoT) data. The basic objective of this article is to explore the potential impact of large data challenges, research efforts directed towards the analysis of IoT data and various tools associated with its analysis. As a result, this article suggests the use of platforms to explore big data in numerous stages and better understand the knowledge we can draw from the data, which opens a new horizon for researchers to develop solutions based on open research challenges and topics.


Big Data Analytics Internet of Things Hadoop Massive data Structured data Unstructured data 



This work was possible thanks to Senescyt of Ecuador for the financing of research studies at the Polytechnic Institute of Leiria, Portugal and to the FCT project UID/CEC/4524/2016.


  1. 1.
    Tavana, M., Puranam, K.: Handbook of Research on Organizational Transformations through Big Data Analytics, p. 109 (2012)Google Scholar
  2. 2.
    Marjani, M., et al.: Big IoT data analytics: architecture, opportunities, and open research challenges. IEEE Access PP(99), 1 (2017)Google Scholar
  3. 3.
    Tiainen, P.: New opportunities in electrical engineering as a result of the emergence of the Internet of Things. AaltoDoc, Aalto Univ. (2016)Google Scholar
  4. 4.
    Beyer, M.: Gartner says solving ‘Big Data’ challenge involves more than just managing volumes of data. AaltoDoc, Aalto Univ. (2011)Google Scholar
  5. 5.
    Acharjya, D.P., Ahmed, K.: A survey on Big Data analytics: challenges, open research issues and tools. Int. J. Adv. Comput. Sci. Appl. 7(2), 511–518 (2016)Google Scholar
  6. 6.
    Mital, R., Coughlin, J., Canaday, M.: Using Big Data technologies and analytics to predict sensor anomalies. In: Proceedings of the Advanced Maui Optical and Space Surveillance Technologies Conference, p. 84 (2014)Google Scholar
  7. 7.
    Golchha, N.: Big data-the information revolution. Int. J. Adv. Res. 1, 791–794 (2015)Google Scholar
  8. 8.
    Tsai, C.-W.: Big Data analytics: a survey. J. Big Data 2, 1–32 (2015)CrossRefGoogle Scholar
  9. 9.
    Russom, P.: Big Data Analytics. TDWI Best Pract. Rep., pp. 1–35 (2011)Google Scholar
  10. 10.
    LaValle, S., Lesser, E., Shockley, R., Hopkins, M.S., Kruschwitz, N.: Big Data, analytics and the path from insights to value. MIT Sloan Manag. Rev. 52, 21 (2011)Google Scholar
  11. 11.
    CollaB, O.: Big Data Definition, Open Framework, Information Management Strategy & Collaborative Governance| Data & Social Methodology - MIKE2.0 Methodology, 2015.
  12. 12.
    Gantz, J., Reinsel, D.: The digital universe in 2020: Big Data, bigger digital shadows, and biggest growth in the far east. IDC Anal. Future (2012)Google Scholar
  13. 13.
    U. S. Profile, The Digital Universe in 2020: Big Data, Bigger Digital Shadows, and Biggest Growth in the Far East — United States, pp. 1–7 (2013)Google Scholar
  14. 14.
    Atzori, L., Iera, A., Morabito, C.: The Internet of Things: a survey, pp. 2787–2805 (2010)CrossRefGoogle Scholar
  15. 15.
    Hsieh, H.-C., Lai, C.-H.: Internet of Things architecture based on integrated PLC and 3G communication networks. IEEE Access, pp. 853–856Google Scholar
  16. 16.
    Kwon, O., Lee, N., Shin, B.: Data quality management, data usage experience and acquisition intention of big data analytics. Int. J. Inf. Manag. 34, 387–394 (2014)CrossRefGoogle Scholar
  17. 17.
    Alvarado, J.C.: Estudio descriptivo de técnicas aplicadas en herramientas Open Source y comerciales para visualización de …, January 2017, 2016Google Scholar
  18. 18.
    Hashema, I.A.T., Yaqoob, I., Anuara, N.B., Mokhtara, S., Gania, A., Khanb, S.U.: The rise of ‘Big Data’ on cloud computing: review and open research issues. Inf. Syst. 47, 98–115 (2015)CrossRefGoogle Scholar
  19. 19.
    Kuo, M.-H., Sahama, T., Kushniruk, A.W., Borycki, E.M., Grunwell, D.K.: Health Big Data analytics: current perspectives, challenges and potential solutions. Int. J. Big Data Intell. 1, 114–126 (2014)CrossRefGoogle Scholar
  20. 20.
    Huang, Z.: Extensions to the k-Means algorithm for clustering large data sets with categorical values. Data Min. Knowl. Discov. 2(3), 283–304 (1998)MathSciNetCrossRefGoogle Scholar
  21. 21.
    T. A. S. Foundation., Apache Hadoop (2014).
  22. 22.
    Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. In: Proceedings of the 6th Symposium on Operating Systems Design and Implementation, pp. 137–149 (2004)Google Scholar
  23. 23.
    Kaisler, S., Armour, F., Money, W., Espinosa, J.A.: Big Data issues and challenges, vol. 5, no. 2013, pp. 2013–2015 (2015)Google Scholar
  24. 24.
    E. Consulting: The importance of scalability in big data processing.
  25. 25.
  26. 26.
    Mishra, N., Lin, C.C., Chang, H.T.: A cognitive adopted framework for IoT Big-Data management and knowledge discovery prospective. Int. J. Distrib. Sens. Networks 2015(March), 1–13 (2015)Google Scholar
  27. 27.
    Chen, X.-Y., Jin, Z.-G.: Research on key technology and applications for internet of things. Phys. Procedia 33, 561–566 (2012)CrossRefGoogle Scholar
  28. 28.
    Spark – A modern data processing framework for cross platform analytics Deploying Spark on HPE Elastic Platform for Big DataGoogle Scholar
  29. 29.
    A. S. Foundation: Apache Storm (2015)Google Scholar
  30. 30.
    T. O. Center: Introducción a Hadoop y su ecosistema.
  31. 31.
    Ingersoll, G.: Introducing apache mahout: Scalable, commercial friendly machine learning for building intelligent applications, pp. 1–18. White Paper, IBM Developer Works (2009)Google Scholar
  32. 32.
    Isard, M., Budiu, M., Yu, Y., Birrell, A., Fetterly, D.: Dryad: distributed data-parallel programs from sequential building blocks. ACM SIGOPS Oper. Syst. Rev. 41, 59–72 (2007)CrossRefGoogle Scholar
  33. 33.
    Chen, C.L.P., Zhang, C.Y.: Data-intensive applications, challenges, techniques and technologies: a survey on Big Data. Inf. Sci. 275, 314–347 (2014)Google Scholar
  34. 34.
    Kelly, J.: Apache Drill Brings SQL-Like, Ad Hoc Query Capabilities to Big Data (2013).,_Ad_Hoc_Query_Capabilities_to_Big_Data
  35. 35.
    Huang, T., Lan, L., Fang, X., An, P., Min, J., Wang, F.: Promises and challenges of big data computing in health sciences. Big Data Res. 2, 2–11 (2015)CrossRefGoogle Scholar
  36. 36.
    Castella: Introducción a Hadoop y su ecosistema (2013).
  37. 37.
    A. S. Foundation: Spark 0.8.0: This document gives a short overview of how Spark runs on clusters, to make it easier to understand the components involved. (2014).
  38. 38.
    I. d. i. d. Conocimiento: 7 Herramientas Big Data para tu empresa (2016).
  39. 39.
    Acharjya, D.P., Dehuri, S., Sanyal, S. (eds.): Computational Intelligence for Big Data Analysis (2015)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Fabián Constante Nicolalde
    • 1
    • 2
  • Fernando Silva
    • 1
  • Boris Herrera
    • 2
  • António Pereira
    • 1
    • 3
  1. 1.School of Technology and Management, Computer Science and Communications Research CentrePolytechnic Institute of LeiriaLeiriaPortugal
  2. 2.Universidad Central del EcuadorQuitoEcuador
  3. 3.Information and Communications Technologies UnitINOV INESC Innovation-Delegation Office at LeiriaLeiriaPortugal

Personalised recommendations