Ontology-Based Integrated Monitoring of Hadoop Clusters in Industrial Environments with OPC UA and RESTful Web Services

  • Kamil Folkert
  • Marcin FojcikEmail author
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 522)


Contemporary industrial and production systems produce huge amounts of data in various models, used for process monitoring, predictive maintenance of the machines, historical analysis and statistics, and more. Apache Hadoop brings a cost-effective opportunity for Big Data analysis, including the data generated in various industries. Integrating Hadoop into industrial environments creates new possibilities, as well as many challenges. The authors of this paper are involved into commercial and scientific projects utilizing Hadoop for industry as predictive analytics platform. In such initiatives the lack of standardization of monitoring of the industrial process in terms of Hadoop cluster utilization is especially perplexing. In this paper, authors propose the methodology of monitoring Hadoop in industrial environments, based on dedicated ontology and widely adopted standards: OPC Unified Architecture and RESTful Web Services.


OPC UA Big data Hadoop Process monitoring Ontology REST 



This work was supported by the European Union from the European Social Fund (grant agreement number: UDA-POKL.04.01.01-00-106/09).


  1. 1.
    Gartner. Big Data definition. Gartner IT glossary. Accessed 28 Jan 2015
  2. 2.
    Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)CrossRefGoogle Scholar
  3. 3.
    Shvachko, K., Kuang, H., Radia, S., Chansler, R.: The hadoop distributed file system. In: 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST). IEEE (2010)Google Scholar
  4. 4.
    Bakshi, K.: Considerations for big data: architecture and approach. In: Aerospace Conference, 2012. IEEE (2012)Google Scholar
  5. 5.
    White, T.: Hadoop: The Definitive Guide. O’Reilly Media Inc, Sebastopol (2012)Google Scholar
  6. 6.
    Bahga, A., Madisetti, V.K.: Analyzing massive machine maintenance data in a computing cloud. IEEE Trans. Parallel Distrib. Syst. 23(10), 1831–1843 (2012)CrossRefGoogle Scholar
  7. 7.
    Kiss, I., Genge, B., Haller, P., Sebestyen, G.: Data clustering-based anomaly detection in industrial control systems. In: 2014 IEEE International Conference on Intelligent Computer Communication and Processing (ICCP). IEEE (2014)Google Scholar
  8. 8.
    Scholten, B.: The Road to Integration: A Guide to Applying the ISA-95 Standard in Manufacturing. Isa, USA (2007)Google Scholar
  9. 9.
    EMC2-Factory. Accessed 28 Jan 2015
  10. 10.
  11. 11.
    Cupek, R., Drewniak, M., Zonenberg, D.: Online energy efficiency assessment in serial production-statistical and data mining approaches. In: 2014 IEEE 23rd International Symposium on Industrial Electronics (ISIE). IEEE (2014)Google Scholar
  12. 12.
    Cupek, R., Fojcik, M., Sande, O.: Object oriented vertical communication in distributed industrial systems. In: Kwiecień, A., Gaj, P., Stera, P. (eds.) CN 2009. CCIS, vol. 39, pp. 72–78. Springer, Heidelberg (2009) CrossRefGoogle Scholar
  13. 13.
    Haaland Thorsen, K.A., Rong, C.: Towards dataintegration from WITSML to ISO 15926. In: Sandnes, F.E., Zhang, Y., Rong, C., Yang, L.T., Ma, J. (eds.) UIC 2008. LNCS, vol. 5061, pp. 626–635. Springer, Heidelberg (2008) CrossRefGoogle Scholar
  14. 14.
    King, R.L.: Information services for smart grids. In: Power and Energy Society General Meeting-Conversion and Delivery of Electrical Energy in the 21st Century, pp. 1–5. IEEE (2008)Google Scholar
  15. 15.
    Van Deursen, D., Poppe, C., Martens, G., Mannens, E., Walle, R.: XML to RDF conversion: a generic approach. In: 2008 International Conference on Automated Solutions for Cross Media Content and Multi-channel Distribution, AXMEDIS 2008. IEEE (2008)Google Scholar
  16. 16.
    Stopper, M., Katalinic, B.: Service-oriented architecture design aspects of OPC UA for industrial applications. In: Proceedings of the International Multi-Conference of Engineers and Computer Scientists (2009)Google Scholar
  17. 17.
    Rohjans, S., Fensel, D., Fensel, A.: OPC UA goes semantics: Integrated communications in smart grids. In: 2011 IEEE 16th Conference on Emerging Technologies and Factory Automation (ETFA), pp. 1–4 (2011)Google Scholar
  18. 18.
    Thorsen, K.A.H., Torbjørnse, O.F., Rong, C.: Automatic web service detection in oil and gas. In: Ślęzak, D., Kim, T., Chang, A.C.-C., Vasilakos, T., Li, M.C., Sakurai, K. (eds.) FGCN/ACN 2009. CCIS, vol. 56, pp. 193–200. Springer, Heidelberg (2009) CrossRefGoogle Scholar
  19. 19.
    Hortonworks Data Platform. Accessed 29 Jan 2015
  20. 20.
    Richardson, L., Ruby, S.: RESTful Web Services. O’Reilly Media Inc, USA (2008) Google Scholar
  21. 21.
    Clavel, M., Durán, F., Eker, S., Lincoln, P., Martí-Oliet, N., Meseguer, J., Talcott, C.: Introduction to OPC UA performance. In: Clavel, M., Durán, F., Eker, S., Lincoln, P., Martí-Oliet, N., Meseguer, J., Talcott, C. (eds.) CN 2012, CCIS 291. LNCS, vol. 4350, pp. 1–28. Springer, Heidelberg (2007) CrossRefGoogle Scholar
  22. 22.
    Folkert, K., Fojcik, M., Cupek, R.: Efficiency of OPC UA communication in java-based implementations. In: Kwiecień, A., Gaj, P., Stera, P. (eds.) CN 2011. CCIS, vol. 160, pp. 348–357. Springer, Heidelberg (2011) CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Institute of InformaticsSilesian University of TechnologyGliwicePoland
  2. 2.Sogn Og Fjordane University CollegeFørdeNorway

Personalised recommendations