Advertisement

Smart Grids Data Management: A Case for Cassandra

  • Gil PinheiroEmail author
  • Eugénia Vinagre
  • Isabel Praça
  • Zita Vale
  • Carlos Ramos
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 620)

Abstract

The objective of this paper is to present a SMACK based platform for microgrids data storage and management. The platform is being used in a real microgrid, with an infrastructure that monitors and controls 3 buildings within the GECAD - ISEP/IPP campus, while, at the same time, receives and manages data sources coming from different types of buildings from associated partners, to whom intelligent services are being provided. Microgrid data comes in different formats, different rates and with an increasing volume, as the microgrid itself covers more customers and areas. Based on the atual available computational resources, a Big Data platform based on the SMACK stack was implemented and is presented. The Cassandra component of the stack has evolved. AC version 2 is still supported until the version 4 release, and is often still used in production environments. However, a new stable version, version 3, introduces major optimizations in the storage that bring disk space savings. The main focus of this work is on the Data Storage and the formalization of the data mapping in Cassandra version 3, which is contextualized by means of a short example with data coming from the monitoring infrastructure of the microgrid.

Keywords

Big Data Storage Smart Grids Cassandra 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1. Skopik, F., Ma, Z.: Attack Vectors to Metering Data in Smart Grids under Security Constraints. In: 2012 IEEE 36th Annual Computer Software and Applications Conference Workshops. pp. 134–139. IEEE (2012).Google Scholar
  2. 2. Chandarana, P., Vijayalakshmi, M.: Big Data analytics frameworks. In: 2014 International Conference on Circuits, Systems, Communication and Information Technology Applications (CSCITA). pp. 430–434. IEEE (2014).Google Scholar
  3. 3. Shvachko, K., Kuang, H., Radia, S., Chansler, R.: The Hadoop Distributed File System. In: 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST). pp. 1–10. IEEE (2010).Google Scholar
  4. 4. Zaharia, M., Franklin, M.J., Ghodsi, A., Gonzalez, J., Shenker, S., Stoica, I., Xin, R.S., Wendell, P., Das, T., Armbrust, M., Dave, A., Meng, X., Rosen, J., Venkataraman, S.: Apache Spark: A Unified Engine for Big Data Processing. Commun. ACM. 59, 56–65 (2016).Google Scholar
  5. 5. Felix Covrig, C., Ardelean, M., Vasiljevska, J., Mengolini, A., Fulli, G., Amoiralis, E., Sánchez Jiménez, M., Filiou, C., European Commission. Joint Research Centre. Institute for Energy and Transport.: Smart grid projects outlook 2014. Publications Office (2014).Google Scholar
  6. 6. Vinagre, E., Gomes, L., Vale, Z.: Electrical energy consumption forecast using external facility data. Proc. - 2015 IEEE Symp. Ser. Comput. Intell. SSCI 2015. 659–664 (2016).Google Scholar
  7. 7. Gomes, L., Lefrancois, M., Faria, P., Vale, Z.: Publishing real-time microgrid consumption data on the web of Linked Data. In: 2016 Clemson University Power Systems Conference (PSC). pp. 1–8. IEEE (2016).Google Scholar
  8. 8. Estrada, R., Ruiz, I.: Big Data SMACK. Apress, Berkeley, CA (2016).Google Scholar
  9. 9. Mayilvaganan, M., Sabitha, M.: A cloud-based architecture for Big-Data analytics in smart grid: A proposal. In: 2013 IEEE International Conference on Computational Intelligence and Computing Research. pp. 1–4. IEEE (2013).Google Scholar
  10. 10. Zhou, D., Guo, J., Zhang, Y., Chai, J., Liu, H., Liu, Y., Huang, C., Gui, X., Liu, Y.: Distributed Data Analytics Platform for Wide-Area Synchrophasor Measurement Systems. IEEE Trans. Smart Grid. 1–9 (2016).Google Scholar
  11. 11. Chebotko, A., Kashlev, A., Lu, S.: A Big Data Modeling Methodology for Apache Cassandra. In: 2015 IEEE International Congress on Big Data. pp. 238–245. IEEE (2015).Google Scholar
  12. 12. Datastax: Estimating partition size, https://docs.datastax.com/en/landing_page/doc/landing_page/planning/planningPartitionSize.html. Accessed 23 Nov 2016
  13. 13. Datastax Academy: Physical: Partition Size, https://academy.datastax.com/courses/ds220-data-modeling/physical-partition-size. Accessed 20 Nov 2016
  14. 14. Lebresne, S.: Putting some structure in the storage engine, http://www.datastax.com/2015/12/storage-engine-30. Accessed 4 Dec 2016
  15. 15. Morton, A.: Introduction To The Apache Cassandra 3.x Storage Engine, http://thelastpickle.com/blog/2016/03/04/introductiont-to-the-apache-cassandra-3-storage-engine.html. Accessed 4 Dec 2016

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Gil Pinheiro
    • 1
    Email author
  • Eugénia Vinagre
    • 1
  • Isabel Praça
    • 1
  • Zita Vale
    • 1
  • Carlos Ramos
    • 1
  1. 1.GECAD – Research Group on Intelligent Engineering and Computing for Advanced Innovation and Development, Institute of EngineeringPolytechnic of Porto (ISEP/IPP)PortoPortugal

Personalised recommendations