Abstract
We introduce the BDA dynamics and explore the associated applications in renewable energy sector with a focus on data-driven innovation. Our study draws on the exponential growth of renewable energy initiatives over the last decades and on the paucity of literature to illustrate the use of BDA in the energy industry. We conduct a qualitative field study in the UK with stakeholder interviews and analyse our results using thematic analysis. Our findings indicate that no matter if the importance of the energy sector for ‘people’s well-being, industrial competitiveness, and societal advancement, old fashioned approaches to analytics for organisational processes are currently applied widely within the energy sector. These are triggered by resistance to change and insufficient organisational knowledge about BDA, hindering innovation opportunities. Furthermore, for energy organisations to integrate BDA approaches, they need to deal with challenges such as training employees on BDA and the associated costs. Overall, our study provides insights from practitioners about adopting BDA innovations in the renewable energy sector to inform decision-makers and provide recommendations for future research.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aggarwal, N., Nicasio, A., DeSilva, R., Boiler, M., & Lewis-Fernández, R. (2013). Barriers to implementing the dsm-5 cultural formulation interview: A qualitative study. Culture, Medicine, and Psychiatry, 37(3), 505–533. https://doi.org/10.1007/s11013-013-9325-z
Agrawal, R., & Srikant, R. (2000). Privacy-preserving data mining. ACM SIGMOD Record, 29(2), 439–450. https://www.researchgate.net/publication/262235629_Privacy-preserving_data_mining [Accessed 13 Apr. 2021].
Akter, S., Bandara, R., Hani, U., Fosso Wamba, S., Foropon, C., & Papadopoulos, T. (2019). Analytics-based decision-making for service systems: A qualitative study and agenda for future research. International Journal of Information Management. https://doi.org/10.1016/j.ijinfomgt.2019.01.020
Akter, S., Michael, K., Uddin, M. R., McCarthy, G., & Rahman, M. (2020). Transforming business using digital innovations: The application of AI, blockchain, cloud and data analytics. Annals of Operations Research. https://doi.org/10.1007/s10479-020-03620-w
Alahakoon, D., & Yu, X. (2016). Smart electricity meter data intelligence for future energy systems: A survey. IEEE Transactions on Industrial Informatics, 12(1), 425–436. https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=7063262&tag=1. Accessed 14 Dec 2020.
Altin, M., Goksu, O., Teodorescu, R., Rodriguez, P., Jensen, B., & Helle, L. (2010). Overview of recent grid codes for wind power integration. In 2010 12th international conference on optimization of electrical and electronic equipment (pp.1–5). https://upcommons.upc.edu/handle/2117/11465 Accessed 13 Apr 2021.
Awudu, I., Wilson, W. W., Fathi, M., Bachkar, K., Dahl, B., & Acquaye, A. (2020). Application of big data copula-based clustering for hedging in renewable energy systems. International Journal of Revenue Management, 11, 237–263.
Babbie, E. (2013). The basics of social research. Cengage learning (6th edn. pp. 280–294). Belmont US.
Baig, M. I., Shuib, L., & Yadegaridehkordi, E. (2019). Big data adoption: State of the art and research challenges. Information Processing and Management, 56, 102095.
Balac, N., Sipes, T., Wolter, N., Nunes, K., Sinkovits, B. & Karimabadi, H. (2013). Large scale predictive analytics for real-time energy management. In 2013 IEEE international conference on big data. https://ieeexplore.ieee.org/abstract/document/6691635 [Accessed 13 Apr. 2021].
Batistič, S., & van der Laken, P. (2019). History, evolution and future of big data and analytics: A bibliometric analysis of its relationship to performance in organisations. British Journal of Management, 30(2), 229–251.
Bazeley, P., & Jackson, K. (2013). Qualitative data analysis with NVivo (2nd ed., pp. 1–23). SAGE.
Bell, E., Bryman, A., & Harley, B. (2019). Business research methods (4th ed., pp. 353–530). Oxford University Press.
Bello-Orgaz, G., Jung, J. & Camacho, D. (2016). Social big data: Recent achievements and new challenges. Information Fusion, 28, 45–59. https://www.sciencedirect.com/science/article/pii/S1566253515000780 [Accessed 13 Apr. 2021].
Bibri, S. E. (2018). Data science for urban sustainability: Data mining and data-analytic thinking in the next wave of city analytics. Urban Book Series. https://doi.org/10.1007/978-3-319-73981-6_4
Bibri, S. E., & Krogstie, J. (2020). The emerging data–driven Smart City and its innovative applied solutions for sustainability: The cases of London and Barcelona. Energy Informatics. https://doi.org/10.1186/s42162-020-00108-6
Bose, R. (2009). Advanced analytics: Opportunities and challenges. Industrial Management and Data Systems, 109(2), 155–172. https://pdfs.semanticscholar.org/7c91/c58581fa17a2da4dcf1e8bd281854cc35527.pdf [Accessed 13 Apr. 2021].
Boyatzis, R. (2009). Transforming qualitative information (3rd ed., pp. 1–54). Sage Publications.
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa
Causone, F., Carlucci, S., Ferrando, M., Marchenko, A., & Erba, S. (2019). A data-driven procedure to model occupancy and occupant-related electric load profiles in residential buildings for energy simulation. Energy and Buildings. https://doi.org/10.1016/j.enbuild.2019.109342
Ceci, M., Cassavia, N., Corizzo, R., Dicosta, P., Malerba, D., Maria, G., Masciari, E. & Pastura, C. (2014). Innovative power operating center management exploiting big data techniques. In: Proceedings of the 18th International Database Engineering and Applications Symposium on—IDEAS '14 (pp.1–6). https://www.researchgate.net/publication/286142328_Big_data_techniques_for_renewable_energy_market [Accessed 13 Apr. 2021].
Chen, H., Chiang, R. & Storey, V. (2012). Business intelligence and analytics: From big data to big impact, 36(4), 1165–1188. https://pdfs.semanticscholar.org/f5fe/b79e04b2e7b61d17a6df79a44faf358e60cd.pdf [Accessed 13 Apr. 2021].
Chen, M., Mao, S., Zhang, Y. & Leung, V. (2014b). Big data. 1st edn. Springer (pp.1–20). https://www.springer.com/gp/book/9783319062440 [Accessed 13 Apr. 2021].
Chen, P. & Zhang, C. (2014). Data-intensive applications, challenges, techniques and technologies: A survey on Big Data. Information Sciences, 275, 314–347. https://www.sciencedirect.com/science/article/pii/S0020025514000346 [Accessed 13 Apr. 2021].
Chen, M., Mao, S., & Liu, Y. (2014a). Big data: A survey. Mobile Networks and Applications, 19(2), 171–209. https://doi.org/10.1007/s11036-013-0489-0#citeas
De Coninck, N. (2017). The relationship between big data analytics and operations research. Universiteit Gent. https://libstore.ugent.be/fulltxt/RUG01/002/351/191/RUG01-002351191_2017_0001_AC.pdf Accessed 14 Dec 2020.
Davenport, T. (2014). Big data work: Dispelling the myths, uncovering the opportunities (1st ed., pp. 1–20). Harvard Business Pr.
Davis, F. (1985). A technology acceptance model for empirically testing new end-user information systems: Theory and results. Ph.D. Massachusetts Institute of Technology, Sloan School of Management.
Denzin, N., & Lincoln, Y. (2013). The landscape of qualitative research (4th ed., pp. 7–11). Sage.
Diamantoulakis, P., Kapinas, V. & Karagiannidis, G. (2015). Big data analytics for dynamic energy management in smart grids. Big Data Research, 2(3), 94–101. https://www.sciencedirect.com/science/article/pii/S2214579615000283. Accessed 14 Dec 2020.
Dremel, C., Herterich, M. M., Wulf, J., & Vom Brocke, J. (2020). Actualising big data analytics affordances: A revelatory case study. Information and Management, 57(1), 103121.
Dubey, R., Gunasekaran, A., Childe, S. J., Papadopoulos, T., Luo, Z., Wamba, S. F., & Roubaud, D. (2019). Can big data and predictive analytics improve social and environmental sustainability? Technological Forecasting and Social Change, 144, 534–545.
Edwards, R. & Holland, J. (2013). What is qualitative interviewing? 2nd edn. London: Bloomsbury, pp.2,4,20,21.
Ericsson, (2014). Horizon scan: ICT and the future of utilities. Smart Cities. Ericsson, pp.1–44. https://www.ericsson.com/assets/local/news/2014/12/ict-and-the-future-of-utilities.pdf [Accessed 13 Apr. 2021].
Escobedo, G., Jacome, N. & Arroyo-Figueroa, G. (2017). Big data and analytics to support the renewable energy integration of smart grids—case study: Power solar generation. In Proceedings of the 2nd international conference on internet of things, big data and security (pp.2–5). https://www.researchgate.net/publication/317299122_Big_Data_Analytics_to_Support_the_Renewable_Energy_Integration_of_Smart_Grids_Case_Study_Power_Solar_Generation [Accessed 13 Apr. 2021].
Fan, J., Han, F. & Liu, H. (2014). Challenges of big data analysis. National Science Review, 1(2), 293–314. https://academic.oup.com/nsr/article/1/2/293/1397586 [Accessed 13 Apr. 2021].
Finlay, S. (2014). Predictive analytics, data mining and big data. 1st ed. Basingstoke: Palgrave Macmillan, pp. 39–49,65–78.
Gandomi, A. & Haider, M. (2015). Beyond the hype: Big data concepts, methods, and analytics. International Journal of Information Management, 35(2), 137–144. https://www.sciencedirect.com/science/article/pii/S0268401214001066. Accessed 14 Dec 2020.
Gillham, B. (2000). The research interview (1st ed., pp. 21–26). Continuum.
GOV (2019). Digest of United Kingdom energy statistics 2019. Renewable sources of energy (pp.1–15). London: GOV. https://www.gov.uk/government/statistics/renewable-sources-of-energy-chapter-6-digest-of-united-kingdom-energy-statistics-dukes [Accessed 13 Apr. 2021].
Grant, C. & Osanloo, A. (2014). Understanding, selecting, and integrating a theoretical framework in dissertation research: Creating the blueprint for your "House". Administrative Issues Journal Education Practice and Research, pp.1–5. https://files.eric.ed.gov/fulltext/EJ1058505.pdf [Accessed 13 Apr. 2021].
Gunasekaran, A., Papadopoulos, T., Dubey, R., Fosso-Wamba, S., Childe, S., Hazen, B., & Akhter, S. (2017). Big data and predictive analytics for supply chain and organisational performance. Journal of Business Research, 70, 308–317.
Günther, W. A., Mehrizi, M. H. R., Huysman, M., & Feldberg, F. (2017). Debating big data: A literature review on realising value from big data. The Journal of Strategic Information Systems, 26(3), 191–209.
Gupta, M., & George, J. (2016). Toward the development of a big data analytics capability. Information and Management, 53(8), 1049–1064.
Gupta, S., Chen, H., Hazen, B. T., Kaur, S., & Gonzalez, E. D. S. (2019). Circular economy and big data analytics: A stakeholder perspective. Technological Forecasting and Social Change, 144, 466–474.
Halper, F. (2014). Predictive analytics for business advantage. TDWI Best Practices Report. TDWI, pp.1–10. https://vods.dm.ux.sap.com/previewhub/ITAnalyticsContentHubANZ/downloadasset.2014-03-mar-17-21.predictive-analytics-for-business-advantage-pdf.pdf [Accessed 13 Apr. 2021].
Hazen, B. T., Skipper, J. B., Boone, C. A., & Hill, R. R. (2018). Back in business: Operations research in support of big data analytics for operations and supply chain management. Annals of Operations Research, 270(1), 201–211.
Hu, J. & Vasilakos, A. (2016). Energy big data analytics and security: Challenges and opportunities. IEEE Transactions on Smart Grid, 7(5), 2423–2436. https://ieeexplore.ieee.org/abstract/document/7466849 [Accessed 13 Apr. 2021].
Huberman, M. (1990). Linkage between researchers and practitioners: A qualitative study. American Educational Research Journal, 27(2), 363–391. https://doi.org/10.3102/00028312027002363
IDC, (2018). Data age 2025: The digitisation of the world from edge to core. Seagate. https://www.seagate.com/files/www-content/our-story/trends/files/idc-seagate-dataage-whitepaper.pdf. Accessed 14 Dec 2020.
Jeble, S., Dubey, R., Childe, S. J., Papadopoulos, T., Roubaud, D., & Prakash, A. (2018). Impact of big data and predictive analytics capability on supply chain sustainability. The International Journal of Logistics Management. https://doi.org/10.1108/IJLM-05-2017-0134
Jeffery, S., Alonso, G., Franklin, M., Wei H. & Widom, J. (2006). A pipelined framework for online cleaning of sensor data streams. In 22nd International Conference on Data Engineering (ICDE'06). https://ieeexplore.ieee.org/document/1617508 [Accessed 13 Apr. 2021].
Jetzek, T., Avital, M., & Bjorn-Andersen, N. (2014). Data-driven innovation through open government data. Journal of Theoretical and Applied Electronic Commerce Research. https://doi.org/10.4067/S0718-18762014000200008
Karafili, E., Spanaki, K., & Lupu, E. (2018). An argumentation reasoning approach for data processing. Computers in Industry, 94, 52–61.
Khan, S., Subbarao, G. & Reddy, V. (2016). Hace theorem based data mining using big data. Research Inventy: International Journal of Engineering and Science, 6(5), 1–5. http://www.researchinventy.com/papers/v6i5/N0605083087.pdf [Accessed 13 Apr. 2021].
Khanra, S., Dhir, A., & Mäntymäki, M. (2020). Big data analytics and enterprises: A bibliometric synthesis of the literature. Enterprise Information Systems, 14(6), 737–768.
King, N. (2014). Using interviews in qualitative research. In C. Cassel & G. Symon (Eds.), Essential guide to qualitative methods in organisational research (pp. 11–20). Sage.
Kotu, V., & Deshpande, B. (2014). Predictive analytics and data mining (1st ed., pp. 1–15). Morgan Kaufmann.
Kristoffersen, E., Blomsma, F., Mikalef, P., & Li, J. (2020). The smart circular economy: A digital-enabled circular strategies framework for manufacturing companies. Journal of Business Research, 120, 241–261.
Kusiak, A. (2009). Innovation: A data-driven approach. International Journal of Production Economics. https://doi.org/10.1016/j.ijpe.2009.06.025
Kwon, O., Lee, N. & Shin, B. (2014). Data quality management, data usage experience and acquisition intention of big data analytics. International Journal of Information Management, 34(3), 387–394. https://www.sciencedirect.com/science/article/pii/S0268401214000127. Accessed 14 Dec 2020.
LaValle, S., Lesser, E., Shockley, R., Hopkins, M.S. & Kruschwitz, N. (2011). Big data, analytics and the path from insights to value. MIT Sloan Management Review, 52(2), 21–31. https://tarjomefa.com/wp-content/uploads/2017/08/7446-English-TarjomeFa.pdf. Accessed 14 Dec 2020.
Lee, H. L. (2018). Big data and the innovation cycle. Production and Operations Management. https://doi.org/10.1111/poms.12845
Malladi, S. (2013). Adoption of Business Intelligence & Analytics in Organisations: An Empirical Study of Antecedents. In AMCIS—Proceedings of the 19th Americas Conference on Information Systems. https://pdfs.semanticscholar.org/2772/919ae1a0bc57d26f9f082fed32e408a2aaae.pdf. Accessed 14 Dec 2020.
Manyika, J., Chui, M., Brown, B., Bughin, J., Dobbs, R., Roxburgh, C., & Hung Byers, A. (2011). Big data: The next frontier for innovation, competition, and productivity. McKinsey Global Institute.
Marvasti, A., Holstein, J., & Gubrium, J. (2012). The SAGE handbook of interview research (2nd ed., pp. 347–360). Sage Publications.
Merriam, S. (1998). Qualitative research and case study applications (2nd ed., pp. 27–43). Jossey-Bass.
Merriam, S. (2002). Qualitative research in practice (1st ed., pp. 1–10). Jossey-Bass.
Mikalef, P., Boura, M., Lekakos, G., & Krogstie, J. (2019). Big data analytics and firm performance: Findings from a mixed-method approach. Journal of Business Research, 98, 261–276.
Mikalef, P., van de Wetering, R., & Krogstie, J. (2021). Building dynamic capabilities by leveraging big data analytics: The role of organizational inertia. Information and Management, 58(6), 103412.
Miles, M. & Huberman, A. (1994). Qualitative data analysis. 2nd ed. Thousand Oaks: Sage, pp.1–10, 288–295.
Mortenson, M., Doherty, N. & Robinson, S. (2015). Operational research from Taylorism to Terabytes: A research agenda for the analytics age. European Journal of Operational Research, 241(3), 583–595. https://www.sciencedirect.com/science/article/pii/S037722171400664X. Accessed 14 Dec 2020.
Oussous, A., Benjelloun, F., Ait Lahcen, A. & Belfkih, S. (2018). Big data technologies: A survey. Journal of King Saud University: Computer and Information Sciences, 30(4), 431–448. https://www.sciencedirect.com/science/article/pii/S1319157817300034?via%3Dihub#b0500 [Accessed 13 Apr. 2021].
Palinkas, L., Horwitz, S., Green, C., Wisdom, J., Duan, N., & Hoagwood, K. (2013). Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Administration and Policy in Mental Health and Mental Health Services Research, 42(5), 533–544. https://doi.org/10.1007/s10488-013-0528-y#citeas
Papadopoulos, T., Sing, S. P., Spanaki, K., Gunasekaran, A., & Dubey, R. (2021). Towards the next generation of manufacturing: Implications of big data and digitalization in the context of industry 4.0. Production Planning and Control. https://doi.org/10.1080/09537287.2020.1810767
Patton, M. (2002). Qualitative research and evaluation methods by Michael Quinn Patton (3rd ed., pp. 242–246). Sage Publications Limited.
Patton, M. (2005). Qualitative research. Encyclopedia of Statistics in Behavioral Science. https://doi.org/10.1002/0470013192.bsa514
Raguseo, E. (2018). Big data technologies: An empirical investigation on their adoption, benefits and risks for companies. International Journal of Information Management, 38(1), 187–195. https://www.sciencedirect.com/science/article/pii/S0268401217300063 [Accessed 13 Apr. 2021].
Ramanathan, R., Philpott, E., Duan, Y., & Cao, G. (2017). Adoption of business analytics and impact on performance: a qualitative study in retail. Production Planning and Control, 28(11–12), 985–998. https://doi.org/10.1080/09537287.2017.1336800
Rogers, E. (2003). Diffusion of innovation (5th ed., pp. 5–100). The Free Press.
Sagiroglu, S. & Sinanc, D. (2013). Big data: A review. In 2013 international conference on Collaboration Technologies and Systems (CTS) (pp.1–7). https://ieeexplore.ieee.org/abstract/document/6567202 [Accessed 13 Apr. 2021].
Schoenherr, T., & Speier-Pero, C. (2015). Data science, predictive analytics, and big data in supply chain management: Current state and future potential. Journal of Business Logistics, 36(1), 120–132. https://doi.org/10.1111/jbl.12082
Sena, V., Bhaumik, S., Sengupta, A., & Demirbag, M. (2019). Big data and performance: What can management research tell us? British Journal of Management, 30(2), 219–228.
Sharmila, P., Baskaran, J., Nayanatara, C., & Maheswari, R. (2019). A hybrid technique of machine learning and data analytics for soptimised distribution of renewable energy resources targeting smart energy management. Procedia Computer Science, 165, 278–284.
Sivarajah, U., Kamal, M., Irani, Z. & Weerakkody, V. (2017). Critical analysis of big data challenges and analytical methods. Journal of Business Research, 70, 263–286. https://www.sciencedirect.com/science/article/pii/S014829631630488X [Accessed 13 Apr. 2021].
Sorescu, A. (2017). Data-driven business model innovation. Journal of Product Innovation Management. https://doi.org/10.1111/jpim.12398
Spanaki, K., Gürgüç, Z., Adams, R., & Mulligan, C. (2018). Data supply chain (DSC): Research synthesis and future directions. International Journal of Production Research. https://doi.org/10.1080/00207543.2017.1399222
Spanaki, K., Karafili, E., & Despoudi, S. (2021). AI applications of data sharing in agriculture 4.0: A framework for role-based data access control. International Journal of Information Management. https://doi.org/10.1016/j.ijinfomgt.2021.102350
Sun, S., Cegielski, C. G., Jia, L., & Hall, D. J. (2018). Understanding the factors affecting the organizational adoption of big data. Journal of Computer Information Systems, 58, 193–203.
Sun, S., Hall, D. J., & Cegielski, C. G. (2020). Organisational intention to adopt big data in the B2B context: An integrated view. Industrial Marketing Management, 86, 109–121.
Suri, H. (2011). Purposeful sampling in qualitative research synthesis. Qualitative Research Journal, 11(2), 63–75. https://pdfs.semanticscholar.org/e287/d5579e587325ebaf789834124c4f94969de4.pdf [Accessed 13 Apr. 2021].
Tankard, C. (2012). Big data security. Network Security, 2012(7), 5–8. https://www.sciencedirect.com/science/article/pii/S1353485812700636 [Accessed 13 Apr. 2021].
Tannahill, B. & Jamshidi, M. (2014). System of systems and big data analytics: Bridging the gap. Computers and Electrical Engineering, 40(1), 2–15. https://www.sciencedirect.com/science/article/pii/S004579061300298X [Accessed 13 Apr. 2021].
Turner, D. (2010). Qualitative interview design: A practical guide for novice investigators. The Qualitative Report, 15(3), 754–760. https://nsuworks.nova.edu/tqr/vol15/iss3/19 [Accessed 13 Apr. 2021].
Wamba, S. F., Gunasekaran, A., Akter, S., Ren, S. J., Dubey, R., & Childe, S. J. (2017). Big data analytics and firm performance: Effects of dynamic capabilities. Journal of Business Research, 70, 356–365. https://doi.org/10.1016/j.jbusres.2016.08.009
Wang, Y., Kung, L. & Byrd, T. (2018). Big data analytics: Understanding its capabilities and potential benefits for healthcare organisations. Technological Forecasting and Social Change, 126, 3–13. https://www.sciencedirect.com/science/article/pii/S0040162516000500#bb0260 [Accessed 13 Apr. 2021].
Wang, G., Gunasekaran, A., Ngai, E. W. T., & Papadopoulos, T. (2016). Big data business analytics in logistics and supply chain management: Certain investigations for research and applications. International Journal of Production Economics, 176, 98–110.
Watson, H. J. (2014). Tutorial: Big data analytics: Concepts, technologies, and applications. Communications of the Association for Information Systems. https://aisel.aisnet.org/cais/vol34/iss1/65/ [Accessed 13 Apr. 2021].
Wu, X., Zhu, X., Wu, G. & Ding, W. (2014). Data mining with big data. IEEE Transactions on Knowledge and Data Engineering, 26(1), pp. 97–107. https://ieeexplore.ieee.org/document/6547630 [Accessed 13 Apr. 2021].
Yousefi, A., Ameri Sianaki, O., & Jan, T. (2019). Big data analytics for electricity price forecast. Advances in Intelligent Systems and Computing. https://doi.org/10.1007/978-3-030-15035-8_90
Zhang, H., Song, M., & He, H. (2020). Achieving the success of sustainability development projects through big data analytics and artificial intelligence capability. Sustainability, 12(3), 949.
Zhou, K., Fu, C., & Yang, S. (2016). Big data driven smart energy management: From big data to big insights. Renewable and Sustainable Energy Reviews, 56, 215–225.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Kava, H., Spanaki, K., Papadopoulos, T. et al. Data analytics diffusion in the UK renewable energy sector: an innovation perspective. Ann Oper Res 333, 717–742 (2024). https://doi.org/10.1007/s10479-021-04263-1
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10479-021-04263-1