1 Introduction

“If we are to live healthy, fulfilling lives on this planet in the future, we must find new, life-affirming values and forge new patterns of living and working together” (Benn et al. 2014).

The progress in Information and Communication Technology (ICT) is leading societies of the 21st century into an ever-increasing digitalized world, where information and knowledge becomes readily available to more and more people every day. Societies are becoming landscapes mediated by different digital media platforms, digital services, and technologies that allow sensing, data capture and analytics. This progress and change in technology inevitably leads to a change in how societies are organized, and how their members interact with each other. Massive amounts of data are generated every moment from a growing number of sources. Companies are realizing that the data they own and the way they use them can provide them with a competitive edge. Big data and business analytics are also challenging existing modes of business and well-established companies. Yet, there is limited understanding of how organizations need to change to embrace these technological innovations, and the business shifts they entail which can lead to business and societal transformation (Chen et al. 2012; Loebbecke and Picot 2015).

Big data, business analytics, and “smart” environments have attracted great attention over the past few years in driving organizational decision making, as organizations are working on how on how to give purpose to the data, and get value-driven answers that will increase their performance (Mayhew et al. 2016), influencing different members in the society (e.g., individuals, businesses, governments) (Chen et al. 2012). Big data may be one the most significant technological disruptions in business and academic ecosystems in recent years (Agarwal and Dhar 2014). As the label itself indicates, big data refers to large volumes of data generated and made available online and in digital media ecosystems. Big data are generated from different type of sources, such as the multiple transactions performed daily, posts made on social media, or from the increasing number of sensors installed in numerous objects (e.g., mobile phones, home appliances, cars, etc.). Big data analytics is a tool that goes beyond pattern analysis, allows the prediction of events (George et al. 2014), and supports artificial intelligence that is able to automatize processes, transform companies and create new types of business as it can do now (Ng 2018), as well as to create value for the development of sustainable and prosperous societies.

Companies have been considered responsible for multiple challenges of the society, with social, environmental, and economic consequences (Porter and Kramer 2019), even though the vast majority of CEOs state that achieving business sustainability is of high priority for them (Hoffman 2018). To this end, focus has also been on developing and marketing sustainable products and services that impact society in general. Furthermore, citizens are becoming more responsible and aware of the impacts coming from the consumption of the various services and goods (e.g., environmental, data protection) and require them to be produced/operate in sustainable and transparent ways. Addressing social, environmental, and economic challenges creates value for numerous stakeholders in a society. For example, a company can gain a competitive advantage by investing in wellness programs that take advantage of big data. Specifically, companies may offer free wearable fitness devices to their employees if they agree to reach specific activity goals, urging them to exercise more and improve their health. The latter improves the quality of life of the employees and their families, increases productivity for the company, reduces medical costs for the government overtime, and may impact research as well since the data can also be used for medical studies.

As big data tools and applications spread, they will inevitably change long-standing ideas about decision making, management practices, competitive strategy formulation, and value creation. Big data and business analytics, mainly driven by practice, have been used to predict or explain what caused certain outcomes (Reed and Dongarra 2015). Indeed, recently in their commentary on the role of big data in IS, Agarwal and Dhar (2014) highlight how the experiments conducted on Facebook were popular in the media, because they raised issues regarding the protection of privacy and individuals’ rights. Since then, big data analytics have evolved and are increasingly used by companies, economists, marketers, or political scientists to analyze and predict numerous phenomena, from stock prices, to purchase behavior and voting intentions. Going further, the generated knowledge from big data and business analytics may be used to target people with specific information to influence perceptions, attitudes, or behavior, over time. At the time of writing the present editorial, the case of Cambridge Analytica is receiving increased attention, because of its role in the US elections of 2016, and how it used big data from Facebook, which were passed to third parties, for innovative political marketing campaign. Besides the impact of this practice and its significance, and European Union’s efforts to put into practice the General Data Protection Regulation (GDPR); it is interesting to note that the issues of data privacy and individuals’ rights on their digital footprint remain. Indeed, similar problems continue to exist suggesting that different types of approaches and changes in culture are required (Mayer-Schönberger and Cukier 2013). Considering that technology evolves rapidly with social media existing for over a decade, and that we generate massive amounts of data every moment, the lack of consensus and regulations poses a significant barrier towards the adoption of big data for achieving digital transformation while developing sustainable societies.

There is a growing need for findable, accessible, interoperable and reusable infrastructures and data management standards that provide greater access to the information in the society (Wilkinson et al. 2016). Investing in such infrastructures enables thriving innovation and digitalization of the city services and jumpstarts a wide-range of technology ecosystems. Digital infrastructures are now integral in numerous fields (e.g., business, health, transportation, finance), but the question remains on how we can give purpose to the data and extract actionable insight; by going beyond technical innovations and security issues, asking the right questions, and bridging business transformation with big data analytics for value creation that accelerates the sustainable development of the society.

In the 21st century’s ecosystems none of their actors can be seen in isolation, instead all of them need to actively interact and collaborate with each other to create knowledge and innovate, while evolving their interrelations, leading to new technologies and companies, and increased value. The cross-disciplinary nature of IS discipline can be the driving force that will give meaning to big data and improve the relations among data and business models (Agarwal and Dhar 2014). Digitalization of services and transformation of contemporary business models is needed to accelerate the creation of sustainable societies. New digital business models will not only be more accurate and efficient but also go beyond economic needs, and address societal challenges generating shared value that impacts the companies, organizations, consumers, and the public at large (Porter and Kramer 2019). Big data play a key role in this transformation and combining them from multiple sources, sharing them with various stakeholders, and analyzing them in different ways allows the achievement of digital transformation and creation of sustainable societies. A comprehensive analysis of the big data and business analytics ecosystem and its interdependencies, enables the development of frameworks that will provide solutions that benefit all the actors within the ecosystem.

2 Conceptualizing big data and business analytics ecosystems

The term ecosystem has been proposed to describe the interaction system which includes living organisms and their non-living environment (Tansley 1935). Since the term was coined, it has been used in different areas including biology, business, management, technology, and innovation leading to more specific ecosystems such as business ecosystems (Moore 1996), or innovation ecosystems that focus on value creation for firms (Adner and Kapoor 2010). An ecosystem in the area of management, technology and innovation is defined as a historically self-organized or managerially designed multilayer social network that consists of actors that have different attributes, decision principles, and beliefs (Tsujimoto et al. 2017). Thus, such an ecosystem should be viewed as a highly complex system that can organize itself and requires long term data collection. Furthermore, as an ecosystem consists of multiple hierarchical layers, cooperation, collaboration, and coopetition among its actors is required but it may be difficult to be achieved. Also, the relations among the actors of an ecosystem cannot remain solely within the business context, instead they are likely to extend to different contexts, like personal or procedural relations. Since the actors are different from one another it is necessary to examine their attributes and beliefs to better understand their behavior, their capabilities, and their needs, which in turn will improve the efficiency, coherency, and the performance of the ecosystem overall.

Drawing from the business ecosystems (Moore 1996), the big data value chain has been proposed recently to model high level activities within information systems, and is placed in the core of the big data ecosystem at a micro level, while multiple stakeholders exist at the meso and macro level (Curry 2016). When referring to big data and business analytics, the term ecosystem describes the environment created and supported by the numerous actors, that comprise the ecosystem, their perpetual data generation along with their interactions and interrelations. Such ecosystems already exist in the industry within or between different sectors (e.g., Apple, Google, Intel, Microsoft). However, if our ultimate goal is to create sustainable societies we need to evolve existing ecosystems, or develop new ones, to be more dynamic and actively include more stakeholders of their stakeholders, taking into account both their capabilities and needs. The works from Knabke and Olbrich (2017) and Mikalef et al. (2017) in this special issue, highlight both the importance and potential of dynamic capabilities in increasing business intelligence and value, thus improving our current understanding on how firms may evolve their role and position in current big data and business analytics ecosystems.

A strong analytics capability is key to digital transformation, as organizations that want to compete in the digital economy will have to invest in various resources including people, processes and technology of data and analytics (Carlsson 2017). Such resources can be categorized into tangible, intangible, and human skills and they can lead to increased performance and create competitive advantage for organizations (Grant 1991). As big data have become a necessary resource in creating value, big data analytics capability is defined as the ability of a data actor to effectively deploy technology and talent to capture, store and analyze data, towards value creation, business change, and societal change (Gupta and George 2016; Loebbecke and Picot 2015; Mikalef et al. 2017). To achieve this a data-driven culture is required, which will allow decision-makers to base their decisions more on insight rather than instinct (McAfee and Brynjolfsson 2012). Furthermore, along with data-driven culture, organizational learning is a critical aspect of effective deployment of big data initiatives (Mikalef et al. 2017; Vidgen et al. 2017), which allows data actors to exploit existing knowledge and continuously explore new knowledge in order to keep up with unpredictable market conditions. Finally, both technical and managerial-oriented skills are critical to derive value from big data investments (Wamba et al. 2017). Thus, big data analytics capability includes basic resources and technology (tangible), technical and managerial skills (human skills), and data driven culture and organizational learning.

Interactions among academia, industry, and government are essential to create the needed technological, institutional, and psychological conditions to innovate in a knowledge-based society, creating the triple-helix model (Etzkowitz and Carvalho de Mello 2004). The triple-helix model has been extended to the quadruple-helix model by incorporating the civil society (Carayannis and Campbell 2009), which includes individuals and citizens. Building on the above discussion, we posit that a big data and business analytics ecosystem comprises of the data actors (i.e., academia, industry/private organizations, government/public organizations, civil society, and individuals/entrepreneurs), who generate and use big data. We differentiate individuals as entrepreneurs from the civil society, considering their importance in creativity, innovation, economic growth, and social impact (Drucker 2014). The data actors need to develop big data analytics capabilities, in their respective contexts, which will lead to value creation, business change, and societal change. This is an iterative process based on which the data actors use their experience to constantly improve and evolve their big data analytics capabilities, and increase the generated value that impacts both business and society. The successful implementation of this process is key to digital transformation and the creation of sustainable societies, creating the big data and business analytics ecosystems. Figure 1 presents the Digital Transformation and Sustainability (DTS) model, which conceptualizes the big data and business analytics ecosystem and the factors that need to cooperate, coordinate, and collaborate to enable the use of big data towards the achievement of digital transformation and the creation of sustainable societies.

Fig. 1
figure 1

The digital transformation and sustainability (DTS) model

Full size image

In 21st century’s ecosystems the evolution of digital economy and its combination with big data have led to the advancement of traditional economic and business concepts and the development of new ones (George et al. 2014). As all the actors of a big data and business analytics ecosystem generate vast amount of data every moment (e.g., while browsing the internet, using social media, using sensor networks, performing business transactions, etc.), it is critical to capture and analyze such data, from multiple sectors, because it will help to better understand the behavior, capabilities, and needs of the respective actors. Furthermore, it allows to develop and offer better products and services, in different aspects (e.g., quality, precision, timeliness) within the ecosystem. The work from He et al. (2017) in this special issue shows how user generated content from social media can be analyzed to improve consumer service quality. Indeed, they argue that the combination of social media analytics with traditional methods of data gathering and analysis (e.g., surveys, focus groups) may lead to a more realistic understanding of consumers’ capabilities and needs.

Both private and public organizations have been implementing different techniques to benefit from the available big data and business analytics, (e.g., mobile, network, web, and text analytics). The main focus of private organizations is to increase business value, while of public organizations to offer better services to the citizens and improve society. However, private and public organizations are not at the same level of maturity in relation to their big data analytics implementations, partly because they have different goals, but also because they have different resources and access to different types of data (Kim et al. 2014). For example, a company may own data from web-based or mobile transactions with customers, or sensor-based data from mobile phones. On the other hand, a public organization may own data from transactions with citizens and data from legacy systems within its departments. Okwechime et al. (2017) in this special issue argue that public organizations are trying to catch up with organizations from the private sectors in the use of big data. Within the context of an ecosystem it is expected that knowledge transfer from companies could help public organizations to improve their skills and outcomes from analyzing big data, however Okwechime et al. (2017) point out that if their respective differences are not considered, friction and disconnection among them may occur. In order to achieve digital transformation, we need to evolve and change the ways that public and private organizations in the ecosystem (as well as the rest of the actors) interact, cooperate, and collaborate. Combining big data from contemporary technologies (e.g., smartphones) with current low technology services (e.g., water and electricity utilities) is a way to achieve digital transformation and create sustainable societies (George et al. 2014). To this end, Golightly et al. (2017), describe how different sectors (e.g., power generation, transport, manufacturing) are adopting big data and analytics tools for data-driven decisions and solutions. Their findings show that organizations need to go beyond their common data-driven design approaches, improve users’ ability to interpret data and use it to take decision-driven approaches. Similar approaches have been proposed in the public sector (Matheus et al. 2018), where helping governments to understand big and open data through data-driven dashboards can help decision-driven processes and improve the relation among citizens with governments.

Different types of data are available to the actors of the ecosystem, depending on their role, however it has been shown that they can create the necessary opportunities that may disrupt industries and transform societies. Based on the notion of creative economy, many societal problems may be solved through individual creativity, instead of big companies or organizations, which requires the convergence of technologies and knowledge, leading to the creation of new companies and economic growth (Howkins 2002). Individuals, such as entrepreneurs and innovators, can use available big data (e.g., public or open data) to develop new products or services that can transform a market or an industry (George et al. 2014). An individual may also drive innovation as a customer if the companies provide the necessary tools to design and develop their own products (Thomke and Von Hippel 2002), thus leading to value co-creation (Vargo et al. 2008). Big data and business analytics, as resources, may be transformed to assets that interconnect firms with customers and foster cooperation and value co-creation (Xie et al. 2016). Thus, a stakeholder may have multiple roles and contribute in multiple ways in creating value within the ecosystem. Indeed, the value of big data and business analytics has been established for addressing complex technical and business challenges (Chen et al. 2012). However their social value still remains unclear (Agarwal and Dhar 2014), with most of the studies in the area overlooking the potential of big data and business analytics into solving social problems (Zicari 2014). Since our goal is digital transformation towards the creation of sustainable societies, it is critical to answer the question on how well big data and business analytics can be used to solve complex societal challenges.

A core issue in the above discussion is the education of professionals that are capable of undertaking such tasks. While much discussion has focused around the role of the data scientist in contemporary organizations (Davenport and Patil 2012), recent literature also expanded on the role of managers in the age of big data (Ransbotham et al. 2015). Such practice-driven articles highlight the importance that big data analytics have on a growing number of organizations, and the requirement of individuals to be knowledgeable about the use of big data analytics. In fact, several recent studies have delved on the multiplicity of necessary skills that are looked for in industry (De Mauro et al. 2016; Mikalef et al. 2018). Such studies show that there are large gaps in the skills that graduates possess and those that are needed in industry. In addition, they illustrate the diversity of  the necessary skills, that amongst others include skills in programming, research methods, data handling, visualization tools, soft skills, domain knowledge, and strong cooperation competencies. With the importance of big data analytics growing in contemporary business ecosystems, the need for highly skilled individuals is ever increasing. Thus, a major area of interest in the upcoming years will be on the development of course curricula and support learning tools that facilitate education in such broad skill-sets, and rapid adaptation according to market demands.

In today’s digital era, public and private organizations, academia, industry, governments, and individuals generate vast amounts of ubiquitous data, spawning into new capabilities and opportunities, and creating value through new business strategies, policies, etc. Thus, we view society as a big data and business analytics ecosystem on which data, information, and knowledge is shared and transferred among its stakeholders, in order to achieve digital transformation and create sustainable societies. The formation of such ecosystems allows the collaboration, cooperation, and coopetition among its multiple actors, towards the creation of new possibilities for companies and governments to address consumers’ and citizens’ needs, that can also lead to new business opportunities for entrepreneurs, innovators, and firms in order to develop innovative digital data-based designs and transform current business models. Lastly, a big open data ecosystem will empower the individuals in controlling their lives, choosing what to share, and inspire them to be more active by assuming other roles within the ecosystem. The creation of big data and business analytics ecosystems has the potential to lead to digital transformation, and at the same time it may transform theory and practice in IS, management, technology, and innovation.

3 Focus of the special issue

The main objective of this special issue is to provide theoretical discussion and empirical support to better understand what is the role of big data and business analytics in 21st century’s ecosystems, and to develop a research agenda for the future. The call for papers generated a number of submissions but not all of them met the criteria of this special issue. After two or three rounds of review, five papers were finally accepted. The papers appearing in this special issue cover different aspects surrounding the areas of big data and business analytics ecosystems, while exploring how analytics may create or increase business value through various perspectives.

The first two papers address dynamic capabilities and how they can improve business intelligence agility and business value. The two papers complement each other. Knabke and Olbrich (2017) empirically demonstrate the importance of organizations’ assets, external, and internal view in increasing business intelligence agility, while Mikalef et al. (2017) provide a systematic literature review in the area of big data analytics and identify how big data analytics capabilities may increase business value (i.e., innovation, agility, firm performance). Furthermore, Mikalef et al. (2017) call for more empirical studies in the area and develop a research agenda which distinguishes six core areas of investigation, with the study by Knabke and Olbrich (2017) answering to this call in the present special issue. Next, the work from He et al. (2017) is a quantitative study that analyses content produced by individuals on social media and examines how it can improve service quality, which may have a potential impact on both public and private sector since service quality is important to all users, either as citizens or as consumers. Finally, the special issue includes two papers that take a qualitative approach to examine the role of big data and their value in public and private sector organizations. The works from Okwechime et al. (2017) and Golightly et al. (2017) provide a comprehensive picture and complement each other as they provide insight on how big data can be used in public and private organizations settings, respectively, for increased performance, better services, and improved solutions to existing problems.

Knabke and Olbrich (2017) take a dynamic capability approach to better understand and explain agile business intelligence systems. In detail, the authors highlight the need to bridge the gap between organizations’ long-term strategies and agile adaption to dynamic environments based on shifting market demands. This study attempts to answer how, and which, dynamic capabilities influence business intelligence agility, and if emerging technologies influence business intelligence agility. The authors develop a research model based on which dynamic capabilities (i.e., adoption of business intelligence assets, market understanding and intimacy with business intelligence, and business operations with business intelligence) influence business intelligence agility. The model is tested through a quantitative study that includes 110 participants that work in various industries. The findings show that adoption of emerging technologies, well-trained staff, corresponding organizational structures, and strategy alignment are essential for an organization to achieve business intelligence agility. This study offers theoretical grounding on the importance of asset adoption and integration of external and internal view for more agile supply of information and decision preparation, in today’s turbulent business environments.

Mikalef et al. (2017) posit that the technologies that enable big data correspond only partially to their value, instead multiple and complex relations, infused with organizational capabilities, are required to unlock their full potential for organizations and firms in increasing value and gaining competitive advantage. The study draws from the resource-based theory and dynamic capabilities view of the firm to develop a research framework that improves our understanding of the role of big data analytics in offering firms a competitive advantage and increasing business value. The authors adopt the notion of big data analytics capability which refers to a firm’s proficiency in orchestrating and managing its big data-related resources to gain strategic and operational insight. Through a systematic literature review, Mikalef et al. (2017) define big data, big data analytics, and big data analytics capabilities, while highlighting their main differences. Their findings lead to six thematic areas for big data analytics research, along with a discussion on what mechanisms should be leveraged to harness the great potential of big data analytics for organizations. The paper provides a holistic research framework for big data analytics that will lead to more theoretically driven research in the area, while at the same time it contains notions that may be adopted by different firms and organizations.

He et al. (2017) link industry and civil society, as they use user generated content from social media to investigate consumer behaviour. In detail, they propose a framework that based on social media analytics aims to improve service quality. To this end, the authors suggest that combining well-defined traditional metrics on service quality with big data analytics platforms, will lead to improved service quality and impact business value. Based on the fact that an increasing number of consumers post their shopping experience on social media, the authors perform a case study about two large retailers in the United States of America. After gathering about half a million tweets, they analyse social media mentions, perform sentiment analysis, and compare different service quality metrics. The findings verify the fact that social media analytics can provide useful insight on what consumers think or feel and allow a better understanding of their perceptions on service quality. The authors argue that combining social media analytics with traditional methods of data gathering and analysis (e.g., surveys, focus groups) will lead to a more realistic understanding of consumer needs.

Okwechime et al. (2017) identify new ways on how big data can be used by public sector organizations to offer better solutions to smart cities. In detail, they posit that the successful implementation of big data by the public sector for smart cities initiatives, depends on the organizations’ ability to embed acquired knowledge. To this end, the authors take an organizational learning approach to examine how public organizations exploit big data to carry out smart city initiatives, and present a conceptual model that outlines an iterative process which forms a learning ecosystem within organizations. Their study shows that the implementation of big data may affect the knowledge and power dynamics in terms of how these organizations operated before. The findings show that knowledge finding and reframing, along with inter-organizational collaborations and ex-post evaluations are needed for organizations to learn how to use and introduce big data, thus providing the groundwork for smart cities to fully explore and exploit the potential of big data.

Golightly et al. (2017) examine the role of big data in the industry and private organizations, in order to identify the factors that support or inhibit the design and deployment of predictive asset management. They differentiate from previous studies in the area of predictive management that focus on the technology (i.e., sensors and algorithms), by addressing user and organizational issues. To this end, the authors perform a cross-sector analysis to identify issues across the data-driven technology lifecycle, through conception, development, and long-term deployment. By conducting interviews with experts from different sectors, a framework is created that covers project motivations and conception, design and development, and operation. Their findings provide important themes towards decision driven approach to design, instead of more common data driven approaches. The paper contributes towards the need to improve data interpretation by users, as well as the improvement of management and maintenance of key assets that can increase business value.

4 Conclusions and the way ahead

The value of digital transformations that emerges through big data analytics ecosystems is an area that is going to attract a lot of attention in the upcoming years. In doing so, it is important to understand first the different actors, the data they generate, and how they interact, and second the necessary capabilities that need to be developed to harness this potential. The proposed Digital Transformation and Sustainability (DTS) model is a step towards this direction. Developing a data-driven culture within organizations, investing in appropriate technology, fostering technical and managerial skills, and promoting a climate of organizational learning are critical factors in realizing value. It is also important to recognize that value can emerge through different means, and can therefore be captured through different measures. While some organizations or entrepreneurs may focus on driving business value and keeping ahead of competitors, others may have a view of facilitating societal change, therefore generating value that impacts both them and the society overall. As business models become increasingly more oriented towards societal and individual needs, so will big data analytics ecosystems emerge as a driver of digital transformation and sustainability to fulfil this need. To this end, we propose the following research directions:

  • The role of data actors. How actively may data actors be involved in order to shape the digital transformation and development of the sustainable societies? These actors generate the data, own the data, and have the potential to benefit from the data. The data actors are typically involved in top-down approaches of data analysis, but they may also be involved in participatory (bottom-up) endeavors shaping how digital transformation will impact and change society.

  • Data capacities and availability. Further research is required on regulations around data in the digital society (e.g., GDPR). This needs to take into account the capacity and availability of big data, as well as differences between countries, continents, and cultures towards the creation of unified practices and regulations. This will require improved integration and adoption of powerful data collection technologies (e.g., smartphone sensors) and methods (e.g., real-time analysis).

  • Adoption at leadership and management level. Future research needs to examine how the different leaders and organizational structures are willing to adopt and implement data-driven methods in their decision-making processes. This requires a change in the culture of current organizations, which will also allow them to revise their contemporary routines to create value and achieve business or social change.

  • Data-driven sustainable development. Current practices and strategies are expected to build upon data-driven methods, thus we need a deeper understanding of how they can co-exist and co-evolve in the digital society. Various challenges exist before such a transformation can be achieved, and thus we need to change the existing process of how we design information technology and digital practices in our research.