Big Data-Driven Innovation in Industrial Sectors

Abstract

This chapter provides the conceptual background and overview of big data-driven innovation in society. Specifically, it examines the nature of data-driven innovation, exemplars of big data-driven innovations in sectors spanning healthcare, public sector, finance, media, energy, and transport. It discusses core enablers for these innovations highlighting factors and challenges associated with the adequate diffusion, uptake, and sustainability of big data-driven initiatives. Finally, it presents policy recommendations to guide the development of a big data innovation ecosystem.

Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

9.1 Introduction

Regardless of what form it takes, data has the potential to tell stories, identify cost savings and efficiencies , new connections and opportunities, and enable improved understanding of the past to shape a better future (US Chamber of Commerce Foundation 2014). Big data connotes the enormous volume of information including user-generated data from social media platforms (i.e. Internet data); machine, mobile, and GPS data as well as the Internet of Things (industrial and sensor data); business data including customer , inventory, and transactional data (enterprise data ); datasets generated or collected by government agencies, as well as universities and non-profit organizations (public data) (US Chamber of Commerce Foundation 2014). For many businesses and governments in different parts of the world, techniques for processing and analysing these large volumes of data (big data) constitute an important resource for driving value creation, fostering new products , processes , and markets, as well as enabling the creation of new knowledge (OECD 2014). In 2013 alone, the data-driven economy added an estimated $67 billion in new value to the Australian economy , equivalent to 4.4 % of its gross domestic product or the whole of its retail sector (Stone and Wang 2014).

As a source of economic growth and development, big data constitutes an infrastructural resource that could be used in several ways to produce different products and services. It also enables creation of knowledge that is vital for controlling natural phenomenon, social systems, or organizational processes and supports complex decision-making (OECD 2014). In this vein, the international development community and the United Nations are seeking political support at the highest levels on harnessing data-driven innovations to support sustainable development, particularly under the new global Sustainable Development Goals (SDGs) (Independent Expert Advisory Group on Data Revolution 2014). Similarly, cities like Helsinki, Manchester, Amsterdam, Barcelona, and Chicago are leveraging big and open data from open sensor networks, public sector processes, and crowdsourced social data to improve mobility, foster co-creation of digital public services, and in general enable better city intelligence to support more effective city planning and development (Ojo et al. 2015).

At the same time, there is a growing understanding of the challenges associated with the exploitation of big data in society. These challenges range from paucity of requisite capacity (e.g. data literacy ) to ethical dilemma in handling big data and how to incentivize the participation of other critical stakeholders in adopting and leveraging big data-driven innovation to tackle societal challenges (Hemerly 2013; Insight Centre for Data Analytics 2015).

This chapter describes what is involved in big data-driven innovation, provides examples of big data-driven innovations across different sectors, and synthesizes enabling factors and challenges associated with the development of a big data innovation ecosystem . The chapter closes by offering practical (policy) recommendations on how to develop viable big data innovation programs and initiatives.

9.2 Big Data-Driven Innovation

Innovation is an iterative process aimed at the creation of new products , processes , knowledge , or services by the use of new or even existing knowledge (Kusiak 2009). Data-driven innovation entails exploitation of any kind of data in the innovation process to create value (Stone and Wang 2014). The emerging trend of big data-driven innovation is leading to the development of data-driven goods and services and can enable data-driven planning, data-driven marketing, and data-driven operations across all industrial sectors and domains. From the economic perspective, data as a non-rivalrous good or commons such as oil serves an infrastructural resource (from a functional perspective) that could be exploited simultaneously by many users or actors for different competing or complementary ends. The demand for data in this sense according to the OECD (2014) is driven primarily by downstream productive activities that require data as an input and, in fact, a non-trivial capital. In addition, the same authors assert that data resources may be used as input into a wide variety of goods, including private, public, and social goods. In other words, big data potentially offers significant returns to scale and scope.

Big data-driven innovations are implicitly associated with a value chain model or more precisely a “virtual value chain ” specifying how the data of interest will be gathered, organized, selected, transformed into products or services, and distributed (Rayport and Sviokla 1995; Piccoli 2012). Big data value chains as discussed in Chap. 3 are at the core of delivering data-driven innovation using big data technology. At the organizational level, at least two categories of strategic initiatives could result from big data-driven innovation and its underlying big data value chain. The first category of initiatives aims to make information available on aspects of organizational processes and services to enable improvements. In general, by instrumenting organizational operations, large amounts of data (i.e. big data) are generated that inform or drive required changes (Piccoli 2012). The second set of initiatives is external facing and involves exploitation of customer data such as search and user logs, transaction records, and other customer-generated contents to drive long-tail marketing , targeted and personalized recommendation , increased sale, and customer satisfaction . A popular example of this is Netflix’s collaborative filtering algorithm to predict user movie ratings (Chen and Storey 2012). Yet another example is Google’s use of users search behaviour to target advertising (US Chamber of Commerce Foundation 2014).

In the United States, hundreds of companies are utilizing open and big data (such as weather and GPS data) as key resources to generate value across different sectors including finance and investment, education, environment and weather, housing and real estate, and food and agriculture (US Chamber of Commerce Foundation 2014). The next section elaborates on a number of data-driven transformations across different sectors including telecommunication, healthcare, public sector, finance and insurance, media and entertainment, energy, and transport.

9.3 Transformation in Sectors

The BIG Project examined how big data technologies can enable business innovation and transformation within different sectors by gathering big data requirements from vertical industrial sectors, including health , public sector , finance , insurance , telecom , media , entertainment , manufacturing , retail , energy , and transport . There are a number of challenges that need to be addressed before big data-driven innovation is generally adopted. Big data can only succeed in driving innovation if a business puts a well-defined data strategy in place before it starts collecting and processing information. Obviously, investment in technology requires a strategy to use it according to commercial expectations; otherwise, it is better to keep current systems and procedures. Organizations within many sectors are now beginning to take the time to understand where this strategy should take them.

The full results of this analysis are available in Zillner et al. (2014). Part III of this book provides a concise summary of the key findings from a selected number of sectors. The remainder of this chapter provides an executive summary of the findings from each sector together with discussion and analysis.

9.3.1 Healthcare

Investigation of the healthcare sector in Chap. 10 revealed several developments, such as escalating healthcare costs , increased need for healthcare coverage, and shifts in provider reimbursement trends, which have triggered the demand for big data technology. In the sector the availability and access of health data is continuously improving, the required big data technology (such as advanced data integration and analytics technologies) are in place, and first-mover best-practice applications have demonstrated the potential of big data technology. However, the big data revolution in the healthcare domain is in a very early stage with the most potential for value creation and business development unclaimed as well as unexplored. Current roadblocks to big data-driven innovation are the established system incentives of the healthcare system that hinders collaboration and, thus, data sharing and exchange. The trend towards value-based healthcare delivery will foster the collaboration of stakeholders to enhance the value of the patient’s treatment, and thus will significantly foster the need for big data applications.

9.3.2 Public Sector

The investigation of the public sector in Chap. 11 showed that the sector is facing some important challenges—the lack of productivity compared to other sectors, budgetary constraints, and other structural problems due to the aging population that will lead to an increasing demand for medical and social services, together with the foreseen lack of a young workforce in the future.

The public sector is increasingly aware of the potential value to be gained from big data-driven innovation via improvements in effectiveness and efficiency and with new analytical tools. Governments generate and collect vast quantities of data through their everyday activities, such as managing pensions and allowance payments, tax collection, etc. The main requirements, mostly non-technical, from the public sector are:

• (i) Interoperability : An obstacle to exploit data assets due to the fragmentation of data ownership and the resulting data silos.

• (ii) Legislative support and political willingness: The process of creating new legislation is often too slow to keep up with fast-moving technologies and business opportunities.

• (iii) Privacy and security issues : The aggregation of data across administrative boundaries in a non-request-based manner is a real challenge.

• (iv) Big data skills : Besides technical people, there is a lack of knowledge regarding the potential of big data in business-oriented people.

9.3.3 Finance and Insurance

As covered in Chap. 12 the finance and insurance sector is the clearest example of a data-driven industry. Big data represents a unique opportunity for most banking and financial services organizations to leverage their customer data to transform their business, realize new revenue opportunities, manage risk, and address customer loyalty . However, similarly to other emerging technologies, big data inevitably creates new challenges and data disruption for an industry already faced with governance, security, and regulatory requirements, as well as demands from the increasingly privacy-aware customer base.

At this moment not all finance companies are prepared to embrace big data, legacy information infrastructure, and organizational factors being the most significant barriers for its wide adoption in the sector. The deployment of big data solutions must be aligned with business objectives for a successful adoption of the technology to return the maximum business value .

9.3.4 Energy and Transport

Chapter 13 examines the sectors of energy and transport which from an infrastructure perspective, as well as from resource efficiency and quality of life perspectives, are very important for Europe. The high quality of the physical infrastructure and global competitiveness of the stakeholders needs to be maintained with respect to the digital transformation and big data-driven innovation .

The analysis of the available data sources in energy as well as their use cases in the different categories for big data value : operational efficiency , customer experience , and new business models make it clear that a mere utilization of existing big data technologies as employed by the online data businesses will not be sufficient. Domain- and device-specific adaptations are necessary for use in the cyber-physical systems of oil, gas, electrical, and transport. Innovation regarding privacy and confidentiality preserving data management and analysis is a primary concern of all energy and transport stakeholders that are dealing with customer data, be it business-to-consumer or business-to-business. Without satisfying the need for privacy and confidentiality , there will always be uncertainty around regulation and customer acceptance of new data-driven offering.

The increasing intelligence embedded in the infrastructures will enable the “in-field ” analysis of the data to deliver “smart data ”. This seems to be necessary, since the analytics involved will require much more elaborate algorithms than for other sectors such as retail. Additionally, the stakes are very high since the optimization opportunities are within critical infrastructures .

9.3.5 Media and Entertainment

The media and entertainment industries have frequently been at the forefront of adopting new technologies. Chapter 14 details the key business problems that are driving media companies to look at big data-driven innovation as a way to reduce the costs of operating in an increasingly competitive landscape, and at the same time, the need to increase revenue from delivering content. It is no longer sufficient to publish a newspaper or broadcast a television programme—contemporary operators must drive value from their assets at every stage of the data lifecycle .

Media players are also more connected with their customers and competitors than ever before—thanks to the impact of disintermediation, content can be generated, shared, curated, and republished by literally anyone. This means that the ability of big data technologies to ingest and process many different data sources, and if required even in real-time, is a valuable asset companies are prepared to invest in.

As with the telecom industry, the legal and regulatory aspects of operating within Europe cannot be disregarded. As one example, it is critical that just because it is technically possible to accumulate vast amounts of detail about customers from their service usage, call centre interactions, social media updates, and so on, it does not mean that it is ethical to do so without being transparent about how the data will be used. Europe has much stronger data protection rules than the United States, meaning that individual privacy and global competitiveness will need to be balanced.

9.3.6 Telecommunication

The telecom sector seems to be convinced of the potential of big data technologies. The combination of benefits within marketing and offer management, customer relationship , service deployment, and operations can be summarized as the achievement of the operational excellence for telecom players.

There are a number of emerging big data telecom-specific commercial platforms available in the market that provide dashboards, reports to assist decision-making processes, and can be integrated with business support systems (BSS) . Automatic actuation on the network as a result of the analysis is yet to come. Besides these platforms, Data as a Service (DaaS) is a trend some operators are following, which consists of providing companies and public sector organizations with analytical insights that enable third parties to become more effective.

Another very important factor within the sector is related to policy. The Connected Continent framework, aimed at benefiting customers and fostering the creation of the required infrastructure for Europe to become a connected community, at first sight, will most probably result in more strict regulations for telco players. A clear and stable framework is very important to foster investment in technology, including big data solutions.

9.3.7 Retail

The retail sector will be dependent on the collection of in-store data , product data , and customer data . To be successful in the future, retailers must have the ability to extract the right information out of huge data collections acquired in instrumented retail environments in real time . Existing business intelligence for retail analytics must be reorganized to understand customer behaviour and to be able to build more context-sensitive, consumer- and task-oriented recommendation tools for retailer-consumer dialog marketing.

9.3.8 Manufacturing

The core requirements in the manufacturing sector are the customization of products and production—“lot size one”—the integration of production in the larger product value chain , and the development of smart products .

The manufacturing industry is undergoing radical changes with the introduction of IT technology on a large scale. The developments under “Industry 4.0 ” include a growing number of sensors and connectivity in all aspects of the production process. Thus, data acquisition is concerned with making the already available data manageable, i.e., standardization and data integration are the biggest requirements. Data analysis is already applied in intra-mural applications and will be required for more integrated applications that cover complete logistics chains across factories in the production chain and even into the post-sale usage of (smart) products. Production planning needs to be supported by data-based simulation of these complete environments.

Complex and smart machinery , e.g., airplane engines, can benefit from big data-based predictive maintenance where sensor and context information is used with machine learning algorithms to avoid unnecessary maintenance and to schedule protective repairs when failures are predicted. Given the additional infrastructure costs, manufacturers are using new business models where machinery is leased and not sold; and in turn sensor data and services are owned and executed by the manufacturer and not the user of machinery. This leads to challenges in regulations and contracts concerning data ownership.

The European manufacturing sector can be both a market leader using big data in the context of Industry 4.0, and a leading market, where manufacturing big data is integrated in the larger product value chain and smart products can be put to use.

9.4 Discussion and Analysis

The analysis of the key findings across the sectors indicates that it is important to distinguish the technical from the business perspective. From a technological perspective, big data applications represent an evolutionary step. Big data technologies, such as decentralized networking and distributed computing for scalable data storage and scalable data analytics , semantic technologies and ontologies , machine learning , natural language processing , and other data mining techniques have been the focus of research projects for many years. Now these techniques are being combined and extended to address the technical challenge faced in the big data paradigm.

When analysed from the business perspective, it becomes clear that big data applications have a revolutionary —sometimes even disruptive —impact on the existing industrial business-as-usual practices. If thought through: new players emerge that are better suited to offer services based on mass data. Underlying business processes change fundamentally. For instance in the healthcare domain, big data technologies can be used to produce new insight about the effectiveness of treatments and this knowledge can be used to increase quality of care. However, in order to benefit from the value of these big data applications, the industry requires new reimbursement models that reward the quality instead of quantity of treatments. Similar changes are required in the energy industry: energy usage data from end users would have benefits for multiple stakeholders such as energy retailers, distribution network operators, and new players such as demand response providers and aggregators, energy efficiency service providers. But who is to invest in the technologies that would harvest the energy data in the first place? New participatory business value networks are required instead of static value chains .

Within all industries the 3 Vs of big data, volume , velocity , and variety , have been of relevance. In addition, industrial sectors that are already reviewing themselves in the light of the big data era add further Vs to reflect sectorial-specific aspects and to adapt the big data paradigm to their particular needs. Many of those extensions, such as data privacy , data quality , data confidentially , etc., address the challenge of data governance , while other extensions, such as value, address the fact that the potential business value of big data applications is yet unexplored and may not be well understood within the sector.

Within all industrial sectors it became clear that it was not the availability of technology, but the lack of business cases and business models that is hindering the implementation of big data. Usually, a business case needs to be clearly defined and convincing before investment is made in new applications. However, in the context of big data applications, the development of a concrete business case is a very challenging task. This is due to two reasons. First, as the impact of big data applications relies on the aggregation of not only one but also a large variety of heterogeneous data sources beyond organizational boundaries, the effective cooperation of multiple stakeholders with potentially diverging or at first orthogonal interests is required. Thus, the stakeholders ’ individual interests and constraints—which in addition are quite often moving targets—need to be reflected within the business case. Second, existing approaches for developing business models and business cases usually focus on single organizations and do not provide guidance for dynamic value networks of multiple stakeholders within a digital single market .

9.5 Conclusion and Recommendations

Data-driven innovation has the potential to impact all sectors of the economy. However to realize these, potential policymakers need to develop coherent policies for the use of data. This could be achieved by: (1) supporting education that focuses on data science skills , (2) removing the barriers to create a digital single market , (3) stimulating the necessary investment environment needed for big data technology, (4) making public data accessible through open data and removing data silos, (5) providing competitive technical infrastructure , and (6) promoting balanced legislation , and at the same time, policy must address issues such as privacy and security , ownership and transfer, and infrastructure and data civics (Hemerly 2013). In this vein, there are calls for a magna carta for data to address questions on how big data technologies could facilitate discrimination and marginalization; how to ensure that contracts between individuals and powerful big data companies or governments are fair; and where to situate the responsibility for the security of data (Insight Centre for Data Analytics 2015). In our opinion, further and sustainable progress in big data-driven innovation is contingent on actions by governments in collaboration with other major stakeholders in developing the right policy and regulatory environment based on empirical evidences from systematic research around some of the questions advanced above.