A Data Ecosystem is a socio-technical system enabling value to be extracted from data value chains supported by interacting organizations and individuals. Within an ecosystem, data value chains can be oriented to business and societal purposes. The ecosystem can create the conditions for a marketplace competition between participants or enable collaboration among diverse, interconnected participants that depend on each other for their mutual benefit. Data Ecosystems can be formed in different ways around an organisation or community technology platforms, or within or across sectors (Curry 2016).
Creating a European data ecosystem would “bring together data owners, data analytics companies, skilled data professionals, cloud service providers, companies from the user industries, venture capitalists, entrepreneurs, research institutes and universities” (DG Connect 2013). However, in the early 2010s, there was no coherent data ecosystem at the European level (DG Connect 2013), and Europe was lagging behind in the adoption of big data. To drive innovation and competitiveness, Europe needed to foster the development and broad adoption of data technologies, value-adding use cases and sustainable business models. There were significant challenges to overcome.
4.1 Challenges
To understand the difficulties that existed in establishing a European data ecosystem, it is useful to look at the multiple challenges (Cavanillas et al. 2016a) that needed to be overcome:
-
Low rates of big data adoption: The European industry was lagging in the adoption of big data solutions. Many businesses and NGOs were uncertain of how to apply the technology within their operations, what the return on investment would be and how to deal with non-technical issues such as data privacy.
-
A disconnection between data owners and data innovators: Many data owners (often large organisations) possessed large datasets, but they could not fully utilize big data’s innovation potential. Data entrepreneurs and innovators (often SMEs and researchers) had vital insights on how to extract the value but lacked access to the data to prove their innovation. This mismatch created an impasse which needed to be overcome if innovation was to flourish.
-
Lack of technical and non-technical big data skills: A key challenge for Europe was the provision of appropriately skilled people who had an excellent grasp of the best practices and technologies for delivering big data solutions. There was a shortage of data scientists and engineers who had expertise in analytics, statistics, machine learning, data mining and data management. Strong domain knowledge of how to apply big data know-how within organisations to create value was and still is a critical but rare skill.
-
Next-generation technologies: US organizations had mainly driven the first generation of big data technology. It was essential to develop European leadership in the next generation of big data technology. Leadership in this space was critical for job creation and prosperity by creating a European-wide competency in technology and applications.
A thriving data ecosystem would need to overcome these challenges and bring together the ecosystem stakeholders to create new business opportunities, more access to knowledge and benefits for society. For Europe to seize this opportunity, action was needed.
4.2 A Call for Action
Big data offers tremendous untapped potential value for many sectors, however, there was no coherent data ecosystem in Europe. As Commissioner Kroes explained, “The fragmentation concerns sectors, languages, as well as differences in laws and policy practices between EU countries” (European Commission 2013; Neelie 2013). To develop its data ecosystem, Europe needed strong players along the big data value chain, in areas ranging from data generation and acquisition, through data processing and analysis, to curation, usage, service creation and provisioning. Each link in the value chain needed to be strong so that a vibrant big data value ecosystem could evolve.
The cross-fertilisation of a broad range of organisations (business, research and society) and data was seen as the critical enabler for advancing the data economy in Europe. Stakeholders from all along the Data Value Chain needed to be brought together to create a basis for cooperation to tackle the complex and multidisciplinary challenges to create an optimal business environment for big data that would accelerate adoption within Europe. During the ICT 2013 Conference, Commissioner Kroes called for a European public-private partnership on big data to create a coherent European data ecosystem that stimulates research and innovation around data, as well as the uptake of cross-sector, cross-lingual and cross-border data services and products.
4.3 The Big Data Value PPP (BDV PPP)
Europe needed to aim high and mobilise stakeholders throughout society, industry, academia and research to enable the creation of a European big data value economy. It needed to support and boost agile business actors; deliver products, services and technology; and provide highly skilled data engineers, scientists and practitioners along the entire big data value chain. The goal was an innovation ecosystem in which value creation from big data flourishes.
To achieve these goals the European contractual Public-Private Partnership on Big Data Value (BDV PPP) was signed on 13 October 2014. This marked the commitment of the European Commission, industry and partners from academia to build a data-driven economy across Europe, mastering the generation of value from big data and creating a significant competitive advantage for European industry, thus boosting economic growth and jobs.
The BDV PPP commenced in 2015 and was operationalised with the launch of the Leadership in Enabling and Industrial Technologies (LEIT) work programme of Horizon 2020. The BDV PPP activities addressed the development of technology and applications, business model discovery, ecosystem validation, skills profiling, regulatory and IPR environments, and many social aspects.
With an initial indicative budget from the European Union of €534M for the period 2016–2020 and €201M allocated in total by the end of 2018, the BDV PPP has already mobilised €1570M in private investments since the launch of the PPP (€467M for 2018). Forty-two projects were running at the beginning of 2019 and the BDV PPP in only 2 years developed 132 innovations of exploitable value (106 delivered in 2018, 35% of which are significant innovations), including technologies, platforms, services, products, methods, systems, components and/or modules, frameworks/architectures, processes, tools/toolkits, spin-offs, datasets, ontologies, patents and knowledge. Ninety-three percent of the innovations delivered in 2018 had economic impact and 48% had societal impact. By 2020, the BDV PPP had projects covering a spectrum of data-driven innovations in sectors including advanced manufacturing, transport and logistics, health, and bioeconomy. These projects have advanced the state of the art in key enabling technologies for big data value and in non-technological areas such as providing solutions, platforms, tools, frameworks, best practices and invaluable general innovations, setting up firm foundations for a data-driven economy and future European competitiveness in data and AI.
The BDV PPP has supported the emergence of a comprehensive data innovation ecosystem for achieving and sustaining European leadership in big data and delivering the maximum economic and societal benefits to Europe – its businesses and citizens. In 2018 alone, the BDV PPP organised 323 events (including European Big Data Value Forum, BDV PPP Summit, seminars and conferences) outreaching over 630,000 participants, and taking into account mass media. The number of people outreached and engaged in dissemination activities has been estimated at 7.8 million by the Monitoring Report 2018 (Big Data Value PPP Monitoring Report 2018 2019). According to the European Data Market Study,3 there has been a significant expansion of the European Data Economy in recent years:
-
The number of Data Companies increased to 290,000 in 2019, compared to 283,300 in 2018.
-
The revenues of Data Companies in the European Union reached €83.5B in 2019 compared to €77B in the previous year, with a growth rate of 8%.
-
The baseline for Data Professionals in the European Union in 2013 was 5.77 million. The number of data professionals increased to a total of 7.6 million by 2019 in the EU28, corresponding to 1.836 million jobs created for data professionals since 2013.
4.4 Big Data Value Association
The Big Data Value Association (BDVA) is an industry-driven international non-profit organisation which has grown over the years to over 220 members all over Europe, with a well-balanced composition of large, small and medium-sized industries as well as research and user organisations. BDVA has over 25 working groups organised in Task Forces and subgroups, tackling all the technical and non-technical challenges of big data value.
BDVA served as a private counterpart to the European Commission to implement the Big Data Value PPP programme. BDVA and the Big Data Value PPP pursued a common shared vision of positioning Europe as the world leader in the creation of big data value. BDVA is also a private member of the EuroHPC Joint Undertaking and one of the leading promoters and driving forces of the AI, Data and Robotics Partnership planned for the next framework programme Multiannual Financial Framework (MFF) 2021–2027.
The mission of BDVA was “to develop the Innovation Ecosystem that will enable the data-driven digital transformation in Europe delivering maximum economic and societal benefit, and, to achieve and to sustain Europe’s leadership on Big Data Value creation and Artificial Intelligence.” BDVA enabled existing regional multi-partner cooperation, to collaborate at European level through the provision of tools and know-how to support the co-creation, development and experimentation of pan-European data-driven applications and services, and know-how exchange. To achieve its mission, in 2017 BDVA defined four strategic priorities (Zillner et al. 2017):
-
Develop Data Innovation Recommendations: Providing guidelines and recommendations on data innovation to the industry, researchers, markets and policy-makers
-
Develop Ecosystem: Developing and strengthening the European big data value ecosystem
-
Guiding Standards: Driving big data standardisation and interoperability priorities and influencing standardisation bodies and industrial alliances
-
Know-How and Skills: Improving the adoption of big data through the exchange of knowledge, skills and best practices
BDVA developed a joint Strategic Research & Innovation Agenda (SRIA) on Big Data Value (Zillner et al. 2017). It was initially fed by a collection of technical papers and roadmaps (Cavanillas et al. 2016a) and extended with a public consultation that included hundreds of additional stakeholders representing both the supply and the demand side. The BDV SRIA defined the overall goals, main technical and non-technical priorities, and a research and innovation roadmap for the BDV PPP. The SRIA set out the strategic importance of big data, described the Data Value Chain and the central role of Ecosystems, detailed a vision for big data value in Europe in 2020, analysed the associated strengths, weaknesses, opportunities and threats, and set out the objectives and goals to be accomplished by the BDV PPP within the European research and innovation landscape of Horizon 2020 and at national and regional level.