1 Introduction

This chapter presents a best practice guide for the operation of Big Data and Artificial Intelligence Centres of Excellence (BDAI CoE). The goal of the guide is to foster collaboration and share best practices among existing Centres of Excellence (CoEs) and support the establishment of new CoEs within Europe.

The best practice guide is conceptualised as a framework to capture appropriate practices for operating a BDAI CoE. The framework was developed following a phased design science process, starting from a literature review to create an initial framework which was enhanced with the findings of a multi-case study of existing successful CoEs. Each case study involved an in-depth analysis and a series of in-depth interviews with CoE leadership.

The resulting best practice framework models a CoE using open systems theory (Von Bertalanffy 1968) that comprises input (environment), transformation (CoE) and output (impact). The framework conceptualises the internal operation of the CoE as a set of high-level capabilities including strategy, governance, structure, funding, and people and culture. The core capabilities of the CoE include business development, collaboration, research support services, technical infrastructure, experimentation/demonstration platforms, Intellectual Property (IP) and data protection, education and public engagement, policy outreach, technology and knowledge transfer, performance and impact assessment.

Initial insight from this work indicates that there is a wide range of practices that are needed to operate a BDAI CoE successfully. Some practices (governance, financial management, human resources) in the BDAI CoE environment are arguably the same as found in traditional businesses. However, other practices are unique to a BDAI CoE and are substantially different from conventional business practices. In particular, collaboration is a crucial practice between the CoE and industry players, balancing the need for scientific advancement and the transfer of technology to the industry.

The rest of the chapter is structured as follows: the Sect. 2 details what a CoE is and how it plays a fundamental role in the creation and sharing of research and innovation within the local and national innovation ecosystems. Section 3 sets out the methodology used in the design and refinement of the framework. In the Sect. 4, we describe the best practice framework for CoEs in big data and AI, including objectives, environment, strategic and operational capabilities, and impact. Section 5 outlines how the framework can be used by a CoE to support its strategic direction and operational decisions over time, and how a new CoE can use it in the start-up phase. Section 6 explores the critical success factors of CoE as defined by a survey of CoE managers. Finally, the chapter concludes with a summary.

2 Innovation Ecosystems and Centres of Excellence

To understand the essence and nature of a CoE, it is essential to understand the wider setting and context in which the CoE is situated. To this end, we introduce the key elements of the innovation ecosystem in which CoEs exist to understand their role within the national innovation ecosystem and the broader technological ecosystem.

National Innovation Ecosystems constitute networks of public and private sector institutions that generate value from the development and applications of new technologies. They play a crucial role in the socio-economic development of countries (Mowery et al. 1993; Fagerberg and Srholec 2008).

In this chapter, we focus on the networks around big data and AI technologies and their roles in the creation and sustainability of CoEs. In particular, our interest lies in the national or pan-European innovation systems that have a significant investment regarding funding and workforce directed towards addressing the challenges and leveraging the opportunities of big data and AI. We focus on the concept of the CoE to identify the characteristics of the thriving organisations, mainly public sector and universities that are leading the technological developments around big data and AI in Europe.

In natural ecosystems, smart organisms control their energy. In business ecosystems, smart companies manage their information and information flows (Kim et al. 2010). Regarding data, the ecosystem metaphor is used to describe the data environment supported by a community of interacting organisations and individuals. Data Ecosystems are formed in different ways around an organisation and community technology platforms, or within or across sectors. Data Ecosystems exist within many industrial sectors where a vast amount of data moves between actors within complex information supply (Cavanillas et al. 2016).

Beyond data, the AI Innovation Ecosystem (Zillner et al. 2020) is complex and diverse. It contains multiple types of stakeholders, and, to be effective, there needs to be alignment and collaboration between them. It is central for the sharing of assets, technology skills and knowledge. It provides a scale to achieve consensus and critical mass around the generation of value through innovation that no single partner alone could achieve. It expresses the collaborative purpose that binds organisations and individuals together in achieving success in deploying AI. A functional data and AI ecosystem must bring together the key stakeholders with clear benefits for all. The key actors in a big data and AI ecosystem (Zillner et al. 2017), as illustrated in Fig. 1, are as follows:

  • End-User: a person or organisation from the public or private sector that leverages AI technology and services to their advantage

  • Application Provider: an organisation that uses AI technology for developing a vertical AI application (e.g. to be offered as AI service)

  • User: a person who either knowingly or unknowingly uses or is impacted by a system product or service that uses AI

  • Data Supplier: a person or any organisation (public or private) that creates, collects, aggregates and transforms data from both public and private sources

  • Technology Creator: typically, an organisation (of any size) that creates tools, platforms, services, hardware and technical knowledge

  • Broker: an organisation that connects the supply and demand for AI assets (such as skills, data, algorithms, and infrastructure) needed for developing AI applications by providing a channel for exchanging AI assets

  • Innovator and Entrepreneur: drives the development of innovative AI technology, products and services

  • Researcher and Academic: researches and investigates new algorithms, hardware, technologies, methodologies and business models; provides skills and training in AI and assesses the societal aspects of AI

  • Regulator: assesses AI systems for compliance with regulation, privacy and legal norms

  • Standardisation Body: defines technology standards (consensus-based, de facto and formalised) to promote the global adoption of AI technology

  • Investor, Venture Capitalist and Incubator: provides resources and services to develop the commercial potential of the ecosystem

  • Citizen: a person who will or will not develop trust in AI technologies

Fig. 1
figure 1

The micro, meso and macro levels of a big data and AI ecosystem (Curry 2016)

An effective European AI Innovation Ecosystem facilitates the cross-fertilisation and exchange between participants that lead to new AI-powered value chains that can improve business and society and deliver benefits to people. A productive European AI Innovation Ecosystem is an essential component to overcome key adoption challenges. Within the ecosystem model, researchers and academics play research and innovation roles. Traditionally, within universities, academic departments and schools often work towards the establishment of a specific-purpose CoE to drive a research and innovation mission for big data and AI.

2.1 What Are Centres of Excellence?

Excellence as a concept has many varying definitions depending on the area of focus, that is, whether it is research, development, education or management. It is a complex concept that is difficult to define and operationalise due to its dynamic and multidimensional nature (Schmidt and Krogh Graversen 2017). Hellström states that “excellence is a term for the political and the scientific community: this is because its evaluative dimensions vary within a common theme which most researchers can relate to, and it is often tangible enough for external interests to partake and discuss its implications” (Hellström 2011). According to the OECD (2014), a CoE relates to promoting high-quality scientific research, facilitating basic research through funding, promoting the internationalisation of national research, raising the profile of the host institution through the establishment of a CoE, formulating influential research groups and collaborations, and attracting experts and highly skilled researchers. Another view by Ohno-Machado (2014) relates CoEs to data science skills, technical and policy infrastructure for data acquisition, efficient storage and management, knowledge generation, data security, and privacy protection and sector-wide collaboration. Aksnes et al. (2012) have identified three basic schemes for CoEs in Nordic countries, and these include programmes that focus on scientific excellence, schemes that aim for innovation excellence and programmes that address societal challenges.

Similarly, Hellström (2011, 2012) have developed an analytical framework for analysing CoE schemes according to their strategic orientation, institutional and operational conditions, and impact and capacity building attributes. In this regard, they classify CoE schemes according to the following strategic directions: basic and strategic research, innovation and advanced technological development, and social and economic development. In this context, we define a BDAI CoE as follows:

“A Big Data and Artificial Intelligence Centre of Excellence is an organisation or organisational unit within a national system of research and education that provides leadership in research, innovation and training for Big Data and AI technologies.”

The defining characteristic is its focus on enabling technologies and societal impacts of big data and AI. Within this broad scope, a CoE can serve as a common place for accumulating and creating knowledge that addresses challenges of big data and AI, opens new avenues of knowledge-based economies, guides policy instruments in the era of digital life, and informs the public about the externalities of technological advances based on information processing. Based on context consideration, we use the above-listed classification to categorise BDAI CoEs according to their primary strategic orientations.

3 Methodology

The framework was developed following a phased process, starting from a literature review to create an initial version which was enhanced with the findings of a multi-case study of existing successful BDAI CoEs. The CoEs were selected based on a mix of size, posture (from basic to applied research) and geographical balance.

The production of the framework followed two types of information-gathering exercises carried out on three BDAI CoEs that were selected as case studies. First, literature review (done as desktop research) provided secondary data on each case and, second, a series of interviews with senior managers (12 in total) of the selected CoEs produced primary data, also on each case. The elicited information was reviewed to cross-check correctness with the various sources and to fine-tune it for the best quality, including readability, understandability, navigability, organisation and presentation.

The methodology follows design science principles within a rigorous design process that facilitates the engagement of scholars, as well as ensuring consistency by providing a meta-model for structuring the methodology. The design science approach used is closely aligned with the three design science research cycles (Relevance Cycle, Rigor Cycle and Design Cycle) proposed by Hevner (2007).

In this approach, we had step-by-step activities that began with recognising the problem at hand, followed by statements of objectives to be actualised in the tasks. We engaged in the design and development of the framework. Next, we evaluated the framework, which was followed by the demonstration of how it could be used. Finally, we communicated the framework to users. The steps in the methodology are:

  • Identification and motivation of the problem

  • Definition of objectives for the framework

  • Design and development of the BDAI CoE framework

  • Evaluation of the framework

  • Demonstration of the use of the framework

  • Communication of the framework

A research methodology based on the Delphi method is employed for capturing the best practices and guidelines for CoEs (Linstone and Turoff 1975). The Delphi method is primarily used for forecasting with the help of a panel of experts over multiple iterations. Our methodology uses a two-round approach for capturing and refining best practices and guidelines with the help of a panel of seven CoE managers. The objective of the interviews was to capture the collective intelligence and experience of the interviewees within a framework for BDAI CoEs. The experts on BDAI CoEs were interviewed with participation from several CoEs across Europe.

4 Best Practice Framework for Big Data and Artificial Intelligence Centre of Excellence

The objective of the framework is to develop a best practice guide for use in promoting value generation and sharing of ideas within the big data and AI innovation ecosystem.

The goals are to:

  • Foster collaboration and promote sharing of best practices and know-how among CoEs and national initiatives

  • Provide expert guidance and (non-financial) support to member states looking to establish a new national CoE for big data and AI.

Within the framework as illustrated in Fig. 2, there is a process flow in the form of a value chain starting from the environment (which supplies input) through the core BDAI CoE capabilities (which process the input) to the output represented by the impact of the output received by the society under various categories: economic, scientific and societal. There is a backward flow (feedback) from the impact of a CoE back to the CoE and to the environment in which the CoE operates. For example, a CoE may hire personnel it trained as postgraduates or receive income from services rendered to a partner, which can return value to the CoE. Similarly, the impact created can influence the environment in which it operates, particularly regarding policymaking and funding decisions. The quality of output from a CoE is often the most significant determinant of funding decisions by funding agencies.

Fig. 2
figure 2

Framework for Big Data and AI Centre of Excellence

4.1 Environment

As described in the literature on organisational science, the “environment means forces difficult to control from inside that demanded a response” (Weisbord 1976). The external environment comprises forces that initiate organisational change (Burke and Litwin 1992). In the context of a BDAI CoE, the environment is defined as three forces: industry, policy and citizens.

4.1.1 Industry

The term industry refers to companies, start-ups and businesses that are carrying out economic activities related to big data and AI. While the big data and AI industry would directly affect the strategy and performance of a BDAI CoE, the relative strengths or weaknesses of other industrial sectors may be reflected in the core elements of the BDAI CoE framework. A recent Norwegian study indicated that the industry provides increasingly significant financial support (more than doubled since the 1980s) for academic research while the proportion of basic funding is decreasing (Gulbrandsen and Smeby 2005). In a study carried out among Norwegian university professors, a clear relationship exists between industry funding and research performance. Professors with industrial funding are often engaged in applied research and frequently produce entrepreneurial results, they collaborate more with other researchers both in academia and in industry, and they report more scientific publications (Perkmann and Walsh 2007; Gulbrandsen and Smeby 2005).

The industry in the context of the BDAI CoE framework is defined as follows:

“The ecosystem of companies surrounding a big data and AI Centre of Excellence that is associated with the creation of economic value, at both national and European levels.”

Establishing and maintaining strategic industry—research collaborations should be a priority for BDAI CoEs. Inter-organisational network relationships in the context of “open innovation”, the role of practices such as collaborative research, university-industry CoEs, contract research, and academic consulting are the basic needs of existing CoEs (Perkmann and Walsh 2007).

The industry demands for big data and AI tools and services drive research focus on the development of these innovative technologies through collaborative research, contract research and consultation services with industry participants. Industry-focused CoEs are highly user-centric in the design of their technologies, and, as such, they work very closely with their end-users to co-design functional solutions to the users’ respective challenges.

In the field of big data and AI, CoEs within the EU focus on different domains and trends, while the industries mainly drive decisions within a country. However, international development in science and technology also has an important impact on local trends and decision-making by the management of organisations. For example, within Ireland, the IT, medical and pharmaceutical industries are significant parts of the economy; therefore, data analytics research CoEs focus on providing cutting-edge technology tools and services for these sectors. Centres within economies dominated by petrochemicals focus on the development of data analytics for the digitalisation of oil and gas exploration and related developments in geology domains.

New or emerging CoEs should focus on the areas of interest of the country of operation to align the CoEs’ strategic interest with the national strategic interest. This enables the country to provide better funding and policy support for a CoE. As seen from the case studies, where interests diverge, a CoE could run into problems in balancing its priorities. Evidence from the survey indicates that internal capabilities, such as supportive governance, exemplary strategy implementation, the existing units for business development, a simplified IP arrangement process and advanced outreach programmes, are needed to promote university—industry collaborations.

4.1.2 Policy

Policies and regulations can be divided into two broad categories: research and innovation policy and data protection policy. The first policy defines the goals of funding available to CoEs and influences the alignment of the elements within a CoE with those goals. The second policy primarily focuses on clarifying rules about data usage, data ownership, data localisation and data portability (Ron 2016), which are critical to the operation of a CoE.

Policy in the context of the BDAI CoE framework is defined as follows:

“The policy is defined as the set of public laws, regulations and principles that govern research and innovation activities at the national and European level, as well as dictating the access, manipulation and distribution of data.”

A dedicated agency or agencies in each country support research activities and provide funding support when needed. The reason for the use of dedicated agencies to fund and support research institutions is that these agencies are specialised in designing arrangements and policies that help to align the research institutions’ strategic interests with the country’s overall educational system, particularly STEM subjects, research and development, and development of expertise. The agencies help to prioritise areas of research, not just for the country but also among existing CoEs in the country to avoid unnecessary duplication of research effort and funding. The agencies also monitor the performances of CoEs to ensure impacts are up to expectations considering the investment funding provided to them. For example, the Department of Business, Enterprise and Innovation (DBEI) in Ireland has the responsibility of enacting research-related policies and helps in setting national strategic directions regarding stem disciplines, including Science and Technology and Innovation (STI). In addition to the DBEI, the Science Foundation of Ireland (SFI), Enterprise Ireland (EI) and the Industrial Development Authority (IDA) are Irish Government agencies that not only fund Research and Innovation (R&I) development initiatives but also play crucial roles in planning and deciding the direction of the country’s technological development, including the development of expertise. Generally, policy formulation fosters academic-industry collaboration as a way to facilitate technology transfer from the academic/research institutions to the industry where research results are applied in practice. Successful CoEs have developed strong working relationships with these agencies to implement policy, but also to shape it.

It is essential for new and existing CoEs to ensure close alignment with funding agencies and national research and innovation agendas. For example, one CoE was aligned with a national digital transformation agenda. As part of the transformation process, the CoE was charged with the research and development initiatives for the CoE of a specific sector of national importance. There could arise a considerable number of funding issues, where a CoE interest fails to align well with the national research agenda that is pursued by the funding agencies.

4.1.3 Societal

Citizens or civil society communities play an important role within the external environment of a BDAI CoE. Social, political and cultural values influence the progress of scientific research and technological innovation in society (Bijker et al. 1987). The state of a societal environment around a BDAI CoE can be assessed using frameworks produced by organisations such as the Organisation for Economic Co-operation and Development (OECD) or the United Nations (UN). In this regard, we use the following three indices: the Human Development Index (HDI), the Global Competitiveness Index (GCI) and the Global Innovation Index (GII).

Societal in the context of the BDAI CoE framework is defined as follows:

“The societal environment of a BDAI CoE comprises the state of human development as measured by composite statistics and indexes, and the national priorities for human development regarding the UN Sustainable Development Goals and H2020 Societal Challenges.”

There is a feedback loop between the societal influence on a CoE and the impact of the CoE’s output on the society. Society influences a CoE through various policies and research agenda directives. The societal influences on a CoE include the existing science and technology goodwill of a country, the ability to attract high-level research expertise and industry, and the ability to harness the available expertise and research output. The presence of more expertise and companies enables research institutions to produce higher-quality outputs that are driven by the demand for the output and the availability of quality skills. The identified interdependence between society and research institutions works systematically to sustain the research environment as well as the industrial environment. In this sense, the industrial or corporate entities serve as the user entities for research output, as well as research collaborators providing the problems and challenges for which solutions need to be designed.

Thriving research organisations prioritise the publication of research output, attend international science and technology conferences, and get involved in collaborative research contracts or projects. These are avenues that publicise the inventions of a CoE and add to its popularity, helping it to stand out from the crowd. The CoEs within our study had an excellent national and international record of performances in science and technology development initiatives. The CoEs support the countries’ rise in the Global Indicators, which creates a positive feedback loop by attracting an inflow of personnel and companies which further drive quality output.

4.2 Strategic Capabilities

The strategic capabilities of the framework include strategy, governance, structure, funding, people and culture.

4.2.1 Strategy

Strategy in the context of the BDAI CoE framework is defined as follows:

“The means by which a CoE intends to achieve its overall mission and goals.”

A dynamic and innovative research environment has a clear and visible strategy which has been formulated by a senior research and management group (Schmidt and Krogh Graversen 2017). Successful CoEs have well-defined, distinct, narrow-ranged research areas which are unique in their region (or country) (Schmidt and Krogh Graversen 2017). The strategy of a CoE is not limited to corporate body management activity. Unlike companies which define their future goals and can independently plan how they achieve them, CoEs often have research agendas handed down to them by funding agencies in a top-down approach. This commonly results in a situation where CoEs force severe performance challenges, which can create occasional conflicts of interest between a CoE and its funding partners and host university or affiliated educational institutions. The management act of strategising is needed to define goals to be pursued by the CoE and to plan ways to achieve them. Prioritising strategic goals is critical to make the best use of available resources and create a focus on the mission of the CoE.

The BDAI CoE study discovered that the strategy design processes in the studied CoE cases were similar. For example, in all cases studied, the management

  • clearly defines and lists its strategic goals and objectives

  • tries to align the CoE’s strategic goals and mission with national (and European) research goals and objectives

  • is market-focused and directed towards industry challenges and demands when designing its future goals and objectives

  • is working hard to attain knowledge development and technology transfer to the industry through collaboration with industry partners

On the other hand, there are specific approaches that are different in the case studies.

For example, some CoEs carry out widespread consultations to gather information to formulate strategies. Such consultations included dialogue with stakeholders in the research ecosystem and with their staff, and research and funding organisations and affiliated educational institutions.

Some CoEs break down strategic goals into manageable objectives or activities and use Key Performance Indicators (KPIs) to measure performances towards objectives, goals, mission and vision. These KPIs cover impact areas including economic, commercialisation and academic, and they are operationalised.

Applied CoEs focus on developing a robust interface with industry partners. This approach helps the CoEs to:

  • Identify technical, social and cognitive barriers to use of technology

  • Define, reinforce and maintain mutual understanding and a shared vision with industry partners

  • Track evolving technologies and challenges

  • Establish new industrial collaborations

Through this approach, CoEs can identify constraints in existing tools, identify opportunities for changes to transform end-user work practices, and transfer knowledge and expertise via a feedback loop in the innovation cycle. This end-user knowledge allows them to engage in industrial projects and to justify continued basic funding from funding agencies.

Finally, decision-making through consensus of all members at the CoEs on major matters requires holding several meetings and using procedures to prepare and anchor decision-making and to run processes that enable achievement of a consensus.

The BDAI CoE study reveals that in the strategy design process, CoEs consider the following factors in the definition and design of future goals, objectives and priorities:

  • Design strategic goals to align with the national research agenda

  • Design strategic goals to align with market demands and trends, bearing in mind future needs and developments

  • Break down overloaded research agenda from funding institutions into strategic goals and objectives

  • Break down goals into manageable activities and measure each with KPIs

  • Operationalise the KPIs into daily activities

Strategy Formulation

A broad dialogue is necessary to design robust strategies for a CoE. The formulation of the strategy needs to go beyond the senior management group and be inclusive of all stakeholders, including researchers and students. The process of soliciting contributions to strategy design needs to be all-inclusive. For example, one CoE holds an annual general strategy meeting to gather ideas from everyone on how to advance the CoE. It is also crucial that the strategy formulation opens a dialogue with industry stakeholders, host university(s) and researchers from the broader ecosystem. This dialogue with stakeholders is regarded as very important for a CoE’s future success as it offers the stakeholders an opportunity to articulate their priorities. For instance, some stakeholders may prefer the development of an international profile, while others suggest the development of national and local priorities.

Alignment of KPIs with Strategy

As part of the strategic initiatives of a CoE, the management should strive to design KPIs to measure the performances of their organisation towards the set goals.

In this sense, the CoE’s management should operationalise some clearly defined strategies by formulating them into objectives that are measurable using properly designed KPIs. The measurement of those KPIs should be on a regular periodic basis, for example quarterly, bi-annually or annually.

4.2.2 Governance

Governance in the context of the BDAI CoE framework is defined as follows:

“The means by which a CoE achieves decision-making and operations.”

Joynson and Leyser (2015) propose a set of good research practices for high-quality science regarding research governance and integrity, which include training in good research practice, openness about the consequences of misconduct, and adoption of appropriate ethical review processes.

Core to the effective governance of a CoE is a strong governance body and management team. The governance body of a CoE can go by a range of names, which include Governing Council (GC), Centre Steering Committee (CSC) or General Assembly (GA). The composition of the governing body usually consists of both internal and external members. Internal members typically include the CoE’s Director or Chief Executive Officer (CEO) and a few top-level officials which could be both academic and non-academic staff. External members can be drawn from industry partners. Despite the similarity in the composition of the governing body, differences exist to some extent. For example, some CoEs include an independent observer, an official from the Technology Transfer Office (TTO), or members of governmental departments.

The governing body of a CoE holds regular meetings, about twice a year in some CoEs and up to three or four times a year in other CoEs. Some CoEs use a Strategy Board to complement the activities of the governing body. The Strategy Board is charged with the responsibility of drafting the strategic goals as well as overseeing the day-to-day operations of the CoE. These boards are composed of the top leadership personnel of the CoE. Often CoEs maintain an Executive Team and together with the CEO of the CoE report the CoE’s operations to the GC. In reverse, the GC disseminates its information through the Executive Team to the general members of the CoE. This approach is bottom-up and top-down information dissemination.

The management team of a CoE needs to plan and coordinate research activities, define and prioritise research target areas, and emphasise research productivity and quality (Schmidt and Krogh Graversen 2017). The management team should lead by example by supporting high ethical standards and paying attention to the responsible conduct of research. They should ensure policies that promote being the “best” within the scientific enterprise, and within a context that encourages responsible conduct (Schmidt and Krogh Graversen 2017).

In general, the governing body has the role of making the top-level decisions and approving the strategic goals, objectives and priorities of the CoE. Whatever the composition is, there is a significant value that each member brings to the governing board. For example, an independent observer assumes the role of suppressing biases in judgements or dealing with areas of conflict of interest during decision-making processes. Similarly, the role of the Principal Scientific Investigator in the governing body is to introduce ideas from an in-depth research point of view, which is necessary for delivering research targets.

The bottom-up and top-down information dissemination approach is useful in ensuring accountability, contribution to the decision-making process and an allowance for general inclusivity. It also enables the governing body to monitor the CoE’s performances through KPIs.

4.2.3 Structure

Structure in the context of the BDAI CoE framework is defined as follows:

“How a CoE is designed in terms of levels, roles, units, decisions, and accountability.”

An appropriate CoE structure depends on the type of institutions and the level of decision-making, as defined by Bleiklie and Kogan (2007)

  • CoE as a “republic of scholars”: Leadership and decision-making is at the collegial level by independent scholars.

  • CoE as a “stakeholder organisation”: (1) Institutional autonomy is considered a basis for strategic decision-making by leaders who are assumed to see it as their primary task to satisfy the interests of major stakeholders. (2) The interests of other stakeholders therefore circumscribe academic freedom.

Schmidt and Krogh Graversen (2017) identified that dynamic research environments have flexible organisational structure which may consist of a core researcher group and some attached members or affiliates. Successful CoEs have an organisational structure with high adaptability to internal and external changes.

One of the most critical findings in the case studies is that the structure of a CoE is designed to ensure representation of stakeholders, including host institutions (or affiliate educational institution), industry partners, funding agencies and key staff of the CoE. The structure is designed to facilitate operations and support decision-making and governance that enables coordination and integration of the activities of the CoE for consistency and synergy.

In the design of the structure of a CoE or in guiding the evolving features of the structure, it is important to consider the size of the CoE and the scope of activities. It is also essential to consider the interdependency of the various roles that must work together to optimise resource utilisation to maximise outcomes. Structures enable the efficient running of an entity – the roles, the reporting lines and the accountability for the respective responsibilities. The structure facilitates information dissemination and enforcement of rules and regulations, and thus can also play a key role in the development of suitable cultural practices.

4.2.4 Funding

Funding in the context of the BDAI CoE framework is defined as follows:

“The availability, diversity and sustainability of the monetary support for carrying out research and educational activities in the CoE.”

Funding practices for a CoE need to ensure that it is provided with sufficient funding and that it has diverse external funding sources to supplement basic research funding. Funding practices in CoEs with a focus on applied research look to secure funding in the form of collaborative or contract research, with industry partners facilitating technology transfer. Joynson and Leyser (2015) highlight two good research practices for high-quality science through the adoption of diverse funding approaches and the clear communication of funding opportunities and assessment criteria funding that are critical to the recruitment of new researchers, which is a key success factor of a CoE.

From the BDAI CoE case study, the result shows that funding models are provided in a cycle with a fixed period to address specific long-term objectives (e.g. 4, 6 or 8 years). Funding schemes come in mixed models comprising diverse funding sources. A mixed funding model pushes a CoE to explore multiple funding sources such as national, industry (local and international) and European funding sources (e.g. H2020). The industry funding sources could further be broken down into contract research with large multinational companies or with small and medium enterprises (SMEs), as well as with start-up companies. However, there are challenges involved in dealing with SMEs and start-up organisations because of their income level and undefined strategies and goals. Extra funding sources beyond the basic sources usually supplied by funding agencies can also be in the form of services delivered as consulting services by CoEs to other corporate entities or organisations in the not-for-profit sector or even educational sector. The extra funding could also come from national funders that facilitate organisations to sponsor projects financially for a CoE to execute them. In the European Commission (EC), most international funding sources come from EC H2020 and FP7 projects. Participation in projects sponsored by these funding sources in addition to collaborative research with industry partners helps CoEs to obtain extra income to augment their basic funding requirements.

A CoE’s sources of funding can be listed as follows:

  • Engagement or collaboration with industry partners in collaborative research and consultation services to industry members.

  • Participation in EU projects under mainly Horizon 2020 and also FP7 Research and Innovation projects.

  • A business development role can be used to develop better engagement capability with the industry partners, which helps a CoE to negotiate more contracts for more income.

  • Some CoEs pursue commercialisation and spinouts as well as scientific inventions and publications. However, some are more specialised in the scope of research and innovation development.

Additional funding is often needed to enable a CoE to finance specific interests that the funding agencies may not want to fund. However, funding policy requirements may pose some challenges for a CoE in that it may be required to provide a given amount of its funding needs to become eligible for funding supply from its financiers. For example, one CoE studied needs to provide up to 25% of its funding needs to be eligible for continued funding from funders. This places the management under pressure to collaborate with industrial partners even when it is not a priority to enter into such a contract.

4.2.5 People

People in the context of the BDAI CoE framework are defined as follows:

“The people required to carry out specific tasks towards the goals of the organisation.”

CoEs are affiliated to educational institutions, which appear, in most cases, to be the primary sources of personnel supply, particularly CoEs that run academic courses such as master’s, PhD and postdoctoral training. In the case of all CoEs, the host universities provide the human resources policies that guide the personnel practices in the CoE.

To gain a broader scope of expertise to bring into their CoEs, the management of research institutions advertise vacancies in both local and international fora, and this enables them to build a range of options into the selection process. CoEs also use some cultural practices:

  • To keep their people in a good state of mental health and social well-being (e.g. community volunteering programmes, excursions, walking and cycling activities). If the CoE is not hosted at a single physical site, these can be online activities (e.g. online mindfulness, virtual coffee sessions and online game tournaments, especially during the Covid-19 pandemic).

  • To help build their skill and careers through various activities—for example, lunch seminars with invited speakers and on-the-job training of partners’ workers on internship programmes. A programme at the CoE combines researchers and partners representatives for cross-fertilisation of skills, weekly meetings featuring occasional invited speakers and thesis programmes involving public speaking.

  • To help in the integration of new in-takes through mentoring programmes and to get them up to speed with others.

  • To eliminate preferential treatment (e.g. unconscious bias programme) and ensure gender equality and gender mix, programmes like staff diversity, gender equality and women’s networking, are organised. In addition, local language training programmes for non-native speakers take place.

To make people feel at home, CoEs use programmes to bring about a feeling of togetherness in a common purpose. For example, one CoE organises an International Cultural Day, which is an event where the different cultures of the various represented nationalities are displayed and celebrated, including the provision of food from various nationalities. A feeling of togetherness can also be achieved through the creation of a friendly environment, where individuals voice their concerns. This helps achieve collaboration and teamwork necessary for productivity in the CoE. Joynson and Leyser (2015) propose a set of good theoretical research practices for high-quality science. These practices include providing adequate training programmes for researchers, being open and clear about consequences of misconduct, and the adoption of appropriate ethical review processes.

4.2.6 Culture

Culture in the context of the BDAI CoE framework is defined as follows:

“The underlying values, beliefs and norms that drive the teams and the CoE as a whole.”

Culture is a critical part of the CoE. Schmidt and Krogh Graversen (2017) identified that a successful CoE has a working climate based on internalised norms grounded in a research tradition. The working environment should be open to new ideas, methods and approaches. Staff within the CoE have research autonomy during the research process. The working climate is based on teamwork with close cooperation among research staff. Finally, they identify that culture encourages internal professional and social dialogues.

The case results point to the common fact that most CoEs have a mix of local and international culture. A key question is how CoEs use cultural practices to achieve a spirit of togetherness and inclusivity that reduces conflicts, eliminates preferential treatment and maximises productivity.

The effective use of cultural practices in CoEs helps the management to mitigate problems and helps staff to attain high levels of productivity:

  • Integration of new in-take

  • Collaboration and teamwork

  • Welfare programmes

  • Researchers/staff personal skills development

  • Inclusivity and voice

  • Support for outreach

  • Elimination of preferential treatment and achieving gender equality

Culture plays a vital role in the level of interrelationship and interaction existing between people in an organisation. Culture is connected to the degree of collaboration that is possible in an organisation and has a direct impact on the success of the organisation. Culture is developed or guided to evolve into practices that support healthy sharing, caring and support of one another, a situation that enables people in an organisation to feel a sense of togetherness, giving them an opportunity to voice their concerns and contribute to decision-making processes and general shared goals. Like corporate organisations, research institutions also recognise the strong need for good cultural practices in a workplace and how to use their impact to direct success.

The BDAI CoE study reveals that CoEs use various programmes to enhance cultural practices and to make things happen in the way they are desired. For example, in the case of integration of new in-takes, some CoEs use mentoring and orientation programmes to familiarise recruits with their operation and culture. Welfare programmes cater for students and staff to make them feel valued and to get the best out of them for their success and that of the CoE. As they cooperate and collaborate to deliver for the success of their institutions, researchers in research institutions, particularly student researchers, often have personal career development needs. To compensate for their individual needs, leading research institutions provide career and personal development programmes for their workers.

4.3 Operational Capabilities

Operational capabilities in the context of the BDAI CoE framework are defined as follows:

“The operational capability is the ability of a CoE to perform a coordinated set of tasks, utilising organisational resources for the achievement of its mission and goals.”

The BDAI CoE framework identifies a set of operational capabilities needed to operate a CoE. These capabilities are detailed in Table 1.

Table 1 Core operational capabilities of the BDAI CoE framework

Capabilities maintained by a CoE are partly dependent on their areas of focus and partly conditioned by their need to meet stakeholder demands. There is a wide range of capabilities within the studied CoEs. Some of the highlights from the case studies are as follows:

One CoE exercises an elaborate plan of outreach in the form of Education and Public Outreach (EPE) programmes for which a Subject Matter Expert is employed. The elaborate EPE process is informed by the importance attached to it by the government’s interest in making the public aware and also taking advantage of science and innovation outcomes.

A CoE with an applied focus to bring the best of services and products to their industry partners adopts a practical process of demonstrating their prototypes contained in a catalogue of demonstrators, IPs and the state-of-the-art analytics and visualisation technology reviews to their partners. This capability brings research outcomes to its network of industry members to which it also delivers services such as seminars, conferences and consultation to create awareness and disseminate information to the end-users of its technologies. The process of garnering collaboration with partners uses two calls for demonstrator proposal. Later, a team filters the proposals received and rates the accepted ones. Finally, the rated proposals are decided upon by the senior management of the CoE, which makes final proposal choices.

Another CoE developed an iterative three-stage process of innovation cycle methodology called Scalable Innovation Cycle (SIC), in which the CoE carries out a user-led generation of ideas and validation of results. The CoE’s processes are highly user-centred, and hence it aligns them closely with the end-user-centric methodologies. The goal of this methodology is to combine research with real-world deployment to meet real business problems. Being iterative, SIC requires the use of a series of feedback among pilots, prototypes and experiments to identify new challenges and gaps to perfect results.

The results of these case studies show that there are various capabilities, and these capabilities tend to differ from CoE to CoE depending mostly on their strategic research domain and end-user needs. With this in mind, it is hard to pinpoint one capability as the best approach to research as there are reasons that support the use of individual capabilities in each CoE.

However, whatever capability is in use in a CoE, there is a need for it to be regularly well operationalised and measured for the desired outcome. KPIs should be designed by breaking down a capability into stages of work, and metrics should be put in place to measure performances at each stage over a given time interval or periodically.

4.4 Impact

Impact in the context of the BDAI CoE framework is defined as follows (Harland and O’Connor 2015):

“The direct and indirect ‘influence’ of research or its ‘effect on’ an individual, a community or society as a whole, including benefits to the economic, social, human and natural capital.”

The definition of the impact metrics and their measurement methods are a significant part of the impact assessment methodology. The following subsections provide guidelines from the literature on how to measure the economic, scientific and societal impact of research output. The impact on the environment and society would be seen in reports of innovation activities derived from field research about impact areas such as economic, scientific and societal. The parameters to understand impacts could be measured through the KPIs being monitored by the BDAI CoE and those monitored by the country government agencies in which the BDAI CoE is located. For example, the economic impact could be how a CoE and industry partnership or collaboration in research and technology is bringing about a measurable increase in commercial activities, companies created through commercialisation, spinouts and jobs creation, and skills development. There are reports which provide a narration of these measures for the government and government agencies to use in support of policymaking for performance review and educational purposes.

4.4.1 Economic Impact

Economic impact in the context of the BDAI CoE framework is defined as follows:

“The economic impact is the effect on commerce, employment, or incomes generated from Big Data and AI research in general and by the CoE in particular.”

As described in Adams (2016), the examples of best practices for the assessment of economic impact are:

  • Funders need to be sure that job creation is reported consistently across multiple organisations so researchers need an agreed standard such as “full-time equivalent jobs created” to avoid counting part-time roles.

  • Claims of impact remain assertions unless there is an independent validation of impact evidence.

  • Evaluators require an audit trail to use impact data for evaluation purposes.

  • Impact evidence must be collected over time, attributing each impact to original research or expertise and tracing the developing sequences of activities.

  • Evidence types can vary widely depending on the discipline, the stakeholders and the changes that have occurred.

  • Impact evidence can include quantitative reports of increased sales for a commercial stakeholder or quality of life improvements.

  • Qualitative testimonials can directly attribute changes to the research, or the contributions made by researchers because of their expertise.

  • Impact information needs a standard structure and categorisation.

A digital research report by Digital Science & Research Ltd. that was released in March 2016 suggests the following best practices for a Research Excellence Framework to improve both the quality and value of future CoEs (Adams 2016):

  • To ensure that the full range of meaningful impacts can be recognised, consider extending eligible periods both for impacts and for the research on which they were based.

  • Require listing of funders and grant references in the case study template.

  • To aid assessment and further use, consider developing guidance on certain types of evidence where appropriate, e.g. sales, staff numbers, company investment.

  • Where possible, re-use information from other systems, e.g. ORCID.

4.4.2 Scientific Impact

Scientific impact in the context of the BDAI CoE framework is defined as follows:

“The scientific impact of a CoE is the returns on research investment assessed qualitatively or quantitatively within the academic sphere.”

The assessment of the scientific impact of a CoE helps funding agencies to evaluate returns on research investment from a research impact perspective. The scientific result can be assessed qualitatively or quantitatively. An analysis carried out by Sutherland et al. (2011) identifies the following practices for quantifying the impact and relevance of scientific research:

  • Qualitative approaches: This approach involves expert panels evaluating impact, for example as high, medium or low, based on written descriptions of impact.

  • Quantitative approaches: This approach involves numerical indicators derived from scoring systems or questionnaires focused on the various possible impacts of a research programme or project.

4.4.3 Societal Impact

Societal impact in the context of the BDAI CoE framework is defined as follows:

“The societal impact of a CoE is its impact on human lives and health, organisational capacities, societal behaviours and the environment.”

A variety of frameworks and models are proposed to quantify and measure societal impact (Penfield et al. 2014; Bornmann 2013; Sutherland et al. 2011). Such a variety of frameworks might also be reflected by the impact assessment methods adopted by national funding agencies across Europe. Regardless of the specifics of assessment tools or methods, the underlying objective of assessing societal impact is to understand the social externalities of research and innovation activities undertaken in a BDAI CoE.

Impact on the environment and society can be captured by reporting activities which are conducted by several agencies such as the United Nations Human Development Index (UNHDI), GCI, GII, Knowledge Impact (KI) and Knowledge Fusion (KF) rankings agencies or organisations. These rankings are measurements that also categorise measures into impact areas such as economic, scientific and societal. The parameters to understand impacts could also be measured through some KPIs being monitored by the individual BDAI CoE, on the one hand, and those monitored by the research-funding agencies and other government agencies of the country in which the BDAI CoE is located, on the other hand.

Societal impact can be reached through various practices that CoEs can adopt to influence the relationship between research and society (non-academic community). Societal impacts, as defined by Molas et al. (2002), are part of a conceptual framework for analysing third-stream activities and categorised as follows:

  • Research CoEs have capabilities in two main areas: (a) knowledge capabilities and (b) physical facilities. These capabilities are developed as CoEs that carry out their core functions of teaching and research.

  • Using the means at their disposal, CoEs carry out three main sets of activities; they (c) do research, (d) teach, and (e) communicate the results of their work.

The type of economic impact a CoE has on the economy in which it exists is dependent on the research areas it specialises in and how that drives economic output. For example, a large-scale CoE may have broad research areas which cut across data analytics applicable in many domains such as media analytics, optimisation and decision analytics. It also participates in other domains such as personal sensing, sustainable IT, e-government, machine learning and Semantic Web. On the other hand, a CoE may have a narrower domain focus with industry-centric capability for producing various data analytics and visualisation tools. Centres may also focus on a single industrial domain. The visible outcome of a CoE does not depend entirely on its output because it also depends on the amount of publicity the CoE has provided on its scientific outcome. Publicity on a CoE’s research result is essential in that it helps to create public awareness (locally and internationally) and attract partners for collaboration, creating an avenue for technology transfer.

Conversely, collaboration opportunities previously involved have the potential to bring more opportunities to the CoE because previous engagements serve as an opening for further engagements. This is the cyclical aspect which calls for adequate investment in various ways by which research output can be publicised, and it should include the national agenda of the country in which the CoE is located, as well as the funding agencies’ contribution towards publicity and exposure to opportunities. Many countries have put in place policies to drive outreach activities from CoEs to the public, while individual CoEs also make an effort to get involved in presentations at conferences as well as sending entries to scientific publications. Another important consideration for impact is the quality of research output. Good-quality and innovative research output sells itself while bad results fail. This would therefore be a good reason to invest in world-class researchers and infrastructure, in addition to a continuous study of the trends in the markets both in the local and international environment.

Scientific impact is constituted by additions to the state of the art in science and technology which are made known to the public through publications in scientific journals and conferences, as mentioned above. A culture of documentation of research processes and findings on a regular basis can help provide information necessary for preparing articles on the outcome of research endeavours. Documentation should be given priority in research exercises not only for project purposes but also for article writing and presentation at scientific conferences. Societal impact is linked to economic impact with the use of research outcomes in the industry, thereby creating new companies, jobs and economic values which benefit the entire society. Also, societal impact refers to the direct benefit derived by people when they use technology items and when technology helps to create better conditions around them, e.g. reduction in poverty levels and crime and disease control and prevention, as well as helping humanity sustain a greener environment in any way possible.

4.4.4 Impact Measured Through KPIs

Whichever category an impact belongs to, it can be measured through specific indicators that can capture perceivable improvements due to the outcomes of a CoE. KPIs (as described in Table 2) are basic indicators that can be measured with defined metrics designed to provide measures of benefits produced regarding economic, scientific and societal advantages. For example, in Ireland, the principal research financing agencies, such as SFI, EI and IDA, have together developed a set of KPIs to measure research performances and their impacts on the nation’s goals based on their research outcomes. SFI demands that a research centre’s targets be ambitious and achievable and reflect the strategic and commercial positioning of the centre. The centre’s targets will therefore be part of the basis for evaluation of the centre’s proposal. Also, funded centres’ metrics will be reported against defined KPIs and evaluated against the targets on an annual basis (Roche et al. 2013). SFI selected 13 KPIs and used these to score each centre under relevant performance indicators and targets broken down into four categories: academic outputs, human capital outputs, funding diversification and commercialisation. All of these must be aligned with the objectives of the research centres’ programmes as well as the overall SFI objectives per Agenda 2020.

Table 2 Sample impact KPIs

SFI evaluates a research centre’s performance periodically using evaluation instruments such as the Metrics Governance report and balanced score card, the annual report of the centre, the annual census report (including financial reporting) and site visitations with the external panel (Roche et al. 2013).

5 How to Use the Framework

An assessment of how the capabilities are contributing to the CoE’s overall goals and objectives has taken place. This gap analysis between what the CoE wants and what it is achieving positions the framework as a management tool for aligning the operational capabilities of the CoE with its objectives.

The framework focuses on the execution of two key actions:

  • Define the goal and posture of the CoE.

  • Develop and manage the CoE’s strategic and operational capability over time.

Here we outline these actions in more detail and discuss their implementation.

Defining the Scope and Goal

First, the CoE must define the scope of its efforts. Agreeing on the desired posture (from basic to applied research) has a significant impact on the CoE and thus on its goals and priorities. Second, the organisation must define the goals of its effort. It is important to be clear on the CoE’s objectives and the role of its capabilities in enabling those objectives. Having a transparent agreement between the internal and external stakeholders of the CoE can tangibly help achieve those objectives.

Develop and Manage Strategic and Operational Capabilities

Once the scope and goals of the CoE are clear, the CoE must identify its current capabilities by examining across its different operational and strategic functions. This helps the CoE to have a clear view of its current capabilities. Comparisons with the best practices identified within the framework can help identify key areas for action and improvement. To develop capability over time, the CoE should:

  • Develop a roadmap and action plan

  • Add a yearly follow-up review of capabilities to ensure their fitness for purpose and alignment with the CoE’s objectives

Agreeing on stakeholder ownership for each priority area is critical to developing both short-term and long-term action plans for developing and improving capabilities.

The decision to use the BDAI CoE framework to improve operations of a CoE should not involve re-inventing the wheel. The concepts it contains have been theorised and applied extensively by many successful CoE organisations. These concepts and the manner of implementing them can be harnessed to support the development and growth of big data and AI-oriented research entities. The plan to use the BDAI CoE framework may need to incorporate an enhancement of the operations of existing big data and AI CoEs, including the manner of drafting the strategic direction, seeking funding, collaboration, information dissemination and outreach practices. By considering the elements of the BDAI CoE framework, which include strategy, governance and structure, funding, culture and capabilities, it is clear that appropriate practices under each of these elements may need to be (re-)designed into the activities and the general operations of a CoE may need to be performed in achieving the strategic goals of the CoE.

The management team needs to evaluate all factors in the framework, such as environmental, industry and societal, which have a significant influence on the way a CoE may be run. They should consider the needs of its “customers” to know what is currently in demand as well as industry trends. Within such a competitive research and innovation landscape, the management team must decide on the specific value direction the CoE must explore so they can guide the process of resource allocation and talent development or recruitment.

5.1 Framework in Action

The framework is being used by a number of CoEs which contributed to its creation. The CoEs use the framework in different ways, from the training and onboarding of staff to planning the design of new or enhanced organisational capabilities. The framework has also been used to guide the creation of a new CoE. The GATE projectFootnote 1 was a Horizon 2020 WIDESPREAD-2016-2017 TEAMING Phase 1 programme that aspired to create a sustainable business plan for the creation of the first CoE in big data in Bulgaria. The purpose of this big data centre is to produce excellent science by seamlessly integrating related fields and associating complementary skills. GATE aspired to add value to knowledge, to strengthen the capacity of researchers, to educate and train early-stage researchers, to disseminate and promote projects, and to achieve international visibility and scientific as well as industrial connectivity. With innovation pillars like data-driven government (public services based on open data), data-driven industry (manufacturing and production), data-driven society (smart and sustainable cities) and data-driven science (big data technology stack in the scientific community), GATE had set its aim high to fulfil its goal.

The framework was used in an advisory capacity in the GATE project by sharing best practices at several meetings and workshops. The framework also supported the CoE to determine their research strategy and business plan. Overall, very positive feedback has been received by the GATE as they built the first big data CoE in Bulgaria, paving the way for more CoEs to start spreading in Eastern Europe in the future. In the words of Professor Sylvia Ilieva (Director GATE CoE) the framework helped the centre “in difficult very first steps of structuring and organisation [and] guided the building of GATE sustainable model on the collective experience and best practices”. She continued that it “helped at specialising [the] GATE mission and focus to be complementary, but competitive to the other 55 Centres in Western Europe”.

6 Critical Success Factors for Centres of Excellence

Critical success factors are a range of key enablers that CoEs, like corporate bodies, employ to achieve success in their operations. While some are very easily identifiable, e.g. funding availability and a mix of employees’ capabilities and cooperation, other success factors are not quite salient, e.g. the role of culture in the success of a CoE. Similarly, some success factors are common to a majority of CoEs, e.g. the importance of enough funding towards success, possession of world-class researchers, collaboration with important partners and output publicity. Other factors are peculiar to individual CoEs because certain factors apply to the research focus of a CoE. However, whatever the key success factor is, it is the responsibility of the management team to identify it early enough and to harness it to drive success in the required direction.

This section reports the findings of the BDAI CoE case studies as success factor recommendations for existing CoEs and potential ones for their research operations. These factors are gathered from interviews with the CoEs’ senior management using a series of open-ended questions:

  1. 1.

    What are the common difficulties faced by the CoE in achieving its objectives?

  2. 2.

    What factors contribute to/enable the success of the CoE?

  3. 3.

    What are the typical mechanisms deployed to address success factors and challenges in the CoE?

  4. 4.

    What would you need to do to be more successful?

Challenges are the drawbacks to the progress of any organisation, while the success factors facilitate progress. Therefore, the management team of an organisation, according to its mandate, has to devise strategies and practices to eliminate or at least mitigate challenges and other risks to success. Success factors can be leveraged to drive the development of capabilities to meet the CoE’s goal.

6.1 Challenges

The key challenges identified in our interviews are detailed in Table 3. They are aligned to the related strategic or operational capability. The list does not have an order of priority.

Table 3 Summary of challenges

6.2 Success Factors

The factors with which the CoEs’ leadership contribute to their success are detailed in Table 4. They are aligned to the related strategic or operational capability.

Table 4 Summary of success factors

6.3 Mechanisms to Address Challenges

The mechanisms deployed by the CoE’s leadership to address their challenges are detailed in Table 5. They are aligned to the related strategic or operational capability.

Table 5 Summary of mechanisms to address challenges

6.4 Ideal Situation

According to the CoEs’ leadership, the ideal conditions for the operation of their CoEs are detailed in Table 6. They are aligned to the related strategic or operational capability.

Table 6 Summary of ideal situations

7 Summary

This chapter presented a best practice framework for the operation of BDAI CoE. The goal of the framework is to foster collaboration and share best practices among existing centres and support the establishment of new CoEs within Europe. The framework was developed following a phased design science process, starting from a literature review to create an initial framework which was enhanced with the findings of a multi-case study of existing successful CoEs. The chapter outlined how the framework can be used by a CoE to support its strategic direction and operational decisions over time, and how a new CoE can use it in the start-up phase. Based on the analysis of the case studies, the chapter explored the critical success factors of CoEs as defined by a survey of CoE managers.