The Role of Network Topology and the Spatial Distribution and Structure of Knowledge in Regional Innovation Policy: A Calibrated Agent-Based Model Study

Abstract

Recently, new regional innovation policy paradigms emerged that transcend a long-lived dispute on whether either regional specialization, diversification, or rather related variety is most conducive to regional innovativeness. This includes ‘smart specialization’ in which regions are deliberately specialized and connected following technological relatedness, ‘gatekeepers’ in which there are pipelines between dense regional networks, and a ‘regional mix’ of knowledge that allows sustained ‘branching’. We develop and use an agent-based model to study the conjecture that, to stimulate (supra)regional innovativeness, social planners need to consider both, in conjunction, the mix of technological knowledge in regions and the (inter)regional innovation network topology. We use this agent-based model to evaluate the performance of and study the internal mechanisms of these new policy paradigms in a variety of scenarios. To increase the external validity of our findings, we calibrate the knowledge graph searched by the agents in our model to the OECD patent database. We confirm that access to related variety is important, yet that, on top of that, access to incidentally related knowledge is crucial to prevent high-level lock-in and ensure long term technological progress. Moreover, we find that networks with regional gatekeepers are particularly innovative, because these gatekeepers form ‘knowledge hubs’, create short paths to potential partners, and enlarge the total pool of knowledge. In case agents have few relationships, we find exceptionally high performance for the gatekeeper network in combination with regional diversification. The smart specialization network is a solid second option, although it lacks access to incidentally related knowledge and thus will ultimately fall behind whenever agents have relatively few relationships. The study elaborates on specific scenarios to reveal intricacies in the relationship between knowledge distribution, network topology, and the structure of interrelationships between knowledge fields.

1 Introduction

In the knowledge-based perspective on innovation economics literature, there is an ongoing debate on whether technological specialization or rather diversification of research and development activities is conducive to regional innovativeness (Glaeser et al. 1992; Panne 2004; Paci and Usai 2000; Feldman and Audretsch 1999), or, as an additional alternative, whether regions should contain a mix of technologically related research activities (cf. Balland et al. 2015; Bristow 2010; Frenken et al. 2007). Taking technological discoveries as due to finding a suitable combination of existing knowledge (cf. Arthur 2009), firms in diversified regions face many unrelated knowledge flows and ‘infeasible’ combinations, thus lowering dynamic efficiency, while, in contrast, firms in specialized regions may be dynamically efficient but may not have access to technological path-breaking combinations and thus face a lock-in (Menzel and Fornahl 2010; Saxenian 1996; Hassink 2005; Martin and Sunley 2011). Arguably, this body of literature may overemphasize the regional character of knowledge flows. While actual co-location (and tighter governance) adds to the efficiency of product definition, knowledge recombination, and diffusion (Maskell and Malmberg 1999; Pinch et al. 2003), organizational proximity of innovation activities compensates for geographical distance (Capaldo and Petruzzelli 2014). As such, arguably, to enhance regional innovativeness, a social planner¹ should not only look at the composition of the regional knowledge, but also at the (possibly interregional) topology of innovation networks and notably whether there is collaboration with agents active in different technological fields (cf. Grant and Baden-Fuller 1995, 2004; Pyka and Küppers 2002; Pyka 2002). In this paper, the contention is that the regional specialization versus diversification versus related variety debate (and the related purely regional innovation policy paradigms) cannot be settled without taking the (inter)regional innovation network into the consideration. Regional innovation systems feature a mix of public and private, intra- and inter-industry, and local and global knowledge flows (Autant-Bernard et al. 2013; Camagni and Capello 2013). Following these insights, social planners were confronted with new regional innovation policy paradigms. In this paper, we focus on three prominent ones. Firstly, the ‘smart innovation’ policy paradigm (Foray 2014; Foray et al. 2009, 2011) argues that a social planner, such as the European Commission, best prioritizes technological development in regions in particular technological fields, building upon existing strengths, and assigns other regions ‘co-innovator’ roles with specializations in other fields. In following the smart specialization paradigm, the social planner thus deliberately creates a patchwork of complementary, specialized regions connected through an interregional network. This network is crucial as, under fully fledged regional specialization, realizing path-breaking inventions requires access to alien technological knowledge that may be located in other regions (Bathelt et al. 2004; Rosenkopf and Almeida 2003). Secondly, the ‘branching’ policy paradigm (Asheim et al. 2011; Boschma 2011) [and related ‘regional resilience’ paradigm (cf. Balland et al. 2015; Bristow 2010)] argues that regions may pick particular mixes of technological fields that can continue to branch in a self-sustained manner, independent of knowledge from outside the region. Thirdly, in the ‘regional gatekeeper’ literature, a more specific network form is prescribed. Hereby innovation networks are regionally dense, and there are regionally central gatekeepers which monitor and filter technological knowledge in other regions to then diffuse that knowledge within their own region (Graf 2011; Graf and Krüger 2011; Spencer 2003; Breschi and Lenzi 2015; Bathelt et al. 2004).²

The research question now is: given a particular regional distribution and interrelationship of technological knowledge, notably of specialized fields, which network topology should a social planner pick to enhance technological progress in these regions, on average? In fact, we are able to prescribe the combination of knowledge distribution and network topology which a social planner should pick. To transcend the ongoing qualitative arguments and conflicting empirical findings, we define and use an agent-based model to experimentally investigate the relationship of specialization/related variety/diversification and intra-/inter-regional innovation network topologies (and notably the smart and gatekeeper network) in conjunction. Given that such an agent-based model provides full control over the topology of the network, we study combinations of spatial knowledge distribution and (inter)regional innovation networks which are not discussed in literature. Moreover, the agent-based model also allows us to study the number of relationships of each of the agents on technological progress. Ultimately, the agent-based model enables us to get insights in mechanisms at work on the complex interplay of spatial distribution, technological interrelationships, and network topology, and insights in the performance of policy paradigms followed by social planners.

In addition to addressing the topical debate on which regional network innovation policy is to be preferred, we extend the debate. Our claim is that the significance of collaboration with firms in other fields and in other regions depends on structural features of the underlying technological knowledge and the distribution of knowledge over regions. In preceding simulation studies, we revealed how collaboration of agents in different specialized regions is predominantly significant if technological knowledge accumulation is ‘progressive’ (i.e. not building upon own ancestors, which would be ‘conservative’), which is the case for some technological fields more than for others (see Vermeulen and Pyka 2014b, a). In these studies, however, the technological structure is highly simplified such that conclusions have limited external validity. In the study at hand, the structural features of the technological knowledge searched by the agents in the model is empirically calibrated to citation and classification statistics of patents in the OECD patent database. In the spatial agent-based model presented here, the world consists of sea and land regions where a (controlled and fixed) number of firm agents resides in each land region. Each agent is specialized in one technological field, is engaged in knowledge discovery, and has a number of relationships with agents with whom it can (but need not) collaborate in knowledge discovery. The specializations of the agents in each region as well as the topology of the innovation network are now independent variables and the absolute ‘amount’ and ‘advancedness’ of knowledge discovered are studied. We see this paper and its model as the first step in a research line in which various regional innovation network policies are studied for realistic knowledge structures while controlling innovation system properties.

The structure of this paper is as follows. In the following section, we provide an overview of the literature on (inter)regional innovation policy paradigms and notably their preoccupation with knowledge specialization/diversity and with the role of (inter)regional innovation network relationships. As our principal claim in previous work already was that the structure of the knowledge determines which (inter)regional innovation network structure is most efficient in technology development, we subsequently discuss existing ‘technology discovery models’.³ We then specify our own ‘technological knowledge graph’ model, how it is generated and calibrated to the OECD patent database, followed by an operational definition of the spatial agent-based simulation model and the search heuristics followed by agents in searching and unlocking new parts of that knowledge graph. This is followed by a section with an overview and discussion of simulation results. The last section provides conclusions.

2 Economics of Innovation: The Spatial and Network Dimensions

In the knowledge-based perspective of the firm, it is dynamically more efficient to conduct research in collaborative rather than in integrated or market governance forms (Grant and Baden-Fuller 1995, 2004). This is corroborated by the fact that strategic alliances (Hagedoorn 2002) and innovation networks (cf. Pyka and Küppers 2002; Pyka 2002) have emerged as persistent forms of organizing research and development of new technologies. Given that technological knowledge with a substantial tacit component (cf. Polanyi 1966; Johnson et al. 2002) is conveyed and combined more efficiently when done face-to-face (Gertler 2003), one expects and also observes geographical clustering of vertically specialized but technologically related firms active in the same industry (Feldman 1994). Moreover, firms active in the same industry tend to co-locate to find specialized component suppliers, to share a pool of skilled labor, capture (unintended) knowledge flows, and reap localized scale economies: the so-called Marshall–Arrow–Romer externalities. Not surprisingly, scholars of economics of innovation focus on the regional nature of innovation collaboration (e.g. Cooke 2001; Asheim and Coenen 2005; Bathelt et al. 2004; Neffke et al. 2011; Karlsson and Stough 2005; MacKinnon et al. 2002; Vermeulen and Paier 2016). Consequently, popular regional innovation policy paradigms (including also new paradigms such as smart specialization and branching, discussed later) have in common that innovation-based economic growth is stimulated by formation of technologically specialized clusters (e.g. Cumbers and MacKinnon 2004; Boschma 2014). It is acknowledged, though, that technologically specialized regions (and, notably, ‘industrial districts’) tend to focus on merely incremental extension of the locally supported product designs. Consequently, a region (and the local networks contained in it) may thus fall behind others if they fail to absorb, imitate, or leapfrog the technology developed elsewhere (Menzel and Fornahl 2010; Saxenian 1996; Hassink 2005; Martin and Sunley 2011). Other authors argue that regions should, rather than exploit the Marshall–Arrow–Romer externalities, stimulate technological diversity to stimulate innovation. The argument is that (path-breaking) innovation comes about by cross-fertilization of different technological fields. For an in-depth discussion of the conflict between these two bodies of literature see Glaeser et al. (1992), Panne (2004), Paci and Usai (2000), Feldman and Audretsch (1999), and the possibly methodological causes of this conflict, see Beaudry and Schiffauerova (2009).

In recent literature, this perspective is refined in two directions. Firstly, the related variety literature (Frenken et al. 2007) argues that diversity is not beneficial per se. It is only the case if two or more knowledge bases can be fruitfully combined. Problematic is that it is not known a priori whether or not particular knowledge bases can be fruitfully combined into path-breaking innovations (cf. Castaldi et al. 2015). Policy interventions may of course ensure sustained presence of fruitful multi-industry knowledge diversity (cf. Balland et al. 2015; Bristow 2010). Note that, purely from the perspective of regional growth, a diversified region is more resilient to shocks to or falling behind of one of the industrial activities (see Christopherson et al. 2010, for an extensive discussion of resilience). The ‘regional resilience’ paradigm calls for a balanced, diverse portfolio of sectors to prevent a region from getting into a technological lock-in and regional economic stagnation. Regions with particular combinations of sectors may allow for sustained creation of opportunities for cross-fertilization, by ‘branching’ into new but technologically related directions (Asheim et al. 2011; Boschma 2011). Regional innovation policy aiming for regional resilience may stimulate a ‘smart mix’ of related fields providing options for sustained branching.

Given that it may not be known a priori what other technological knowledge base gives rise to fruitful cross-fertilization, the, in retrospect, unrelated variety in a diverse region may hence be a source of dynamic inefficiency, while the positive agglomeration externalities of regional specialization are not reaped. That said, being co-located in a region is not sufficient nor necessary for innovation to take place (Boschma 2005): a channel of communication has to be present and all parties have to be willing to exchange and cross-fertilize knowledge, which can then take place regardless of the location. Moreover, so we add, diversity in the region need not impair innovation efficiency as long as the agents active in unrelated fields are not connected in the innovation network. While actual co-location (and tighter governance) would add to the efficiency of product definition, knowledge recombination, and diffusion (Maskell and Malmberg 1999; Pinch et al. 2003), organizational proximity of innovation activities compensates for geographical distance (Capaldo and Petruzzelli 2014).

Innovation policy literature stresses that there is no one-size-fits-all regional innovation policy, but needs to be tailored to circumstances specific for the region (cf. Tödtling and Trippl 2005). Each regional innovation system calls for its particular mix of public and private, intra- and inter-industry, and local and global knowledge flows (Autant-Bernard et al. 2013; Camagni and Capello 2013). Moreover, given regional disparities, it makes sense for a social planner such as the European Commission to differentiate the extent and direction in which innovation is stimulated in the various regions. The ‘smart innovation’ concept (Foray 2014; Foray et al. 2009, 2011) elaborates on this idea in that a social planner, such as the European Commission, best prioritizes technological development in particular sectors, thus deliberately creating a patchwork of complementary, specialized regions. Based on existing competitive advantages and knowledge strengths, regions are stimulated to specialize in particular fields, whereby regions specialized in other fields are designated to support innovative activities in the focal regions. A necessary complement to this patchwork of specialized regions is a network of research collaborations between organizations in different regions following technological relationships. Extending Camagni and Capello (2013), we define smart innovation network policies as those policies that seek to “enhance local knowledge production by exploiting existing local knowledge specificities through intra- and interregional research and development relationships”.

Literature with an innovation network perspective on regional development argues that regional lock-in may be overcome by ensuring that local agents have relationships with agents in other regions to acquire and absorb (only) relevant knowledge otherwise unavailable to them. In literature, there is a particular interest for so-called gatekeepers (Graf 2011; Graf and Krüger 2011; Spencer 2003; Breschi and Lenzi 2015) or pipelines (Bathelt et al. 2004). These gatekeepers function as filter for unrelated knowledge available outside the region and translate and diffuse related knowledge in the regional network. Generally, we take the purpose of innovation networks to, on the one hand, connect pools of technological knowledge, regardless of the geographical distances, and, on the other hand, increase efficiency by limiting the relationships to those technological relevant. In this perspective, the topology of innovation networks may overcome ‘pollution’ with technologically unrelated knowledge in diversified regions as well as overcoming the lock-in in specialized regions devoid of technological related knowledge. Stimulating innovation and regional economic development combines interventions on either side of the continuum: (i) enhancing innovation networks by facilitating access of (supposedly) relevant technological knowledge given its location, and (ii) altering the mix of industrial knowledge present in the region, e.g. by ensuring presence of agents active in other sectors, given the network topology.

Table 1 contains the claims discussed above and actually reveals how these claims are unspecific or rather specific with regard to the network (implicitly) presumed. Using the agent-based model presented below, we study all combinations.

Table 1 Claims in literature on the efficiency and mechanisms at work in regions with a particular knowledge mix (specialized, related variety, diversified) versus the (inter)regional innovation network topology (regional, gatekeeper, or smart interregional network)

3 Knowledge-Based Model of Technology Discovery

3.1 Existing Models of Technology Discovery

In order to study our claims on technology discovery using a computational agent-based model, we need an operational model of both technological knowledge and how this is obtained. Literature does not provide a standard one, but rather many different ones, which are often tailored to particular research questions. Conceptually, though, there is consensus that technology discovery is an evolutionary process of recombination of, experimentation with, and selection of technologies, followed by diffusion, (imperfect) imitation, and adaptation of technologies, ultimately culminating in localized, evolving repositories of artifacts, tools, and associated technological knowledge (cf. Arthur 2009; Childe 1936; Basalla 1988). Given our explicit focus on the knowledge-based perspective in the geography of innovation literature, we follow Arthur (2009, p. 21): “novel technologies arise by combination of existing technologies. [...] The overall collection of technologies bootstraps itself upward from the few to the many and from the simple to the complex.”. In Arthur’s perspective, technologies harness phenomena (e.g. optical, electrical) and put particular effects to use. Metaphorically, Arthur perceives technologies as hidden underground in ‘seams’ of related effects and mankind gradually ‘mines’ deeper from the top down. At the top, effects are primitive and may be serendipitously discovered. The discovery of ‘deep’ effects (e.g. chemical, quantum) is preceded by discovery of simpler effects, and requires advanced knowledge and modern technologies. Technologies based on quantum effects could not have been discovered without using tools harnessing electrical effects, for instance. “Phenomena cumulate by bootstrapping their way forward: effects are captured and devices using these effects are built and uncover further effects.”. While some clusters of phenomenologically related technologies may require research or tool technologies to be discovered, other technologies simply combine ‘modules’ as developed in different fields, e.g. a microscope combines optical lens technologies with mechanical positioning technologies.

Conclusively, a model of technology search ideally features that (i) technology gradually accumulates, (albeit possibly in a bursty nature, upon discovery of particular ‘seams’), (ii) there is an inherent order in which technologies are (to be) discovered, and (iii) new technology comes about by combining existing technologies or its discovery requires using existing technologies. For accumulation and expansion, the search space should expand non-convexly or be so large that search does not reach the boundaries during the modeled time horizon. Combining or extending existing technologies should (be able to) yield technologies outside the readily acquired set. Secondly, to model the inherent order in which technologies are to be discovered, one can think of, e.g. distance in search space that has to be traversed. Thirdly, there is a precedence relationship between technologies. Note that ancestors can be certain tools or research instruments required to ‘uncover’ certain phenomena or ‘seams’ of effects (to stick to Arthur’s metaphors) as well as that particular technology indeed simply is a modular combination of existing technologies. If there are multiple ancestors required to discover a more advanced technology, this meets the requirement of ‘non-convex accumulation’, ‘inherent order’, as well as ‘combinatorial search’.

Few combinatorial technology search models found in literature model meet all three requirements. We briefly go over the most prominent ones and then present our own model. In the models discussed here, technology search is modeled by having agents combine (discovered) elements, thereby seeking to meet or maximize a particular objective (e.g. Gilbert et al. 2001; Arthur and Polak 2006; Chie and Chen 2013; Chen and Chie 2006; Korhonen and Kasmire 2013; Padgett et al. 2003; Vermeulen et al. 2016a). Often the set of elements is fixed and given, and technology search merely is finding the ‘fittest’ or most ‘attractive’ combination. In the NK-landscape approach, technology search is modeled as the process of combining elements (from a fixed set) which jointly and interactively determine the combination’s fitness (cf. Frenken 2006; Woodard and Clemons 2012; Auerswald et al. 2000). In Padgett et al. (2003), a technology is successfully constructed when a combination of ‘processes’ forms a ‘hyper-cycle’. Technology search concerns randomly combining processes from a given set of processes. In Korhonen and Kasmire (2013), technical elements (numbers) are interconnected by transformations (arithmetic operators) into a system which is considered technically feasible if it has a preset outcome. Technology search concerns finding a combination of elements and operators that has that particular outcome. In Gilbert et al. (2001) and Vermeulen et al. (2016a), units of technological knowledge are combined into a product that generates a payoff. Technology search concerns finding a set of units with the highest payoff, both by altering the firm’s portfolio of units and trying different combinations of units from that portfolio (or from the portfolios of firms collaborating in innovation). Korber and Paier (2014) extend the Gilbert–Pyka–Ahrweiler model and calibrate knowledge discovery opportunities using empirical co-occurrence rates. In the current setup, these models feature accumulation of technology, but in a narrow sense: there is no bootstrapping into new sets of technologies, so ultimately, the landscape is confined. In Morone and Taylor (2010), technologies are ordered in a predefined, fixed network structure. Agents explore this network to discover new technologies, however these technologies only feasible when all of its ancestors are discovered and feasible. So, also here there is accumulation and a natural order, plus new technologies that can be generated on the fly. Below we will present a model in the vein of Morone and Taylor (2010).

3.2 Our Directed Graph Search Approach

In line with notably Morone and Taylor (2010) and following our earlier experimental work contained in Vermeulen and Pyka (2014b) and Vermeulen and Pyka (2014a), technological knowledge is perceived as ‘units’ that are ordered in directed graphs, from primitive (upstream) to advanced (downstream). Moreover, like Morone and Taylor (2010), one agent or a collective of agents tries to unlock units in this directed graph by combining units that it already possesses. A single agent or a collective of agents discovers a knowledge unit downstream in the graph if it has discovered and puts together the exact set of primitive ancestors found upstream in the graph. The idea is that primitive knowledge has to be acquired before more advanced knowledge can be understood. An example of this is that man needs to know ‘how to control fire’ and ‘how to contain liquid’ before it can learn how to ‘boil liquid’. The discovery of technological knowledge thus expands the technology search space non-convexly. Note that our model thus meets the three requirements mentioned above: (i) knowledge accumulates (yet it is stored in the agents’ knowledge repositories), (ii) there is an inherent order in which knowledge is (to be) discovered (namely from primitive to advanced), and (iii) new knowledge is discovered by combining readily discovered knowledge.

3.3 Empirical Calibration with Patents?

In Vermeulen and Pyka (2014b) and Vermeulen and Pyka (2014a), we used a parameterized algorithm to generate a stylized knowledge graph. In order to study the controversies in the innovation policy paradigms and the empirical findings on which they are based, it is paramount to use a knowledge graph with a real-world structure and real-world complexity. Inspired by Korber and Paier (2014), we seek to empirically calibrate the search process. In our case, we calibrate the directed graph of knowledge units using statistics on the classifications and citations of patents in the European Patent Office’s patent database PATSTAT. By doing so, the simulation results are arguably externally more valid than when using a highly stylized and simplified structure.

A patent database such as the one of the European Patent Office forms an extensive, valuable source of combinatorial precedence relationships of knowledge. After all, each patent provides backward citations to older patents and thereby related technological knowledge. Moreover, as patents receive a (highly detailed) classification code, actual interrelationships of different ‘fields’ of technological knowledge can be derived from the patent database as well. We can thus determine how often knowledge from one field is used in the discovery of knowledge of the same or another field. In seeking to stimulate innovativeness, this ‘co-occurrence’ of fields has ramifications for the desirable topology of innovation networks, hereby mediated by the geographical distribution of the (agents with) technological knowledge in these fields.

For now, we limit ourselves to the top-level of the International Patent Classification system, i.e. to the level of ‘sections’ purely because of scale issues in our computer implementation. We do acknowledge that also the classification of patents into a ‘class’, ‘subclass’, ‘group’ and ‘subgroup’ is valuable, and even that the knowledge structure may change by adding these lower-level classifications. The ‘section’ level of the International Patent Classification consists of the eight ‘sections’: (A) Human Necessities, (B) Preforming operations; Transporting, (C) Chemistry; Metallurgy, (D) Textiles; Paper, (E) Fixed constructions, (F) Mechanical engineering; Lighting; Heating; Weapons; Blasting; (G) Physics, and (H) Electricity. In our directed knowledge graph, we will classify each knowledge unit as to be part of one of eight ‘technological knowledge fields’, associated with these eight sections.

Apart from having to limit the depth of classification in our calibration, there are several other caveats in using patents to study technological knowledge. If anything, the patent database is not all-encompassing: not all inventions are patentable and not all are patented. Moreover, the inventions that are patented differ in ‘quality’ (for these last points, see Griliches 1957). This may in part relate to strategic patenting of firms as they may create ‘thickets’ of patents around the same technology to (i) discourage competitors from moving into a particular technology field and to secure a de-facto monopoly (cf. Hall et al. 2012, 2015) or (ii) defend against encroachment by competitors pending particular regulation (see e.g. Vermeulen et al. 2016b). Despite the occurrence of e.g. low quality patents or patent thickets, the technological relationships of the various fields and hence the importance of bringing knowledge from these fields together remain valid.

Table 2 Notation

3.4 Algorithm for Graph Generation

We use a straightforward method to ‘statistically calibrate’ the cumulative, combinatorial structure of the technological knowledge graph to the European Patent Office’s database PATSTAT. Each knowledge unit in the knowledge graph is assigned to one of $m = 8$ fields, associated with the eight patent sections in the International Patent Classification code. The knowledge graph is arranged in ‘tiers’, the equivalent of years in the patent database. We initialize Tier 0 with $N_0$ unique initial knowledge units evenly distributed over the $m = 8$ fields. We then use the algorithm described below to expand the knowledge graph recursively, hereby using the citation and classification statistics of patents in the PATSTAT database. See Table 2 for the specification of the notation used in our description.

After studying the distribution of the number of backward and forward citations, we decided to use only the patents granted during the years 1988–1994 for calibration. Starting with an earlier year would underestimate the number of cited patents (‘input knowledge’) while ending with a later year would underestimate the number of citing patents (‘output knowledge’).

The first step is to generate the number $N_t$ of knowledge units for tier t. For this we use a simple period-on-period growth model $N_t = (1 + \rho ) N_{t-1}$, where $\rho = 0.15$ is the period-on-period growth rate as computed from our extensive patent dataset.

The second step is to determine the field for each of these $N_t$ knowledge units. For each unit, determine its field x by drawing it from the discrete, empirical distribution $\mathcal {F}$ of the occurrence rates of fields. The distribution $\mathcal {F}$ is obtained by counting the number of times a patent has a (first) classification in a particular field and then normalizing by the total number of patents in our set. The empirical distribution $\mathcal {F}$ is plotted in Fig. 1.

Fig. 1

Number of patents by first classification in the eight patent sections

Fig. 2

Histogram of normalized number of backward citations by the first patent section (thin lines) and the empirical approximation used (thick)

The third step is to determine the number of backward citations B for each of these $N_t$ units. For each unit, draw the number B from the discrete, empirical distribution $\mathcal {B}$. The empirical distribution $\mathcal {B}$ of the number of backward citations by patents’ first field is plotted in Fig. 2. While the number of backward citations may differ from field to field, we find that this difference is actually limited. Given this similarity in the distributions for the various fields, the citation data $\mathcal {B}$ is aggregated and approximated with one and the same empirical distribution (plotted in thick lines).

The fourth step is to determine the field for each of these B citations. For this we first construct the ‘co-occurrence’ matrix $\mathcal {C}$ of the first field of a citing and the first field of the cited patent.⁴ For each of these B citations we have to determine the field of the cited unit. Say, from a unit in field x, draw the field y from the empirical distribution in the x-th row $\mathcal {C}_{x\cdot }$ of the matrix $\mathcal {C}$. For clarity of exposition, the matrix $\mathcal {C}$ as extracted from the patent database is depicted in the chord diagram in Fig. 3. Hereby, the width of the circle arc represents the fraction of patents with a particular first field (see also the distribution $\mathcal {F}$). Each of the chords connect two circle arcs, indicating a relationship of two fields. The width of a chord at a circle arc x connected to a circle arc y specifies the relative number of patents in field x citing a patent in field y (i.e. $\mathcal {C}_{xy}$).

The fifth step is to determine the period-lag for each of these B citations. This lag is drawn from the discrete, empirical distribution $\mathcal {L}$ of the lag (in number of periods) from cited and citing patent. This period-lag is obtained by looking at the number of years between the year of publication of the citing and cited patents. This distribution is plotted in the dark-shaded histogram in Fig. 4. Ignoring a short period between patent application and grant during which there are no (or very few) backward citations, the number of backward citations can be approximated within a few percentages with $\hat{\mathcal {L}}(\Delta ) = g \Delta ^p$ with $\Delta$ the lag in number of periods, $g = 13$ and $p = 3.1$. While generating Tier $i = 1, \ldots$ of the knowledge graph, the lag $\Delta$ is drawn based on the truncated discrete distribution, i.e. from $\hat{\mathcal {L}}(i-1), \hat{\mathcal {L}}(i-2), \ldots , \hat{\mathcal {L}}(0)$.

Fig. 3

Chord diagram of the first patent sections of citing and cited patents

Fig. 4

Histogram of year-lag of citing to cited patent

As such, we have created a number of knowledge units, which have been assigned a field according to the empirical occurrence rate and a number of ancestors with a field and time-lag which are also drawn from empirical distributions. We thus generate a next tier in a directed graph of knowledge units that is calibrated to rate of (co-)occurrences of patent classifications as well as the lag and backward citations figures of a real-world patent database. A flowchart of this algorithm is given in Fig. 5.

At the start of a simulation run, fifteen tiers are generated recursively, such that, given the distribution of the citation time-lag $\mathcal {L}$, new tiers are unlikely to add additional descendants (‘forward citations’) to existing knowledge units. Moreover, as soon as search (described later) unlocks knowledge units closer than ten tiers from the most advanced tier, new tiers are generated, again with the reason that all descendants of primitive knowledge should be known. We do this because of the search heuristic followed by the agents, as will be described in the next section. Figure 6 contains an illustration of a generated knowledge graph, here with only four tiers.

Fig. 5

Flowchart of the algorithm to generate the next tier in the knowledge graph using the empirical statistics of the PATSTAT patent database

Fig. 6

Illustration of a few tiers of directed graph of technological knowledge units as generated by calibrated using the patent database

4 Spatial Agent-Based Model

To determine what network topology or spatial distribution of knowledge a social planner should implement to stimulate the discovery of technological knowledge, we use an agent-based model in which we can study the topology and the spatial distribution in conjunction. In the agent-based model, there are $r = 8$ regions (‘land’ cells as opposed to ‘sea’ cells), $m = 8$ fields (each associated with a patent classification section in the patent database), $n = 8$ agents per region, and a bipartite matrix assigning knowledge sections to agents (as their technological specialization).

Each agent starts with all of the primitive knowledge units within its specialization (i.e. the ‘nodes’ at the bottom in Fig. 6). In each period of the simulation, each agent investigates a single combination of knowledge units in its own repository and the repositories of its partners. There is no economic principle (e.g. a demand market) that rewards knowledge discovery or paces search. Moreover, there is no entry or exit of agents. Finally, the circular layout of regions is fixed for all simulations. This simple setup ensures undistorted observations of the effects the role of the network topology and spatial distribution of knowledge the advancement of technological knowledge.

As explained above, in this paper, certain combinations of technological knowledge units ‘unlock’ a unique, more advanced knowledge unit. Agents are hence engaged in searching for successful combinations of knowledge units found in their own repository complemented with knowledge units found in the repositories of their network partners. As described in Sect. 3, many technology discovery models found in literature have agents that are engaged in trial-and-error recombination of technical elements, whereby the number of elements available is constant (and generally low) and thus form a concave set. In the model presented here, combinations of primitive knowledge units produce more advanced knowledge units that may very well not yet be in the pool of available units. As the knowledge units that are discovered can in turn be combined to try to discover even more advanced knowledge, the number of units is endogenously and non-convexly growing. Consequently, the number of possible combinations explodes with the number of simulation steps. In case the number of feasible combinations (i.e. those ‘unlocking’ a new unit) is a non-trivial percentage, there is a rapid plateauing in the number of discoveries if such a trial-and-error search heuristic is followed. As such, it is disputable that research is, in the real world, naive trial-and-error.

In reality, inventors have visions of the technology that they seek to create, then isolate operating principles to be exploited and technological solutions to try (cf. Arthur 2009). Our operationalization of Arthur’s observation is that the agents, firstly, serendipitously ‘see’ a potential extension of a randomly drawn knowledge unit that they already possess (i.e. a descendant in the knowledge graph). In the current simulation results, the agent only sees extensions on the next tier of the knowledge graph. Secondly, agents know which technical specializations are required for this extension, but without knowing which knowledge units exactly. The searching agent approaches partners with one of the required specializations in uniform random order with the request to test whether any of their knowledge units would be adequate. Once all of the required knowledge units are present, the new knowledge unit is ‘unlocked’. This newly discovered unit is then also added to the repository of all agents that have contributed and are specialized in the field of the output knowledge. This search heuristic is depicted in the flowchart in Fig. 7. In new rounds, this knowledge can be used for further recombination.

Under these assumptions, the knowledge discovery search is then not a purely random ‘trial-and-error’ of recombining knowledge units, but rather inquiries with a deliberately chosen, technologically specific selection of partners. Note that this does not alter the pool of possible discoveries, but merely compresses the time scale compared to simulation runs with a random selection of knowledge units and/or collaboration partners.

5 Simulation Setup and Results

5.1 Simulation Scenarios

Simulations are run for 1000 period, with $r = 8$ regions placed in a circle (such that each region has exactly two immediate neighboring regions for a purpose explained below), and $n = 8$ agents per region, whereby each agent is specialized in only one of eight fields. With regard to the spatial distribution of technological fields over regions, we study three scenarios. Firstly, the ‘specialized’ scenario in which the $n = 8$ agents in a particular region are active in the same field and start out with the same primitive knowledge units. Each region has a different specialization (Field A, B, ..., H). Secondly, the ‘diversified’ scenario in which each agent in a particular region has a different field of specialization, i.e. the first agent is specialized in (and starts with primitive knowledge units in) field A, the second in field B, and so forth, and the eighth agent is specialized in field H. The setup is the same in each of the $r = 8$ regions. Thirdly, the ‘related variety’ scenario in which each region has agents active in a mix of fields. Hereby, each region has its unique, main field of activity (like in the ‘specialized’ scenario), yet the region possibly contains agents engaged in probabilistically most related fields. This is initialized as follows. Each region gets an agent specialized in a unique field, say field x, while the other $n - 1$ agents are randomly assigned a field based on the empirical co-occurrence distribution $\mathcal {C}_{x \cdot }$, i.e. the x-th row of the co-occurrence matrix $\mathcal {C}$. To be able to have perfectly specialized regions while covering all fields (in certain simulation scenarios), we set r equal to m, and to be able to have perfectly diversified regions while covering all fields, we set n equal to m.

Fig. 7

Search heuristic

Fig. 8

Illustrations of each of the network structures for $k=2$, here with specialized regions. a Gatekeeper, b regional, c smart

With regard to the innovation network imposed by the social planner, we study three network topologies. Firstly, the ‘regional’ network topology, in which each agent is recursively connected to k randomly drawn other agents in the same region. As such, the work consists of r disconnected subnetworks. The total number of ties is rnk and the average degree is 2k.

Secondly, the ‘gatekeeper’ topology in which each region has exactly one ‘gatekeeper’. Only this gatekeeper is connected to agents outside the region, and then in particular exclusively to other gatekeepers. Within the region, each non-gatekeeper agent is connected to the gatekeeper. Moreover, each non-gatekeeper is connected to $k - 1$ other agents in the region, whereby these connections are recursively picked randomly from all possible connections not yet existing. To meet the correlation of geographical and organizational proximity, we have chosen to lay out the regions in a circle and thus make sure that each gatekeeper is in contact with the two gatekeepers in adjacent regions. Any other configuration with 8 regions would yield asymmetries in the number of contacts. As such, there are r connections between gatekeepers in total. To make sure that the total number of ties (and thereby the average degree and average ‘transaction costs’) are equal across the three network topologies, $r (k - 1)$ random, unique ties between gatekeepers are created.

Thirdly, the ‘smart’ topology connects each agent with k other agents based on field of specialization. Hereby, an agent with specialization x is connected to an agent with specialization y with probability $\mathcal {C}_{xy}$ (see the description of $\mathcal {C}$ given before). Empirically, the higher $\mathcal {C}_{xy}$, the more likely a patent in field x is citing a patent in field y. So, the higher $\mathcal {C}_{xy}$, the more often collaboration yields an invention. In case there are more agents with specialization y, one is picked randomly from those with whom there is not yet a connection. The number of ties is rnk and the average degree is 2k. Figure 8 contains illustrations of each of the three network topologies. Note that the ‘regionality’ of the ties in the ‘smart’ topology reflects the spatial distribution of knowledge. In Fig. 9, examples of the ‘smart’ network topology for both the case with specialized and the case with diversified regions are shown. We see that the smart network with diversified regions is in fact more interregional. This is caused by the fact that many inventions are actually combinations of pieces of knowledge within one and the same field, i.e. the on-diagonal elements on the co-occurrence matrix $\mathcal {C}$ are relatively high. Alternatively, the chord diagram in Fig. 3 reveals that cited patents often have a classification in the same field as the citing patent. In case of diversified regions, agents active in the same field are in fact found in other regions.

Fig. 9

Examples of smart topologies for specialized and diversified regions, with $k = 2$ ties initiated by each agent. With diversified regions, more ties are interregional so as to collaborate with others active in the same technology field. a Example of a smart network topology for specialized regions, b example of a smart network topology for diversified regions

For $k = 1, 3$, the total number of ties are 64, 192 respectively in all network topologies. Consequently, any differences in knowledge discovery (rates) are then due to structural differences in the topology (that is, with the spatial distribution of technological knowledge held equal), not because of differences in the number of relationships. In reality, firms incur transaction costs for maintaining (innovation network) relationships, so by ensuring that the topologies have equal numbers of relationships, we can ignore these costs in our analysis and conclusions.

We thus get a $3 \times 3$ contingency table of possible scenarios for the spatial distribution of technological fields versus the innovation network topology. For each of the $3 \times 3$ scenarios, we also varied the number of links $k = 1, 3$ each agent itself creates (explained above as to why we pick this ‘ego-centered’ operationalization) and the ‘depth’ $d = 1, 2$ of the collaboration, whereby $d = 2$ means there is triadic closure. For each possible scenario, we simulated 100 cases for 1000 periods. To provide more insight in the potential innovativeness, we provide the $m \times m$ matrices of the average minimum shortest path lengths between agents by technological specialization in “Appendix A”. Despite the appearance and naming of the network topologies, the traditional metrics of social network structure provide limited insight in the performance in terms of rate and advancedness of inventions.⁵ As such, we refrain from reporting these metrics.

Fig. 10

Plots of the average maximum advancedness of the knowledge discovered for different parameter settings. Plotted are the average (thick), the top and bottom 20% (thin), and the top and bottom 5% over all simulation runs. a $k=1, d=1$, b $k=1, d=2$, c $k=3, d=1$, d $k=3, d=2$

Fig. 11

Plots of the cumulative number of unique discoveries over time for different parameter settings. Plotted are the average (thick), the top and bottom 20% (thin), and the top and bottom 5% over all simulation runs. a $k=1, d=1$, b $k=1, d=2$, c $k=3, d=1$, d $k=3, d=2$

5.2 Scenario Simulation Results

We study the technological progress using two measures. Firstly, we measure the ‘average maximum advancedness’ of knowledge. Over time, agents discover knowledge at increasingly higher tiers in the knowledge graph. By taking the average of the indices of the most advanced tier reached by each agent, we obtain the ‘average maximum advancedness’. We plot this advancedness over time. However, while this measures how advanced technology becomes, i.e. the ‘depth’ of the technological knowledge, it does not measure the ‘breadth’ of discoveries, i.e. how many of the possible technologies at each tier have actually been discovered. So, secondly, we measure the total size of the ‘technological knowledge pool’ by counting the total number of unique knowledge units across all agents. We plot this cumulative number over time as well. We also study the percentage of knowledge units discovered by tier and by field using density plots. As explained in “Appendix B”, we use uniform occurrence rates ($\mathcal {F} = (1/m, \ldots , 1/m)$) rather than the original rates plotted in Fig. 1 to overcome distortion in the simulation results caused by very low frequencies of knowledge units in fields D, E, and F.

In Fig. 10, we plot the average maximum advancedness for the $3 \times 3$ scenarios (specialized/related variety/diversified regions $\times$ regional/gatekeeper/smart network topologies) for the four combinations of the number of ties $k = 1, 3$ (rows) and the partnering depth $d = 1, 2$ (columns) for collaboration with own partners only ($d = 1$) or also with partners of partners ($d = 2$, triadic closure). In general, we see that there is an initial phase with fast technological progress (caused by the low number of tiers yet generated and the fact that the lion share of knowledge units on these tiers are in the possession of agents for further recombination, see Sect. 3.4), the increase in average maximum advancedness settles at slowly decreasing rate (see Fig. 10), while the total number of unique inventions keeps on increasing steadily (see Fig. 11). Comparative analysis reveals that technological progress in regional networks is almost always outperformed by the progress in the other two topologies, regardless of spatial knowledge distribution, while smart networks are generally slightly outperformed by the gatekeeper networks particularly if the number of partners is limited (k and d low). In regional networks, the size of the pool of knowledge in various fields is simply more limited, either in terms of diversity or magnitude. Fundamental results on the average minimum path lengths between agents active in different fields contained in “Appendix A” show that regional networks (i) lack access to knowledge in different fields in case of a specialized region, or (ii) lack access to knowledge in the same field in case of a diversified region, so has a limited pool of knowledge. Given the high number of backward citations and the frequent cross-references between knowledge fields (as captured by the co-occurrence matrix $\mathcal {C}$, see the chord diagram in Fig. 3), access to knowledge in other fields is crucial. In fact, comparing the advancedness for the regional networks, we see that the advancedness—although not high in general—is higher for the case with diversified knowledge than for the case with related variety in the region. So, while related variety is to be preferred over specialization (in which the region gets into a lock-in soon), the access to incidentally related knowledge provided by diversification is crucial for long term ‘branching’. Across all network topologies, the technological advancedness is as high as or higher in the diversified region as it is in the other two. The fundamental results in “Appendix A” also reveal that the average minimum shortest path length is—on average—also slightly lower in case of diversified regions than in case of regions with related variety, with a notable exception for the shortest path to agents in the same knowledge field.

We also see that both the advancedness and total number of discoveries are high in case of the gatekeeper topology for most connection settings (k and d), but saliently for low k ($k = 1$) with triadic closure ($d = 2$). Analysis of the knowledge repository of the gatekeepers reveals that they accumulate an extensive set of knowledge, provide access to the aforementioned incidentally related knowledge and thus are the motor for discoveries in their principal region, in connected regions, but indirectly also in other regions. For high connectivity cases ($k=3$), the performance of the smart network is somewhat higher than that of the gatekeeper network because agents in the smart network now also have partners active in not only the most but also less related knowledge fields, while gatekeeper networks have a geographical bias as they connect in the first place to other gatekeepers the two adjacent regions.

In Fig. 11, we plot the total number of unique knowledge units for each of $3 \times 3$ scenarios and the four parameter combinations. In general, the number of discoveries is low if the connectivity is low ($k = 1$ and $d = 1$), plus, as we already saw, also the advancedness of the discoveries is low. The number of unique discoveries increases substantially with more connections ($k = 3$) particularly in conjunction with triadic closure ($d=2$). Looking at the average minimum shortest path length statistics in “Appendix A”, the root cause is the fact that, apart from the exceptional regional case, agents in all fields are often directly connected to agents in any other field and with relatively short path lengths. While this holds in general, there a more subtle insights gained from detailed analysis. Firstly, in case of a regional topology without interregional ties, we see that the effect of more ties or triadic closure is relatively limited and, if there is any salient effect, this is only the case when the region is diversified. All in all, agents in other regions make discoveries in other branches of the knowledge graph and, for further progress in each of these branches, agents active in these branches should be able to access knowledge in the other branches. From this we conclude, that there is a strong effect of the ‘size of the knowledge pool’ and that access to the knowledge bases of other agents is significant in both the absolute number and advancedness of inventions. Secondly, if the region is diversified, the regional topology generates more discoveries than the smart topology. This is indicative of a phenomenon we call ‘high level lock-in’: in the smart topology, agents may lack access to knowledge in incidentally related fields, which effectively stifles discoveries in certain subgraphs of the knowledge graph. We discuss this ‘high level lock-in’ phenomenon in detail, later. In short, the cause of the ‘high level lock-in’ is that smart topologies lack connections to merely incidentally required knowledge, yet have (excessive) access to knowledge in the same and technologically strongly related fields. This is corroborated by the high average minimum shortest path lengths reported in the $k=1$ and $k=3$ tables in “Appendix A” for technologically weakly connected fields. Interestingly, the reverse is true if the region contains related variety or if the region is specialized: smart topologies generally outperform the merely regionally connected networks in case the region hosts specialized or a mix of related knowledge. Thirdly, there is an exceptionally high relative number of discoveries for the gatekeeper topology with triadic closure for diversified regions, which is essentially the culmination of three effects: (i) diversification provides access to ‘incidentally related’ knowledge, (ii) gatekeepers function as knowledge hubs, and (iii) triadic closure provides access to a bigger pool of knowledge. Triadic closure through gatekeepers provides agents with short paths to many other agents, even already for $k=1$. Fourthly, the increase in the curves seem to taper off, whereby the increase in advancedness decreases rather quickly and in the number of inventions only very gradually. A primary cause of the decreasing rate of progress and discovery is the increasing ‘breadth’ of undiscovered potential ancestors from tier to tier.⁶ Note that all combinatorial technology search models reviewed in this paper feature this leveling off. However, unlike these models, the cause is not combinatorial trial-and-error search. In our model, each agent asks all of its partners with a particular specialization (in random order) for suitable knowledge units to unlock targeted knowledge in a single period, so all combinations of knowledge units present among partners are considered. As such, the tapering off of progress is due to the fact that more and more advanced branches in the knowledge graph remain unlocked and hence knowledge therein cannot be exploited to unlock even more and more advanced knowledge.

Fig. 12

Density plots of the percentage of discoveries possible per tier (X-axis) per each of the m fields (Y-axis) after T periods for different parameter settings. The darker, the higher the percentage. Completely black (white) means 100% (0%) of knowledge units available in the knowledge graph is actually discovered at $T=1000$. a $k=1, d=1$, b $k=1, d=2$, c $k=3, d=1$, d $k=3, d=2$

In Fig. 12, we plot the percentage of discovered knowledge units for each tier (with Tier 0 on the left and Tier 30 on the right) for each of the fields (with Field A at the top and Field H at the bottom). The differences in discovery rates from scenario to scenario are striking. In line with the results presented above, the rate of discovery increases strongly with connectivity (i.e. increasing k and d) and whether or not agents connect their partners (i.e. for $d=2$ there is triadic closure, while for $d=1$ there is not). When agents have more (potential) connections, they have access to more and more varied sources of (related) knowledge. Insight is obtained through more detailed analysis. We see that, for the gatekeeper and smart topologies, the frontier is about equally advanced, yet the percentage of knowledge discovered across all tiers is superior in diversified regions, generally. For regional topologies, both the advancedness and percentage of discoveries is particularly high if the region is diversified. Conclusively, although the rate of discovery within regions may be enhanced by diversification, there may still be high level lock-in due to limited access to knowledge in branches developed in other regions. Moreover, a salient feature of knowledge discovery not showing up in the plots of the advancedness and the number of inventions is the ‘profile’ in the percentage of discoveries across different knowledge fields. We see that, for many simulation scenarios, knowledge discovery in Field E and, to lesser extent, Field D and Field F is falling behind (has a lighter shade) the rest. Surprisingly, though, for several scenarios with a diversified knowledge mix and many local relationships (regional and gatekeeper network topologies), the percentage of discoveries is rather higher (has a darker shade). The root cause is that particularly for these knowledge fields, only a few other fields matter mostly (e.g. Field D relies strongly on Field B and C, Field F relies strongly on B) and the network topologies exploit profound access to these fields in the region and access to fields with incidentally related knowledge. In fact, smart network topologies have less access to the incidentally related knowledge. Looking at the path length statistics contained in “Appendix A”, we see that, for instance, agents in Field D are relatively well-connected to agents in Field B and C (high number of connections, which is reported between square brackets and low average path length), but the opposite holds for agents in other fields (so, low number of connections and high average path length). Moreover, where the local density of the gatekeeper and regional networks enhances access to incidentally related knowledge in case of diversified regions, this incidentally related knowledge is not or rarely present in case of regions with specialized knowledge or related variety. The density plots thus highlight the intricate relationship between knowledge distribution, network topology, and the technological interrelationships between the various knowledge fields.

5.3 High Level Lock-In

While the term ‘lock-in’ is generally associated with the inability to innovate due to technological specialization, there is a form of ‘lock-in’ also in the case of recombining exclusively within a set of (strongly) related variety knowledge. In the simulation results for the different scenarios, we observed that the ‘smart network’ may ultimately be outperformed by the ‘gatekeeper network’ particularly if the collaboration depth is $d = 2$ and $k = 3$. Inspection of simulation logs revealed that agents in smart networks sooner or later do not have access to knowledge in ‘incidentally related’ fields. In this $d=2$ and $k = 3$ case, the few random connections of the gatekeeper to the gatekeepers in other regions gives access to path-breaking (or rather ‘subgraph unlocking’) knowledge. This is corroborated by the path length statistics contained in “Appendix A” in the light of the co-occurrence matrix plotted in Fig. 3: in smart network topologies, agents have many connections and short path lengths to agents active in related fields, but few connections and long path lengths to agents active in less-related fields. Simulation results not included in this paper revealed that also pure random network topologies outperform smart networks for the same reason. The key implication is that while smart networks and access to related variety is performance enhancing, access to incidentally related variety remains crucial for discoveries in the long run. We consider this a counterpart of the ‘weakness of strong ties’ and how ‘weak ties’ may be a long-run competitive advantage.

A similar principle of ‘high level lock-in’ applies for purely regional networks and has ramifications for ‘branching’ policies. Looking at Fig. 11, we see that the cumulative number of discoveries in regional network is increasing faster if the region is diversified than if the region contains related variety. Analysis of simulation logs revealed that quite early on knowledge that is only incidentally related is not found, effectively blocking discovery of large subgraphs in the knowledge graph. From this, we see that a branching strategy in a region with merely related variety (see Boschma 2011; Asheim et al. 2011, advocating this regional innovation policy) is risky if there is no pipeline to such merely incidentally related knowledge.

5.4 Mediating Effect of the Knowledge Structure

The findings in this papers are particular to the knowledge graph as calibrated to the patent database. As discussed, there are several caveats to the patent classification code level we used for calibration and also to using patents in the first place. Our motivation to do this paper is to analyze the role of innovation network topology in knowledge discovery given a particular spatial distribution of knowledge. Yet, we also claim that the underlying technological knowledge structure is a (strong) mediator. Here we study more in depth, how the knowledge structure affects the innovativeness of the network topology given a spatial knowledge distribution. From this, a social planner can derive which network topology to pick. However, we restrict our analysis to changing the co-occurrence matrix $\mathcal {C}$ and leave the distribution of the number of backward citations $\mathcal {B}$ and the distribution of the citation lag $\mathcal {L}$ unaltered. We construct the knowledge co-occurrence matrix as follows:

$$\begin{aligned} \mathcal {C}(\zeta ) \leftarrow (1 - m \, \zeta ) \, I + \zeta \, U, \end{aligned}$$

whereby I is the identity matrix and U is the unit matrix with all entries equal to 1. In our simulations, we pick $\zeta = 0.005, 0.010, \ldots , 0.120$ to vary the co-occurrence matrix between the case where each field uses nearly only knowledge from its own field (i.e. mostly self-referential) when $\zeta \downarrow 0$ to the case where fields are all nearly equally related to one another when $\zeta \uparrow 1/m$.

After inspection, we dismiss less interesting cases in which many agents are connected to many others already (e.g. $k = 3$ and $d = 2$). Rather, we select the case with $k=2$ and $d=1$ with a few, yet non-trivial number of connections. Figure 13 contains plots of the cumulative number of unique discoveries after 1000 periods for the range of $\zeta$ values for each of the studied network topologies (regional, gatekeeper, smart), for each of the three studied geographical knowledge distributions (specialization, diversification, related variety). We see that, in general, in the gatekeeper network cases, high numbers of unique discoveries are attained. This is caused by the fact that the gatekeepers function as knowledge hub either between different regions specialized in different fields or between diversified pools of knowledge and thus stimulate innovativeness in all regions connected to. Gatekeeper networks even significantly outperform the smart and regional networks for near-identity co-occurrence matrices, i.e. for more self-referential knowledge. Interestingly, we see in Fig. 13a that for specialized regions and regions with related variety, the performance of smart and gatekeeper network topologies is (statistically) equal for near-uniform co-occurrence matrices. This is caused by the fact that, for near-uniform co-occurrence matrices, smart network topologies are nearly random, thus have relatively short paths to virtually all types of knowledge (particularly because the number of links $k > 1$). In contrast, gatekeepers still first connect to gatekeepers that are immediate neighbors and connect with uniform randomly drawn other gatekeepers if the number of ties k permits. The focus on regional ties for non-gatekeepers and the focus on immediate neighbors for gatekeepers limits the access to other types of knowledge, hence the relatively weak performance.

Conclusively, if knowledge is highly cross-referential, relationships in the (inter)regional innovation networks should be formed on the basis of knowledge relationships and rather ignore geographical factors. This is particularly the case for (strongly) uniform knowledge structures with specialized regions; any geographical bias in collaboration relationships hampers innovativeness. This is corroborated by looking at the performance of the smart network topology. Although not the best performing network, the number of discoveries in the smart network cases is consistent across the three scenarios for the spatial knowledge distribution. This is because of the focus on technological rather than geographical relatedness. The merely sub-optimal performance is due to absence of access to incidentally related technological knowledge.

Another phenomenon observed is that the number of discoveries in the gatekeeper (dashed) and regional (dotted) network changes from down- to upward sloping if the regions contain diverse rather than specialized knowledge or related variety. Whenever regions contain specialized knowledge or related variety while discoveries often require knowledge from other fields (high $\zeta$), the local network density and geographical bias towards immediate neighbors in gatekeeper networks hampers access to that knowledge from other fields. Whenever regions are diversified, though, the local network density is in fact conducive to a high discovery rate. This observation validates the role of the geographical bias in innovativeness in the previous paragraphs.

Fig. 13

Number of discoveries new to the world at time 1000 for different stylized knowledge matrices, ranging from a near-identity matrix ($\zeta = 0.005$) to a near-uniform matrix ($\zeta = 0.12$), for a regional (dotted), smart (continuous), and gatekeeper (dashes) network topology. Plotted are the averages and 90% confidence interval. a Regions contain specialized knowledge, b regions contain related variety, c regions contain diverse knowledge

5.5 Technological Focus Stifles Progress

One of the first findings that actually led to a change in our operational search algorithm is a form of lock-in which may very well exist in the real world. Analysis of logs of single runs revealed that if agents initiate R&D only if the discovered knowledge falls within their own field of specialization, the discovery of knowledge in fields different than the fields sought to extend is inhibited. Suppose that a knowledge unit, say X in field x can be combined with a knowledge unit Y in field y (which may or may not be equal to field x) and that the outcome is knowledge unit Z in field z with $z \ne x$. If a technologically focused agent active in field x focuses exclusively on discoveries in its own field, it will not conduct R&D to discover Z unless approached by an agent active in field z. However, if $x \ne z$ and $y \ne z$, no agent will seek to discover technological knowledge Z. The effects on further development may be substantial, after all, given the high number of backward citations, see $\mathcal {B}$, this knowledge Z is itself likely to be required to discover more advanced knowledge. Consequently, the technological focus of agents in their R&D activities in fact inhibits discovery of whole subgraphs of the knowledge graph.

Figure 14 plots the descriptive statistics for both the case in which agent do (bottom row) and do not (top row) have an explicit technological focus, i.e. whether agents only seek to discover knowledge in its own field of expertise (bottom row) or not (top row). We see that a technological focus of agents causes the total number of discoveries to drop significantly (compare Fig. 14c with Fig. 14f), but that this is not only due to lower advancedness of the discoveries (compare Fig. 14b with Fig. 14e), but rather that the percentage of discoveries across all tiers drops (compare Fig. 14a with Fig. 14d). Since (partnerships of) agents do not unlock technologies that are not in their field of expertise, whole subgraphs remain undiscovered, thereby limiting future options for recombination as well, with—as we see here—significant consequences. In Sect. 6, we provide recommendations on how policy makers may stimulate agents to broaden their research scope.

Fig. 14

Plots of descriptive statistics for $k=3$ and $d = 2$ in case without (top row) or with (bottom row) explicit technological focus. Here we use the knowledge structure $\mathcal {C}(\zeta )$ with $\zeta = 0.05$ for clarity of exposition. a Density plot of % discovered with no technological focus, b maximum advantage with no technological focus, c total number of unique knowledge units discovered with no technological focus, d density plot of % discovered with technological focus, e maximum advantage with technological focus, f total number of unique knowledge units discovered with technological focus

6 Conclusions and Policy Implications

In this paper, we studied how a social planner seeking to enhance innovativeness of one or multiple regions should take into account the (inter)regional innovation network and the spatial distribution and structure of technological knowledge. Our findings confirm our conjecture that the regional specialization versus diversification versus related variety debate (and the associated policy paradigms) cannot be settled without taking the (inter)regional innovation network into consideration. Harking back to Table 1, we see that several of the cells in the contingency table that are empty (i.e. these particular combinations were not explicitly discussed in the reviewed theoretical literature) have been studied using our agent-based model. Generally, though, the research question ‘given a particular geographical distribution of and interrelationship of technological knowledge, what network topology should a social planner pick to enhance innovativeness?’ has no simple answer. A short answer revolves around the fact access to related variety but particularly also ‘incidentally related’ knowledge is crucial for long term technological progress. There are several combinations of a spatial knowledge distribution and a network topology that inhibit access to such incidentally related knowledge. Not only regional networks in case regions have specialized knowledge or related variety, but also smart (interregional) networks with agents in specialized regions, or regions with specialized knowledge or related variety connected by gatekeepers that connect on geographical rather than technological basis. That said, although diverse regions are generally performing relatively well, notably because agents have access to ‘incidentally related’ knowledge in each of the studied network topologies, performance is greatly enhanced by increasing the number of relationships (e.g. through triadic closure) and access to knowledge outside the region (notably through gatekeepers).

Our main conclusions and this short answer do rest on the premise that the knowledge graph truly is as calibrated. However, in the long answer to the research question found below, we elaborate also on how the significance of collaboration with firms in other fields and in other regions depends on structural features of the underlying technological knowledge. First of all, strictly regional networks and thereby industrial district policies are to be dismissed: strictly regional networks are almost always outperformed by the other two topologies, irrespective of the spatial knowledge distribution, if not because of hampered access to (incidentally) related knowledge, then simply because the total pool of knowledge available to agents is smaller. Fundamental analysis in Sect. 5.4 showed this also is the case for regional networks if the technological knowledge field is (mostly) self-referential, i.e. if the co-occurrence matrix is (nearly) the identity matrix. Moreover, smart networks are in turn doing about as well as but are a few times outperformed by the gatekeeper networks. Given the high number of backward citations and frequent cross-references between knowledge fields, access to knowledge in other fields is crucial. Persistent innovativeness requires access to merely ‘incidentally’ related knowledge and smart networks provide this less often than do gatekeeper networks. For high connectivity scenarios, this is because gatekeeper networks have ‘weak’ ties to random knowledge pools, while smart networks have mostly ties based on technological co-occurrence probabilities, thus overlooking the importance of incidentally related knowledge for path-breaking or rather subgraph unlocking.

For the same reason, the performance of the recently advocated ‘regional branching’ policy paradigm (in which regions merely host a mix of ‘related variety’) is performing sub par. As agents cannot access incidentally related knowledge, particular subgraphs in the knowledge graph are not unlocked such that innovativeness ultimately tapers off. We coined the term for ‘high-level’ lock-in, which distinguishes stifling of technological progress because agents are limited to specialized knowledge of that because agents have access to related variety for branching, but not to incidentally related knowledge required to unlock alternative subgraphs of knowledge. Interestingly, we also find that the ‘smart specialization policy’ followed by, e.g. the European Commission, has a drawback for the same reason, namely that there is limited access to incidentally related knowledge. Not surprisingly, the smart specialization policy performs better if there are more relationships and when there is triadic closure that connects partners that are otherwise not collaborating.

We find that the gatekeeper network has remarkably high advancedness of and total number of discoveries, across all connection settings (k and d), but saliently for cases with few connections (low k or d). Analysis of the knowledge repository of the gatekeepers reveals that they accumulate an extensive and relatively diverse set of knowledge and thus are the motor for discoveries in their principal region, in connected regions, but indirectly also in other regions. Fundamental analysis in Sect. 5.4 on the moderating role of the knowledge structure reveals that the gatekeeper network topology is generally performing better than the smart and regional network topologies. However, if the knowledge structure is nearly uniform ($\zeta \uparrow 1/m$), the smart network outperforms the gatekeeper network if regions are highly specialized and gatekeepers have few and mostly geographically proximate relationships. Moreover, the smart network topology has a consistent performance across the various spatial distributions of knowledge. The general implication here is that technological relatedness should prevail over geographical relatedness in constructing (inter)regional relationships.

Apart from the sweeping conclusions on the type of network topology that a social planner should prefer in conjunction to the geographical distribution of knowledge (and a particular knowledge structure, which we presume to have calibrated correctly), there are a few general recommendations we can make. As a first general recommendation, it should be noted that technological knowledge is fairly self-referential such that buffering against inefficient interaction across knowledge fields is commendable. Finding a balance between collaborating with immediate peers, to parties active in frequently related fields, and ultimately also parties only incidentally related fields seems to be a challenge. The simulations reveal that gatekeepers do not only provide this buffer, but create short paths, provide triadic closure, and provide rich knowledge repository, and are as such a valuable asset in innovation networks.

An important observation we did was that ‘technological focus’ of agents strongly impedes technological progress, not only of the individual agent but also of the region and thereby the world. This ‘technological focus’ may be argued to be the flip side of the medal when it comes to the benefits of vertical specialization. We argue that social planners should make sure that firms should also care about research when it merely applies its current knowledge outside its field of expertise. The social planner could stimulate applied research, install research valorization centers, stimulate horizontal diversification of firms, etc.

As discussed before, there are several caveats to using patents in general and using patents in the way we did. Apart from fairly limited expositions in the history of technology literature, there are, to our knowledge, no serious alternatives available that we can use to calibrate knowledge graphs. This while we need such a knowledge graph with a real-world structure to address research questions as ours in this paper. That said, we are aware that the method proposed in the paper for empirical calibration and also the level of classification codes that we have used (at the patent section level) may well be improved. For one, we can imagine that a higher granularity of knowledge is desired, e.g. to the level of subsections, class, or even subclass.

However, this paper is, in our view, a first step in a potentially fruitful research line that addresses research questions that require the control that agent-based models can provide, yet are not purely fundamental with regard to actual technology/knowledge search processes. In our view, a model such as ours thus moves research in the direction of providing more specific and yet externally valid implications for social planners and policy makers when it comes to stimulating innovativeness.

Appendices

Appendix A: Average Minimum Shortest Paths Between Agents by Specialization

To assess the chance to make particular combinations of knowledge units, the topologies and knowledge distributions in case of each of the $3 \times 3$ scenarios in technology discovery, the shortest path length to agents with (other) specializations is computed. Given that the topologies, generally, contain asymmetries, the various agents active in one and the same field may have different shortest path lengths to other agents. As the depth d of relationships of agents should be equal or higher than the minimum shortest path length for agents to be able to make combinations, the latter is reported. The tables in Figs. 15 and 16 report the average minimum shortest path length of 100 different network and knowledge distribution instances for m times m pairwise combinations between fields. To compare the inherent chances of making combinations across the $3 \times 3$ scenarios, we provide the matrix of average minimum shortest path lengths of the pairwise combinations of fields for each of these scenarios. The average minimum shortest path length of pairs of fields is used in explaining differences in the average maximum advancedness, the cumulative number of unique discoveries, and density of discoveries.

Fig. 15

Matrix of the pairwise average minimum shortest paths by field for each of the $3 \times 3$ scenarios, for $k = 1$. Between square brackets the percentage of paths between agents specialized in the two fields

Fig. 16

Matrix of the pairwise average minimum shortest paths by field for each of the $3 \times 3$ scenarios, for $k = 3$. Between square brackets the percentage of paths between agents specialized in the two fields

Fig. 17

Density plots of the average percentage of knowledge discovered at $T=1000$ for 100 simulation runs (for connectivity settings $k=2$ and $d=1$) for the four combinations of using the empirical (co-)occurrence rates or rather the controlled rates (i.e. $\mathcal {F} = (1/m, \ldots , 1/m)$ and $\mathcal {C}(\zeta = 0.04)$). a Empirical occurrence $\mathcal {F}$, empirical co-occurence $\mathcal {C}$, b empirical occurrence $\mathcal {F}$, controlled co-occurence $\mathcal {C}$ with $\zeta = 0.04$, c uniform occurrence $\mathcal {F}$, empirical co-occurence $\mathcal {C}$, d uniform occurrence $\mathcal {F}$, controlled co-occurence $\mathcal {C}$ with $\zeta = 0.04$

Appendix B: Simulation Onset Particularities Due to Low Occurrence

From the recursive algorithm described in Sect. 3.4, it is clear that the number of knowledge units in the first few tiers is relatively low and only gradually grows. The number of knowledge units in these first few tiers is of course particularly low for fields with a relatively low occurrence rate $\mathcal {F}$. In Fig. 1, we see that particularly fields D, E, and F have a low occurrence rate. As such, in various simulation runs it does occur that the first few tiers have no knowledge units in either of these fields. Consequently, in the case agents are technologically focused, there are agents that will not be engaged in research and development. Moreover, in expanding the knowledge graph, knowledge units in other fields have no ancestors in these fields to cite. Ultimately, though, with the increase in total number $N_t$ of knowledge units per tier grows and the chances grow rapidly that there are knowledge units in these ‘rare’ fields. However, it is clear that the simulation results thus feature ‘onset behavior’ due to initialization of the simulation during early periods that gradually vanishes over simulation time. Although this particular ‘onset behavior’ is not an anomaly of our operationalization, we have decided to alter the occurrence rates to completely uniform, i.e. $\mathcal {F} = ( 1/m, \ldots , 1/m )$, such that the simulation results differ purely due to the differences in the co-occurrence matrix $\mathcal {C}$. By doing so, we focus purely on the effect of the relationships between fields as captured in matrix $\mathcal {C}$ and not the occurrence rates $\mathcal {F}$. Comparing the average percentages of discovered knowledge at the end of simulation runs (here for $k=2$ and $d=1$, but the results remain structurally the same for other settings) for cases with both the empirical and uniform occurrence rate (i.e. comparing Fig. 17a with Fig. 17c), we see that the ‘irregularities’ in the early tiers on the left-hand side are gone. The effects of, in addition, also controlling the knowledge graph by using the co-occurrence matrix $\mathcal {C}(\zeta = 0.04)$ can be seen by comparing these figures with Fig. 17b, d.