Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Creative resources in the life sciences in academic research and in industry are not being used efficiently. This phenomenon, often described as the innovation gap or science-to-market gap [1], is very much to the detriment of the economy and society. The core strategy of the European Union’s “Europe 2020” features the “Innovation Union” as one of seven flagship initiatives “to ensure that innovative ideas can be turned into products and services that create growth and jobs [2].” To meet the challenges of the global knowledge society, European governments and other public funding bodies increasingly demand that public research organizations, including universities (together called PROs), exploit their results from invention and discovery through technology transfer (TT). This so-called “third task”—in addition to the tasks of research and teaching—enable PROs to capitalize more fully on public funding for the benefit of society. Innovations stemming from public research will be defined in this chapter as value-creating novelties [3].

The commercialization activities arising out of public research are typically performed by a technology transfer office (TTO). In essence, the task of a TTO is to find a commercial partner willing and able to develop and further test novel ideas into product solutions that people are willing to pay for, thereby creating value. TT is defined by the Organization for Economic Co-operation and Development (OECD) as intellectual property (IP) management, whose purpose is “to identify, protect, exploit and defend intellectual property [4].”

The question how efficiently or effectively this innovation process works has to be asked for the PRO as a whole. This is because it reflects both its output and the outcome of research production. To quote the old saying: “If it isn’t measured, it can’t be managed,” the purpose of performance measurement is thus to create transparency of results and achievements, support the defined strategy and decision making, and provide guidance for behavioral change, such as integrating innovation into research operations. Because no indicators exist either for measuring the intangible results of research or for creating multidimensional outputs from research or the quality of interaction among the players involved, a systematic attempt will be made to design a flexible and holistic concept of performance measurement. “Holistic” according to [5] incorporates:

  • Focus on relevant stakeholders

  • Balance between the resulting demands (as far as possible)

  • Integration in a systems approach, and

  • Sustainable and long term in orientation.

2 Research Production and Innovation

Holistic performance measurement of TT must take into account all dimensions at the level of the PRO. Detailed information is needed about the causal linkages of the setting in which innovation occurs, the processes of innovation, and the outcome of innovation.

PROs are an essential component in the innovation chain. Faculty and researchers produce basic novel knowledge, some of which is inventive and patentable. The bulk of these inventions occur early in the discovery phase of the innovation life cycle. Far from being a true product, they require additional research and experiments to further refine and advance these early inventions and to strengthen the patent application prior to commercialization. Today, many TTOs have created and manage a gap fund to finance such proof-of-concept studies to reduce risk and further advance their inventions in a bid to make them more attractive to industry.

One causal linkage deserves special emphasis. Patents as standard innovation indicators play a central role in the commercialization of life science inventions because only patents provide the temporary exclusivity and protection required by the pharmaceutical industry to recoup its huge investments in drug development. Thus patenting in academia deserves close scrutiny. This is because inventing and patenting are in fact two separate phenomena [6] that require different strategies and processes. Responsibilities must be clearly allocated to ascertain who is in control of which part of the innovation process.

Usually it is the researcher’s decision to disclose their invention that triggers the commercialization process at the TTO. If the researcher decides only to publish and not to reveal the invention to the TTO, their invention will be lost for commercialization since it will become public knowledge and no longer be proprietary. Thus, failing to disclose inventions represents the first gap in the innovation chain. It is a myth to believe that a TT manager can just “walk the halls” and pick inventions like cherries that after commercial evaluation will be patented and successfully marketed. Researchers themselves are responsible for disclosing their invention to the TTO. Therefore, invention disclosures constitute the only direct measure of researchers’ innovation activities. By contrast, the number of patents (patent applications, patents granted) reflects the output of research as a whole, including the TTO’s activities. This accords with the assistant role of the TTO as described in the “assisted innovation model” [7]. The TTO forms both an integral part of the research production process and the linear innovation chain, facilitating moving inventions further along the value chain, thus closing the gap [7].

As patents are created as a result of the co-production between the researchers and their TTO, a PRO must take a twofold approach to capture the value of its investment in research and discovery: (1) ensure that all possible inventions are disclosed and (2) maintain a TTO ready and able to mine these inventions successfully. However, bearing in mind that the chances of meaningful commercial success are very low (estimated at below 1% of all invention disclosures), success can be defined as collaborations with industry, licensing to an established company, or founding a spin-off. Different inputs will yield different outputs and outcomes, just as the efficiency and effectiveness of processes will differ with expertise and experience. Although several indicators and benchmarks already exist for TT, they must be evaluated (holistically) at the level of the PRO.

The Production Model of Research: Holistic performance measures for innovation from research have been developed using the decision-oriented production model of research [8]. This approach has a twofold advantage. First, it makes it possible to adapt performance measurement to the relevant decision-maker or stakeholder. Second, it is a tool to define the extent to which a PRO wishes to engage in innovation and take responsibility for it. The model contains four dimensions at the level of the PRO: input, processes, output, and outcome. These dimensions, if aligned sequentially, form four elements in the innovation chain (Fig. 8.1).

Fig. 8.1
figure 1figure 1

Decision-oriented research production model aligned for holistic performance measurement along the four elements of the innovation chain. The elements in the top row represent the research elements and those in the bottom row the commercial elements leading to innovation. Both must be provided with adequate resources (input) and activities (processes) to yield the desired output and outcome. The outcome “innovation” is only obtained by achieving market impact (modified according to [3, 8])

Full size image

Input

  • Structure and resources (funding, personnel) available for research and TT, including the support of the commercialization process.

Processes

  • Research production of new knowledge includes all research activities, specifically inventions, plus all services offered by the TTO. If inventions are not disclosed to the TTO, internal innovation gaps may arise, leading to an early break in the innovation chain.

Output

  • Productivity and efficiency of the PRO with respect to publications, patents, and commercialization results such as licensing contracts.

Outcome

  • The overall impact on society can be divided into institutional impact and market impact. Only market impact can be associated with innovation (any value generated such as licensing income can be reinvested in research input).

3 Conceptual Design of Holistic Performance Measurement in Technology Transfer

The TT process is complex in nature because it must take into consideration the interdependence between input, research production, TT, the various stakeholders, and the multiple goals of a PRO. To replicate this complexity, the conceptual design of performance management needs to make use of the above-mentioned decision-oriented production model of research [8] and should contain the following elements:

  • Analysis of the interests of the relevant stakeholders: government, management, and the researchers themselves

  • Alignment of possible goals to respective indicators

  • Examples of evaluation methods applicable to TT

3.1 Stakeholders in Technology Transfer

The most relevant stakeholders in the innovation process in the academic context are the government or other funding agencies, the PRO management, and the researchers themselves. Each of them will have a different interest in innovation, thus producing multiple, potentially conflicting goals.

The Government: The government (or other public funding bodies) funds the production of new knowledge, stipulating that funding be used sparingly and economically by maximizing desirable side effects (such as TT and international collaborations, equal opportunities, and promoting young researchers) and at the same time minimizing risks. The government might also fund specific programs to foster innovation (such as for establishing spin-off companies).

Management: The PRO’s management is responsible for the overall strategy and budget from public money.Footnote 1 The management sets priorities and balances the multiple interests of the funding bodies and other relevant interest groups. The strategy is often devised together with the so-called visible scientists whose expert knowledge constitutes the most important capital of a PRO. Managing such experts, who thrive on the privilege of academic freedom, is admittedly challenging. Since the research budget is generally decreasing rather than increasing, any change in strategy will result in reallocating funds, such as for those for innovation, at the expense of other activities. Without management’s support it might be difficult to pursue commercialization activities efficiently. Management might also prefer short-term visible achievements over long-term investments in innovation. As such, management might choose to establish collaborations with industry creating immediate research money, rather than investing long term in the infrastructure for TT. This is regardless of how small the investment is compared with research expenses. Ultimately, some PROs may wisely do both.

Faculty: Researchers and inventors certainly constitute the most important stakeholders in innovation. It is well known that their motivation influences the outcome of TT [9, 10]. Researchers have three ways to convey information about their work results—publish, patent or do both. However, not all researchers are equally motivated to patent. A researcher is primarily motivated by “solving the puzzle” [6], that is, answering scientific questions not necessarily connected to patenting or commercialization. Second, researchers are driven to receive recognition from their peers and to establish their reputation within their scientific community. Third, money does not seem to be the primary motivation for most researchers because most research fails to generate patentable and commercially viable inventions. Therefore, although inventors usually receive a share of about 30% of the revenues received by the PRO, this motivation is the least powerful. This share can be substantial in a very few cases, but might take a long time to materialize due to long development times.

The propensity to disclose inventions differs among researchers. Those performing basic research are less motivated to patent compared with those who are willing to invest additional time to validate their invention. This also holds true for emerging young talents who first have to build their scientific reputation. In essence, researchers will be motivated to engage in commercial activities the more they feel their resulting share will be adequate in relation to the efforts required [6]. Role models (and potentially feelings of envy) and the fact that some researchers like to “gamble” are further factors affecting researchers’ propensity to patent.

3.2 Indicators and Goals in Technology Transfer

Bearing the stakeholders in mind, the most visible indicator of added value from public research is reflected in the millions of dollars in licensing revenues that some of the large American universities receive. However, this does not automatically mean that these universities and their TTOs are best in class or best in process. This is because very few extremely successful (blockbuster) inventions can generate huge licensing revenues, whereas the bulk of inventions produce only a lower income. As a result, only 16% of TTOs in the United States operate profitably, while more than 50% accrue losses [10]. Moreover, licensing revenues produced by public research constitute only a fraction of the impact and benefit to the economy and society. They reflect market success, which can be neither planned nor influenced by the TTO or PRO. Using licensing revenues as the main indicator of the quality of the TTO or outcome of research is thus not recommended [11].

The set of indicators from the annual Association of University Technology Managers (AUTM) survey represents the gold standard for measuring TTO performance in the United States and Canada [12]. The survey contains a wealth of information collected for more than 30 years and has been expanding to include other continents and countries. In fact, the European Commission (EC) published a guideline with seven key quantitative indicators with definitions matching the AUTM survey [13]. For example, these allow comparison with the United States on a country level or benchmarking with specific institutions:

  • Three indicators highlight the commercial potential of public research and constitute the output of research and the prerequisites for commercialization: (1) number of invention disclosures, (2) number of patent applications, and (3) number of patents granted.

  • Three indicators reflect the commercialization of public research by companies: (4) number of new licensing agreements, (5) number of spin-offs and (6) gross licensing income (in Euros or dollars) and thus measure the outcome of TT.

  • In addition, one indicator was introduced that serves as a process indicator: (7) the number of new collaboration agreements with industry (excluding consortia being funded by the EC or other public money). This makes it possible to determine the focus of the transfer activities (i.e., collaboration with industry or licensing).

Care must be taken to measure performance with indicators relevant to the goals and strategy of a PRO. Measuring the wrong things may subsequently lead innovation efforts down the wrong path. Four goals in TT are usually identified: service, transfer, profit, and regional development. Each goal can be further broken down into strategic and operational goals and connected with one key indicator as follows [14].

Service: Service for the researchers as customers means focusing on the internal relationships. This is done, for example, by delivering prompt service, providing expertise in business development in specific research areas, and minimizing transaction times. The key indicator for service is the satisfaction survey conducted for a customer. Note: Because the researcher may demand services that far exceed the budget, satisfaction surveys could be conducted cost-effectively via a focused interest group.

Transfer: Maximizing transfer means maximizing marketing to industry (i.e., e­xternal relationships). For each invention the TTO assesses the most potent way of commercialization, giving preference to exclusive licenses and web-based click-licensing. Researchers should be highly motivated and hand in as many invention disclosures as possible. The number of transferred technologies represents the key indicator for transfer. Note: This is a composite indicator and not so easy to assess. Alternatively, the number of license deals can be used as a more direct indicator.

Profit: Maximizing profit involves focusing on large markets with a high potential income. It is achieved, for example, by aggressively collecting royalties and pursuing IP infringers. The key indicator for profit is gross revenue minus direct costs. Note: Although this goal is pursued primarily by only 10% of the TTOs [9], revenues will always be an important measure of innovation.

Regional Development: Creating jobs in the region in particular is done by providing extensive coaching and teaching on how to set up a company and establishing links to science parks and investors. The key indicator here is the number of spin-offs and the number of jobs created. Note: Universities usually have a closer affinity to their region than non-university research centers.

Rather than just listing the four goals, it is preferable to identify and pursue one primary goal [14]. However, all of the above goals are essential to TT activities, and none of them can be ignored. Therefore, it is important to balance these goals and set priorities that can be changed from time to time if deemed necessary. Goals 1 and 2 are both personnel-intensive, because they require extensive marketing efforts. Although the extent to which they are pursued might be restricted by staff and budgetary limitations, marketing (internal and external) needs to be continuously improved.

3.3 Evaluation Methods and Best Practices

The above analysis on stakeholders’ interests, goals, and indicators provides a systematic framework for holistic performance measurement. Making use of the concept outlined above, a PRO will first need to design a strategy geared to innovation and then start measuring performance operationally. Thus, prior to evaluation, the expectations, purpose, and goals will be laid out and provide the necessary guidance.

Strategically, after reaching agreement on what is relevant in innovation or TT, a PRO will formulate its mission statement and goals or even stipulate them in its statues or policies. The process may well create tensions and trigger discussions on the importance of traditional academic values versus engaging in entrepreneurial activities and creating added value. However, most, if not all, PROs are subject to performance-based budgeting which increasingly includes patent indicators and other innovation measures.

Operationally, a PRO will need to initiate an evaluation cycle or integrate the performance measurement of the TT activities into its own evaluation culture and make use of established evaluation methods. At the Deutsches Krebsforschungszentrum the standard process every 5 years includes self-evaluation, benchmarking with others, and external review. This process is described in more detail below.

Self-evaluation: Self-evaluation involves an analysis of the current situation, for example by summarizing TT results in a business report or plan, including an estimate of the value of the patent portfolio and a forecast of potential licensing income. A SWOT analysis is a helpful tool to visualize strengths, weaknesses, opportunities and threats and to devise action items.

Benchmarking: Benchmarking compares one’s own performance with others, searching for the best processes, activities, and results. The aim is to learn from them to improve one’s own processes, activities, and results. Quantitative data are available from the AUTM survey. Other data may be obtained from the literature [3, 9, 15, 16]. For a true comparison, the data must be standardized either with respect to the number of researchers potentially producing patentable results or the amount of research dollars received in relevant fields (engineering, medicine, life sciences, natural sciences, computing, and so on).

External Evaluation: External evaluation might be conducted as a peer review on site. Peer review is admittedly time-consuming and has often been criticized but it remains the oldest and the most widely accepted qualitative tool for monitoring research. If done regularly in a standardized manner, such evaluation will provide a feedback mechanism for continuous improvement and learning. For the evaluation of TT activities a mixture of the following types of experts with different professional specialties should avoid creating any bias: administrative or scientific directors of a PRO, directors of TT offices, representatives from industry and venture capital and, ideally, reputable researchers with commercial experience.

Such external evaluation may even be performed to support management in devising a strategy for TT and mapping the ensuing operational steps. Management will traditionally include visible scientists and relevant committees. Such on-site peer review will result in a written expert’s opinion on topics such as (1) culture and ethos, including mission, self-image, internal and external outreach, (2) governance of the TTO, (3) business activities (IP management, collaborations with industry, and startup formation), (4) business development activities including gap or validation funding, (5) strategic orientation for the future, and (6) staffing.

For holistic performance management, it is useful to formulate a set of key questions along the four dimensions of the research production model with respect to innovations (input, processes, output, and outcome). Table 8.1 summarizes a set of key questions and indicators. Moreover, these questions may be applied to create an innovation score card as described by Levy [3].

Table 8.1 Key questions in four dimensions to holistically measure performance and innovative activity of a PRO (modified from [3])
Full size table

4 Conclusions and Recommendations

In summary, this chapter has detailed the elements of holistic performance measurement for PROs to fulfill their roles of fostering, supporting, and enabling innovation and managing their “third task” activities. Although much has already been achieved there is still untapped potential for innovation from academia. Unraveling this potential is in fact highly complex. The decisive measurable first step for researchers is disclosing their inventions before publishing. The number of unreported cases is not known, but they constitute the innovation potential to be addressed. Patenting and publishing can be performed simultaneously. However, because researchers are under great pressure to publish their findings, there might not be enough time to exemplify promising inventions prior to patenting, thus making it difficult to produce strong patents for commercialization. TTOs are more directly involved in the innovation process and are thus a genuine organizational mechanism that closes the gap between invention and innovation. More still needs to be learned about the consequences of introducing measures, such as performance-based budgeting. It is hoped—and there is reason to hope—that fostering innovation will not lead to a decrease in the quality of basic research.

Integrating innovation as a third task into any research organization requires a holistic approach, because innovation may be perceived as counter to the tradition and culture of academic institutions. However, changes are necessary to increase innovation activities within a PRO for the benefit of society. Changes are already happening with novel therapeutic concepts, such as personalized or translational medicine. PROs have assumed responsibilities in early drug discovery formerly only vested with the pharmaceutical industry, such as establishing high-throughput screening facilities, medicinal chemistry, and early clinical trials [17]. The key aspect of this public commitment is the alarmingly decreasing number of innovative new medicines over a period of many years. This downward trend is detrimental to the healthcare system and to patients [18]. However, companies such as Genentech are still very successfully developing new and innovative drugs. There is reason to believe that their special skills in turning academia’s novel ideas into novel drugs constitute the key factor for success.

Inspired by this example, many pharmaceutical companies are now creating novel collaboration models with academic institutions in early drug discovery. Both pharmaceutical companies and academia are striving to overcome cultural and organizational barriers and boundaries between them and thus are becoming more flexible in interacting with one another. Learning to integrate the best of both worlds seems like a promising way to create added value. For example, the collaborative innovation alliance between the Deutsches Krebsforschungszentrum and Bayer Healthcare can be highlighted where joint projects are performed together along defined milestones, directly feeding successful projects into the internal Bayer pipeline. This risk-and-reward sharing partnership model is based on interactions at all organizational levels in an open and mutually beneficial atmosphere of an exchange of ideas [19]. In a few years’ time the ultimate measure of success of such academic institutions and industry alliances will be the introduction of innovative treatments into patient care.