Skip to main content
SpringerLink
Log in

Innovation, Evaluation and Measurement: Macro-Level and Firm-Level Perspectives

  • Chapter
  • First Online:
The Quadruple Innovation Helix Nexus

Abstract

Innovation is recognised as a difficult domain to assess and to measure. Innovation indicators are necessary to characterise innovation dynamics and to assess the effects of public policies supporting innovation or, from a micro perspective, return on investment, including the creation of conditions conducive to research, development and innovation activities. Moreover, it is also important to observe the role of the different actors, whether companies, the main drivers of innovation, or other entities in the innovation system. At country level, research has highlighted the importance of analysing innovation performance. Also, several studies have focused approaches developed by international organisations, in particular the European Commission. Part I of this chapter relates to a macro-level perspective applied to innovation measurement. Companies are eager to develop and apply methodologies contributing to capture innovation results. A problem in innovation management is how to do it. A framework model was proposed and applied, considering three levels of evaluation of innovation activities and projects, following a Return on Investment approach. Part II of this chapter aims to analyse how companies are managing innovation, which practices have been implemented and what framework could be designed to promote their capabilities to evaluate and measure innovation. It will contribute to the understanding of innovation measurement at firm level and to a more systematic approach for innovation management.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 119.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  • Abreu, M., Grinevich, V., Kitson, M., & Savona, M. (2010). Policies to enhance the “hidden innovation” in services: Evidence and lessons from the UK. The Service Industries Journal, 30(1), 99–118. http://doi.org/10.1080/02642060802236160

  • Adams, R., Bessant, J., & Phelps, R. (2006). Innovation management measurement: A review. International Journal of Management Reviews, 8(1), 21–47. http://doi.org/10.1111/j.1468-2370.2006.00119.x

  • Al-Aali, A., & Teece, D. J. (2013). Towards the (strategic) management of intellectual property: Retrospective and prospective. California Management Review, 55(4), 15–30.

    Article  Google Scholar 

  • Alexy, O., & Dahlander, L. (2014). Managing open innovation. In The Oxford handbook of innovation management. New York: Oxford University Press.

    Google Scholar 

  • Andrew, J. P., Haanaes, K., Michael, D. C., Sirkin, H. L., & Taylor, A. (2009). Measuring innovation 2009: The need for action (BCG senior management survey). Boston: The Boston Consulting Group Inc..

    Google Scholar 

  • Archibugi, D., & Coco, A. (2005). Measuring technological capabilities at the country level: A survey and a menu for choice. Research Policy, 34(2), 175–194. http://doi.org/10.1016/j.respol.2004.12.002

  • Archibugi, D., & Pianta, M. (1996). Measuring tecnological change through patents and innovation. Technovation, 16(9), 451–468.

    Article  Google Scholar 

  • Archibugi, D., Denni, M., & Filippetti, A. (2009). The technological capabilities of nations: The state of the art of synthetic indicators. Technological Forecasting and Social Change, 76(7), 917–931. http://doi.org/10.1016/j.techfore.2009.01.002

  • Armbruster, H., Bikfalvi, A., Kinkel, S., & Lay, G. (2008). Organizational innovation: The challenge of measuring non-technical innovation in large-scale surveys. Technovation, 28(10), 644–657. http://doi.org/10.1016/j.technovation.2008.03.003

  • Baghai, M., Smit, S., & Viguerie, S. P. (2008). The granularity of growth. McKinsey.

    Google Scholar 

  • Barney, J. (1991). Firm resources and sustained competitive advantage. Journal of Management, 17(1), 99–120. http://doi.org/10.1177/014920639101700108

  • Berkhout, F. (2014). Sustainable innovation management. In M. Dodgson, D. M. Gann, & N. Phillips (Eds.), The oxford handbook of innovation management. Oxford: Oxford University Press.

    Google Scholar 

  • Caetano, I. (2010). Métricas e Indicadores de Inovação. In Guia de Boas Práticas de Gestão da Inovação (3a ed., pp. 196–201). COTEC Portugal.

    Google Scholar 

  • Caraça, J., Ferreira, J., & Mendonça, S. (2006). Modelo de Interacções em Cadeia: Um modelo de Inovação para a Sociedade do Conhecimento. Report. COTEC Portugal.

    Google Scholar 

  • Caraça, J., Lundvall, B. A., & Mendonça, S. (2009). The changing role of science in the innovation process: From queen to cinderella? Technological Forecasting and Social Change, 76(6), 861–867.

    Article  Google Scholar 

  • Carayannis, E. G., & Campbell, D. F. J. (2012). Mode 3 knowledge production in quadruple helix innovation systems: 21st-century democracy, innovation, and entrepreneurship for development. New York: Springer.

    Book  Google Scholar 

  • Carayannis, E. G., & Provance, M. (2008). Measuring firm innovativeness: towards a composite innovation index built on firm innovative posture, propensity and performance attributes. International Journal of Innovation and regional Development, 1(1), 90–107.

    Article  Google Scholar 

  • Chesbrough, H. W. (2003). The era of open innovation. MIT Sloan Management Review, 44(3).

    Google Scholar 

  • Chiesa, V., & Frattini, F. (2009). Evaluation and performance measurement of research and development: Techniques and perspectives for multi-level analysis. Cheltenham: Edward Elgar.

    Book  Google Scholar 

  • CMMI Institute (2015). Why is measurement of data management maturity (DMM) so important? White paper. CMMI Institute.

    Google Scholar 

  • Cordero, R. (1990). The measurement of innovation performance in the firm: An overview. Research Policy, 19, 185–192.

    Article  Google Scholar 

  • Portugal, C. O. T. E. C. (2007). Innovation scoring: Support manual. COTEC Portugal: Lisbon.

    Google Scholar 

  • David, P. A., Hall, B. H., & Toole, A. A. (1999). Is public R&D a complement or substitute for private R&D? A review of the econometric evidence (NBER Working Papers 7373). Cambridge:National Bureau of Economic Research, Inc.

    Google Scholar 

  • Dervitsiotis, K. N. (2010). A framework for the assessment of an organisation’s innovation excellence. Total Quality Management & Business Excellence, 21(9), 903–918. http://doi.org/10.1080/14783363.2010.487702

  • Dewangan, V., & Godse, M. (2014). Towards a holistic enterprise innovation performance measurement system. Technovation, 34(9), 536–545.

    Article  Google Scholar 

  • Dodgson, M., Gann, D. M., & Phillips, N. (2014). Perspectives on innovation management. In M. Dodgson, D. M. Gann, & N. Phillips (Eds.), The Oxford handbook of innovation management (1st ed.). Oxford: Oxford University Press. Retrieved from http://www.oxfordhandbooks.com/view/10.1093/oxfordhb/9780199694945.001.0001/oxfordhb-9780199694945

  • Dosi, G., & Orsenigo, L. (1988). Coordination and transformation: An overview of structures, behaviours and change in evolutionary environments. In Technical change and economic theory (pp. 13–37). London: Columbia University Press.

    Google Scholar 

  • Edison, H., bin Ali, N., & Torkar, R. (2013). Towards innovation measurement in the software industry. Journal of Systems and Software, 86(5), 1390–1407. http://doi.org/10.1016/j.jss.2013.01.013

  • Edquist, C. (Ed.). (1997). Systems of innovation: technologies, institutions, and organizations. Washington/ London: Pinter.

    Google Scholar 

  • Edquist, C., & Zabala-Iturriagagoitia, J. M. (2015). The innovation union scoreboard is flawed: The case of Sweden – not being the innovation leader of the EU. Papers in Innovation Studies 2015/16, Lund University, CIRCLE – Center for Innovation, Research and Competences in the Learning Economy.

    Google Scholar 

  • Foyn, F. (2013). Innovation and R&D surveys in Norway. In Handbook of innovation indicators and measurement (pp. 135–177). Northampton: Edward Elgar Pub.

    Google Scholar 

  • Freeman, C. (1987). Technology policy and economic performance: Lessons from Japan. London: Pinter Publishers.

    Google Scholar 

  • Freeman, C., & Soete, L. (2009). Developing science, technology and innovation indicators: What we can learn from the past. Research Policy, 38(4), 583–589. http://doi.org/10.1016/j.respol.2009.01.018

  • Fujimoto, T. (2014). Innovation management in Japan. In The Oxford handbook of innovation management (1 ed., pp. 335–354). Oxford: Oxford University Press.

    Google Scholar 

  • Gama, N., Silva, M. M. da, & Ataíde, J. (2007). Innovation scorecard: A balanced scorecard for measuring the value added by innovation. In P. F. Cunha & P. G. Maropoulos (Eds.), Digital enterprise technology (pp. 417–424). Boston: Springer US. Retrieved from http://link.springer.com/10.1007/978-0-387-49864-5_49

  • Gatignon, H., Tushman, M. L., Smith, W., & Anderson, P. (2002). A structural approach to assessing innovation: Construct development of innovation locus, type, and characteristics. Management Science, 48(9), 1103–1122.

    Article  Google Scholar 

  • Gault, F. (2013). Innovation indicators and measurement: Challenges. In Handbook of innovation indicators and measurement. Northampton: Edward Elgar Pub.

    Chapter  Google Scholar 

  • Gault, F. (2014). Where are innovation indicators, and their applications, going? (UNU-MERIT Working Paper Series). United Nations University.

    Google Scholar 

  • Godinho, M. M. (2013). Inovação em Portugal. Fundação Francisco Manuel dos Santos.

    Google Scholar 

  • Grant, R. M. (1991). The resource-based theory of competitive advantage: Implications for strategy formulation. California Management Review, 33(3), 114–135. http://doi.org/10.2307/41166664

  • Grant, R. M. (1996). Toward a knowledge-based theory of the firm: Knowledge-based theory of the firm. Strategic Management Journal, 17(S2), 109–122. http://doi.org/10.1002/smj.4250171110

  • Griliches, Z. (1979). Issues in assessing the contribution of research and development to productivity growth. Bell Journal of Economics, The RAND Corporation, 10(1), 92–116.

    Article  Google Scholar 

  • Hall, B. H. (1996). The private and social returns to research and development. In B. L. R. Smith & C. E. Barfield (Eds.), Technology, R&D, and the economy (pp. 140–183). Washington: The Brookings Institution.

    Google Scholar 

  • Hall, B. H., Lotti, F., & Mairesse, J. (2013). Evidence on the impact of R&D and ICT investments on innovation and productivity in Italian firms. Economics of Innovation and New Technology, 22(3), 300–328. http://doi.org/10.1080/10438599.2012.708134

  • Hall, B. H., Mairesse, J., & Mohnen, P. (2010). Measuring the returns to R&D. In Handbook of the economics of innovation. Amsterdam: North Holland.

    Google Scholar 

  • Ijichi, T. (2013). Innovation surveys: Experience from Japan. In Handbook of innovation indicators and measurement (pp. 196–216). Northampton: Edward Elgar Pub.

    Google Scholar 

  • James, S. D., Leiblein, M. J., & Lu, S. (2013). How firms capture value from their innovations. Journal of Management, 39(5), 1123–1155. http://doi.org/10.1177/0149206313488211

  • Lay Tin, K. (2005). Measuring innovation performance (National Library Board, Singapore).

    Google Scholar 

  • Leonard, W. N. (1971). Research and development in industrial growth. Journal of Political Economy, 79(2), 232–256.

    Article  Google Scholar 

  • Lev, B. (2001). Intangibles: Management, measurement, and reporting. Washington: Brookings Institution.

    Google Scholar 

  • Lundvall, B. A. (Ed.). (1992). National systems of innovation: Towards a theory of innovation and interactive learning. London: Pinter Publishers.

    Google Scholar 

  • Maghsoudi, S., Duffield, C., & Wilson, D. (2015). Innovation evaluation: Past, current models and a framework for infrastructure projects. International Journal of Innovation Science, 7(4), 281–298. http://doi.org/10.1260/1757-2223.7.4.281

  • Manoochehri, G. (2010). Measuring innovation: Challenges and best practices. California Journal of Operations Management, 8(1), 67–73.

    Google Scholar 

  • Mansfield, E., Rapoport, J., Schnee, J., Wagner, S., & Hamburger, M. (1971). Research and innovation in the modern corporation. New York: Norton.

    Book  Google Scholar 

  • McGrath, R. G. (2013). Transient advantage. Harvard Business Review, 91, 62–70.

    Google Scholar 

  • Milbergs, E., & Vonortas, N. (2005). Innovation metrics: Measurement to insight. George Washington University.

    Google Scholar 

  • Muller, A., Välikangas, L., & Merlyn, P. (2005). Metrics for innovation: guidelines for developing a customized suite of innovation metrics. Strategy & Leadership, 33(1), 37–45. http://doi.org/10.1108/10878570510572590

  • Neely, A., Mills, J., Platts, K., Richards, H., Gregory, M., Boume, M., & Kennerley, M. (2000). Performance measurement system design: developing and testing a process-based approach. International Journal of Operations & Prodution Management, 20(10), 1119–1145.

    Article  Google Scholar 

  • Nelson, R. R. (1991). Why do firms differ, and how does it matter? Strategic Management Journal, 12, 61–74.

    Article  Google Scholar 

  • Nelson, R. R., & Winter, S. G. (1982). An evolutionary theory of economic change. Cambridge, MA: Belknap Press of Harvard University Press.

    Google Scholar 

  • OECD (2015). Frascati manual 2015: Guidelines for collecting and reporting data on research and experimental development. The measurement of scientific, technological and innovation activities. Paris: OECD Publishing.

    Google Scholar 

  • Perani, G. (2015). Empirical Studies of Business Innovation: lessons for the revision of the Oslo Manual, Eurostat G4, Power Point Presentation, Piacenza, 13.11.2015, accessible at: http://dipartimenti.unicatt.it/dises-Innovation_Perani_151113.pdf, on 28 Apr 2016.

  • Pereira, T. S. (2007). Enhancing science policy and management in south east Europe: S&T statistics and indicators systems (Science Policy Series No. 4). Veneza: UNESCO-BRESCE.

    Google Scholar 

  • Peters, B., & Rammer, C. (2013). Innovation panel surveys in Germany. In Handbook of innovation indicators and measurement (pp. 135–177). Northampton: Edward Elgar Pub.

    Google Scholar 

  • Planview (2014). A new framework for assessing your innovation program: Introducing the innovation management. White Paper. Planview.

    Google Scholar 

  • Porter, M. E. (1998). Clusters and the new economy. Harvard Business Review, 76(6).

    Google Scholar 

  • Radjou, N. (2004). Firms confront growing innovation demand. (Trends 30 Jun 2004). Forrester.

    Google Scholar 

  • Salter, A., & Alexy, O. (2014). The nature of innovation. In The Oxford handbook of innovation management. New York: Oxford University Press.

    Google Scholar 

  • Shenhar, A., & Dvir, D. (2007). Reinventing project management: The diamond approach to successful growth and innovation. Boston/Massachusetts: Harvard Business School Press.

    Google Scholar 

  • Simões, V. C. (2008). Improving innovation scoreboards: Finding a way forward. Presented at the IV Symposium COTEC Europa (27 June), Napoli, Italy.

    Google Scholar 

  • Smith, M., Busi, M., Ball, P., & Van Der Meer, R. (2008). Factors influencing an organisation’s ability to manage innovation: A structured literature review and conceptual model. International Journal of Innovation Management, 12(04), 655–676. http://doi.org/10.1142/S1363919608002138

  • Software Engineering Institute. (2010). CMMI® for development, version 1.3. Technical Report. http://www.sei.cmu.edu/reports/10tr033.pdf

  • Sprenger, M., Mettler, T., & Winter, R. (2016). A viability theory for digital businesses: Exploring the evolutionary changes of revenue mechanisms to support managerial decisions. Inf Syst Front. http://doi.org/10.1007/s10796-016-9638-x

  • Teece, D. J. (1986). Profiting from technological innovation: Implications for integration collaboration, licensing and public policy. Research Policy, 15, 285–305.

    Article  Google Scholar 

  • Teece, D. J. (2009). Dynamic capabilities and strategic management: Organizing for innovation and growth. Oxford: Oxford University Press.

    Google Scholar 

  • Teece, D. J. (2010). Technological innovation and the theory of the firm: The role of enterprise-level knowledge, complementarities, and (dynamic) capabilities. In Handbook of the economics of innovation. Amsterdam: North Holland.

    Google Scholar 

  • Teixeira, E. O., & Werther, W. B. (2013). Resilience: Continuous renewal of competitive advantages. Business Horizons, 56(3), 333–342. http://doi.org/10.1016/j.bushor.2013.01.009

  • van der Panne, G., van Beers, C., & Kleinknecht, A. (2003). Success and failure of innovation: A literature review. International Journal of Innovation Management, 07(03), 309–338. http://doi.org/10.1142/S1363919603000830

  • Walker, H., & Phillips, W. (2009). Sustainable procurement: Emerging issues. International Journal of Procurement Management, 2(1), 41. http://doi.org/10.1504/IJPM.2009.021729

  • Westerman, G., Bonnet, D., & McAfee, A. (2014). Leading digital: Turning technology into business transformation. Boston, MA: Harvard Business Review Press.

    Google Scholar 

  • Zhang, M. Y. (2014). Innovation management in China. In The oxford handbook of innovation management (1st ed., pp. 355–374). Oxford: Oxford University Press.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. COTEC Portugal and INDEG-ISCTE Executive Education, Lisbon, Portugal

    Isabel Caetano

Authors
  1. Isabel Caetano
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. P-BIO Portugal’s Biotechnology Industry Organization, Coimbra, Portugal

    Sara Paulina De Oliveira Monteiro

  2. Department of Information Systems and Technology Management, School of Business George Washington University, Washington, District of Columbia, USA

    Elias G. Carayannis

Appendices

Appendix A: Survey Questions Considered as Variables

Business strategy

1.1. The company has strategic objectives associated with innovation?

1.2. The company has an annual plan of innovation activities?

1.3. The company applies management indicators associated with innovation?

1.4. The company applies other indicators or metrics that somehow contribute to understand and to assess the impact of innovation?

Activities and Innovation Projects

2.1. The company applies quantitative criteria, and other such criteria, when evaluating RDI projects, particularly associated with the demonstration of their potential impact and return on investment?

2.2. The company applies quantitative criteria, and other such criteria, when assessing, in addition to RDI projects, other innovation activities such as the idea-management process or knowledge-management process?

2.4. The company develops and applies some technologies and tools supporting the management of innovation activities?

Accounting

3.1. The company applies R&D cost accounting in analytic terms?

3.1.2. If so, can you identify the investments, costs, and profits associating these terms with projects?

3.2. The company applies cost accounting, in analytic terms, to other innovation activities, in addition to R&D?

Notes

  1. 1.

    A previous version of this chapter was presented at ISPIM Conference in Porto on 22 June 2016. The author is grateful to COTEC Portugal and Deloitte Portugal, including to team members from ROI special project—Alexandre Andrade, Mariana Trigo Pereira, João Pereira and João Viegas, for support and exchange of information for part of this study. In addition, the author expresses gratitude to Professors Paulo Bento and Vitor Corado Simões for their encouragement, comments and suggestions.

  2. 2.

    ROI (%) = ((Gain from investment—Cost from investment)/Cost from investment) × 100.

Copyright information

© 2017 The Author(s)

About this chapter

Cite this chapter

Caetano, I. (2017). Innovation, Evaluation and Measurement: Macro-Level and Firm-Level Perspectives. In: De Oliveira Monteiro, S., Carayannis, E. (eds) The Quadruple Innovation Helix Nexus. Palgrave Studies in Democracy, Innovation, and Entrepreneurship for Growth. Palgrave Macmillan, New York. https://doi.org/10.1057/978-1-137-55577-9_7

Download citation

Publish with us

Policies and ethics

Search

Navigation