This article presents a set of performance indicators based on the Portuguese case for managing the performance of research and cultural creation activities and their impact on the regional context in polytechnic higher education institutions. Five theoretical dimensions have been proposed and empirically validated to frame the diversity of these activities and, for each one of them, a set of performance indicators (which total 29) was defined. The Delphi technique was applied to reach consensus among specialists (polytechnic higher education institutions’ presidents) on the performance indicators; two rounds of the technique application have led to a final set of indicators (comprising 23 indicators). The final set of performance indicators emerged as valid to actually be used by polytechnic higher education institutions, not only as a self-instrument to assess and improve their performance, but also as a management tool allowing the institutions to establish goals, define their own profile and the strategies to achieve it.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
The main theoretical explanations regarding the differences between commercialization and engagement activities and the main inspiration for the framework can be found in Perkmann et al. (2013).
Abreu, M., Grinevich, V., Hughes, A., & Kitson, M. (2009). Knowledge exchange between academics and the business, public and third sectors. London: University of Cambridge and Imperial College London.
Cave, M., Hanney, S., Henkel, M., & Kogan, M. (1997). The use of performance indicators in higher education: The challenge of the quality movement. London: Jessica Kingsley.
E3M (2010). “Needs and constraints analysis of the three dimensions of third mission activities”, E3M: European indicators and ranking methodology for university third mission. www.e3mproject.eu/docs/Three-dim-third-mission-act.pdf. Accessed 12 March 2018.
Ferreira, V. (1999). O Inquérito por Questionário na Construção de Dados Sociológicos. In A. S. Silva & J. M. Pinto (Eds.), Metodologia das Ciências Sociais (pp. 165–196). Porto: Edições Afrontamento.
File, J., de Weert, E., Vossensteyn, H., Kaiser, F., Jongbloed, B., Goedegebuure, L., et al. (2013). Policy challenges for the Portuguese polytechnic sector: A report for the Portuguese polytechnics coordinating council (CCISP). Twente: CHEPS.
Gibbons, M., Nowotny, H., Schwartzman, S., Trow, M., Limoges, C., & Scott, P. (1994). The new production of knowledge: The dynamics of science and research in contemporary societies. London: Sage.
Gordon, T., & Pease, A. (2006). RT Delphi: An efficient, “round-less” almost real time Delphi method. Technological Forecasting and Social Change, 73(4), 321–333.
Hasson, F., & Keeney, S. (2011). Enhancing rigour in the Delphi technique research. Technological Forecasting and Social Change, 78, 1695–1704.
Hsu, C.-C., & Sandford, B. A. (2007). The Delphi technique: Making sense of consensus. Practical Assessment, Research & Evaluation, 12, 1–8.
Hughes, A., & Kitson, M. (2012). Pathways to impact and the strategic role of universities: New evidence on the breadth and depth of university knowledge exchange in the UK and the factors constraining its development. Cambridge Journal of Economics, 36(3), 723–750.
Kyvik, S. (2004). Structural changes in higher education systems in Western Europe. Higher Education in Europe, 29(3), 393–409.
Landeta, J. (2006). Current validity of the Delphi method in social sciences. Technological Forecasting and Social Change, 73(5), 467–482.
Linstone, H. A., & Turoff, M. (Eds.) (2002). The Delphi method: Techniques and applications. New Jersey: “Authors”.
Machado, M. d. L., Brites Ferreira, J., Santiago, R., & Taylor, J. (2008). Reframing the non-university sector in Europe: Convergence or diversity? In J. Taylor, J. Brites Ferreira, M. d. L. Machado, & R. Santiago (Eds.), Non-university higher education in Europe (pp. 245–260). Dordrecht: Springer.
Melo, A. I., & Sarrico, C. S. (2015). Performance management systems and their influence on the governance structures of Portuguese universities: A case study. In I. M. Welpe, J. Wollersheim, S. Ringelhan, & M. Osterloh (Eds.), Incentives and performance (pp. 81–97). Cham: Springer International Publishing Switzerland.
Melo, A. I., Sarrico, C. S., & Radnor, Z. (2010). The influence of performance management systems on key actors in universities. Public Management Review, 12(2), 233–254.
Ministério da Ciência, Tecnologia e Ensino Superior. (2015). Programa de modernização e valorização dos institutos superiores politécnicos. Lisboa: Ministério da Ciência, Tecnologia e Ensino Superior.
Neave, G. (1979). Academic drift: Some views from Europe. Studies in Higher Education, 4(2), 143–159.
Perkmann, M., Tartari, V., McKelvey, M., Autio, E., Broström, A., D’Este, P., et al. (2013). Academic engagement and commercialisation: A review of the literature on university-industry relations. Research Policy, 42, 423–442.
Pollitt, C., & Bouckaert, C. (2011). Public management reform. Oxford: Oxford University Press.
REF 2014. (2011). Assessment framework and guidance on submissions. Bristol: REF.
Rosa, M., Tavares, D., & Amaral, A. (2006). Funding systems and their effects on higher education systems – Country study – Portugal. Paris: OECD.
Saaty, T. L. (2004). Decision making – The analytic hierarchy and network processes (ahp/anp). Journal of Systems Science and Systems Engineering, 13(1), 1–35.
Samfira, E.-M., & Raţă, G. (2015). Assessing education needs at tertiary level: The focus group method. Animal Science and Biotechnologies, 48(2), 223–227.
Sarrico, C. (2010). Indicadores de Desempenho para Apoiar os Processos de Avaliação e Acreditação de Cursos. Lisboa: Gabinete de Estudos e Análise, A3ES.
Stokes, D. E. (1997). Pasteur's quadrant: Basic science and technological innovation. Brookings Institution Press.
Tecău, A. S., & Tescaşiu, B. (2015). Nonverbal communication in the focus-group. Bulletin of the Transilvania University of Brasov – Series V: Economic Sciences, 8(2), 119–125.
Teixeira, P., & Dill, D. (Eds.). (2011). Public vices, private virtues – Assessing the effects of marketization in higher education. Rotterdam: Sense Publishers.
Tight, M. (2015). Theory development and application in higher education research: The case of academic drift. Journal of Educational Administration and History, 47(1), 84–99.
Van Vught, F. & Ziegele, F., (2010). U-Multirank: Design and testing the feasibility of a multi-dimensional global university ranking. Twente: University of Twente, CHEPS. https://www.eurashe.eu/library/modernising-phe/mobility/quality/WG4%20R%20multirank_en%20final.pdf. Accessed 12 March 2018.
Van Vught, F,. Kaiser, F., File, J., Gaethgens, C., & D. Westerheijden (2010). U-Map: The European Classification of Higher Education Institutions. Twente, University of Twente, https://ris.utwente.nl/ws/portalfiles/portal/5141399/U-MAP_final_report.pdf. CHEPS. Accessed 12 March 2018.
Wright, J., & Giovinazzo, R. (2000). Delphi – Uma ferramenta de apoio ao planejamento prospectivo (Delphi – A support tool towards prospective planning). Cadernos de Pesquisas em Administração, 1(12), 54–65.
This research was supported by the Portuguese Foundation for Science and Technology (FCT) under grants 02/INDICADORES/2014 and 05/INDICADORES/2014. It has also been supported by FCT through project PEst-OE/CED/UI0757/2013 (funded by the Programme COMPETE). This work was also financially supported by the project POCI-01-0145-FEDER-029427- funded by FEDER, through COMPETE2020 - Programa Operacional Competitividade e Internacionalização (POCI), and by national funds (OE), through FCT/MCTES.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Biscaia, R., Melo, A.I., Machado, I. et al. Performance indicators for research and cultural creation activities in polytechnic higher education institutions: a consensus building approach. Tert Educ Manag (2019). https://doi.org/10.1007/s11233-019-09035-y
- Performance management
- Performance measurement
- Higher education policy
- Performance indicators
- Higher education institutions
- Research policy