Abstract
Key performance indicators are crucial for the strategic planning of both, public and private institutions. Composite indicators are a useful tool for conveying summary information about overall performance in a relatively simple way. The construction of these indicators implies several stages concerning collection of data, selection of criteria and individual indicators, normalization and weighting of criteria and indicators, aggregation and comparison of overall performance of the alternatives or options. However, due to the different nature of the decision criteria and indicators traditional normalization methods are not always suitable. This is the case in the proposal of composite indicators measuring students’ adequacy degree in higher education institutions. This concept involves several dimensions, criteria and individual indicators of diverse nature. In this paper, we try to overcome the normalization problems arising from this situation proposing a normalization method based on similarity of the options with an ideal target. Once indicators have been normalized, weights are determined and aggregation is conducted in an easy way allowing the overall assessment of the students in terms of their adequacy level. Based on this overall assessment an adequacy index is then proposed allowing classification of the students in three different groups reflecting different levels of difficulties in terms of their adequacy. Extra activities can be then planned by the institution in order to re-inforce the adequacy level of the students. The adequacy index proposal has been developed in the context of the Industrial University of Santander (Colombia).
Similar content being viewed by others
Notes
The current legal minimum monthly wages are known in Colombia as SMMLV, which is the Spanish acronym for “Salario Mínimo Mensual Legal Vigente”.
The diagnostic test of mathematics-UIS, is intended to measure, through learning outcomes, the degree to which students achieve certain competencies.
The EFAI-4 test is the Factorial Assessment of Intellectual Skills for people 16 years of age or over. In our case, only the scores obtained in the numerical test will be taken into account.
The test of Saber 11 is the examination carried out by the Colombian State for Admission to Higher Education. For this paper, only the scores obtained by the students in the Mathematics test were taken into account.
References
Acuña-Soto, C. M., Liern, V., & Pérez-Gladish, B. (2018). Multiple Criteria performance evaluation of YouTube mathematical educational videos by IS-TOPSIS. Operational Research International Journal. https://doi.org/10.1007/s12351-018-0405-2.
Antunes da Silva, F., & Borsato, M. (2017). Organizational Performance and Indicators: Trends and Opportunities. Procedia Manufacturing, 11, 1925–1932.
Asif, M., & Searcy, C. (2014). A composite index for measuring performance in higher education institutions. International Journal of Quality & Reliability Management, 31(9), 983–1001.
Badawy, M., Abd El-Aziz, A. A., Idress, A. M., Hefny, H., & Hossam, S. (2016). A survey on exploring key performance indicators. Future Computing and Informatics Journal, 1, 47–52.
Behzadian, M., Otaghsara, S. K., Yazdani, M., & Ignatius, J. (2012). A state-of the-art survey of TOPSIS applications. Expert Systems with Applications, 39, 13051–13069.
Bradley, G. (2013). Professional suitability of students at the point of selection: What is it realistic to assess? British Journal of Occupational Therapy, 76(4), 200–202.
Cables, E., Lamata, M. T., & Verdegay, J. L. (2016). RIM—reference ideal method in multicriteria decision making. Information Sciences, 337–338, 1–10.
Canós, L., Casasús, T., Liern, V., & Pérez, J. C. (2014). Soft computing methods for personnel selection based on the valuation of competences. International Journal of Intelligent Systems, 29, 1079–1099.
Epstein, M. J., & Roy, M. J. (2001). Sustainability in action: Identifying and measuring the key performance drivers. Long Range Planning, 34, 585–604.
Freudenberg, M. (2003). Composite Indicators of Country Performance: a Critical Assessment. In OECD Science, Technology and Industry Working Papers, 2003/16. Organisation for Economic Co-operation and Development (OECD), Paris.
Gaudreault, C., Samson, R., & Stuart, P. (2009). Implications of choices and interpretation in LCA for multi-criteria process design: de-inked pulp capacity and cogeneration at a paper mill case study. Journal of Cleaner Production, 17(17), 1535–1546.
Gnaldi, M., & Ranalli, G. (2016). Measuring University performance by means of composite indicators: A robustness analysis of the composite measure used for the Benchmark of Italian Universities. Social Indicators Research, 129(2), 659–675.
Hwang, C. L., & Yoon, K. (1981). Multiple attribute decision making: Methods and applications. New York: Springer.
Ibáñez-Forés, V., Bovea, M. D., & Pérez-Belis, V. (2014). A holistic review of applied methodologies for assessing and selecting the optimal technological alternative from a sustainability perspective. Journal of Cleaner Production, 70, 259–281.
Ivanova, I., Arcelus, F. J., & Srinivasan, G. (1999). An assessment of the measurement properties of the human development index. Social Indicators Research, 46(2), 157–179.
Karavanas, A., Chaloulakou, A., & Spyrellis, N. (2009). Evaluation of the implementation of best available techniques in IPPC context: an environmental performance indicators approach. Journal of Cleaner Production, 17(4), 480–486.
Kerzner, H. (2011). Project management metrics, KPIs, and dashboards. New Jersey: Wiley.
Miller, J., & Koerin, B. (1998). Can we assess suitability at admission? A review of MSW application procedures. Journal of Social Work Education, 34(3), 437–453.
Munda, G. & Nardo, M. (2005). Constructing consistent composite indicators: The issue of weights. European Commission Directorate-General Joint Research Centre Institute for the Protection and Security of the Citizen European Communities: EC Publishing.
Munda, G., Nardo, M., Saisana, M., & Srebotnjak, T. (2009). Measuring uncertainties in composite indicators of sustainability. International Journal of Environmental Technology and Management, 11, 7–26.
Murias, P., de Miguel, J. C., & Rodriguez, D. (2008). A composite indicator for university quality assessment: The case of Spanish higher education system. Social Indicators Research, 89(1), 129–146.
Nardo, M., Saisana, M., Saltelli, A., & Tarantola, S. (2015). Tools for composite indicators building, European Commission, EUR 21682 EN. Italy: Institute for the Protection and Security of the Citizen, JRC Ispra.
OECD. (2008). Handbook on constructing composite indicators: Methodology and user guide. OECD: OECD Publishing.
Ouenniche, J., Pérez-Gladish, B., & Bouslah, K. (2017). An out-of-sample framework for TOPSIS-based classifiers with application in bankruptcy prediction. Technological Forecasting and Social Change. https://doi.org/10.1016/j.techfore.2017.05.034.
Parada, S. E., Fiallo, J. E., & Blasco, O. (2015). Construcción de indicadores sintéticos basados en juicio experto: Aplicación a una medida integral de la excelencia académica. Rect@, 16, 51–67.
Pedrycz, W., Ekel, P., & Parreiras, R. (2010). Fuzzy multicriteria decision-making: Models, algorithms and applications (1st ed.). New York: Wiley.
Saisana, M., Saltelli, A., & Tarantola, S. (2005). Uncertainty and sensitivity analysis techniques as tools for the quality assessment of composite indicators. Journal of the Royal Statistical Society Series A (General), 168, 307–323.
Saisana, M. & Tarantola, S. (2002). State-of-the-art report on current methodologies and practices for composite indicator development. European Commission, Joint Research Centre, Institute for the Protection and the Security of the Citizen, Technological and Economic Risk Management Unit.
Saltelli, A. (2007). Composite indicators between analysis and advocacy. Social Indicators Research, 81(1), 65–77.
SEA-UIS, (2018). https://www.uis.edu.co/webUIS/es/estudiantes/excelenciaAcademica/index.html. Accessed 15 Oct 2017.
Sharpe, A. (2004). Literature review of frameworks for macro-indicators. Ottawa: Centre for the Study of Living Standards.
Smilkstein, G. (1978). The family APGAR: A proposal for family function test and its use by physicians. The Journal of Family Practice, 6(6), 1231–1239.
Tangsubkul, N., Beavis, P., Moore, S. J., Lundie, S., & Waite, T. D. (2005). Life cycle assessment of water recycling technology. Water Resource Management, 19(5), 521–537.
Vega, A., Aguarón, J., García-Alcaraz, J., & Moreno-Jiménez, J. M. (2014). Notes on dependent attributes in TOPSIS. Procedia Computer Science, 31, 308–317.
Vignes, R. P. (2001). Use limited life-cycle analysis for environmental decision-making. Chemical Engineering Progress, 97(2), 40–54.
Wang, Y. J. (2014). A fuzzy multi-criteria decision-making model by associating technique for order preference by similarity to ideal solution with relative preference relation. Information Sciences, 268, 169–184.
Zeng, W., & Guo, P. (2008). Normalized distance, similarity measure, inclusion measure and entropy of interval-valued fuzzy sets and their relationship. Information Sciences, 178, 1334–1342.
Zheng, B. (1993). An axiomatic characterization of the Watts poverty index. Economics Letters, 42, 81–86.
Zhou, P., & Ang, B. W. (2009). Comparing MCDA aggregation methods in constructing composite indicators using the Shannon-Spearman measure. Social Indicators Research, 94, 83–96.
Zhou, P., Ang, B. W., & Zhou, D. Q. (2010). Weighting and aggregation in composite indicator construction: a multiplicative optimization approach. Social Indicators Research, 96, 169–181.
Zyoud, S. H., & Fuchs-Hanusch, D. (2017). A bibliometric-based survey on AHP and TOPSIS techniques. Expert Systems with Applications, 78, 158–181.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Parada, S.E., Blasco-Blasco, O. & Liern, V. Adequacy Indicators Based on Pre-established Goals: An Implementation in a Colombian University. Soc Indic Res 143, 1–24 (2019). https://doi.org/10.1007/s11205-018-1979-z
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11205-018-1979-z