Abstract
In this work we discuss how Emergency Departments (EDs) can be ranked on the basis of multiple indicators. This problem is of absolute relevance due to the increasing importance of EDs in regional healthcare systems and it is also complex as the number of indicators that have been proposed in the literature to measure ED performance is very high. Current literature faces this problem using synthetic (or numerically aggregated) indicators of a set of performance measures but, although simple, this solution has a number of drawbacks that make this choice inefficient: a compensation effect among the indicators; a high degree of subjectivism in the indicators weighting; opacity in the decision making; all the EDs are considered to be comparable. Indeed, the situations in which EDs are comparable (i.e. when all the performance of one ED are not lower than the performance indicators of the other) are a minority and incomparability is by itself a source of information that should be used to identify situations for which different policy actions should be designed. In this work we propose to use non compensatory composite indicators and partial ordering theory to rank and compare EDs giving value to the reasons of such an incomparability. These methods are applied on a case study of 19 EDs in an administrative region in Italy.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
When a synthetic indicator is, for instance, a weighted sum of the elementary indicators, compensation means that a good value of such an indicator may be the results of a very good value for some indicators which masks potentially critical values for other indicators.
The concrete danger of death that characterizes red triage accesses may compromise the data quality on their waiting spans as in same situations medical staff gives priority to the patient’s assistance rather than to a timely update of the patients’ tracking system.
In Italy regional tariffs are suggested by the Ministry of Health, but each regional district has the opportunity to arbitrarily revise them.
In the following \(_{E} ed_{i}\) will indicate the profile of the ith ED for the Cost-efficiency set of indicators and \(_{E} ed_{i}\) the profile of the i-th ED for the Quality set.
It is possible to find a sort of analogy between the L-axis and the penalisation coefficient of MPI in formula (2): both the two techniques try to include in the analysis a measure of the degree of concordance among the indicators.
References
Alessandrini, E. A., & Knapp, J. (2011). Measuring quality in pediatric emergency care. Clinical Pediatric Emergency Medicine, 12(2), 102–112.
Bhat, K., & Patil, G. (2007). Posac, data based weights, and mutual probability methods for multicriterion prioritization: A study in the theory and application of ranking methods, Center for Statistical Ecology and Environmental Statistics, Technical Reports and Reprint Series, p. 0703
Brüggemann, R., & Carlsen, L. (2014a). Incomparable—what now? Match, 71(3), 699–716.
Brüggemann, R., & Carlsen, L. (2014b). Incomparable: what now II? Absorption of incomparabilities by a cluster method. Quality and Quantity, 49(4), 1633–1645.
Brüggemann, R., & Carlsen, L. (2015). Incomparable - what Now, III. Incomparabilities, elucidated by a simple version of ELECTRE III and a fuzzy partial order approach. Match, 73(2), 277–302.
Brüggemann, R., & Patil, G. P. (2011). Ranking and prioritization for multi-indicator systems—introduction to partial order applications. Berlin: Springer.
Cremonesi, P., di Bella, E., & Montefiori, M. (2010). Cost analysis of emergency department. Journal of Preventive Medicine and Hygiene, 51(4), 157–163.
Cremonesi, P., di Bella, E., Montefiori, M., & Persico, L. (2015). The robustness and effectiveness of the triage system at times of overcrowding and the extra costs due to inappropriate use of emergency departments. Applied Health Economics and Health Policy, 13(5), 507–514.
De Muro, P., Mazziotta, M., & Pareto, A. (2011). Composite Indices of development and poverty: An application to MDGs. Social Indicators Research, 104(1), 1–18.
di Bella, E., Corsi, M & Leporatti L. (2016a). POSET analysis of panel data with POSAC. In M. Fattore and R. Brüggeman (eds), Partial order concepts in applied sciences, Springer, in press.
di Bella, E., Corsi, M., & Leporatti, L. (2015). A multi-indicator approach for smart security policy making. Social Indicators Research, 122(3), 653–675.
di Bella, E., Corsi, M., Leporatti, L., & Cavalletti, B. (2016). Wellbeing and sustainable development: A multi-indicator approach. Agriculture and Agricultural Science Procedia, 8, 784–791.
Duda, R. O., Hart, P. E., & Stork, D. G. (2001). Pattern classification (2nd ed.). New York: Wiley.
Graff, L., Stevens, C., Spaite, D., & Foody, J. (2002). Measuring and improving quality in emergency medicine. Academic Emergency Medicine, 9(11), 1091–1107.
Guttmann, A., Razzaq, A., Lindsay, P., Zagorski, B., & Anderson, G. M. (2006). Development of measures of the quality of emergency department care for children using a structured panel process. Pediatrics, 118(1), 114–123.
Huang, I. B., Keisler, J., & Linkov, I. (2011). Multi-criteria decision analysis in environmental sciences: ten years of applications and trends. Science of the Total Environment, 409(19), 3578–3594.
Hung, G. R., & Chalut, D. (2008). A consensus-established set of important indicators of pediatric emergency department performance. Pediatric Emergency Care, 24(1), 9–15.
Levaggi, R., & Montefiori, M. (2013). Definition of a prospective payment system to reimburse emergency departments. BMC Health Services Research, 13, 409.
Lindsay, P., Schull, M., Bronskill, S., & Anderson, G. (2002). The development of indicators to measure the quality of clinical care in emergency departments following a modified-delphi approach. Academic Emergency Medicine, 9(11), 1131–1139.
Martinetti, E. C., & von Jacobi, N. (2012). Light and shade of multidimensional indexes: How methodological choices impact on empirical results. In F. Maggino & G. Nuvolati (Eds.), Quality of life in Italy: Research and Reflections, Social Indicators Research Series 48 (pp. 69–103). Dordrecht: Springer.
Mazziotta, M., & Pareto, A. (2011). Un indice sintetico non compensativo per la misura della dotazione infrastrutturale: un’applicazione in ambito sanitario. Rivista di Statistica Ufficiale, 1, 63–79.
Mazziotta, M., & Pareto, A. (2015). Comparing two non-compensatory composite indices to measure changes over time: a case study. Statistika, 95(2), 44–53.
Ministero Della Salute (2007) Progetto Mattoni SSN. Milestone 1.3—Definizione del sistema di valutazione dei pazienti (triage PS e 118). Available online: http://www.mattoni.salute.gov.it/ Accessed 10 July 2016.
Munda, G., & Nardo, M. (2005). Non-compensatory composite indicators for ranking countries: A defensible setting (p. 21833). EUR: EUR Report.
OECD. (2008). Handbook on constructing composite indicators. Methodology and user guide. Paris: OECD Publications.
Patil, G., & Taillie, C. (2004). Multiple indicators, partially ordered sets, and linear extensions: Multicriterion ranking and prioritization. Environmental and Ecological Statistics, 11, 199–228.
Raveh, A., & Landau, S. (1993). Partial order scalogram analysis with base coordinates (POSAC): Its application to crime patterns in all the states in the United States. Journal of Quantitative Criminology, 9(1), 83–99.
Schull, M. J., Guttmann, A., Leaver, C. A., Vermeulen, M., Hatcher, C. M., Rowe, B. H., et al. (2011). Prioritizing performance measurement for emergency department care: consensus on evidence based quality of care indicators. Canadian Journal of Emergency Medicine, 13(05), 300–309.
Shye, S. (1985). Multiple scaling. The theory and application of partial order scalogram analysis. Amsterdam: North-Holland.
Shye, S., & Amar, R. (1985). Partial order scalogram analysis by base coordinates and lattice mapping of items by their scalogram roles. In D. Canter (Ed.), Facet theory: Approaches to social research (pp. 277–298). New York: Springer.
Solberg, L. I., Asplin, B. R., Weinick, R. M., & Magid, D. J. (2003). Emergency department crowding: consensus development of potential measures. Annals of Emergency Medicine, 42(6), 824–834.
Sørup, C. M., Jacobsen, P., & Forberg, J. L. (2013). Evaluation of emergency department performance-a systematic review on recommended performance and quality-in-care measures. Scandinavian Journal of Trauma Resuscitation & Emergency Medicine, 21(1), 62–76.
Tregunno, D., Ross Baker, G., Barnsley, J., & Murray, M. (2004). Competing values of emergency department performance: Balancing multiple stakeholder perspectives. Health Services Research, 39(4 Pt 1), 771–792.
UNDP. (2010). Human Development Report 2010. New York: Palgrave MacMillan.
Vidoli, F., & Mazziotta, C. (2013). Robust weighted composite indicators by means of frontier methods with an application to European infrastructure endowment. Statistica Applicata, Italian Journal of Applied Statistics, 23(2), 259–282.
Voigt, K., Welzl, G., & Brüggemann, R. (2004). Data analysis of environmental air pollutant monitoring systems in Europe. Environmetrics, 15(6), 577–596.
Welch, S. J., Asplin, B. R., Stone-Griffith, S., Davidson, S. J., Augustine, J., Schuur, J., et al. (2011). Emergency department operational metrics, measures and definitions: results of the second performance measures and benchmarking summit. Annals of Emergency Medicine, 58(1), 33–40.
Acknowledgements
The authors would like to thank Regione Liguria for his valuable co-operation in providing the data.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
di Bella, E., Gandullia, L., Leporatti, L. et al. Ranking and Prioritization of Emergency Departments Based on Multi-indicator Systems. Soc Indic Res 136, 1089–1107 (2018). https://doi.org/10.1007/s11205-016-1537-5
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11205-016-1537-5