Performance Measurement in the Health sector: Uses of Frontier Efficiency Methodologies and Multi-Criteria Decision Making
- 913 Downloads
Performance measurement is essential for improvement. Although techniques for collecting, analyzing, and reporting data fall within the quantitative skill set of the organization, translating data into information that managers need to promote performance improvement requires a different and more subtle skill set. Health services sector is a complex area that is unique in all its characteristics. It has too many dimensions to be fitted into a simple singular unit and it is therefore essentially very difficult to approach the measurement of the performance of healthcare services by using one method or another . Frontier Efficiency Methodologies and Multi-criteria Decision Making have been used rigorously in recent years. Data Envelopment analysis [1, 2, 3] is proven to be a useful tool in measuring efficiency and productivity of hospitals and health care related units [4, 5]. Multi-criteria decision making approach has also been adopted extensively for performance analysis in healthcare sector [6, 8, 9, 10].
The 9 papers comprising this special volume of the Journal of Medical Systems contribute to the theory and application of Data Envelopment Analysis (DEA). Papers were included in the special volume after a rigorous refereeing process, and represent only a small fraction of the total number of submitted manuscripts.
The volume opens with an application focused on the performance of dialysis facilities. Nick Kontodimopoulos, Nikolaos D. Papathanasiou, Angeliki Flokou, Yannis Tountas and Dimitris Niakas examine the impact of non-discretionary factors on DEA and SFA technical efficiency differences. They investigated the effect of factors such as operating environment on the efficiency. In a sample of 124 dialysis facilities, technical efficiency was compared according to ownership, region, years in operation and size. With second-stage Tobit regression, DEA and SFA efficiency was regressed against these environmental factors to determine their potential for predicting technical efficiency, as well as the efficiency differences between the two frontier methods. DEA expectedly generated lower mean efficiency scores than SFA, due to the “random effects” term computed by the latter, in addition to “true” inefficiency. This finding was consistent for the subgroups formed on the basis of the environmental factors. Thy found that half the variation in the DEA-SFA efficiency differences was explained by environmental factors. This suggests that in addition to market instabilities, luck, and other related phenomena, decision-makers in their effort to determine optimal resource allocation, should point their attention to the potentially useful insight provided by environmental factors.
In another application in measuring technical efficiency in primary health care in the Spanish region of Extremadura, José Manuel Cordero Ferrera, Eva Crespo Cebada and Luis R. Murillo Zamorano investigated the effect of exogenous variables on efficiency. In this context, the exogenous variables are represented by the main characteristics of the covered population. They argued that with using multi-stage process in DEA allows calculating more accurate efficiency scores that can reflect the performance of units more properly. Their results show that the inclusion of the exogenous variables in the evaluation has a great impact on both the values of efficiency scores and the rank of units.
In the same area of searching for factors that affect on efficiency, Jos L.T. Blank and Bart L. Van Hulst investigated governance characteristics that could explain the performance of hospitals using a case of Dutch hospitals. In particular, they focused on accounting for cost inefficiency measures on the part of hospital corporate governance. They used bootstrap techniques in order to obtain more efficient estimates of the effects of governance on the efficiency. They results showed that part of the cost efficiency can be explained with governance. In particular they find that a higher remuneration of the board as well as a higher remuneration of the supervisory board does not implicate better performance.
Angeliki Flokou, Nick Kontodimopoulos and Dimitris Niakas used a two stage analysis which in the second stage they used cross-evaluation and cluster analysis in a sample of Greek NHS hospitals. While DEA is used in the first stage, in a second stage (post-DEA), aggressive and benevolent cross-efficiency formulations and clustering were employed, to validate (or not) the initial DEA scores. They used “maverick index” to sort the peer-appraised hospitals. Initially they indentified ten hospitals as benchmark (100% efficient), but using the aggressive and benevolent formulations showed that two and four of them respectively were at the lower end of the maverick index list. On the other hand, only one 100% efficient (self-appraised) hospital was at the higher end of the list, using either formulation. Cluster analysis produced a hierarchical “tree” structure which dichotomized the hospitals in accordance to the cross-evaluation results, and provided insight on the two-dimensional path to improving efficiency.
The fifth application is also a multiple stage approach for performance improvement of primary healthcare practice. Martha T. Ramírez-Valdivia, Sergio Maturana and Sonia Salvo-Garrido demonstrated their method using a case of Chilean primary healthcare practice. They argued that the efficiency score by itself is of limited value for decision making; hence they used a multivariate tool to explain the effect of relevant factors. First, they used a cluster analysis to homogenize the units under study. Second, they employed DEA to estimate the efficiency levels. Third, they used bootstrapping to estimate confidence intervals for the efficiency scores, and a Biplot method to identify adequate variables to include in the Tobit Model, which is their last stage. They identified six factors associated with rural municipalities’ operational efficiency, and two factors with urban ones.
Using two stage analysis Herbert F. Lewis, Thomas R. Sexton and Melissa A Dolan, employed an efficiency-based multi-criteria decision model to guide an ambulatory surgery centres (ASC) for identifying their optimal procedure mix. In the first stage they used DEA to calculate the efficiency of each procedure based on the resources required to perform the procedure, the revenue it generates, and its risk of complications. In the second stage they used the DEA factor efficiency scores in a bottleneck program to optimize the mix of procedures while satisfying the ASC’s resource and operational constraints.
The seventh study is a novel application of DEA in the appraisal and relative performance evaluation of nurses. Ibrahim H. Osman, Lynn N. Berbarya, Yusuf Sidania, Baydaa Al-Ayoubib and Ali Emrouznejad used data from an intensive care unit for both nurses and employers in an era of clinical governance, increased accountability and high standards of health care services. In this study they proposed a DEA model for the appraisal and relative performance evaluation of nurses. They validated the model by applying it on 32 nurses working at an intensive care unit at one of the most recognized hospitals in Lebanon.
In another study, Paul Rouse, Julie Harrison and Nikki Turner used activity based costing (ABC) and DEA to investigate the cost and performance of 24 New Zealand primary care practices. They indicated that the DEA has a high level focus typically using aggregated data in the form of inputs and outputs, while ABC is more detailed and oriented around very disaggregated data. Hence they used a case study of immunisation activities to illustrate how DEA and ABC can be used in conjunction to improve performance analysis and benchmarking. They results show that practice size, socio-economic environment, parts of the service delivery process as well as regular administrative tasks are major cost and performance drivers for general practices in immunisation activities.
Finally, Chun-Ling Chuang, Peng-Chan Chang, Rong-Ho Lin investigated the evaluation of hospital operational efficiency for better resource allocation and cost effectiveness. First they used several DEA-based models, and then they identified a DEA-artificial neural network (ANN) model and claimed that this model is more capable than the DEA and DEA-assurance region (AR) models of measuring operational efficiency and recognizing the best-performing hospital. At the end they also used the classification and regression tree (CART) efficiency model to utilize and extract rules for improving resource allocation of medical institutions.
Overall, the papers included in the special volume give us a small but representative picture of the current research in the area of non-parametric efficiency and productivity evaluation using DEA for performance measurement in the health sector. All of the papers contribute either to the theory or applications in the field and should be of interest to both academics and practitioners.
To conclude, we are grateful to all the authors and to the reviewers who made this special volume possible. Although it was not possible to include all submitted manuscripts, the editors hope that all authors found the feedback helpful for their future work. We thank Dr. R. R. Graham, Editor-in-Chief of the Journal of Medical Systems, for giving us the opportunity to prepare this special issue.
- 2.Emrouznejad, A., and De Witte, K., COOPER-framework: A unified process for non- parametric projects. European Journal of Operational Research, (2010) 207(3):1573-1586.Google Scholar
- 8.Hariharan, S., Dey, P. K., Kumar, A. Y., and Moseley, H. S. L., Analytic hierarchy process as a tool for the performance measurement on intensive care services in hospital. J Crit Care Med 20(2):117–124, 2005.Google Scholar