The joint adoption of DEA and BSC, two methods coming from different fields, may enhance the process of evaluating the performance of organizations. As mentioned in Sect. 2, the synergies between the two methodologies for the performance evaluation of museums have been exploited in Basso et al. (2018) with the proposal of a two-stage DEA–BSC model for the evaluation of the performance of a set of museums.
The idea exploited in Basso et al. (2018) is first (in stage 1) to measure the museums performance according to each BSC perspective; a proper DEA model is therefore designed for each BSC perspective, by choosing the input and output variables relevant for that perspective.
As for the connection among the different BSC perspectives, in the museum sector and more in general in the whole cultural sector, we do not find a hierarchy between the BSC perspectives which is clearly defined and generally accepted. This is due to the very nature of the nonprofit organizations that are common in this sector and to the difficulty to measure the fulfilment of the social purpose of their activity, which is directed toward stakeholders with different objectives; for a comprehensive discussion on this issue, see Basso et al. (2018). As a consequence, we have opted for a simultaneous consideration of the different BSC perspectives and therefore the four DEA–BSC models are all set at the same level in the first stage.
At the end of the first stage, four efficiency indicators are therefore obtained, one for each BSC perspective; these indicators capture the different dimensions of the museum’s performance. Subsequently, the second stage of the evaluation process uses the DEA approach once again to aggregate the performance scores of the four BSC perspectives into an overall performance indicator. A graphical representation of the basic flow chart of the two-stage DEA–BSC model is given in Fig. 1.
First stage
The four DEA models associated with the four BSC perspectives in the first stage have a similar structure: they are all variable returns-to-scale DEA models with output orientation and virtual weights restrictions (see Basso et al. 2018).
The input and output variables were carefully selected by designing a special BSC scheme tailored to meet museums’ peculiarities. Indeed, the variables of the DEA models focus on the main characteristics of the museum activities which affect each BSC perspective; moreover, the variables have to be computable for all the museums investigated. Table 1 displays the input and output variables of the four DEA–BSC models used at the first stage.
Table 1 Input and output variables of the four DEA–BSC models of the first stage With reference to the DEA model associated with the Customer perspective, the outputs are somewhat related to the stakeholders of the museums; among the output variables we find the number of: museum visitors, Web site visitors, catalogues sold in the museum bookshop, museum members, and the donations received.
The outputs considered in the DEA model for the internal process perspective are instead related to the main purposes of the museum in terms of preservation of cultural heritage (conservation and restoration costs), expansion of museum’s collections (amount spent for new acquisitions), increase in value for the museum’s customers (measured by the number of visitors).
The personnel training (number of hours per employee) and one or more sustainability indicators make up the outputs of the DEA model for the Innovation and learning perspective. As for the sustainability indicators, we list a few indicators: the presence of innovative lighting, an indicator of environmental sustainability that measures the adoption of appropriate environmental saving measures, and the number of different facilities for people with disabilities; in the empirical application, we consider a comprehensive indicator computed as a linear combination of such indicators; for the details, see Basso et al. (2018).
Finally, in the financial perspective, the outputs are the different entries that form museum’s income (income from tickets, sponsorships, donations, public funding, and other incomes).
As for the inputs of the first-stage DEA–BSC models, each model has a single input: the insured value, as a proxy for the value of the museum exhibits in the customer perspective model; the total operating costs in the internal process perspective model, including the personnel costs that represent the highest share of costs but have not been considered separately in accordance with the suggestions of the BSC approach (other DEA contributions for museums, not using a BSC approach, make a different choice and explicitly consider the personnel costs or number; see, for example, Mairesse and Vanden 2002; Pignataro 2002; Del Barrio-Tellado and Herrero-Prieto 2019); the expenditure (the algebraic sum of operating costs, financial costs and taxes) in the financial perspective model.
A separate consideration has to be made for the innovation perspective model, for which indeed a specific input is not required, since the outputs are a ratio (the number of hours per employee) and a quality indicator (the sustainability indicator). On the other hand, it is possible to show (Lovell and Pastor 1999) that an output oriented DEA model without inputs is equivalent to an output oriented DEA model with a single constant input. In a way, a constant input can be seen as a “dummy input” having the same value for all decision-making units (Cherchye et al. 2007).
As regards the structure of the four DEA models associated with the four BSC perspectives, they all have the same structure: each of them is a variable returns-to-scale DEA model, with output orientation and restrictions on the output weights (see Basso, Casarin, Funari, 2018). The choice of variable returns-to-scale gives more flexibility, since it does not impose that the returns-to-scale are constant, but at the same time, it does not prevent them from being constant.
Actually, we may observe that the innovation perspective model could equivalently be written as a constant returns-to-scale model, since Lovell and Pastor (1999) proved that an output oriented variable returns-to-scale model with a single constant input is equivalent to the corresponding constant returns-to-scale model.
As for the weights restrictions imposed on the DEA models, they serve to ensure that we do not completely overlook some important variables in the computation of museum’s efficiency score.
The virtual weights restrictions considered are applied only to the weights of the outputs and set lower and upper limits on the proportion of each virtual output (defined as the product of the level of output \(r'\) and the related optimal weight) with respect to the total virtual output:
$$\begin{aligned} L_{r'} \le \frac{u_{r'} y_{r'j}}{\sum _{r=1}^{t} u_r y_{rj}} \le U_{r'} \quad r'=1,\ldots ,t; \quad j=1,\ldots ,n \end{aligned}$$
(5)
where \(L_{r'}\) and \(U_{r'}\) represent the lower and upper percentage values set for the virtual output \(r'\) (on virtual weights restrictions see (Wong and Beasley 1990).
For example, in the application of the two-stage DEA–BSC model carried out in Basso et al. (2018) on the municipal museums of Venice (the “MUVE museums”), the lower and upper bounds referred to the first output variable “Visitors” in the DEA model designed for the Customer perspective are \(L_{1}^C=0.4\) and \(U_{1}^C=0.9\), respectively. These bounds entail that the visitors variable will affect the performance score of the customer perspective model in a percentage that may range from \(40\,\%\) to \(90\,\%\): Since exhibiting its collections to the public is an important museum’s mission, we need to give this variable a relevant role in the computation of the performance score. We refer to Table 6 in Basso et al. (2018), p. 79, for the values chosen for all the virtual weights restrictions in the analysis of the MUVE museums.
The “MUVE museums” are a set of 11 municipal museums of Venice managed by the MUVE Foundation, including several renowned museums such as the Doge’s Palace and the Correr Museum. We have carried out some empirical investigations applying the DEA–BSC–AHP models on data of the MUVE museums referred to the year 2013. The data have been kindly provided by the MUVE Foundation and are reported in Tables 2, 3, 4 and 5, which show the input and output variables of all DEA–BSC models.
Table 2 Values of the input and output variables of the Customer Perspective model for the MUVE museums (percentage values) Table 3 Values of the input and output variables of the internal process perspective model for the MUVE museums (percentage values) Table 4 Values of the input and output variables of the innovation perspective model for the MUVE museums (percentage values) Table 5 Values of the input and output variables of the financial perspective model for the MUVE museums (percentage values) In the analysis carried out with the DEA–BSC–AHP three-system model that will be presented later, in Sects. 5 and 7, the four DEA–BSC models of the first stage coincide with those presented in Basso et al. (2018); of course, since both the data used and the weights restrictions adopted are the same, the results of the first stage coincide with those obtained in Basso et al. (2018) and are reported in Table 6, which summarizes the values of the performance scores obtained for each BSC perspective. It can be seen that the performance of MUVE museums differ according to the BSC perspective considered; moreover, none of the museums is efficient with respect to all perspectives.
Table 6 Performance scores obtained for each BSC perspective at the first stage of the DEA–BSC model in the assessment of the MUVE museums (reworking from Basso et al. (2018)) Second stage
The computation of the performance scores related to the different BSC perspectives allows to identify the areas where a museum is stronger or weaker, in comparison with the other museums under evaluation.
However, it is useful to summarize the performances achieved by the museums across the different areas in a synthetic indicator. In a way, the idea is similar to that of the graduation grade, which synthesizes the grades obtained in all exams in a unique score.
A DEA approach, looking for the most favorable weights for each organization, can be exploited also at the second stage, and a synthetic index can be computed starting from the four performance scores obtained at the end of the first stage.
The DEA model developed by Basso et al. (2018) in the second stage has four output variables that coincide with the performance scores associated with the four BSC perspectives, and a constant input (see Table 7). It is once again a variable returns-to-scale DEA model with output orientation and virtual weights restrictions, with a structure similar to the DEA models of the first stage. Of course, this model, too, could equivalently be written as a constant returns-to-scale model given that it has a single constant input.
Table 7 Input and output variables of the DEA–BSC model at the second stage In the empirical application to the MUVE museums, the following lower and upper bounds were chosen to restrict the virtual output weights of the DEA model in the second stage:
Customer performance score
|
\(L_1^2 = 0.2\)
|
\(U_1^2 = 0.8\)
|
Internal process performance score
|
\(L_2^2 = 0.05\)
|
\(U_2^2 = 0.5\)
|
Innovation and learning performance score
|
\(L_3^2 = 0.05\)
|
\(U_3^2 = 0.5\)
|
Financial performance score
|
\(L_4^2 = 0.05\)
|
\(U_4^2 = 0.5\)
|
These restrictions emphasize the importance of the Customer perspective in the museum context but ensure at the same time that no BSC perspective is overlooked in the computation of the overall performance indicator. Let us point out that, though in the DEA–BSC model at the second stage the four BSC perspectives are set all at the same level, without imposing a hierarchical order, the use of virtual weights restrictions allows one to somewhat grade their importance.
The values of the overall performance indicator computed for the MUVE museums in the second stage of the evaluation process are reported in Table 8, together with the relative position in the ranking.
Table 8 Overall indicator and ranking obtained at the second stage with the DEA–BSC model for the MUVE museums (reworking from Basso et al. (2018))