Eppur si muove: an evaluation of museum policy reform in Italy

Abstract

This study evaluates the effectiveness of a novel cultural policy to recover the valorization function of Italian state museums by increasing managerial autonomy. The policy is part of the Franceschini reform, introduced in 2014, that concerns the governance of cultural institutions and was justified by the severe budget constraints resulting from economic crisis and a spending review. Given the staggered implementation of the reform across 32 museums that obtained special autonomy, we apply a difference-in-differences strategy that allows to properly investigate the causal effect of the policy reform on museums’ outcome measures of exhibition function, as visitors and revenue from admissions. Our findings robustly support the effectiveness of the cultural policy reform in terms these outcomes. Although we provide some suggestive evidence of the efficacy of museums’ management (through a number of communication and supplementary services), this study suffers of the limitations due to the lack of more appropriate measures on how successful changes in management have been. However, results aim to offer policy recommendations in favor of greater managerial and financial autonomy of state museums that, while preserving their conservation function, can promote the widespread utilization of collections ensuring museums’ greater sustainability.

1 Introduction

The goal of this paper is to provide an empirical investigation of the effectiveness of a novel cultural policy measure to enhance the valorization function of state museums’ collections in Italy. In 2014, Italian cultural institutions (in particular museums) were affected by a reform in their governance aimed at reinstituting their education and research mission leading to a greater uptake of cultural heritage by the public. Specifically, one of the main features of the reform was special autonomy, which was awarded to 32 selected museums; these museums constitute the sample for our causal effect investigation.

Museums are permanent, accessible, non-profit institutions that research, acquire, conserve, communicate, and exhibit the cultural heritage of a country with the aim of promoting learning, participation, and the well-being of the community.¹ In view of this, one objective of museums is to contribute to economic development through a growth in cultural, human, and social capital. To achieve these goals, the traditional mission of museums over the last few decades, which mainly focused on conservation of their collections (including preservation, interpretation, research, and display), has been re-thought in line with the enhancement of valorization, utilization, communication, and exhibition to meet the needs of visitors and improve the social and economic impact of cultural goods on the community (Anderson, 2004; Weil, 1995, 1999). Indeed, a museum has to provide both local residents and foreign visitors with education and entertainment content (Mairesse and Desvallées 2010), contributing to the creation of a better educated and informed population (Johnson, 2003), and reassessing national identity in a changing political climate (Dodd et al., 2012).²

To address the trade-off between conservation and valorization (or utilization) of collections, new models (and strategies) of governance of state museums and, to an even greater extent, the design of ad hoc policies are required. This need is amplified by the general lack of public resources and, in recent years, by the economic crisis that saw the imposition of budget restrictions on public museums in many countries. Since the 1980s, the New Public Management (NPM) philosophy (Hood,1995) has promoted the diffusion of an economic-managerial approach in public sector organizations. Accordingly, some studies point out that a transfer of managerial and financial responsibilities to autonomous public bodies may be better than either private or centralized public management for the provision of public services (Besley & Ghatak, 2003). NPM has inspired administrative reforms that redesign the strategic orientation of public museums from a focus on conservation to a more contemporary focus on accessibility.

Since 2014, the governance of Italian state museums has been affected by a substantial reform designed from the perspective of NPM: the Franceschini reform³ that brought Italy more in line with international best practices. The reform was part of a cultural project aimed at reinstituting the education and research mission (lost in the past decades—see Argan, 1957) of cultural heritage in the light of widespread culture as a pillar of economic growth. One of the most important features of the reform was the identification of 32 state museums and archaeological parks eligible for special autonomy, specifically, organizational, scientific, accounting, and financial autonomy. This policy advocates the management of museums being carried out by directors (no longer the Ministry) responsible for ensuring their performance and the economy, efficiency, and effectiveness of activities as well as for enhancing the promotion of artistic heritage. This paper aims to evaluate the effectiveness of this policy reform in pursuing their new function of the valorization/utilization of their collections. We test whether special autonomy is an incentive for museum management to realize the valorization function, measured by the number of visitors and the amount of revenue from admissions. Indeed, at a time of tight public budget constraints, effective valorization of museums’ cultural heritage can increase the financial resources directors have available to re-invest in valorization.

Due to restricted financial resources, at the beginning of 2016, autonomy was awarded to 22 museums, and, at the end of 2017, to the remaining 10. Therefore, in the 2 years between early 2016 and the end of 2017, only 22 special museums were affected by the reform. We exploit the staggered pattern in the achievement of special autonomy as an as good as exogenous shock in the outcomes of interest. Indeed, exogeneity is assumed because no criteria involving visitors and/or revenue was adopted in the selection of the first and second tranche of special museums. This exogenous shock thus allows us to implement a difference-in-differences (diff-in-diff) approach, comparing the outcomes of museums (visitors and revenue) between the treated (the first museums being autonomous) and the control (the last 10) groups of museums before and after the reform was applied. However, when visitors and revenue are considered as outcomes, an endogeneity problem may arise due to possible omitted variables catching characteristics of museums affecting the outcomes. The staggered pattern in the achievement of special autonomy by the 32 special museums allows us to deal with this endogeneity issue by studying, in a natural experimental setting, the causal effect of the policy reform: the reform, being an exogenous shock, makes the treatment and the control group unrelated to such unobserved museums’ characteristics affecting the dependent variable of our analysis.

Our findings suggest that the policy reform increased the share of visitors by more than 3.7 percentage points and the share of revenue by more than 4.3 percentage points in the treatment group of museums compared to the counterfactual. We also give graphical support to our findings by plotting the Kernel density distribution of both the outputs in the treatment and control group of museums before and after the reform: the shift of the distribution to the right after the reform is more pronounced for the treatment group than the control group. The results remain robust to the external factors affecting the development of the outcome variables, as the tourist flows, and to the location of museums in regional capitals. Moreover, we verify that the increase in revenue is only due to the growth in the number of visitors because there is no significant change in the new managements’ pricing strategy.

We provide evidence of the parallel trend in museums’ outcomes for the treatment and control groups in the pre-reform period and of the dynamic of the outcome variables with an event study estimation. Moreover, our results are robust to a number of sample size variations.

This paper contributes to the literature on the effectiveness of the NPM view of museum management, whose empirical literature is still scarce. It falls into the more general debate addressing whether ownership structures and organizational models of cultural institutions affect differently their efficiency and effectiveness in the trade-off between preservation and utilization. Indeed, looking at the distinction between public and private museums, the former, with public grants covering their budget, have low incentives to search for alternative income sources, being oriented mainly toward the promotion of the social role of museums and improvements in people’s culture. Moreover, when a centralized bureaucratic organization manages cultural institutions, it fails in coordinating the diverse museum functions and in promoting visitor-oriented activities (Benhamou, 1998; Frey & Pommerehne, 1989). Private museums on the other hand have “market-oriented” government with greater attention to additional financial sources such as sponsorship and donations (Rushton, 2000); moreover, to ensure the organization’s financial sustainability, they also have a greater incentive to engage in managerial practices and strategies, particularly the optimal price strategy, to maximize revenue (Frey & Meier, 2006). However, there is a lack of empirical evidence showing the effect of ownership on museums’ efficiency and performance. Camarero et al. (2011) find public funding has a significant and negative impact on museums’ capacity to invest in technological innovation, whereas evidence of the effect of public ownership on museums’ economic and social performance is mixed. Studies on Italian museums find that, according to some measures of effectiveness, autonomous or outsourced public museums show the best performance, while private museums achieve a better performance than those publicly managed in the traditional way (Beretta et al., 2019; Bertacchini et al., 2018; Leva et al., 2019). In a very recent work, Guccio et al. (2020) show that the operational environment (per capita income levels and larger hospitality sectors) also matters in positively affecting museums’ efficiency; moreover, they show that private ownership is associated with a higher level of efficiency and that state museums present a lower level of efficiency than local museums. Our main findings highlight the effectiveness of a policy oriented to providing state museums with greater managerial autonomy. We show that the design of such policies can be interpreted as an attempt by policymakers to preserve the “welfare-oriented” conservation function of state museums and, at the same time, to enhance the fruition of collections by providing greater museum sustainability due to its more “market-oriented” managerial approach.

The recent research of del Barrio-Tellado and Herrero-Prieto (2019), evaluating the performance of publicly owned Spanish museums, reveals that good levels of efficiency in terms of management do not guarantee success when attracting visitors, and there seems to be a trade-off between the two goals. In this respect, our paper, using a number of communication and supplementary services provided by museums (as the number of guided tours, of coffees and meals at restaurants, of audio guides, and of gadgets) offers evidence that seems to mitigate the del Barrio-Tellado and Herrero-Prieto (2019)’s trade-off.⁴ We find that the reform increased the number of such services by about 5.3 percentage points more in the treatment group than in the control group. An increase of about 4.8 p.p occurs when we consider the revenue from those services.

Although communication and supplementary services are under the direct management of directors (rather than number of visitors and amount of revenue from admissions, whose trend may be affected by other specific conditions—for example, the trend in demand—external from management control), their analysis allows us to give only a “suggestive” evidence of the effectiveness of managerial autonomy. Indeed, supplementary services are not direct measures of museum management autonomy, as temporary exhibitions or fundraising (whose data are not actually available) could be. Therefore, our study evaluates a museum cultural policy in terms of two variables (visitors and revenue from admissions) related to the exhibition function of museums. Not much can be said about how successful changes in management have been. In this sense, a criticism may arise referring to the unchanged pricing strategy that is an aspect of the museums’ own capacity to act.

Our study, although subject to the limitations pointed out just above, provides robust policy indications for defining future cultural management strategy in the context of public budget crises.

The paper continues as follows. Section 2 outlines the institutional framework and data. Section 3 develops the empirical methodology. Section 4 provides the results of the empirical analysis, where Subsection 4.1 summarizes the main findings and Subsection 4.2 discusses the validity of the identification assumptions and the dynamic impact of the reform. Subsection 4.3 offers some robustness checks in terms of a number of sample size variation (in Subsection 4.3.1) and of the provision of communication and supplementary services (in Subsection 4.3.2). In Sect. 5, we conclude.

2 Institutional framework, data, and variables

Italian cultural heritage is huge. Italy can count nearly 5000 sites, including museums, monuments, and archaeological areas; in terms of visits, they range from small to world-level superstars. In Italy, private and governmental museums (with different degrees of autonomy) coexist. Private cultural institutions are about 1/3 while 2/3 are publicly owned; within that, $10\%$ are owned and managed by the state and they account for around $50\%$ of visitors. On the whole, Italian artistic heritage has traditionally been considered an inestimable and undervalued source of wealth for the country, and its quality and quantity should be valued to really promote growth.

Until the mid-1990s, Italian state museums were managed directly by national or local governments, with no autonomous budget, spending power, or revenue. They were managed in the light of traditional conservation and the availability of their collections to the public. In recent years, best international practice has recognized satisfaction of visitor needs and, through the communication and dissemination of culture, improving their impact on economic and social development as central roles for museums. Since the 1990s, in the context of reduced public expenditure and budgetary constraints, a new ideological view of museum management has begun to develop (Bertacchini & Dalle Nogare, 2015). The Italian Ministry for Cultural Heritage, Activities and Tourism (MiBACT), directed by Dario Franceschini, devised a reform aimed at re-organizing the management of cultural heritage in terms of its structure, governance, accounting, and financial rules. The Franceschini reform finalized a process, beginning in 2013 in the Commissione Bray, for greater autonomy for museums with specific characteristics (Casini, 2014, 2016, 2018, 2017). The reform was introduced by Decree n. 171/2014 with the aim of enhancing the culture (i.e., of reinstituting the education and research mission of the cultural good) and re-organizing the MiBACT in light of the needs of the general public spending review by providing museums with technical-scientific and financial autonomy to achieve the redefined objectives of their mission.

One of the most important features of the reform was the special autonomy given to 32 museums. Indeed, based on the evaluation of museums’ specific characteristics (in terms of visitors, employees, revenue, dimensions, collections, and geographical distribution), the Ministry compiled a list of 32 museums (30 museums and 2 archaeological parks) eligible for special autonomy, specifically, organizational, scientific, accounting, and financial autonomy. Decree 171/2014 defined two different kinds of institutions with special autonomy: the soprintendenze speciali⁵ and state museums with special autonomy. For the 32 “special” museums, the Decree prescribed the appointment of directors according to international competition. The directors, together with a board of councilors and a scientific committee, must manage both the conservation and enhancement of museum collections, and define the price of tickets and opening hours.⁶

Not all the museums on the list could achieve autonomy at the same time due to budgetary restraints. Indeed, the Ministry selected 22 museums from the full list of 32 that began autonomy at the beginning of 2016, since the procedures for designating directors were not completed until the end of 2015. The prescription of the reform was completed by Decree n. 44/2016, which began designation procedures for directors of the remaining 10 museums; the appointment of directors was completed at the end of 2017. All 32 museums had achieved autonomy by 2018. This staggered introduction of the reform up to the end of 2017 allows us to identify the first 22 museums as the treatment group (exposed to the reform) and the remaining 10 museums as the control group (unaffected by the reform). Our empirical investigation comprises those 32 museums affected by the special autonomy policy that Table 1 shows according to their group and province. The Table clearly reveals their homogeneity within Italian territory. They are mostly concentrated in the province of Rome, with 10 museums, some in the treatment group and some in the control group. Next, the province of Napoli, with 5 museums, and then Firenze with 3 museums.

Table 1 List of museums in the treatment and control group

We measure the new managements’ performance in enhancing the utilization function by the number of visitors and revenue from admissions. The main source of data is the MiBACT website. It covers all Italian museums and galleries, archaeological sites, monuments, and other similar institutions, and collects information on visitors, revenue, and the type of services and activities provided by them. A comprehensive set of that information is available from 2010; thus, the time span of our analysis is 2010–2017 (in 2018, all the 32 museums are autonomous).⁷ Visitors as well as revenue are particularly important for the analyzed scenario. Indeed, the protection and conservation of cultural heritage implies high costs, and the possibility of covering at least a part of them with ticket revenue guarantees the possibility of carrying out these activities regularly, particularly in the face of limited public resources. In the empirical analysis, we use the following as dependent variables: the share of visitors, calculated as a ratio of the number of visitors each year and the sum of visitors over 2010–2017, for each museum; thereafter Visitors;⁸ the share of revenue, calculated as a ratio of the amount of revenue each year and the sum of revenue over 2011–2017,⁹ for each museum; thereafter Revenue.¹⁰ We have taken the share of such measures (their annual value over the entire period) to normalize their size because in our sample there are both very large and small museums in terms of visitors and revenue.

At the beginning of the period (2010), the mean of the share of visitors (over museums) was 12% while at the end of the period (2017) it became 15.19%. The growth rate of the share of revenue between the beginning and the end of the time span under consideration is still more evident and amounts to 100%. When we look separately at the treated and untreated museums, the data show that the share of visitors in the treatment group moves from the 11.4% to the 16.4% (always between the beginning and the end of the period) while the control group experienced a decrease in its visitor share by about 1 percentage point. The pattern of revenue is positive and more pronounced for both the treated (13 percentage points increase) and control museums (7 percentage points increase).

We also collect information on the population between 6 and 15 years old, the population with a bachelor’s degree, and the per-capita value added, the number of overnight stays, all on a provincial basis. These are important variables affecting our outcomes of interest. The population between 6- and 15-year-old proxies that share of visitors coming on “school trips”; it is constructed as a ratio of the amount of the population 6–15 years old each year and the sum of the population 6–15 years old over 2010–2017, for province i at time t (Pop 6–15). The graduate population proxies that share of the population with a sufficient cultural background to enjoy visiting museums; it is constructed as a ratio of the amount of the population with a bachelor’s degree each year and the sum of the population with a bachelor’s degree over 2010–2017, for province i at time t (Pop High School). The per-capita value added proxies the wealth of each province; it is constructed as a ratio of the per-capita value added each year and the sum of per-capita value added over 2010–2017, for province i at time t (Per-Capita VA). We collect the number of overnight stays (as the number of nights spent by tourists) to construct an index proxying the tourist flows, as the ratio between the number of overnight stays and the population, both in the provinces where each museum is located (TF).¹¹ Finally, for each museum in the dataset, we collect data on complementary and supplementary services as the number of/revenue from guided tours, coffees and meals at the restaurants, audio guides and gadgets.¹²

Table 2 presents descriptive statistics on the average of both the outcome variables under analysis in the treatment and control groups. The average is calculated using data on the variables in the 6 years before the reform, 2010–2015 (Before), and in the 2 years immediately after it, 2016–2017 (After). Looking at the Before period, the statistics show that all the outcome variables are on average higher in the control group than in the treatment group.

The reverse, however, happens in the After period, where the outcomes in the treatment group are significantly higher than that in the control group. The temporal change (After-Before) is positive for both groups of museums and it seems to be larger (in absolute value) for the treatment group. Almost all differences are statistically significant at conventional levels. Looking separately at each measure, the share of visitors in the treated museums grows by 4.1 percentage points after the reform while the share of revenue grows by 9.9 percentage points.¹³ These statistics seem a first corroboration of the effectiveness of the special autonomy policy implemented by the Franceschini reform we are testing.

Table 2 T-test on the average of variables

Figure 1 illustrates the evolution of the outcome variables over the time span under analysis for museums belonging to the treatment and control groups. Panels 1a and 1b show, respectively, the trend for Visitors and Revenue. In both panels, before the reform, the line referring to the variables in the control group (red line) is above that in the treatment group (blue line); both lines seem to follow a parallel trend. At the time of the reform (shown by the vertical line at 2015 in all the graphs), the trend of variables in the two groups begin to change; the two lines cross just after the vertical lines. Henceforth, the treated museums have overtaken the untreated ones.

Fig. 1

Trend of the outcome variables in treatment and control group of museums. Notes. Panels a and b compare, respectively, the trend of Visitors and Revenue variables in treatment (blue line) and control (red line) group of museums. Years from 2010 to 2017; for Revenue years from 2011 to 2017 (color figure online)

3 Empirical strategy

We study the impact of the policy giving special autonomy to museums embedded in the Franceschini reform on the mission of museums to enhance the valorization of their collections. The staggered implementation of the reform allows us to perform a diff-in-diff approach where we compare the outcomes of interest (number of visitors and revenue from admissions) between the treated (the first 22 achieving special autonomy) and untreated (the last 10) groups of museums before and after the reform occurred. Since being or not being included in the first group of museums to achieve special autonomy (that is, being exposed to the treatment) was as good as random (that is, no criteria for prioritizing the 32 museums in the timing of achieving special autonomy are declared in the Decree), we consider the treatment and control groups’ status as unrelated to other unobserved museum characteristics affecting the dependent variable. Table 12 in Appendix presents a list of museums according to the two main outcomes of interest and their membership of the treatment or control group. That is, it presents the mean of the number of visitors, in increasing order, as well as of the amount of revenue from admissions over the period 2010–2015 (2011–2015 for revenue) and 2016–2017, i.e., before and after the reform, respectively. Table 12 supports the randomness of the assignment of museums to the two groups: despite the increasing order of the number of visitors (that does not correspond to the increasing order of the revenue from admissions), it seems that no priority in the achievement of special autonomy (that is, in assignment to the treatment and control group) linked to the number of visitors (or amount of revenue) has been given to museums.

Moreover, Table 12 highlights that the Gallerie degli Uffizi, the Parco Archeologico di Pompei and the Sopr. Speciale Colosseo e l’Area Archeologica Centrale di Roma are outliers in terms of both visitors and revenue from admissions.¹⁴ In the time span under consideration, the Gallerie degli Uffizi, the Parco Archeologico di Pompei and the Sopr. Speciale Colosseo e l’Area Archeologica Centrale di Roma attracted more than 60% of the total number of visitors of all the other 29 autonomous museums. For these reasons, in the baseline analysis, we exclude them from the treated museums.

The panel structure of our database combined with the exogenous treatment allows us to isolate the effect of the policy reform on variables of interest from any time-specific effect—their trends—which might drive the results. We run museum-level regressions and compare the change in the dependent variables across the two groups of museums in the 6 years just before the reform and in the 2 years just after the introduction of the reform. We also focus on the shorter term change in the outcomes before and after the reform (2014–2017), since it is likely that there were no sharp changes in the institutional environment other than the reform that could have differently affected our variables within a short period of time.

Our key identification assumption is based on the parallel trend assumption: without the policy intervention, the differences in the museums’ outcomes would have remained constant over time between the treatment and control groups. An initial visual support to the assumption of a common trend in the outcome variables among the treated and untreated museums in the years preceding the reform is given by Fig. 1. We provide formal tests of the common trend assumption in Sect. 4.2.

We first estimate the mean impact of the reform of 2015 on the Visitors and Revenue using the following specification:

$$\begin{aligned} Y_{it}=\alpha _{i} + \beta _{1} \mathrm{{Treat}}_{i}+ \beta _{2} (\mathrm{{Treat}}_{i}*\mathrm{{Reform}}_{t})+\lambda _{t} +\gamma ' X_{it}+\epsilon _{it}. \end{aligned}$$

(1)

${Y_{it}}$ is the outcome of interest of museum i at time t. $\alpha _i$ represents museum fixed effects that control for unobserved heterogeneity across museums. $Treat_{i}$ is a dummy taking the value of 1 for museums affected by the reform and 0 otherwise; it allows us to control for any unobserved time-invariant characteristics that may differ across museums in the two groups. $Treat_{i}$*$Reform_{t}$ measures the treatment effect of our interest, i.e., the difference in the dependent variables between the treatment and control groups before and after the reform. It is the interaction between $Treat_{i}$ and $Reform_{t}$, a dummy which takes the value 1 in the years after the reform (2016 and 2017) and 0 otherwise (2010–2015). $\lambda _t$ is a vector of year dummies and controls for the year-by-year temporal trends. X is the vector of the control variables that are the population between 6 and 15 years, the population with a bachelor’s degree, and the per-capita value added, all on a provincial basis. Finally, $\epsilon _{it}$ is an error term. Our parameter of interest is $\beta _2$, which measures the treatment effect, that is, the difference in the temporal variations of outcomes between the museums affected and unaffected by the reform. If we expect special autonomy to increase the valorization of the museums’ collections, we should observe a $\beta _2$ positive and significantly different from zero.

The order of our results in Sect. 4 are: in Subsection 4.1, we present estimation results of the baseline model with several variations (effects of tourist flows, regional capital and pricing strategy); in Subsection 4.2, we discuss the common trend assumption hypothesis; in Subsection 4.3, several robustness checks are provided related to the sample size and the provision of complementary services.

4 Results

4.1 Baseline analysis

As first evidence of the effect of the special autonomy policy reform, Table 3 reports the results of the estimation of Eq. 1 where the dependent variables are the share of all museums’ visitors and the share of revenue from admissions. When the number of clusters (in our case, the number of museums, 29) is limited (generally, lower than 50), the usual cluster-robust standard error estimates would be downward biased (Cameron & Miller, 2015). Thus, to overcome this bias, we implement the wild cluster bootstrap methodology proposed by Cameron et al. (2008) to test the null hypothesis that each parameter of the model equals zero.¹⁵ In each of the following tables presenting our estimation results, the bootstrapped p values of the null hypothesis are shown in round brackets.

Table 3 Estimation results

A first look at Table 3 shows that the coefficient of the interaction term Treat*Reform is statistically significant and positive everywhere, suggesting that the reform has increased the number of museum visitors as well as the revenue from admissions. In the first four columns, we consider the full time span. In columns 1 and 3, we estimate the baseline specifications including only the museums and year Fixed Effects (FE). In terms of the size of estimated effect, the result in column 1 suggests that after the introduction of the reform, the average share of visitors in the treatment group of museums increased by 3.82 percentage points more than in the control group.

The magnitude of the treatment coefficient becomes slightly smaller when we improve the specification by introducing the control variables we describe above (column 2); given that an improved specification should allow removing biases, this pattern is consistent with the presence of an upward bias in our initial estimate. Moreover, the stability of the main coefficients after including controls enables us to defend the causal interpretation of the results by excluding a possible omitted variable bias. Column 3 presents estimation results when the dependent variable is Revenue. The reform increased the share of revenue by 4.85 percentage points more in the treatment group than in the counterfactual and the decreasing size of the coefficient of Treat*Reform as we improve the specification (column 4), confirms an upward bias in the initial estimate.

To control for any possible sharp changes in the institutional environment other than the reform affecting the outcome variables, we restrict the pre-treatment period to the two years just before the reform, to have the same number of years in the pre- and post-reform. Therefore, the time span of the empirical analysis is 2014–2017. The results are presented in columns 5–8 in Table 3 for Visitors and Revenue. The coefficient of interest continues to be positive and highly significant, and its magnitude is greater than in the full period. More precisely, the increase in the share of visitors in the treatment group caused by the policy reform comes by 6.2–5.89 percentage points, while we count a more than 7 percentage points increase when we look at the share of revenue.¹⁶

Fig. 2

Kernel density plots for the treatment and control group. Time-span 2014–2017. Notes. The blue and green lines refer to the distribution, respectively in the treatment and control group of museums, of the share of visitors (panel a) and of the share of revenue (panel b) before the reform, 2014 and 2015. The red and yellow lines refer to the distribution of the same variable after the reform, 2016 and 2017 (color figure online)

To further complement the regression analysis proposed in Table 3 (columns 5–8), Fig. 2 shows the plot of the Kernel densities of the share of visitors (in panel 2a) and the share of revenue (in panel 2b) in the treatment and control group in the pre-reform period (Before) and in the post-reform period (After). Both graphs clearly show that, although the distribution shifts to the right for both the groups after the reform, the shifting is more pronounced for the treatment group than for the control group, corroborating our results about the effectiveness of the reform about the two variables of interest.¹⁷

One can argue that external factors affecting the development of the variable of interest (mainly the visitors), such as tourist flows, could also drive demand in the more iconic museums or those that are located in areas of greatest tourist appeal. It would seem that such areas coincide in many in the treatment group. Thus, we control for tourist flows and we detect whether such flows affect the causal effect of museum policy reform on the variables of interest. We construct an index proxying the tourist flows through the ratio between the number of overnight stays (as the number of nights spent by tourists) and the population, both in the provinces where each museum is located (TF).¹⁸ We interact this variable with the treatment dummy (Treat*Reform*TF). Table 4 shows estimation results.

Table 4 Estimation results controlling for the rate of tourism

The treatment variable maintains its positive sign and the significance becomes stronger than in Table 3 as well as its magnitude: the effect of the reform in enhancing the number of visitors and revenue from admissions is about 8 percentage points for Visitors (column 5) and about 8.5 percentage points for Revenue (column 7). The interaction term Treat*Reform*TF is negative but never significantly different from 0, meaning that tourist flows do not affect the impact of the reform on the outcome variables.

Related to the previous point, the analysis of del Barrio-Tellado and Herrero-Prieto (2019), evaluating the efficiency of the Spanish state-run network of museums, shows that National Museums, usually located in tourist cities, are more efficient and are focused on attracting visitors (as final output), whereas provincial museums (managed by regional governments) perform better in managerial functions or intermediate outputs (new exhibitions, publications, museums’ activities). In the context of our analysis of Italian special museums, this concern can be appropriate if we consider museums located in a regional capital (“capuluogo di regione”) with respect to a museum located in a provincial capital (“capuluogo di provincia”). Dealing with the possibility that the location of museums could drive our main results, we control for a dummy variable taking the value of 1 if the museum is located in a regional capital and 0 otherwise (RC) and we interact this dummy with the treatment one (Treat*Reform*RC).¹⁹

Table 5 Estimation results controlling for the location of museums in regional capitals

Table 5 shows estimation results. Interestingly, the interaction term Treat*Reform*RC is negative and highly significant, meaning that the effect of the reform in enhancing visitors and revenue from admissions is stronger in museums located in provincial capitals than in museums located in regional capitals. In more detail, looking at the first four columns of Table 5, the location in a regional capital undermines the effect of the reform by about 4.6 percentage points for Visitors and by about 7 percentage points for Revenue. Restricting the sample to 2014–2017 (columns 5–8) confirms the results.

The two outcome variables we use measure the incentive to enhance the valorization function of the new management in terms of the quantity of visitors as well as the ticket pricing strategy. Indeed, the increase in the revenue for admissions that we show as a result of the policy reform could have been driven by both the increase in visitors (already tested and verified) and a variation in ticket prices. Data on the price of tickets are not available, but we can obtain them by considering the revenue from admissions and the number of visitors²⁰ to disentangle the effect of the reform on the two components of revenue. Thus, we estimate Eq. 1 where the dependent variable is the average price of tickets (thereafter Price) each year for each museum. The results shown in Table 6 seem to suggest that the reform did not affect directors’ pricing strategy; thus, the increase in revenue from admissions has been entirely due to the increase in visitors.

Table 6 Estimation results

Our findings must be interpreted with caution. Indeed, as a possible first bias, we have to take into account that many of the museums in our study belong to a “Circuito museale” (museum system) that provides a single ticket for entry to all the museums within the “Circuito”. In this case, we are unable to assign to each museum its “share” of the cumulative ticket. Moreover, as a second bias, the total revenue from admissions also consists of revenue from exhibitions of temporary collections, for which museums issue a dedicated ticket. Thus, the average price we have calculated may not reflect the actual price.

4.2 Discussion on the identification assumptions

The goal of our study is to estimate the causal impact of the policy reform for museums on some dimensions of the valorization function of state museums. The central question here is about the validity of the basic common trend assumption on the basis of the diff-in-diff empirical strategy. Accordingly, in the pre-reform period, the treatment and control group of museums have to be significantly similar in the trend of the outcome variables.

To carry out a formal test of the parallel trend assumption, we follow the very recent approach in the literature on diff-in-diff in our investigation of the trends in outcomes for both treatment and control groups in the periods before the reform. We specify a general “fully flexible” model as in Mora and Reggio (2019) that includes the interactions of the groups (treatment/control) dummies with each of the pre- and post-reform time dummies.²¹ This model allows us also to study the full dynamic effects of the reform, i.e., how the outcomes of interest adjust over time. This is as follows:

$$\begin{aligned} Y_{it}=\alpha _{i} +\beta _{1}\mathrm{{Treat}}_{i}+\sum _{t=2010}^{2018} \nu _{t}\cdot {\mathrm{{Treat}}_{i}}\times {B_{t}}+\lambda _{t}+\epsilon _{it}, \end{aligned}$$

(2)

where t from 2010 to 2018 is an index for years prior to the reform ($t=2010--2015$, $t=2011--2015$ for the share of revenue) and after the reform ($t=2016, 2017$). ${B_{t}}$ is the complete set of dummy variables for each year from 2010 to 2017. Since we are controlling for time-invariant characteristics of the treatment group, not all of the diff-in-diff coefficients are identified. For this reason, we normalize $\nu _{2015}=0$ (2015 is the year of the reform, that is, the last pre-treatment period) so that the remaining coefficients measure the effect relative to the period of the reform ($t=2016,2017$). Therefore, the dynamic coefficients $\nu _{t}$ allow us to measure the effect of the reform every year before and after it is implemented. If the common trend assumption is satisfied, we should observe that all the $\nu _{t}$ before the reform ($t=2010--2014$) are not significantly different from zero.

Table 7 Dynamic impact estimates of the reform

Table 7 presents the estimation results of the dynamic effect of the reform on the main outcomes of interest according to the model as in Eq. 2. It shows all the coefficients of the interaction between the groups (treatment/control) dummies and the times dummies. The effect of the reform appears soon in 2016 (Treat*$B_{2016}$ is positive and significant), and it reinforces in magnitude and significance in 2017 (Treat*$B_{2017}$). We perform a Wald test for the null hypothesis of the absence of dynamics in treatment effects after the reform under the model as in Eq. 2. The bootstrapped p values of the test performed on specifications, respectively, in columns 1, 2, 3, and 4 in Table 7 are equal to 0.18, 0.20, 0.11, and 0.18. Thus, we accept the null of equal treatment impact at conventional level. This finding allows us to support the validity of the results of the model as in Eq. 1 shown in Table 3.

The dynamic model (Eq. 2) allows the implementation of the test on parallel paths proposed by Mora and Reggio (2019). First, note that all the coefficients Treat*$B_{2010}$ -Treat*$B_{2014}$ in Table 7 are not significantly different from zero; also the bootstrapped p value of the F-test that all their coefficients are jointly equal to zero (the last row of Table 7) confirms this finding, supporting the parallel trend assumption between the two groups of museums before the reform.

We complement the dynamic analysis shown in Table 7 with Fig. 3 that plots the 95% confidence intervals of $\nu _t$ in Eq. 2, respectively, for the variables Visitors (graph 3a) and Revenue (graph 3b). We recall that the reference year is 2015, normalized to zero. Figure 3 shows that there is no evidence of differential pre-trends across the two outcome variables, supporting the validity of the basic identification assumption of our empirical model. The two graphs also highlight the positive impact of the reform, showing an increase in both visitors and revenue for admissions after the 2015.

Fig. 3

Event study estimates. a Visitors. b Revenue. Note. The graphs report coefficients and confidence intervals estimated according to specification as in Eq. 2. Estimations contain museum and year FE. Robust standard errors clustered at the museum level

In the case of multiple pre-reform periods, Mora and Reggio (2019) show that the effects of the reform can be estimated under a family of alternative Parallel-q assumptions where q is the number of pre-treatment periods (in our case $q=6$). The authors show that when all Parallel-q hold, the common trend assumption is fulfilled; that is, the common trend assumption implies Parallel-1-Parallel-6 simultaneously.²² the results of the test (that refer to the specifications in Table 7) show that the p values of the Wald test on the equality between Parallel-1-Parallel-6 are, respectively, for Visitors and Revenue, equal to 0.90 and 0.84. They support a common pre-treatment dynamics in all the outcome variables.²³

4.3 Robustness checks

In this section, we undertake some robustness checks. We focus on various sample size variations and on the provision of communication and supplementary services.

4.3.1 Sample size variation analysis

In the baseline analysis, we excluded from the sample of museums the Gallerie degli Uffizi, the Parco Archeologico di Pompei, and the Sopr. Speciale Colosseo e l’Area Archeologica Centrale di Roma because outliers with respect to the rest of the museums. As a first robustness check, we perform estimations over the full sample of 32 museums interested by the reform. The results are in columns 1 and 2 of Table 8, respectively, for the visitors and revenue, and replicates those in Table 3.

Our diff-in-diff identification strategy relies on a comparison of museums that are as similar as possible. In our sample of museums, three more museums with respect to the Gallerie degli Uffizi, the Parco Archeologico di Pompei, and the Sopr. Speciale Colosseo e l’Area Archeologica Centrale di Roma appear quite different according to the evidence shown in Table 12 in the Appendix. They are the Museo di Capodimonte, Galleria dell’Accademia di Firenze and the Museo Miramare, all belonging to the treatment group of museums. Thus, to take into account these further dissimilarities, we replicate the diff-in-diff approach of Eq. 1 by excluding them from the sample. The results are presented in columns 3 and 4 of Table 8. The sign and significance of the treatment variable is still strongly preserved, confirming the effectiveness of the reform in enhancing the valorization of the museums’ collections. The magnitude of the effect is a bit stronger than in the baseline analysis.

Table 8 Estimations on the sample size variations

In our sample, 50% of museums (belonging to the treatment and control group) are part of a territorial museum system (“Circuito museale”). The territorial museum system allows museums within a homogeneous geographical area to make an agreement to enhance the different cultural characteristics of the area and to achieve a more efficient management of services. It is the tool through which local authorities implement cooperation, qualification, development of services, and promote the protection and enhancement of the cultural and environmental heritage of their territory. Membership of a territorial museum system may provide museums with some spill-over effects, generating a possible distortion in the results. Taking this into consideration, we restrict the sample to museums that do not belong to a territorial museum system.²⁴ The estimation results are presented in columns 5 and 6 of Table 8 and confirm the main results of the effectiveness of the reform.

4.3.2 Communication and supplementary services’ analysis

In recent years, more and more museums have been offering services aimed at widening museum accessibility (e.g., evening openings, upon-request openings, and temporary exhibitions), supporting activities to improve the fruition of collections (e.g., the availability of brochures, the presence of audio guides and guided tours, the provision of childcare activities, and so on) and supplementary services (e.g., coffee bar, restaurants, and gadgets). The supply of all these services has been driven by the increasingly stringent budget constraints experienced by public museums in recent years. Indeed, as underlined by Cellini et al. (2020), “museums supply complementary services as potential alternative sources of revenue, sometimes necessary to support the traditional ‘core-business’ of museums, namely conservation, research, and exhibitions”.

While our analysis robustly proves that the new management reached objectives of enhancing the valorization of the cultural heritage housed by museums through the sharp increase in visitors, the effectiveness of managerial autonomy might be properly tested by considering the mentioned services that are completely under management discretion (indeed, the main outcome variables we considered might be affected by other external factors outside management autonomy, for example, the trend in demand).

Unfortunately, comprehensive data on all these services are not available. The MiBACT website provides yearly data (from 2010 to 2017) only on some communication and supplementary services. That is: the number of guided tours, coffees, (meals at the) restaurants, audio guides, and gadgets.²⁵

Thus, to deeply evaluate the effect of the managerial change due to the Franceschini reform, we run the baseline specification of Eq. 1 where the dependent variables are, one-by-one, the sum of all the listed services, until obtaining, with the last summation, a single synthetic indicator. We start considering one of the most common supplementary services: guided tours, for which the MiBACT website provides the richer set of data. As for the main outcome variables, we consider the share of guided tours (calculated as the number of guided tours each year over the sum of guided tours in 2010–2017, by museum—thereafter Guided Tours). Then, we add the share of coffees (thereafter Coffees), the share of (meals at the) restaurants (thereafter Restaurants), the share of audio guides (thereafter Audio Guides) and, finally, the share of gadgets (thereafter Gadgets).²⁶

Table 9 Estimations: number of supplementary services

The estimation results are in Table 9. An overall look at the treatment coefficient Treat*Reform highlights a positive and highly significant sign, confirming the effectiveness of the policy also on supplementary services. About the Guided Tours in column 1, the increase is of 6.86 percentage points in treated museums with respect to the counterfactual group. The increase reaches 7.6 percentage points when considering the sum of Guided Tours, Coffees and Restaurants (column 3); it ends at a 5.38 percentage points increase when all the services are added (column 5).

Following the presentation of the main analysis, we show in Table 10, the results where the dependent variables are the revenue from the mentioned communication and supplementary services. As for Table 9, we start by the revenue from guided tours and we add, one-by-one, the revenue of all the other services until we reach a comprehensive indicator, in column 5. The increase in the revenue of guided tours, on average over the full period, is 8.2 percentage points more in the treatment group than in the control group after the reform took place; the size becomes weaker and close to 5 percentage points when we add revenue from coffees, restaurants, audio guides, and gadgets (column 5).

Table 10 Estimations: revenue from supplementary services

Figure 4 in Appendix further complements the regression analysis in Tables 9 and 10 by plotting the Kernel densities of the share of all services in the treatment and control group in the pre- and post-reform period restricting the time span to 2014–2017. Once again, the distribution of both the treatment and control group of museums shifts to the right after the reform but the distance between the two distributions Before–After the reform is wider for the treatment group than the counterfactual.

Although the previous analysis supports the validity of the Franceschini reform, this part of our study is subject to limitations due to the availability of data on other more relevant variables about the functions under the direct responsibility of directors such as, for example, the number of temporary exhibitions or fundraising (public or private), which might be more representative of management autonomy. In more detail, the ISTAT survey on museum²⁷ provides a variable counting the number of temporary exhibitions organized by museums but it is available only for 1 year before the reform (2015) and 1 year after the reform (2017). We perform estimation over this very small sample (less than 60 observations) and we find no significant results. Moreover, data on fundraising are available only after 2016 (thus, only one observation per museum) because it is governed by the so called “Art Bonus” that has been in force since 2015. Thus, being the supplementary services an unsuitable measure of how successful changes in management have been, the previous results offers only a suggestive evidence of the effectiveness of the museum cultural policy.

5 Conclusions

Over the last few years, the traditional mission of museums has been reconsidered in light of visitors’ needs as well as the social and economic impact of cultural heritage on the whole community. This has resulted in an increasing focus on the dissemination of culture and the appeal of the museums to tourists. This explains why the performance of cultural institutions has fired debate on the management of such institutions, carried out in the context of stringent spending reviews. As a result, an important policy reform concerning Italian state museums was introduced, providing some of them with a greater degree of managerial, technical-scientific, and financial autonomy. The reform aimed at allocating to the museum’s director autonomy in both the management of conservation and promotion of its collections. To provide empirical evidence of the effectiveness of this important feature of the Franceschini reform in encouraging the valorization of museums’ collections, we estimate the causal effect of the special autonomy on the number of visitors and the revenue from admissions. We apply the novel—in this field of analysis—diff-in-diff technique and an event-study dynamic specification, that allows to overcome endogeneity concerns about such indicators, by exploiting the staggered implementation of the reform over the 32 special museums and the random assignment of museums to treatment and control groups.

Our findings show an increase in the visitors as well as in revenue from admission in treated museums compared to the control ones. They remain robust controlling for tourist flows, the location of museums in regional capitals and to a number of sample size variations. The dynamic specification of the empirical model provides evidence of parallel trend in the museums’ outcomes for treated and control group in the pre-reform period, that offers valid support to the basic identification assumption to the diff-in-diff approach. Moreover, the event-study graphs show the effectiveness of the reform in increasing visitors and revenue from admission just after its implementation.

While some concerns can be raised about the main outcome variables (visitors and revenue from admission) because they can be affected by other factors (such as the increase in demand) outside direct managerial autonomy, we provide only suggestive evidence of the effectiveness of the policy reform by considering a number of communication and supplementary services. Although results confirm the efficacy of the change in management, they have to be considered not final given the lack of data on more appropriate measures of the new management activities. Moreover, in this respect, some criticism may be pointed out given that we find no alteration in the pricing strategy.

Therefore, our study has verified the reform’s scope of success in terms of two variables (visitors and revenue from admissions) related to the exhibition function of museums, but it is not conclusive in the scope that autonomy has had in the museum management. Further research in this sense is needed.

However, the reform process has progressed (and still continues) quite slowly and only a small subgroup of governmental museums has been affected by the greater degree of autonomy we have analyzed here.

While there is still a long way to go, this study highlights the effectiveness of the special autonomy in promoting a more widespread utilization of cultural heritage. However, the trade-off between preservation and valorization of collections remains central in public policies concerning cultural goods. It seems that special autonomy can be a suitable policy for reducing the effects of this trade-off. Indeed, the preservation and conservation of cultural heritage requires huge financial resources; the greater financial autonomy as well as the different incentives and priorities the new directors face has resulted in greater dissemination and fruition of cultural and artistic heritage that in turn has resulted in increased revenue that contributes toward museum self-sustainability. From a policy-making perspective, the public value generated by the growth in cultural, human, and social capital achieved by this new managerial perspective can provide a method for framing and targeting public investment at a time of rising demand and limited public spending budgets.

Appendix

See Fig.  4 and Tables 11,  12,  13,  14.

Table 11 Descriptive statistics

Table 12 Museums in treatment and control group by the number of visitors and the amount of revenues, 2010–2015/2016–2017

Table 13 Estimation results including Treat and Reform

Table 14 Estimation results. Dep. Var.: ln(revenue)

Fig. 4

Kernel density plots for the treatment and control group. Time span 2014–2017. Notes. The blue and green lines refer to the distribution, respectively, in the treatment and control group of museums, of the share of all the supplementary services (panel a) and of the share of revenue by those services (panel b) before the reform, 2014 and 2015. The red and yellow lines refer to the distribution of the same variable after the reform, 2016 and 2017 (color figure online)