Introduction

The Townscape Heritage Initiatives (THI) programme was launched in 1998 to address problems of ‘disrepair, erosion of quality and under use of structures in areas where historic buildings predominate’ (Heritage Lottery Fund (HLF), 2002). The initiative was intended to assist places that had heritage assets but also demonstrable social and economic needs. THI schemes were set up to operate with partners, local governments as well as national and international agencies, who all contributed to common funds, with the HLF portion limited to a maximum of 75 per cent but more generally being between 20 and 50 per cent. What properties and work were eligible as well as the level of grant aid was defined at the outset of each scheme. The first phase of the THI programme supported 118 separate schemes throughout the United Kingdom, totalling some £92 million spread over six annual bidding rounds. Subsequent expansions of the programme have now seen aid to about 175 different locations (some towns have more than one THI scheme in place) with a budget in excess of £170 million.

In 1999, the HLF engaged the Department of Planning in the School of the Built Environment at Oxford Brookes University (OBU) to undertake a 10-year project to evaluate the success of the THI programme as described above. Written by the two research directors for the project, the paper reports this research and its outcomes to date.

The HLF followed the rationale set out by Tiesdell et al (1996) that urban regeneration is a three-way process that needs to involve not just the physical upgrading of historic buildings, but also economic activity and social use of urban centres. It was never a primary objective of the HLF that investment in the built heritage would have regeneration benefits beyond townscape improvements; however, they anticipated that such benefits might, and indeed ought to, flow from such investment. The research was designed to find out whether this was in fact the case, and the extent of such benefits. That is, in the way that Maclaren (1996) suggested, they wanted to gauge whether and to what extent their interventions were moving communities in the right direction. The HLF commissioned the research not in order to claim credit for change but in order to help determine how, as a major player in heritage investment, it can be more effective in encouraging the best kind of change.

In 2004, a peer-reviewed article appeared in the academic literature, which outlined the methodology that had been adopted to evaluate the effectiveness of HLF contributions to the THI (Shipley et al, 2004). The current article builds on the previous work by addressing the very pertinent question: did the evaluation methodology developed in 1999 and described in 2004 work? In order to deal with this central question we consider three separate points in order. First, was it possible to gather the data that the researchers proposed in the planned evaluation? Second, was it possible to use those data to evaluate the degree to which the stated HLF goals for the programme were met? With regard to these first two questions the work addresses important questions raised by Rhodes et al (2005) concerning both the theory and efficacy of this type of programme evaluation. Finally, in what way will the evaluation be of value to the HLF in determining the characteristics either of the places that are likely to benefit from THI-type schemes in the future, or the characteristics of the schemes that were successful from which best practices might be derived? It should be emphasised that this article is about the validity and utility of the evaluation methodology not about the performance of the individual THI schemes. Nonetheless, the final part of the paper discusses the performance of individual cases insofar as they are helpful in constructing a predictive model of heritage investment potential. It should be noted that the majority of schemes involved a considerable amount of local commitment by various agencies and individuals, and even in an academic context it would be invidious and insensitive to expose the apparent failure of such commitment.

Brief Overview of the THI Evaluation Approach

Although it is not possible here to outline the entire evaluation methodology, a brief synopsis will be useful at this juncture.

Selecting the sample of sites

Because there are a large number of places in the THI programme it was not possible to study each in detail. A sample of 18 representative sites was chosen using the following criteria: size of the site to ensure that examples of villages, towns and cities were all evaluated; amount of the grant and total budget; geographical diversity and local characteristics, that is within metropolitan areas, isolated, resource based, industrial, rural agricultural (Figure 1). Table 1 gives a more detailed breakdown of the characteristics of the sample group.

Table 1 Characteristics of the sample group
Full size table
Figure 1
figure 1

THI evaluation project sites.

Full size image

One of the original sites (Tower Hamlets in London) was dropped from the programme and two of the sites, though physically separate, are administered jointly. The current study, therefore, encompasses 17 distinct townscapes, three in Wales, three in Northern Ireland, three in Scotland and eight in England. The total contribution of HLF grants to these sites is in the neighbourhood of £18 million, which represents anywhere from 20 to 64 per cent of the total THI funding for the schemes. Expenditure on individual schemes examined in this study ranged from £7.6 million in Liverpool with a relatively low percentage HLF input of £1.5 million to smaller schemes in places such as Newport Pagnell where all the work was done for £550 000 with the HLF contribution of £225 000 representing 50 per cent. The average HLF contribution to all THI schemes is £971 000 while the average contribution to the study sample sites is £1 059 000. The sample is therefore roughly representative of the funding level as well as of the geographical distribution and community profile.

Methodology

The HLF posed four questions for the research from which general conclusions might be drawn and which might help in the efficiency and effectiveness of their future funding allocations. The questions were to be applied to each case study:

  • Has the THI contributed to the community's sustainability through encouraging community involvement and access – that is, has it enhanced quality of life?

  • Has the THI improved the area's appearance?

  • Has the THI facilitated investment in the area?

  • Has the THI created greater social and business confidence?

The aim of the HLF was for the programme to have a positive influence on all of these themes, in all places benefiting from its investment, through the careful expenditure of funds. The aim of the research was to capture the degree to which the programme as a whole was successful in this intention and the reasons for this. While the four themes, quality of life, appearance, investment, and image and confidence, formed the core of the investigation, they are obviously very broad and encompass physical, economic and social dimensions. Therefore, each theme was to be investigated by subdividing it into four thematic indicators of success.

The concept of an indicator is extremely helpful in evaluative research (Maclaren, 1996; Hart, 1999). In essence, an indicator provides a gauge for reviewing change over time, informed often by a range of different but related data. Munday and Roberts (2006) discuss the various problems that face researchers in deciding on appropriate ways of evaluating progress, but as with their study of sustainability in Wales the investigators must eventually decide on a defensible set of indicators they can use. The indicators developed for the THI evaluation were designed to be easily measured: employment, education, sense of community and security were used as indicators of quality of life; townscape quality, public space, private space and façades were taken as indicators of changes in appearance – specifically related to heritage quality; land use changes, retail usage, capital values and traffic flow became indicators of investment – specifically related to economic regeneration; while media coverage, attitudes of citizens, visitation and business vitality were indicators of image and confidence.Footnote 1 In 2009, Oktay Vehbi and Önal Hoskara outlined a model for evaluating regeneration in historic urban quarters. The THI evaluation approach could have been the prototype for the model they proposed.

On the basis of this thematic structure of main themes and indicators, we determined the appropriate types of data to be collected or generated that would inform the key indicators. Having determined this we, of course, had to devise methods for gathering the data. In summary, this came down to four data streams or sources: household questionnaires, interviews, a townscape survey and information from secondary sources. The decision to use these predicated on two conditions: the research budget and the likely value of the data that would be available or that could be generated from these different methods.

Household surveys consisting of over 30 questions were sent to 500 people living in and around the THI areas. Questions covered a wide range of topics from socioeconomic status such as household income and type of employment to attitudinal issues such as their satisfaction with shopping opportunities in their area and opinions about their elected representatives. Interviews were conducted with THI officials, key local authority personnel, business people and civic leaders. When it was not possible to conduct face-to-face interviews, the open-ended questions were supplied to the informants who filled them out and returned them. The townscape survey was conducted by the OBU team members and consisted of land use mapping and the use of a pro forma that measures 25 different elements of given views (Reeve et al, 2007). These elements included the cleanliness of the streets, the quality of restoration work and pedestrian safety. In addition, photographs were taken, always from the precisely the same location, so that details of changes could be visually assessed. Anywhere from 30 to 50 different views were observed in each THI site. The secondary data collected included national statistics on education achievement and absenteeism, employment levels and trends, property values and crime rates.

For each indicator, a number of these measures were synthesised in the evaluation. A summary of findings for each indicator were written up in the text of reports on each site, while measures for each were also entered into what is called a Balanced Scorecard (Kaplan and Norton, 1996). Any given item of measurement was only used once in the Balanced Scorecard and all measures were rendered into a number out of five. Responses on the questionnaire were registered on a 5-point Likert Scale; elements in the Townscape Survey were scored out of 5 and ratios derived from secondary data were calibrated on a 5-point scale (Trochim, 2005). Only interview responses were left out of the scoring and used only for qualitative evaluation. This Balanced Scorecard approach ensures that, as the name implies, the scores for economic factors, opinions, observations and statistical data were given equal weight.

Each site was visited initially by teams of researchers from Oxford Brookes between 1999 and 2001. Questionnaires were sent and secondary data collected and from this information baseline reports were prepared. After 5 years of the THI scheme implementation each of the sites was again visited and data were collected in 2005 and 2006. Detailed reports evaluating the changes were prepared in 2007. After 10 years, a third set of site visits and a further evaluation will be undertaken.

This approach was discussed extensively with the client (the HLF) before the study began and was shaped not only to meet their expectation, but also to ensure that the methodology was appropriate. Programme Evaluation of this kind has a long and distinguished pedigree and is based on solid theoretical work (Bichman, 1987; Rog and Fournier, 1997; Khakee, 1998). At the same time, the urgency and utility of evaluating planning initiatives is being increasingly recognised (Baum, 2001; Seasons, 2003; Laurian et al, 2004; Edwards and Thomas, 2005; Goodlad et al, 2005), and the body of practice is growing, especially but not exclusively in North America (Robertson, 1997; Lopes-Balsas, 2000; Rypkema and Wiehagen, 2000; Shipley et al, 2006).

Caveats regarding the approach

The study was not designed to monitor expenditure on specific properties, measure the precise return on investment or to arrive at some sort of formula to predict likely outputs in terms of jobs created or amount of floor space brought back into use for a given sum invested. Nor was it intended to compare one THI site with another. The aim was to measure change, and hopefully progress with respect to the goals of the HLF, in each location. The findings of the research should not be taken in any case as a critique of the dedication, community spirit or integrity of the individuals involved in any of the sites. The THI Programme Evaluation has been intended to take an honest, fair and impartial look at what actually happened in the sample sites following the application of a rigorous and well-defined set of methods. In this approach, the study is in line with other current efforts to arrive at suitable methods for measuring sustainable progress towards desirable and broad community goals.

Was Gathering the Proposed Data Possible?

In considering the collection of data it will be most useful to look both at the challenges and the strengths of each of the data streams in order: questionnaires, interviews, townscape survey and secondary data.

Questionnaires

One possible weakness that might be perceived in the process of collecting information from locally distributed questionnaires is the response rate and number of respondents. While a standard 500 surveys were distributed to selected postal walks in each THI site, except West Wemyss, where there were fewer than 500 inhabitants, the average response rate was about 20 per cent or about 100 returns. The number of returns at the time of the baseline study ranged as high as 140 and 150 in Glasgow and Wigtown and as low as 50 in the Liverpool Rope Walks. At the time of the 5-year follow-up study the number of returns dropped in most places by about 3–5 per cent. There were exceptions such as Middlesbrough, where the number of returns dropped by half and Draperstown, where the number actually increased slightly.

Another drawback with the questionnaires was that often a number of respondents had no opinion (that is, answered ‘don’t know’) and sometimes had a slightly distorting influence on scores.

The strengths of the questionnaires were more pronounced. Although the number of respondents was generally between 10 and 25 per cent and was often less in the 5-year review than in the baseline study, the count remained at viable levels for statistical significance. There can be considerable confidence in changes of mean scores for questions that show shifts in attitudes, for example response to the question of whether the town is a good place to work might show a small increase in the mean score but a more noticeable shift in the numbers who moved away from the most pessimistic views.

Cross referencing often, though not always, gave good evidence of the accuracy of responses, for example comparing questionnaire response on employment to national statistics on employment. In Bloxwich, Bodmin, Burslem and Newport Pagnell the questionnaire reported unemployment rate was exactly the same as the rate found in the national statistical data. In other areas, however, there was wide variation that was not easy to explain. Where the questionnaire responses deviated from the national statistics for the area it was most common that those reporting being unemployed represented a much lower rate than the officially reported figure.

In the end, the richness of detail in the questionnaire responses was very useful in making comparisons between the time of the baseline report and the time of the 5-year review. Examples will be discussed below in the section dealing with how the research data was interpreted. It can also be noted that Kendall's rank correlation was computed for each set of questionnaires to examine the statistical associations between answers to questions, and no unforeseen or disconfirming relations were found. The questionnaire has also provided a large set of data currently being reviewed on the influence of the THI as a programme of investment in the heritage on resident's perception of quality of place, and of their appreciation or perception of heritage value.

Interviews

Finding the same people who had been interviewed for the baseline studies was one of the principal challenges in conducting the 5-year review. This was partly because both THI staff and Local Authority personnel had moved on. A second problem was that responses from officials often seemed to be coloured by an impression that the OBU team might have an influence on the awarding of future grants and they therefore wanted to be positive in their portrayal of the success of the THI at the expense of honesty. Finally, interviews that took place in group settings sometimes appeared to restrict the directness of responses that might have been forthcoming in private individual interviews. This was evident when politicians and Local Authority staff were both present. In the cases where this situation occurred, the OBU interviewers arrived to find that such meetings had been organised. When making their own interview arrangements OBU team members always preferred individual sessions.

As with the questionnaires, however, the positive aspects of the interviews outweighed any problems. They often provided concrete examples of the way the schemes functioned. The OBU interviewer in Bodmin, for example, was told how THI-funded repairs on certain buildings had led directly to Council's insistence that other buildings outside the THI area be brought to the same standard. They gave colour and dimension to feelings and impressions of developments in different sites and, under the right conditions, allowed individuals to provide honest feedback. In Burslem, the part played by personality conflicts was outlined discretely, but honestly. That was aided by the fact that the problems had been overcome.

The challenge of finding the people who had been interviewed for the baseline study was sometimes overcome by simply locating them in their new positions and asking them to reflect on their previous experience. Although face-to-face interviews were not always possible local people were often very good at providing thoughtful written responses to the open-ended questions in the interview guideline that was mailed out to specific individuals. Finally, in other cases when it was not possible to interview the same people as talked to during the baseline study, new contacts were sought out. This was particularly the case with local business people, shopkeepers and estate agents.

Townscape survey

The method for observing townscape changes that has been developed over many years at OBU, and which has been discussed in the academic literature, proved to be a most reliable tool in evaluating the THI sites (Reeve et al, 2007). Researchers were able to complete the land use mapping, which is the first part of the townscape survey, without difficulty, although determining upper floor uses remains somewhat speculative and in several cases was deemed not possible.

Before the 5-year review site visits, a page was prepared for each of the 30–50 views that were to be evaluated. These pages included copies of the previous picture in order to ensure exact duplication of the scene and angle to be photographed, as well as the pages containing previous scores to allow easy comparison. Twenty-five separate aspects are scored out of 5 for each view, and therefore the cumulative impact of even the subtlest changes can be determined. In the field, an overall impression score is also given for each view before the detailed scoring is undertaken. The impression score is the researcher's first instinctive notion of the view's numerical value. The impressions scores were compared with the cumulative detailed scores at the end of every day and, if there were discrepancies, the views were either revisited or careful comparisons made using the baseline picture and the 5-year review photo.

In a few cases, notably in the Liverpool Rope Walks area, the changes in the townscape were so dramatic that it was sometimes difficult to take a photograph of the same view and from the same position where the baseline picture had been taken. In most places, however, the views had not changed so dramatically.

Secondary data

Assembling secondary data presented more challenges than did gathering information from the other sources. The problems arose in two specific areas. Statistical information gathered at the local level proved to be virtually impossible to obtain, while nationally collected data often changed in either nature or in the areas for which they were reported.

In the first case it meant that traffic counts, footfall surveys, planning application statistics and other items that might have been useful are not included in the analysis. Frustratingly, the OBU team had planning permission figures for some sites, notably Burslem and Pembroke Dock for 1999, but these proved useless when similar figures for 2005/2006 were not forthcoming. Local estate sales information was also not easy to obtain. These difficulties in collecting locally generated counts and statistics can largely be attributed to the workload of local authority officials and private business people.

The problem with the national data is that some measures that were available 5–7 years ago are no longer collected. This was the case, for example, with deprivation indexes. In other cases, such as figures by ward for employment levels, the ward boundaries changed, making precise comparisons difficult from one time to another. Finally, there continue to be differences in the way statistics are collected in England and Wales, Scotland and Northern Ireland. For example, in England and Wales there were statistics available for ‘school exam achievements – percentage of 15-year-old pupils gaining 5 or more GCSES at grades A*–C’, while no comparable information was available for the Northern Irish sites.

The use of triangulation of data as a characteristic of the OBU methodology, however, allowed for these difficulties to be overcome. There was a sufficient number of statistical measures in the original plan that, even when some were missing, it remained possible to use other data to compensate. Owing to the richness of the data it was also possible to substitute figures even within specific measures, as was the case when changed ward boundaries still allowed comparative ratios to be determined. For example, if the 1990s wards covering the THI areas yielded a specific ratio when compared to the surrounding district or county, the ratio for 2006 wards with somewhat different boundaries, but still including the THI area, could be expected to yield similar insights into how the THI site compared to the broader area. As the resulting figure is a ratio and not an absolute number or percentage, it retains its value for the purpose of analysis.

On balance, the data collection methods laid out in the original OBU plan and endorsed by the HLF at the outset of the THI evaluation proved reasonably robust, replicable and accurate. However, the next iteration of the research is likely to be more streamlined as a result of budgetary considerations, and the research team will review the relative value of the different methods for data collection that have been applied to date. The research will also be modified through a careful analysis of the findings to date, particularly as they have a bearing on the question of cost effectiveness of the programme.

Was it Possible to Use the Data to Evaluate the HLF's Goals for the THI

The straightforward answer to this question is ‘yes’. The data from the four streams of information gathered provided sufficient material for the OBU team to come to clear conclusions regarding most of the indicators. It was also possible to determine not only what changes were taking place, but also what some of the causal links were between the THI spending and observable outcomes. The following sections illustrate this point by reference to specific cases.

Quality of life indicators

In the case of determining changes in the employment and income situation, by consulting different data streams it was possible to better understand the picture. Although in Bloxwich the mean score for income as reported on the household questionnaires only changed by one tenth of a per cent, rising from 2.46 to 2.47 between 2000 and 2006, it can be seen that there were more people reporting lower incomes and a significant number moving up slightly in the middle-income range (Figure 2). Responses to other questions also show more part-time employment. As with other cases, the national employment statistics indicate an increase in employment, but that is because of more part-time work. The overall reading of the employment and income indicator, therefore, is of a situation tending to decline but not in free fall. It can be speculated with some degree of assurance that THI improvements, which are maintaining businesses, are playing a part in preventing a worse situation.

Figure 2
figure 2

Reported incomes in Bloxwich.

Full size image

Creswell Model Village and Bodmin provide examples of changes in the quality of life indicator relating to crime and security and their relation to townscape improvement. Creswell underwent one of the most dramatic physical transformations of any THI site, whereas changes in Bodmin were more modest. However, the metamorphosis in attitudes about crime was considerable in both areas. In Creswell, the sense of daytime security reported in the household surveys increased from 65 per cent of the people feeling safe up to 90 per cent between 2000 and 2006. Perception of night-time safety increased from 25 to 45 per cent. People's feeling that there would be more crime in the coming five years decreased from 80 per cent in 2000 down to 45 per cent in 2006. The actual crime rate in Creswell during this time, as recorded in the national statistics, in fact increased a little from 80 per thousand of population to 90. People nevertheless felt much safer in their newly refurbished environment. In Bodmin, the number of people reporting crimes remained virtually the same between 1999 and 2005. But on close examination of the survey data it turned out that assault, sexual assault, robbery and almost every other crime category had gone down, whereas reports of vandalism had increased dramatically. The most plausible conclusion is that while actual crime decreased, people's renewed pride in the community had increased, and they were much more aware of things such as spray-painted graffiti and broken bottles that detracted from that sense of civic pride.

Townscape improvement indicators

The townscape surveys often present a much clearer scene, partly owing to their graphic character. Land use change was dramatic in some cases such as the Liverpool Rope Walks THI (Figure 3). Here, thousands of square metres of previously vacant floor space was brought back into use. Many of the pre-existing buildings were not only vacant in 2000 but were derelict and in advanced stages of decay. Along with restoration and adaptive reuse there was also considerable new building that followed strict design guidelines.

Figure 3
figure 3

Liverpool Rope Walks.

Full size image

In other places, however, the changes were much more subtle, yet still captured through the OBU townscape evaluation procedure. In Newport Pagnell, the restoration of shop fronts contributed to a general upgrading of the town that was noted in the Townscape Survey and that can be related to an increase in the desire of residents to shop locally and an increase in their willingness to invest in maintenance of their own properties, factors that became evident from the household questionnaires. Many other THI sites show similar improvements. Some townscape elements, such as ‘legibility’, might escape the conscious notice even of people who use the street regularly, yet they contribute to overall sense of improvement. For example, on one of the streets approaching the town centre of Newport Pagnell there is a bicycle gate that was almost invisible as one approached it in 2000, but which could be clearly seen in 2006 (Figure 4). The new bicycle gate was part of the town's public realm improvement.

Figure 4
figure 4

Newport Pagnell, illustration of subtle change in the legibility of the street.

Full size image

Townscape land use mapping sometimes revealed other factors. In the case of Burslem, there had clearly been changes in use with some previously vacant properties coming back into use but there was no net gain in occupancy (Figure 5). That change without perceivable improvement was reflected in household survey results that failed to register any noticeable increase in people's confidence. At the same time it must be remembered that THI is a long-term investment. Even in places such as Burslem there are many promising signs that may yet come to fruition because of the basic work of conservation-led regeneration. New housing development around the old town centre may provide the additional customer base that will be served by shops housed in buildings that have been saved through the THI scheme.

Figure 5
figure 5

Burslem.

Full size image

Economic regeneration indicators

The way that the various data streams informed a reading of economic regeneration indicators can be seen by referring to the situation in one of the more successful sites, Glasgow Merchant City, and one of the less fortunate THI schemes, Newport's Lower Dock Street in South Wales. In the case of Glasgow, we can focus on the land use change indicator and for Newport on the retail usage and demand indicator. In the Glasgow example, the balanced scorecard rating for the land use change indicator increased by 15 per cent between 2001 and 2006 while the retail usage and demand indicator for Newport lost almost 3 per cent during the same period.

At the baseline stage it was evident from the stakeholder interviews and written responses, especially from talking to estate agents, that the Merchant City part of Glasgow was becoming gentrified. Loft apartments were beginning to appear on the upper floors of some buildings. This trend is clearly continuing with the emphasis in the phase two THI on fostering a cultural- and arts-based quarter. In 2006, estate agents indicate that residential prices were rising, although the term they used was ‘aspirational’. On the basis of information supplied and observation, vacancy rates fell from around 44 properties in 2001 to 33 in 2006. The emphasis in land use was also shifting away from commercial and office, to more residential, retail and entertainment. The residential element consists of a mix of up-market owner occupied apartments, typically at around £180 000, and housing association rented property charging social rents. In terms of land use change, the indicator points clearly towards economic regeneration.

The retail usage and demand indicator in Newport tells a different story. There simply was not a lot of change in attitudes within the THI area between 2000 and 2005, in spite of investment in building renovation. People's opinion of business and shopping opportunities remained quite grim (Table 2). Fewer people held the worst views of prospects but more were unsure and this had the effect of slightly lowering the Balanced Scorecard calculation.

Table 2 Question responses from Newport
Full size table

Image and confidence building indicators

What the OBU evaluation has been able to find are many changes that might not have been predicted. In Colwyn Bay, for example, the score for the indicator of Citizen's Attitudes rose by a surprising 10 per cent. In the 2001 survey, 60 per cent of respondents indicated that the town had serious problems. This number in 2005 was down to 23 per cent. As a place to visit, to shop and to socialise, local people felt more positive than they did in 2001. Where half of those questioned in the baseline study said it was not a good place to work, only a quarter still felt that way in 2005 (Table 3). This change in attitude was measured in spite of very modest upgrades in the urban fabric. The indicator of townscape quality, measured by detailed observation of 39 separate views, had not changed. The indicator of public space management had only improved a few percentage points. Here it seems that simple enhancements of elements such as plantings and shop front canopies appear to have resulted in people feeling much better about the town (Figure 6).

Table 3 Attitude changes in Colwyn Bay
Full size table
Figure 6
figure 6

Shop front canopies in Colwyn Bay

Full size image

The indicator of business vitality and investment is one of the most important for gauging change. When we look at the Merchant City we see a THI area that remains complex, partly because of its location out of the commercial heart of Glasgow but a place where vacancy rates have fallen and where more businesses are operating. This is in part because of the enlightened policy of allowing shops to be occupied for no rent initially, and with the THI bearing the cost of initial refurbishment. The questionnaire findings suggest that the area is improving as a place for doing business with the mean scorecard measure going from 2.13 up to 3.35 out of 5. The stakeholder interview evidence supports this and also indicates that the right policy is being pursued in encouraging small businesses to invest in the area. There are still obstacles to business growth, such as high rates and low footfall, but investment has clearly been significant with the Fraser Suites making a major contribution to the vitality and attractiveness to visitors and businesses alike in the area. And, finally, the townscape survey shows that vitality is up and dereliction down, which are both indicators of an improving business context.

All of the observations outlined above as examples are derived from the solid evidence of questionnaires, interviews, townscape surveys and statistics and have enabled the evaluation team to draw clear conclusions about the indicators of success in THI sites.

Summary of Key Findings

What is the likely value of the research to the HLF?

This section sets out the key findings to date from the research that, in a sense, encapsulate the value of the work for the client. That is, the findings suggest an explanation of what conditions are likely to be necessary for investment to create both townscape and broader regeneration benefits. The findings are presented in two forms: three hypotheses, followed by a more detailed analysis in the form of a ‘predictive’ model, highlighting what seem to be key variables that, taken together, predict likely regeneration outcomes from investment. The section begins with a general overview of the schemes and their relative success.

Although the THI evaluation has another iteration to be completed, some conclusions can begin to be drawn on the evidence gathered thus far about conditions for success in THI schemes. This can be done by taking the four main themes together. Of those reviewed, the schemes that have produced the most improvement across most themes are Glasgow, Liverpool, Newport Pagnell, Creswell, West Wemyss and, to a lesser degree, Colwyn Bay and Bodmin. Wigtown has shown significant improvement, although this is only reflected in the qualitative data – attitudes and perceptions, rather than in the hard quantitative evidence. The least improved places are Draperstown, Newry, Pembroke Dock, Newport, Cleator Moor and Burslem. The others have remained largely static (Figure 7).

Figure 7
figure 7

Comparative success in different THI schemes

Full size image

When we ask ourselves why these differences in success have occurred a number of hypotheses can be posited. These hypotheses represent the key findings from the work to date, and focus on the factors that appear to explain success or lack of it. First, the schemes that have been most successful are in locations where local economic and social trends are already favourable. These sites include The Merchant City Glasgow, Liverpool and Newport Pagnell. This hypothesis is certainly in line with Pike et al (2007) in their exploration of what is meant by ‘regional development, its historical context, its geographies in space, territory, place and scale’ (p. 1253). They recognise that development is inevitably socially uneven and that the benefits of any kind of development are going to be regionally determined. It should not be surprising then that in spite of HLF efforts to be fair and equitable in their allocations of support, some THI sites are bound to be more successful and that is partly owing to the ambient economic conditions.

Second, the schemes where conservation investment has been focussed and highly visible have been successful. In Creswell and West Wemyss, large proportion of buildings were included in the scheme in a small and concentrated area, whereas in Bodmin a landmark building was targeted. In contrast, in the Draperstown and Burslem physical improvements were spread more thinly across a whole settlement, often taking the form of detailed and less obvious restoration. Work on the public realm, where appropriate, can also be important with Bloxwich serving a good example.

Third, THI schemes have made a significant difference in locations where demand and community aspirations for conservation have been latent. The Liverpool Rope Walks and Colwyn Bay are good cases in point. In describing the challenges of regeneration in Cardiff since 1990, Punter (2007) concluded that plans that were too grand and objectives that were too high can lead to failure. He discussed the ‘mismatch between the development ambitions and market size and trends’ (p. 357) and suggested that more incremental approaches tailored to the needs and capacities of the place were better than mega projects.

The converse of the point above seems to occur where there has been little economic or social potential, and in places such as Newport and Pembroke Dock success has been limited mostly to townscape improvement. Punter's (2007) warnings, particularly concerning smaller centres in Wales apply equally here. Town centre improvements have been made in Newport and specific large-scale building restorations were carried out in Pembroke Dock but neither place, at this time, has the market demand to support large-scale rejuvenation in the form of new residents with capital to invest. They may in the future, but there remains that mismatch between ambition and reality.

The final hypothesis is that lack of success is associated with places that have entrenched economic and social problems, where demand for housing and business premises is low, and where there has been long-term and extensive neglect. The Northern Irish towns of Newry, Draperstown and Moneymore are examples as are the English sites of Cleator Moor, Burslem and Middlesbrough. This hypothesis regarding regional problems concurs with the findings of Lagendijk (2007).

Although the THI schemes have had dramatically positive impacts on some sites and modest but significant impacts on others, even those where real measurable success has been more elusive, it is very likely that THI investment arrested decline. With their built heritage enhanced, these sites have the potential for future growth and prosperity that they would not have had without the conservation of their built heritage.

Reflecting on the four key goal of the programme, a question might be posed: is there any evidence of any correlation between them in the sample? That is, if the townscape improves, does it follow that image and confidence are boosted? Intriguingly, the headline findings (see Figure 7) suggest not in any obvious way. It is as if each goal is acting independently of others – as a general rule: although in particular cases it can be seen – for example West Wemyss – that there is a positive association between two factors (in this case townscape improvement and economic improvement; in another there is a negative association – for example Newport, between quality of life and economic improvement. We have found no generalisable and compelling case to see a clear, generalisable and potentially explanatory link between goals across the sample. However, once the final iteration of the research is complete this question will be revisited.

Predicting success and failure

In addition to the hypotheses and observations set out above, the value of the work in terms of the THI programme, and possibly more generally with respect to understanding how investment in built heritage can deliver wider regeneration outcomes, lies in our identification of key variables and factors that seem to have a direct bearing on projects and that are generalisable. There is not space in this article to go into much detail on this, and a few remarks will have to suffice, although the authors are currently preparing another paper on this theme.

Our analysis of the results from the research to date suggests that a number of key variables have an affect on the effectiveness of THI projects. These can be separated into intrinsic and extrinsic variables. The intrinsic variables are those that are dependent on the scheme itself. In essence, success depends on some fairly obvious issues that the research has confirmed, including

  • the scale of the scheme relative to the size of the area requiring regeneration: the more comprehensive a scheme the greater its impact;

  • the more experienced and developed a partnership the greater the impact;

  • the commitment, skills and continuity of key staff;

  • the length of time allowed within the programme to commit funding;

  • the more focused a scheme's objectives on quality conservation and physical regeneration, the greater the impact; and

  • the quality and timeliness of project monitoring.

The extrinsic variables are those that are independent of the scheme, and relate usually to its local and regional context. They include:

  • the local economic context is the single most important factor in determining a scheme's success;

  • the presence or absence of appropriate conservation expectation and building skills (having a direct impact on the quality of conservation work); and

  • the relationship of the scheme to the wider regeneration and conservation strategy operating in a location.

The research has also led to a better understanding of those things that need to be present if investment in the heritage in the style of the THI is likely to be beneficial. We have termed these ‘predictive factors’. Table 4 sets these out, organised under three themes that have emerged from the research, and represents a modification of the four thematic indicators around which the research was originally organised. The table also suggests for the client the type of evidence that might be called for to determine whether the predictive factor is likely to be present.

Table 4 Predictive factors
Full size table

As a research team we have proposed that a framework based on the above might be developed to predict likely outcomes (but not outputs, which would be impossible to do); but it could equally be used to assess the risk that a scheme will not deliver significant improvements. The key risk factors would be assessed against the set of critical conditions listed in the table. The risk clearly increases the worse a place is performing as measured against these. Again this hypothesis would need to be tested both against our own work and a broader set of examples, but could be done post hoc, in a review of all schemes where information to set against each predictive factor is available.

What is critical in assessing the likely outcomes of an application, as well as its risks, is to recognise the volatility and unpredictability of certain key variables. That is, certain conditions are more likely to change over time and have a more significant impact than others. The economic context is one such, in particular levels of investment and demand, along with the cost of credit. On the other hand, the underlying or intrinsic townscape and conservation quality is a relatively static factor, although it will degrade over time without some sort of investment. Again, further research would need to be conducted to determine which risk factors are key to predicting failure.

Conclusion

This article has described the methods used in an ongoing and longitudinal evaluation project, examining the effectiveness of the THI in the United Kingdom. In summary, this evaluation has shown that the THI programme, usually combined with other conservation investment, has potentially turned some places around, stabilised others and prevented some from further deterioration. Of course, it has not been able to demonstrate this by the use of a set of controls – places where investment has not happened, with the same set of characteristics as the schemes with THI investment. However, we are confident that the research is sufficiently robust to claim that there is more than an association between investment and improvement in many of the schemes examined. This research has also allowed some key lessons to be drawn that will hopefully inform future directions for conservation investment policy. Along with the work of Otsuka and Reeve (2007) and others on town centre management, the THI evaluation project provides an insight into the key elements in successful conservation-led urban regeneration. It is expected that these lessons will inform future HLF decisions in terms of policy, procedures and in the awarding of specific support grants. Although this article's intention was not to examine or compare in detail a small number of the cases examined – largely because of the limit of space – we believe that the more comprehensive analysis of the samples has allowed us to better explain the merits and operations of the method, to touch on key findings and to at least hint at the direction in which theory in respect of heritage investment might develop.