Sunteți pe pagina 1din 16

Urban Studies, Vol. 41, No.

4, 757–772, April 2004

An Indicator-based Approach to Measuring

Sustainable Urban Regeneration Performance:
Part 2, Empirical Evaluation and Case-study

Lesley Hemphill, Stanley McGreal and Jim Berry

[Paper first received, January 2003; in final form, October 2003]

Summary. This paper examines the sustainability of current urban regeneration practice,
through the application of weighted indicators and a points scoring framework. The analysis
applies the hierarchical model discussed in the preceding paper of this journal issue to case
studies of waterfront areas and cultural quarters in three European cities: Belfast, Dublin and
Barcelona. The evaluation permits performance comparisons to be made between the case studies
regarding the sustainability of regeneration areas and projects, variations on an indicator set
basis and the sensitivity of scores. Conclusions are drawn concerning regeneration practice, the
extent to which sustainability principles are adhered to, potential policy benefits and the
applicability of the model.

Sustainable development and urban regener- sustainable development, the role of urban
ation have become increasingly recognised regeneration in fulfilling sustainability objec-
as complementary goals, but there remains tives remains in its infancy.
only limited evidence of performance evalua- Sustainable Development: The UK Strat-
tions that investigate the extent to which egy (DOE, 1994) recognised the importance
regeneration projects have achieved sustain- of urban regeneration in contributing to sus-
able outcomes (Carley and Kirk, 1998; tainability by developing areas in an efficient
DETR, 1998a; OECD, 2000). Likewise, the way whilst making them more attractive
dissemination of what constitutes ‘good sus- places in which to live and work. In a similar
tainability practice’ as discussed in the ac- vein, the Urban Task Force in its “Towards
companying paper and how this can be an Urban Renaissance” report (DETR,
benchmarked remains underinvestigated. 1999b) advocated the need for a
What is becoming increasingly obvious is the “comprehensive package of regeneration
prominent place given in UK policy to the measures to address both the physical regen-
issue of sustainability and ultimately how eration of an area and the economic and
this can be best achieved (DETR, 1998b, social needs of the local population” (p. 133).
1998c, 1998d, 1999a, 1999b, 2000). How- This is further supported by the UK govern-
ever, despite this policy drive to instigate ment’s Urban White Paper Our Towns and
Lesley Hemphill, Stanley McGreal and Jim Berry are in the Centre for Research on Property and Planning, School of the Built
Environment, University of Ulster, Shore Road, Newtownabbey, County Antrim, Northern Ireland, BT37 0QB. Fax: 028 9036 6826.
E-mail:; and
0042-0980 Print/1360-063X On-line/04/040757–16  2004 The Editors of Urban Studies
DOI: 10.1080/0042098042000194098

Cities: The Future—Delivering an Urban gathering data appropriate for the indicators.
Renaissance (DETR, 2000) which identified Expert opinion plays a major part in the
bringing brownfield land and buildings back empirical investigation which seeks to assess
into constructive use as a key objective to the measurement of urban regeneration prac-
exploit their potential in contributing to the tice in light of sustainability principles. A
quality of urban life. The Sustainable Devel- detailed performance analysis of six indepen-
opment Strategy for the UK: A Better Quality dent case studies is presented utilising the 52
of Life (DETR, 1999a) lists regeneration as indicators discussed in the accompanying pa-
one of a number of aspects contributing to per. The performance of each case-study area
sustainable development whilst indicating is compared for each of the five indicator
that building sustainable communities re- groups in the model and against an overall
mains a priority for government action. How- aggregated score to provide an indication of
ever, Imrie and Thomas (1995) consider that relative performance. Sensitivity analysis is
the proliferation of new urban policy initia- applied to each indicator to assess different
tives since the early 1980s has not been options. The paper provides the basis for the
matched by systematic monitoring and evalu- use of the model for evaluation and policy
ation. Likewise, Turok (1989) states that the purposes, notably in the identification of the
scope and form of evaluative research have issues where present urban regeneration
often been illogical, weak and increasingly practice is not fully embracing the sustain-
concerned with a narrow range of quantitat- ability agenda. Conclusions are drawn on the
ive evaluative criteria. Such a focus on the respective case-study performances and the
quantitative measurement of policy impacts successes or shortcomings of present regen-
has failed to recognise the importance of the eration policy in creating sustainable out-
subjective, experiential ‘life-world’ of human comes, potential policy benefits and
well-being (Burns, 2000). Indeed, it is this applicability of the model.
failure to appreciate the human aspects of
urban policy through the use of purely quan-
titative methods of evaluation that has often
Sustainable Regeneration Performance
led to the disquiet expressed over urban pol-
icy evaluation.
Hakim (2000) argues that too often the The subject of measuring sustainability per-
secondary analysis of an existing data-set is formance has been assessed in detail in the
proposed at the expense of qualitative re- accompanying paper; however, an equally
search. Similarly Wong (2002) points to- important aspect of measurement is how to
wards the need for more qualitative research benchmark this performance. Benchmarking
to measure satisfactorily intangible issues, can be considered an effective means of
such as community identity and institutional helping to deliver better services by compar-
capacity. Perhaps of greater significance is ing performance against an accepted norm
the potential to combine both quantitative and learning from other areas/organisations
and qualitative techniques within any given (Audit Commission, 2000). This is all the
evaluation to ensure that their respective more evident in today’s society with headline
strengths can be capitalised on. Indeed, qual- (DETR, 1999a) and best value (ODPM,
itative research can be used in combination 2002) indicators providing two examples of
with virtually all other types of study, in- circumstances where indicators have been
forming the interpretation of more imper- used to measure and subsequently bench-
sonal statistical data (Hakim, 2000). mark performance. Benchmarking enables
The research reported in this article builds the comparison of quantitative performance
upon the concepts developed in the preced- data, identifies how performance differs from
ing paper in this issue and draws upon a other areas, how it has changed over time
combination of techniques as a means of and whether it can act as the spur for im-

provement or an example of how to achieve and work (21.5 per cent); Resource use (17.5
‘good practice’ (Audit Commission, 2000). per cent); Buildings and land use (18.9 per
Practitioners need to be able to establish cent); Transport and mobility (22.1 per cent);
the baseline economic and social conditions Community benefits (20 per cent). Further
against which they can measure the progress discussion of the indicators, the points scor-
and improvement resulting from regeneration ing framework and how they were selected
projects (Bristow, 1999). Likewise, it is ac- and tested is provided in the accompanying
knowledged that local authorities and devel- paper.
opers require practical tools to measure the
sustainability of developments (both build-
Case-study Selection
ings and infrastructure) at site or estate level
(Brownhill and Rao, 2002) to ensure that In testing the validity of the model and seek-
they meet planning requirements and contrib- ing to achieve a meaningful set of results,
ute to wider sustainability objectives. Simi- rigorous selection criteria were drawn up to
larly, a crucial part of benchmarking and the ensure that the same basic characteristics
use of indicators is to help guide appropriate were present in each case-study area thereby
action through the setting of suitable targets enabling a full comparative evaluation be-
(European Commission DGXI, 1996). The tween individual schemes. These criteria in-
Audit Commission (2000) identifies the key clude: previous track record of the case-study
elements of the benchmarking process— area, the hybrid of contacts available, indi-
namely, select service/project, identify re- cator coverage, completed schemes, size and
sources, identify partners, define and collect scale of scheme, and the occurrence of wa-
measures, compare performance, find best terfront and cultural regeneration projects
practice/targets, plan change, implement and within prospective locations.
monitor outcomes. The selection of the most appropriate case-
This paper adopts a similar benchmarking study locations, as governed by these criteria,
strategy to the measurement of sustainable identified a number of potential urban regen-
urban regeneration and utilises the model eration schemes embracing ‘good practice’.
developed and discussed in this journal issue. However, the requirement for differing
Although the points scoring parameters schemes including both cultural and water-
reflect current legislative or policy standards, front examples reduced the range of possibil-
it is important to stress the dynamic charac- ities. A further consideration was the number
teristics of the model and its flexibility to of case-study areas. Literature sources sug-
permit change as and when new technology gest that the significance and robustness of
or information renders current standards as research findings increases with the number
insufficient to meet sustainability require- of sites in which a survey is conducted,
ments. although the largest single gain occurs when
the number of sites is increased from one to
two (Sudman, 1976). This was supported by
Evaluative Model Overview
Yin (1984) who indicated that survey results
The preceding paper provides a detailed re- can be strong with a small number of case
sumé of the methodology underpinning the studies provided that they are carefully se-
research identifying a four-phase process lected. Based upon these considerations,
(Figure 1). This paper deals primarily with three case-study cities were selected with
the application of the evaluative model at each providing two individual studies to
case-study level. In particular, the focus is reflect the waterfront and cultural themes
upon phase four—the case-study application underpinning the ‘designated regeneration
and sensitivity analysis. The model is based area’. It is considered that the six case studies
on 52 individual indicators which have been provided a sufficient and diverse range of
weighted into five main groups: Economy examples to allow the indicators to be tested.

Figure 1. Methodological approach.

Belfast provides examples of mainstream Cathedral Quarter, formerly Northside, on

UK-based initiatives including the develop- the northern city-centre fringe of Belfast,
ment corporation model, Laganside Corpor- although possessing many key physical at-
ation, the last remaining development tributes, has been a difficult area to regener-
corporation excluding the recent revival of ate with many different initiatives (Berry and
the model in Thames Gateway and extensive McGreal, 1993) the latest of which focuses
use of gap-funding through urban develop- upon its ethos as a cultural location.
ment grant. In Belfast, two key schemes The second case-study city, Dublin, pro-
Laganside and Cathedral Quarter fit the vides many contrasts with Belfast from a
wider selection criteria of waterfront and cul- policy perspective (Berry and McGreal,
tural quarters respectively (Table 1). The for- 1992). Both of the selected schemes in
mer has a proven national and international Dublin (Table 2), the Docklands area and the
track record. Laganside schemes received cultural quarter of Temple Bar have
recognition from the RICS in 1998 by win- benefited significantly from the use of tax-
ning their ‘Reclamation of Contaminated ation breaks. Temple Bar is widely accepted
Land Award’ and were the subject of an as being a good model of cultural regener-
OECD evaluation study (OECD, 2000). The ation with the area promoted as the flagship

Table 1. Belfast case studies: summary

Laganside—Waterfront Cathedral Quarter—Cultural

140 ha site situated 1 km from Belfast city centre 12.1 ha site located on the northern fringe of the
city centre
Area first designated in 1987 with the scheme Area designated in 1998 with the scheme pro-
details highlighted in the Laganside Concept posals highlighted in the Cathedral Quarter Re-
Plan generation Strategy (previously Northside)
Area falls under the control of the Laganside Area falls under the control of the Laganside
Corporation, a form of development corporation Corporation, but treated as a distinct area separ-
set up to oversee the development of the area ate from the main Laganside site
Area consists of 6 key commercial sites and 6 Area (will) consist of 2 apartment developments;
apartment developments, as well as environmen- a hotel; various cultural and arts developments;
tal improvements to River Lagan and the intro- building refurbishments; a multi-storey car park;
duction of a cross-harbour road and rail bridge environmental improvements to streetscape

Table 2. Dublin case studies: summary

Dublin Docklands—Waterfront Temple Bar—Cultural

526 ha site located in the northern area of the 12 ha site located in the heart of Dublin city
traditional dockland; 85 ha available for develop- centre
ment and a further 10 ha for open space
Area first designated in 1986 although an official Area designated in 1991 with development car-
Master Plan was not adopted until 1997 ried out in line with the Temple Bar Develop-
ment Programme 1996
Area falls under the control of the Dublin Dock- Area fell under the control of 2 independent
lands Development Authority (DDDA) which re- companies—Temple Bar Renewal Ltd (respon-
placed the Custom House Docks Development sible for deciding funding applications) and
Authority (CHDDA) in 1997 Temple Bar Properties Ltd (responsible for car-
rying out the development work)
Area consists of 6 main development areas each Development carried out in 2 main phases:
of which have various objectives ranging from 1991–96 and 1996–1999. Area consists of an
promoting residential use, developing landscap- Irish Film Centre, infrastructural improvements,
ing, promoting public transport, promoting apartment developments, retail lettings, building
tourism/hotel facilities, protecting natural habi- refurbishments, provision of innovative archaeo-
tats, supporting cultural development and the logical component, promotion of area as a cul-
expansion of enterprise uses tural quarter

project to mark Dublin’s year as European approach taken to the stimulation of private
City of Culture in 1991 (Montgomery, 1995). investment between tax-based incentives and
In a similar vein, the Docklands area has grant-based mechanisms as applied in
been widely advocated as a success (Gahan, Belfast.
1993) and long-term sustainability is one of The selection of Barcelona (Spain) widens
the objectives of the Dublin Docklands Mas- the research to a European perspective, facil-
ter Plan making the area an ideal case study itating comparison with the UK and Ireland
in which to apply the indicators (DDDA, dimension. Barcelona also provided two very
1997). In developing the contrast to Belfast, different regeneration schemes—namely, the
the Dublin case studies provided the oppor- Ciutat Vella (Old City) as the cultural exam-
tunity to compare and contrast the differing ple and the Olympic Village as the water-

Table 3. Barcelona case studies: summary

Olympic Village—Waterfront Ciutat Vella—Cultural

47 ha site located between the Cuitadella Park 386 ha site located in the old historical centre of
and the centre of the Poblenou area—part of a the city and constitutes one of the 10 districts
zone of coastal facilities (130 ha) into which Barcelona is divided
Decision to regenerate the area coincided with Area designated as an Area of Integrated Re-
the announcement that Barcelona was to be the habilitation (AIR) in 1986 with 4 special plans
venue for the 1992 Olympic Games. In 1989, for internal reform (PERI) to cover the key areas
various changes were made to the Plan General in the old city: El Ravel, Casc Antic, Barceloneta
to allow a change in the function of the area and Barri Gotic
Area fell under the control of an ad hoc body Area fell under the control of a public–private
called Holding Olimpico SA (HOLSA) who company called Promocio Ciutat Vella SA
were responsible for co-ordinating the develop- (PROCIVESA) to carry out or co-ordinate the
ments main works envisaged in the special plans
Area consists of various 5-storey apartment The main elements of PERI plans included land
blocks with commercial and retail facilities be- clearance, redevelopment or refurbishment of
low, the twin Olympic Towers (Hotel and property, creation of public open space and the
Office); and high-quality public space provision of modern infrastructure

front example (Table 3). The decision to use tors and include: government departments/
Barcelona as a case-study location was based agencies and other development organ-
on a number of counts, but primarily the isations, literature/official reports, other re-
international reputation that the city has search, end-user surveys of occupiers/
achieved since the Olympic Games. Indeed, employers, consultation with a range of
the Urban Task Force report (DETR, 1999b) professionals in the built environment
charted some of the regeneration work un- (architects/property developers/estate agents),
dertaken as being a good example of the way questionnaire surveys (employees/residents)
forward for future practice. Furthermore, the and site survey and investigation.
urban design and architectural quality of The five separate indicator sets—namely,
what has been developed have been docu- Economy and work, Resource use, Buildings
mented in journals, culminating in the RIBA and land use, Transport and mobility and
awarding the area with a gold medal for the Community benefits for each case study were
outstanding quality of the transformations evaluated on both an individual and collec-
(DETR, 1999b). tive basis in accordance with the points scor-
ing criteria and rationale highlighted in the
associated paper. The case-study results are
Case-study Results considered under three headings: indicator
Accessing information to enable each of the set analysis, benchmark analysis and sensi-
indicators to be assessed required the utilis- tivity analysis.
ation of a combination of highly varied meth-
ods, applied consistently across each of the
Indicator Set Analysis
case studies. Some provided hard
quantifiable data that could be directly ap- This analysis initially reviews the scoring on
plied to the points scoring framework, whilst an indicator basis, followed by the overall
others provided more qualitative information points for each indicator group, supple-
requiring interpretation prior to the allocation mented by a brief overview of the relative
of points. The sources of information utilised performance of the individual case studies in
reflect the complexity and variety of indica- delivering sustainable regeneration out-

Table 4. Economy and work scoring performance

Economy and work indicators LS CQ DD TB OV CV

Number of jobs created per 1000 square metres 8 10 6 4 4 2

Net jobs created 6 4 6 4 8 8
Percentage of new enterprises still operating after 3 10 8 10 6 10 8
Quality of jobs created 4 10 4 8 6 8
Leverage ratios 4 4 4 4 2 2
Performance of incentive mechanisms 6 1 6 2 6 8
Partnership structure performance 6 5 8 5 8 8
Effectiveness of exit strategy 7 5 8 2 8 7
Incorporation of training programmes 8 7 7 7 6 6
End-user scheme satisfaction 8 5 6 6 8 7

Total score 67 59 65 48 66 64
Percentage score 67 59 65 48 66 64
Weighted score (weighting factor 21.5) 1440.5 1268.5 1397.5 1032.0 1419.0 1376.0
Ranking 1 5 3 6 2 4

comes. Derivation of these indicators has cess in terms of the number of jobs created
been discussed in detail in the accompanying and the quality of those jobs with both indi-
paper. Their richness stems from the number cators scoring highly. However, the rela-
and comprehensive range of indicators em- tively poor performance of the incentive
ployed across the five component areas and mechanisms indicator, within the Cathedral
the extensive nature of the evidence. Quarter and Temple Bar areas, seems to indi-
cate that incentives had little influence on the
decision of businesses to move to these loca-
Economy and work. This constitutes a subset tions. This also reflects a high number of in
of 10 indicators formulated to evaluate the situ end-users. From the sustainability per-
performance of economic strategies as well spective, the findings of the Economy and
as the tangible outcomes of the investment work indicators show that in most of the
process in terms of the employment gener- regeneration schemes partnership structures,
ated. For this indicator set, the three water- exit strategies and training programmes are
front localities of Laganside Belfast, the important, although the low scores awarded
Olympic Village Barcelona and Dublin to the leverage ratios suggest a continuing
Docklands achieved the highest scores dependency on public-sector money to se-
reflecting the economic emphasis initially cure long-term success.
needed in these areas to stimulate investment
(Table 4). Of the individual indicators, the
percentage of new enterprises still operating Resource use. The Resource use indicators
after 3 years showed the best performance relate to the effective use of the physical
across all case-study areas, supporting the resources whilst dealing with environmental
need for longevity of businesses in the long- issues such as the reclamation of materials,
term sustainability of regeneration schemes. waste minimisation, energy efficiency and
The 2 Barcelona case studies are the only conservation. This indicator group produced
examples to embrace significantly the indi- a similar set of results across 5 of the case-
cator relating to the percentage of net jobs study areas with only 7 per cent covering the
created: both provide over 30 per cent of top 5 performers (Table 5). The Barcelona
their workforce from their local area. The case studies lead on these indicators with the
Cathedral Quarter Belfast had notable suc- Olympic Village scoring 64 per cent and the

Table 5. Resource use scoring performance

Resource use indicators LS CQ DD TB OV CV

Reclamation of building materials 2 2 2 6 2 8

Retention of environmental features 6 6 4 2 2 6
Household waste disposal 2 2 2 2 6 6
Firms undertaking waste minimisation 10 8 10 10 10 8
Energy efficiency: building lay-out/design 8 7 8 9 8 5
Energy efficiency: building materials/construction 6 4 8 8 6 5
Conservation of built heritage 6 10 2 10 6 10
Incorporation of environmental design 4 4 10 6 10 4
Performance of environmental management 8 6 5 1 8 5

Total score 52 49 51 54 58 57
Percentage score 58 54 57 60 64 63
Weighted score (weighting factor 17.5) 910.0 857.5 892.5 945.0 1015.0 997.5
Ranking 4 6 5 3 1 2

Ciutat Vella 63 per cent. The Cathedral regeneration schemes upon property and de-
Quarter Belfast provided the poorest per- velopment outcomes. The top-ranked loca-
formance in this group (54 per cent) tions are the Olympic Village (78 per cent),
reflecting cost constraints involved in im- followed by Ciutat Vella (74 per cent) with
proving the energy efficiency of older build- Laganside, Cathedral Quarter and Dublin
ings. All case studies scored well in the Docklands all scoring 73 per cent (Table 6).
waste minimisation indicator demonstrating These results place the Barcelona case stud-
a very high percentage of firms actively ies collectively as the best performers in this
undertaking waste audits. However, this sector corresponding with their well-recog-
positive treatment of waste did not extend to nised position at the forefront of urban de-
household waste disposal, with the Barcelona sign and the creation of buildings with
case studies the only areas to perform above architectural distinction.
the less than 20 per cent band—indicating Contrasts are apparent in terms of the indi-
that in the UK and Ireland households are vidual indicator performances. The analysis
falling well short of the 40 per cent recycling demonstrates that the three waterfront case
target. As expected, the three cultural case studies demonstrate a high ratio of open
studies substantially outperformed the water- space to built form, whilst the cultural case
front areas regarding the conservation of studies balance this with having a much
built heritage, but conversely were lower higher ratio of converted/redeveloped build-
ranked in terms of incorporating environ- ings to new build. Nevertheless, all case
mental design into the building stock. With studies illustrate high density, a balance of
the exception of Ciutat Vella and Temple uses (apart from Dublin Docklands) and the
Bar, the case studies illustrated that little or complete removal of any contaminated land.
no reclamation of building materials oc- The office rental indicator, with the excep-
curred. This finding is somewhat surprising tion of Temple Bar, implies that favourable
in sustainability terms, given the high levels rents are achievable with relatively high lev-
of land filling of such waste. els of occupancy in areas that previously
were classified as weak market locations.
Buildings and land use. The best overall The performance of the property variables
performance in five of the case studies is for reflects the acceptable levels of design and
the Buildings and land use indicator group, quality of the final built product. However,
reflecting the emphasis of these particular the main vulnerability from a sustainability

Table 6. Buildings and land use scoring performance

Buildings and land use indicators LS CQ DD TB OV CV

Ratio of open space: built form 10 6 10 4 10 6

Ratio of converted buildings: new build 2 10 4 10 2 6
Reclamation of contaminated land 10 10 10 10 10 10
Density levels in relation to plot size 8 10 10 8 8 10
Balance of uses 8 6 4 10 8 10
Occupancy levels 8 6 10 8 6 10
Office rental v. CBD rents 8 8 8 2 10 8
Quality of final product 8 7 8 6 8 7
Design quality 8 6 7 9 9 7
Quality of public space 6 6 5 4 8 5
Usage of public space 6 8 6 6 8 6
Quality of private space 6 4 5 3 7 4

Total score 88 87 87 80 94 89
Percentage score 73 73 73 67 78 74
Weighted score (weighting factor 18.9) 1663.2 1644.3 1644.3 1512.0 1776.6 1682.1
Ranking 3 4 4 6 1 2

perspective, shown by the rankings, is the port links and travelling habits in relation to
perception of poorer quality private space in leisure activities perform well. However,
four of the case studies, the exceptions being with the exceptions of Temple Bar, the
the Olympic Village and Laganside. Olympic Village and to lesser degree in La-
ganside, the land allocation dedicated
Transport and mobility. Transport and mo- specifically for pedestrian space and move-
bility indicators encapsulate matters pertain- ment is ranked low with the converse that too
ing to infrastructure improvement, travel high a proportion of land is devoted to roads.
habits in relation to a range of activities and For residential car parking provision, the
car parking provision. This indicator set pro- scores reflect that fewer spaces are allocated
duced a varied performance across the case within schemes, whereas commercial car
studies, with only the Olympic Village (73 parking provision points towards an overpro-
per cent) and Temple Bar (70 per cent) pro- vision with the result that the public transport
ducing high scores reflecting their excellent option becomes less lucrative. This raises
internal accessibility and proximity advan- particular concern in meeting sustainability
tages within their respective city centres objectives. Clearly, further deliverability is
(Table 7). The Dublin Docklands was ranked required across the case studies in integrated
third (63 per cent) ahead of Cathedral Quar- land-use and transport planning.
ter, Belfast (62 per cent). Laganside (56 per
cent) had the lowest score for Transport and Community benefits. Community benefits
mobility, closely followed by Ciutat Vella capture a highly variable range of indicators,
(58 per cent), making the Belfast case studies with the performance of Temple Bar (78 per
the weakest collective performers. Given that cent) significantly ahead of the next-highest-
a gap of 17 per cent separates the top and ranked case studies—namely, Ciutat Vella
bottom performers, there is ample evidence (67 per cent) and Olympic Village (66 per
to suggest that the importance placed upon cent). The high score associated with Temple
accessibility and discouraging private car use Bar reflects attempts to foster close links
differs greatly between case-study locations. with the community (Table 8). In compari-
Those indicators measuring public trans- son, the Dublin Docklands (54 per cent) had

Table 7. Transport and mobility scoring performance

Transport and mobility indicators LS CQ DD TB OV CV

Land devoted to roads 4 6 8 4 6 4

Land devoted to pedestrians 6 2 2 8 8 2
Road improvements 8 5 4 5 7 5
Work travelling habits 4 4 6 6 6 6
Leisure travelling habits 6 10 10 10 10 10
Public transport links 8 8 10 7 10 8
Car-parking provision—residential 6 8 8 8 8 6
Car-parking provision—commercial 4 8 4 8 4 4
Integration of land use and public transport 5 5 5 7 7 7

Total score 51 56 57 63 66 52
Percentage score 56 62 63 70 73 58
Weighted score (weighting factor 22.1) 1127.1 1237.6 1259.7 1392.3 1458.6 1149.2
Ranking 6 4 3 2 1 5

Table 8. Community benefits scoring performance

Community benefits indicators LS CQ DD TB OV CV

Access to open space 8 8 8 10 10 10

Access to leisure facilities 8 6 8 10 10 10
Access to retail facilities 8 10 8 10 4 6
Access to educational needs 4 8 4 8 6 8
Access to medical facilities 2 6 4 4 10 6
Access to entertainment facilities 8 8 6 10 8 6
Access to cultural facilities 4 4 4 10 4 6
Access to housing 6 4 6 6 7 7
On-site retail facilities 4 5 5 8 5 8
LA21 effectiveness 7 4 6 6 8 6
Community ownership 4 5 5 6 5 6
Community group involvement 3 1 1 5 2 1

Total score 66 69 65 93 79 80
Percentage score 55 58 54 78 66 67
Weighted score (weighting factor 20.00) 1320.0 1380.0 1300.0 1860.0 1580.0 1600.0
Ranking 5 4 6 1 3 2

the lowest score followed by the two Belfast On an individual indicator basis, there is a
examples of Laganside (55 per cent) and low level of community group involvement
Cathedral Quarter (58 per cent). For Dublin and, at best, only partly developed com-
Docklands and Laganside, this in part reflects munity ownership of the schemes across all
the predominant emphasis upon a physical case studies reflecting that, with the excep-
development programme particularly in the tion of the Ciutat Vella, none is a truly
early stages of the regeneration process. The established residential community area. In
scores indicate that more attention needs to terms of accessibility, the outcome is vari-
be placed upon improving community rela- able. Certain facilities such as retailing,
tions at the outset of regeneration schemes leisure and entertainment perform at reason-
rather than being slowly addressed in later ably encouraging levels in terms of sustain-
phases. ability principles, although more alarmingly

Table 9. Overall points scoring summary

Case-study location Weighted score Percentage score Scaling Overall ranking

Laganside 6460.8 62 ‘Good’ (5)

Cathedral Quarter 6387.9 62 ‘Good’ (6)
Dublin Docklands 6494.0 63 ‘Good’ (4)
Temple Bar 6741.3 65 ‘Good’ (3)
Olympic Village 7249.2 70 ‘Good–excellent’ (1)
Ciutat Vella 6804.8 66 ‘Good’ (2)

others such as education and medical facili- 5191.0 to 6125.38) indicating above-average
ties produced a less satisfactory result apart performance, 60–69 per cent (6229.2 to
from Olympic Village. Likewise, the fact 7163.58) classified as good, 70–79 per cent
that accommodation in these areas has be- (7267.4 to 8201.78) good–excellent and
come sought after and hence is highly priced above 80 per cent (weighted score 8305.6)
can exclude lower-income households. This excellent. However, the spread of scores and
is apparent in the variable performance of the associated banding have sufficient distinction
housing accessibility indicator and raises so- also to enable application of the model to
cial equity issues concerning—for example, areas considered to be failing from a regener-
housing affordability. This grouping of indi- ation perspective.
cators shows that out-turn performance is Each of the six case studies received high
skewed more towards the remunerative de- total scores and can be classified as falling
liverables, whereas the community compo- within the ‘good’ category, with the Olympic
nents lag considerably with the result that Village Barcelona achieving the distinction
sustainability objectives are not being fully of falling within the ‘good–excellent’ cate-
embraced. gory. A relatively small percentage (8 per
cent) separates the top and bottom perform-
ers, reflecting the exemplar status of these
Benchmarking Analysis
three cities in terms of regeneration. On the
The overall weighted scores, the summation basis of this analysis, the top-ranked case-
of the product of the total points times the study location is the Olympic Village with 70
weighting factor, permit an analysis of each per cent (weighted score 7249.2) and Ciutat
case study against the theoretical maximum Vella has the next-best performance with 66
possible score of 10 382. The median score per cent (weighted score 6804.8). Temple
of 5191 is taken to represent average per- Bar ranks third with 65 per cent (weighted
formance with the weighted points band score 6741.3). Outside the top three case
(4152.8 to 5078.18) in the decile below the studies in terms of sustainability perform-
median considered to be of below-average ance, the difference between the scores is
performance. A case study with a weighted small. Dublin Docklands (63 per cent,
total below 40 per cent of the maximum weighted score 6494) just edges out Lagan-
possible score is rated as poor in relation to side (62 per cent, weighted score 6460.8) for
sustainability practice. However, as all six fourth place, whilst the Cathedral Quarter (62
case studies were selected on the basis of per cent, weighted score 6387.9) although in
recognised or potential achievements in re- the bottom position is still classified in sus-
generation, it is important to have a better tainability terms as good (Table 9). It is the
discrimination of weighted scores above the weighted score that an area or project re-
median. Again, deciles permit a rational cate- ceives that is important.
gorisation of scores into nominal groups with The findings illustrate that the model is
the band 50–59 per cent (weighted score capable of discriminating between compara-

tive overall performance levels on a case- data collection phase, substituting the orig-
study basis. Significantly, each of the case inal score with the lower score where any
studies confirmed that sustainability is be- difference occurred, with the converse occur-
coming a more integral part of regeneration ring in the case of the higher variation. Both
practice, with none of the areas below the sets of scores provide an alternative view of
‘good’ category. Furthermore, no noticeable performance and represent the extreme of the
pattern was established between those case distribution of scores.
studies undertaken in waterfront locations The results of the sensitivity analysis illus-
compared with the more culturally based trate the effect that a change in the points
projects; rather, differences are more related allocated to any indicator would have on the
to the city than defined locations within a weighted scores and relative ranking of the
city. case studies. In terms of the low variation,
The case studies that have been completed some case-study areas proved to be more
or those nearing completion provided the sensitive to a change in points total than
best performances, notably the Olympic Vil- others (Table 10). For example, the Olympic
lage (70 per cent), Ciutat Vella (66 per cent) Village shows the smallest percentage drop
and Temple Bar (65 per cent), as these areas (4 per cent) in the overall performance level
benefit from more established markets and indicating that it is the least sensitive to a fall
greater investor, resident and employee in points total. Further confirmation of top
confidence. This suggests that the perform- ranking for Olympic Village (66 per cent) is
ance levels of the other areas should improve illustrated by an increase in the gap (to 6 per
as schemes mature. In this respect, Dublin cent) to the next-highest-scored location,
Docklands (63 per cent), Laganside (62 per Ciutat Vella (60 per cent), on the basis of
cent) and Cathedral Quarter (62 per cent) lowest score variation. The largest drop in
have performed well given these constraints points was experienced by Laganside and
and just lag the performance levels set by the Cathedral Quarter (8 per cent), followed by
top-ranked case studies. Temple Bar (7 per cent), resulting in both
Belfast case studies displaying a similar per-
formance level (54 per cent of possible
Sensitivity Analysis
weighted score) and confirming their relative
The regeneration characteristics evaluated by status as the weakest performers, although
the indicators are complex, with a degree of still above average. The utilisation of the low
qualitative evidence from specific expert score variation also shows a narrowing of the
groups as discussed in the accompanying gap between the Dublin case studies to 1 per
paper. Hence, sensitivity analysis is deployed cent, demonstrating that the Temple Bar re-
to allow for any uncertainty and to analyse sults were more sensitive to change than the
the effect that a change in the scores may Docklands (Table 10).
have on the overall result. Sensitivity simula- Perhaps the most significant result of ap-
tions are undertaken whereby higher and plying the low score variation is the effect on
lower variations of the final scores of several the overall ranking. In all cases, the appli-
individual indicators were chosen to test cation of the low score variation saw a real-
what impact, if any, this would exert on the location of the case studies from the ‘good’
overall final points scoring. This demon- category to the lower ‘above average’ tier of
strates how sensitive the results are to a the benchmark scale and the top-ranked
change in the individual indicator scores and Olympic Village area (6823.9) was reallo-
indicates whether or not a particular case cated to the upper half of the ‘good’ cate-
study should be reallocated to another per- gory. Of the case studies reallocated to the
formance level on the benchmark scale. The ‘above average’ category, Ciutat Vella
lower score variation was taken from alterna- (6195.7), Temple Bar (5982.8) and the
tive pessimistic responses received in the Dublin Docklands (5948.3) are in the upper

Table 10. Comparison and ranking of sensitivity analysis (low variation)

Case-study location Weighted Percentage Percentage Scaling Original Low

score score difference category ranking ranking

Laganside 5588.5 54 -8 ‘Average–good’ (5) (5)

Cathedral Quarter 5485.5 54 -8 ‘Average–good’ (6) (6)
Dublin Docklands 5948.3 57 -6 ‘Average–good’ (4) (4)
Temple Bar 5982.8 58 -7 ‘Average–good’ (3) (3)
Olympic Village 6823.9 66 -4 ‘Good’ (1) (1)
Ciutat Vella 6195.7 60 -6 ‘Average–good’ (2) (2)

Table 11. Comparison and ranking of sensitivity analysis (high variation)

Case study location Weighted Percentage Percentage Scaling category Original High
score score difference ranking ranking

Laganside 7003.9 68 ⫹6 ‘Good’ (5) (5)

Cathedral Quarter 7164.7 69 ⫹7 ‘Good’ (6) (3)
Dublin Docklands 6788.0 65 ⫹2 ‘Good’ (4) (6)
Temple Bar 7115.8 69 ⫹4 ‘Good’ (3) (4)
Olympic Village 7849.9 76 ⫹6 ‘Good–excellent’ (1) (1)
Ciutat Vella 7251.3 70 ⫹4 ‘Good–excellent’ (2) (2)

half, whilst Laganside (5588.5) and Ca- of the ‘good–excellent’ tier with the other
thedral Quarter (5485.5) are placed in the case studies consolidating their position
middle to lower part (Table 10). The analysis within the respective categories, notably the
suggests that the overall scores awarded for Olympic Village (7849.9), Cathedral Quarter
the case studies are sensitive to a downward (7164.7), Temple Bar (7115.8) and Lagan-
change in points total, with all six experienc- side (7003.9). The Dublin Docklands experi-
ing a reallocation to the next lower category. enced little change in weighted points total,
Concerning the high score variation, some with only a slight improvement to the mid
case studies demonstrated a greater sensi- position of the ‘good’ category (Table 11).
tivity to points change than others (Table The overall scores awarded for the case stud-
11). The Dublin Docklands showed the ies have proved to be less sensitive to an
smallest increase (2 per cent) in points total, upward change in points total, suggesting
followed by Temple Bar and Ciutat Vella (4 that respondents may have adopted optimis-
per cent), illustrating that the Docklands is tic assessments of regeneration areas in their
the least sensitive to a rise in points total original response to particular indicators.
from the high score variation. Cathedral As none of the case studies experienced
Quarter (7 per cent), the Olympic Village greater than a 7 per cent change in the points
and Laganside (6 per cent) experience the total allocated by selecting either the high or
greatest increase in points. The analysis low score variations, it can be concluded that
based on a high score variation establishes the original scores awarded can be treated
Cathedral Quarter (69 per cent), demonstrat- with a high degree of confidence. Further-
ing its latent potential, as the third-ranked more, the overall rankings developed from
case-study area behind the Olympic Village the sensitivity analysis showed very little
(76 per cent) and Ciutat Vella (70 per cent), movement, with the exception of the Ca-
which consolidate their positions at the top. thedral Quarter case, which experienced a
The application of the high score variation major upward change attributable to its in-
moved Ciutat Vella (7251.3) to the margins herent potential. The outcome offers con-

siderable support to the scores allocated and reliance to be placed upon the weights as-
the rankings developed as well as the re- cribed to the components tier. In addition, the
liability of the data sources used. model has flexibility in that parameters can
be changed to reflect new standards arising
from shifts in policy or technological devel-
opments, thereby contributing to its effec-
Within urban regeneration policy and prac- tiveness in the evaluation of mechanisms in
tice, the application of performance indica- creating sustainable urban environments.
tors is becoming increasingly significant in The weightings applied to the respective
measuring sustainability outputs. However, components enhance the utility of the model
the use of methodologies to benchmark and as a comparative tool in the analysis of sus-
assess the performance of sustainability re- tainability practice and outcomes. In this re-
mains largely underresearched. Weighted in- spect, the model is capable of application
dicators and a points scoring framework across regeneration areas and projects of dif-
developed in the preceding paper are applied fering size, scale, level of perceived success/
to both waterfront-led and culture-led regen- delivery and at different phases of a
eration schemes in three European cities to particular initiative. Indeed, the model is use-
measure achievements in the delivery of sus- able as a tool to test at different stages the
tainable developments/environments. The degree to which a particular regeneration
study demonstrates that the more established project is adhering to the principles of sus-
and mature regeneration locations (Olympic tainability. Thus, at strategic points over the
Village, Ciutat Vella and Temple Bar) de- life cycle of the project, the model could be
liver better sustainability performance than utilised to inform and influence strategy in a
those areas which are less well developed, corrective capacity by highlighting any
highlighting the importance of timing in re- significant divergence from sustainability ob-
generation areas. The use of sensitivity jectives. Ideally, a project/regeneration area
analysis to test the various options, particu- should be surveyed before mid term to en-
larly for those indicators relying on qualita- sure that sufficient time is available to im-
tive evidence, indicates that the points score plement change if scores are low implying
allocated to the various case studies in the that the scheme that is unlikely to achieve
empirical investigation can be treated with sustainability targets. In theory, resurvey
confidence. Significantly, the analysis shows could potentially occur at several stages over
that both of the Barcelona case studies, the the period of a project as well as forming part
Olympic Village and Ciutat Vella, and Tem- of the overall evaluation at the end of the
ple Bar, Dublin, display a high degree of initiative.
adherence to sustainability principles. Potential policy benefits stem from the
A key parameter of the hierarchical model model’s holistic approach and its applicabil-
developed in the accompanying paper and ity across different regeneration mechanisms.
tested in this analysis is its inclusiveness in In assessing the extent to which regeneration
terms of 5 indicator sets in the components areas are conforming to sustainability cri-
tier and 52 separate indicators. The robust- teria, the model provides a valuable tool in
ness of the model is demonstrated by its policy analysis. Application, as demonstrated
application in different regeneration typolo- by the case studies, illustrates where good
gies (waterfronts and cultural quarters) and practice is occurring—notably, in the
in different urban contexts in terms of city Barcelona examples and particularly the
characteristics, institutional structures and Olympic Village. An added feature and po-
policy frameworks. Development of the tentially of greater significance is the ability
model through the use of expert opinion is a of the model to operate at a components level
key aspect of the methodological approach, which, from a policy perspective, discrimi-
the comprehensive nature of which allows nates between those aspects where the regen-

eration initiative is working well and those sure success, Urban Environment Today, 62(21
matters which require further policy inter- January), p. 13.
BROWNHILL, D. and RAO, S. (2002) A Sustainabil-
vention. From the analysis contained in this ity Checklist for Developments: A Common
paper, the strong performance of Laganside Framework for Developers and Local Authori-
(Belfast) in terms of Economy and work is ties. Watford: CRC Ltd.
highlighted, but equally apparent is the BURNS, R. B. (2000) Introduction to Research
weaker performance in relation to Com- Methods. London: Sage Publications Ltd.
CARLEY, M. and KIRK, K. (1998) Sustainable by
munity benefits and Transport and mobility. 2020? A Strategic Approach to Urban Regen-
Hence, policy priorities in Laganside need to eration for Britain’s Cities. Bristol: The Policy
be redirected to the latter issues, whereas in Press, for the Joseph Rowntree Foundation.
the case of Temple Bar (Dublin) the model DDDA (DUBLIN DOCKLANDS DEVELOPMENT AUTH-
ORITY) (1997) Dublin Docklands Area—Master
highlights a converse situation.
Plan 1997. Dublin: DDDA.
The capacity of the model to drill down DETR (DEPARTMENT OF THE ENVIRONMENT,
and examine components is important from a TRANSPORT AND THE REGIONS) (1998a) Sustain-
policy and decision-making perspective, par- able Regeneration Good Practice Guide. Lon-
ticularly in the allocation and targeting of don: HMSO.
funds and other resources by allowing pol- DETR (1998b) Planning for the Communities of
the Future. London: HMSO.
icy-makers to identify areas requiring action. DETR (1998c) Planning for Sustainable Develop-
This in turn offers the possibility to influence ment: Towards Better Practice. London:
future performance and guide policy and HMSO.
practice. On a wider perspective, the appli- DETR (1998d) Sustainability Counts. London:
cability of performance indicators allows HMSO.
DETR (1999a) Sustainable Development Strategy
policy approaches to be assessed on their for the UK: A Better Quality of Life. London:
ability to deliver sustainable development HMSO.
and assess whether partnership structures are DETR (1999b) Towards an Urban Renaissance:
effective. Regeneration performance outputs Final Report of the Urban Task Force. London:
are increasingly expected to show joined-up DETR.
DETR (2000) Our Towns and Cities: The Future
holistic thinking across programmes by pro- Delivering an Urban Renaissance. London:
moting physical, social, economic and en- HMSO.
vironmental criteria. The indicator-based DOE (DEPARTMENT OF THE ENVIRONMENT) (1994)
model advocated and developed in these two Sustainable Development: The UK Strategy.
papers allows an evaluation of the extent to Cm 2426. London: HMSO.
which integration is being achieved and sus- Sustainable Cities. Expert Group on the Urban
tainability objectives delivered within urban Environment. Luxembourg: European Com-
regeneration. mission.
GAHAN, J. (1993) Fiscal measures in promoting
urban regeneration: Custom House Docks,
Dublin, in: J. N. BERRY, W. S. MCGREAL and
References W. DEDDIS (Eds) Urban Regeneration, Prop-
erty Investment and Development, pp. 95–107.
AUDIT COMMISSION (2000) Getting Better All the London: E&FN Spon.
Time: Making Benchmarking Work. London: HAKIM, C. (2000) Research Design: Successful
HMSO. Designs for Social and Economic Research.
BERRY, J. N. and MCGREAL, W. S. (1992) Fiscal London: Routledge.
mechanisms and inner city development, Jour- IMRIE, R. and THOMAS, H. (1995) Changes in
nal of Property Finance, 3(4), pp. 543–552. Local Governance and their Implications for
BERRY, J. N. and MCGREAL, W. S. (1993) Public Urban Policy Evaluation, in: R. HAMBLETON
sector initiatives in the regeneration of Belfast, and H. THOMAS (Eds) Urban Policy Evalu-
in: J. N. Berry, W. S. MCGREAL and W. DEDDIS ation: Challenge and Change, pp. 123–138.
(Eds) Urban Regeneration, Property Invest- London: Paul Chapman Publishing.
ment and Development, pp. 193–214. London: MONTGOMERY, J. (1995) The story of Temple Bar:
E&FN Spon. creating Dublin’s cultural quarter, Planning
BRISTOW, H. (1999) Developing the tools to mea- Practice and Research, 10(2) pp. 135–172.

ODPM (OFFICE OF THE DEPUTY PRIME MINISTER) TUROK, I. (1989) Evaluation and understanding in
(2002) Best Value Performance Indicators local economic policy, Urban Studies, 26,
2002/2003. London: ODPM. pp. 587–606.
OECD (ORGANISATION FOR ECONOMIC CO-OPER- WONG, C. (2002) Developing indicators to inform
ATION AND DEVELOPMENT) (2000) Urban Re- local economic development in England, Ur-
naissance: Belfast’s Lessons for Policy and ban Studies, 39(10), pp. 1833–1863.
Partnership. Paris: OECD. YIN, R. K. (1984) Case Study Research: Design
SUDMAN, S. (1976) Applied Sampling. New York: and Methods. London: Sage.
Academic Press.