How will the 2014 Commonwealth Games impact on Glasgow’s health, and how will we know?

McCartney, Gerry (2010) How will the 2014 Commonwealth Games impact on Glasgow’s health, and how will we know? MD thesis, University of Glasgow.

Abstract

Context:

The City of Glasgow is the most deprived city in the UK. Over the last 30 years the mortality rate has become an outlier when compared with the rest of the UK and rest of Europe. Even when the effects of poverty and deprivation are taken into account, the mortality and ill-health in the population is higher than would be expected (the ‘Glasgow effect’). This has led to a series of health, social and economic interventions in the City designed to change these trends and improve outcomes for Glasgwegians, but with limited success.

In 2007 the City placed a bid to host the 2014 Commonwealth Games. One of the stated aims of this bid was the potential for the Games to generate a positive legacy in terms of health, social and economic outcomes.

Glasgow was successful in its bid to host the Games and there have since been concerted efforts by the Scottish Government and Glasgow City Council to ensure that the potential of the Games to generate a positive legacy is fully realised.
Aims:

The aim of this thesis is to consider what the impact of the Games will be on the health of Glasgow, and how the impact can be evaluated.

Methods:

There are five sections to this thesis: an analysis of the Scottish Government’s theory of how the event will generate impacts; a systematic review of the literature; a health impact assessment of the Games; generation of an evaluation framework for the event; and a synthesis which considers what the net impact of the Games is likely to be.

A systematic review was undertaken to determine the evidence for these impacts from previous major multi-sport events (1978-2008). It involved a search of the following databases (ASSIA, BHI, Cochrane, Econlit, EMBASE, ERIC, HMIC, IBSS, MEDLINE/Pre-MEDLINE, PsycINFO, Sociological Abstracts, Sportdiscus, Web of Knowledge, Worldwide Political Science Abstracts) as well as a grey literature search for events occurring between 1978 and 2008. Studies of any design which assessed the health and socio-economic impacts of major multi-sport events on the host population were included. Studies using exclusively estimated data rather than actual data; investigating host population support for an event; media portrayals of host cities; or describing new physical infrastructure were excluded. Studies were selected and critically appraised by two independent reviewers. All data extraction was checked by a second reviewer. The narrative synthesis used ESRC guidelines.

The analysis of the Scottish Government’s theory was based on the consultation document published shortly after the announcement that Glasgow had won the right to host the Games. The document was dissected to expose the proposed mechanisms through which change was thought likely to occur and the expected legacy outcomes. This formed a ‘theory of change’: an explicit chain between the intervention and outcome along which impacts are thought to occur.

A Health Impact Assessment (HIA) methodology was then used to identify the likely mechanisms of impact for the 2014 Games, including those proposed by the Scottish Government. The HIA was not used to predict the net impact of the Games but instead to produce over 150 recommendations for City Council decision-makers detailing how the positive impacts might be maximised, and the negative impacts mitigated.

Following this, an analysis of how better quality evidence might be gathered from the 2014 Games was detailed using and critiquing the current approaches to the evaluation of complex social interventions.

The last section of the thesis draws upon the evidence collected throughout the thesis to predict the likely impacts of the 2014 Games and then consider how academics might benefit from using a ‘critical pathways’ approach to informing the policy-making process in the future.

Results:

The systematic review included 54 studies. The study quality was poor with 69% of studies repeat cross-sectional and 85% of quantitative studies assessed as being below 2+ on the Health Development Agency appraisal scale, often because of a lack of comparison group. Five studies reported health outcomes including suicide, and hospital presentations for paediatrics, children with asthma, and illicit-drug related problems. The data did not indicate clear negative or positive health impacts. The most frequently reported outcomes were economic (18 studies) which were similar enough to perform a narrative synthesis. The overall impact on economic growth and employment was unclear. Two thirds of the economic studies reported increased economic growth or employment immediately after the event but all of these used some estimated data in their models, failed to account for opportunity costs or examined only short-term impacts. In contrast, the economic studies that did not use estimated data reported mixed impacts or a decrease in employment and economic growth. The transport outcomes were similar enough to synthesise. The synthesis showed that event-related interventions including restricted car use and public transport promotion were associated with significant short-term reductions in traffic volume, congestion or pollution in 4 out of 5 cities.

The Scottish Government’s theory of change for the Commonwealth Games was analysed and was found to contain three separate types of mechanism. Direct impacts are those arising from actions intrinsic to the hosting of the Games and which are unlikely to be replicated in their absence. This includes the development of Games-related infrastructure and the potential increase in tourism resulting from media exposure during the event. Project impacts are those arising from interventions associated with the Games, where the intervention could be implemented in another context or time without the backdrop of the event. This might include a physical activity programme branded as being part of the Games intervention. The other category of mechanism of action is catalytic impacts. These are the additional impacts expected to result from existing programmes, policies and projects in the presence of the Games. The Government document described a range of outcomes for each of its themes (healthier, wealthier and fairer, smarter, greener, safer and stronger) using these three types of mechanism.

The HIA did not make a prediction of the health or socioeconomic impact of the Games because it formed part of the corporate processes of the City council, and a prediction of the impacts was not one of the council’s objectives for the HIA. Instead, the HIA made over 150 recommendations in an attempt to maximise the positive benefits and mitigate the negative impacts. The HIA concluded that the Games will most likely influence health most through impacts on the economy, civic pride, engagement in decision-making, the provision of new infrastructure and participation in cultural events. A range of recommendations were made reflecting the available evidence and the collective wisdom of the public and participants in the HIA process. These included: a recommendation that a higher proportion of the new housing built to accommodate athletes during the event should be made available as social housing; transport policies before, during and after the event should promote active transport and make public transport more affordable and accessible; and increased public involvement in the decision-making processes about the use of the new sports infrastructure after the event. Further evaluation is required to assess how successful the HIA process was in terms of community participation and validity, on the question of how effective the HIA was at influencing policy-makers, and on whether the impacts discussed in the HIA were realised.

Evaluation of the impact of the 2014 Games is likely to be challenging because of the complex nature of the intervention. Three types of mechanism of impact were identified: project effects, direct effects and catalytic effects. The project effects arise from specific projects or programmes that are undertaken in the context of the Games, but which could be implemented in the absence of such an event. The evaluation of these could be improved if either a prospective cohort study (where the participants in the projects can be predicted in advance) or retrospective cohort study (where the participants will only be known after they have taken part) is arranged. This will require to be undertaken in combination with qualitative studies and the creation of a theory of change to understand why any such project effects are (or are not) seen. The direct impacts of the Games, that is the impacts that occur as a direct result of playing host (such as the impact on tourism), require a different approach to achieve a quality evaluation. First, a theory of change to identify the critical pathways in generating impacts should be elucidated. Next, a combination of a cohort study and an ecological study (using routine data and a series of comparison areas identified in advance), should be used to identify the attributable impacts of the Games. Qualitative work alongside these studies will be required to understand why the impacts occur (or not). For the economic impacts specifically, an ecological design or economic modelling should be performed using routine statistical data (rather than estimates) and taking account of the opportunity costs. The catalytic impacts are particularly difficult to evaluate as it is not yet clear what these impacts might be. These potential impacts will require being identified using regular qualitative work with key individuals within the public and private sector in Glasgow, and this information will then need to be used to design quantitative studies to test these hypotheses.

The synthesis discussed whether or not the Games could legitimately be described as a health improvement intervention. It found that some of the critical steps in the intervention were very similar to the tried and tested mechanisms used over many decades in the West of Scotland in attempt to improve the health and social conditions (economic growth and improved environment), without success. The ability of the Games to impact on the other critical steps (sports participation, increased volunteering and increased pride) is not supported by the evidence from previous events, and it is difficult to see what is different about the plans for the 2014 Games that might generate a different result. It was therefore concluded that the 2014 Games are unlikely to be an effective health improvement, and are unlikely to generate the plethora of social and economic benefits that pepper the bid document and legacy plans.

Discussion:

There are high expectations from Government that the 2014 Games will deliver a plethora of health and social benefits. The evidence from previous events is of poor quality, and there is an absence of evidence of positive impacts occurring. Given that a publication bias towards positive impacts is expected, it is unlikely that large positive health or socioeconomic benefits have occurred from major multi-sport events in the last 30 years.

Health impact assessment can be used as an effective method of engaging the public and can be used to inform policy-making with evidence. Although the HIA did not predict the net overall impact of the Games, it is possible to make evidence-informed recommendations that are likely to maximise the potential for positive impacts and minimise the potential for negative impacts.

The quality of evidence on the impacts of major sports events could be improved if a theory of change evaluation framework was applied to the event and if this was used to design a series of qualitative, cohort and ecological studies with appropriate comparison groups. However, it is unlikely that the 2014 Games will have a large positive impact on the health of Glaswegians or on socioeconomic outcomes because there is little evidence that the likely critical pathways have been successfully used in generating positive impacts despite similar attempts in the past. Some critics of major sports events have also made a plausible case for their being important negative consequences from playing host.

The strengths of the thesis include the use of robust methodologies for the systematic review and health impact assessment, and the innovative use of a critical pathways approach for estimating whether or not the net impact of the Games will be positive.

The weaknesses of the thesis include the reliance on the Government’s published work to discern the theory of change; the 34 studies that could not be obtained for possible inclusion in the systematic review; the limited evidence base upon which to make recommendations in the HIA; and the reliance on an accurate theory of change to predict the net impact of the Games, including the absence of emergent impacts from the complex Glaswegian context.

Conclusions:
The 2014 Games are unlikely to generate a large positive impact for health or the socioeconomic determinants of health. There is potential for unintended negative consequences to occur. The impacts of the Games are most likely to be optimised if the HIA recommendations are acted upon by decision-makers, and these impacts will only be accurately known if there is work to improve the quality of the evaluation. It is possible that the marketing of the Games as an intervention for health and social improvement might deflect attention from more important determinants of health in the city. In that vein, it may be more reasonable to make few other claims for the Games than that it will provide public entertainment and a festival for the population, and to minimise the opportunity costs that the Games will generate.

• Item Type: Thesis (MD)
• Qualification Level: Doctoral
• Keywords: Glasgow, Commonwealth Games, Systematic review, Health Impact Assessment, Evaluation.
• Subjects: R Medicine > RA Public aspects of medicine > RA0421 Public health. Hygiene. Preventive Medicine; H Social Sciences > H Social Sciences (General)
• Colleges/Schools: College of Medical Veterinary and Life Sciences > School of Health & Wellbeing
• Supervisor's Name: Hanlon, Professor Phil and Bond, Professor Lyndal
• Date of Award: 2010
• Depositing User: Dr Gerry McCartney
• Unique ID: glathesis:2010-2136
• Copyright: Copyright of this thesis is held by the author.
• Date Deposited: 19 May 2011
• Last Modified: 10 Dec 2012 13:51
• URI: https://theses.gla.ac.uk/id/eprint/2136