How to measure the IMPACT of major events
Major events – from arts festivals to sporting spectaculars – all have impacts beyond bums on seats. They create broader economic, social and cultural impact on the cities and towns that host them. Identifying the extent of that impact is a critical role of market and social research, collecting data and conducting critical analysis to allow funding partners, key stakeholders and the community to best understand the value of such festivals and spectaculars.
Professor Barry Burgan, Deputy Dean at Bond University, is a leading authority on measuring the impact of majors events. He provides some insight into where the impact from events come from, and how to measure it, economically, culturally and beyond.
…
Economic impact is the most common form of event evaluation. At the start it should be recognised as a relevant, but only a partial view of the outcomes associated with an event. Economic impact relates (usually) to the short term impact on jobs and incomes. Clearly cultural and sporting events have a much broader contribution than this—they impact on general health and well-being, on creativity, and regional brand and image etc etc. Studies (and the literature) have focused on economic impact, but there is general agreement that the other aspects are in many cases of more interest. Economic impact is an important part, but only a part, of the evaluation in events that have tourism attraction as a major objective.
One of the constraints of evaluating events in the broader context is the difficulty in defining the outcomes quantitatively, but that does not reduce the need for more comprehensive understanding of the event space (Mair and Whitford (2013), Sydney Morning Herald (2016)).
Because evaluations focus on economic impact assessment it is important to consider some of the issues in the assessment and reporting. Firstly there needs to be a clear distinction between the economic activity associated with an event and the economic impact of an event.
Economic activity is a measure of the aggregate size of an event and associated expenditures — the total that people who attend, spend on tickets, on merchandise, on food and beverages (in and outside the event). The source of data for evaluations is data on the characteristics of the event (attendance, tickets by postcode, financials) and surveys of attendees (including questions such as where they came from, whether as visitors they came for the event, extended their stay, how much they spent and on what). This is of course subject to some sample variation.
Economic impact is the estimate of new or created contribution to income (not expenditure) and jobs in the region under consideration. This firstly allows for transfer affects and assumes for example that local attendees would mainly spend their money on some other local activity if the event was not held, that some visitors who attend the event would have come to the region anyway, that government funding provided would have been spent on other activities — and as such these expenditures are transferred rather than new or created.
The economic impact is made of direct impacts (the incomes and jobs created in servicing the needs of those attending the event based on the new expenditure) and the induced impact (after allowing for flow on or what are sometimes called multiplier effects).
There remains some debate in the literature (and some misunderstanding in practice) about the appropriate models to be used for assessing economic impact. Some event evaluations stop at estimating created expenditure — but of course not all expenditure is equal in the way it impacts on the region. Interestingly the way it seems to be going is that this is more used for smaller and/or regional events — yet it is in those cases that a lot of the expenditure actually does not result in outcomes in the region (because it is spent on imported food and drink, or external artists etc).
The majority of event analyses use Input-Output models to estimate both the direct and induced impacts. Such models are economy wide models giving underling industry ratios (income and jobs to expenditure ) to allow calculation of direct effects and industry linkages which can be used to estimate the induced or flow on effects. The use of IO models has been cautioned in both the literature and in some audits as overstating the outcomes. Early in the debate, this criticism was based on the use of incorrect and overstating multipliers and was valid. Regional multipliers for value added (or income) are generally around 0.4–0.6 on average, and total multipliers (direct and induced) are around 0.9–1.2.
The actual outcomes depend on the nature of the expenditure and the size of the region. The smaller the region, the lower the induced impact as there is more leakage to imports. So, it is important to use a model for the region and not apply broader level (especially national) multipliers — which is a common mistake. But, if a report is reviewed and one is looking at a state or sub-state level then multipliers much different to the above should be questioned. It should also be noted that some studies include only partial multipliers which will make them lower again.
Secondly it is broadly acknowledged that induced effects based on IO tables incorporate some simplistic assumptions — the main one being that industries can expand and contract without constraint, and that there are constant returns to scale. Therefore the alternative economic impact model that is often proposed in the literature (as being superior) is a Computable General Equilibrium model (Li). This model adds in constraints in terms of supply, can allow for non-constant returns to scale and provides a little more information (depending on the model). In my view, if you are looking at the national impacts of a major event it is critical that you use a CGE model as there are possible labour market constraints, balance of trade constraints etc.
However, at the regional and state level this argument is logically not as strong — as many of the supply constraints are less relevant, particularly in a long run analysis. While some of the events economic impact literature suggests that IO models generate higher results than CGE at the state level — this is dependent on the assumed parameters and closures that are used in the model — and experiences from other studies show that long run closures of CGE models at the state level tend to produce similar results to properly used IO models. In addition there is little empirical evidence for the parameters applied in a CGE model at the regional level and so it is heavily assumption laden. Therefore, IO models used properly can be considered to be sufficient for the purpose of assessing the impact at a regional level.
So all of this detail is about the technical aspects of economic impact modelling . My conclusion is that the economic impact of an event is a valid consideration in the evaluation of outcomes of an event, but is only one aspect. Increasingly we are focusing in our research on understanding the broader benefits and costs in a benefit cost perspective. This includes longer term economic impacts, social well being outcomes etc. The economic impact is usually produced as different paradigm base, but can be integrated into the cost benefit analysis (Burgan and Mules) (2001). It must be recognised that a full cost benefit analysis would require assessment of some costs of an event that are not considered in economic impact often because the evidence does not exist (e.g. impact of congestion in public spaces) (Gong (2012), Mueller Institute (2016))
Events analysed for their economic impact will always have a positive outcome (as will any other net expenditure on projects)—the issue is that the extent of the impact and whether it is “sufficient” as a partial return for investment (of course along with the other benefits). Rigorous studies ensure a defensible and reliable methodology—and should explain it. Unfortunately, there are many studies that present/focus on the results and not on the assumptions — which do not allow others to understand the approach used and the impact this might have on the results.
(Originally posted via Square Holes’ ezine:
Square Holes 73: How to measure the IMPACT of major events)
References:
Spectrum , Sydney Morning Herald, “40 Years of the Sydney Festival”, 02 Jan 2016
Mulier Institute & Utrecht University, “Creating social impact with sport events”, July 2016
