Effective forecasting during and beyond the pandemic

You’d be right to be sceptical about the accuracy of any forecast that claims with certainty that events will unfold along a single path resulting in a specific set of circumstances.  In hindsight, how we got from A to B is seemingly obvious. and we tell ourselves that it couldn’t have happened any other way, but when we are yet to arrive at B the path is far less clear and any number of events can throw us off course. 

Effective forecasting is not saying that this is what will happen, but what might happen based upon the likelihood of a series of events playing out in a specific way at a specific time under specific conditions – the more events you are having to take account of, the less likely any single outcome will occur. 

“The trick is to work out what is useful information to feed into your forecast (the signal) and what is merely a distraction (the noise)”

David Reece

What is forecasting?

Forecasting is defined as “a statement of what is judged likely to happen in the future, based on information you have now.” There are two key points here.

  • Firstly, the phrase, “likely to happen,” which begs the question what does likely mean? More than 50% chance? Even a 51% chance of something happening isn’t great odds.
  • The second point relates to “information you have now,” which is the key behind any forecast modelling. The trick is to work out what is useful information to feed into your forecast (the signal) and what is merely a distraction (the noise).

Let’s take a quick detour into the world of polling and use the 2016 US Election as an example. In the run up to the Election, most polls had the Democratic nominee, Hilary Clinton with a much greater chance of winning the Presidency over the Republican challenger, Donald Trump. The poll aggregator website, FiveThirtyEight, run by forecasting expert Nate Silver and his team, built a forecasting model using a vast array of sources to predict the outcome. The model gave Hilary Clinton a 71.4% chance of victory compared to 28.6% chance of victory for Donald Trump – that’s basically a 1 in 4 chance of Trump taking the Whitehouse – and as we saw, that time round, Trump secured victory.

US Election Prediction from FiveThirtyEight in the 2016 US Presidential Election

But the only way to know how accurate that forecast from FiveThirtyEight was, would be to run the Election 4 more times and see if Clinton won in any of those occasions.

Of course, different information might also have produced a different forecast, in which Trump had a much greater chance of winning, but the model was based on the information it had now, which was constantly updated as new information was gathered.

So, if forecasting doesn’t tell you with certainty what is going to happen, then what is the point of forecasting at all? Shouldn’t we just plan for every eventuality and make decisions in response to events as they unfold? Well, yes, you could do that, and in the case of something like an election where 2 candidates are in with a chance of winning then it makes sense to plan for 2 possible outcomes, but when there are many more factors at play with any number of possible outcomes, having an informed idea of how things might turn out makes it much easier to plan for the most likely scenarios.

Guiding principles: the 5 Ps of Forecasting

To be effective, forecasting should state the likelihood of certain events happening under certain circumstances – it’s no good having a forecast which states for Event A you’re going to sell many tickets whereas for Event B you’re going to sell fewer tickets.  Forecasts should be quantifiable to be useful so we should take care to work out what we mean when we say ‘many’. How many tickets? What type of tickets? At what price? Over what timeframe?

We have devised the 5 Ps of forecasting to act as guiding principles to help make your forecasts more effective.  Think of them as a series of questions you need to ask yourself as you build your forecasting model.  

The Five Ps of Forecasting

1. Purpose

Effective forecasting is about being really clear what you are trying to achieve with your forecasting and what you’re trying to model – whether that’s:

  • sales, which can be broken down into sales for specific segments
  • income and again, which type, ticket income, secondary income, donation income etc.
  • or something else entirely like membership conversion rates. 

All of which are important, all of which are related, and an overall business model will take into account all of these measures.

2. Past and Present Evidence

Any forecasting model is only as good as the information you provide it with, which for the purposes of forecasting the examples above, can be split into three broad categories:

  1. Historic data
  2. Market research
  3. Industry trends

The best forecasting models are able to use all three of these types to improve their accuracy – past data provides the baseline, market research gives you an indication of how people are feeling right now, and industry trends add context.

3. Precision

Firstly, getting your data clean and correct is so important.  After all your forecast is only as good as the data you put in – garbage in equals garbage out. 

This means taking time to code your data, ensuring that both your objective variables (like day of week or time of year) as well as your subjective variables (like genre and popularity of an artist or title), can be used to establish suitable comparators for what you want to forecast in future. 

External factors might also be at play, which although not something you have direct control over will need to be taken into account as these might explain why a usually strong-selling Saturday in June underperforms due to heavy rain that day, or due to significant competition. 

4. Practicality

This is closely linked to precision, but the question here is about what you can reasonably be expected to forecast and over what timeframe?  The further you try to forecast into the future the greater the uncertainty and therefore the inaccuracy of your forecast.  Take the weather, for example; in the short-term the forecast is precise as meteorologists have all the data they need, but start going beyond 5 days and the number of possible outcomes increases dramatically.  Minor variances in forecasts become magnified over time. 

In the current climate where there is more uncertainty than normal, trying to predict too far into the future becomes increasingly challenging.  Forecasting might only be practical over a short period of time. 

5. Process

There are going to be lots of unknowns and a lack of data so you need to keep refreshing your forecasts as new information becomes available.  This is an iterative process – review, refine, reforecast.  As Nate Silver puts it in The Signal and the Noise, “If you have reason to think that yesterday’s forecast was wrong, there is no glory in sticking to it.” It’s still worth the effort though. Without a forecasting model then you’re flying blind, making decisions based on instinct and intuition, which are as often wrong as they are right.

“Without a forecasting model then you’re flying blind”

David Reece

Forecasting for 2021

Of course, 2021 is not going to look like any previous year and continuing restrictions will result in artificially less capacity than usual so you are already operating in unknown territory.  However, the purpose of forecasting for 2021 is still to estimate sales and income for a specific exhibition, event, production etc. to inform planning, budgets and target setting. 

Looking at past data establishes your starting point, e.g. under historic conditions for these types of event, performance, exhibition etc. sales and income were X, and looking at present data from market research and industry trends informs how you need to make adjustments.  For example, PwC are projecting a 6% drop in GDP going into 2021, which could be used as a top-level indication of change in demand across the board before making further adjustments. 

The degree of precision is determined in part by the quality of the data available to you, but let’s break down what you’re forecasting into smaller chunks, look at particular segments of your audience and forecast how each of these groups might be affected in future as far as practicality allows.

Audience Segments

Take your regulars – usually members, season pass holders, subscribers and highly frequent. They are generally more likely to respond to any surveys you have sent out during this time so sentiment surveys provide an idea of how many intend to visit in 2021 and you can then compare that with their past behaviour.  Keep in mind that surveys can tell you more or less how many customers you are going to lose but you need to multiply that by their frequency and party size to estimate the drop in tickets, and by their average annual spend to estimate the loss in income.  Likewise, a similar approach can be taken with infrequents, who generally have a much higher churn rate, and because of concerns and restrictions about travelling expect the focus to shift towards those who are local. 

Families and seniors are reported to be more sensitive to risk due to Covid-19 so expect less reliance on these groups in the short-term, but those that live locally are more likely to return. A segment that is still likely to take a big hit is clearly international visitors, where you’ll want to use other data as proxy, e.g. air travel projections.  You could also consider groups and travel trade as a distinct segment, as well as younger adults and make predictions about what proportion are likely to return.

Having established your forecasting model, remember this is part of an iterative process, which means making adjustments to your base scenario as circumstances change and new information becomes available, e.g. on-going sentiment trackers, economic forecasts etc. 

Adaptability is key

As we think about the various stages of opening that we all find ourselves in, and as we look toward the future, it’s important to remember that adaptability is key. In the UK, for the rest of 2020, we could assume a very limited vaccine rollout, limited capacity restrictions, social distancing, travel restrictions, local lockdowns, and a possible second wave. But what about early 2021? There’s Brexit, wider vaccine rollout, the impact of winter flu, continued lockdowns, limited demand.  And by summer 2021, who knows?  A widespread vaccine roll-out, some kind of Covid passport, increase in tourism, easing of restrictions, increased capacity?  All of which are going to have an impact on both supply (what you can put on) and demand (what people are willing to buy) and as a result what you should do in response. 

In summary, effective forecasting takes time and effort, and once you’ve set your forecasts for the year you can’t even sit back and relax, but the reward for that time and effort is informed planning, better target setting with realistic goals and evidence-based strategy. 

David Reece
David Reece

Deputy CEO David Reece has led our team of consultants and analysts for more than a decade, working with cultural organisations and attractions globally

Leave a Reply

Your email address will not be published. Required fields are marked *

Say hello

To set up a conversation without obligation, please fill in the form below and we'll get in touch.