Market research: When might a Surveymonkey be bananas?

When I was the Marketing Director of a producing theatre, I used to dream that some day someone would say to me – “I’ve got thoughts on the artwork for this show, but as you’re the professional I think I’ll trust your judgement and keep my opinions to myself”.

The reality is that, as any arts marketer will tell you, EVERYONE’s an arts marketer – or they think they are. It can be  frustrating to listen to people conflate what they personally like or dislike, with what they think will engage an audience.

In turn, there’s also an expectation that every arts marketer is also a market researcher. Though there are many talented researchers in the sector, it isn’t a universal skill.

When SurveyMonkey may be bananas

Market research is an important part of the response to heightened instability in the forces which affect demand, motivation and intent to engage with the arts.

DIY market research is widespread, as barriers to entry are low and low-cost surveying options like the feature-rich and market-leading platform Surveymonkey are readily available. Surely it’s straightforward to ask a question, and then collect responses on a five point scale from “strongly agree” to “strongly disagree”?

It’s often impractical for reasons of cost and time to seek professional help with market research but, as both a commissioner and provider of research, I know a good market researcher adds great value. 

What then are the things to focus on when undertaking market research in-house?

Get the question design right 

One of the frustrations with surveys is realising, after everyone has completed it, that you asked the wrong questions. This sounds improbable but is extremely common. You ask people which ice cream flavour they prefer and, when you see the results, you realise what you really wanted to know was whether they prefer ice cream or a savoury snack. And might that change at different points in the visit? 

Think through what you really want to find out from market research and what supplementary questions you should ask in particular circumstances. As a general rule, a long survey is a bad survey, however it’s conducted.

Don’t use a survey to ask what you already know

To extend the ice cream example, if a venue wants to know the most popular ice cream, it probably already has the answer in its ePOS (electronic point of sale system).

Likewise, asking audiences their favourite genre of performance to assist artistic programming will only tell you their ‘stated preference’. Often their real behaviour, or ‘revealed preference’, as captured in your ticketing system is more valuable.

Asking respondents to tell you things you already know is a sure-fire way to increase survey abandonment.

When respondents second-guess why they’re being asked

If you ask a direct question about how much a respondent is willing to pay, you can discard the results. Respondents know why you’re asking and will consciously lowball their answers. Market research for pricing is a highly specialist area – ask for help.

Separate beliefs from behaviours 

It’s common to ask questions like: “When do you next plan to come to the theatre?” This will tell you what the respondent believes. Follow it up with a question like: “Do you have a booking for an event with us in the future?”, which will reveal their behaviour. The two questions may yield very different results, which is useful information in itself.

Be aware of your respondent profile

If surveying your own database, two things are vital. First, what is the profile of that database in terms of purchasing behaviour, location, age, social class? Second, what is the profile of the responses? How does it align or diverge from the overall population?

Ask questions that allow you to profile and qualify the responses. Organisations with research capacity may then weight the responses accordingly, usually best done by an expert as it can substantially change the results.

When Baker Richards co-led the national Culture Restart audience research programme, we understood the profile of respondents – predominantly older, more highly engaged cultural attendees (as is the case with nearly all arts organisations’ surveys).

Ask the right people

Organisations with large contacts databases often don’t reach beyond their own data. Rather than preaching to the converted, they really need to find out what people who don’t come to church think.  

Market research agencies have ways of accessing panels of off-database respondents and know when such a panel should be census-balanced or targeted at certain types of respondents. Market research is most useful when it covers the whole potential market, not just the people on your database.

Choose the right method

Organisations often use quantitative research – surveys with scores, drop downs, Likert scales and the like – when a qualitative survey might be more valuable and enlightening – such as when you want to understand attitudes, motivations and feelings, or to test out how patrons interact with a new product or service. 

Qualitative research, which tends to use data from conversations rather than surveys, is best undertaken by a skilled professional. It’s hard to correct for failings in the process – such as one voice dominating and swaying all the members of a focus group.

It’s common to undertake a survey once and fail to recognise that responses may change over time. How often do you need to ask a question before the result becomes reliable?

Misleading data is worse than no data

In World War II, the US Center for Naval Analyses, concerned that bombers were regularly being shot down over Germany, analysed the distribution of bullet holes on returning planes to understand where they were most vulnerable.  

Damage was concentrated on the wings and upper body of the planes, so plans were made to strengthen these areas. In fact the reverse was true, as they were only looking at planes that returned to base. What their analysis was actually revealing was where a plane could take heavy fire and still make it home. It’s always useful to have someone external to your situation challenge your interpretation of data to see if it holds up.

With great uncertainty in the market, the sector will undoubtedly benefit from rigorous market research to underpin both its tactics and strategy. It’s vital to avoid research pitfalls – when in doubt, should you ask an expert for help? I’d “strongly agree”. 

Robin Cantrill-Fenwick
Robin Cantrill-Fenwick

Robin is Chief Executive of Baker Richards

Leave a Reply

Your email address will not be published. Required fields are marked *

Say hello

To set up a conversation without obligation, please fill in the form below and we'll get in touch.