Now that you have described what you want to measure, you can begin to design the methods you will use to capture these.
The next stage is to plan how you are going to capture the hard and rich data, from both inside and outside the event.
This image shows some of the methods you may want to think about.
Hard Data - What
The gathering of hard data should be aligned and hard-wired into your event planning and logistics. Some tips are:
- Keep a spreadsheet of artists and staff involved in the event and complete this with both projected and actual figures.
- Use the projected figures you have indicated to funders and partners and which you have built into your risk management and financial planning (for example: box office projections).
- Build monitoring of attendance and engagement into your contracts and agreements with artists and workshop leaders.
- Ensure the production or site manager is responsible for monitoring audience numbers – build this into your risk assessment process.
- Have a nominated team member or volunteer to monitor the press and media coverage of the event including social media hits and engagements.
Hard Data - Who
If part of your success criteria is engaging with people from a specific demographic or geographic group you should build this into your design for data capture. Many funders will require this but it is also vital information for you to understand who you are reaching – and who you are not. You can then respond to this creatively and practically in your planning for future events.
Remember to consider what hard data you need for each of the ‘who’ that you plan to engage with.
This could include:
You might want to make use of the following tools:
- Facebook and Twitter have in-built analytics
- Most box office and ticketing systems can also give you information useful for analysis (for example post-codes)
- For free events or for more complex information you should build data collection into your survey activity.
This demographic data can help you compare against general demographic information for residents and visitors.
If you repeat the survey each year then you will also be able to identify audience trends and planned or unexpected changes in audience profile.
Build your survey directly into Google forms then the data will be automatically stored into a spread-sheet for easier manipulation into simple but powerful visuals.
Collecting post-codes and storing them in a spreadsheet enables you to make use of mapping software to see exactly where the impact of your event is felt.
It can also be mapped onto other mapped data so that you can build up a narrative about who your audiences are – particularly in understanding areas of low cultural engagement (by comparing with the audience agency website) and areas of multiple deprivation (by comparing with government open data).
Rich data helps you to understand how the event was experienced by the people you engaged with.
- Social media is a useful tool for understanding your qualitative ‘rich’ data.
- Surveys are a very effective tool for capturing rich data. You should design your questions in line with the effect you want to identify.
- Imagine yourself in their situation – what would you want to share about an experience?
Designing your survey
Surveys are one of the most powerful ways of generating a large quantity of both hard and rich data about the people engaging with and experiencing your event.
- Google forms provide a free platform which you can customise with your own logo and visual design. Information is stored directly into google sheets for easy manipulation into data visualisation and analysis.
- The survey data is also date and time coded so you can see when people submitted their form.
- Use this online system for all aspects of your surveys – audience, artists and participants.
- You may want to incentivise participation through exclusive offers and premium services.
- Set a target for your survey respondents that you are comfortable with. For most events between 250 and 500 should give you a good indication but the more the better.
When developing your questions think about:
- Provide a strong message which explains why evaluation is important– people like to know that their participation will be useful and valuable.
- Only ask the questions that you need to ask – short is sweet.
- Only use surveys for information that you cant answer through other methods – less is more.
- Map your questions against your qualitative outcomes – do they tell you all you need to know?
- Do you need to include options so you can drill down into data for each event?
- Have more qualitative than quantitative questions – show people you care what they think of the event.
- Have at least one open question to enable people to be creative – it should feel like a conversation not a check-list.
- The tone should be the same as the event’s general communication campaign – be friendly.
- Imagine asking the questions to a range of people yourself – are they comfortable to ask?
- Consider when and how people might engage with the survey - are you capturing anticipation before an event or their assessment afterwards?
- Are they likely to be in a group at an event talking to an interviewer or encounter this online, possibly alone?
- What format do you need the information in order to tell the story you want about the event?
There are a number of different ways to phrase questions, which will enable you to show and share the rich data in a number of ways including closed choices such as multiple-choice, check-boxes, linear scale and drop-down menu; and open choices such as short descriptions and paragraphs.
Closed questions quantify answers so they can be shared as graphs and charts.
Open questions will provide you with a richer more personal impression of your festival. Personal testimonies and quotes can be used to enrich your evaluation. For Hastings we asked respondents to provide one word to describe their experience of the festival. We could then put this into the Wordle app in order to paint a word picture of the event. These can be a simple but powerful tool in assessing the experiential impact of your festival.
Respondents can interact with the survey in two ways:
- Live– through a conversation with a volunteer who can fill in the survey online.
- Online – via links provided on social media and web platforms.
The most effective campaign will combine both methods – you should never rely on self-directed online surveys alone.
For the ‘Live’ surveys,
- Ensure that volunteers feel confident about the online survey form. Provide training and ensure they have the right tools for the job. They will need a smartphone or pad with sufficient Wi-Fi or data coverage.
- Ensure volunteers are aware of any ethical issues and ensure that full consent is provided by respondents.
- Make sure volunteers are visibly part of the event team – you could provide them with t-shirts or a lanyard and event ID.
- Pair volunteers up at events so that they have support – and can chose to gamify the survey collection.
- Brief volunteers to use their judgement to chose a range of different people to try and be as representative of the audience as they can.
- Ask volunteers to also capture photographs and films of the event with their smart-phone / pad.
- Ensure that volunteers make themselves known to the on site event organiser at the beginning and end of their shift. This should form part of our event risk assessment.
- Always make sure that they have an event mobile number to call if they are concerned and that you have details of their name, address and who should be contacted in an emergency. This should form part of your safety procedures for all staff and volunteers.
For the On-line surveys,
- Ensure that the survey is visible on your web-site and across all your social media platforms.
- Keep reminding your target respondents of the value of the evaluation – and reminding them of the importance of the survey.
- Incentivise and encourage participation through competitions and premium offers.
- Send out the survey to your mailing list – and if you don’t have one then consider setting one up. MailChimp offers a free subscription for lists of under 2000.
Social media can also be a powerful tool in capturing and conveying an experiential narrative of your event. Remember the following tips:
- Integrate your evaluation plans into your whole communication strategy and campaign.
- Have a clear lead for social media will be responsible for monitoring and evaluating communication, before, during and after the event.
- Take a lead in social media– make sure that you have a clear identity on all platforms that will be relevant to who you want to engage with.
- Make sure that there is a simple and memorable hashtag (#) used on all platforms.
- Ensure this # is visible on print, online and physically at the events.
- Incentivise and encourage social media activity through competitions and premium offers.
- Make use of the in-built analytics for platforms.
- Use online platforms to ‘storify’ the event.
Social Media Analytics:
Arts and culture evaluation toolkit