Why it’s important to make space for unexpected outcomes
Even the best-laid plans are subject to change. Setting clear goals and metrics is a vital part of evaluation but how can you plan to understand any unexpected outcomes from your work? Here, three experts share their tips for expansive approaches to evaluation, avoiding a restrictive “success or failure” mindset, and learning how to benefit from your work’s unforeseen twists and turns.
“Open it up!” – Avoiding bias in evaluation
Lara Ratnaraja, a cultural consultant specialising in diversity, collaboration and cultural policy, recommends an approach that is “rigorous” – capturing hard data about specific goals that have been set in advance – and “expansive” in how that data is interpreted, and used to expand your knowledge.
Tip 1: Avoid making assumptions about your audience
Biases are assumptions, Lara says, and making assumptions can close you off from learning. “If you don’t start evaluation with bias, then unexpected outcomes will open up your perspective.”
These biases might come in the form of assumptions about your audience. Perhaps you assume only a certain demographic of people might be interested in your work, or perhaps you do the opposite, and don’t take a more informed and specific enough approach to thinking about who your audience might be.
“I do believe there’s an audience for everything, but be realistic,” Lara advises. “If you say, ‘I want to see more diversity in my audience’, but your evaluation doesn’t show you that, then it’s a negative, right? But that is a bias in what you think diversity is. Maybe you’re in an area which is more monocultural, so think about other factors such as barriers around class.” Unexpected, or even unintended outcomes, can be valuable learning opportunities, provided you use them to spark further questions: “Open up your mindset, don’t close it!”
Proactively avoid biases by being genuinely curious about your audience’s experiences of the work, and the wider context your work exists within. “We don’t often ask, ‘How did the space feel?’, ‘Did you feel welcome?’”
Tip 2: Reflect on your motivations
Without overlooking any responsibilities to funders and stakeholders, Lara encourages you reflect on your own reasons for evaluation. “Think deeply about what you might get out of this,” she encourages. “If your art is a learning process, then your evaluation should be a learning process too. Who is your evaluation for? Why do you want to evaluate? How open are you to the challenges that your evaluation might give you? We can often assume there’s a fixed standard for culture, and evaluate against that. But if that’s not your standard, then how are you going to evaluate?”
Tip 3: Embrace plural perspectives
“I like it when feedback comes out of left field,” Lara says, “that means your work has motivated someone, or energised someone, and it could be positive or negative. If people feel safe to be negative, with no fear of repercussion, then it’s not a binary thing. Your work might be challenging to some people, but does that make it wrong?”
“The really glorious, unexpected outcomes are when someone sees something in your work that you haven’t seen. Be excited about that potential! Look at evaluation as collecting a plural set of perspectives – some of them are right for you, and some of them might be right for you in the future. Evaluation is iterative.”
Using creative evaluation methods to plan for the unknown
Dr Emma McDowell, Lecturer in Cultural & Creative Industries at the University of Leeds, advocates for creative and unusual evaluative methods to help you find out what you don’t already know about your work. It might sound counterintuitive, but you’ll get the best results by planning for the unexpected.
Tip 1: Ask unexpected questions in audience surveys
“For me, the unexpected will always come from the ways that people make meaning,” Emma says, emphasising the value in letting audiences tell you what was valuable about their experience, on their own terms, and beyond simple box-ticking.
“If you use a survey, make space for one question that’s a bit weird. Even something as simple as, ‘What three words best describe this show?’ Think of the kind of question that would make a statistician cringe – how do you measure that? – because there’s real value in creating a space where people can be creative in their responses.”
Tip 2: Explore mixed evaluation methods
Emma suggests an approach of “structured flexibility”, all caveated by the understanding that your evaluation process might be partially determined by various stakeholders. “It’s definitely valuable to have a clear sense of what you’re doing, and then doing what you say you’re going to do,” she says. But beyond that? “Harness your curiosity!”
The Centre for Cultural Value has a wealth of resources about alternative evaluation methods that can be combined with more established processes, such as surveys. Begin with their guide to planning a mixed-methods framework, and then explore toolkits for conducting walking interviews, amplifying diverse voices within your feedback, and anti-racist audience research.
Tip 3: Re-frame ‘failure vs success’
Avoid the blame game, and “accept that multiple things can be true,” Emma urges. “Sometimes ten people will come, and that might be a failure financially, but those ten people are the right people. That’s better than bringing in thirty people who were expecting something completely different, and now they’re annoyed!”
She recommends FailSpace, a toolkit from Leila Jancovich, David Stevenson, Lucy Wright and Malaika Cunningham, as a brilliant resource for supporting open, honest conversations about failure and what we can learn from it. It conceives failure as a broad church, rather than a mindset to take only if/when things go wrong, and offers a really helpful guiderail for embedding this attitude within your evaluation from the start.
“It’s rare for anything to be a complete failure,” Emma says, reassuringly. “And the opposite is true, too – it’s rare for anything to be a complete success, although we like to pretend that it is! Instead, it’s all about the articulation of learning.”
Digital case study: Embracing risk in Louise Orwin’s Famehungry
Famehungy, a show that takes place simultaneously in a theatre and on TikTok, grew out of performance artist Louise Orwin’s curiosities about the app’s (potentially) enormous audiences. Could a pivot to the platform supersize her career or compromise her artistic integrity?
Audiences who buy a ticket to Famehungry in a theatre will see Louise performing elaborate, repetitive routines designed to maximise her visibility within TikTok’s mysterious algorithms, or to test and critique the app’s censorial “community guidelines”. All the while, her performance is streamed live to TikTok audiences, whose comments and reactions fill the screen at the back of the stage. The show operates around a game: can Louise reach a certain number of online likes, before the hour’s up? Or, before the app kicks her off for violating those opaque “guidelines”?
“It’s all about questions of survival,” she says, “how can we live in a hyper-capitalist world and survive as an artist?”
Moving goalposts
During early works-in-progress, Famehungry had a goal of 4,000 likes. After a Fringe run in Edinburgh and two weeks at the SoHo Playhouse in New York, some performances were peaking at 72,000. During others, Louise had been banned from the app across multiple back-up accounts, or violently trolled by Neo-Nazis.
To Louise, “smashing the likes” doesn’t necessarily mark success or failure for the show. Instead, she measures her ability to keep the show reactive to, and curious about, the changeability of TikTok as a platform, and her ability to maintain balance and interactivity between online and in-person audiences. This might mean improvising, mid-show, if she hits that goal too early, or having the flexibility to facilitate a conversation between an online viewer and someone in the front row.
In a stroke of fate, the US’s TikTok ban was enforced on the same day she flew to New York to begin her run. Downloading the app on UK soil, and the use of VPNs, meant that she could make it work on a technical level, but on a societal level – “I knew I needed to address it in the show,” she says. “What does it mean to use an app with this big, Trump-endorsed stamp on it? I felt like the show was right at its edge of discomfort.”
Open to unexpected feedback
“I wanted to experiment with new ways of meeting audiences, new ways of talking to audiences,” Louise says. Even so, the amount of feedback she was able to access through TikTok came as a welcome surprise. “Not only do we get the replay of the livestream but we get a replay of the comments. I’ve been reading them every night.
“It’s actually really beautiful – the TikTok audience can comment on their experience all the way through the show, which is something that the in-person audience aren’t allowed to do in the same way. I’m able to see, yes, the work is doing what I want it to be doing. It’s creating an audience for performance art. It is an enticing, exhilarating experience, and it’s making them think.”
Not all feedback has been positive. One performance to a group of school children – i.e. TikTok natives – resulted in the in-person audience using the app to leave targeted, derogatory comments. Even so, Louise says, it was fascinating to reflect on their real-time realisation that “they could essentially become performers in the show”.
Evaluating a show about success
Louise’s evaluation process, conducted with her producer, takes in audience response, sales targets, critical responses, and the response from the venue. But, as befits Famehungry, this process also examines her own artistic growth. The biggest success, she says, is in the feeling that “I’ve been training for this kind of work for ages. I love to create situations of risk, and that’s when audiences respond best to my work, when they can feel how alive it is.”
Written by Katie Hawthorne
How useful was this resource?
You may also be interested in
A practical framework for evaluating digital cultural experiences
Areas to consider when evaluating a digital project
Well-designed evaluation can help you understand the types of audience engaging and therefore whether your marketing is well targeted
Join our Mailing List:
Follow us on social media:
