What I offer here is the first in a series of posts on how we measure the impact of the arts sector. I will begin with a story about the evaluation of a new mural festival near Boston. Following that, I’ll describe how our consulting team is helping NYC’s Department of Cultural Affairs coach their community partners on measuring the impact of different investments in neighborhoods. Finally, I’ll introduce our measurement-based approach to cultural planning.
Before getting to those stories, we should affirm why measurement is important for the sector.
First, arts professionals often make presumptions about the value of the arts. The problem is that those presumptions are not accepted, let alone understood, by others outside the field. Thus it becomes important to demonstrate correlations, causality or simple relationships between the arts and other good things.
Second, funders are looking for ways to defend support for the arts, and seek an effective measure of their “return on investment.” That is more easily done with numbers than with words.
Third, you can only move what you can measure. That is to say that you can’t really claim progress unless you can measure, and thus prove, progress.
So how does this measurement work in the real world? Our first story is about measuring the impact of a new mural festival in Lynn, MA (a suburb of Boston). Lynn used to be a very tough town, immortalized by a song that included the following stanzas.
Lynn, Lynn, the city of sin
You never come out the way you came inYou ask for water, but they give you gin
The girls say no, yet they always give in
These days, Lynn is diverse and busy with an active downtown, great old buildings and a bustling economy. Unfortunately, it still gets a bad rap as a scary place.
And that’s where our story begins.
Along come a few folks determined to change perceptions of the community, using the arts. They dream up the idea of a mural festival to occur in their downtown during the summer of 2017. They convince building owners to allow blank walls to be painted. They commission artists to come to Lynn to create murals on those blank walls. And they make it a festival, with commemorations and celebrations over 10 days.
The Beyond Walls Festival is an immediate hit. People in the community journey downtown to see what’s going on; people from elsewhere come to Lynn to be a part of the activities. There’s inescapable buzz and positive energy all over the community. The small, stretched nonprofit organization that is producing the festival knows they’re doing something great. They have a sense that measuring that greatness is important but they’re swamped with the festival itself. It isn’t until a short period after the festival ends that they manage to convince Mass Development to help them organize an impact study.
We are commissioned to do that work, so we roll up our sleeves and see what raw data we have to work with. We collect attendance data from the festival. We organize surveys of community residents, merchants and the artists who participated in the festival. We collate research on traditional and online media coverage of the event.
Our team develops an input/output model to project economic impact using multipliers purchased from the Bureau of Economic Analysis in Washington, DC. But the lack of good input information from local merchants and the inherently conservative nature of a county-based model leads to low-impact numbers.
We collect great stories in the surveys about changing perceptions of the downtown area and the community, but we can’t prove a change in tax revenue, business openings and local employment. We also collect comprehensive information on every article or post on the festival, but we can’t prove how media coverage directly impacted the local economy.
We can show value and positive impact, but it is expressed more through stories than through statistics. We learn the hard lesson that an after-the-fact scramble for data won’t be sufficient, and that good measurement is often a function of good preparation.
But the real value of our work is to prepare the festival organizers for the next year’s event — specifically by identifying new and better sources of quantifiable data that can be collected before, during and after the festival, and thus provide a lot stronger case for the impact of the event.
Next month, I’ll continue with what our consulting team is doing with NYC to coach community partners to measure the impact of arts investments in neighborhoods.
