Research is often designed to compare two things to each other. Likewise, these designs for listening differ mostly in what the control group should be for your situation. You should always have one over-arching goal in mind: Systematically listening to people, so that you can incorporate their views into a better project design. This means you need to know what your goal is for the project, and you need to know what you don't know about the project. That will dictate what you ask people.
The narratives you collect will have more depth and detail than the multiple choice questions that follow in the survey.
Later, during analysis, you will use both kinds of data to create benchmarks and compare your group of stories against some control group. Your program design will dictate which listening design will work best.
Focus on the location and the people that live there. Ask an open-ended question like,
"Talk about a time when a person or organization tried to help someone or change something in your community. What happened?"
From 2010-2012 we collected nearly 60,000 responses to this question, searchable online.
All stories in a collection will be connected by their proximity in space and time, and often this alone is able to reveal patterns that should inform project design. For example, the Kenyan NGO VAP interviewed girls in their program and learned that rape was a major life issue. Too many girls were bringing it up in generic "community mapping" stories - more than a third - to ignore. So they changed the program to address this immediate need. The following year they used this process to hone in on aspects of youth crime that were amenable to after-school lessons. We summarized their journey in this case study from these (1,2) blog posts.
In 2012 our GlobalGiving storytelling project was based in many parts of East Africa. We collected and published community maps for each community:
In the past creating these maps was a manual process, but soon they will be automatically visualized from the stories and their meta data. "Meta data" are the little bits of related data around stories, such as who scribed them and where the story took place. It is safe to assume that stories with overlapping people and places and dates are important signals for program managers, and that visualizing it can help them make smarter decisions (or even make them smarter decision-makers).
Impact is a vague, ambiguous word. We disambiguate "Impact" by offering many different approaches to measuring it, depending on the best way to gather signals from people in your context.
Some past storytelling questions give you a sense for how to map the problem, or issue, that comes to mind in association with a topic:
Please tell a story about a time when you had to choose between protecting the environment and maintaining a livelihood. Include if/how individuals or organizations were involved in the conflict.
Please tell a story about a time when you tried to get a job. What helped you get a job?
Please talk about a specific time that you felt more visible in your community. What happened and how did it expose some hidden need or issue? What would you like to do to help address it?
This prompt will not yield as many insightful stories about the problem. Instead is gives an organization information to diagnose the problems with one specific solution to the problem (their existing project):
Please tell a story on a most significant change that you have observed based on your experience as a participant in our program(s).
Good story prompting questions undergo design evolution based on field testing:
First draft: Please tell a story about a time when you had to work with someone different from yourself.
Second draft: Please tell a story about a time when a conflict arose because you had to work with someone from a different background (religious, cultural, ethnic etc.) to yourself.
Third draft: Please tell a story about a time when a person changed someone else's perception of them or challenged a prejudice or misunderstanding.
Collect conflict narratives
Good writers know that narratives require conflict to be interesting. There are as many different kinds of conflicts in fiction as there are programs to solve real-life conflicts. Narratives about having to choose between two things, or explain why things happened the way they did will yield longer, more complex, deeper narratives. Yet all of this needs to fit in under 150 words for good storytelling.
And these illustrate how GlobalGiving has used it in our own network:
Talk about your experience approaching a grantmaking or funding organization that either did or did not grant you funding. What was your relationship like? Did you receive support from them?
Please tell a story about a time when when a nonprofit listened, acted, and learned to become more effective at fundraising on GlobalGiving. Did they became more effective in real life?
As a past global giver, why did you give to GlobalGiving or to this project in particular?
Some prompts are too specific to be comparable to any other stories:
Please tell a story about a time when YaLa Africa tried to help and empower you or your community through micro-gardening and nutrition training.
You'll notice we don't prompt storytellers to describe the impact directly. Instead we ask them to describe specific events and how they felt about these events, so that the Impact can emerge from the collect
You will select from a pool of over 40 questions in building your survey. Most are optional, and you certainly can't use more than half of these in a single survey (it must fit on one page). Depending on what you want to learn, use these follow-up questions to extend these narratives in ways that allow for specific quantitative comparisons that reflect assumptions you make about your work.
Freedom | Fun
Knowledge | Respect | Creativity | Self-esteem |
Food and shelter| Security| Family and friends| Physical needs
Choose a point on the line.
What else would have made a difference in your story?
It might seem obvious that the people who can best explain what's not working with your program are the people who never chose to participate in the first place. These are the neighbors, friends, relatives, and nearby shop keepers of the people you serve. And yet it is so rare when organizations do interview them. Take this example from one of our partner's project reports:
"We were not happy and tried to figure out what went had gone wrong with the Vision Centre. We conducted a random sample survey and interviewed the few patients who visited the centre, random people in the nearby area, and also the staff. With the survey we came to know that the location of the Vision Centre is not appropriate, it is difficult to reach and requires more conveyance cost. Also, we came to know that we failed to understand the customer's needs to providing specific frame/design/range of spectacles as asked by them. Now we have changed our inventory materials to suit the customer's demands." (From our globalgiving.org/fail-forward/ essays)
Sometimes it makes no sense to interview neighbors of the people you serve. In that case, each program participant can serve as his or her own control if you invite them to share two stories. The first story can be "How does organization X help you?" and the second one, "how has some other organization helped you?" With variations on this within-subjects control design you can make many comparisons.
What else do the people you serve care about or need?
What other organizations are having an impact on the lives of the people you serve?
How do people feel about various life issues that intersect with the problems your program claims to address?
Is there one demographic group that you are reaching more (or failing to reach)?
This is explained further in our essay, demonstrating the power of the two-story rule.
Impact is positive change over time. The number one reason Impact is hard to measure is that the people with the money and the power don't want to wait for time to pass - they want to know immediately. But if you don't ask people to describe life before an intervention, you will find it hard to measure change. Mathematically, it is impossible, though people often use weak data from elsewhere as a proxy for the baseline.
To be able to look at how a collection of narratives is changing over time, you need (at a minimum) to ask people before and after the program. If programs are ongoing, then you can ask periodically. Very strict researchers would ask the same people at regular intervals, but in the real world getting an organization to just ask two times (before and after) would be a huge improvement over what they have done in the past.
Journaling - if you have volunteers working for weeks or months, have them keep a journal. After, scan and datify the content as stories. So instead of two stories (pre and post), you would have a dozen or more stories from the same person about an issue. Growth is easy to see with journaling.
Focus groups - Informal discussions can be augmented by during a transcript of stories shared into data. Leave a tape recorder running, or use an app like dictadroid to convert it to MP3 and email for transcription immediately after.
The best data is the kind that already exists. All you need to do is add your unique part and use our comparison tools to look at how peoples' experiences differ. You can use existing stories in the collection to look at information about the work of similar organizations, and benchmark your program against theirs.
How to read the chart above: For the program on the left (Mermbo), girls are the soul focus, and symbols for self-esteem and respect are important themes to these girls. For the program on the right (Sita Kimya), also aiming to prevent rape in Nairobi slums, we see that both boys and girls are involved. With additional analysis (that we'll teach you how to do), it turns out that the Sita Kimya program got the basic message out ("Stop Rape!") but never connected the idea that rape is one way that HIV is spreading in the community. That's a pretty important omission, and a major focus of the related project (Mrembo). Based on details people include or omit in stories, it is possible to discover many surprising things about programs they write about.
This approach was inspired by the Framingham Heart Study, a longitudinal health study that has discovered hundreds of important risk factors for heart disease, simply by following people over time with an open-inquiry approach. Our goal is to create a "Framingham Heart Study" on the root causes of poverty.
In the past we curated a collection of over 57,000 stories from Kenya and Uganda (2010-2013). Read about how organizations have used storytelling to be more effective organizations: