Surveys
Surveys are commonly used to collect assessment information from large numbers of people. They can be simple cards distributed in program notes to audience members, surveys mailed to representative samples of a population, exit polls (a combination of survey and interview), or telephone polls of randomly selected households. Surveys generally collect demographic information (gender, age, occupation, education, income, family size) preferences (What do you do for recreation? What arts events have you attended?) and opinions (Do you like modern dance?).
Surveys are so common it’s easy to forget that the successful ones are quite sophisticated. The most successful do-it-yourself surveys are those conducted with a known audience. If you distribute a demographic and opinion survey to renewing subscribers, you can be reasonably sure that you understand the characteristics and opinions of your subscribers.
However, it is easy to collect misleading information. If you conduct a telephone survey of your community, you may get a limited picture. For example, afternoon calls will only reach people who do not work outside the home, and evening calls will reach people who stay home. A small survey may suggest a trend that is statistically insignificant, and the results you obtain may be solely a matter of chance.
For general population surveys, it is highly recommended that you seek professional advice. An experienced consultant or graduate student at a school of business or sociology may either design a survey or conduct one. see Sample Survey Questions (PDF)
Interviews
Another way to gather survey information is the one-on-one interview. A good interview requires preparation and skill. You must determine what information is needed, who to ask, and how to formulate questions. The information needed depends, of course, on the organization and its plan. Typically, external perceptions are sought regarding community needs and interests, opportunities and threats, and potential resources.
A good interviewer prepares a list of questions that will guide the interview. However, the best interviews seem more like conversations, with questions emerging naturally from the flow of discussion. Don’t ask your questions in lockstep fashion. An answer to an open-ended first question may well anticipate your second and third questions. Inexperienced interviewers often make the mistake of talking too much. Listening is key. It is vitally important to listen actively, restating important points to be sure they were accurately understood.
Take extensive notes. A tape recorder may help, although it may inhibit the person you are interviewing. If you want to use a tape recorder, be sure to ask permission first. Interview notes and tapes should remain confidential. Ask for permission to attribute any direct quotes; otherwise, report only summaries.
Steps of Survey Research:
- Identify the problem to be solved or general questions to be answered. What are the characteristics of our audiences? How do people find out about us? What kinds of programming would appeal to the people who currently are not attending? Focus groups and interviews might be useful in identifying an area to explore by survey.
- Identify who you want to survey. Start by identifying the overall population (community, neighborhood, or existing audience), a zip code, a demographic, a particular group (i.e., kids 11-17 years old). There are professional services through PR firms to help you get the list you are looking for.
- Design the survey questionnaire. Compose questions to elicit the required information.
Questions can be:
Open (What kind of music do you like?)
Closed (I like country and western music. True or False)Tip: Be very strategic in your use of open questions. Those data are much harder to compile and use than that from the closed questions.
- Test your questionnaire beforehand with a smaller sample to weed out confusing questions or other problems. Some common errors are two questions in one, overly complicated questions, leading questions, and jargon.
- Distribute the questionnaires. Consider doing your survey online. It allows you to capture the information in a database, and many people are more willing to do a quick web survey than return a paper form in the mail.
- Incentivize them. Offer rewards for doing the survey. Have people fill out a contact form separately to enter them in drawing for prizes, coupons, free tickets, etc. Use the contact information to build out your contact list. With a live audience, you can give everyone a sticker like they do in voting booths. People like rewards, even tiny ones.
- Monitor and begin compiling the results. You want to have database in which to enter your results. This can be done in any spreadsheet or database program.
- Look for patterns in open questions. Read through all the surveys, noting frequent responses, and pulling out the well-constructed comments.
- Analyze the results. With a computer program you can easily generate graphic portrayals of the data, such as bar and pie charts.
- Share the results with the participants, your members, staff, and decision makers. People like to see how their feedback stacks up to others and get a sense of what you are going to do with it.
Focus Groups
Focus groups are interviews of a group of people with common interests. The approach was originally developed for commercial market research and has found applications in the areas of nonprofit planning. Focus groups are a simple way to determine what a large number of people are likely to think about a given subject by interviewing a few carefully selected representatives. You can get a general sense of preferences or reactions to specific programs.
How to Run a Focus Group:
- Determine what information you seek and compose specific questions to elicit that information. Limit focus groups to one or two general topics and no more than four or five specific questions.
- Determine which community segments you wish to hear from and select a representative sample. These might be groups such as your audience, people who are not attending, artists, educators, etc. A common error is to invite only those who care most about the arts and then to assume that the entire population feels the same way. People who do not see themselves as interested in the arts may need an incentive to participate.
- Schedule focus group meetings. Always convene more than one meeting, as a single meeting could provide information unique to that group’s dynamics. Usually focus groups consist of six to ten people. The minimum number is four, and twelve is the manageable maximum. Allow about one hour per meeting.
- The meetings require a moderator who introduces the topic, poses questions, and facilitates the conversation. The moderator can direct the conversations or stay in the background and only intervene when the discussion gets off track.
- The meeting agenda might include the following: Explain the reason for the meeting, introduce the general topic, introduce participants, and then pose the first specific question. After some discussion of the first question, move on to subsequent questions. Summarize the key points at the conclusion of the meeting.
- Be as specific as possible. If someone says she likes film, probe and find out what types. Ask questions that your group is capable of answering. Don’t ask people who have never been to your programs what they think of your programs. To determine what people think of your programs, show a video segment and invite reactions to it.
- Professional focus group researchers tape-record or videotape the focus groups, transcribe the discussions, and closely analyze the results. At the very least, take detailed notes.
- During the second and third focus groups, ask the same questions. Be sure not to let the group know what the first one said. The point of conducting multiple meetings is to determine whether perceptions are shared. If you ask leading questions “We’ve been hearing that our ticket prices are too low. What do you think?” you’ll get biased results.
- If you have conducted two focus group meetings and can successfully predict what the third one will say, you have done enough. If you keep getting new or conflicting information, you should continue.
- At the conclusion of all the focus groups, review the notes with an eye for patterns of interest and references that show up in more than one meeting. Test conclusions for validity by comparing them to other sources of assessment information such as ticket sales, competitors’ trends, surveys, or first-hand observations.
Adapted from
Arts Extension Service Fundamentals of Local Arts Management
