Asking Good Questions: Surveys

Surveys are probably the most common method of gathering information from customers, and are particularly useful for understanding the proportions of your customers that fit a particular description.
Asking Good Questions: Surveys - Articles

Updated: August 8, 2017

Asking Good Questions: Surveys

For example, you might want to know the percent of your customers who heard about you through your brochures, the average rating your customers give to potential products for next season, or the top three positive and negative qualities of your business identified by your customers.

Methods

The first step is to develop the set of questions to include in the survey. Most questions on a survey are likely to be closed-ended, meaning that you provide a set of pre-defined answers that customers can choose. Open-ended questions, or those without predetermined answer categories, can be used as well. However, if the number of customers you're contacting is large, answers to open-ended questions can be hard to summarize. (Tips for asking good questions can be found in Listening to Your Customers: Asking Good Questions.)

Next, you need to determine how you're going to give the survey to your customers and how you're going to get it back from them. Primary delivery methods for direct market ag businesses include:

  • Mail: surveys are mailed to customers, and usually include a stamped envelope to return the completed survey. To increase the number of people who respond, it is helpful to contact them multiple times. A common approach is to send them the survey, then a reminder letter or postcard about a week later. For those who don't respond to this first contact, it might be helpful to follow up either with another copy of the survey or with a phone call (depending on your budget and how many have not responded). Mail surveys can be expensive and time consuming, because of the printing and postage. This approach works best when you have the contact information for your customers. It provides the most confidentiality and convenience for customers.
  • Hand out: surveys can be given to customers at a meeting, point of purchase, or delivery. This approach works well when you don't have contact information for customers. It also allows you to answer questions customers may have about the survey. The completed surveys can either be left with you or mailed back at a later time.
  • Telephone: customers are called and asked a series of questions over the telephone. Telephone surveys are less expensive than mail surveys, but can be extremely time-consuming if you're doing all the interviews yourself. The questions need to be written in such a way that they are easy to ask and answer verbally. Telephone surveys also need to have a limited number of questions, as people are easily distracted when they are on the phone.
  • Electronic (web, email): this is rapidly becoming a preferred approach for many businesses because it is relatively easy to create the survey and many survey web sites compile the data for you. This approach might be appropriate for you if most or all your customers have internet access and some basic skills in its use. If most of your customers have access, but a portion does not, you might consider combining electronic with other approaches that are more appropriate. To increase the number of people who respond, it is helpful to contact them via email multiple times. Common tools for web-based surveys include SurveyMonkey and Zoomerang.

Regardless of your approach, you need to develop an introduction for the survey packet, telephone conversation, or invitation to the electronic survey. This should include the reason for doing the survey, how their information will be used, if the information will be confidential or anonymous, and how to contact you if they have questions.

Analyzing the data

Answers to open-ended questions should be organized by question. This will help you to identify and summarize common themes or ideas associated with each topic or issue.

Analyzing answers to closed-ended questions can be done by hand (although this can be extremely time-consuming) or by computer. Basic spreadsheet software (such as Microsoft Excel) can be used to enter and analyze the survey responses. You would need to enter the responses to each question from each survey into Excel. For some questions, this will be a number. For example, you might have asked customers the total number of people in their household. You would simply enter this number into Excel. For other questions, you may have asked customers to check the answer that best fits them. To simplify data entry, assign a number to each answer category. For example, you asked each customer to identify their top reason for buying from you, and gave them five possible reasons (quality of products, quantity of products, organic certification, presentation of products, friendliness of owner/staff). You would assign a number from 1-5 to each answer category. You would enter the number associated with the answer category that each customer chose.

Once all the surveys are entered, you can create frequency tables, or tables that describe how many people (or what percentage) chose each answer to a question. For example, you might want to know the percent of your customers that live in the four surrounding townships. You can also calculate other statistics, such as averages (average number of times customers visited your booth over the season) and ranges (the minimum and maximum amounts spent by customers when they visited your booth). To compare sub-groups among your customers, you would need to select each set and then calculate the frequencies or statistics separately. For example, it might be helpful to know the percent of new versus long-term customers who were highly satisfied with your business.

Cost

The cost of surveys varies greatly depending on your choice for delivering the survey and the number of people you want to contact. Mail is the most expensive, as it requires printing and postage. Handing out surveys is less expensive, requiring printing of the surveys (there might be postage as well if customers are to return the survey via mail). Telephone surveys will incur telephone usage charges. Electronic methods are the least expensive. Popular web-based services may offer limited service for free but charge for more advanced service.

Advantages of Surveys

  • Gathers a broad range of responses from customers
  • Allows for comparisons among groups of customers
  • Efficient, cost-effective method for gathering information from a large number of customers
  • Information received will represent all customers, if sample chosen properly (or all customers surveyed)
  • Survey responses are confidential or anonymous, making participants feel more comfortable expressing their opinions
  • Respondents can fill out a survey at their own convenience, and usually in a short amount of time.

Disadvantages of Surveys

  • Can be expensive to design and implement, depending on length and number of customers contacted
  • Results may be invalid if survey not designed or administered correctly
  • Survey design and data analysis can be daunting
  • Sometimes difficult to obtain contact information for all customers
  • Respondents may skip or misunderstand some questions.

Example

Suppose you are a dairy with on-farm cheese-making facility, selling through on-farm retail store, internet, and local retail stores

Goal:

  • to develop marketing materials
  • to reach new internet customers;
  • to increase number of repeat internet customers

Data Needed:

Who are the internet customers? How did they find the website? What was their evaluation of the website and their purchases? What products would they like to purchase in the future?

Methods/Procedures:

Because you want to know several types of information, a survey is the most appropriate approach. This particular set of customers has internet access and some skills in its use, so an electronic survey is a logical choice. You've had a total of 120 internet customers since you first established the website. This number is manageable, particularly in an on-line survey, so you decide to contact all 120.

The free basic service offered by SurveyMonkey limits the survey to 10 questions and the number of customers to100. It is unlikely that you'll get more than 100 of your 120 customers to respond, so this service fits your needs. You decide to ask mostly closed-ended questions because the number of questions you want to cover and the number of customers is relatively large. Using SurveyMonkey, you create a survey that asks questions such as:

  • Please check the answer that best describes how you first heard about our website. (Answer categories might include: internet search engine such as Google, farm-specific website such as LocalHarvest.com or AgMap, link from another website [list], product purchased in a store, recommendation from a friend)
  • Overall, how satisfied were you with the products you purchased? (very unsatisfied, unsatisfied, satisfied, very satisfied)
  • Please rate the following features of our website using the following scale: very poor, poor, satisfactory, good, and very good. (The list of features might include attractiveness, ease of use, download speed, etc.; or it could identify content, such as the welcome page, product descriptions, farm information, checkout process, etc.
  • Below is a list of products we are considering offering in the future. Please check the products you would seriously consider buying from us in the next two years.

To administer this survey, you would develop an invitation email that describes the project, gives your contact information, assures customers that their answers are confidential, and provides them with the link to the survey. About a week after sending the invitation, send a reminder email and encourage customers to respond. You might consider sending one more reminder about two weeks later to those who haven't responded.

SurveyMonkey generates a report of the results, listing the number and percentage of people in each answer category.

Like the other methods discussed in this series using surveys is not as difficult as it may seem at first. Carefully considering your information needs and the questions you need to ask are vital whether you use a survey, interview, or focus group.

July 2007

Authors

Energy and Society, specifically Community Impacts of Marcellus Shale Development Community Development and Public Engagement Water Quality Environmental Governance Systems Women in Agriculture

More by Kathryn Brasier, Ph.D.