A question we often hear form our clients at the early stages of a research project is about their expected response rate.  What response rate should we get?  Is it enough?  Is it better or worse than others?  Should we give away a prize to insure a higher response rate?

Response rate is important.  It tells us a lot about the target population’s attitude regarding the survey and their belief in our willingness to listen and take action.  Response rate, however, is not as important as the number of responses.  If we have a population of 100 people and 20% of that population respond to our survey (20 responses) we do not know much about the 100 people.  There is a high probability that the 20 responses could be substantively different than the 80 non respondents because it is composed of only 20 different people.  If, however, we have a 20% response rate from a population of 200,000 (40,000 responses) there is a very low probability that 40,000 different people could collectively be way outside of the norm.

Additionally, response rate will vary for many different reasons.  Understanding this variation and its causes is informative about respondent attitudes and affinity but does not necessarily give more information about the accuracy of the findings.

Since 2001 we have completed alumni and donor survey projects with over 140 universities and colleges.  Within that group, we have had response rates as low as 5% and as high as 35%.  The distribution sets have ranged from 3,000 to 150,000.

In my judgment, response rate is a function of:

1. Affinity between target group and the institution doing the survey,
2. Demographic characteristics of the distribution pool,
3. Structure of the survey instrument,
4. Belief of the respondents that someone will read the surveys and take action as a result of the findings.

For instance, current givers or current members will usually respond at a rate that is two to 5 times larger than never givers.  If the database is oversampling never givers then the response rate will be lower.

Finally, I cannot overemphasize the last consideration.  If your alumni believe that you will actually listen to their responses and take action on the results they are much more likely to respond.  When you survey, always let them know what you heard and what you are going to do as a result of what you heard.  When you take action that is based on what you heard, let them know!  Finally, ask again.  We call this the Alumni Partnership

Model(c) and it is a critical part of a successful feedback systems, higher response rates, and more engagement.

One other thing to keep in mind is that response rate is much less important than having a sufficiently large number of responses. Statistical confidence is not a function of response rate but is based on the number of responses, their demographic distribution, and the methodology of participant selection, survey methodology, and instrument design.

Related posts:

1. Partnering With Your Stakeholders
2. Some things to keep in mind if you are planning an alumni survey:
3. On the Metrics of the AAS