How can I pilot test the evaluation design and settings?

Pilot testing will allow you to 'try out' your evaluation before finalizing it and sending it out to your respondents. It will allow you to:

• Check the presentation, wording and clarity of the content of the evaluation.
• Check the deployment methods.
• Test how long it will take the respondent to complete the evaluation.
• Ensure the relevance and accuracy of the data you collect.

You can pilot test your evaluation using any of the responses collection options on the Collect Responses page. Once you have completed the pilot test, you can clear the responses collected by clicking on the [Clear Responses] icon located to the right of the evaluation name.

Checklist

The following provides a sample checklist of questions to consider during the pilot test:

• Do the respondents understand the objective of the evaluation?
• Do the respondents feel comfortable answering the questions?
• Is the wording of the evaluation clear?
• Is the timeframe for responding clear to the respondents?
• Are the answer choices compatible with the respondents’ experience in the matter?
• Do any of the items require respondents to think too long or hard before responding? If so, which ones?
• Which items produce irritation, embarrassment, or confusion?
• Do any of the questions generate response bias? If so, which ones?
• Do the answers collected reflect what you want in regards to the purpose of the evaluation?
• Is there enough diversity in the answers received?
• Is the evaluation too long?
• According to your test audience, have any other important issues been overlooked?

(Source: Iraossi, G. (2006). The Power of Survey Design: A User's Guide for Managing Surveys, Interpreting Results, and Influencing Respondents. Washington, D.C.: The World Bank)

Other Testing Tips:

When testing, you might also consider the following in questions.

1. Did you include a clear introduction explaining who you are, what the evaluation is about, how to move through the evaluation, etc.?
2. If there are sections that need to be described separately, have you added pages or sections into the design?
3. Is the Required Answer feature active on relevant questions?

Back

Print

Article Quick Jump :

How can I pilot test the evaluation design and settings? What types of questions can I add to my evaluation?