Question Evaluation

Question evaluation is the process of testing and analyzing how survey questions perform. With an understanding of how questions perform, survey managers and data users have full confidence in the validity of the data, as well as an opportunity to revise questions to better fit their studies. Some of the strengths of question evaluation are listed below.

Question evaluation can determine if a question...

Ideally, questions should capture constructs they were created to capture. However, it is important to know if questions are not capturing intended constructs. Understanding constructs that questions are capturing allows survey managers to modify questions, and allows data users to understand the potential range of constructs that may have been captured in the data. Both of these measures increase the data validity.

Respondents' interpretations of a question may differ depending on their social experiences and demographic factors. For example, respondents with different education levels might interpret or comprehend question terminology differently.

There are many possible reasons why respondents might have difficulty answering a question. Question evaluation can help to shed light on this process.

Additional Information

Question Evaluation is required for all new surveys or questions. The Office of Management and Budget stipulates in Section 1.4 of The Standards and Guidelines for Statistical Surveys:

Agencies must ensure that all components of a survey function as intended when implemented in the full-scale survey and that measurement error is controlled by conducting a pretest of the survey components or by having successfully fielded the survey components on a previous occasion.

Related Conferences and Workgroups