Details
| Original language | English |
|---|---|
| Pages (from-to) | 401-417 |
| Number of pages | 17 |
| Journal | Journal of Survey Statistics and Methodology |
| Volume | 6 |
| Issue number | 3 |
| Publication status | Published - 3 Oct 2017 |
| Externally published | Yes |
Abstract
In empirical social research, using questions with an agreement scale, also known as agree/disagree (A/D) questions, is a popular technique for measuring attitudes and opinions. Methodological considerations, however, suggest that such questions require effortful cognitive processing and are prone to response bias, such as acquiescence. Therefore, many researchers recommend the use of item-specific (IS) questions, which are based on tailored response categories and seem to imply less response burden. In this study, we investigate the cognitive processing of A/D and IS questions in web surveys, using eye-tracking methodology. On the basis of recordings of respondents' eye movements, we are able to draw conclusions about how respondents process survey questions and to evaluate how they process information. Our results indicate that IS questions require deeper processing than A/D questions. Interestingly, the eye-tracking data reveals that this phenomenon is only observable for the response categories but not for the question stems; this indicates that the stems do not differ in terms of cognitive effort. We therefore argue that the observed differences are directly attributable to a more intensive processing of the IS response categories. Practically speaking, this additionally indicates a more thoughtful processing of the response categories and, thus, might lead to more well-considered and appropriate responses.
Keywords
- Asking manner, Cognitive effort, Eye-tracking experiment, Question format, Response behavior, Web survey
ASJC Scopus subject areas
- Mathematics(all)
- Statistics and Probability
- Social Sciences(all)
- Social Sciences (miscellaneous)
- Decision Sciences(all)
- Statistics, Probability and Uncertainty
- Mathematics(all)
- Applied Mathematics
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
In: Journal of Survey Statistics and Methodology, Vol. 6, No. 3, 03.10.2017, p. 401-417.
Research output: Contribution to journal › Article › Research › peer review
}
TY - JOUR
T1 - New insights on the cognitive processing of agree/disagree and item-specific questions
AU - Höhne, Jan Karem
AU - Lenzner, Timo
PY - 2017/10/3
Y1 - 2017/10/3
N2 - In empirical social research, using questions with an agreement scale, also known as agree/disagree (A/D) questions, is a popular technique for measuring attitudes and opinions. Methodological considerations, however, suggest that such questions require effortful cognitive processing and are prone to response bias, such as acquiescence. Therefore, many researchers recommend the use of item-specific (IS) questions, which are based on tailored response categories and seem to imply less response burden. In this study, we investigate the cognitive processing of A/D and IS questions in web surveys, using eye-tracking methodology. On the basis of recordings of respondents' eye movements, we are able to draw conclusions about how respondents process survey questions and to evaluate how they process information. Our results indicate that IS questions require deeper processing than A/D questions. Interestingly, the eye-tracking data reveals that this phenomenon is only observable for the response categories but not for the question stems; this indicates that the stems do not differ in terms of cognitive effort. We therefore argue that the observed differences are directly attributable to a more intensive processing of the IS response categories. Practically speaking, this additionally indicates a more thoughtful processing of the response categories and, thus, might lead to more well-considered and appropriate responses.
AB - In empirical social research, using questions with an agreement scale, also known as agree/disagree (A/D) questions, is a popular technique for measuring attitudes and opinions. Methodological considerations, however, suggest that such questions require effortful cognitive processing and are prone to response bias, such as acquiescence. Therefore, many researchers recommend the use of item-specific (IS) questions, which are based on tailored response categories and seem to imply less response burden. In this study, we investigate the cognitive processing of A/D and IS questions in web surveys, using eye-tracking methodology. On the basis of recordings of respondents' eye movements, we are able to draw conclusions about how respondents process survey questions and to evaluate how they process information. Our results indicate that IS questions require deeper processing than A/D questions. Interestingly, the eye-tracking data reveals that this phenomenon is only observable for the response categories but not for the question stems; this indicates that the stems do not differ in terms of cognitive effort. We therefore argue that the observed differences are directly attributable to a more intensive processing of the IS response categories. Practically speaking, this additionally indicates a more thoughtful processing of the response categories and, thus, might lead to more well-considered and appropriate responses.
KW - Asking manner
KW - Cognitive effort
KW - Eye-tracking experiment
KW - Question format
KW - Response behavior
KW - Web survey
UR - http://www.scopus.com/inward/record.url?scp=85054914647&partnerID=8YFLogxK
U2 - 10.1093/JSSAM/SMX028
DO - 10.1093/JSSAM/SMX028
M3 - Article
AN - SCOPUS:85054914647
VL - 6
SP - 401
EP - 417
JO - Journal of Survey Statistics and Methodology
JF - Journal of Survey Statistics and Methodology
SN - 2325-0984
IS - 3
ER -