Details
| Original language | English |
|---|---|
| Pages (from-to) | 955-980 |
| Number of pages | 26 |
| Journal | Journal of the Royal Statistical Society. Series A: Statistics in Society |
| Volume | 185 |
| Issue number | 3 |
| Publication status | Published - 20 May 2022 |
| Externally published | Yes |
Abstract
Images might provide richer and more objective information than text answers to open-ended survey questions. Little is known, nonetheless, about the consequences for data quality of asking participants to answer open-ended questions with images. Therefore, this paper addresses three research questions: (1) What is the effect of answering web survey questions with images instead of text on breakoff, noncompliance with the task, completion time and question evaluation? (2) What is the effect of including a motivational message on these four aspects? (3) Does the impact of asking to answer with images instead of text vary across device types? To answer these questions, we implemented a 2 × 3 between-subject web survey experiment (N = 3043) in Germany. Half of the sample was required to answer using PCs and the other half with smartphones. Within each device group, respondents were randomly assigned to (1) a control group answering open-ended questions with text; (2) a treatment group answering open-ended questions with images; and (3) another treatment group answering open-ended questions with images but prompted with a motivational message. Results show that asking participants to answer with images significantly increases participants' likelihood of noncompliance as well as their completion times, while worsening their overall survey experience. Including motivational messages, moreover, moderately reduces the likelihood of noncompliance. Finally, the likelihood of noncompliance is similar across devices.
Keywords
- breakoff, images, motivational messages, noncompliance, smartphone, web survey
ASJC Scopus subject areas
- Mathematics(all)
- Statistics and Probability
- Social Sciences(all)
- Social Sciences (miscellaneous)
- Economics, Econometrics and Finance(all)
- Economics and Econometrics
- Decision Sciences(all)
- Statistics, Probability and Uncertainty
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
In: Journal of the Royal Statistical Society. Series A: Statistics in Society, Vol. 185, No. 3, 20.05.2022, p. 955-980.
Research output: Contribution to journal › Article › Research › peer review
}
TY - JOUR
T1 - A new experiment on the use of images to answer web survey questions
AU - Bosch, Oriol J.
AU - Revilla, Melanie
AU - Qureshi, Danish Daniel
AU - Höhne, Jan Karem
N1 - Funding Information: German Science Foundation, through the Collaborative Research Center 884 “Political Economyof Reforms”, 139943784; European Research Council (ERC) under the European Unions Horizon 2020 research and innovation programme, 849165 Funding information
PY - 2022/5/20
Y1 - 2022/5/20
N2 - Images might provide richer and more objective information than text answers to open-ended survey questions. Little is known, nonetheless, about the consequences for data quality of asking participants to answer open-ended questions with images. Therefore, this paper addresses three research questions: (1) What is the effect of answering web survey questions with images instead of text on breakoff, noncompliance with the task, completion time and question evaluation? (2) What is the effect of including a motivational message on these four aspects? (3) Does the impact of asking to answer with images instead of text vary across device types? To answer these questions, we implemented a 2 × 3 between-subject web survey experiment (N = 3043) in Germany. Half of the sample was required to answer using PCs and the other half with smartphones. Within each device group, respondents were randomly assigned to (1) a control group answering open-ended questions with text; (2) a treatment group answering open-ended questions with images; and (3) another treatment group answering open-ended questions with images but prompted with a motivational message. Results show that asking participants to answer with images significantly increases participants' likelihood of noncompliance as well as their completion times, while worsening their overall survey experience. Including motivational messages, moreover, moderately reduces the likelihood of noncompliance. Finally, the likelihood of noncompliance is similar across devices.
AB - Images might provide richer and more objective information than text answers to open-ended survey questions. Little is known, nonetheless, about the consequences for data quality of asking participants to answer open-ended questions with images. Therefore, this paper addresses three research questions: (1) What is the effect of answering web survey questions with images instead of text on breakoff, noncompliance with the task, completion time and question evaluation? (2) What is the effect of including a motivational message on these four aspects? (3) Does the impact of asking to answer with images instead of text vary across device types? To answer these questions, we implemented a 2 × 3 between-subject web survey experiment (N = 3043) in Germany. Half of the sample was required to answer using PCs and the other half with smartphones. Within each device group, respondents were randomly assigned to (1) a control group answering open-ended questions with text; (2) a treatment group answering open-ended questions with images; and (3) another treatment group answering open-ended questions with images but prompted with a motivational message. Results show that asking participants to answer with images significantly increases participants' likelihood of noncompliance as well as their completion times, while worsening their overall survey experience. Including motivational messages, moreover, moderately reduces the likelihood of noncompliance. Finally, the likelihood of noncompliance is similar across devices.
KW - breakoff
KW - images
KW - motivational messages
KW - noncompliance
KW - smartphone
KW - web survey
UR - http://www.scopus.com/inward/record.url?scp=85130462555&partnerID=8YFLogxK
U2 - 10.1111/rssa.12856
DO - 10.1111/rssa.12856
M3 - Article
AN - SCOPUS:85130462555
VL - 185
SP - 955
EP - 980
JO - Journal of the Royal Statistical Society. Series A: Statistics in Society
JF - Journal of the Royal Statistical Society. Series A: Statistics in Society
SN - 0964-1998
IS - 3
ER -