TY - JOUR

T1 - A new experiment on the use of images to answer web survey questions

AU - Bosch, Oriol J.

AU - Revilla, Melanie

AU - Qureshi, Danish Daniel

AU - Höhne, Jan Karem

N1 - Funding Information: German Science Foundation, through the Collaborative Research Center 884 “Political Economyof Reforms”, 139943784; European Research Council (ERC) under the European Unions Horizon 2020 research and innovation programme, 849165 Funding information

PY - 2022/5/20

Y1 - 2022/5/20

N2 - Images might provide richer and more objective information than text answers to open-ended survey questions. Little is known, nonetheless, about the consequences for data quality of asking participants to answer open-ended questions with images. Therefore, this paper addresses three research questions: (1) What is the effect of answering web survey questions with images instead of text on breakoff, noncompliance with the task, completion time and question evaluation? (2) What is the effect of including a motivational message on these four aspects? (3) Does the impact of asking to answer with images instead of text vary across device types? To answer these questions, we implemented a 2 × 3 between-subject web survey experiment (N = 3043) in Germany. Half of the sample was required to answer using PCs and the other half with smartphones. Within each device group, respondents were randomly assigned to (1) a control group answering open-ended questions with text; (2) a treatment group answering open-ended questions with images; and (3) another treatment group answering open-ended questions with images but prompted with a motivational message. Results show that asking participants to answer with images significantly increases participants' likelihood of noncompliance as well as their completion times, while worsening their overall survey experience. Including motivational messages, moreover, moderately reduces the likelihood of noncompliance. Finally, the likelihood of noncompliance is similar across devices.

AB - Images might provide richer and more objective information than text answers to open-ended survey questions. Little is known, nonetheless, about the consequences for data quality of asking participants to answer open-ended questions with images. Therefore, this paper addresses three research questions: (1) What is the effect of answering web survey questions with images instead of text on breakoff, noncompliance with the task, completion time and question evaluation? (2) What is the effect of including a motivational message on these four aspects? (3) Does the impact of asking to answer with images instead of text vary across device types? To answer these questions, we implemented a 2 × 3 between-subject web survey experiment (N = 3043) in Germany. Half of the sample was required to answer using PCs and the other half with smartphones. Within each device group, respondents were randomly assigned to (1) a control group answering open-ended questions with text; (2) a treatment group answering open-ended questions with images; and (3) another treatment group answering open-ended questions with images but prompted with a motivational message. Results show that asking participants to answer with images significantly increases participants' likelihood of noncompliance as well as their completion times, while worsening their overall survey experience. Including motivational messages, moreover, moderately reduces the likelihood of noncompliance. Finally, the likelihood of noncompliance is similar across devices.

KW - breakoff

KW - images

KW - motivational messages

KW - noncompliance

KW - smartphone

KW - web survey

UR - http://www.scopus.com/inward/record.url?scp=85130462555&partnerID=8YFLogxK

U2 - 10.1111/rssa.12856

DO - 10.1111/rssa.12856

M3 - Article

AN - SCOPUS:85130462555

VL - 185

SP - 955

EP - 980

JO - Journal of the Royal Statistical Society. Series A: Statistics in Society

JF - Journal of the Royal Statistical Society. Series A: Statistics in Society

SN - 0964-1998

IS - 3

ER -