TY - JOUR

T1 - Innovating the collection of open-ended answers

T2 - The linguistic and content characteristics of written and oral answers to political attitude questions

AU - Gavras, Konstantin

AU - Höhne, Jan Karem

AU - Blom, Annelies G.

AU - Schoen, Harald

N1 - Funding Information: The authors thank Daniel Qureshi (University of Frankfurt) and Stephan Schlosser (University of Göttingen) for their support during data collection. The data collection was funded by the German Research Foundation (Deutsche Forschungsgemeinschaft, DFG) through the Collaborative Research Center 884 ‘Political Economy of Reforms’ (SFB 884; project ID: 139943784). We thank the Board of the SFB 884 for supporting our research. Open access funding was enabled and organized by ProjektDEAL.

PY - 2022/7

Y1 - 2022/7

N2 - The rapid increase in smartphone surveys and technological developments open novel opportunities for collecting survey answers. One of these opportunities is the use of open-ended questions with requests for oral instead of written answers, which may facilitate the answer process and result in more in-depth and unfiltered information. Whereas it is now possible to collect oral answers on smartphones, we still lack studies on the impact of this novel answer format on the characteristics of respondents' answers. In this study, we compare the linguistic and content characteristics of written versus oral answers to political attitude questions. For this purpose, we conducted an experiment in a smartphone survey (N = 2402) and randomly assigned respondents to an answer format (written or oral). Oral answers were collected via the open source ‘SurveyVoice (SVoice)’ tool, whereas written answers were typed in via the smartphone keypad. Applying length analysis, lexical structure analysis, sentiment analysis and structural topic models, our results reveal that written and oral answers differ substantially from each other in terms of lengths, structures, sentiments and topics. We find evidence that written answers are characterized by an intentional and conscious answering, whereas oral answers are characterized by an intuitive and spontaneous answering.

AB - The rapid increase in smartphone surveys and technological developments open novel opportunities for collecting survey answers. One of these opportunities is the use of open-ended questions with requests for oral instead of written answers, which may facilitate the answer process and result in more in-depth and unfiltered information. Whereas it is now possible to collect oral answers on smartphones, we still lack studies on the impact of this novel answer format on the characteristics of respondents' answers. In this study, we compare the linguistic and content characteristics of written versus oral answers to political attitude questions. For this purpose, we conducted an experiment in a smartphone survey (N = 2402) and randomly assigned respondents to an answer format (written or oral). Oral answers were collected via the open source ‘SurveyVoice (SVoice)’ tool, whereas written answers were typed in via the smartphone keypad. Applying length analysis, lexical structure analysis, sentiment analysis and structural topic models, our results reveal that written and oral answers differ substantially from each other in terms of lengths, structures, sentiments and topics. We find evidence that written answers are characterized by an intentional and conscious answering, whereas oral answers are characterized by an intuitive and spontaneous answering.

KW - open-ended questions

KW - political attitudes

KW - sentiment analysis

KW - smartphone surveys

KW - structural topic modelling

KW - text data

KW - voice data

UR - http://www.scopus.com/inward/record.url?scp=85125065992&partnerID=8YFLogxK

U2 - 10.1111/rssa.12807

DO - 10.1111/rssa.12807

M3 - Article

AN - SCOPUS:85125065992

VL - 185

SP - 872

EP - 890

JO - Journal of the Royal Statistical Society. Series A: Statistics in Society

JF - Journal of the Royal Statistical Society. Series A: Statistics in Society

SN - 0964-1998

IS - 3

ER -