TY - JOUR

T1 - Differences in measurement quality depending on recall: results for a question about trust in the parliament

AU - Revilla, Melanie

AU - Höhne, Jan Karem

AU - Rettig, Tobias

N1 - Funding Information: Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. This article uses data from wave 51 of the German Internet Panel (Blom et al. ). A study description can be found in Blom et al. (). The German Internet Panel is part of the Collaborative Research Center 884 (SFB 884) funded by the German Research Foundation (DFG)—Project Number 139943784–SFB 884.

PY - 2023/6

Y1 - 2023/6

N2 - A large body of studies estimate the measurement quality of survey questions using multitrait-multimethod (MTMM) experiments. In these experiments, respondents are asked (nearly) identical questions at least twice. Most commonly, this is done within the same survey by, for instance, changing the response scale characteristics. However, it has frequently been argued that the estimates from these experiments might be biased when respondents recall their first answer and base their second answer on the initial one. So far, only little is known about the impact of memory effects on the estimates of measurement quality. In this study, we conducted a MTMM experiment in the probability-based German Internet Panel (N = 4,401) to investigate whether and to what extent measurement quality estimates differ across respondent groups varying in terms of recall. For this purpose, we use a survey question on trust in the German parliament that was asked with three different response scales. The results reveal that the recall of answers and memory effects vary significantly depending on the scales used for the trust in the parliament question. We also found significant differences in measurement quality across different recall groups. However, more refined research is necessary to better understand the association between answer recall and measurement quality estimates.

AB - A large body of studies estimate the measurement quality of survey questions using multitrait-multimethod (MTMM) experiments. In these experiments, respondents are asked (nearly) identical questions at least twice. Most commonly, this is done within the same survey by, for instance, changing the response scale characteristics. However, it has frequently been argued that the estimates from these experiments might be biased when respondents recall their first answer and base their second answer on the initial one. So far, only little is known about the impact of memory effects on the estimates of measurement quality. In this study, we conducted a MTMM experiment in the probability-based German Internet Panel (N = 4,401) to investigate whether and to what extent measurement quality estimates differ across respondent groups varying in terms of recall. For this purpose, we use a survey question on trust in the German parliament that was asked with three different response scales. The results reveal that the recall of answers and memory effects vary significantly depending on the scales used for the trust in the parliament question. We also found significant differences in measurement quality across different recall groups. However, more refined research is necessary to better understand the association between answer recall and measurement quality estimates.

KW - Measurement quality

KW - Memory effects

KW - Multitrait-multimethod (MTMM)

KW - Probability-based online panel

KW - Recall of survey answers

KW - Trust in the parliament

UR - http://www.scopus.com/inward/record.url?scp=85132151436&partnerID=8YFLogxK

U2 - 10.1007/s11135-022-01441-6

DO - 10.1007/s11135-022-01441-6

M3 - Article

AN - SCOPUS:85132151436

VL - 57

SP - 2125

EP - 2146

JO - Quality and Quantity

JF - Quality and Quantity

SN - 0033-5177

IS - 3

ER -