Details
| Original language | English |
|---|---|
| Pages (from-to) | 2125-2146 |
| Number of pages | 22 |
| Journal | Quality and Quantity |
| Volume | 57 |
| Issue number | 3 |
| Early online date | 20 Jun 2022 |
| Publication status | Published - Jun 2023 |
| Externally published | Yes |
Abstract
A large body of studies estimate the measurement quality of survey questions using multitrait-multimethod (MTMM) experiments. In these experiments, respondents are asked (nearly) identical questions at least twice. Most commonly, this is done within the same survey by, for instance, changing the response scale characteristics. However, it has frequently been argued that the estimates from these experiments might be biased when respondents recall their first answer and base their second answer on the initial one. So far, only little is known about the impact of memory effects on the estimates of measurement quality. In this study, we conducted a MTMM experiment in the probability-based German Internet Panel (N = 4,401) to investigate whether and to what extent measurement quality estimates differ across respondent groups varying in terms of recall. For this purpose, we use a survey question on trust in the German parliament that was asked with three different response scales. The results reveal that the recall of answers and memory effects vary significantly depending on the scales used for the trust in the parliament question. We also found significant differences in measurement quality across different recall groups. However, more refined research is necessary to better understand the association between answer recall and measurement quality estimates.
Keywords
- Measurement quality, Memory effects, Multitrait-multimethod (MTMM), Probability-based online panel, Recall of survey answers, Trust in the parliament
ASJC Scopus subject areas
- Mathematics(all)
- Statistics and Probability
- Social Sciences(all)
- General Social Sciences
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
In: Quality and Quantity, Vol. 57, No. 3, 06.2023, p. 2125-2146.
Research output: Contribution to journal › Article › Research › peer review
}
TY - JOUR
T1 - Differences in measurement quality depending on recall: results for a question about trust in the parliament
AU - Revilla, Melanie
AU - Höhne, Jan Karem
AU - Rettig, Tobias
N1 - Funding Information: Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. This article uses data from wave 51 of the German Internet Panel (Blom et al. ). A study description can be found in Blom et al. (). The German Internet Panel is part of the Collaborative Research Center 884 (SFB 884) funded by the German Research Foundation (DFG)—Project Number 139943784–SFB 884.
PY - 2023/6
Y1 - 2023/6
N2 - A large body of studies estimate the measurement quality of survey questions using multitrait-multimethod (MTMM) experiments. In these experiments, respondents are asked (nearly) identical questions at least twice. Most commonly, this is done within the same survey by, for instance, changing the response scale characteristics. However, it has frequently been argued that the estimates from these experiments might be biased when respondents recall their first answer and base their second answer on the initial one. So far, only little is known about the impact of memory effects on the estimates of measurement quality. In this study, we conducted a MTMM experiment in the probability-based German Internet Panel (N = 4,401) to investigate whether and to what extent measurement quality estimates differ across respondent groups varying in terms of recall. For this purpose, we use a survey question on trust in the German parliament that was asked with three different response scales. The results reveal that the recall of answers and memory effects vary significantly depending on the scales used for the trust in the parliament question. We also found significant differences in measurement quality across different recall groups. However, more refined research is necessary to better understand the association between answer recall and measurement quality estimates.
AB - A large body of studies estimate the measurement quality of survey questions using multitrait-multimethod (MTMM) experiments. In these experiments, respondents are asked (nearly) identical questions at least twice. Most commonly, this is done within the same survey by, for instance, changing the response scale characteristics. However, it has frequently been argued that the estimates from these experiments might be biased when respondents recall their first answer and base their second answer on the initial one. So far, only little is known about the impact of memory effects on the estimates of measurement quality. In this study, we conducted a MTMM experiment in the probability-based German Internet Panel (N = 4,401) to investigate whether and to what extent measurement quality estimates differ across respondent groups varying in terms of recall. For this purpose, we use a survey question on trust in the German parliament that was asked with three different response scales. The results reveal that the recall of answers and memory effects vary significantly depending on the scales used for the trust in the parliament question. We also found significant differences in measurement quality across different recall groups. However, more refined research is necessary to better understand the association between answer recall and measurement quality estimates.
KW - Measurement quality
KW - Memory effects
KW - Multitrait-multimethod (MTMM)
KW - Probability-based online panel
KW - Recall of survey answers
KW - Trust in the parliament
UR - http://www.scopus.com/inward/record.url?scp=85132151436&partnerID=8YFLogxK
U2 - 10.1007/s11135-022-01441-6
DO - 10.1007/s11135-022-01441-6
M3 - Article
AN - SCOPUS:85132151436
VL - 57
SP - 2125
EP - 2146
JO - Quality and Quantity
JF - Quality and Quantity
SN - 0033-5177
IS - 3
ER -