TY - JOUR

T1 - Measuring Income (In)equality

T2 - Comparing Survey Questions With Unipolar and Bipolar Scales in a Probability-Based Online Panel

AU - Höhne, Jan Karem

AU - Krebs, Dagmar

AU - Kühnel, Steffen M.

N1 - Funding Information: The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: We acknowledge financial support by the German Science Foundation through the Collaborative Research Center 884 “Political Economy of Reforms” at the University of Mannheim (Germany).

PY - 2022/2

Y1 - 2022/2

N2 - In social science research, unipolar and bipolar scales are commonly used methods in measuring respondents’ attitudes and opinions. Compared to other rating scale characteristics, scale polarity (unipolar and bipolar) and its effects on response behavior have rarely been addressed in previous research. To fill this gap in the literature, we investigate whether and to what extent fully verbalized unipolar and bipolar scales influence response behavior by analyzing observed and latent response distributions and latent thresholds of response categories. For this purpose, we conducted a survey experiment in a probability-based online panel and randomly assigned respondents to a unipolar or bipolar scale condition. The results reveal substantial differences between the two rating scales. They show significantly different response distributions and measurement non-invariance. In addition, response categories (and latent thresholds) of unipolar and bipolar scales are not equally distributed. The findings show that responses to unipolar and bipolar scales differ not only on the observational level but also on the latent level. Both rating scales vary with respect to their measurement properties, so that the responses obtained using each scale are not easily comparable. We recommend not considering unipolar and bipolar scales as interchangeable.

AB - In social science research, unipolar and bipolar scales are commonly used methods in measuring respondents’ attitudes and opinions. Compared to other rating scale characteristics, scale polarity (unipolar and bipolar) and its effects on response behavior have rarely been addressed in previous research. To fill this gap in the literature, we investigate whether and to what extent fully verbalized unipolar and bipolar scales influence response behavior by analyzing observed and latent response distributions and latent thresholds of response categories. For this purpose, we conducted a survey experiment in a probability-based online panel and randomly assigned respondents to a unipolar or bipolar scale condition. The results reveal substantial differences between the two rating scales. They show significantly different response distributions and measurement non-invariance. In addition, response categories (and latent thresholds) of unipolar and bipolar scales are not equally distributed. The findings show that responses to unipolar and bipolar scales differ not only on the observational level but also on the latent level. Both rating scales vary with respect to their measurement properties, so that the responses obtained using each scale are not easily comparable. We recommend not considering unipolar and bipolar scales as interchangeable.

KW - item response theory

KW - latent thresholds

KW - measurement invariance

KW - online survey

KW - rating scales

KW - scale polarity

UR - http://www.scopus.com/inward/record.url?scp=85081597577&partnerID=8YFLogxK

U2 - 10.1177/0894439320902461

DO - 10.1177/0894439320902461

M3 - Article

AN - SCOPUS:85081597577

VL - 40

SP - 108

EP - 123

JO - Social science computer review

JF - Social science computer review

SN - 0894-4393

IS - 1

ER -