Investigating direction effects in rating scales with five and seven points in a probability-based online panel

Publikation: Beitrag in FachzeitschriftArtikelForschungPeer-Review

Autoren

  • Jan Karem Höhne
  • Dagmar Krebs
  • Steffen M. Kühnel

Organisationseinheiten

Externe Organisationen

  • Universität Pompeu Fabra (UPF)
  • Justus-Liebig-Universität Gießen
  • Georg-August-Universität Göttingen
  • Deutsches Zentrum für Hochschul- und Wissenschaftsforschung (DZHW)
Forschungs-netzwerk anzeigen

Details

OriginalspracheEnglisch
Seiten (von - bis)193-204
Seitenumfang12
FachzeitschriftSurvey research methods
Jahrgang17
Ausgabenummer2
PublikationsstatusVeröffentlicht - 8 Aug. 2023

Abstract

Survey questions with rating scales are a common method in attitude measurement. Similar to other scale characteristics, scale direction and its effects on answer behavior and data quality deserves special attention. This particularly applies to scale direction effects across different scale lengths. In order to contribute to the current state of research, we investigate scale direction effects across scales with five and seven points by analyzing observed and latent answer distributions including composite reliabilities. We conducted an experiment in the probability-based German Internet Panel (N = 4676) using questions on achievement and job motivation that vary scale direction (i.e., decremental and incremental) and scale length (i.e., five and seven points). The results reveal differences between scales with five and seven points. Five-point scales are more robust against scale direction effects than their seven-point counterparts. In ad-dition, decremental and incremental scales with five points are invariant. This does not apply to decremental and incremental scales with seven points. However, composite reliabilities are higher for scales with seven points than for scales with five points. This is irrespective of the scale direction.

ASJC Scopus Sachgebiete

Zitieren

Investigating direction effects in rating scales with five and seven points in a probability-based online panel. / Höhne, Jan Karem; Krebs, Dagmar; Kühnel, Steffen M.
in: Survey research methods, Jahrgang 17, Nr. 2, 08.08.2023, S. 193-204.

Publikation: Beitrag in FachzeitschriftArtikelForschungPeer-Review

Höhne JK, Krebs D, Kühnel SM. Investigating direction effects in rating scales with five and seven points in a probability-based online panel. Survey research methods. 2023 Aug 8;17(2):193-204. doi: 10.18148/srm/2023.v17i2.8006
Höhne, Jan Karem ; Krebs, Dagmar ; Kühnel, Steffen M. / Investigating direction effects in rating scales with five and seven points in a probability-based online panel. in: Survey research methods. 2023 ; Jahrgang 17, Nr. 2. S. 193-204.
Download
@article{2c01b0f99afb4400b2530d9c3e3f3e69,
title = "Investigating direction effects in rating scales with five and seven points in a probability-based online panel",
abstract = "Survey questions with rating scales are a common method in attitude measurement. Similar to other scale characteristics, scale direction and its effects on answer behavior and data quality deserves special attention. This particularly applies to scale direction effects across different scale lengths. In order to contribute to the current state of research, we investigate scale direction effects across scales with five and seven points by analyzing observed and latent answer distributions including composite reliabilities. We conducted an experiment in the probability-based German Internet Panel (N = 4676) using questions on achievement and job motivation that vary scale direction (i.e., decremental and incremental) and scale length (i.e., five and seven points). The results reveal differences between scales with five and seven points. Five-point scales are more robust against scale direction effects than their seven-point counterparts. In ad-dition, decremental and incremental scales with five points are invariant. This does not apply to decremental and incremental scales with seven points. However, composite reliabilities are higher for scales with seven points than for scales with five points. This is irrespective of the scale direction.",
keywords = "Composite reliabilities, Data quality, Latent means, Measurement invariance, Online survey, Scale direction effects, Survey-satisficing",
author = "H{\"o}hne, {Jan Karem} and Dagmar Krebs and K{\"u}hnel, {Steffen M.}",
year = "2023",
month = aug,
day = "8",
doi = "10.18148/srm/2023.v17i2.8006",
language = "English",
volume = "17",
pages = "193--204",
journal = "Survey research methods",
issn = "1864-3361",
publisher = "European Survey Research Association",
number = "2",

}

Download

TY - JOUR

T1 - Investigating direction effects in rating scales with five and seven points in a probability-based online panel

AU - Höhne, Jan Karem

AU - Krebs, Dagmar

AU - Kühnel, Steffen M.

PY - 2023/8/8

Y1 - 2023/8/8

N2 - Survey questions with rating scales are a common method in attitude measurement. Similar to other scale characteristics, scale direction and its effects on answer behavior and data quality deserves special attention. This particularly applies to scale direction effects across different scale lengths. In order to contribute to the current state of research, we investigate scale direction effects across scales with five and seven points by analyzing observed and latent answer distributions including composite reliabilities. We conducted an experiment in the probability-based German Internet Panel (N = 4676) using questions on achievement and job motivation that vary scale direction (i.e., decremental and incremental) and scale length (i.e., five and seven points). The results reveal differences between scales with five and seven points. Five-point scales are more robust against scale direction effects than their seven-point counterparts. In ad-dition, decremental and incremental scales with five points are invariant. This does not apply to decremental and incremental scales with seven points. However, composite reliabilities are higher for scales with seven points than for scales with five points. This is irrespective of the scale direction.

AB - Survey questions with rating scales are a common method in attitude measurement. Similar to other scale characteristics, scale direction and its effects on answer behavior and data quality deserves special attention. This particularly applies to scale direction effects across different scale lengths. In order to contribute to the current state of research, we investigate scale direction effects across scales with five and seven points by analyzing observed and latent answer distributions including composite reliabilities. We conducted an experiment in the probability-based German Internet Panel (N = 4676) using questions on achievement and job motivation that vary scale direction (i.e., decremental and incremental) and scale length (i.e., five and seven points). The results reveal differences between scales with five and seven points. Five-point scales are more robust against scale direction effects than their seven-point counterparts. In ad-dition, decremental and incremental scales with five points are invariant. This does not apply to decremental and incremental scales with seven points. However, composite reliabilities are higher for scales with seven points than for scales with five points. This is irrespective of the scale direction.

KW - Composite reliabilities

KW - Data quality

KW - Latent means

KW - Measurement invariance

KW - Online survey

KW - Scale direction effects

KW - Survey-satisficing

UR - http://www.scopus.com/inward/record.url?scp=85167903913&partnerID=8YFLogxK

U2 - 10.18148/srm/2023.v17i2.8006

DO - 10.18148/srm/2023.v17i2.8006

M3 - Article

AN - SCOPUS:85167903913

VL - 17

SP - 193

EP - 204

JO - Survey research methods

JF - Survey research methods

SN - 1864-3361

IS - 2

ER -