TY - JOUR

T1 - Data quality, experimental artifacts, and the reactivity of the psychological subject matter

AU - Feest, Uljana

N1 - Funding Information: I would like to thank Julie Zahle, Caterina Marchionni, and Marion Godman, as well as members of the audience at the workshop, “Reactivity in the Research Process” (Bergen, February 2020) for an inspiring workshop and helpful discussions. More recent versions of this paper were presented at Hannover University (April 2020), the Max Planck Cognition Academy (January 2021), the Ghent/Brussels Work-In-Progress-Colloquium (March 2021), and the philosophy colloquium at the University of Warsaw (April 2021). Many thanks to participants of each of these events. I am also grateful to Duygu Uygun Tunç as well as the two extremely helpful referees for this journal for their thorough and constructive feedback. Finally, and as always, the work of Jim Bogen and Jim Woodward has been invaluable in shaping the analyses presented here (though, of course, any errors and misunderstandings are all mine).

PY - 2022/2/3

Y1 - 2022/2/3

N2 - While the term “reactivity” has come to be associated with specific phenomena in the social sciences, having to do with subjects’ awareness of being studied, this paper takes a broader stance on this concept. I argue that reactivity is a ubiquitous feature of the psychological subject matter and that this fact is a precondition of experimental research, while also posing potential problems for the experimenter. The latter are connected to the worry about distorted data and experimental artifacts. But what are experimental artifacts and what is the most productive way of dealing with them? In this paper, I approach these questions by exploring the ways in which experimenters in psychology simultaneously exploit and suppress the reactivity of their subject matter in order to produce experimental data that speak to the question or subject matter at hand. Highlighting the artificiality of experimental data, I raise (and answer) the question of what distinguishes a genuine experimental result from an experimental artifact. My analysis construes experimental results as the outcomes of inferences from the data that take material background assumptions as auxiliary premises. Artifacts occur when one or more of these background assumptions are false, such that the data do not reliably serve the purposes they were generated for. I conclude by laying out the ways in which my analysis of data quality is relevant to, and informed by, recent debates about the replicability of experimental results.

AB - While the term “reactivity” has come to be associated with specific phenomena in the social sciences, having to do with subjects’ awareness of being studied, this paper takes a broader stance on this concept. I argue that reactivity is a ubiquitous feature of the psychological subject matter and that this fact is a precondition of experimental research, while also posing potential problems for the experimenter. The latter are connected to the worry about distorted data and experimental artifacts. But what are experimental artifacts and what is the most productive way of dealing with them? In this paper, I approach these questions by exploring the ways in which experimenters in psychology simultaneously exploit and suppress the reactivity of their subject matter in order to produce experimental data that speak to the question or subject matter at hand. Highlighting the artificiality of experimental data, I raise (and answer) the question of what distinguishes a genuine experimental result from an experimental artifact. My analysis construes experimental results as the outcomes of inferences from the data that take material background assumptions as auxiliary premises. Artifacts occur when one or more of these background assumptions are false, such that the data do not reliably serve the purposes they were generated for. I conclude by laying out the ways in which my analysis of data quality is relevant to, and informed by, recent debates about the replicability of experimental results.

KW - Data reliability

KW - Experimental artifacts

KW - Experimental inferences

KW - Philosophy of data

KW - Philosophy of psychology

KW - Reactivity

KW - Replication crisis

UR - http://www.scopus.com/inward/record.url?scp=85124412059&partnerID=8YFLogxK

U2 - 10.1007/s13194-021-00443-9

DO - 10.1007/s13194-021-00443-9

M3 - Article

AN - SCOPUS:85124412059

VL - 12

JO - European Journal for Philosophy of Science

JF - European Journal for Philosophy of Science

SN - 1879-4912

IS - 1

M1 - 13

ER -