Details
| Original language | English |
|---|---|
| Pages (from-to) | 623-634 |
| Number of pages | 12 |
| Journal | Research evaluation |
| Volume | 32 |
| Issue number | 4 |
| Publication status | Published - 23 Oct 2023 |
Abstract
Peer review has developed over time to become the established procedure for assessing and assuring the scientific quality of research. Nevertheless, the procedure has also been variously criticized as conservative, biased, and unfair, among other things. Do scientists regard all these flaws as equally problematic? Do they have the same opinions on which problems are so serious that other selection procedures ought to be considered? The answers to these questions hints at what should be modified in peer review processes as a priority objective. The authors of this paper use survey data to examine how members of the scientific community weight different shortcomings of peer review processes. Which of those processes’ problems do they consider less relevant? Which problems, on the other hand, do they judge to be beyond remedy? Our investigation shows that certain defects of peer review processes are indeed deemed irreparable: (1) legitimate quandaries in the process of fine-tuning the choice between equally eligible research proposals and in the selection of daring ideas; and (2) illegitimate problems due to networks. Science-policy measures to improve peer review processes should therefore clarify the distinction between field-specific remediable and irremediable flaws than is currently the case.
Keywords
- field of science, peer review, problems, randomization, research grants
ASJC Scopus subject areas
- Social Sciences(all)
- Education
- Social Sciences(all)
- Library and Information Sciences
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
In: Research evaluation, Vol. 32, No. 4, 23.10.2023, p. 623-634.
Research output: Contribution to journal › Article › Research › peer review
}
TY - JOUR
T1 - Peer review’s irremediable flaws
T2 - Scientists’ perspectives on grant evaluation in Germany
AU - Barlösius, Eva
AU - Paruschke, Laura
AU - Philipps, Axel
N1 - Funding Information: This work was supported by the Bundesministerium für Bildung und Forschung grant number: 01PW18004.
PY - 2023/10/23
Y1 - 2023/10/23
N2 - Peer review has developed over time to become the established procedure for assessing and assuring the scientific quality of research. Nevertheless, the procedure has also been variously criticized as conservative, biased, and unfair, among other things. Do scientists regard all these flaws as equally problematic? Do they have the same opinions on which problems are so serious that other selection procedures ought to be considered? The answers to these questions hints at what should be modified in peer review processes as a priority objective. The authors of this paper use survey data to examine how members of the scientific community weight different shortcomings of peer review processes. Which of those processes’ problems do they consider less relevant? Which problems, on the other hand, do they judge to be beyond remedy? Our investigation shows that certain defects of peer review processes are indeed deemed irreparable: (1) legitimate quandaries in the process of fine-tuning the choice between equally eligible research proposals and in the selection of daring ideas; and (2) illegitimate problems due to networks. Science-policy measures to improve peer review processes should therefore clarify the distinction between field-specific remediable and irremediable flaws than is currently the case.
AB - Peer review has developed over time to become the established procedure for assessing and assuring the scientific quality of research. Nevertheless, the procedure has also been variously criticized as conservative, biased, and unfair, among other things. Do scientists regard all these flaws as equally problematic? Do they have the same opinions on which problems are so serious that other selection procedures ought to be considered? The answers to these questions hints at what should be modified in peer review processes as a priority objective. The authors of this paper use survey data to examine how members of the scientific community weight different shortcomings of peer review processes. Which of those processes’ problems do they consider less relevant? Which problems, on the other hand, do they judge to be beyond remedy? Our investigation shows that certain defects of peer review processes are indeed deemed irreparable: (1) legitimate quandaries in the process of fine-tuning the choice between equally eligible research proposals and in the selection of daring ideas; and (2) illegitimate problems due to networks. Science-policy measures to improve peer review processes should therefore clarify the distinction between field-specific remediable and irremediable flaws than is currently the case.
KW - field of science
KW - peer review
KW - problems
KW - randomization
KW - research grants
UR - http://www.scopus.com/inward/record.url?scp=85183950789&partnerID=8YFLogxK
U2 - 10.1093/reseval/rvad032
DO - 10.1093/reseval/rvad032
M3 - Article
AN - SCOPUS:85183950789
VL - 32
SP - 623
EP - 634
JO - Research evaluation
JF - Research evaluation
SN - 0958-2029
IS - 4
ER -