Details
| Original language | English |
|---|---|
| Article number | 99 |
| Journal | Empirical software engineering |
| Volume | 27 |
| Issue number | 4 |
| Early online date | 7 May 2022 |
| Publication status | Published - Jul 2022 |
Abstract
Code review is an important process in software engineering – yet, a very expensive one. Therefore, understanding code review and how to improve reviewers’ performance is paramount. In the study presented in this work, we test whether providing developers with explicit reviewing strategies improves their review effectiveness and efficiency. Moreover, we verify if review guidance lowers developers’ cognitive load. We employ an experimental design where professional developers have to perform three code review tasks. Participants are assigned to one of three treatments: ad hoc reviewing, checklist, and guided checklist. The guided checklist was developed to provide an explicit reviewing strategy to developers. While the checklist is a simple form of signaling (a method to reduce cognitive load), the guided checklist incorporates further methods to lower cognitive demands of the task such as segmenting and weeding. The majority of the participants are novice reviewers with low or no code review experience. Our results indicate that the guided checklist is a more effective aid for a simple review,while the checklist supports reviewers’ efficiency and effectiveness in a complex task. However, we did not identify a strong relationship between the guidance provided and code review performance. The checklist has the potential to lower developers’ cognitive load, but higher cognitive load led to better performance possibly due to the generally low effectiveness and efficiency of the study participants. Data and materials: https://doi.org/10.5281/zenodo.5653341. Registered report: https://doi.org/10.17605/OSF.IO/5FPTJ.
Keywords
- Checklist-based reading, Code review, Cognitive load, Reviewing strategies
ASJC Scopus subject areas
- Computer Science(all)
- Software
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
In: Empirical software engineering, Vol. 27, No. 4, 99, 07.2022.
Research output: Contribution to journal › Article › Research › peer review
}
TY - JOUR
T1 - Do explicit review strategies improve code review performance?
T2 - Towards understanding the role of cognitive load
AU - Gonçalves, Pavlína Wurzel
AU - Fregnan, Enrico
AU - Baum, Tobias
AU - Schneider, Kurt
AU - Bacchelli, Alberto
N1 - Funding Information: Open access funding provided by University of Zurich. P. Wurzel Gonçalves, E. Fregnan, and A. Bacchelli gratefully acknowledge the support of the Swiss National Science Foundation through the SNF Projects No. PP00P2_170529.
PY - 2022/7
Y1 - 2022/7
N2 - Code review is an important process in software engineering – yet, a very expensive one. Therefore, understanding code review and how to improve reviewers’ performance is paramount. In the study presented in this work, we test whether providing developers with explicit reviewing strategies improves their review effectiveness and efficiency. Moreover, we verify if review guidance lowers developers’ cognitive load. We employ an experimental design where professional developers have to perform three code review tasks. Participants are assigned to one of three treatments: ad hoc reviewing, checklist, and guided checklist. The guided checklist was developed to provide an explicit reviewing strategy to developers. While the checklist is a simple form of signaling (a method to reduce cognitive load), the guided checklist incorporates further methods to lower cognitive demands of the task such as segmenting and weeding. The majority of the participants are novice reviewers with low or no code review experience. Our results indicate that the guided checklist is a more effective aid for a simple review,while the checklist supports reviewers’ efficiency and effectiveness in a complex task. However, we did not identify a strong relationship between the guidance provided and code review performance. The checklist has the potential to lower developers’ cognitive load, but higher cognitive load led to better performance possibly due to the generally low effectiveness and efficiency of the study participants. Data and materials: https://doi.org/10.5281/zenodo.5653341. Registered report: https://doi.org/10.17605/OSF.IO/5FPTJ.
AB - Code review is an important process in software engineering – yet, a very expensive one. Therefore, understanding code review and how to improve reviewers’ performance is paramount. In the study presented in this work, we test whether providing developers with explicit reviewing strategies improves their review effectiveness and efficiency. Moreover, we verify if review guidance lowers developers’ cognitive load. We employ an experimental design where professional developers have to perform three code review tasks. Participants are assigned to one of three treatments: ad hoc reviewing, checklist, and guided checklist. The guided checklist was developed to provide an explicit reviewing strategy to developers. While the checklist is a simple form of signaling (a method to reduce cognitive load), the guided checklist incorporates further methods to lower cognitive demands of the task such as segmenting and weeding. The majority of the participants are novice reviewers with low or no code review experience. Our results indicate that the guided checklist is a more effective aid for a simple review,while the checklist supports reviewers’ efficiency and effectiveness in a complex task. However, we did not identify a strong relationship between the guidance provided and code review performance. The checklist has the potential to lower developers’ cognitive load, but higher cognitive load led to better performance possibly due to the generally low effectiveness and efficiency of the study participants. Data and materials: https://doi.org/10.5281/zenodo.5653341. Registered report: https://doi.org/10.17605/OSF.IO/5FPTJ.
KW - Checklist-based reading
KW - Code review
KW - Cognitive load
KW - Reviewing strategies
UR - http://www.scopus.com/inward/record.url?scp=85129584432&partnerID=8YFLogxK
U2 - 10.1007/s10664-022-10123-8
DO - 10.1007/s10664-022-10123-8
M3 - Article
AN - SCOPUS:85129584432
VL - 27
JO - Empirical software engineering
JF - Empirical software engineering
SN - 1382-3256
IS - 4
M1 - 99
ER -