Details
| Original language | English |
|---|---|
| Pages (from-to) | 1762-1798 |
| Number of pages | 37 |
| Journal | Empirical Software Engineering |
| Volume | 24 |
| Issue number | 4 |
| Early online date | 2 Jan 2019 |
| Publication status | Published - 15 Aug 2019 |
Abstract
Change-based code review is a software quality assurance technique that is widely used in practice. Therefore, better understanding what influences performance in code reviews and finding ways to improve it can have a large impact. In this study, we examine the association of working memory capacity and cognitive load with code review performance and we test the predictions of a recent theory regarding improved code review efficiency with certain code change part orders. We perform a confirmatory experiment with 50 participants, mostly professional software developers. The participants performed code reviews on one small and two larger code changes from an open source software system to which we had seeded additional defects. We measured their efficiency and effectiveness in defect detection, their working memory capacity, and several potential confounding factors. We find that there is a moderate association between working memory capacity and the effectiveness of finding delocalized defects, influenced by other factors, whereas the association with other defect types is almost non-existing. We also confirm that the effectiveness of reviews is significantly larger for small code changes. We cannot conclude reliably whether the order of presenting the code change parts influences the efficiency of code review.
Keywords
- Change-based code review, Code ordering, Cognitive load, Cognitive support, Individual differences, Working memory
ASJC Scopus subject areas
- Computer Science(all)
- Software
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
In: Empirical Software Engineering, Vol. 24, No. 4, 15.08.2019, p. 1762-1798.
Research output: Contribution to journal › Article › Research › peer review
}
TY - JOUR
T1 - Associating working memory capacity and code change ordering with code review performance
AU - Baum, Tobias
AU - Schneider, Kurt
AU - Bacchelli, Alberto
N1 - Funding Information: We thank all participants and all pre-testers for the time and effort they donated. We furthermore thank Sylvie Gasnier and G?nter Faber for advice on the statistical procedures and Javad Ghofrani for help with double-checking the defect coding. We thank Bettina von Helversen from the psychology department at the University of Zurich for advice on the parts related to the theory of cognitive load. Bacchelli gratefully acknowledges the support of the Swiss National Science Foundation through the SNF Project No. PP00P2_170529.
PY - 2019/8/15
Y1 - 2019/8/15
N2 - Change-based code review is a software quality assurance technique that is widely used in practice. Therefore, better understanding what influences performance in code reviews and finding ways to improve it can have a large impact. In this study, we examine the association of working memory capacity and cognitive load with code review performance and we test the predictions of a recent theory regarding improved code review efficiency with certain code change part orders. We perform a confirmatory experiment with 50 participants, mostly professional software developers. The participants performed code reviews on one small and two larger code changes from an open source software system to which we had seeded additional defects. We measured their efficiency and effectiveness in defect detection, their working memory capacity, and several potential confounding factors. We find that there is a moderate association between working memory capacity and the effectiveness of finding delocalized defects, influenced by other factors, whereas the association with other defect types is almost non-existing. We also confirm that the effectiveness of reviews is significantly larger for small code changes. We cannot conclude reliably whether the order of presenting the code change parts influences the efficiency of code review.
AB - Change-based code review is a software quality assurance technique that is widely used in practice. Therefore, better understanding what influences performance in code reviews and finding ways to improve it can have a large impact. In this study, we examine the association of working memory capacity and cognitive load with code review performance and we test the predictions of a recent theory regarding improved code review efficiency with certain code change part orders. We perform a confirmatory experiment with 50 participants, mostly professional software developers. The participants performed code reviews on one small and two larger code changes from an open source software system to which we had seeded additional defects. We measured their efficiency and effectiveness in defect detection, their working memory capacity, and several potential confounding factors. We find that there is a moderate association between working memory capacity and the effectiveness of finding delocalized defects, influenced by other factors, whereas the association with other defect types is almost non-existing. We also confirm that the effectiveness of reviews is significantly larger for small code changes. We cannot conclude reliably whether the order of presenting the code change parts influences the efficiency of code review.
KW - Change-based code review
KW - Code ordering
KW - Cognitive load
KW - Cognitive support
KW - Individual differences
KW - Working memory
UR - http://www.scopus.com/inward/record.url?scp=85059568157&partnerID=8YFLogxK
U2 - 10.1007/s10664-018-9676-8
DO - 10.1007/s10664-018-9676-8
M3 - Article
AN - SCOPUS:85059568157
VL - 24
SP - 1762
EP - 1798
JO - Empirical Software Engineering
JF - Empirical Software Engineering
SN - 1382-3256
IS - 4
ER -