What Makes a Code Review Useful to OpenDev Developers? An Empirical Investigation
\textit{Context: } Due to the association of significant efforts, even a minor improvement in the effectiveness of Code Reviews(CR) can incur significant savings for a software development organization.
\textit{Aim:} This study aims \textit{ to develop a finer grain understanding of what makes a code review comment useful to OSS developers, to what extent a code review comment is considered useful to them, and how various contextual and participant-related factors influence its degree of usefulness}.
\textit{Method:} On this goal, we have conducted a three-stage mixed-method study. We randomly selected 2,500 CR comments from the OpenDev Nova project and manually categorized the comments. We designed a survey of OpenDev developers to better understand their perspectives on useful CRs. Combining our survey-obtained scores with our manually labeled dataset, we trained two regression models - one to identify factors that influence the usefulness of CR comments and the other to identify factors that improve the odds of `Functional’ defect identification over the others.
\textit{Key findings:} The results of our study suggest that a CR comment’s usefulness is dictated not only by its technical contributions, such as defect findings or quality improvement tips but also by its linguistic characteristics, such as comprehensibility and politeness. While a reviewer’s coding experience is positively associated with CR usefulness, the number of mutual reviews, comment volume in a file, the total number of lines added /modified, and CR interval have the opposite associations. While authorship and reviewership experiences for the files under review have been the most popular attributes for reviewer recommendation systems, we do not find any significant association of those attributes with CR usefulness.
\textit{Conclusion:} We recommend discouraging frequent code review associations between two individuals as such associations may decrease CR usefulness. We also recommend authoring CR comments in a constructive and empathetic tone. As several of our results deviate from prior studies, we recommend more investigations to identify context-specific attributes to build reviewer recommendation models.
Thu 18 JulDisplayed time zone: Brasilia, Distrito Federal, Brazil change
11:00 - 12:30 | Empirical Studies 2Ideas, Visions and Reflections / Research Papers / Industry Papers / Journal First at Acerola Chair(s): Iftekhar Ahmed University of California, Irvine | ||
11:00 18mTalk | State Reconciliation Defects in Infrastructure as Code Research Papers Md Mahadi Hassan Auburn University, John Salvador Auburn University, Shubhra Kanti Karmaker Santu Auburn University, Akond Rahman Auburn University Pre-print | ||
11:18 18mTalk | Understanding and Detecting Annotation-induced Faults of Static Analyzers Research Papers Huaien Zhang The Hong Kong Polytechnic University, Southern University of Science and Technology, Yu Pei The Hong Kong Polytechnic University, Shuyun Liang Southern University of Science and Technology, Shin Hwei Tan Concordia University | ||
11:36 18mTalk | What Makes a Code Review Useful to OpenDev Developers? An Empirical Investigation Journal First | ||
11:54 9mTalk | The Patch Overfitting Problem in Automated Program Repair: Practical Magnitude and a Baseline for Realistic Benchmarking Ideas, Visions and Reflections Justyna Petke University College London, Matias Martinez Universitat Politècnica de Catalunya (UPC), Maria Kechagia University College London, Aldeida Aleti Monash University, Federica Sarro University College London | ||
12:03 9mTalk | Automating Issue Reporting in Software Testing: Lessons Learned from Using the Template Generator Tool Industry Papers Lennon Chaves SIDIA Institute of Science and Technology, Flávia Oliveira Sidia Institute of Science and Technology, Leonardo Tiago Sidia Institute of Science and Technology | ||
12:12 9mTalk | Reproducibility Debt: Challenges and Future Pathways Ideas, Visions and Reflections Zara Hassan Australian National University, Christoph Treude Singapore Management University, Michael Norrish Australian National University, Graham Williams Australian National University, Alex Potanin Australian National University Link to publication DOI | ||
12:21 9mTalk | A Vision on Open Science for the Evolution of Software Engineering Research and Practice Ideas, Visions and Reflections Edson OliveiraJr State University of Maringá, Fernanda Madeiral Vrije Universiteit Amsterdam, Alcemir Rodrigues Santos State University of Piauí, Christina von Flach Federal University of Bahia, Sérgio Soares Universidade Federal de Pernambuco Pre-print |