Thu 18 Jul 2024 11:36 - 11:54 at Acerola - Empirical Studies 2 Chair(s): Iftekhar Ahmed

\textit{Context: } Due to the association of significant efforts, even a minor improvement in the effectiveness of Code Reviews(CR) can incur significant savings for a software development organization.

\textit{Aim:} This study aims \textit{ to develop a finer grain understanding of what makes a code review comment useful to OSS developers, to what extent a code review comment is considered useful to them, and how various contextual and participant-related factors influence its degree of usefulness}.

\textit{Method:} On this goal, we have conducted a three-stage mixed-method study. We randomly selected 2,500 CR comments from the OpenDev Nova project and manually categorized the comments. We designed a survey of OpenDev developers to better understand their perspectives on useful CRs. Combining our survey-obtained scores with our manually labeled dataset, we trained two regression models - one to identify factors that influence the usefulness of CR comments and the other to identify factors that improve the odds of `Functional’ defect identification over the others.

\textit{Key findings:} The results of our study suggest that a CR comment’s usefulness is dictated not only by its technical contributions, such as defect findings or quality improvement tips but also by its linguistic characteristics, such as comprehensibility and politeness. While a reviewer’s coding experience is positively associated with CR usefulness, the number of mutual reviews, comment volume in a file, the total number of lines added /modified, and CR interval have the opposite associations. While authorship and reviewership experiences for the files under review have been the most popular attributes for reviewer recommendation systems, we do not find any significant association of those attributes with CR usefulness.

\textit{Conclusion:} We recommend discouraging frequent code review associations between two individuals as such associations may decrease CR usefulness. We also recommend authoring CR comments in a constructive and empathetic tone. As several of our results deviate from prior studies, we recommend more investigations to identify context-specific attributes to build reviewer recommendation models.

Thu 18 Jul

Displayed time zone: Brasilia, Distrito Federal, Brazil change

11:00 - 12:30
Empirical Studies 2Ideas, Visions and Reflections / Research Papers / Industry Papers / Journal First at Acerola
Chair(s): Iftekhar Ahmed University of California, Irvine
11:00
18m
Talk
State Reconciliation Defects in Infrastructure as Code
Research Papers
Md Mahadi Hassan Auburn University, John Salvador Auburn University, Shubhra Kanti Karmaker Santu Auburn University, Akond Rahman Auburn University
Pre-print
11:18
18m
Talk
Understanding and Detecting Annotation-induced Faults of Static Analyzers
Research Papers
Huaien Zhang The Hong Kong Polytechnic University, Southern University of Science and Technology, Yu Pei The Hong Kong Polytechnic University, Shuyun Liang Southern University of Science and Technology, Shin Hwei Tan Concordia University
11:36
18m
Talk
What Makes a Code Review Useful to OpenDev Developers? An Empirical Investigation
Journal First
Asif Kamal Turzo Wayne State University, Amiangshu Bosu Wayne State University
11:54
9m
Talk
The Patch Overfitting Problem in Automated Program Repair: Practical Magnitude and a Baseline for Realistic Benchmarking
Ideas, Visions and Reflections
Justyna Petke University College London, Matias Martinez Universitat Politècnica de Catalunya (UPC), Maria Kechagia University College London, Aldeida Aleti Monash University, Federica Sarro University College London
12:03
9m
Talk
Automating Issue Reporting in Software Testing: Lessons Learned from Using the Template Generator Tool
Industry Papers
Lennon Chaves SIDIA Institute of Science and Technology, Flávia Oliveira Sidia Institute of Science and Technology, Leonardo Tiago Sidia Institute of Science and Technology
12:12
9m
Talk
Reproducibility Debt: Challenges and Future Pathways
Ideas, Visions and Reflections
Zara Hassan Australian National University, Christoph Treude Singapore Management University, Michael Norrish Australian National University, Graham Williams Australian National University, Alex Potanin Australian National University
Link to publication DOI
12:21
9m
Talk
A Vision on Open Science for the Evolution of Software Engineering Research and Practice
Ideas, Visions and Reflections
Edson OliveiraJr State University of Maringá, Fernanda Madeiral Vrije Universiteit Amsterdam, Alcemir Rodrigues Santos State University of Piauí, Christina von Flach Federal University of Bahia, Sérgio Soares Universidade Federal de Pernambuco
Pre-print