Wed 17 Jul 2024 17:39 - 17:48 at Mandacaru - Program Repair and Synthesis Chair(s): Fernanda Madeiral

With the growing interest on Large Language Models (LLMs) for fault localization and program repair, ensuring the integrity and generalizability of the LLM-based methods becomes paramount. The code in existing widely-adopted benchmarks for these tasks was written before the the bloom of LLMs and may be included in the training data of existing popular LLMs, thereby suffering from the threat of data leakage, leading to misleadingly optimistic performance metrics. To address this issue, we introduce ConDefects, a dataset developed as a complement to existing datasets, meticulously curated with real faults to eliminate such overlap. ConDefects contains 1,254 Java faulty programs and 1,625 Python faulty programs. All these programs are sourced from the online competition platform AtCoder and were produced between October 2021 and September 2023. We pair each fault with fault locations and the corresponding repaired code versions, making it tailored for in fault localization and program repair related research. We also provide interfaces for selecting subsets based on different time windows and coding task difficulties. While inspired by LLM-based tasks, ConDefects can be adopted for benchmarking ALL types of fault localization and program repair methods. The dataset is publicly available, and a demo video can be found at https://www.youtube.com/watch?v=22j15Hj5ONk

Wed 17 Jul

Displayed time zone: Brasilia, Distrito Federal, Brazil change

16:00 - 18:00
Program Repair and SynthesisDemonstrations / Research Papers / Ideas, Visions and Reflections at Mandacaru
Chair(s): Fernanda Madeiral Vrije Universiteit Amsterdam
16:00
18m
Talk
A Deep Dive into Large Language Models for Automated Bug Localization and Repair
Research Papers
Soneya Binta Hossain University of Virginia, Nan Jiang Purdue University, Qiang Zhou Amazon Web Services, Xiaopeng LI Amazon Web Services, Wen-Hao Chiang Amazon Web Services, Yingjun Lyu Amazon Web Services, Hoan Nguyen Amazon Web Services, Omer Tripp Amazon Web Services
DOI
16:18
18m
Talk
CORE: Resolving Code Quality Issues Using LLMs
Research Papers
Nalin Wadhwa Microsoft Research, India, Jui Pradhan Microsoft Research, India, Atharv Sonwane Microsoft Research, India, Surya Prakash Sahu Microsoft Research, India, Nagarajan Natarajan Microsoft Research India, Aditya Kanade Microsoft Research, India, Suresh Parthasarathy Microsoft Research, India, Sriram Rajamani Microsoft Research Indua
16:36
18m
Talk
Towards Effective Multi-Hunk Bug Repair: Detecting, Creating, Evaluating, and Understanding Indivisible Bugs
Research Papers
Qi Xin Wuhan University, Haojun Wu Wuhan University, Jinran Tang Wuhan University, Xinyu Liu Wuhan University, Steven P. Reiss Brown University, Jifeng Xuan Wuhan University
16:54
18m
Talk
ProveNFix: Temporal Property guided Program RepairDistinguished Paper Award
Research Papers
Yahui Song National University of Singapore, Xiang Gao Beihang University, Wenhua Li National University of Singapore, Wei-Ngan Chin National University of Singapore, Abhik Roychoudhury National University of Singapore
DOI Pre-print
17:12
18m
Talk
Towards AI-Assisted Synthesis of Verified Dafny Methods
Research Papers
Md Rakib Hossain Misu University of California Irvine, Crista Lopes University of California Irvine, Iris Ma University of California Irvine, James Noble Independent. Wellington, NZ
DOI Pre-print
17:30
9m
Talk
Execution-free program repair
Ideas, Visions and Reflections
Bertrand Meyer Constructor Institute Schaffhausen, Li Huang Constructor Institute Schaffhausen, Ilgiz Mustafin Constructor Institute, Manuel Oriol Constructor Institute Schaffhausen
17:39
9m
Talk
ConDefects: A Complementary Dataset to Address the Data Leakage Concern for LLM-based Fault Localization and Program Repair
Demonstrations
Yonghao Wu Beijing University of Chemical Technology, Zheng Li Beijing University of Chemical Technology, Jie M. Zhang King's College London, Yong Liu Beijing University of Chemical Technology
17:48
9m
Talk
ASAC: A Benchmark for Algorithm Synthesis
Demonstrations
Zhao Zhang Peking University, Yican Sun Peking University, Ruyi Ji Peking University, Siyuan Li Peking University, Xuanyu Peng University of California, San Diego, Zhechong Huang Peking University, Sizhe Li Peking University, Tianran Zhu Peking University, Yingfei Xiong Peking University
Pre-print Media Attached