Large language models (LLMs) have shown impressive effectiveness in various software engineering tasks, including automated program repair (APR). In this study, we take a deep dive into automated bug fixing utilizing LLMs. In contrast to many deep learning-based APR methods that assume known bug locations, rely on line-level localization tools, or address bug prediction and fixing in one step, our approach uniquely employs LLMs to predict bug location at the token level and subsequently utilizes them for bug fixing. This methodological separation of bug localization and fixing using different LLMs enables effective integration of diverse contextual information and improved incorporation of inductive biases. We introduce Toggle: Token-Granulated Bug Localization and Repair, a comprehensive program repair framework that integrates a bug localization model, an adjustment unit, and a bug-fixing model. Toggle takes a buggy function as input and generates a complete corrected function. We investigate various styles of prompting to the bug fixing model to identify the most effective prompts that better utilize the inductive bias and significantly outperform others. Toggle achieves the new state-of-the-art (SOTA) performance on the CodeXGLUE code refinement benchmark, and exhibits better and comparable performance on several other widely-used APR datasets, including Defects4J. In the Defects4J benchmark, our approach consistently ranks above other methods, achieving superior results in the Top-10, Top-30, Top-50, and Top-100 metrics. Additionally, this paper examines Toggle’s generalizability to unseen data, evaluates the effectiveness of various prompts, investigates the impact of additional contextual information such as buggy lines and code comments on bug localization, and explores the importance of the adjustment unit. Our extensive experiments offer valuable insights and answers to critical research questions.
Thu 18 JulDisplayed time zone: Brasilia, Distrito Federal, Brazil change
10:30 - 11:00 | |||
10:30 30mPoster | DyPyBench: A Benchmark of Executable Python Software Posters Islem BOUZENIA University of Stuttgart, Bajaj Piyush Krishan University of Stuttgart, Michael Pradel University of Stuttgart | ||
10:30 30mPoster | Shadows in the Interface: A Comprehensive Study on Dark Patterns Posters Liming Nie Nanyang Technological University, Yangyang Zhao Zhejiang Sci-Tech University, Chenglin Li Zhejiang Sci-Tech University, Xuqiong Luo Changsha University of Science and Technology, Yang Liu Nanyang Technological University | ||
10:30 30mPoster | Do Large Language Models Recognize Python Identifier Swaps in their Generated Code? Posters DOI Pre-print Media Attached File Attached | ||
10:30 30mPoster | Understanding Developers' Discussions and Perceptions on Non-Functional Requirements: The Case of the Spring Ecosystem Posters Anderson Oliveira Pontifical Catholic University of Rio de Janeiro (PUC-Rio), João Lucas Correia Pontifical Catholic University of Rio de Janeiro (PUC-Rio), Wesley Assunção North Carolina State University, Juliana Alves Pereira Pontifical Catholic University of Rio de Janeiro (PUC-Rio), Rafael de Mello Federal University of Rio de Janeiro (UFRJ), Daniel Coutinho Pontifical Catholic University of Rio de Janeiro (PUC-Rio), Caio Barbosa Pontifical Catholic University of Rio de Janeiro (PUC-Rio), Paulo Vítor C. F. Libório Pontifical Catholic University of Rio de Janeiro (PUC-Rio), Alessandro Garcia Pontifical Catholic University of Rio de Janeiro (PUC-Rio) | ||
10:30 30mPoster | ProveNFix: Temporal Property guided Program Repair Posters Yahui Song National University of Singapore, Xiang Gao Beihang University, Wenhua Li National University of Singapore, Wei-Ngan Chin National University of Singapore, Abhik Roychoudhury National University of Singapore | ||
10:30 30mPoster | PBE-based Selective Abstraction and Refinement for Efficient Property Falsification of Embedded Software Posters | ||
10:30 30mPoster | A Transferability Study of Interpolation-Based Hardware Model Checking to Software Verification Posters DOI Media Attached | ||
10:30 30mPoster | Evaluating and Improving ChatGPT for Unit Test Generation Posters Zhiqiang Yuan Fudan University, Mingwei Liu Fudan University, Shiji Ding Fudan University, Kaixin Wang Fudan University, Yixuan Chen Yale University, Xin Peng Fudan University, Yiling Lou Fudan University | ||
10:30 30mPoster | Testing AI Systems Leveraging Graph Perturbation Posters Zhaorui Yang University of California, Riverside, Haichao Zhu Tencent America, Qian Zhang University of California, Riverside | ||
10:30 30mPoster | Predictive Program Slicing via Execution Knowledge-Guided Dynamic Dependence Learning Posters Aashish Yadavally University of Texas at Dallas, Yi Li University of Texas at Dallas, Tien N. Nguyen University of Texas at Dallas | ||
10:30 30mPoster | Unprecedented Code Change Automation: The Fusion of LLMs and Transformation by Example Posters Malinda Dilhara University of Colorado Boulder, Abhiram Bellur University of Colorado Boulder, Timofey Bryksin JetBrains Research, Danny Dig University of Colorado Boulder, JetBrains Research | ||
10:30 30mPoster | A Deep Dive into Large Language Models for Bug Fixing Posters Soneya Binta Hossain University of Virginia, Nan Jiang Purdue University, Qiang Zhou Amazon Web Services, Xiaopeng LI Amazon Web Services, Wen-Hao Chiang Amazon Web Services, Yingjun Lyu Amazon Web Services, Hoan Nguyen Amazon Web Services, Omer Tripp Amazon Web Services | ||
10:30 30mPoster | A Quantitative and Qualitative Evaluation of LLM-based Explainable Fault Localization Posters Sungmin Kang Korea Advanced Institute of Science and Technology, Gabin An Korea Advanced Institute of Science and Technology, Shin Yoo Korea Advanced Institute of Science and Technology | ||
10:30 30mPoster | IRCoCo: Immediate Rewards-Guided Deep Reinforcement Learning for Code Completion Posters Bolun Li Shandong Normal University, Zhihong Sun Shandong Normal University, Tao Huang Shandong Normal University, Hongyu Zhang Chongqing University, Yao Wan Huazhong University of Science and Technology, Chen Lyu Shandong Normal University, Ge Li Peking University, Zhi Jin Peking University | ||
This room is conjoined with the Foyer to provide additional space for the coffee break, and hold poster presentations throughout the event.