Only diff is Not Enough: Generating Commit Messages Leveraging Reasoning and Action of Large Language Model
Commit messages play a vital role in software development and maintenance. While previous research has introduced various Commit Message Generation (CMG) approaches, they often suffer from a lack of consideration for the broader software context associated with code changes. This limitation resulted in generated commit messages that contained insufficient information and were poorly readable. To address these shortcomings, we approached CMG as a knowledge-intensive reasoning task. We employed ReAct prompting with a cutting-edge Large Language Model (LLM) to generate high-quality commit messages. Our tool retrieves a wide range of software context information, enabling the LLM to create commit messages that are factually grounded and comprehensive. Additionally, we gathered commit message quality expectations from software practitioners, incorporating them into our approach to further enhance message quality. Human evaluation demonstrates the overall effectiveness of our CMG approach, which we named Omniscient Message Generator (OMG). It achieved an average improvement of 30.2% over human-written messages and a 71.6% improvement over state-of-the-art CMG methods.
Wed 17 JulDisplayed time zone: Brasilia, Distrito Federal, Brazil change
11:00 - 12:30 | Software Maintenance and Comprehension 1Research Papers / Ideas, Visions and Reflections / Demonstrations at Acerola Chair(s): Wesley Assunção North Carolina State University | ||
11:00 18mTalk | Enhancing Function Name Prediction using Votes-Based Name Tokenization and Multi-Task Learning Research Papers Xiaoling Zhang Institute of Information Engineering, Chinese Academy of Sciences, School of Cyber Security, University of Chinese Academy of Sciences,, Zhengzi Xu Nanyang Technological University, shouguo yang Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China, Zhi Li Institute of Information Engineering, Chinese Academy of Sciences, China, Zhiqiang Shi Institute of Information Engineering, Chinese Academy of Sciences, School of Cyber Security, University of Chinese Academy of Sciences,, Limin Sun Institute of Information Engineering, Chinese Academy of Sciences, School of Cyber Security, University of Chinese Academy of Sciences, DOI Pre-print | ||
11:18 18mTalk | Only diff is Not Enough: Generating Commit Messages Leveraging Reasoning and Action of Large Language Model Research Papers Jiawei Li University of California, Irvine, David Faragó Innoopract GmbH & QPR Technologies, Christian Petrov Innoopract GmbH, Iftekhar Ahmed University of California, Irvine | ||
11:36 18mTalk | Towards Efficient Build Ordering for Incremental Builds with Multiple Configurations Research Papers Jun Lyu Nanjing University, Shanshan Li Software Institute, Nanjing University, He Zhang Nanjing University, Lanxin Yang Nanjing University, Bohan Liu Nanjing University, Manuel Rigger National University of Singapore | ||
11:54 18mTalk | Unprecedented Code Change Automation: The Fusion of LLMs and Transformation by Example Research Papers Malinda Dilhara University of Colorado Boulder, Abhiram Bellur University of Colorado Boulder, Timofey Bryksin JetBrains Research, Danny Dig University of Colorado Boulder, JetBrains Research Pre-print | ||
12:12 9mTalk | Variability-Aware Differencing with DiffDetectiveBest Demo Paper Demonstrations Paul Maximilian Bittner Paderborn University, Alexander Schultheiß Paderborn University, Benjamin Moosherr University of Ulm, Timo Kehrer University of Bern, Thomas Thüm Paderborn University Pre-print Media Attached | ||
12:21 9mTalk | From Models to Practice: Enhancing OSS Project Sustainability with Evidence-Based Advice Ideas, Visions and Reflections Nafiz Imtiaz Khan Department of Computer Science, University of California, Davis, Vladimir Filkov University of California at Davis, USA Link to publication DOI | ||