Thu 18 Jul 2024 10:30 - 11:00 at Lounge - Poster Session 2

Large Language Models (LLMs) have transformed natural language processing and generation activities in recent years. However, as the scale and complexity of these models grow, their ability to write correct and secure code has come under scrutiny. In our research, we delve into the critical examination of Large Language Models (LLMs) like ChatGPT-3.5 (text-davinci-002-render-sha), legacy Bard (PaLM 2), and Gemini (gemini-1.0-pro-001), and their proficiency in generating accurate and secure code, particularly focusing on the occurrence of identifier swaps within the code they produce. Our methodology encompasses the creation of a diverse dataset comprising a range of coding tasks designed to challenge the code generation capabilities of these models. Further, we employ Pylint for an extensive code quality assessment and undertake multi-shot identifier swap testing to evaluate the models’ ability to maintain context and coherence over sequential coding prompts. Our findings contribute to the understanding of LLMs’ performance\r\nin code generation, highlighting potential areas for improvement in ensuring the reliability and security of LLM-generated code in software development applications.

Thu 18 Jul

Displayed time zone: Brasilia, Distrito Federal, Brazil change

10:30 - 11:00
Poster Session 2Posters at Lounge
10:30
30m
Poster
DyPyBench: A Benchmark of Executable Python Software
Posters
Islem BOUZENIA University of Stuttgart, Bajaj Piyush Krishan University of Stuttgart, Michael Pradel University of Stuttgart
10:30
30m
Poster
Shadows in the Interface: A Comprehensive Study on Dark Patterns
Posters
Liming Nie Nanyang Technological University, Yangyang Zhao Zhejiang Sci-Tech University, Chenglin Li Zhejiang Sci-Tech University, Xuqiong Luo Changsha University of Science and Technology, Yang Liu Nanyang Technological University
10:30
30m
Poster
Do Large Language Models Recognize Python Identifier Swaps in their Generated Code?
Posters
Sagar Bhikan Chavan IIT Gandhinagar, Shouvick Mondal IIT Gandhinagar
DOI Pre-print Media Attached File Attached
10:30
30m
Poster
Understanding Developers' Discussions and Perceptions on Non-Functional Requirements: The Case of the Spring Ecosystem
Posters
Anderson Oliveira Pontifical Catholic University of Rio de Janeiro (PUC-Rio), João Lucas Correia Pontifical Catholic University of Rio de Janeiro (PUC-Rio), Wesley Assunção North Carolina State University, Juliana Alves Pereira Pontifical Catholic University of Rio de Janeiro (PUC-Rio), Rafael de Mello Federal University of Rio de Janeiro (UFRJ), Daniel Coutinho Pontifical Catholic University of Rio de Janeiro (PUC-Rio), Caio Barbosa Pontifical Catholic University of Rio de Janeiro (PUC-Rio), Paulo Vítor C. F. Libório Pontifical Catholic University of Rio de Janeiro (PUC-Rio), Alessandro Garcia Pontifical Catholic University of Rio de Janeiro (PUC-Rio)
10:30
30m
Poster
ProveNFix: Temporal Property guided Program Repair
Posters
Yahui Song National University of Singapore, Xiang Gao Beihang University, Wenhua Li National University of Singapore, Wei-Ngan Chin National University of Singapore, Abhik Roychoudhury National University of Singapore
10:30
30m
Poster
PBE-based Selective Abstraction and Refinement for Efficient Property Falsification of Embedded Software
Posters
Yoel Kim Kyungpook National University, Yunja Choi Kyungpook National University
10:30
30m
Poster
A Transferability Study of Interpolation-Based Hardware Model Checking to Software Verification
Posters
Dirk Beyer LMU Munich, Po-Chun Chien LMU Munich, Marek Jankola LMU Munich, Nian-Ze Lee LMU Munich
DOI Media Attached
10:30
30m
Poster
Evaluating and Improving ChatGPT for Unit Test Generation
Posters
Zhiqiang Yuan Fudan University, Mingwei Liu Fudan University, Shiji Ding Fudan University, Kaixin Wang Fudan University, Yixuan Chen Yale University, Xin Peng Fudan University, Yiling Lou Fudan University
10:30
30m
Poster
Testing AI Systems Leveraging Graph Perturbation
Posters
Zhaorui Yang University of California, Riverside, Haichao Zhu Tencent America, Qian Zhang University of California, Riverside
10:30
30m
Poster
Predictive Program Slicing via Execution Knowledge-Guided Dynamic Dependence Learning
Posters
Aashish Yadavally University of Texas at Dallas, Yi Li University of Texas at Dallas, Tien N. Nguyen University of Texas at Dallas
10:30
30m
Poster
Unprecedented Code Change Automation: The Fusion of LLMs and Transformation by Example
Posters
Malinda Dilhara University of Colorado Boulder, Abhiram Bellur University of Colorado Boulder, Timofey Bryksin JetBrains Research, Danny Dig University of Colorado Boulder, JetBrains Research
10:30
30m
Poster
A Deep Dive into Large Language Models for Bug Fixing
Posters
Soneya Binta Hossain University of Virginia, Nan Jiang Purdue University, Qiang Zhou Amazon Web Services, Xiaopeng LI Amazon Web Services, Wen-Hao Chiang Amazon Web Services, Yingjun Lyu Amazon Web Services, Hoan Nguyen Amazon Web Services, Omer Tripp Amazon Web Services
10:30
30m
Poster
A Quantitative and Qualitative Evaluation of LLM-based Explainable Fault Localization
Posters
Sungmin Kang Korea Advanced Institute of Science and Technology, Gabin An Korea Advanced Institute of Science and Technology, Shin Yoo Korea Advanced Institute of Science and Technology
10:30
30m
Poster
IRCoCo: Immediate Rewards-Guided Deep Reinforcement Learning for Code Completion
Posters
Bolun Li Shandong Normal University, Zhihong Sun Shandong Normal University, Tao Huang Shandong Normal University, Hongyu Zhang Chongqing University, Yao Wan Huazhong University of Science and Technology, Chen Lyu Shandong Normal University, Ge Li Peking University, Zhi Jin Peking University

Information for Participants
Thu 18 Jul 2024 10:30 - 11:00 at Lounge - Poster Session 2
Info for room Lounge:

This room is conjoined with the Foyer to provide additional space for the coffee break, and hold poster presentations throughout the event.