Thu 18 Jul 2024 14:36 - 14:54 at Pitanga - Security and Privacy 1 Chair(s): Gias Uddin

Neural Code Completion Tools (NCCTs) have reshaped the field of software engineering, which are built upon the language modeling technique and can accurately suggest contextually relevant code snippets. However, language models may emit the training data verbatim during inference with appropriate prompts. This memorization property raises privacy concerns of NCCTs about hard-coded credential leakage, leading to unauthorized access to applications, systems, or networks. Therefore, to answer whether NCCTs will emit the hard-coded credential, we propose an evaluation tool called Hard-coded Credential Revealer (HCR). HCR constructs test prompts based on GitHub code files with credentials to reveal the memorization phenomenon of NCCTs. Then, HCR designs four filters to filter out ill-formatted credentials. Finally, HCR directly checks the validity of a set of non-sensitive credentials. We apply HCR to evaluate three representative types of NCCTs: Commercial NCCTs, open-source models, and chatbots with code completion capability. Our experimental results show that NCCTs can not only return the precise piece of their training data but also inadvertently leak additional secret strings. Notably, two valid credentials were identified during our experiments. Therefore, HCR raises a severe privacy concern about the potential leakage of hard-coded credentials in the training data of commercial NCCTs.

Thu 18 Jul

Displayed time zone: Brasilia, Distrito Federal, Brazil change

14:00 - 15:30
Security and Privacy 1Ideas, Visions and Reflections / Industry Papers / Research Papers at Pitanga
Chair(s): Gias Uddin York University, Canada
14:00
18m
Talk
Investigating Documented Privacy Changes in Android OS
Research Papers
Chuan Yan University of Queensland, Mark Huasong Meng National University of Singapore, Fuman Xie University of Queensland, Guangdong Bai University of Queensland
14:18
9m
Talk
A Preliminary Study on the Privacy Concerns of Using IP Addresses in Log Data
Ideas, Visions and Reflections
Issam Sedki Concordia University
14:27
9m
Talk
Personal Data-Less Personalized Software Applications
Ideas, Visions and Reflections
Sana Belguith University of Bristol, Inah Omoronyia University of Bristol, Ruzanna Chitchyan University of Bristol
14:36
18m
Talk
Your Code Secret Belongs to Me: Neural Code Completion Tools Can Memorize Hard-coded Credentials
Research Papers
Yizhan Huang The Chinese University of Hong Kong, Yichen LI The Chinese University of Hong Kong, Weibin Wu Sun Yat-sen University, Jianping Zhang The Chinese University of Hong Kong, Michael Lyu The Chinese University of Hong Kong
14:54
18m
Talk
Unveil the Mystery of Critical Software Vulnerabilities
Industry Papers
Shengyi Pan Zhejiang University, Lingfeng Bao Zhejiang University, Jiayuan Zhou Huawei, Xing Hu Zhejiang University, Xin Xia Huawei Technologies, Shanping Li Zhejiang University
15:12
9m
Talk
AgraBOT: Accelerating Third-Party Security Risk Management in Enterprise Setting
Industry Papers
Mert Toslali IBM Research, Edward Snible IBM Research, Jing Chen IBM Research, Alan Cha IBM Research, USA, Sandeep Singh IBM, Michael Kalantar IBM Research, Srinivasan Parthasarathy IBM Research