Thu 18 Jul 2024 14:54 - 15:12 at Mandacaru - SE4AI 1 Chair(s): Qinghua Lu

Deep neural networks (DNNs) are widely used in various application domains such as image processing, speech recognition, and natural language processing. However, testing DNN models may be challenging due to the complexity and size of their input domain. Particularly, testing DNN models often requires generating or exploring large unlabeled datasets. In practice, DNN test oracles, which identify the correct outputs for inputs, often require expensive manual effort to label test data, possibly involving multiple experts to ensure labeling correctness. In this paper, we propose DeepGD, a black-box multi-objective test selection approach for DNN models. It reduces the cost of labeling by prioritizing the selection of test inputs with high fault-revealing power from large unlabeled datasets. DeepGD not only selects test inputs with high uncertainty scores to trigger as many mispredicted inputs as possible but also maximizes the probability of revealing distinct faults in the DNN model by selecting diverse mispredicted inputs. The experimental results conducted on four widely used datasets and five DNN models show that in terms of fault-revealing ability: (1) White-box, coverage-based approaches fare poorly, (2) DeepGD outperforms existing black-box test selection approaches in terms of fault detection, and (3) DeepGD also leads to better guidance for DNN model retraining when using selected inputs to augment the training set.

Thu 18 Jul

Displayed time zone: Brasilia, Distrito Federal, Brazil change

14:00 - 15:30
14:00
18m
Talk
Harnessing Neuron Stability to Improve DNN Verification
Research Papers
Hai Duong George Mason University, Dong Xu University of Virginia, ThanhVu Nguyen George Mason University, Matthew B Dwyer University of Virginia
14:18
18m
Talk
MirrorFair: Fixing Fairness Bugs in Machine Learning Software via Counterfactual Predictions
Research Papers
Ying Xiao King's College London / Southern University of Science and Technology, Jie M. Zhang King's College London, Yepang Liu Southern University of Science and Technology, Mohammad Reza Mousavi King's College London, Sicen Liu Southern University of Science and Technology, Dingyuan Xue Southern University of Science and Technology
14:36
9m
Talk
Using Run-time Information to Enhance Static Analysis of Machine Learning Code in Notebooks
Ideas, Visions and Reflections
Yiran Wang Linköping University, José Antonio Hernández López Linkoping University, Ulf Nilsson Linköping University, Daniel Varro Linköping University / McGill University
Link to publication DOI
14:45
9m
Talk
Human-Imperceptible Retrieval Poisoning Attacks in LLM-Powered Applications
Ideas, Visions and Reflections
Quan Zhang Tsinghua University, Binqi Zeng Central South University, Chijin Zhou Tsinghua University, Gwihwan Go Tsinghua University, Heyuan Shi Central South University, Yu Jiang Tsinghua University
14:54
18m
Talk
DeepGD: A Multi-Objective Black-Box Test Selection Approach for Deep Neural Networks
Journal First
Zohreh Aghababaeyan University of Ottawa, Canada, Manel Abdellatif Software and Information Technology Engineering Department, École de Technologie Supérieure, Mahboubeh Dadkhah The School of EECS, University of Ottawa, Lionel Briand University of Ottawa, Canada; Lero centre, University of Limerick, Ireland
15:12
9m
Talk
Testing Learning-Enabled Cyber-Physical Systems with Large-Language Models: A Formal Approach
Ideas, Visions and Reflections
Xi Zheng Macquarie University, Aloysius K. Mok University of Texas at Austin, Ruzica Piskac Yale University, Yong Jae Lee University of Wisconsin Madison, Bhaskar Krishnamachari University of Southern California, Dakai Zhu The University of Texas at San Antonio, Oleg Sokolsky University of Pennsylvania, USA, Insup Lee University of Pennsylvania
15:21
9m
Talk
GAISSALabel: A tool for energy labeling of ML models
Demonstrations
Pau Duran Universitat Politècnica de Catalunya (UPC), Joel Castaño Fernández Universitat Politècnica de Catalunya (UPC), Cristina Gómez Universitat Politècnica de Catalunya, Silverio Martínez-Fernández UPC-BarcelonaTech
Link to publication Pre-print