Using Run-time Information to Enhance Static Analysis of Machine Learning Code in Notebooks
A prevalent method for developing machine learning (ML) prototypes involves the use of notebooks. Notebooks are sequences of cells containing both code and natural language documentation. When executed during development, these code cells provide valuable run-time information. Nevertheless, current static analyzers for notebooks do not leverage this run-time information to detect ML bugs. Consequently, our primary proposition in this paper is that harvesting this run-time information in notebooks can significantly improve the effectiveness of static analysis in detecting ML bugs. To substantiate our claim, we focus on bugs related to tensor shapes and conduct experiments using two static analyzers: 1) PYTHIA, a traditional rule-based static analyzer, and 2) GPT-4, a large language model that can also be used as a static analyzer. The results demonstrate that using run-time information in static analyzers enhances their bug detection performance and it also helped reveal a hidden bug in a public dataset.
Thu 18 JulDisplayed time zone: Brasilia, Distrito Federal, Brazil change
14:00 - 15:30 | SE4AI 1Journal First / Ideas, Visions and Reflections / Research Papers / Demonstrations at Mandacaru Chair(s): Qinghua Lu Data61, CSIRO | ||
14:00 18mTalk | Harnessing Neuron Stability to Improve DNN Verification Research Papers Hai Duong George Mason University, Dong Xu University of Virginia, ThanhVu Nguyen George Mason University, Matthew B Dwyer University of Virginia | ||
14:18 18mTalk | MirrorFair: Fixing Fairness Bugs in Machine Learning Software via Counterfactual Predictions Research Papers Ying Xiao King's College London / Southern University of Science and Technology, Jie M. Zhang King's College London, Yepang Liu Southern University of Science and Technology, Mohammad Reza Mousavi King's College London, Sicen Liu Southern University of Science and Technology, Dingyuan Xue Southern University of Science and Technology | ||
14:36 9mTalk | Using Run-time Information to Enhance Static Analysis of Machine Learning Code in Notebooks Ideas, Visions and Reflections Yiran Wang Linköping University, José Antonio Hernández López Linkoping University, Ulf Nilsson Linköping University, Daniel Varro Linköping University / McGill University Link to publication DOI | ||
14:45 9mTalk | Human-Imperceptible Retrieval Poisoning Attacks in LLM-Powered Applications Ideas, Visions and Reflections Quan Zhang Tsinghua University, Binqi Zeng Central South University, Chijin Zhou Tsinghua University, Gwihwan Go Tsinghua University, Heyuan Shi Central South University, Yu Jiang Tsinghua University | ||
14:54 18mTalk | DeepGD: A Multi-Objective Black-Box Test Selection Approach for Deep Neural Networks Journal First Zohreh Aghababaeyan University of Ottawa, Canada, Manel Abdellatif Software and Information Technology Engineering Department, École de Technologie Supérieure, Mahboubeh Dadkhah The School of EECS, University of Ottawa, Lionel Briand University of Ottawa, Canada; Lero centre, University of Limerick, Ireland | ||
15:12 9mTalk | Testing Learning-Enabled Cyber-Physical Systems with Large-Language Models: A Formal Approach Ideas, Visions and Reflections Xi Zheng Macquarie University, Aloysius K. Mok University of Texas at Austin, Ruzica Piskac Yale University, Yong Jae Lee University of Wisconsin Madison, Bhaskar Krishnamachari University of Southern California, Dakai Zhu The University of Texas at San Antonio, Oleg Sokolsky University of Pennsylvania, USA, Insup Lee University of Pennsylvania | ||
15:21 9mTalk | GAISSALabel: A tool for energy labeling of ML models Demonstrations Pau Duran Universitat Politècnica de Catalunya (UPC), Joel Castaño Fernández Universitat Politècnica de Catalunya (UPC), Cristina Gómez Universitat Politècnica de Catalunya, Silverio Martínez-Fernández UPC-BarcelonaTech Link to publication Pre-print | ||