Wed 17 Jul 2024 17:12 - 17:30 at Pitomba - AI4SE 2 Chair(s): Jingyue Li

Search components in e-commerce apps, often complex AI-based retrieval systems, are susceptible to bugs, including ones leading to missed recalls. Missed recalls are instances where an entry should appear in the search results based on the algorithmic and business logic, but doesn’t. They cause dissatisfaction among shop owners and can impact the app’s profitability. However, testing for missed recalls is challenging due to difficulties in generating user-aligned test cases and the absence of oracles. In this paper, we introduce mrDetector, the first automatic testing approach specifically for missed recalls. To tackle the test case generation challenge, we first study how users construct queries during searching. We then use these findings to create a CoT prompt containing multiple examples and guide the LLM generation process. To address the lack of oracles, we learn from users who create multiple queries for one shop and compare search results, and provide a test oracle through a metamorphic relation. Extensive experiments using open access data demonstrate that mrDetector outperforms all baselines with the lowest false positive ratio. Experiments with real industrial data show that mrDetector discovers over one hundred missed recalls with only 17 false positives. Corresponding engineers accept all representative missed recalls mrDetector finds.

Wed 17 Jul

Displayed time zone: Brasilia, Distrito Federal, Brazil change

16:00 - 18:00
AI4SE 2Industry Papers / Research Papers at Pitomba
Chair(s): Jingyue Li Norwegian University of Science and Technology (NTNU)
16:00
18m
Talk
MonitorAssistant: Simplifying Cloud Service Monitoring via Large Language Models
Industry Papers
Zhaoyang Yu Tsinghua University, Minghua Ma Microsoft Research, Chaoyun Zhang Microsoft, Si Qin Microsoft Research, Yu Kang Microsoft Research, Chetan Bansal Microsoft Research, Saravan Rajmohan Microsoft, Yingnong Dang Microsoft Azure, Changhua Pei Computer Network Information Center at Chinese Academy of Sciences, Dan Pei Tsinghua University, Qingwei Lin Microsoft, Dongmei Zhang Microsoft Research
16:18
18m
Talk
Code-Aware Prompting: A study of Coverage guided Test Generation in Regression Setting using LLM
Research Papers
Gabriel Ryan Columbia University, Siddhartha Jain AWS AI Labs, Mingyue Shang AWS AI Labs, Shiqi Wang AWS AI Labs, Xiaofei Ma AWS AI Labs, Murali Krishna Ramanathan AWS AI Labs, Baishakhi Ray Columbia University, New York; AWS AI Lab
16:36
18m
Talk
A Machine Learning-Based Error Mitigation Approach for Reliable Software Development on IBM’s Quantum Computers
Industry Papers
Asmar Muqeet Simula Research Laboratory and University of Oslo, Shaukat Ali Simula Research Laboratory and Oslo Metropolitan University, Tao Yue Beihang University, Paolo Arcaini National Institute of Informatics
16:54
18m
Talk
Multi-line AI-assisted Code Authoring
Industry Papers
Omer Dunay Meta Platforms, Inc., Daniel Cheng Meta Platforms Inc., Adam Tait Meta Platforms, Inc., Parth Thakkar Meta Platforms, Inc., Peter C Rigby Meta / Concordia University, Andy Chiu Meta Platforms, Inc., Imad Ahmad Meta Platforms, Inc., Arun Ganesan Meta Platforms, Inc., Chandra Sekhar Maddila Meta Platforms, Inc., Vijayaraghavan Murali Meta Platforms Inc., Ali Tayyebi Meta Platforms Inc., Nachiappan Nagappan Meta Platforms, Inc.
17:12
18m
Talk
Combating Missed Recalls in E-commerce Search: a CoT-prompting Testing Approach
Industry Papers
Shengnan Wu School of Computer Science, Fudan University, Yongxiang Hu Fudan University, Yingchuan Wang School of Computer Science, Fudan University, Jiazhen Gu The Chinese University of Hong Kong, Jin Meng Meituan Inc., Liujie Fan Meituan Inc., Zhongshi Luan Meituan Inc., Xin Wang Fudan University, Yangfan Zhou Fudan University
Pre-print
17:30
18m
Talk
Automated Unit Test Improvement using Large Language Models at Meta
Industry Papers
Mark Harman Meta Platforms, Inc. and UCL, Jubin Chheda Meta platforms, Anastasia Finogenova Meta platforms, Inna Harper Meta, Alexandru Marginean Meta platforms, Shubho Sengupta Meta platforms, Eddy Wang Meta platforms, Nadia Alshahwan Meta Platforms, Beliz Gokkaya Meta Platforms