TestGen automatically generates unit tests, carved from serialized observations of complex objects, observed during app execution. We describe the development and deployment of TestGen at Meta. In particular, we focus on the scalability challenges overcome during development in order to deploy observation-based test carving at scale in industry. So far, TestGen has landed 518 tests into production, which have been executed 9,617,349 times in continuous integration, finding 5,702 faults. Meta is currently in the process of more widespread deployment. Our evaluation reveals that, when carving its observations from 4,361 reliable end-to-end tests, TestGen was able to generate tests for at least 86% of the classes covered by end-to-end tests. Testing on 16 Kotlin Instagram app-launch-blocking tasks demonstrated that the TestGen tests would have trapped 13 of these before they became launch blocking.
Fri 19 JulDisplayed time zone: Brasilia, Distrito Federal, Brazil change
11:00 - 12:30 | Testing 4Research Papers / Industry Papers at Pitanga Chair(s): Antonia Bertolino National Research Council, Italy | ||
11:00 18mTalk | Partial Solution Based Constraint Solving Cache in Symbolic Execution Research Papers Ziqi Shuai School of Computer, National University of Defense Technology, China, Zhenbang Chen College of Computer, National University of Defense Technology, Kelin Ma School of Computer, National University of Defense Technology, China, Kunlin Liu School of Computer, National University of Defense Technology, China, Yufeng Zhang Hunan University, Jun Sun School of Information Systems, Singapore Management University, Singapore, Ji Wang School of Computer, National University of Defense Technology, China Pre-print | ||
11:18 18mTalk | Natural Symbolic Execution-based Testing for Big Data Analytics Research Papers Yaoxuan Wu UCLA, Ahmad Humayun Virginia Tech, Muhammad Ali Gulzar Virginia Tech, Miryung Kim UCLA and Amazon Web Services Pre-print | ||
11:36 18mTalk | MTAS: A Reference-Free Approach for Evaluating Abstractive Summarization Systems Research Papers Xiaoyan Zhu Zhejiang Sci-Tech University, Mingyue Jiang Zhejiang Sci-Tech University, Xiao-Yi Zhang University of Science and Technology Beijing, Liming Nie Nanyang Technological University, Zuohua Ding Zhejiang Sci-Tech University | ||
11:54 18mTalk | Observation-based unit test generation at Meta Industry Papers Mark Harman Meta Platforms, Inc. and UCL, Rotem Tal Meta platforms, Alexandru Marginean Meta platforms, Eddy Wang Meta platforms, Nadia Alshahwan Meta Platforms | ||
12:12 18mTalk | Property-based Testing for Validating User Privacy-Related Functionalities in Social Media Apps Industry Papers Jingling Sun University of Electronic Science and Technology of China, Ting Su East China Normal University, Jun Sun School of Information Systems, Singapore Management University, Singapore, Jianwen Li East China Normal University, China, Mengfei Wang ByteDance, Geguang Pu East China Normal University, China | ||