Search-based Software Testing Driven by Automatically Generated and Manually Defined Fitness Functions
Search-based software testing (SBST) typically relies on fitness functions to guide the search exploration toward software failures. There are two main techniques to define fitness functions: (a) automated fitness function computation from the specification of the system requirements, and (b) manual fitness function design. Both techniques have advantages. The former uses information from the system requirements to guide the search toward portions of the input domain more likely to contain failures. The latter uses the engineers’ domain knowledge.
We propose ATheNA, a novel SBST framework that combines fitness functions automatically generated from requirements specifications and those manually defined by engineers. We design and implement ATheNA-S, an instance of ATheNA that targets Simulink models. We evaluate ATheNA-S by considering a large set of models from different domains. Our results show that ATheNA-S generates more failure-revealing test cases than existing baseline tools and that the difference between the runtime performance of ATheNA-S and the baseline tools is not statistically significant. We also assess whether ATheNA-S could generate failure-revealing test cases when applied to two representative case studies: one from the automotive domain and one from the medical domain. Our results show that ATheNA-S successfully revealed a requirement violation in our case studies.
Thu 18 JulDisplayed time zone: Brasilia, Distrito Federal, Brazil change
16:00 - 18:00 | Testing 3Ideas, Visions and Reflections / Demonstrations / Research Papers / Journal First at Pitanga Chair(s): Qi Xin Wuhan University | ||
16:00 18mTalk | Search-based Software Testing Driven by Automatically Generated and Manually Defined Fitness Functions Journal First Federico Formica McMaster University, Tony Fan McMaster University, Claudio Menghi University of Bergamo; McMaster University | ||
16:18 9mTalk | Monitoring the Execution of 14K Tests: Methods Tend to Have One Path that Is Significantly More Executed Ideas, Visions and Reflections Andre Hora UFMG Pre-print Media Attached | ||
16:36 18mTalk | Finding and Understanding Defects in Static Analyzers by Constructing Automated Oracles Research Papers weigang he East China Normal University / University of Technology Sydney, Peng Di Ant Group, Mengli Ming East China Normal University, Chengyu Zhang ETH Zurich, Ting Su East China Normal University, Shijie Li Ant Group, Yulei Sui UNSW | ||
16:54 18mTalk | A Miss Is as Good as A Mile: Metamorphic Testing for Deep Learning Operators Research Papers Jinyin Chen Zhejiang University of Technology, Chengyu Jia Zhejiang University of Technology, Yunjie Yan Zhejiang University of Technology, Jie Ge Zhejiang University of Technology, haibin zheng Zhejiang University of Technology, Yao Cheng TÜV SÜD Asia Pacific Pte. Ltd. | ||
17:12 9mTalk | ExLi : An Inline-Test Generation Tool for Java Demonstrations Yu Liu University of Texas at Austin, Aditya Thimmaiah The University of Texas at Austin, Owolabi Legunsen Cornell University, Milos Gligoric The University of Texas at Austin | ||
17:21 9mTalk | ATheNA-S: a Testing Tool for Simulink Models Driven by Software Requirements and Domain Expertise Demonstrations Federico Formica McMaster University, Mohammad Mahdi Mahboob McMaster University, Mehrnoosh Askarpour McMaster University, Claudio Menghi University of Bergamo; McMaster University | ||
17:30 9mTalk | Test Polarity: Detecting Positive and Negative Tests Ideas, Visions and Reflections Andre Hora UFMG Pre-print Media Attached | ||
17:39 18mTalk | Java JIT Testing with Template Extraction Research Papers Zhiqiang Zang The University of Texas at Austin, Fu-Yao Yu The University of Texas at Austin, Aditya Thimmaiah The University of Texas at Austin, August Shi The University of Texas at Austin, Milos Gligoric The University of Texas at Austin DOI Pre-print |