In infrastructure as code (IaC), state reconciliation is the process of querying and comparing the infrastructure state prior to changing the infrastructure. As state reconciliation is pivotal to manage IaC-based computing infrastructure at scale, defects related to state reconciliation can create large-scale consequences. A categorization of state reconciliation defects, i.e., defects related to state reconciliation, can aid in understanding the nature of state reconciliation defects. We conduct an empirical study with 5,110 state reconciliation defects where we apply qualitative analysis to categorize state reconciliation defects. From the identified defect categories, we derive heuristics to design prompts for a large language model (LLM), which in turn are used for validation of state reconciliation.
From our empirical study, we identify 8 categories of state reconciliation defects, amongst which 3 have not been reported for previously-studied software systems. The most frequently occurring defect category is inventory, i.e., the category of defects that occur when managing infrastructure inventory. Using an LLM with heuristics-based paragraph style prompts, we identify 9 previously unknown state reconciliation defects of which 7 have been accepted as valid defects, and 4 have already been fixed. Based on our findings, we conclude the paper by providing a set of recommendations for researchers and practitioners.
Thu 18 JulDisplayed time zone: Brasilia, Distrito Federal, Brazil change
11:00 - 12:30 | Empirical Studies 2Ideas, Visions and Reflections / Research Papers / Industry Papers / Journal First at Acerola Chair(s): Iftekhar Ahmed University of California, Irvine | ||
11:00 18mTalk | State Reconciliation Defects in Infrastructure as Code Research Papers Md Mahadi Hassan Auburn University, John Salvador Auburn University, Shubhra Kanti Karmaker Santu Auburn University, Akond Rahman Auburn University Pre-print | ||
11:18 18mTalk | Understanding and Detecting Annotation-induced Faults of Static Analyzers Research Papers Huaien Zhang The Hong Kong Polytechnic University, Southern University of Science and Technology, Yu Pei The Hong Kong Polytechnic University, Shuyun Liang Southern University of Science and Technology, Shin Hwei Tan Concordia University | ||
11:36 18mTalk | What Makes a Code Review Useful to OpenDev Developers? An Empirical Investigation Journal First | ||
11:54 9mTalk | The Patch Overfitting Problem in Automated Program Repair: Practical Magnitude and a Baseline for Realistic Benchmarking Ideas, Visions and Reflections Justyna Petke University College London, Matias Martinez Universitat Politècnica de Catalunya (UPC), Maria Kechagia University College London, Aldeida Aleti Monash University, Federica Sarro University College London | ||
12:03 9mTalk | Automating Issue Reporting in Software Testing: Lessons Learned from Using the Template Generator Tool Industry Papers Lennon Chaves SIDIA Institute of Science and Technology, Flávia Oliveira Sidia Institute of Science and Technology, Leonardo Tiago Sidia Institute of Science and Technology | ||
12:12 9mTalk | Reproducibility Debt: Challenges and Future Pathways Ideas, Visions and Reflections Zara Hassan Australian National University, Christoph Treude Singapore Management University, Michael Norrish Australian National University, Graham Williams Australian National University, Alex Potanin Australian National University Link to publication DOI | ||
12:21 9mTalk | A Vision on Open Science for the Evolution of Software Engineering Research and Practice Ideas, Visions and Reflections Edson OliveiraJr State University of Maringá, Fernanda Madeiral Vrije Universiteit Amsterdam, Alcemir Rodrigues Santos State University of Piauí, Christina von Flach Federal University of Bahia, Sérgio Soares Universidade Federal de Pernambuco Pre-print | ||