Context: Research artifacts are part of any research project and must be shared to allow Verifiability, Replicability, and Reuse. To promote artifact-sharing initiatives have been adopted by Software Engineering (SE) journals and conferences. However, there is limited knowledge of the extent to which artifact-sharing has happened and evolved in the community, hindering the understanding of the maturity in the field. Objective: To investigate artifact-sharing practices, trends, transparency, and factors guiding researchers’ sharing decisions to help to culture Open Science policies in Software Engineering. Method: We performed a comprehensive study on encouragements, artifact-sharing aspects, and its evolution in the context of the five main ICSE tracks (Technical, SEIP, SEET, SEIS, and NIER) over the last ten editions, from 2014 to 2023. Following this, we intend to develop a guide to best-tested artifact-sharing practices and inappropriate practices for better-sharing research artifacts. Preliminary Results: Our analysis reveals a growing emphasis on artifact-sharing, driven by ICSE’s prominent role in the Software Engineering community, which paves the way for a more transparent research culture. However, challenges remain, as evidenced by broken links and restricted access. Further efforts are necessary to ensure long-term artifact accessibility and reliability. Our findings highlight the need for improved practices in artifact-sharing at ICSE. Standardization, clear communication about availability, and better reporting on artifact evaluation are essential for advancing open and reproducible research. It can be a first step in helping to culture Open Science policies in Software Engineering.

Open Science in Software Engineering: Unpacking the Black Box of Artifact Sharing (AnaVasconcelos-DoctoralSymposium-FSE24.pdf)406KiB

Graduation at Computer Science from Universidade do Estado de Mato Grosso (2008), Master’s at Computer Science from Universidade Federal de Pernambuco (2012), PhD student at Computer Science from Universidade Federal de Pernambuco (2021).

Mon 15 Jul

Displayed time zone: Brasilia, Distrito Federal, Brazil change

16:00 - 18:00
Empirical Aspects + Life After PhD Panel + ClosingDoctoral Symposium at Sapoti
Chair(s): César França CESAR School, Raula Gaikovina Kula Nara Institute of Science and Technology, Ronnie de Souza Santos University of Calgary
16:00
20m
Talk
Open Science in Software Engineering: Unpacking the Black Box of Artifact Sharing
Doctoral Symposium
Ana Paula Vasconcelos Universidade Federal de Pernambuco / Universidade do Estado de Mato Grosso
File Attached
16:20
20m
Talk
The Replication Crisis in Software Engineering: Guidelines for the Scientific Community in dealing with Threats to Validity and their Mitigation Actions and the Impact of the 3R
Doctoral Symposium
Ivanildo Azevedo Universidade Federal de Pernambuco
File Attached
16:40
50m
Panel
Life After PhD Panel
Doctoral Symposium
César França CESAR School, Maria Teresa Baldassarre Department of Computer Science, University of Bari , Raula Gaikovina Kula Nara Institute of Science and Technology, Ronnie de Souza Santos University of Calgary
17:30
10m
Day closing
Closing
Doctoral Symposium
Maria Teresa Baldassarre Department of Computer Science, University of Bari , Wing Lam George Mason University