Fuzz testing, also known as fuzzing, is a software testing technique aimed at identifying software vulnerabilities. In recent decades, fuzzing has gained increasing popularity in the research community. However, existing studies led by fuzzing experts mainly focus on improving the coverage and performance of fuzzing techniques. That is, there is still a gap in empirical knowledge regarding fuzzing, especially about the challenges developers face when they adopt fuzzing. Understanding these challenges can provide valuable insights to both practitioners and researchers on how to further improve fuzzing processes and techniques.

\textbf{Manual analysis.} We conducted a study to understand the challenges encountered by developers during fuzzing. More specifically, we first manually analyzed 829 randomly sampled fuzzing related GitHub issues and constructed a taxonomy consisting of 39 types of challenges (22 related to the fuzzing process itself, 17 related to using external fuzzing providers). Our manual analysis’ results show that the usability of fuzzers is the most common fuzzing related challenge. More specifically, we find that developers experience significant difficulties when trying to set up, build or use fuzzers. We also find that setting up the fuzzing environment and compiling the fuzzers are common problems faced by developers.

\textbf{Survey with practitioners.} We then surveyed 106 fuzzing practitioners to verify the validity of our taxonomy and collected feedback on how the fuzzing process can be improved. The fuzzing experts’ testimonies reveal several new issues in the field of fuzzing, namely: 1) new fuzzing tools and techniques developed by academics not being useful or usable for the general fuzzing community, 2) fuzzing requiring too much resources and computing power to scale, and 3) the very high barrier of entry preventing average developers from adopting fuzzing for their own projects. Furthermore, multiple experts confirmed the results of our manual analysis by stating that fuzzers are difficult to use alongside build systems. From the survey replies, we also find that practitioners overall have a good experience using continuous fuzzing services such as OSS-Fuzz and that OSS-Fuzz does help alleviate some of the performance problems related to fuzzing.

Our taxonomy, accompanied with representative examples and highlighted implications, can serve as a reference point on how to better adopt fuzzing techniques for practitioners, and indicates potential directions researchers can work on toward better fuzzing approaches and practices.