Cybersecurity concerns about Internet of Things (IoT) devices and infrastructure are growing each year. In response, organizations worldwide have published IoT cybersecurity guidelines to protect their citizens and customers. These guidelines constrain the development of IoT systems, which include substantial software components both on-device and in the Cloud. While these guidelines are being widely adopted, e.g. by US federal contractors, their content and merits have not been critically examined. Two notable gaps are: (1) We do not know how these guidelines differ by the topics and details of their recommendations; and (2) We do not know how effective they are at mitigating real-world IoT failures.

In this paper, we address these questions through a qualitative study of IoT cybersecurity guidelines. We collected 142 IoT cybersecurity guidelines, sampling them for recommendations until saturation was reached at 25 guidelines. From the resulting 958 unique recommendations, we iteratively developed a hierarchical taxonomy following grounded theory coding principles. We measured the guidelines’ usefulness by asking novice engineers about the actionability of each recommendation, and by matching cybersecurity recommendations to the root causes of failures (CVEs and news stories). We report that: (1) Each guideline has gaps in its topic coverage and comprehensiveness; (2) 87.2% recommendations are actionable and 38.7% recommendations can prevent specific threats; and (3) although the union of the guidelines mitigates all 17 of the failures from our news stories corpus, 21% of the CVEs evade the guidelines. In summary, we report shortcomings in each guideline’s depth and breadth, but as a whole they address major security issues. Our results will help software engineers determine which and how many guidelines to study as they implement IoT systems and help publishers improve their guidelines.