Improving search space analysis of fuzzing mutators using cryptographic structures

Chafjiri, Sadegh Bamohabbat; Legg, Phil; Tsompanas, Michail-Antisthenis; Hong, Jun

Abstract

This paper introduces a novel approach to enhance the performance of software fuzzing mutator tools, by leveraging cryptographic structures known as substitution-permutation networks and Feistel networks. By integrating these structures into the existing HonggFuzz fuzzing library, we propose HonggFuzz+ and demonstrate its effectiveness over other leading fuzzers, such as how the method can uncover bugs and edges earlier due to enhanced search space optimisation. By introducing these two structures, we can diversify memory region relationships that can ultimately improve the performance of HonggFuzz. We demonstrate our approach on a range of common software examples from previous software fuzzing literature. Our results show better or as good performance across a range of software targets when compared to other leading fuzzing techniques. We discuss the relevance of the findings and consider future directions for improving software fuzzing search space analysis.

• Presentation Conference Type: Conference Paper (published)
• Conference Name: International Conference on Cyber Security and Privacy
• Start Date: Dec 11, 2023
• End Date: Dec 12, 2023
• Acceptance Date: Nov 16, 2023
• Online Publication Date: Sep 18, 2024
• Publication Date: Sep 18, 2024
• Deposit Date: Dec 15, 2023
• Publicly Available Date: Sep 19, 2025
• Publisher: Springer
• Pages: 153-172
• Series Title: Lecture Notes in Networks and Systems
• Series Number: 1032
• Series ISSN: 2367-3389
• Book Title: AI Applications in Cyber Security and Communication Networks: Proceedings of Ninth International Conference on Cyber Security, Privacy in Communication Networks (ICCS 2023)
• ISBN: 9789819739721
• DOI: https://doi.org/10.1007/978-981-97-3973-8_10
• Public URL: https://uwe-repository.worktribe.com/output/11517833