Skip to content

fitzgen/fuzz-experiment

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

11 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

A Structure-Aware Fuzzing Experiment

Structure-aware fuzzing can better exercise the system under test (SUT) by crafting inputs in the format expected by the SUT, rather than throwing pseudorandom bytes against it. That is, it avoids "shallow" inputs that the SUT will reject early (for example, syntactically invalid source text when fuzzing a programming language's compiler) and only produces inputs that go "deep" into the SUT (e.g. programs that type-check and exercise the mid-end optimizer and backend code generator). The Rust fuzzing ecosystem is largely built around cargo-fuzz and the libfuzzer-sys crate, which provides two methods for structure-aware fuzzing:

  1. Generating structured inputs from scratch with the arbitrary crate

  2. Mutating existing inputs from the fuzzer's corpus in a structure-aware manner, thereby producing new structured inputs, via the fuzz_mutator! hook

While the two methods are not technically mutually exclusive, combining the two can be difficult and engineering resources are finite. So:

If we are only implementing one approach, is generation or mutation better?

Full write up here: https://fitzgen.com/2026/06/01/structure-aware-fuzzing-experiment.html

Benchmark Results

24 Hours of Fuzzing

  • arb has 1.00 +/- 0.00 times more coverage than bottom_up (p = 0.01)
  • mutate has 1.01 +/- 0.00 times more coverage than arb (p = 0.00)
  • top_down has 1.00 +/- 0.00 times more coverage than arb (p = 0.00)
  • mutate has 1.02 +/- 0.00 times more coverage than bottom_up (p = 0.00)
  • top_down has 1.01 +/- 0.00 times more coverage than bottom_up (p = 0.00)
  • mutate has 1.01 +/- 0.00 times more coverage than top_down (p = 0.00)

Image

5 Minutes of Fuzzing

  • bottom_up has 1.01 +/- 0.01 times more coverage than arb (p = 0.04)
  • mutate has 1.47 +/- 0.02 times more coverage than arb (p = 0.00)
  • top_down has 1.06 +/- 0.02 times more coverage than arb (p = 0.00)
  • mutate has 1.45 +/- 0.01 times more coverage than bottom_up (p = 0.00)
  • top_down has 1.05 +/- 0.02 times more coverage than bottom_up (p = 0.00)
  • mutate has 1.38 +/- 0.02 times more coverage than top_down (p = 0.00)

Image

Running the Benchmarks Yourself

# Run the benchmarks.
cargo run --bin benchmark -- path/to/output

# Analyze the benchmark data.
cargo run --bin analyze -- path/to/output

Run with --help to see all supported flags and options.

About

Different approaches to structure-aware fuzzing

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages