LibAFL Code Refactoring & Cleanup

[addisoncrump]

We have lots of places where we have type constraints that are not necessary, old code that is not supported/unused, things which we have no tests for, code is disorganised due to the many hands problem, missing documentation, code in the wrong place (e.g., libafl_libfuzzer has a lot of components that should be in libafl_targets), etc. We need to do some spring cleaning for 0.13!

After each file is checked, either a PR is made with the house cleaning items for that file or it is checked off if there is nothing to do. I think if we focus on this one file at a time, we can crack through it in a few days.

If you are not a maintainer and want to do some of these tasks: please open an issue with the list of files/components you want to tackle so we don't duplicate effort.

Specific things to look for:

• Wherever possible, remove #[allow(...)] and apply the clippy fix instead. This was likely done as an easy way out.
• Wherever possible, use Cow<'static, str> instead of String.
• Fix documentation to avoid obvious things, e.g. "this is a constructor", "this is a module". Write usage notes instead.
• Fix documentation where old information is provided.
• #[deprecate = "<reason>"] tags for no-longer-supported things, or things with better alternatives.
• Remove generic restrictions at the definitions (e.g., we do not need to specify that types impl Serialize, Deserialize, or Debug anymore at the struct definitions).
• Reduce generics to the least restrictive necessary.
• If we could (performantly) use safe code instead of unsafe code: rewrite.
• Traits which have an associated type should refer to the associated type, not the concrete/generic.
• PhantomData should have the smallest set of types needed.
• Wherever possible, trait implementations with lifetime specifiers should use '_ lifetime elision.
• Manual Debug impls should be replaced with .debug_struct chains.
• Complex constructors should be replaced with typed_builder.

The following is a checklist of files which need to be cleaned in the main library files:

`libafl`

• libafl/src/corpus/cached.rs
• libafl/src/corpus/inmemory.rs
• libafl/src/corpus/inmemory_ondisk.rs
• libafl/src/corpus/minimizer.rs
• libafl/src/corpus/mod.rs
• libafl/src/corpus/nop.rs
• libafl/src/corpus/ondisk.rs
• libafl/src/corpus/testcase.rs
• libafl/src/events/hooks/mod.rs
• libafl/src/events/centralized.rs
• libafl/src/events/launcher.rs
• libafl/src/events/llmp.rs
• libafl/src/events/mod.rs
• libafl/src/events/simple.rs
• libafl/src/events/tcp.rs
• libafl/src/executors/combined.rs
• libafl/src/executors/command.rs
• libafl/src/executors/hooks/inprocess.rs
• libafl/src/executors/hooks/inprocess_fork.rs
• libafl/src/executors/hooks/mod.rs
• libafl/src/executors/hooks/timer.rs
• libafl/src/executors/hooks/unix.rs
• libafl/src/executors/hooks/windows.rs
• libafl/src/executors/shadow.rs
• libafl/src/executors/with_observers.rs
• libafl/src/executors/inprocess/inner.rs
• libafl/src/executors/inprocess/stateful.rs
• libafl/src/executors/inprocess/mod.rs
• libafl/src/executors/inprocess_fork/inner.rs
• libafl/src/executors/inprocess_fork/mod.rs
• libafl/src/executors/inprocess_fork/stateful.rs
• libafl/src/executors/differential.rs
• libafl/src/executors/forkserver.rs
• libafl/src/executors/mod.rs
• libafl/src/feedbacks/concolic.rs
• libafl/src/feedbacks/differential.rs
• libafl/src/feedbacks/list.rs
• libafl/src/feedbacks/mod.rs
• libafl/src/feedbacks/nautilus.rs
• libafl/src/feedbacks/new_hash_feedback.rs
• libafl/src/feedbacks/stdio.rs
• libafl/src/feedbacks/transferred.rs
• libafl/src/feedbacks/map.rs
• libafl/src/fuzzer/mod.rs
• libafl/src/generators/gramatron.rs
• libafl/src/generators/nautilus.rs
• libafl/src/generators/mod.rs
• libafl/src/inputs/bytes.rs
• libafl/src/inputs/encoded.rs
• libafl/src/inputs/gramatron.rs
• libafl/src/inputs/mod.rs
• libafl/src/inputs/multi.rs
• libafl/src/inputs/nautilus.rs
• libafl/src/inputs/generalized.rs
• libafl/src/monitors/tui/mod.rs
• libafl/src/monitors/tui/ui.rs
• libafl/src/monitors/disk.rs
• libafl/src/monitors/multi.rs
• libafl/src/monitors/prometheus.rs
• libafl/src/monitors/mod.rs
• libafl/src/mutators/string/unicode_categories.rs
• libafl/src/mutators/encoded_mutations.rs
• libafl/src/mutators/gramatron.rs
• libafl/src/mutators/grimoire.rs
• libafl/src/mutators/mod.rs
• libafl/src/mutators/mopt_mutator.rs
• libafl/src/mutators/multi.rs
• libafl/src/mutators/mutations.rs
• libafl/src/mutators/nautilus.rs
• libafl/src/mutators/scheduled.rs
• libafl/src/mutators/string.rs
• libafl/src/mutators/token_mutations.rs
• libafl/src/mutators/tuneable.rs
• libafl/src/observers/concolic/observer.rs
• libafl/src/observers/concolic/metadata.rs
• libafl/src/observers/concolic/mod.rs
• libafl/src/observers/concolic/serialization_format.rs
• libafl/src/observers/stdio.rs
• libafl/src/observers/stacktrace.rs
• libafl/src/observers/value.rs
• libafl/src/observers/cmp.rs
• libafl/src/observers/list.rs
• libafl/src/observers/mod.rs
• libafl/src/observers/map.rs
• libafl/src/schedulers/accounting.rs
• libafl/src/schedulers/minimizer.rs
• libafl/src/schedulers/powersched.rs
• libafl/src/schedulers/probabilistic_sampling.rs
• libafl/src/schedulers/queue.rs
• libafl/src/schedulers/testcase_score.rs
• libafl/src/schedulers/tuneable.rs
• libafl/src/schedulers/weighted.rs
• libafl/src/schedulers/mod.rs
• libafl/src/stages/push/mod.rs
• libafl/src/stages/push/mutational.rs
• libafl/src/stages/logics.rs
• libafl/src/stages/calibrate.rs
• libafl/src/stages/concolic.rs
• libafl/src/stages/dump.rs
• libafl/src/stages/generalization.rs
• libafl/src/stages/mod.rs
• libafl/src/stages/mutational.rs
• libafl/src/stages/power.rs
• libafl/src/stages/stats.rs
• libafl/src/stages/string.rs
• libafl/src/stages/sync.rs
• libafl/src/stages/tracing.rs
• libafl/src/stages/tuneable.rs
• libafl/src/stages/colorization.rs
• libafl/src/stages/tmin.rs
• libafl/src/state/mod.rs
• libafl/src/common/mod.rs
• libafl/src/lib.rs
• libafl/build.rs

`libafl_bolts`

• libafl_bolts/examples/llmp_test/main.rs
• libafl_bolts/src/compress.rs
• libafl_bolts/src/os/pipes.rs
• libafl_bolts/src/os/unix_shmem_server.rs
• libafl_bolts/src/os/unix_signals.rs
• libafl_bolts/src/os/windows_exceptions.rs
• libafl_bolts/src/os/mod.rs
• libafl_bolts/src/build_id.rs
• libafl_bolts/src/staterestore.rs
• libafl_bolts/src/fs.rs
• libafl_bolts/src/math.rs
• libafl_bolts/src/minibsod.rs
• Improve libafl_bolts/rands.rs #2101
• libafl_bolts/src/anymap.rs
• libafl_bolts/src/cli.rs
• libafl_bolts/src/core_affinity.rs
• libafl_bolts/src/cpu.rs
• libafl_bolts/src/llmp.rs
• libafl_bolts/src/ownedref.rs
• libafl_bolts/src/serdeany.rs
• libafl_bolts/src/shmem.rs
• libafl_bolts/src/lib.rs
• libafl_bolts/src/tuples.rs
• libafl_bolts/build.rs

`libafl_cc`

• libafl_cc/src/ar.rs
• libafl_cc/src/lib.rs
• libafl_cc/src/libtool.rs
• libafl_cc/src/clang.rs
• libafl_cc/src/cfg.rs
• libafl_cc/build.rs

`libafl_concolic`

• libafl_concolic/symcc_libafl/src/lib.rs
• libafl_concolic/symcc_runtime/src/filter/coverage.rs
• libafl_concolic/symcc_runtime/src/filter.rs
• libafl_concolic/symcc_runtime/src/tracing.rs
• libafl_concolic/symcc_runtime/src/lib.rs
• libafl_concolic/symcc_runtime/build.rs
• libafl_concolic/test/dump_constraints/src/main.rs
• libafl_concolic/test/runtime_test/src/lib.rs

`libafl_derive` (does anyone use this?)

• libafl_derive/src/lib.rs

`libafl_frida`

• libafl_frida/src/asan/mod.rs
• libafl_frida/src/asan/asan_rt.rs
• libafl_frida/src/asan/errors.rs
• libafl_frida/src/asan/hook_funcs.rs
• libafl_frida/src/windows_hooks.rs
• libafl_frida/src/drcov_rt.rs
• libafl_frida/src/coverage_rt.rs
• libafl_frida/src/alloc.rs
• libafl_frida/src/cmplog_rt.rs
• libafl_frida/src/executor.rs
• libafl_frida/src/helper.rs
• libafl_frida/src/lib.rs
• libafl_frida/src/pthread_hook.rs
• libafl_frida/src/utils.rs
• libafl_frida/build.rs

`libafl_libfuzzer`

• libafl_libfuzzer/libafl_libfuzzer_runtime/src/tmin.rs
• libafl_libfuzzer/libafl_libfuzzer_runtime/src/corpus.rs
• libafl_libfuzzer/libafl_libfuzzer_runtime/src/feedbacks.rs
• libafl_libfuzzer/libafl_libfuzzer_runtime/src/fuzz.rs
• libafl_libfuzzer/libafl_libfuzzer_runtime/src/lib.rs
• libafl_libfuzzer/libafl_libfuzzer_runtime/src/merge.rs
• libafl_libfuzzer/libafl_libfuzzer_runtime/src/misc.rs
• libafl_libfuzzer/libafl_libfuzzer_runtime/src/options.rs
• libafl_libfuzzer/libafl_libfuzzer_runtime/src/report.rs
• libafl_libfuzzer/libafl_libfuzzer_runtime/src/schedulers.rs
• libafl_libfuzzer/libafl_libfuzzer_runtime/src/observers.rs
• libafl_libfuzzer/libafl_libfuzzer_runtime/build.rs
• libafl_libfuzzer/src/lib.rs
• libafl_libfuzzer/build.rs

`libafl_nyx`

• libafl_nyx/build.rs
• libafl_nyx/src/executor.rs
• libafl_nyx/src/helper.rs
• libafl_nyx/src/lib.rs
• libafl_nyx/src/settings.rs

`libafl_qemu`

• libafl_qemu/libafl_qemu_build/src/main.rs
• libafl_qemu/libafl_qemu_build/src/bindings.rs
• libafl_qemu/libafl_qemu_build/src/build.rs
• libafl_qemu/libafl_qemu_build/src/lib.rs
• libafl_qemu/libafl_qemu_sys/build.rs
• libafl_qemu/libafl_qemu_sys/src/lib.rs
• libafl_qemu/libafl_qemu_sys/src/systemmode.rs
• libafl_qemu/libafl_qemu_sys/src/usermode.rs
• libafl_qemu/libafl_qemu_sys/src/x86_64_stub_bindings.rs
• libafl_qemu/libafl_qemu_sys/build_linux.rs
• libafl_qemu/src/executor/mod.rs
• libafl_qemu/src/executor/stateful.rs
• libafl_qemu/src/aarch64.rs
• libafl_qemu/src/arm.rs
• libafl_qemu/src/asan.rs
• libafl_qemu/src/asan_guest.rs
• libafl_qemu/src/breakpoint.rs
• libafl_qemu/src/calls.rs
• libafl_qemu/src/cmplog.rs
• libafl_qemu/src/command.rs
• libafl_qemu/src/drcov.rs
• libafl_qemu/src/elf.rs
• libafl_qemu/src/emu.rs
• libafl_qemu/src/emu/systemmode.rs
• libafl_qemu/src/emu/usermode.rs
• libafl_qemu/src/helper.rs
• libafl_qemu/src/hexagon.rs
• libafl_qemu/src/hooks.rs
• libafl_qemu/src/i386.rs
• libafl_qemu/src/injections.rs
• libafl_qemu/src/lib.rs
• libafl_qemu/src/mips.rs
• libafl_qemu/src/ppc.rs
• libafl_qemu/src/snapshot.rs
• libafl_qemu/src/sync_backdoor.rs
• libafl_qemu/src/x86_64.rs
• libafl_qemu/src/edges.rs
• libafl_qemu/build.rs
• libafl_qemu/build_linux.rs

`libafl_sugar`

• libafl_sugar/build.rs
• libafl_sugar/src/lib.rs
• libafl_sugar/src/inmemory.rs
• libafl_sugar/src/qemu.rs
• libafl_sugar/src/forkserver.rs

`libafl_targets`

• libafl_targets/src/forkserver.rs
• libafl_targets/src/value_profile.rs
• libafl_targets/src/drcov.rs
• libafl_targets/src/libfuzzer/observers/mod.rs
• libafl_targets/src/libfuzzer/observers/oom.rs
• libafl_targets/src/libfuzzer/mod.rs
• libafl_targets/src/libfuzzer/mutators.rs
• libafl_targets/src/cmps/observers/mod.rs
• libafl_targets/src/cmps/observers/aflpp.rs
• libafl_targets/src/cmps/observers/cmplog.rs
• libafl_targets/src/cmps/stages/mod.rs
• libafl_targets/src/cmps/stages/aflpptracing.rs
• libafl_targets/src/cmps/mod.rs
• libafl_targets/src/windows_asan.rs
• libafl_targets/src/lib.rs
• libafl_targets/src/sancov_cmp.rs
• libafl_targets/src/sancov_8bit.rs
• libafl_targets/src/coverage.rs
• libafl_targets/src/sancov_pcguard.rs
• libafl_targets/build.rs

`libafl_tinyinst`

• libafl_tinyinst/src/lib.rs
• libafl_tinyinst/src/executor.rs

[domenukk]

I would say this is a good step towards 1.0

[flyingmutant]

I want to take on libafl_bolts/rands.rs. I've already submitted a couple of improvement PRs there, and would like to finish the job.