Simple reduction for any* fuzzer-found bug (#2817)

This builds on recent work for deterministic reproduction of fuzzer
testcases using IDs. With that, we can remove all the old auto-reduction
code and make something very similar possible for all* things the
fuzzer script checks for.

The idea is simple: if you run the fuzzer script and it finds a bug,
it prints out the ID it found it with. If you then run

fuzz_opt.py ID

then it runs that exact testcase again, deterministically, making
all the same random choices it made before. The new addition
in this PR is that you can do

fuzz_opt.py ID WASM

which also adds a wasm file. If provided, we still randomly
generate one in the fuzzer script (so that later random numbers
are the same) but we swap in that provided wasm. This then
lets wasm-reduce drive fuzz_opt.py itself as a whole. No more
extracting a testcase and all its commands, it's all done for you.

The fuzzer script will print out hopefully-useful text when it finds
a bug, something like this:

================================================================================
You found a bug! Please report it with

  seed: 4671273171120144526

and the exact version of Binaryen you found it on, plus the exact Python
version (hopefully deterministic random numbers will be identical).

You can run that testcase again with "fuzz_opt.py 4671273171120144526"

The initial wasm file used here is saved as /home/username/binaryen/out/test/original.wasm

You can try to reduce the testcase with

  wasm-reduce /home/username/binaryen/out/test/original.wasm '--command=bash reduce.sh' -t /home/username/binaryen/out/test/t.wasm -w /home/username/binaryen/out/test/w.wasm

where "reduce.sh" is something like

  # check the input is even a valid wasm file
  bin/wasm-opt /home/username/binaryen/out/test/t.wasm
  echo $?

  # run the command
  ./scripts/fuzz_opt.py 4671273171120144526 /home/username/binaryen/out/test/t.wasm > o 2> e
  cat o | tail -n 10
  echo $?

You may want to adjust what is printed there: in the example we save stdout
and stderr separately and then print (so that wasm-reduce can see it) what we
think is the relevant part of that output. Make sure that includes the right
details, and preferably no more (less details allow more reduction, but raise
the risk of it reducing to something you don't quite want).

You may also need to add  --timeout 5  or such if the testcase is a slow one.
================================================================================

The text has full instructions to run the reducer, which should
work in almost all cases - see (*) note below. Because of that
corner case I think it's safer to not run the reducer automatically,
but it's just a quick copy-paste away, and the user can then adjust
the reduce.sh script if necessary.

(*) Well, almost any. There are some corner cases, such as if the
fuzzer generator includes bounds checks in the wasm, reduction
might remove them. We can fix this eventually by making the
bounds checks additions a pass that can be run after the fuzzer
generator, but meanwhile you can work around this by making the
reduction script look for the right thing (i.e. if all it looks for is a
failing return code, that won't be enough as a removed bounds
check will fail but on something else).

0 files changed, 0 insertions, 0 deletions