Fuzzing: ClusterFuzz integration (#7079)

The main addition here is a bundle_clusterfuzz.py script which will package up
the exact files that should be uploaded to ClusterFuzz. It also documents the
process and bundling and testing. You can do

bundle.py OUTPUT_FILE.tgz

That bundles wasm-opt from ./bin., which is enough for local testing. For
actually uploading to ClusterFuzz, we need a portable build, and @dschuff
had the idea to reuse the emsdk build, which works nicely. Doing

bundle.py OUTPUT_FILE.tgz --build-dir=/path/to/emsdk/upstream/

will bundle wasm-opt (+libs) from the emsdk. I verified that those builds
work on ClusterFuzz.

I added several forms of testing here. First, our main fuzzer fuzz_opt.py now
has a ClusterFuzz testcase handler, which simulates a ClusterFuzz environment.
Second, there are smoke tests that run in the unit test suite, and can also be
run separately:

python -m unittest test/unit/test_cluster_fuzz.py

Those unit tests can also run on a given bundle, e.g. one created from an
emsdk build, for testing right before upload:

BINARYEN_CLUSTER_FUZZ_BUNDLE=/path/to/bundle.tgz python -m unittest test/unit/test_cluster_fuzz.py

A third piece of testing is to add a --fuzz-passes test. That is a mode for
-ttf (translate random data into a valid wasm fuzz testcase) that uses random
data to pick and run a set of passes, to further shape the wasm. (--fuzz-passes
had no previous testing, and this PR fixes it and tidies it up a little, adding some
newer passes too).

Otherwise this PR includes the key run.py script that is bundled and then
executed by ClusterFuzz, basically a python script that runs wasm-opt -ttf [..]
to generate testcases, sets up their JS, and emits them.

fuzz_shell.js, which is the JS to execute testcases, will now check if it is
provided binary data of a wasm file. If so, it does not read a wasm file from
argv[1]. (This is needed because ClusterFuzz expects a single file for the
testcase, so we make a JS file with bundled wasm inside it.)

1 files changed, 4 insertions, 4 deletions

diff --git a/test/lit/help/wasm-opt.test b/test/lit/help/wasm-opt.test
index b30f62150..1ac823fa7 100644
--- a/test/lit/help/wasm-opt.test
+++ b/test/lit/help/wasm-opt.test
@@ -41,10 +41,10 @@
 ;; CHECK-NEXT:   --initial-fuzz,-if                            Initial wasm content in
 ;; CHECK-NEXT:                                                 translate-to-fuzz (-ttf) mode
 ;; CHECK-NEXT:
-;; CHECK-NEXT:   --fuzz-passes,-fp                             Pick a random set of passes to
-;; CHECK-NEXT:                                                 run, useful for fuzzing. this
-;; CHECK-NEXT:                                                 depends on translate-to-fuzz (it
-;; CHECK-NEXT:                                                 picks the passes from the input)
+;; CHECK-NEXT:   --fuzz-passes,-fp                             When doing translate-to-fuzz,
+;; CHECK-NEXT:                                                 pick a set of random passes from
+;; CHECK-NEXT:                                                 the input to further shape the
+;; CHECK-NEXT:                                                 wasm
 ;; CHECK-NEXT:
 ;; CHECK-NEXT:   --no-fuzz-memory                              don't emit memory ops when
 ;; CHECK-NEXT:                                                 fuzzing