Fuzzer: Add an option to fuzz with initial wasm contents (#3276)

Previously the fuzzer constructed a new random valid wasm file from
scratch. The new --initial-fuzz=FILENAME option makes it start from
an existing wasm file, and then add random contents on top of that. It
also randomly modifies the existing contents, for example tweaking
a Const, replacing some nodes with other things of the same type, etc.
It also has a chance to replace a drop with a logging (as some of our
tests just drop a result, and we match the optimized output's wasm
instead of the result; by logging, the fuzzer can check things).

The goal is to find bugs by using existing hand-written testcases as
a basis. This PR uses the test suite's testcases as initial fuzz contents.
This can find issues as they often check for corner cases - they are
designed to be "interesting", which random data may be less likely
to find.

This has found several bugs already, see recent fuzz fixes. I mentioned
the first few on Twitter but past 4 I stopped counting...

https://twitter.com/kripken/status/1314323318036602880

This required various changes to the fuzzer's generation to account
for the fact that there can be existing functions and so forth before
it starts to run, so it needs to avoid collisions and so forth.

1 files changed, 35 insertions, 0 deletions

diff --git a/test/unit/test_initial_fuzz.py b/test/unit/test_initial_fuzz.py
new file mode 100644
index 000000000..2ddc29884
--- /dev/null
+++ b/test/unit/test_initial_fuzz.py
@@ -0,0 +1,35 @@
+import subprocess
+from scripts.test import shared
+from . import utils
+
+
+class InitialFuzzTest(utils.BinaryenTestCase):
+    def test_empty_initial(self):
+        # generate fuzz from random data
+        data = self.input_path('random_data.txt')
+        a = shared.run_process(shared.WASM_OPT + ['-ttf', '--print', data],
+                               stdout=subprocess.PIPE).stdout
+
+        # generate fuzz from random data with initial empty wasm
+        empty_wasm = self.input_path('empty.wasm')
+        b = shared.run_process(
+            shared.WASM_OPT + ['-ttf', '--print', data,
+                               '--initial-fuzz=' + empty_wasm],
+            stdout=subprocess.PIPE).stdout
+
+        # an empty initial wasm causes no changes
+        self.assertEqual(a, b)
+
+    def test_small_initial(self):
+        data = self.input_path('random_data.txt')
+        hello_wat = self.input_path('hello_world.wat')
+        out = shared.run_process(shared.WASM_OPT + ['-ttf', '--print', data,
+                                 '--initial-fuzz=' + hello_wat],
+                                 stdout=subprocess.PIPE).stdout
+
+        # the function should be there (perhaps with modified contents - don't
+        # check that)
+        self.assertIn('(export "add" (func $add))', out)
+
+        # there should be other fuzz contents added as well
+        self.assertGreater(out.count('(export '), 1)