| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| ... | |
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This builds on recent work for deterministic reproduction of fuzzer
testcases using IDs. With that, we can remove all the old auto-reduction
code and make something very similar possible for all* things the
fuzzer script checks for.
The idea is simple: if you run the fuzzer script and it finds a bug,
it prints out the ID it found it with. If you then run
fuzz_opt.py ID
then it runs that exact testcase again, deterministically, making
all the same random choices it made before. The new addition
in this PR is that you can do
fuzz_opt.py ID WASM
which also adds a wasm file. If provided, we still randomly
generate one in the fuzzer script (so that later random numbers
are the same) but we swap in that provided wasm. This then
lets wasm-reduce drive fuzz_opt.py itself as a whole. No more
extracting a testcase and all its commands, it's all done for you.
The fuzzer script will print out hopefully-useful text when it finds
a bug, something like this:
================================================================================
You found a bug! Please report it with
seed: 4671273171120144526
and the exact version of Binaryen you found it on, plus the exact Python
version (hopefully deterministic random numbers will be identical).
You can run that testcase again with "fuzz_opt.py 4671273171120144526"
The initial wasm file used here is saved as /home/username/binaryen/out/test/original.wasm
You can try to reduce the testcase with
wasm-reduce /home/username/binaryen/out/test/original.wasm '--command=bash reduce.sh' -t /home/username/binaryen/out/test/t.wasm -w /home/username/binaryen/out/test/w.wasm
where "reduce.sh" is something like
# check the input is even a valid wasm file
bin/wasm-opt /home/username/binaryen/out/test/t.wasm
echo $?
# run the command
./scripts/fuzz_opt.py 4671273171120144526 /home/username/binaryen/out/test/t.wasm > o 2> e
cat o | tail -n 10
echo $?
You may want to adjust what is printed there: in the example we save stdout
and stderr separately and then print (so that wasm-reduce can see it) what we
think is the relevant part of that output. Make sure that includes the right
details, and preferably no more (less details allow more reduction, but raise
the risk of it reducing to something you don't quite want).
You may also need to add --timeout 5 or such if the testcase is a slow one.
================================================================================
The text has full instructions to run the reducer, which should
work in almost all cases - see (*) note below. Because of that
corner case I think it's safer to not run the reducer automatically,
but it's just a quick copy-paste away, and the user can then adjust
the reduce.sh script if necessary.
(*) Well, almost any. There are some corner cases, such as if the
fuzzer generator includes bounds checks in the wasm, reduction
might remove them. We can fix this eventually by making the
bounds checks additions a pass that can be run after the fuzzer
generator, but meanwhile you can work around this by making the
reduction script look for the right thing (i.e. if all it looks for is a
failing return code, that won't be enough as a removed bounds
check will fail but on something else).
|
| |
|
|
|
| |
Just like the parent wasm2c, with NaNs don't compare to self before
and after optimizations. The binaryen optimizer does different things than the
LLVM optimizer there, and NaN bits can change.
|
| |
|
|
|
|
| |
Avoid pass-debug when fuzzing emcc, as it can be slow and isn't
what we care about.
Clean up a loop.
|
| |
|
| |
Since the --roundtrip pass is more general than --fuzz-binary anyways. Also reimplements `ModuleUtils::clearModule` to use the module destructor and placement new to ensure that no members are missed.
|
| |
|
|
|
|
|
|
|
| |
This adds a variant on wasm2c that uses emcc instead of a
native compiler. This helps us fuzz emcc.
To make that practical, rewrite the setjmp glue to only use one
setjmp. The wasm backend ends up doing linear work per setjmp,
so it's quadratic with many setjmps. Instead, do a big switch-loop
construct around a single setjmp.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This adds support for fuzzing with wabt's wasm2c that @binji wrote.
Basically we compile the wasm to C, then compile the C to a native
executable with a custom main() to wrap around it. The executable
should then print exactly the same as that wasm when run in either
the binaryen interpreter or in a JS VM with our wrapper JS for that
wasm. In other words, compiling the wasm to C is another way to
run that wasm.
The main reasons I want this are to fuzz wasm2c itself, and to
have another option for fuzzing emcc. For the latter, we do fuzz
wasm-opt quite a lot, but that doesn't fuzz the non-wasm-opt
parts of emcc. And using wasm2c for that is nice since the
starting point is always a wasm file, which means we
can use tools like wasm-reduce and so forth, which can be
integrated with this fuzzer.
This also:
Refactors the fuzzer harness a little to make it easier to
add more "VMs" to run wasms in.
Do not autoreduce when re-running a testcase, which I hit
while developing this.
|
| |
|
|
|
|
|
|
|
| |
1. Only emit exnref as part of a subtype if exception-handling is
enabled in the fuzzer.
2. Correctly report that funcref and nullref require reference-types
to be enabled.
3. Re-enable multivalue as a normal feature in the fuzzer.
Possibly fixes #2770.
|
| | |
|
| |
|
|
|
|
|
|
| |
Emit tuple.make, tuple.extract, and multivalue control flow, and tuple locals
and globals when multivalue is enabled. Also slightly refactors the top-level
`makeConcrete` function to be more selective about what it tries to
make based on the requested type to reduce the number of trivial nodes
created because the requested type is incompatible with the requested
node.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The main benefit here is comparing VMs, instead of just comparing
each VM to itself after opts. Comparing VMs is a little tricky since there
is room for nondeterminism with how results are printed and other
annoying things, which is why that didn't work well earlier.
With this PR I can run 10's of thousands of iterations without finding
any issues between v8 and the binaryen interpreter. That's after
fixing the various issues over the last few days as found by this:
#2760 #2757 #2750 #2752
Aside from that main benefit I ended up adding more improvements
to make it practical to do all that testing:
Randomize global fuzz settings like whether we allow NaNs and
out-of-bounds memory accesses. (This was necessary here since
we have to disable cross-VM comparisons if NaNs are enabled.)
Better logging of statistics like how many times each handler
was run.
Remove redundant FuzzExecImmediately handler (looks like
after past refactorings it was no longer adding any value).
Deterministic testcase handling: if you run e.g. fuzz_opt.py 42 it
will run one testcase and exactly the same one. If you run without
an argument it will run forever until it fails, and if it fails, it prints out
that ID so that you can easily reproduce it (I guess, on the same
binaryen + same python, not sure how python's deterministic
RNG changes between versions and builds).
Upgrade to Python 3.
|
| |
|
|
|
|
|
|
| |
Tuple operations lower to stacky code, so round tripping from IR to
binary and back is a lossy operation. To help make diagnosing bugs
uncovered by the fuzzer easier, this change writes the original IR
generated by the fuzzer and the IR produced by optimizations to files
that can be inspected after a crash to determine exactly what IR was
emitted.
|
| |
|
|
|
|
| |
It is convenient to have the full command when debugging fuzzing errors.
The fuzzer sometimes fails before running `wasm-reduce` and being able
to reproduce the command right away from the log is very handy in that
case.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
This adds support for the reference type proposal. This includes support
for all reference types (`anyref`, `funcref`(=`anyfunc`), and `nullref`)
and four new instructions: `ref.null`, `ref.is_null`, `ref.func`, and
new typed `select`. This also adds subtype relationship support between
reference types.
This does not include table instructions yet. This also does not include
wasm2js support.
Fixes #2444 and fixes #2447.
|
| |
|
|
|
|
| |
This pass writes and reads the module. This shows the effects
of converting to and back from the binary format, and will be
useful in testing dwarf debug support (where we'll need to see
that writing and reading a module preserves debug info properly).
|
| | |
|
| |
|
|
|
| |
Don't directly import names from shared.py and support.py, and use
prefixes instead. Also this reorders imports based on PEP
recommendation.
|
| |
|
|
|
|
|
|
|
| |
- Adds `items` function for `FeatureOptions` so we can get a vector of
eligible types
- Replaces hardcoded enumeration of MVP types with `getConcreteTypes`,
which also adds v128 type to the list if SIMD is enabled
- Removes `getType()` function; this does not seem to be used anywhere
- Renames `vectorPick` with `pick`
- Use the absolute path for d8 in the fuzzer
|
| |
|
|
| |
overhead (#2345)
|
| |
|
|
|
|
|
|
| |
pep8 specifies 4 space indentation. The use of 2 spaces is, I believe
a historical anomaly where certain large organizations such as google
chose 2 over 4 and have yet to make the switch.
Since there isn't too much code in binaryen today it seems reasonable to
make the switch.
|
| | |
|
| |
|
|
| |
Adds tail call support to fuzzer and makes small changes to handle return calls in multiple utilities and passes. Makes larger changes to DAE and inlining passes to properly handle tail calls.
|
| |
|
|
| |
is the one piece of global state we use (#2237)
|
| |
|
|
|
|
|
| |
When it finds a failing testcase, it reduces the list of optimizations, and then runs wasm-reduce to reduce the wasm itself.
This refactors the testcase handlers into two kinds: one returns a list of commands to run (get_commands()), and we can auto-reduce them. The others get all the parameters and do whatever they want internally, and we can't auto-reduce them yet. If it is useful, auto-reducing could be added to the other handlers (CompareVMs, Wasm2JS, etc.) by modifying them to the new form.
Tested manually by breaking stuff.
|
| | |
|
| |
|
|
|
| |
(#2225)
That way we can still test new flags on modes that do support them (e.g. FuzzExec runs on everything)
|
| |
|
|
|
|
|
| |
After some discussion this seems like a less confusing name: what the pass does is "asyncify" code, after all.
The one downside is the name overlaps with the old emscripten "Asyncify" utility, which we'll need to clarify in the docs there.
This keeps the old --bysyncify flag around for now, which is helpful for avoiding temporary breakage on CI as we move the emscripten side as well.
|
| | |
|
| |
|
|
| |
currently just sign-ext...) (#2200)
|
| | |
|
| |
|
|
|
|
|
|
| |
Gets fuzzing support for Bysyncify working.
* Add the python to run the fuzzing on bysyncify.
* Add a JS script to load and run a testcase with bysyncify support. The code has all the runtime support for sleep/resume etc., which it does on calls to imports at random in a deterministic manner.
* Export memory from fuzzer so JS can access it.
* Fix tiny builder bug with makeExport.
|
| |
|
| |
Create a class for handling the current fuzz testcase, and implement subclasses for the various fuzz things we do. This disentangles a lot of code.
|
| |
|
|
|
| |
This is useful for wasm2js, as we don't emit traps for OOB loads etc. like wasm (like we don't trap on bad float-to-int, as it's too hard in JS, and it's undefined behavior in C anyhow). It may also help general fuzzing, as those traps may make other interesting patterns less likely.
Also add more wasm2js support in the fuzzer, which includes using this no-OOB option.
|
| | |
|
| | |
|
| |
|
| |
Also fix the fuzzer's handling of feature flags so that wasm2js can work.
|
| |
|
|
|
|
|
| |
Implement interpretation of remaining bulk memory ops, add bulk memory
spec tests with light modifications, fix bugs preventing the fuzzer
from running correctly with bulk memory, and fix bugs found by the
fuzzer.
|
| |
|
|
|
|
|
| |
This replaces all uses of __tempMemory__, the old scratch space location, with calls to function imports for scratch memory access. This lets us then implement those in a way that does not use the same heap as main memory. This avoids possible bugs with scratch memory overwriting something, or just in general that it has observable side effects, which can confuse fuzzing etc.
The intrinsics are currently implemented in the glue. We could perhaps emit them inline instead (but that might limit asm.js optimizations, so I wanted to keep our options open for now - easy to change later).
Also fixes some places where we used 0 as the scratch space address.
|
| | |
|
| |
|
| |
Get fuzzing to a runnable state.
|
| |
|
|
| |
nans are allowed, as they are nondetermnistic (#1996)
|
| |
|
|
|
|
|
| |
Get fuzzer to attempt to create almost all features. Pass v8 all the flags to allow that.
Fix fuzz bugs where we read signed_ even when it was irrelevant for that type of load.
Improve wasm-reduce on fuzz testcases, try to replace a node with drops of its children, not just the children themselves.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
| |
* make DE_NAN avoid creating nan literals in the first place
* add a reducer option `--denan` to not introduce nans in destructive reduction
* add a `Literal::isNaN()` method
* also remove the default exception logging from the fuzzer js glue, which is a source of non-useful VM differences (like nan nondeterminism)
* added an option `--no-fuzz-nans` to make it easy to avoid nans when fuzzing (without hacking the source and recompiling).
Background: trying to get fuzzing on jsc working despite this open issue: https://bugs.webkit.org/show_bug.cgi?id=175691
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
| |
The main fuzz_opt.py script compares JS VMs, and separately runs binaryen's fuzz-exec that compares the binaryen interpreter to itself (before and after opts). This PR lets us directly compare binaryen's interpreter output to JS VMs. This found a bunch of minor things we can do better on both sides, giving more fuzz coverage.
To enable this, a bunch of tiny fixes were needed:
* Add --fuzz-exec-before which is like --fuzz-exec but just runs the code before opts are run, instead of before and after.
* Normalize double printing (so JS and C++ print comparable things). This includes negative zero in JS, which we never printed properly til now.
* Various improvements to how we print fuzz-exec logging - remove unuseful things, and normalize the others across JS and C++.
* Properly legalize the wasm when --emit-js-wrapper (i.e., we will run the code from JS), and use that in the JS wrapper code.
|
| |
|
|
|
|
|
| |
* Allow fuzzing from other directories, by looking for wasm-opt relative to the script itself.
* Ignore some VM debug assertions which are fuzz bugs that have already been filed.
* Pick the random seed based on the process ID too, for better parallel fuzzing.
* Remove commandline parsing stuff in fuzz_opt.py, which won't work with the other commandline parsing in test.shared - but we don't need it anyhow.
|
| |
|
| |
We iterated over a set. Instead, iterate over the relevant items in their order in the IR.
|
|
|
A small fuzz script I've been using locally. Runs wasm-opt on random inputs and random passes, looking for breakage or the passes changing something. Can also run VMs before and after the passes, and compare the VMs.
|
