| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
| |
Can now run scripts/fuzz_opt.py --binaryen-bin build/bin [opts...] to fuzz an
out-of-tree build
Handle positional arguments by looking at shared.requested (with options removed)
instead of raw sys.argv
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Comparing to the interpreter, and not just wasm2js to itself (which we've
done on the same file before and after opts) ensures wasm2js has the right
semantics.
To do this, we need to make sure the wasm doesn't contain things where
wasm2js semantics diverge from normal wasm, which includes:
* Legalize so that there are no i64 exports.
* Remove operations JS can't handle with full precision, like i64 -> f32.
* Force all loads/stores to be 1-byte, as unexpectedly-unaligned operations
fail in wasm2js.
This also requires ignoring subnormals when comparing between JS
VMs and the interpreter.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
wasm2js fuzzing should not compare outputs if the wasm would
trap. wasm2js traps on far fewer things, and if wasm would trap
(like an indirect call with the wrong type) it can just do weird undefined
things. Previously, if running wasm2js trapped then we ignored
the output, but that't not good enough, as we need to check if
wasm would, exactly for the cases just mentioned where wasm
would trap but wasm2js wouldn't. So run the wasm interpreter
to see if that happens.
When we see such a trap, ignore everything from that function
call onwards. This at least lets us compare the results of
previous calls, which adds some amount of coverage (before
we just ignored the entire output completely, so only if there
was no trap at all did we do any comparisons at all).
Also give better names than "js.js" to the JS files wasm2js
fuzzing creates.
|
| |
|
|
| |
compilers (#2961)
|
| |
|
|
|
|
| |
Use WASM_RT_SETJMP so we use sigsetjmp when we need to.
Also disable signals in emcc+wasm2c in the fuzzer. emcc looks like
unix, so it enters the ifdef to use signals, but wasm has no signals...
|
| | |
|
| |
|
|
|
|
|
|
|
| |
This finds out which locals are live at call sites that might pause/resume,
which is the set of locals we need to actually save/load. That is, if a local
is not alive at any call site in the function, then it's value doesn't need to
stay alive while sleeping.
This saves about 10% of locals that are saved/loaded, and about 1.5%
in final code size.
|
| |
|
| |
fixes #2915
|
| |
|
|
|
|
|
|
|
| |
When the fuzzer script is given a wasm we don't create a new
one from scratch. But we should still apply --denan and other
things so that we preserve those properties while reducing.
Without this it's possible for reduction to start with a wasm with
no nans but to lose the property eventually, and end up with a
reduced testcase which is not quite what you want.
|
| |
|
|
|
| |
We already avoid that in CompareVMs but Asyncify has the
same issue, as it also can optimize in binaryen but run in
another VM (with different nondeterministic NaN behavior).
|
| |
|
|
|
|
|
| |
FUZZ_OPTS are things like --denan which affect generation of
the original wasm. We were passing those to testcase handlers,
which meant that in addition to the random optimizations we picked
they were also running --denan etc. again. That can be confusing,
so remove it.
|
| |
|
|
|
|
| |
This moves the fuzzer de-NaN logic out into a separate pass. This is
cleaner and also better since the old way would de-NaN once, but then
the reducer could generate code with nans. The new way lets us de-NaN
while reducing.
|
| | |
|
| |
|
|
|
|
|
| |
Currently the fuzzer only generate a script for wasm-reduce only when it
first discovers a case and doesn't do that when a seed is given. But
when you get a bug report with a seed or you want to reproduce the
situation, is still helpful if the fuzzer generates a reduce script for
you.
|
| |
|
|
|
|
|
| |
Use --emit-target-features and --detect-features.
Rename Asyncify handler temp files (I happened to notice they overwrote other files, which was annoying.
Fixes #2831
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
With this, when it finds a bug all you need to do is copy-paste
a single line and it runs the reducer. To do that, it creates a
reducer script and fills it out for you. The script has all the docs
we used to print out to the console, so the console output is
more focused and concise now.
In most cases just running the single line it suggests should
work, however I found that it doesn't always. One reason is
#2831
Also add a missing sys.exit(1), without which we returned 0
despite printing out an error on a failing testcase.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Don't run wasm2c2wasm on large wasm files, as it can OOM the VM.
I've seen this happen on a single wasm on multiple engines, so it's not
a specific engine bug. Sticking to below-average wasm sizes seems ok.
That change means that we check if a vm can run on a per-wasm basis.
That required some refactoring as it means we may be able to run on
the "before" wasm but not the "after" one, or vice versa.
Fix a tiny nondeterminism issue with iterating on a set().
Some tiny docs improvements to the error shown when a bug is found.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This builds on recent work for deterministic reproduction of fuzzer
testcases using IDs. With that, we can remove all the old auto-reduction
code and make something very similar possible for all* things the
fuzzer script checks for.
The idea is simple: if you run the fuzzer script and it finds a bug,
it prints out the ID it found it with. If you then run
fuzz_opt.py ID
then it runs that exact testcase again, deterministically, making
all the same random choices it made before. The new addition
in this PR is that you can do
fuzz_opt.py ID WASM
which also adds a wasm file. If provided, we still randomly
generate one in the fuzzer script (so that later random numbers
are the same) but we swap in that provided wasm. This then
lets wasm-reduce drive fuzz_opt.py itself as a whole. No more
extracting a testcase and all its commands, it's all done for you.
The fuzzer script will print out hopefully-useful text when it finds
a bug, something like this:
================================================================================
You found a bug! Please report it with
seed: 4671273171120144526
and the exact version of Binaryen you found it on, plus the exact Python
version (hopefully deterministic random numbers will be identical).
You can run that testcase again with "fuzz_opt.py 4671273171120144526"
The initial wasm file used here is saved as /home/username/binaryen/out/test/original.wasm
You can try to reduce the testcase with
wasm-reduce /home/username/binaryen/out/test/original.wasm '--command=bash reduce.sh' -t /home/username/binaryen/out/test/t.wasm -w /home/username/binaryen/out/test/w.wasm
where "reduce.sh" is something like
# check the input is even a valid wasm file
bin/wasm-opt /home/username/binaryen/out/test/t.wasm
echo $?
# run the command
./scripts/fuzz_opt.py 4671273171120144526 /home/username/binaryen/out/test/t.wasm > o 2> e
cat o | tail -n 10
echo $?
You may want to adjust what is printed there: in the example we save stdout
and stderr separately and then print (so that wasm-reduce can see it) what we
think is the relevant part of that output. Make sure that includes the right
details, and preferably no more (less details allow more reduction, but raise
the risk of it reducing to something you don't quite want).
You may also need to add --timeout 5 or such if the testcase is a slow one.
================================================================================
The text has full instructions to run the reducer, which should
work in almost all cases - see (*) note below. Because of that
corner case I think it's safer to not run the reducer automatically,
but it's just a quick copy-paste away, and the user can then adjust
the reduce.sh script if necessary.
(*) Well, almost any. There are some corner cases, such as if the
fuzzer generator includes bounds checks in the wasm, reduction
might remove them. We can fix this eventually by making the
bounds checks additions a pass that can be run after the fuzzer
generator, but meanwhile you can work around this by making the
reduction script look for the right thing (i.e. if all it looks for is a
failing return code, that won't be enough as a removed bounds
check will fail but on something else).
|
| |
|
|
|
| |
Just like the parent wasm2c, with NaNs don't compare to self before
and after optimizations. The binaryen optimizer does different things than the
LLVM optimizer there, and NaN bits can change.
|
| |
|
|
|
|
| |
Avoid pass-debug when fuzzing emcc, as it can be slow and isn't
what we care about.
Clean up a loop.
|
| |
|
| |
Since the --roundtrip pass is more general than --fuzz-binary anyways. Also reimplements `ModuleUtils::clearModule` to use the module destructor and placement new to ensure that no members are missed.
|
| |
|
|
|
|
|
|
|
| |
This adds a variant on wasm2c that uses emcc instead of a
native compiler. This helps us fuzz emcc.
To make that practical, rewrite the setjmp glue to only use one
setjmp. The wasm backend ends up doing linear work per setjmp,
so it's quadratic with many setjmps. Instead, do a big switch-loop
construct around a single setjmp.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This adds support for fuzzing with wabt's wasm2c that @binji wrote.
Basically we compile the wasm to C, then compile the C to a native
executable with a custom main() to wrap around it. The executable
should then print exactly the same as that wasm when run in either
the binaryen interpreter or in a JS VM with our wrapper JS for that
wasm. In other words, compiling the wasm to C is another way to
run that wasm.
The main reasons I want this are to fuzz wasm2c itself, and to
have another option for fuzzing emcc. For the latter, we do fuzz
wasm-opt quite a lot, but that doesn't fuzz the non-wasm-opt
parts of emcc. And using wasm2c for that is nice since the
starting point is always a wasm file, which means we
can use tools like wasm-reduce and so forth, which can be
integrated with this fuzzer.
This also:
Refactors the fuzzer harness a little to make it easier to
add more "VMs" to run wasms in.
Do not autoreduce when re-running a testcase, which I hit
while developing this.
|
| |
|
|
|
|
|
|
|
| |
1. Only emit exnref as part of a subtype if exception-handling is
enabled in the fuzzer.
2. Correctly report that funcref and nullref require reference-types
to be enabled.
3. Re-enable multivalue as a normal feature in the fuzzer.
Possibly fixes #2770.
|
| | |
|
| |
|
|
|
|
|
|
| |
Emit tuple.make, tuple.extract, and multivalue control flow, and tuple locals
and globals when multivalue is enabled. Also slightly refactors the top-level
`makeConcrete` function to be more selective about what it tries to
make based on the requested type to reduce the number of trivial nodes
created because the requested type is incompatible with the requested
node.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The main benefit here is comparing VMs, instead of just comparing
each VM to itself after opts. Comparing VMs is a little tricky since there
is room for nondeterminism with how results are printed and other
annoying things, which is why that didn't work well earlier.
With this PR I can run 10's of thousands of iterations without finding
any issues between v8 and the binaryen interpreter. That's after
fixing the various issues over the last few days as found by this:
#2760 #2757 #2750 #2752
Aside from that main benefit I ended up adding more improvements
to make it practical to do all that testing:
Randomize global fuzz settings like whether we allow NaNs and
out-of-bounds memory accesses. (This was necessary here since
we have to disable cross-VM comparisons if NaNs are enabled.)
Better logging of statistics like how many times each handler
was run.
Remove redundant FuzzExecImmediately handler (looks like
after past refactorings it was no longer adding any value).
Deterministic testcase handling: if you run e.g. fuzz_opt.py 42 it
will run one testcase and exactly the same one. If you run without
an argument it will run forever until it fails, and if it fails, it prints out
that ID so that you can easily reproduce it (I guess, on the same
binaryen + same python, not sure how python's deterministic
RNG changes between versions and builds).
Upgrade to Python 3.
|
| |
|
|
|
|
|
|
| |
Tuple operations lower to stacky code, so round tripping from IR to
binary and back is a lossy operation. To help make diagnosing bugs
uncovered by the fuzzer easier, this change writes the original IR
generated by the fuzzer and the IR produced by optimizations to files
that can be inspected after a crash to determine exactly what IR was
emitted.
|
| |
|
|
|
|
| |
It is convenient to have the full command when debugging fuzzing errors.
The fuzzer sometimes fails before running `wasm-reduce` and being able
to reproduce the command right away from the log is very handy in that
case.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
This adds support for the reference type proposal. This includes support
for all reference types (`anyref`, `funcref`(=`anyfunc`), and `nullref`)
and four new instructions: `ref.null`, `ref.is_null`, `ref.func`, and
new typed `select`. This also adds subtype relationship support between
reference types.
This does not include table instructions yet. This also does not include
wasm2js support.
Fixes #2444 and fixes #2447.
|
| |
|
|
|
|
| |
This pass writes and reads the module. This shows the effects
of converting to and back from the binary format, and will be
useful in testing dwarf debug support (where we'll need to see
that writing and reading a module preserves debug info properly).
|
| | |
|
| |
|
|
|
| |
Don't directly import names from shared.py and support.py, and use
prefixes instead. Also this reorders imports based on PEP
recommendation.
|
| |
|
|
|
|
|
|
|
| |
- Adds `items` function for `FeatureOptions` so we can get a vector of
eligible types
- Replaces hardcoded enumeration of MVP types with `getConcreteTypes`,
which also adds v128 type to the list if SIMD is enabled
- Removes `getType()` function; this does not seem to be used anywhere
- Renames `vectorPick` with `pick`
- Use the absolute path for d8 in the fuzzer
|
| |
|
|
| |
overhead (#2345)
|
| |
|
|
|
|
|
|
| |
pep8 specifies 4 space indentation. The use of 2 spaces is, I believe
a historical anomaly where certain large organizations such as google
chose 2 over 4 and have yet to make the switch.
Since there isn't too much code in binaryen today it seems reasonable to
make the switch.
|
| | |
|
| |
|
|
| |
Adds tail call support to fuzzer and makes small changes to handle return calls in multiple utilities and passes. Makes larger changes to DAE and inlining passes to properly handle tail calls.
|
| |
|
|
| |
is the one piece of global state we use (#2237)
|
| |
|
|
|
|
|
| |
When it finds a failing testcase, it reduces the list of optimizations, and then runs wasm-reduce to reduce the wasm itself.
This refactors the testcase handlers into two kinds: one returns a list of commands to run (get_commands()), and we can auto-reduce them. The others get all the parameters and do whatever they want internally, and we can't auto-reduce them yet. If it is useful, auto-reducing could be added to the other handlers (CompareVMs, Wasm2JS, etc.) by modifying them to the new form.
Tested manually by breaking stuff.
|
| | |
|
| |
|
|
|
| |
(#2225)
That way we can still test new flags on modes that do support them (e.g. FuzzExec runs on everything)
|
| |
|
|
|
|
|
| |
After some discussion this seems like a less confusing name: what the pass does is "asyncify" code, after all.
The one downside is the name overlaps with the old emscripten "Asyncify" utility, which we'll need to clarify in the docs there.
This keeps the old --bysyncify flag around for now, which is helpful for avoiding temporary breakage on CI as we move the emscripten side as well.
|
| | |
|
| |
|
|
| |
currently just sign-ext...) (#2200)
|
| | |
|
| |
|
|
|
|
|
|
| |
Gets fuzzing support for Bysyncify working.
* Add the python to run the fuzzing on bysyncify.
* Add a JS script to load and run a testcase with bysyncify support. The code has all the runtime support for sleep/resume etc., which it does on calls to imports at random in a deterministic manner.
* Export memory from fuzzer so JS can access it.
* Fix tiny builder bug with makeExport.
|
| |
|
| |
Create a class for handling the current fuzz testcase, and implement subclasses for the various fuzz things we do. This disentangles a lot of code.
|
| |
|
|
|
| |
This is useful for wasm2js, as we don't emit traps for OOB loads etc. like wasm (like we don't trap on bad float-to-int, as it's too hard in JS, and it's undefined behavior in C anyhow). It may also help general fuzzing, as those traps may make other interesting patterns less likely.
Also add more wasm2js support in the fuzzer, which includes using this no-OOB option.
|
