| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
| |
* Add a new "sleep" fuzzer import, that does a sleep for some ms.
* Add JSPI support in fuzz_shell.js. This is in the form of commented-out async/await
keywords - commented out so that normal fuzzing is not impacted. When we want
to fuzz JSPI, we uncomment them. We also apply the JSPI operations of marking
imports and exports as suspending/promising.
JSPI fuzzing is added to both fuzz_opt.py and ClusterFuzz's run.py.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Similar to call-export*, these imports call a wasm function from outside the
module. The difference is that we send a function reference for them to call
(rather than an export index).
This gives more coverage, first by sending a ref from wasm to JS, and also
since we will now try to call anything that is sent. Exports, in comparison,
are filtered by the fuzzer to things that JS can handle, so this may lead to
more traps, but maybe also some new situations. This also leads to adding
more logic to execution-results.h to model JS trapping properly.
fuzz_shell.js is refactored to allow sharing code between call-export* and
call-ref*.
|
| |
|
|
|
|
|
|
|
|
| |
LLVM recently split the bulk-memory-opt feature out from bulk-memory,
containing just memory.copy and memory.fill. This change follows that,
making bulk-memory-opt also enabled when all of bulk-memory is enabled.
It also introduces call-indirect-overlong following LLVM, but ignores
it, since Binaryen has always allowed the encoding (i.e. command
line flags enabling or disabling the feature are accepted but
ignored).
|
| |
|
|
|
|
|
|
|
|
| |
RemoveUnusedBrs sinks blocks into If arms when those arms contain
branches to the blocks and the other arm and condition do not. Now that
we type Ifs with unreachable conditions as unreachable, it is possible
for the If arms to have a different type than the block that would be
sunk, so sinking the block would produce invalid IR. Fix the problem by
never sinking blocks into Ifs with unreachable conditions.
Fixes #7128.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
IRBuilder is a utility for turning arbitrary valid streams of Wasm
instructions into valid Binaryen IR. It is already used in the text
parser, so now use it in the binary parser as well. Since the IRBuilder
API for building each intruction requires only the information that the
binary and text formats include as immediates to that instruction, the
parser is now much simpler than before. In particular, it does not need
to manage a stack of instructions to figure out what the children of
each expression should be; IRBuilder handles this instead.
There are some differences between the IR constructed by IRBuilder and
the IR the binary parser constructed before this change. Most
importantly, IRBuilder generates better multivalue code because it
avoids eagerly breaking up multivalue results into individual components
that might need to be immediately reassembled into a tuple. It also
parses try-delegate more correctly, allowing the delegate to target
arbitrary labels, not just other `try`s. There are also a couple
superficial differences in the generated label and scratch local names.
As part of this change, add support for recording binary source
locations in IRBuilder.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Previously the only Ifs that were typed unreachable were those in which
both arms were unreachable and those in which the condition was
unreachable that would have otherwise been typed none. This caused
problems in IRBuilder because Ifs with unreachable conditions and
value-returning arms would have concrete types, effectively hiding the
unreachable condition from the logic for dropping concretely typed
expressions preceding an unreachable expression when finishing a scope.
Relax the conditions under which an If can be typed unreachable so that
all Ifs with unreachable conditions or two unreachable arms are typed
unreachable. Propagating unreachability more eagerly this way makes
various optimizations of Ifs more powerful. It also requires new
handling for unreachable Ifs with concretely typed arms in the Printer
to ensure that printed wat remains valid.
Also update Unsubtyping, Flatten, and CodeFolding to account for the
newly unreachable Ifs.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
CodeFolding previously only worked on blocks that did not produce
values. It worked on Ifs that produced values, but only by accident; the
logic for folding matching tails was not written to support tails
producing concrete values, but it happened to work for Ifs because
subsequent ReFinalize runs fixed all the incorrect types it produced.
Improve the power of the optimization by explicitly handling tails that
produce concrete values for both blocks and ifs. Now that the core logic
handles concrete values correctly, remove the unnecessary ReFinalize
run.
Also remove the separate optimization of Ifs with identical arms; this
optimization requires ReFinalize and is already performed by
OptimizeInstructions.
|
| |
|
|
| |
Also add a test that the ClusterFuzz run.py does not warn,
which was helpful when debugging this.
|
| |
|
|
|
|
|
|
|
|
| |
Before, we would simply not export a function that had an e.g. anyref
param. As a result, the modules were effectively "closed", which was
good for testing full closed-world mode, but not for testing degrees of
open world. To improve that, this PR allows the fuzzer to export such
functions, and an "enclose world" pass is added that "closes" the wasm
(makes it more compatible with closed-world) that is run 50% of the
time, giving us coverage of both styles.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The main addition here is a bundle_clusterfuzz.py script which will package up
the exact files that should be uploaded to ClusterFuzz. It also documents the
process and bundling and testing. You can do
bundle.py OUTPUT_FILE.tgz
That bundles wasm-opt from ./bin., which is enough for local testing. For
actually uploading to ClusterFuzz, we need a portable build, and @dschuff
had the idea to reuse the emsdk build, which works nicely. Doing
bundle.py OUTPUT_FILE.tgz --build-dir=/path/to/emsdk/upstream/
will bundle wasm-opt (+libs) from the emsdk. I verified that those builds
work on ClusterFuzz.
I added several forms of testing here. First, our main fuzzer fuzz_opt.py now
has a ClusterFuzz testcase handler, which simulates a ClusterFuzz environment.
Second, there are smoke tests that run in the unit test suite, and can also be
run separately:
python -m unittest test/unit/test_cluster_fuzz.py
Those unit tests can also run on a given bundle, e.g. one created from an
emsdk build, for testing right before upload:
BINARYEN_CLUSTER_FUZZ_BUNDLE=/path/to/bundle.tgz python -m unittest test/unit/test_cluster_fuzz.py
A third piece of testing is to add a --fuzz-passes test. That is a mode for
-ttf (translate random data into a valid wasm fuzz testcase) that uses random
data to pick and run a set of passes, to further shape the wasm. (--fuzz-passes
had no previous testing, and this PR fixes it and tidies it up a little, adding some
newer passes too).
Otherwise this PR includes the key run.py script that is bundled and then
executed by ClusterFuzz, basically a python script that runs wasm-opt -ttf [..]
to generate testcases, sets up their JS, and emits them.
fuzz_shell.js, which is the JS to execute testcases, will now check if it is
provided binary data of a wasm file. If so, it does not read a wasm file from
argv[1]. (This is needed because ClusterFuzz expects a single file for the
testcase, so we make a JS file with bundled wasm inside it.)
|
| |
|
|
|
|
|
|
|
|
| |
When IRBuilder builds an empty non-block scope such as a function body,
an if arm, a try block, etc, it needs to produce some expression to
represent the empty contents. Previously it produced a nop, but change
it to produce an empty block instead. The binary writer and printer have
special logic to elide empty blocks, so this produces smaller output.
Update J2CLOpts to recognize functions containing empty blocks as
trivial to avoid regressing one of its tests.
|
| |
|
|
|
|
|
|
|
|
|
| |
The binary reader has special handling for blocks immediately nested
inside other blocks to eliminate recursion while parsing very deep
stacks of blocks. This special handling did not record binary locations
for the nested blocks, though.
Add logic to record binary locations for nested blocks. This binary
reading code is about to be replaced with completely different code that
uses IRBuilder instead, but this change will eliminate some test
differences that we would otherwise see when we make that change.
|
| |
|
|
|
|
|
|
|
|
|
| |
There were previously two separate code paths for printing function
signatures, one for imported functions and one for declared functions.
The only intended difference was that parameter names were printed for
declared functions but not for imported functions.
Reduce duplication by consolidating the code paths, and add support for
printing names for imported function parameters that have them. Also fix
a bug where empty names were printed as `$` rather than the correct
`$""`.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This adds two new imports to fuzzer modules:
* call-export, which gets an export index and calls it.
* call-export-catch, which does the call in a try-catch, swallowing
any error, and returning 1 if it saw an error.
The former gives us calls back into the wasm, possibly making various
trips between wasm and JS in interesting ways. The latter adds a
try-catch which helps fuzz wasm EH.
We do these calls using a wasm export index, i.e., the index in
the list of exports. This is simple, but it does have the downside that
it makes executing the wasm sensitive to changes in exports (e.g.
wasm-merge adds more), which requires some handling in the fuzzer.
|
| |
|
|
| |
`ModuleUtils::copyTable` was not copying the `indexType` property.
|
| |
|
|
|
|
|
| |
* Remove the code that prevented fuzzing wasm64 test files.
* Ignore a run that hits the V8 implementation limit on memory size.
* Disable wasm64 fuzzing in wasm2js (like almost all post-MVP features).
* Add fuzzer logic to emit a 64-bit memory sometimes.
* Fix various places in the fuzzer that assumed 32-bit indexes
|
| |
|
|
|
| |
Continues the work from #7027 which added throwing from JS, this adds
table get/set operations from JS, to further increase our coverage of
Wasm/JS interactions (the table can be used from both sides).
|
| |
|
|
|
|
|
|
|
|
|
| |
We already generated (throw ..) instructions in wasm, but it makes sense to model
throws from outside as well, as they cross the module boundary. This adds a new fuzzer
import to the generated modules, "throw", that just does a throw from JS etc.
Also be more precise about handling fuzzing-support imports in fuzz-exec: we now
check that logging functions start with "log*" and error otherwise (this check is
now needed given we have "throw", which is not logging). Also fix a minor issue
with name conflicts for logging functions by using getValidFunctionName for them,
both for logging and for throw.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
(#6994)
In #6984 we optimized dropped blocks even if they had unreachable code. In #6988
that part was reverted, and blocks with unreachable code were ignored once more.
However, I realized that the check was not actually for unreachable code, but for
having an unreachable child, so it would miss things like this:
(block
(block
..
(br $somewhere) ;; unreachable type, but no unreachable code
)
)
But it is useful to merge such blocks: we don't need the inner block here.
To fix this, just run ReFinalize if we change anything, which will propagate
unreachability as needed. I think MergeBlocks was written before we had
that utility, so it didn't use it...
This is not only useful for itself but will unblock an EH optimization in a
later PR, that has code in this form. It also simplifies the code by removing
the hasUnreachableChild checks.
|
| |
|
|
| |
Also make Try/TryTables with type none, and not just concrete types as
before.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
When I refactored the optimizeDroppedBlock logic in #6982, I didn't move the
unreachability check with that code, which was wrong. When that function
was called from another place in #6984, the fuzzer found an issue.
Diff without whitespace is smaller. This reverts almost all the test updates
from #6984 - those changes were on blocks with unreachable children.
The change was safe on them, but in general removing a block value in the
presence of unreachable code is tricky, so it's best to avoid it.
The testcase is a little bizarre, but it's the one the fuzzer found and I can't
find a way to generate a better one (other than to reduce it, which I did).
|
| |
|
|
|
|
| |
Just call optimizeDroppedBlock from visitDrop to handle that.
Followup to #6982. This optimizes the new testcase added there. Some older
tests also improve.
|
| |
|
| |
Ensure the "fp16" feature is enabled for FP16 instructions.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Before the PR:
$ bin/wasm-opt test/hello_world.wat --metrics
total
[exports] : 1
[funcs] : 1
[globals] : 0
[imports] : 0
[memories] : 1
[memory-data] : 0
[tables] : 0
[tags] : 0
[total] : 3
[vars] : 0
Binary : 1
LocalGet : 2
After the PR:
$ bin/wasm-opt test/hello_world.wat --metrics
Metrics
total
[exports] : 1
[funcs] : 1
...
Note the "Metrics" addition at the top. And the title can be customized:
$ bin/wasm-opt test/hello_world.wat --metrics=text
Metrics: text
total
[exports] : 1
[funcs] : 1
The custom title can be helpful when multiple invocations of metrics are used
at once, e.g. --metrics=before -O3 --metrics=after.
|
| |
|
|
|
|
|
|
| |
Update the fuzzer to both handle shared types in initial contents and
create and use new shared types without crashing or producing invalid
modules. Since V8 does not have a complete implementation of
shared-everything-threads yet, disable fuzzing V8 when shared-everything
is enabled. To avoid losing too much coverage of V8, disable
shared-everything in the fuzzer more frequently than other features.
|
| |
|
|
|
|
| |
When we switched to the new type printing machinery, we inserted this
extra space to minimize the diff in the test output compared with the
previous type printer. Improve the quality of the printed output by
removing it.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
When creating a reference to `func`, fix the probability of choosing to
continue on to choose some function other than the last one rather than
making it depend on the number of functions. Then, do not eagerly pick
from the rest of the candidate functions. Instead, fall through to the
more general logic that will already pick a random candidate function.
Also move the logic for coming up with a concrete signature down to
where it is needed.
These simplifications will make it easier to update the code to handle
shared types.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
E.g. loading 4 bytes from 2^32 - 2 should error: 2 bytes are past the maximum
address. Before this PR we added 2^32 - 2 + 4 and overflowed to 2, which we
saw as a low and safe address. This PR adds an extra check for an overflow in
that add.
Also add unreachables after calls to segfault(), which reduces the overhead of
the extra check here (the unreachable apparently allows VMs to see that
control flow ends, after the segfault() which is truly no-return).
Fixes emscripten-core/emscripten#21557
|
| |
|
| |
Test was converted using port_passes_tests_to_lit.py.
|
| |
|
|
|
| |
Add the feature and flags to enable and disable it. Require the new feature to
be enabled for shared heap types to validate. To make the test work, update the
validator to actually check features for global types.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
With this PR we generate global.gets in globals, which we did not do before.
We do that by replacing makeConst (the only thing we did before, for the
contents of globals) with makeTrivial, and add code to makeTrivial to sometimes
make a global.get. When no suitable global exists, makeGlobalGet will emit a
constant, so there is no danger in trying.
Also raise the number of globals a little.
Also explicitly note the current limitation of requiring all tuple globals to contain
tuple.make and nothing else, including not global.get, and avoid adding such
invalid global.gets in tuple globals in the fuzzer.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Previously we had passes --generate-stack-ir, --optimize-stack-ir, --print-stack-ir
that could be run like any other passes. After generating StackIR it was stashed on
the function and invalidated if we modified BinaryenIR. If it wasn't invalidated then
it was used during binary writing. This PR switches things so that we optionally
generate, optimize, and print StackIR only during binary writing. It also removes
all traces of StackIR from wasm.h - after this, StackIR is a feature of binary writing
(and printing) logic only.
This is almost NFC, but there are some minor noticeable differences:
1. We no longer print has StackIR in the text format when we see it is there. It
will not be there during normal printing, as it is only present during binary writing.
(but --print-stack-ir still works as before; as mentioned above it runs during writing).
2. --generate/optimize/print-stack-ir change from being passes to being flags that
control that behavior instead. As passes, their order on the commandline mattered,
while now it does not, and they only "globally" affect things during writing.
3. The C API changes slightly, as there is no need to pass it an option "optimize" to
the StackIR APIs. Whether we optimize is handled by --optimize-stack-ir which is
set like other optimization flags on the PassOptions object, so we don't need the
old option to those C APIs.
The main benefit here is simplifying the code, so we don't need to think about
StackIR in more places than just binary writing. That may also allow future
improvements to our usage of StackIR.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Helping #6509, this fixes debugging support for StackIR, which makes it more
possible to use StackIR in more places.
The fix is basically just to pass around some more state, and then to call the
parent with "please write debug info" at the correct times, mirroring the
similar calls in BinaryenIRWriter.
The relevant Emscripten tests pass, and the source map test modified
here produces identical output in StackIR and non-StackIR modes (the
test is also simplified to remove --new-wat-parser which is no longer
needed, after which the test can clearly show that StackIR has the same
output as BinaryenIR).
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
(#6549)
As suggested in #6434 (comment) , lower ref.cast of string views
to ref.as_non_null in binary writing. It is a simple hack that avoids the
problem of V8 not allowing them to be cast.
Add fuzzing support for the last three core string operations, after which
that problem becomes very frequent.
Also add yet another makeTrappingRefUse that was missing in that
fuzzer code.
|
| |
|
|
|
|
|
|
| |
We added string.compare late in the spec process, and forgot to add effects for it.
Unlike string.eq, it can trap.
Also use makeTrappingRefUse in recent fuzzer string generation places that I
forgot, which should reduce the amount of traps in fuzzer output.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
The new text parser is faster and more standards compliant than the old text
parser. Enable it by default in wasm-opt and update the tests to reflect the
slightly different results it produces. Besides following the spec, the new
parser differs from the old parser in that it:
- Does not synthesize `loop` and `try` labels unnecessarily
- Synthesizes different block names in some cases
- Parses exports in a different order
- Parses `nop`s instead of empty blocks for empty control flow arms
- Does not support parsing Poppy IR
- Produces different error messages
- Cannot parse `pop` except as the first instruction inside a `catch`
|
| |
|
|
|
|
|
| |
A little trickier than the others due to the risk of trapping, which this
handles like the other array operations.
Also stop using immutable i16 arrays for string operations - only
mutable ones work atm.
|
| |
|
| |
Also refactor the code a little to make it easier to add this (mostly whitespace).
|
| |
|
|
|
|
|
| |
With this we emit strings spontaneously (as opposed to just getting them from
initial contents).
The relevant -ttf test has been tweaked slightly to show the impact of this
change: now there are some string.new/const in the output.
|
| |
|
|
|
|
|
|
|
|
| |
For types that do not have explicit names, we generate index-based names in the
printer. However, we did not previously ensure that the generated types were not
already used as explicit names, so it was possible to print the same name for
multiple types, which is not valid.
Fix the problem by skipping indices that are already used as type names.
Fixes #6492.
|
| |
|
|
|
|
|
| |
Previously we chose the first with a proper type, and now we start to scan from
a random index, giving later functions a chance too, so we should be emitting a
greater variety of ref.func targets.
Also remove some obsolete fuzzer TODOs.
|
| | |
|
| |
|
|
|
|
|
| |
- Only write explicit function names.
- When merging modules, the name of types, globals and tags in all
modules but the first were lost.
- Set name as explicit when copying a function with a new name.
|
| |
|
|
|
|
| |
Previously we did not optimize a struct or an array with a packed field. As a result a
single packed field in a struct prevented the entire struct from being localized,
which this fixes. This is also useful for arrays as packed arrays are common (e.g. for
string data).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Before this PR we only mutated by replacing an expression with another, which
replaced all the children. With this PR we also do these two patterns:
(A
(B)
(C)
)
=> ;; keep children, replace A
(block
(drop (B))
(drop (C))
(NEW)
)
,
(D
(A
(B)
(C)
)
)
=> ;; keep A, replace it in the parent
(D
(block
(drop
(A
(B)
(C)
)
)
(NEW)
)
)
We also try to replace onto the new D (either A itself, or A's children).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We had two JS files that could run a wasm file for fuzzing purposes:
* --emit-js-shell, which emitted a custom JS file that runs the wasm.
* scripts/fuzz_shell.js, which was a generic file that did the same.
Both of those load the wasm and then call the exports in order and print out
logging as it goes of their return values (if any), exceptions, etc. Then the
fuzzer compares that output to running the same wasm in another VM, etc. The
difference is that one was custom for the wasm file, and one was generic. Aside
from that they are similar and duplicated a bunch of code.
This PR improves things by removing 1 and using 2 in all places, that is, we
now use the generic file everywhere.
I believe we added 1 because we thought a generic file can't do all the
things we need, like know the order of exports and the types of return values,
but in practice there are ways to do those things: The exports are in fact
in the proper order (JS order of iteration is deterministic, thankfully), and
for the type we don't want to print type internals anyhow since that would
limit fuzzing --closed-world. We do need to be careful with types in JS (see
notes in the PR about the type of null) but it's not too bad. As for the types
of params, it's fine to pass in null for them all anyhow (null converts to a
number or a reference without error).
|
| |
|
|
| |
The LLVM wasm backend grows the stack downwards, and this pass did not
fully account for that before.
|
| |
|
| |
Arrays have immutable length, so we can optimize them like immutable fields.
|
