| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
| |
1. Error on retrying due to a wasm-opt issue, locally (in production, we
don't want to error on ClusterFuzz).
2. Move some asserts from test_run_py to the helper generate_testcases
(so that the asserts happen in all callers).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Implement support for both sequentially consistent and acquire-release
variants of `struct.atomic.get` and `struct.atomic.set`, as proposed by
shared-everything-threads. Introduce a new `MemoryOrdering` enum for
describing different levels of atomicity (or the lack thereof). This new
enum should eventually be adopted by linear memory atomic accessors as
well to support acquire-release semantics, but for now just use it in
`StructGet` and `StructSet`.
In addition to implementing parsing and emitting for the instructions,
validate that shared-everything is enabled to use them, mark them as
having synchronization side effects, and lightly optimize them by
relaxing acquire-release accesses to non-shared structs to normal,
unordered accesses. This is valid because such accesses cannot possibly
synchronize with other threads. Also update Precompute to avoid
optimizing out synchronization points.
There are probably other passes that need to be updated to avoid
incorrectly optimizing synchronizing accesses, but identifying and
fixing them is left as future work.
|
| |
|
|
|
|
|
|
|
|
| |
* Add a new "sleep" fuzzer import, that does a sleep for some ms.
* Add JSPI support in fuzz_shell.js. This is in the form of commented-out async/await
keywords - commented out so that normal fuzzing is not impacted. When we want
to fuzz JSPI, we uncomment them. We also apply the JSPI operations of marking
imports and exports as suspending/promising.
JSPI fuzzing is added to both fuzz_opt.py and ClusterFuzz's run.py.
|
| |
|
|
| |
Now that #7149 added support for parsing block parameters, we can run
additional spec tests that previously failed.
|
| |
|
|
|
|
|
|
|
|
|
| |
The new Two fuzz testcase handler generates two wasm files and
then runs them in V8, linking them using JS. It then optimizes at least one of
the two and runs it again, and checks for differences.
This is similar to the Split fuzzer for wasm-split, but the two wasm files
are arbitrary and not the result of splitting.
Also lower CtorEval priority a little. It is not very important.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Similar to call-export*, these imports call a wasm function from outside the
module. The difference is that we send a function reference for them to call
(rather than an export index).
This gives more coverage, first by sending a ref from wasm to JS, and also
since we will now try to call anything that is sent. Exports, in comparison,
are filtered by the fuzzer to things that JS can handle, so this may lead to
more traps, but maybe also some new situations. This also leads to adding
more logic to execution-results.h to model JS trapping properly.
fuzz_shell.js is refactored to allow sharing code between call-export* and
call-ref*.
|
| |
|
|
|
|
|
|
|
|
|
| |
This sends --closed-world to wasm-opt from the fuzzer, when we use that
flag (before we just used it on optimizations, but not fuzz generation). And
TranslateToFuzzReader now stores a boolean about whether we are in closed-
world mode or not.
This has no effect so far, and is a refactoring for a later PR, where we must
generate code differently based on whether we are in closed-world mode
or not.
|
| |
|
|
|
|
|
|
|
| |
The two files are then linked and run by fuzz_shell.js (we had this functionality
already in order to fuzz wasm-split). By adding multiple build and run commands
of both the primary and secondary wasm files, we can end up with multiple
instances of two different wasm files that call between themselves.
To help testing, add a script that extracts the wasm files from the testcase. This
may also be useful in the future for testcase reduction.
|
| | |
|
| |
|
|
|
|
| |
(#7116)
Lower away saturating fptoint operations when we know we are using
emscripten.
|
| |
|
|
|
|
|
|
|
| |
--emscripten is added to the command line if the filename contains the
string
'emscripten', but the current logic includes the cwd, so it fails if the
tests
are run from a directory with 'emscripten' in the name. Fix it so the
test only
includes the basename.
|
| |
|
|
| |
This increases our testing of V8, and helps find bugs where we are
different than V8.
|
| |
|
|
|
| |
This just moves code around. It will allow more code reuse in a later PR.
Also add a bit of test logging.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The main fuzz_shell.js code builds and runs the given wasm. After the refactoring
in #7096, it is simple to append to that file and add more build and run operations,
adding more variety to the code, including cross-module interactions. Add logic
to run.py to do that for ClusterFuzz.
To test this, add a node test that builds a module with internal state that can
actually show which module is being executed. The test appends a build+run
operation, whose output prove that we are calling from the first module to the
second and vice versa.
Also add a ClusterFuzz test for run.py that verifies that we add a variety of
build/run operations.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
This mostly moves the code around and avoids some duplication. It
also tracks the list of exports with both names and values, so that
if we compile more than one module, we can still access exports
from the previous.
Also add a first test of running fuzz_shell.js in node.
This does make build() append the exports, which was done before
on the main module but not the second one. That only affects the
wasm-split fuzzer, which is not active yet, so this is still NFC.
|
| |
|
|
|
|
|
| |
It is ok to use this pass to shape the wasm, but not to test for changes
using fuzz-exec, as the pass is destructive: it can alter observable behavior.
Specifically, it adds casts which can trap in the wasm, which can replace a JS
exception which happens outside, and that difference is noticeable.
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
Before, we would simply not export a function that had an e.g. anyref
param. As a result, the modules were effectively "closed", which was
good for testing full closed-world mode, but not for testing degrees of
open world. To improve that, this PR allows the fuzzer to export such
functions, and an "enclose world" pass is added that "closes" the wasm
(makes it more compatible with closed-world) that is run 50% of the
time, giving us coverage of both styles.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The main addition here is a bundle_clusterfuzz.py script which will package up
the exact files that should be uploaded to ClusterFuzz. It also documents the
process and bundling and testing. You can do
bundle.py OUTPUT_FILE.tgz
That bundles wasm-opt from ./bin., which is enough for local testing. For
actually uploading to ClusterFuzz, we need a portable build, and @dschuff
had the idea to reuse the emsdk build, which works nicely. Doing
bundle.py OUTPUT_FILE.tgz --build-dir=/path/to/emsdk/upstream/
will bundle wasm-opt (+libs) from the emsdk. I verified that those builds
work on ClusterFuzz.
I added several forms of testing here. First, our main fuzzer fuzz_opt.py now
has a ClusterFuzz testcase handler, which simulates a ClusterFuzz environment.
Second, there are smoke tests that run in the unit test suite, and can also be
run separately:
python -m unittest test/unit/test_cluster_fuzz.py
Those unit tests can also run on a given bundle, e.g. one created from an
emsdk build, for testing right before upload:
BINARYEN_CLUSTER_FUZZ_BUNDLE=/path/to/bundle.tgz python -m unittest test/unit/test_cluster_fuzz.py
A third piece of testing is to add a --fuzz-passes test. That is a mode for
-ttf (translate random data into a valid wasm fuzz testcase) that uses random
data to pick and run a set of passes, to further shape the wasm. (--fuzz-passes
had no previous testing, and this PR fixes it and tidies it up a little, adding some
newer passes too).
Otherwise this PR includes the key run.py script that is bundled and then
executed by ClusterFuzz, basically a python script that runs wasm-opt -ttf [..]
to generate testcases, sets up their JS, and emits them.
fuzz_shell.js, which is the JS to execute testcases, will now check if it is
provided binary data of a wasm file. If so, it does not read a wasm file from
argv[1]. (This is needed because ClusterFuzz expects a single file for the
testcase, so we make a JS file with bundled wasm inside it.)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This adds two new imports to fuzzer modules:
* call-export, which gets an export index and calls it.
* call-export-catch, which does the call in a try-catch, swallowing
any error, and returning 1 if it saw an error.
The former gives us calls back into the wasm, possibly making various
trips between wasm and JS in interesting ways. The latter adds a
try-catch which helps fuzz wasm EH.
We do these calls using a wasm export index, i.e., the index in
the list of exports. This is simple, but it does have the downside that
it makes executing the wasm sensitive to changes in exports (e.g.
wasm-merge adds more), which requires some handling in the fuzzer.
|
| |
|
|
|
|
|
| |
It never used the parameter, so remove that (we always access the
features using a global anyhow).
But add a new parameter, the wasm file, which does need to be
passed in for a later PR (so in this PR it is just for future use).
|
| |
|
|
|
|
|
| |
* Remove the code that prevented fuzzing wasm64 test files.
* Ignore a run that hits the V8 implementation limit on memory size.
* Disable wasm64 fuzzing in wasm2js (like almost all post-MVP features).
* Add fuzzer logic to emit a 64-bit memory sometimes.
* Fix various places in the fuzzer that assumed 32-bit indexes
|
| |
|
|
| |
The old code assumed the index was a JS number, but if the table has
64-bit indexes it must be a BigInt. Detect that and cast as needed.
|
| |
|
| |
Followup to #7055
|
| |
|
|
|
|
| |
When fuzzing wasm-merge, we need to avoid the first module not having
an exported table but the second having one, as the way the table operation
imports work, they are sensitive to the existence of such an export, so just
merging in such an export can alter behavior.
|
| |
|
|
|
|
| |
J2CL runs a long pipeline of binaryen opts, including multiple invocations. This adds a
tool that simulates the same process manually, which can be easier to debug with
than running a full J2CL toolchain, which requires Bazel, etc. - this is just a few lines of
bash that does the same thing (at least in simple cases).
|
| |
|
|
|
| |
Continues the work from #7027 which added throwing from JS, this adds
table get/set operations from JS, to further increase our coverage of
Wasm/JS interactions (the table can be used from both sides).
|
| |
|
|
|
|
|
|
|
|
|
| |
We already generated (throw ..) instructions in wasm, but it makes sense to model
throws from outside as well, as they cross the module boundary. This adds a new fuzzer
import to the generated modules, "throw", that just does a throw from JS etc.
Also be more precise about handling fuzzing-support imports in fuzz-exec: we now
check that logging functions start with "log*" and error otherwise (this check is
now needed given we have "throw", which is not logging). Also fix a minor issue
with name conflicts for logging functions by using getValidFunctionName for them,
both for logging and for throw.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
The support is added but not enabled as this is still finding bugs.
The first part here is to add Split testcase handler to the fuzzer,
which runs a wasm, then runs it again after splitting it and then
linking it at runtime, and checking for different results.
The second part is support for linking two modules at runtime
in the fuzzer's JS code, that works in tandem with the first part.
New options are added to load and link a second wasm, and to
pick which exports to run.
|
| |
|
|
|
| |
The blocking bug https://issues.chromium.org/issues/332931390 has been
fixed.
|
| |
|
|
|
|
|
|
|
|
| |
Note: FP16 is a little different from F32/F64 since it can't represent
the full 2^16 integer range. 65504 is the max whole integer. This leads
to some slightly strange behavior when converting integers greater than
65504 since they become infinity.
Specified at
https://github.com/WebAssembly/half-precision/blob/main/proposals/half-precision/Overview.md
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
HeapStoreOptimization (#6882)
This just moves code out of OptimizeInstructions to the new pass. The existing
test is renamed and now runs the new pass instead. The new pass is run right
after each --optimize-instructions invocation, so it should not cause any
noticeable effects whatsoever, making this NFC.
The motivation here is that there is a bug in the pass, see the new testcase
added at the end, which shows the bug. It is not practical to fix that bug in
OptimizeInstructions since we need more than peephole optimizations to do
so. This PR moves the code to a new pass so we can fix it there properly,
later.
The new pass is named HeapStoreOptimization since the same infrastructure
we will need to fix the bug will also help dead store elimination and related
things.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Specified at
https://github.com/WebAssembly/half-precision/blob/main/proposals/half-precision/Overview.md
A few notes:
- The F32x4 and F64x2 versions of madd and nmadd are missing spect
tests.
- For madd, the implementation was incorrectly doing `(b*c)+a` where it
should be `(a*b)+c`.
- For nmadd, the implementation was incorrectly doing `(-b*c)+a` where
it should be `-(a*b)+c`.
- There doesn't appear to be a great way to actually implement a fused
nmadd, but the spec allows the double rounded version I added.
|
| |
|
|
| |
Add the feature flag in V8 invocations, but also disable the feature as it
isn't quite ready yet.
|
| |
|
|
|
|
|
| |
The instructions relaxed_fma and relaxed_fnma have been renamed to
relaxed_madd and relaxed_nmadd.
https://github.com/WebAssembly/relaxed-simd/blob/main/proposals/relaxed-simd/Overview.md#binary-format
|
| |
|
|
| |
Specified at
https://github.com/WebAssembly/half-precision/blob/main/proposals/half-precision/Overview.md
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Spec tests use constants like `ref.array` and `ref.eq` to assert that
exported function return references of the correct types. Support more
such constants in the wast parser.
Also fix a bug where the interpretation of `array.new_data` for arrays
of packed fields was not properly truncating the packed data. Move the
function for reading fields from memory from literal.cpp to
wasm-interpreter.h, where the function for truncating packed data lives.
Other bugs prevent us from enabling any more spec tests as a result of
this change, but we can get farther through several of them before
failing. Update the comments about the failures accordingly.
|
| |
|
|
| |
Specified at
https://github.com/WebAssembly/half-precision/blob/main/proposals/half-precision/Overview.md
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Spec tests pass the value `ref.extern n`, where `n` is some integer,
into exported functions that expect to receive externrefs and receive
such values back out as return values. The payload serves to distinguish
externrefs so the test can assert that the correct one was returned.
Parse these values in wast scripts and represent them as externalized
i31refs carrying the payload. We will need a different representation
eventually, since some tests explicitly expect these externrefs to not
be i31refs, but this suffices to get several new tests passing.
To get the memory64 version of table_grow.wast passing, additionally fix
the interpreter to handle growing 64-bit tables correctly.
Delete the local versions of the upstream tests that can now be run
successfully.
|
| |
|
|
| |
Add comments to the spec test skip list briefly explaining why each
skipped spec test must be skipped.
|
| |
|
|
|
|
|
|
| |
The leading bytes that indicate what kind of heap type is being defined
are bytes, but we were previously treating them as SLEB128-encoded
values. Since we emit the smallest LEB encodings possible, we were
writing the correct bytes in output files, but we were also improperly
accepting binaries that used more than one byte to encode these values.
This was caught by an upstream spec test.
|
| |
|
|
|
|
| |
Run the upstream tests by default, except for a large list of them that
do not successfully run. Remove the local version of those that do
successfully run where the local version is entirely subsumed by the
upstream version.
|
| |
|
|
|
|
| |
* Add interpreter support for exnref values.
* Fix optimization passes to support try_table.
* Enable the interpreter (but not in V8, see code) on exceptions.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Most of our type optimization passes emit all non-public types as a
single large rec group, which trivially ensures that different types
remain different, even if they are optimized to have the same structure.
Usually emitting a single large rec group is fine, but it also means
that if the module is split, all of the types will need to be repeated
in all of the split modules. To better support this use case, add a pass
that can split the large rec group back into minimal rec groups, taking
care to preserve separate type identities by emitting different
permutations of the same group where possible or by inserting unused
brand types to differentiate them.
|
| |
|
|
|
| |
Also use TableInit in the interpreter to initialize module's table
state, which will now handle traps properly, fixing #6431
|
| |
|
|
|
| |
Also we had a mix of os.environ.get and os.getenv. Prefer the former, as the default
value does actual work, so it's a little more efficient to not run it unnecessarily. That is,
os.getenv('X', work()) is less efficient than os.environ.get('X') or work().
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The argument is the minimum benefit we must see for us to decide to optimize, e.g.
--monomorphize --pass-arg=monomorphize-min-benefit@50
When the minimum benefit is 50% then if we reduce the cost by 50% through
monomorphization then we optimize there. 95% would only optimize when we
remove almost all the cost, etc.
In practice I see 95% will actually tend to reduce code size overall, as while we add
monomorphized versions of functions, we only do so when we remove a lot of
work and size, and after inlining we gain benefits. However, 50% or even lower can
lead to better benchmark results, in return for larger code size, just like with
inlining. To be careful, the default is set to 95%.
Previously we optimized whenever we saw any benefit at all, which is the same
as requiring a minimum benefit of 0%. Old tests have the flag applied in this PR
to set that value, so they do not change.
|
| |
|
|
| |
Specified at
https://github.com/WebAssembly/half-precision/blob/main/proposals/half-precision/Overview.md
|
| |
|
|
| |
Specified at
https://github.com/WebAssembly/half-precision/blob/main/proposals/half-precision/Overview.md
|
| | |
|
