| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
| |
Before, we would simply not export a function that had an e.g. anyref
param. As a result, the modules were effectively "closed", which was
good for testing full closed-world mode, but not for testing degrees of
open world. To improve that, this PR allows the fuzzer to export such
functions, and an "enclose world" pass is added that "closes" the wasm
(makes it more compatible with closed-world) that is run 50% of the
time, giving us coverage of both styles.
|
| |
|
|
|
|
|
|
|
|
|
| |
This pass lowers nontrapping FP to int instructions to implement LLVM's
conversion behavior.
This means that they are not fully complete lowerings according to the
wasm spec, but have the same
undefined behavior that LLM does. This keeps the pass simpler and
preserves existing behavior when
compiling without nontrapping-ft.
This will be used in emscripten, so that we can build libraries with
nontrapping-fp and lower them away after link if desired.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The main addition here is a bundle_clusterfuzz.py script which will package up
the exact files that should be uploaded to ClusterFuzz. It also documents the
process and bundling and testing. You can do
bundle.py OUTPUT_FILE.tgz
That bundles wasm-opt from ./bin., which is enough for local testing. For
actually uploading to ClusterFuzz, we need a portable build, and @dschuff
had the idea to reuse the emsdk build, which works nicely. Doing
bundle.py OUTPUT_FILE.tgz --build-dir=/path/to/emsdk/upstream/
will bundle wasm-opt (+libs) from the emsdk. I verified that those builds
work on ClusterFuzz.
I added several forms of testing here. First, our main fuzzer fuzz_opt.py now
has a ClusterFuzz testcase handler, which simulates a ClusterFuzz environment.
Second, there are smoke tests that run in the unit test suite, and can also be
run separately:
python -m unittest test/unit/test_cluster_fuzz.py
Those unit tests can also run on a given bundle, e.g. one created from an
emsdk build, for testing right before upload:
BINARYEN_CLUSTER_FUZZ_BUNDLE=/path/to/bundle.tgz python -m unittest test/unit/test_cluster_fuzz.py
A third piece of testing is to add a --fuzz-passes test. That is a mode for
-ttf (translate random data into a valid wasm fuzz testcase) that uses random
data to pick and run a set of passes, to further shape the wasm. (--fuzz-passes
had no previous testing, and this PR fixes it and tidies it up a little, adding some
newer passes too).
Otherwise this PR includes the key run.py script that is bundled and then
executed by ClusterFuzz, basically a python script that runs wasm-opt -ttf [..]
to generate testcases, sets up their JS, and emits them.
fuzz_shell.js, which is the JS to execute testcases, will now check if it is
provided binary data of a wasm file. If so, it does not read a wasm file from
argv[1]. (This is needed because ClusterFuzz expects a single file for the
testcase, so we make a JS file with bundled wasm inside it.)
|
| |
|
|
|
|
|
|
|
|
|
|
| |
IRBuilder often has to generate new label names for blocks and other
scopes. Previously it would generate each new name by starting with
"block" or "label" and incrementing a suffix until finding a fresh name,
but this made name generation quadratic in the number of names to
generate.
To spend less time generating names, track a hint index at which to
start looking for a fresh name and increment it every time a name is
generated. This speeds up a version of the binary parser that uses
IRBuilder by about 15%.
|
| |
|
|
|
|
| |
Since IRBuilder already knows what labels are used by branches, it is
easy for it to pass that information when finalizing blocks. This avoids
finalization having to walk the blocks looking for branches, speeding up
a future version of the binary parser that uses IRBuilder by 10%.
|
| |
|
|
| |
Since the resulting code has the same undefined behavior as LLVM, make
the pass name reflect that.
|
| |
|
|
|
|
| |
Now that Result and MaybeResult are annotated [[nodiscard]] at the type
level, individual functions and methods that return these types do not
need to be annotated [[nodiscard]] themselves. Remove the newly
redundant annotations.
|
| |
|
|
|
|
| |
Since these types may be carrying errors that need to be handled or
propagated, it is always an error not to use them in some way. Adding
the [[nodiscard]] attribute caused the compiler to find a few instances
where we were incorrectly ignoring results. Fix these places.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
IRBuilder contains a pointer to the current function that is used to
create scratch locals, look up the operand types for returns, etc. This
pointer is nullable because IRBuilder can also be used in non-function
contexts such as global initializers. Visiting the start of a function
sets the function pointer, and after this change visiting the end of a
function resets the pointer to null. This avoids potential problems
where code outside a function would be able to incorrectly use scratch
locals and returns if the IRBuilder had previously been used to build a
function.
This change requires some adjustments to Outlining, which visits code
out of order, so ends up visiting code from inside a function after
visiting the end of the function. To support this use case, add a
`setFunction` method to IRBuilder that lets the user explicitly control
its function context. Also remove the optional function pointer
parameter to the IRBuilder constructor since it is less flexible and not
used.
|
| |
|
|
|
|
|
|
|
|
| |
When IRBuilder builds an empty non-block scope such as a function body,
an if arm, a try block, etc, it needs to produce some expression to
represent the empty contents. Previously it produced a nop, but change
it to produce an empty block instead. The binary writer and printer have
special logic to elide empty blocks, so this produces smaller output.
Update J2CLOpts to recognize functions containing empty blocks as
trivial to avoid regressing one of its tests.
|
| |
|
|
|
|
|
|
|
|
|
| |
The binary reader has special handling for blocks immediately nested
inside other blocks to eliminate recursion while parsing very deep
stacks of blocks. This special handling did not record binary locations
for the nested blocks, though.
Add logic to record binary locations for nested blocks. This binary
reading code is about to be replaced with completely different code that
uses IRBuilder instead, but this change will eliminate some test
differences that we would otherwise see when we make that change.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Instead of setting the local names at the end of binary reading, eagerly
set them before parsing function bodies. This is NFC now, but will fix a
future bug once the binary reader uses IRBuilder. IRBuilder can
introduce new scratch locals, and it gives them the names `$scratch`,
`$scratch_1`, etc. If the name section includes locals with the same
names and we set those local names after parsing function bodies, then
we can end up with multiple locals with the same names. Setting the
names before parsing the function bodies ensures that IRBuilder will
generate different names for the scratch locals.
The alternative fix would be to generate fresh names when setting names
from the name section, but it is better to respect the names in the name
section and use fresh names for the newly introduced scratch locals
instead.
|
| |
|
|
| |
(#7072)
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
IRBuilder introduces scratch locals to hoist values from underneath
stacky code to the top of the stack for consumption by the next
instruction. When it does so, the sequence of instructions from the set
to the get of the scratch local is packaged in a block so the entire
sequence can be made a child of the next instruction. In cases where the
hoisted value comes from a `pop`, this packaging can make the IR
invalid, since `pop`s are not allowed to appear inside blocks.
Detect when this problem might occur and fix it by running
`EHUtils::handleBlockNestedPops` after the function containing the
problem has been constructed.
|
| |
|
|
|
|
|
|
|
| |
Rather than back-patching names when we get to the names section in the
binary reader, skip ahead to read the names section before anything else
so we can use the final names right away. This is a prerequisite for
using IRBuilder in the binary reader.
The only functional change is that we now allow empty local names. Empty
names are perfectly valid.
|
| |
|
|
|
|
|
|
|
|
|
| |
There were previously two separate code paths for printing function
signatures, one for imported functions and one for declared functions.
The only intended difference was that parameter names were printed for
declared functions but not for imported functions.
Reduce duplication by consolidating the code paths, and add support for
printing names for imported function parameters that have them. Also fix
a bug where empty names were printed as `$` rather than the correct
`$""`.
|
| |
|
|
|
|
|
|
| |
This pass lowers away memory.copy and memory.fill operations. It
generates a function that implements the each of the instructions and
replaces the instructions with calls to those functions.
It does not handle other bulk memory operations (e.g. passive segments
and table operations) because they are not used by emscripten to enable
targeting old browsers that don't support bulk memory.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This PR fixes this situation:
(block $out
(local.set $x (struct.new X Y Z))
(struct.set $X 0 (local.get $x) (..br $out..)) ;; X' here has a br
)
(local.get $x)
=>
(block $out
(local.set $x (struct.new (..br $out..) Y Z))
)
(local.get $x)
We want to fold the struct.set into the struct.new, but the br is
a problem: if it executes then we skip the struct.set, and the last
local.get in fact reads the struct before the write. And, if we did this
optimization, we'd end up with the br on the struct.new, so it
would skip that instruction and even the local.set.
To fix this, we use the new API from #7039, which lets us query,
"is it ok to move the local.set to where the struct.set is?"
|
| |
|
|
|
|
| |
When the fuzzer sees an imported segment, it makes it non-imported
(because imported ones would trap when we tried to run them: we don't
have the normal runtime they expect). We had hardcoded i32 offets there,
which need to be generalized.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
I believe the history here is that
1. We added a PickLoadSigns pass. It checks if a load from memory is stored in
a local that is only every used in a signed or an unsigned manner. If it is, we can
adjust the sign of the load (load8_u/s) to do the sign/unsign during the load.
2. The pass finds each LocalGet and looks either 2 or 3 parents above it. For
a sign operation, we need to look up 3, since the operation is x << K >> K. For
an unsigned, we need only 2, since we have x & M. We hardcoded those
numbers 2 and 3.
3. We added the SignExt feature, which adds i32.extend8_s. This does a sign
extend with a single instruction, not two nested ones, so now we can sign-
extend at depth 2, unlike before. Properties::getSignExtValue was updated
for this, but not the pass PickLoadSigns.
The bug that is fixed here is that we looked at depth 3 for a sign-extend, and
we blindly accepted it if we found one. So we ended up accepting
(i32.extend8_s (ANYTHING (x))), which is a sign-extend of something, but
not of x, which is bad.
We were also missing an optimization opportunity, as we didn't look for
depth 2 sign extends.
This bug is quite old, from when Properties got SignExt support, in #3910.
But the blame isn't there - to notice this then, we'd have had to check each
caller of getSignExtValue throughout the codebase, which isn't reasonable.
The fault is mine, from the first write-up of PickLoadSigns in 2017: the code
should have been fully general, handling 2/3 and checking the output when
it does so (adding == curr, that the sign/zero-extended value is the one we
expect). That is what this PR does.
|
| |
|
|
|
| |
This new API lets us ask if a set can be safely moved to a new position. The new
position must be the location of an expression from a particular class (this
allows us to populate the IR once and then query any of those locations).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This adds two new imports to fuzzer modules:
* call-export, which gets an export index and calls it.
* call-export-catch, which does the call in a try-catch, swallowing
any error, and returning 1 if it saw an error.
The former gives us calls back into the wasm, possibly making various
trips between wasm and JS in interesting ways. The latter adds a
try-catch which helps fuzz wasm EH.
We do these calls using a wasm export index, i.e., the index in
the list of exports. This is simple, but it does have the downside that
it makes executing the wasm sensitive to changes in exports (e.g.
wasm-merge adds more), which requires some handling in the fuzzer.
|
| | |
|
| |
|
| |
See https://github.com/WebAssembly/memory64/pull/92
|
| |
|
|
| |
`ModuleUtils::copyTable` was not copying the `indexType` property.
|
| |
|
|
|
|
|
| |
* Remove the code that prevented fuzzing wasm64 test files.
* Ignore a run that hits the V8 implementation limit on memory size.
* Disable wasm64 fuzzing in wasm2js (like almost all post-MVP features).
* Add fuzzer logic to emit a 64-bit memory sometimes.
* Fix various places in the fuzzer that assumed 32-bit indexes
|
| | |
|
| |
|
| |
This allows 64-bit bounds checking to work properly.
|
| |
|
|
| |
Some places assumed a 32-bit index.
|
| |
|
| |
A bunch of places assumed a 32-bit index.
|
| |
|
|
|
| |
When we combine a load/store offset with a const, we must not
overflow, as the semantics of offsets do not wrap.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
CFP is less precise than GUFA, in particular, when it flows around types then
it does not consider what field it is flowing them to, and its core data
structure is "if a struct.get is done on this type's field, what can be read?".
To see the issue this PR fixes, assume we have
A
/ \
B C
Then if we see struct.set $C, we know that can be read by a struct.get $A
(we can store a reference to a C in such a local/param/etc.), so we propagate
the value of that set to A. And, in general, anything in A can appear in B
(say, if we see a copy, a struct.set of struct.get that operates on types A,
then one of the sides might be a B), so we propagate from A to B. But
now we have propagated something from C to B, which might be of an
incompatible type.
This cannot cause runtime issues, as it just means we are propagating more
than we should, and will end up with less-useful results. But it can break
validation if no other value is possible but one with an incompatible type,
as we'd replace a struct.get $B with a value that only makes sense for C.
(The qualifier "no other value is possible" was added in the previous
sentence because if another one is possible then we'd end up with too
many values to infer anything, and not optimize at all, avoiding any error.)
|
| |
|
|
|
|
|
|
| |
This has not been emitted in LLVM since
https://github.com/llvm/llvm-project/commit/3f34e1b883351c7d98426b084386a7aa762aa366.
The corresponding proposed tool-conventions change:
https://github.com/WebAssembly/tool-conventions/pull/236
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
This fixes a regression from #7019. That PR fixed an error on situations with
mixed public and private types, but it made us stop optimizing in valid cases,
including cases with entirely private types.
The specific regression was that we checked if we had an entry in the
map of "can become immutable", and we thought that was enough. But
we may have a private child type with a public parent, and still be able to
optimize in the child if the field is not present in the parent. We also did
not have exhaustive checking of all the states canBecomeImmutable can be,
so add those + testing.
|
| |
|
|
|
|
|
|
|
|
|
| |
This is NFC on 64-bit systems but noticeable on 32.
Also remove the 32-bit path in hash_combine. That isn't necessary for this fix,
but it makes the code simpler and also makes debugging between systems
simpler. It might also avoid problems in future cases, if we are lucky. The only
cost is perhaps a slight slowdown on 32-bit systems, which seems worth it.
Fixes #7046
|
| |
|
|
|
|
|
|
| |
Emscripten's JS loader code for wasm-split isn't prepared for handling
multiple tables that binaryen automatically creates when reference types
are enabled (especially in conjunction with dynamic loading). For now,
disable creation of multiple tables when using Emscripten's table ABI
(distinguished by importing or exporting a table named
"__indirect_function_table".
|
| |
|
|
|
|
|
| |
The old code manually managed it for no good reason that I can see.
After this, there is no difference between callFunction and callFunctionInternal,
so fold them together.
|
| |
|
|
|
| |
Continues the work from #7027 which added throwing from JS, this adds
table get/set operations from JS, to further increase our coverage of
Wasm/JS interactions (the table can be used from both sides).
|
| |
|
|
| |
table.fill was introduced by the reference-types proposal (but also, only
makes sense among the other bulk memory operations, so require both).
|
| | |
|
| |
|
|
|
| |
This makes the behavior consistent with emcc builds where we don't run
finalization, and potentially makes testing and debugging easier.
Emscripten still strips the target features section when optimizing.
|
| |
|
|
|
|
|
|
|
|
| |
Make the ID enum an `int8_t`, and make the Specific ID a `constexpr`
of that type.
This seems more idiomatic and makes some code simpler, see the
change to `find_all.h` which no longer needs a cast to compile.
This has no performance impact.
|
| | |
|
| |
|
|
|
|
|
|
| |
unrefine the output (#7036)
Paradoxically, when a BrOn's castType is refined, its own type (the type it flows out)
can get un-refined: making the castType non-nullable means nulls no longer
flow on the branch, so they may flow out directly, making the BrOn nullable.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Similar to
* https://github.com/WebAssembly/binaryen/pull/6330
* https://github.com/WebAssembly/binaryen/issues/6311
* https://github.com/WebAssembly/binaryen/pull/6912
* https://github.com/WebAssembly/binaryen/issues/5946
This extends the region we ignore the gcc warning in.
The warning:
ninja: job failed: /usr/bin/c++ -I/src/src -I/src/third_party/FP16/include -I/src/third_party/llvm-project/include -I/src -static -DBUILD_LLVM_DWARF -Wall -Werror -Wextra -Wno-unused-parameter -Wno-dangling-pointer -fno-omit-frame-pointer -fno-rtti -Wno-implicit-int-float-conversion -Wno-unknown-warning-option -Wswitch -Wimplicit-fallthrough -Wnon-virtual-dtor -fPIC -fdiagnostics-color=always -O3 -DNDEBUG -UNDEBUG -std=c++17 -MD -MT src/passes/CMakeFiles/passes.dir/Precompute.cpp.o -MF src/passes/CMakeFiles/passes.dir/Precompute.cpp.o.d -o src/passes/CMakeFiles/passes.dir/Precompute.cpp.o -c /src/src/passes/Precompute.cpp
In file included from /src/src/literal.h:27,
from /src/src/wasm.h:36,
from /src/src/ir/boolean.h:20,
from /src/src/ir/bits.h:20,
from /src/src/ir/properties.h:20,
from /src/src/ir/iteration.h:20,
from /src/src/passes/Precompute.cpp:30:
In copy constructor 'wasm::SmallVector<T, N>::SmallVector(const wasm::SmallVector<T, N>&) [with T = wasm::Expression*; long unsigned int N = 10]',
inlined from 'constexpr std::pair<_T1, _T2>::pair(const _T1&, const _T2&) [with _U1 = wasm::Select* const; _U2 = wasm::SmallVector<wasm::Expression*, 10>; typename std::enable_if<(std::_PCC<true, _T1, _T2>::_ConstructiblePair<_U1, _U2>() && std::_PCC<true, _T1, _T2>::_ImplicitlyConvertiblePair<_U1, _U2>()), bool>::type <anonymous> = true; _T1 = wasm::Select* const; _T2 = wasm::SmallVector<wasm::Expression*, 10>]' at /usr/include/c++/13.2.1/bits/stl_pair.h:559:21,
inlined from 'T& wasm::InsertOrderedMap<Key, T>::operator[](const Key&) [with Key = wasm::Select*; T = wasm::SmallVector<wasm::Expression*, 10>]' at /src/src/support/insert_ordered.h:112:29,
inlined from 'void wasm::Precompute::partiallyPrecompute(wasm::Function*)::StackFinder::visitSelect(wasm::Select*)' at /src/src/passes/Precompute.cpp:571:24,
inlined from 'static void wasm::Walker<SubType, VisitorType>::doVisitSelect(SubType*, wasm::Expression**) [with SubType = wasm::Precompute::partiallyPrecompute(wasm::Function*)::StackFinder; VisitorType = wasm::Visitor<wasm::Precompute::partiallyPrecompute(wasm::Function*)::StackFinder, void>]' at /src/src/wasm-delegations.def:50:1:
/src/src/support/small_vector.h:69:35: error: '<unnamed>.wasm::SmallVector<wasm::Expression*, 10>::fixed' may be used uninitialized [-Werror=maybe-uninitialized]
69 | : usedFixed(other.usedFixed), fixed(other.fixed), flexible(other.flexible) {
| ^~~~~~~~~~~~~~~~~~
In file included from /src/src/passes/Precompute.cpp:37:
/src/src/support/insert_ordered.h: In static member function 'static void wasm::Walker<SubType, VisitorType>::doVisitSelect(SubType*, wasm::Expression**) [with SubType = wasm::Precompute::partiallyPrecompute(wasm::Function*)::StackFinder; VisitorType = wasm::Visitor<wasm::Precompute::partiallyPrecompute(wasm::Function*)::StackFinder, void>]':
/src/src/support/insert_ordered.h:112:29: note: '<anonymous>' declared here
112 | std::pair<const Key, T> kv = {k, {}};
| ^~
|
| |
|
| |
Corrected `maybeRefType` declaration to `maybeReftype`.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
TypeMerging works by representing the type definition graph as a
partitioned DFA and then refining the partitions to find mergeable
types. #7023 was due to a bug where the DFA included edges from public
types to their children, but did not necessarily include corresponding
states for those children.
One way to fix the bug would have been to traverse the type graph,
finding all reachable public types and creating DFA states for them, but
that might be expensive in cases where there are large graphs of public
types.
Instead, fix the problem by removing the edges from public types to
their children entirely. Types reachable from public types are also
public and therefore are not eligible to be merged, so these edges were
never necessary for correctness.
Fixes #7023.
|
| |
|
|
|
|
|
|
|
|
|
| |
We already generated (throw ..) instructions in wasm, but it makes sense to model
throws from outside as well, as they cross the module boundary. This adds a new fuzzer
import to the generated modules, "throw", that just does a throw from JS etc.
Also be more precise about handling fuzzing-support imports in fuzz-exec: we now
check that logging functions start with "log*" and error otherwise (this check is
now needed given we have "throw", which is not logging). Also fix a minor issue
with name conflicts for logging functions by using getValidFunctionName for them,
both for logging and for throw.
|
| |
|
|
|
|
|
|
| |
We only checked for the case of the immediate super being public while we
are private, but it might be a grandsuper instead. That is, any ancestor that
is public will prevent GTO from removing a field (since we can only add
fields on top of our ancestors). Also, the ancestors might not all have the
field, which would add more complexity to that particular assertion, so just
remove it, and add comprehensive tests.
|
| |
|
|
|
|
|
|
|
|
|
| |
These were added to avoid common problems with closed world mode, but
in practice they are causing more harm than good, forcing users to work
around them. In the meantime (until #6965), remove this validation to unblock
current toolchain makers.
Fix GlobalTypeOptimization and AbstractTypeRefining on issues that this
uncovers: without this validation, it is possible to run them on more wasm
files than before, hence these were not previously detected. They are
bundled in this PR because their tests cannot validate before this PR.
|
