| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
| |
This makes the behavior consistent with emcc builds where we don't run
finalization, and potentially makes testing and debugging easier.
Emscripten still strips the target features section when optimizing.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Unlike other module elements, types are not stored on the `Module`.
Instead, they are collected by traversing the IR before printing and
binary writing. The code that collects the types tries to optimize the
order of rec groups based on the number of times each type is used. As a
result, the output order of types generally has no relation to the input
order of types. In addition, most type optimizations rewrite the types
into a single large rec group, and the order of types in that group is
essentially arbitrary. Changes to the code for counting type uses,
sorting types, or sorting rec groups can yield very large changes in the
output order of types, producing test diffs that are hard to review and
potentially harming the readability of tests by moving output types away
from the corresponding input types.
To help make test output more stable and readable, introduce a tool
option that causes the order of output types to match the order of input
types as closely as possible. It is implemented by having the parsers
record the indices of the input types on the `Module` just like they
already record the type names. The `GlobalTypeRewriter` infrastructure
used by type optimizations associates the new types with the old indices
just like it already does for names and also respects the input order
when rewriting types into a large recursion group.
By default, wasm-opt and other tools clear the recorded type indices
after parsing the module, so their default behavior is not modified by
this change.
Follow-on PRs will use the new flag in more tests, which will generate
large diffs but leave the tests in stable, more readable states that
will no longer change due to other changes to the optimizing type
sorting logic.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Previously we had passes --generate-stack-ir, --optimize-stack-ir, --print-stack-ir
that could be run like any other passes. After generating StackIR it was stashed on
the function and invalidated if we modified BinaryenIR. If it wasn't invalidated then
it was used during binary writing. This PR switches things so that we optionally
generate, optimize, and print StackIR only during binary writing. It also removes
all traces of StackIR from wasm.h - after this, StackIR is a feature of binary writing
(and printing) logic only.
This is almost NFC, but there are some minor noticeable differences:
1. We no longer print has StackIR in the text format when we see it is there. It
will not be there during normal printing, as it is only present during binary writing.
(but --print-stack-ir still works as before; as mentioned above it runs during writing).
2. --generate/optimize/print-stack-ir change from being passes to being flags that
control that behavior instead. As passes, their order on the commandline mattered,
while now it does not, and they only "globally" affect things during writing.
3. The C API changes slightly, as there is no need to pass it an option "optimize" to
the StackIR APIs. Whether we optimize is handled by --optimize-stack-ir which is
set like other optimization flags on the PassOptions object, so we don't need the
old option to those C APIs.
The main benefit here is simplifying the code, so we don't need to think about
StackIR in more places than just binary writing. That may also allow future
improvements to our usage of StackIR.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
This change removes the "minimal" mode from `LegalizeJSInterface`
which was added in #1883.
The idea behind this change was to avoid legalizing most function except
those we know that JS will be calling. The idea was that for dynamic
linking we always want the non-legalized version to be shared between
wasm module. These days we solve this problem in a different way with
the `legalize-js-interface-export-originals` which exports the original
functions alongside the legalized ones. Emscripten then always
prefers the `$orig` functions when doing dynamic linking.
|
| |
|
| |
See #6088
|
| |
|
|
|
|
|
|
| |
Because we currently strip some data segments (i.e. EM_JS strings)
during `--post-emscripten` this is too late as `--separate-data-segments`
always runs in `wasm-emscripten-finalize`.
Once emscripten switches over to using the pass directly we can remove
the support from `wasm-emscripten-finalize`
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With the goal of supporting null characters (i.e. zero bytes) in strings.
Rewrite the underlying interned `IString` to store a `std::string_view` rather
than a `const char*`, reduce the number of map lookups necessary to intern a
string, and present a more immutable interface.
Most importantly, replace the `c_str()` method that returned a `const char*`
with a `toString()` method that returns a `std::string`. This new method can
correctly handle strings containing null characters. A `const char*` can still
be had by calling `data()` on the `std::string_view`, although this usage should
be discouraged.
This change is NFC in spirit, although not in practice. It does not intend to
support any particular new functionality, but it is probably now possible to use
strings containing null characters in at least some cases. At least one parser
bug is also incidentally fixed. Follow-on PRs will explicitly support and test
strings containing nulls for particular use cases.
The C API still uses `const char*` to represent strings. As strings containing
nulls become better supported by the rest of Binaryen, this will no longer be
sufficient. Updating the C and JS APIs to use pointer, length pairs is left as
future work.
|
| |
|
|
| |
This is no longer needed by emscripten as of:
https://github.com/emscripten-core/emscripten/pull/16529
|
| |
|
|
|
|
|
|
|
| |
This part to finalize is currently not used and was added in preparation
for https://reviews.llvm.org/D75277.
However, the better solution to dealing with this alternative name for
main is on the emscripten side. The main reason for this is that
doing the rename here in binaryen would require finalize to always
re-write the binary, which is expensive.
|
| | |
|
| | |
|
| |
|
|
|
|
| |
This is useful for the case where we might want to finalize
without extracting metadata.
See: https://github.com/emscripten-core/emscripten/pull/15918
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The general shape of the --help output is now:
========================
wasm-foo
Does the foo operation
========================
wasm-foo opts:
--------------
--foo-bar ..
Tool opts:
----------
..
The options are now in categories, with the more specific ones - most likely to be
wanted by the user - first. I think this makes the list a lot less confusing.
In particular, in wasm-opt all the opt passes are now in their own category.
Also add a script to make it easy to update the help tests.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
As suggested in
https://github.com/WebAssembly/binaryen/pull/3955#issuecomment-871016647
This applies commandline features first. If the features section is present, and
disallows some of them, then we warn. Otherwise, the features can combine
(for example, a wasm may enable feature X because it has to use it, and a user
can simply add the flag for feature Y if they want the optimizer to try to use it;
both flags will then be enabled).
This is important because in some cases we need to know the features before
parsing the wasm, in the case that the wasm does not use the features section.
In particular, non-nullable GC locals have an effect during parsing. (Typed
function references also does, but we found a way to apply its effect all the time,
that is, always use the refined type, and that happened to not break the case
where the feature is disabled - but such a workaround is not possible with
non-nullable locals.)
To make this less error-prone, add a FeatureSet input as a parameter to
WasmBinaryBuilder. That is, when building a module, we must give it the
features to use while doing so.
This will unblock #3955 . That PR will also add a test for the actual usage
of a feature during loading (the test can only be added there, after that PR
unbreaks things).
|
| |
|
| |
See https://github.com/emscripten-core/emscripten/issues/13893
|
| |
|
|
|
|
| |
When we can skip function bodies, we still need to parse the start function
for the pthreads case, see details in the comments. This still gives us 99%
of the speedup as the start function is just 1 function and it's not that big,
so with this we return to full speed after the reversion in #3705
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
That PR assumed that wasm-emscripten-finalize does not need to scan
function bodies for metadata. But there is a case where it does, which is
that EM_ASMs with pthreads do still require scanning of the code. So that
approach is not valid.
We could maybe disable the optimization just on pthreads, but I think
major use cases need that. Also there is no simple way to disable it atm,
we'd need changes on both emscripten and binaryen. Also that PR can
no longer be reverted cleanly due to other changes. For all those reasons,
this just disables the optimization so that users of tot are no longer
broken, while we figure out how a valid way to optimize this use case.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
output (#3698)
When not writing output we don't need debug info, as it is not relevant for
our metadata. This saves loading and interning all the names, which takes
several seconds on massive inputs.
This is possible in principle in other tools, but this does not change anything
in them for now. (We do use names internally in some nontrivial ways without
opting in to it, so that would require further refactoring. Also the other tools
almost always do write an output.)
This is not 100% unobservable. If validation fails then the validation error would
just contain the function index instead of the name from the Names section if
there is one. However finalize does not validate atm so that would only matter
if we change that later.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
After sbc100 's work on EM_ASM and EM_JS they are now parsed from
the wasm using exports etc. and so we no longer need to parse function bodies.
As a result if we are not emitting a wasm from wasm-emscripten-finalize then all we are
doing is scanning global structures like imports and exports and emitting metadata
about them. And indeed we do not need to emit a wasm in some cases, specifically
when not optimizing and when using WASM_BIGINT (to avoid needing to
legalize).
We had considering skipping wasm-emscripten-finalize entirely in that situation,
and instead to parse the metadata from the wasm in python on the emscripten
side. However sbc100 had the brilliant idea today to just skip function bodies.
That is very simple to do - no need to write another parser for wasm, and also
look at how simple this PR is - and also it will be faster to run
wasm-emscripten-finalize in this mode than to run python. (With the only
downside that the bytes of the wasm are loaded even if they aren't parsed; but
almost certainly they are in the disk cache anyhow.)
This PR implements that idea: when wasm-emscripten-finalize knows it will
not write a wasm output, it notes "skip function bodies". The binary reader then
skips the bodies and places unreachables there instead (so that the wasm still
validates).
There are no new tests here because this can't be tested - by design it is an
unobservable optimization. (If we could notice the bodies have been skipped,
we would not have skipped them.) This is also why no changes are needed on
the emscripten side to benefit from this speedup. Basically when binaryen sees
it will not need X, it skips parsing of X automatically.
Benchmarking speed, it is as fast as you'd expect: the wasm-emscripten-finalize
step is 15x faster on SQLite (1MB of wasm) and almost 50x faster on the biggest
wasm I have on my drive (40MB of LLVM). (These numbers are on release
builds, without debug info - debug into makes things slower, so the speedup is
lower there, and will need further work.)
Tested manually and also on wasm0 wasm2 other on emscripten.
|
| |
|
| |
See https://github.com/emscripten-core/emscripten/pull/13208
|
| | |
|
| |
|
|
|
|
|
|
| |
This avoids needing to add include wasm-printing if a file doesn't already have it.
To achieve that, add the std::ostream hooks in wasm.h, and also use them
when possible, removing the need for the special WasmPrinter object.
Also stop printing in "full" (print types on each line) in error messages by default. The
user can still get that, as always, using BINARYEN_PRINT_FULL=1 in the env.
|
| |
|
|
|
|
| |
Emscripten no longer needs this information as of
https://github.com/emscripten-core/emscripten/pull/12643.
This also removes the need to export __data_end.
|
| | |
|
| | |
|
| |
|
|
| |
The use of these passes was removed on the emscripten side
in https://github.com/emscripten-core/emscripten/pull/12536.
|
| |
|
|
|
| |
These days we always export the table, except in the
case of dynamic linking, and even then we use the name
`__indirect_function_table`.
|
| |
|
|
|
|
|
| |
This flag disables the features of `wasm-emscripten-finalize`
the replace the mutable global import of `__stack_pointer`.
See the corresponding emscripten change that depends on this
one: https://github.com/emscripten-core/emscripten/pull/12536
|
| |
|
|
|
|
|
| |
Internalizing of the stack pointer is only needed in legacy
PIC mode, since in the new PIC mode we support mutable globals.
Also the additional ASSIGN_GOT_ENTRIES function only exists
in support of the legacy mode.
|
| |
|
|
|
|
|
|
|
|
|
| |
Now that we are renaming invoke wrappers and `emscripten_longjmp_jmpbuf`
in the wasm backend, this deletes all related renaming routines and
relevant tests. Depends on #3192.
Addresses: #3043 and #3081
Companions:
https://reviews.llvm.org/D88697
emscripten-core/emscripten#12399
|
| |
|
| |
This depends on https://github.com/emscripten-core/emscripten/pull/12391
|
| |
|
|
|
|
|
|
| |
This option skips the PIC ABI transforms that are normally done by
wasm-emscripten-finalize and keeps the llvm PIC ABI in place.
The LLVM abi uses mutable globals (GOT.mem.foo and GOT.func.bar) for
data and function offsets rather than accessor functions (g$foo and
g$bar)
|
| |
|
|
|
|
|
|
|
| |
When minimizing wasm changes, leave it as __indirect_function_table
which is what LLVM emits.
This also removes the renaming of the memory. That was never needed
as LLVM already emits "memory" there.
See #3043
|
| |
|
|
|
|
| |
Two new flags here, one to completely removes dynCalls, and another to
limit them to only signatures that contains i64.
See #3043
|
| |
|
|
|
|
|
| |
The minimizeWasmChanges flag now does nothing (but new changes are
coming, so keep it around) - this moves us to always doing the new way of
things. With that we can update the tests.
See #3043
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
wasm-emscripten-finalize renames EM_ASM calls to have the signature in
the name. This isn't actually useful - emscripten doesn't benefit from that. I
think it was optimized in fastcomp, and in upstream we copied the general
form but not the optimizations, and then EM_JS came along which is
easier to optimize anyhow.
This PR makes those changes optional: when not doing them, it just
leaves the calls as they are. Emscripten will need some changes to
handle that, but those are simple.
For convenience this adds a flag to "minimize wasm changes". The idea
is that this flag avoids needing a double-roll or other inconvenience
as the changes need to happen in tandem on the emscripten side.
The same flag can be reused for later changes similar to this one.
When they are all done we can remove the flag. (Note how the code
ifdefed by the flag can be removed once we no longer need the old
way of doing things - that is, the new approach is simpler on the
binaryen side).
See #3043
|
| |
|
|
|
|
|
|
|
|
| |
This helps towards the goal of allowing emscripten to not always modify
the wasm during link. Until now wasm-emscripten-finalize always wrote
an output, while with this PR it only does so if it was asked to, either by
giving it an output filename, or asking for text output.
The only noticeable change from this should be to make what was an
error before (not specify an output or ask for text) into a non-error (run
and print metadata, but do not write the wasm).
|
| | |
|
| |
|
|
|
|
| |
The core logic is still living in EmscriptenGlueGenerator because
its used also by fixInvokeFunctionNames.
As a followup we can figure out how to make these more independent.
|
| |
|
|
|
| |
This new pass takes an optional stack-check-handler argument
which is the name of the function to call on stack overflow.
If no argument is passed then it just traps.
|
| |
|
|
| |
This should not be needed since in emscripten standalone mode we
always include a crt1.o that includes _start.
|
| |
|
|
| |
Doing it this way happens to re-order the __assign_got_entries
function in the module, but its otherwise NFC.
|
| |
|
| |
First step in making wasm-emscripten-finalize use more passes.
|
| |
|
|
|
|
|
|
|
| |
The plan is that for standlone mode we can function just like wasi-sdk
and call the correct main from crt1.c.
For non-standalone mode we still want to export can call main directly
so we rename __main_argc_argv back to main as part of finalize.
See https://reviews.llvm.org/D70700
|
| |
|
|
|
|
|
| |
These are now implemented in assembly as part of emscripten's
compiler-rt.
See: https://github.com/emscripten-core/emscripten/pull/11166
|
| |
|
|
|
| |
If wasm-emscripten-finalize is given the BigInt flag, then we will
be using BigInts on the JS side, and need no legalization at all
since i64s will just be BigInts.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Depends on emscripten-core/emscripten#10741
which ensures that table indexes are unique. With that guarantee,
a main module can just add its function pointers into the table, and
use them based on that index. The loader will then see them in the
table and then give other modules the identical function pointer for
a function, ensuring function pointer equality.
This avoids calling fp$ functions during startup for the main
module's own functions (which are slow). We do still call fp$s
of things we import from outside, as we don't have anything to
put in the table for them, we depend on the loader for that.
I suspect this can also be done with SIDE_MODULES, but did not
want to try too much at once.
|
| |
|
|
|
|
|
|
| |
(#2544)
Without this, the first wasm-opt invocation will remove it. But it
can be very large, and we will soon start to automatically do
updating on it when it exists, so avoid the work if we aren't
actually building a final output with dwarf.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Optionally track the binary format code section offsets,
that is, when loading a binary, remember where each IR
node was read from. This is necessary for DWARF
debug info, as these are the offsets DWARF refers to.
(Note that eventually we may want to do something
else, like first read the DWARF and only then add
debug info annotations into the IR in a more LLVM-like
manner, but this is more straightforward and should be
enough to update debug lines and ranges).
This tracking adds noticeable overhead - every single
IR node adds an entry in a map - so avoid it unless
actually necessary. Specifically, if the user passes in
-g and there are actually DWARF sections in the
binary, and we are not about to remove those sections,
then we need it.
Print binary format code section offsets in text, when
printing with -g. This will help debug and test dwarf
support. It looks like
;; code offset: 0x7
as an annotation right before each node.
Also add support for -g in wasm-opt tests (unlike
a pass, it has just one - as a prefix).
Helps #2400
|
| |
|
|
|
|
|
|
|
| |
In normal mode we call a JS import, but we can't import from JS
in standalone mode. Instead, just trap in that case with an
unreachable. (The error reporting is not as good in this case, but
at least it catches all errors and halts, and the emitted wasm is
valid for standalone mode.)
Helps emscripten-core/emscripten#10019
|
